While intelligent applications (e.g., recommendation systems) prefer different CPU-GPU ratios, GPU pooling technique that decouples the GPU and CPU resources yields substantial flexibility when serving diverse applications. With such architecture, DNN-based recommendation services often offload the compute-intensive neural network layers to the remote GPU pool for high resource utilization. However, such a paradigm results in the long end-to-end latency due to two causes: 1) the intermediate data is copied for multiple times during the entire process in current GPU pooling practices, incurring heavy overheads; 2) the content transferred to the GPU pool involves multiple small tensors, suffering from poor bandwidth efficiency. To solve these problems, we design Zero, a runtime system that incorporates a zero-copy transmission mechanism as well as a dynamic tensor merging policy. The zero-copy transmission mechanism unifies memory management across the inference framework and the RPC framework, accompanied by an elaborated serialization protocol to fully eliminate redundant data copying. Meanwhile, the tensor merging policy deliberately organizes small tensors into larger data blocks, so as to transfer them with higher efficiency. Experimental results show that, compared with prior work, Zero reduces the latency of typical recommendation models by up to 15.1% (10.1% on average).

Multi-threading is a common way for programs to benefit from the multi/many-core design. However, the performance of some parallel programs does not increase/even decrease as the number of cores/threads increases. Our study shows that the performance of a parallel program is impacted bythe number of cores/threads,the thread placement,the inputs of the program. It is nontrivial to identify the optimal number of cores and the corresponding thread placement to maximize the performance, when the input of a program is determined online and the workload of different iterations may not be identical. To resolve the above problem, we proposeOtter, a thread auto-tuning system at runtime for iterative parallel programs. Otter collects the runtime information in the first few iterations and makes decisions on the number of threads and thread placement policy to achieve the goal of improving performance or saving resources. It considers the characteristics of dynamic workload in the iteration process and reduces the time overhead through a migration method. Experiments on a 96-core machine show that Otter improves the performance of the benchmarks by 20.7% and reduces core hours by 51.3% on average compared to the case of running them with all the CPU cores.

With the improvement of the reliability requirements of modern computer systems, fault-tolerant technology has been widely used in all aspects of computer systems as a key mechanism to ensure system reliability. The seL4 microkernel as the world's first complete, formal, machine-checked operating system kernel, has become one of the ideal platforms for safety-critical systems, but the lack of fault-tolerant mechanism reduces its reliability and availability. This paper designs and implements a fault-tolerant backup and recovery mechanism based on checkpoint in the seL4 microkernel and makes the optimization of resource. The experiment demonstrates the performance of the backup and recovery mechanism in terms of time and space.

Virtualization technology, which has achieved great success in server and desktop environments in the last few years, is currently extending itself towards a new territory: embedded system. OKL4 from Open Kernel Labs is a leading virtualization software for embedded systems. Its microkernel approach can improve plain virtualization technologies, but it also brings incomplete IPC (Inter-Process Communication) issues. This paper proposes a novel asynchronous communication mechanism that can generate multiple event channels and efficiently manage concurrent communication requests between virtual machines. This mechanism optimizes the original IPC mechanism, and this paper also proposes a shared-memorybased bulk data transmission mechanism. The final experiments prove its feasibility and demonstrate the specific effect.