We introduce uProcess, a pure userspace process abstraction that enables CPU cores to be rescheduled among applications at sub-microsecond timescale without trapping into the kernel. We achieve this by constructing a special privileged mode in userspace to achieve safe and efficient separation among uProcesses. The core idea is a careful combination of two emerging hardware features - userspace interrupts (Uintr) and memory protection keys (MPK). We materialize the uProcess abstraction by implementing Vessel, a userspace core scheduler that colocates latency-critical and best-effort applications with minimal switching overhead when they time-share CPU cores. Our experiment result shows that Vessel exhibits better overall performance and low latency when multiple applications are colocated.

Deep learning based models have dominated current production recommendation systems. However, the gap between CPU-side DRAM data accessing and GPU processing still impedes their inference performance. GPU-resident cache can bridge this gap, but we find that existing systems leave the benefits to cache the embedding table, a huge sparse structure, on GPU unexploited. In this paper, we present Fleche, a holistic cache scheme with detailed designs for efficient GPU-resident embedding caching. Fleche (1) uses one cache backend for all embedding tables to improve the total cache utilization, and (2) merges small kernel calls into one unitary call to reduce the overhead of kernel maintenance (e.g., kernel launching and synchronizing). Furthermore, we carefully design the cache query workflow for finer-grain parallelism. Evaluations with real-world datasets show that compared with the prior art, Fleche significantly improves the throughput of embedding layer by 2.0 -- 5.4×, and gets up to 2.4× speedup of end-to-end inference throughput.

Nuclear emergency evacuation is challenging and dangerous, with time constraints, resource limitations, and radiation exposure risks. The development of the Internet of Things (IoT) and artificial intelligence enables us to build an intelligent evacuation system to help mitigate this problem a great deal. In this article, we design the nuclear emergency parallel evacuation system based on the artificial systems (A), computational experiments (C), and parallel execution (P) approach and intelligent vehicle collaborative systems (IVCs). In this system, the evacuation risks of different regions under various possible scenarios are simulated and evaluated in the artificial systems. With data adversarially generated from the artificial systems and collected from sensors all over the area, an optimization model is proposed to find the optimal evacuation plans for emergent evacuation scenarios in the computational experiments. Eventually, the most suitable running strategy of autonomous buses will be selected and carried out based on the parallel execution in accordance with the real scene. A case study is conducted, and results indicate that the system can serve as an efficient tool for future nuclear emergency evacuation planning.

Modern recommendation systems in industry often use deep learning (DL) models that achieve better model accuracy with more data and model parameters. However, current opensource DL frameworks, such as TensorFlow and PyTorch, show relatively low scalability on training recommendation models with terabytes of parameters. To efficiently learn large-scale recommendation models from data streams that generate hundreds of terabytes training data daily, we introduce a continual learning system called Kraken. Kraken contains a special parameter server implementation that dynamically adapts to the rapidly changing set of sparse features for the continual training and serving of recommendation models. Kraken provides a sparsity-aware training system that uses different learning optimizers for dense and sparse parameters to reduce memory overhead. Extensive experiments using real-world datasets confirm the effectiveness and scalability of Kraken. Kraken can benefit the accuracy of recommendation tasks with the same memory resources, or trisect the memory usage while keeping model performance.

Learning in digital systems could be enhanced by applying a learn-by-doing mechanism. In this paper the implementation of a web-based remote FPGA laboratory for Computer Organization course is proposed. The projects created for this course are designed towards entry-level users thus producing a trouble-free virtualization of experimental equipment that in turn maximizes educational gain. The network laboratory had made learners complete all the experimental activities through the Internet without the limitation of time, space and resources. We show also some preliminary results compared between Fall 2016 and Fall 2017 semester, which highlight improved student engagement, learning and achievement.

Hardware-friendly network quantization (e.g., binary/uniform quantization) can efficiently accelerate the inference and meanwhile reduce memory consumption of the deep neural networks, which is crucial for model deployment on resource-limited devices like mobile phones. However, due to the discreteness of low-bit quantization, existing quantization methods often face the unstable training process and severe performance degradation. To address this problem, in this paper we propose Differentiable Soft Quantization (DSQ) to bridge the gap between the full-precision and low-bit networks. DSQ can automatically evolve during training to gradually approximate the standard quantization. Owing to its differentiable property, DSQ can help pursue the accurate gradients in backward propagation, and reduce the quantization loss in forward process with an appropriate clipping range. Extensive experiments over several popular network structures show that training low-bit neural networks with DSQ can consistently outperform state-of-the-art quantization methods. Besides, our first efficient implementation for deploying 2 to 4-bit DSQ on devices with ARM architecture achieves up to 1.7× speed up, compared with the open-source 8-bit high-performance inference framework NCNN [31].