Latency-critical (LC) services are usually co-located with best-effort applications to improve the resource utilization. Lots of studies have been proposed to guarantee the performance of LC services in the co-location by managing shared resources. However, even the LC service has enough resources, its performance may still severely degrade because of memory reclaim caused by the operating system. We therefore propose Priority Memory Reclaim (PMR) which can eliminate impact of memory reclaim on LC services as much as possible. PMR consists of two techniques: priority page swapping and adaptive watermark configuration. Experiment results show that PMR can greatly improve performance of LC services. Specifically, PMR can reduce the 99%-ile latency and maximum latency of LC services by up to 92.32% and 95.87% while improving the throughput by up to 4.75x.

In-memory key-value stores (IMKVSes) serve many online applications. They generally adopt the fork-based snapshot mechanism to support data backup. However, this method can result in query latency spikes because the engine is out-of-service for queries during the snapshot. In contrast to existing research optimizing snapshot algorithms, we address the problem from the operating system (OS) level, while keeping the data persistent mechanism in IMKVSes unchanged. Specifically, we first study the impact of the fork operation on query latency. Based on findings in the study, we propose Async-fork, which performs the fork operation asynchronously to reduce the out-of-service time of the engine. Async-fork is implemented in the Linux kernel and deployed into the online Redis database in public clouds. Our experiment results show that Async-fork can significantly reduce the tail latency of queries during the snapshot.

While current deep learning frameworks are mainly optimized for dense-accessed models, they show low throughput and poor scalability in training sparse-accessed recommendation models. Our investigation shows that the poor performance is due to the parameter synchronization bottleneck. We therefore propose BiPS, a bi-tier parameter synchronization system that alleviates the parameter update and the sparse-accessed parameters communication bottleneck. BiPS includes a bi-tier parameter server that accelerates the traditional CPU-based parameter update process, a hotness-aware parameter placement and communication policy to balance the workloads between CPU and GPU and optimize the communication of sparse-accessed parameters. BiPS overlaps the worker computation with the synchronization stage to enable parameter updates in advance. We implement BiPS and incorporate it into mainstream DL frameworks including TensorFlow, MXNet, and PyTorch. The experimental results based on various deep learning frameworks show that BiPS greatly speeds up the training of recommenders (5 - 9$\times$) as the model scale increases, without degrading the accuracy.