Modern stream processing systems often rely on embedded key-value stores, like RocksDB, to manage the state of long-running computations. Evaluating the performance of these stores when used for streaming workloads is cumbersome as it requires the configuration and deployment of a stream processing system that integrates the respective store, and the execution of representative queries to collect measurements.

The traffic load sent to key-value (KV) stores varies over long timescales of hours to short timescales of a few microseconds. Long-term variations present the opportunity to save power during low or medium periods of utilization. Several techniques exist to save power in servers, including feedback-based controllers that right-size the number of allocated CPU cores, dynamic voltage and frequency scaling (DVFS), and c-state (idle-state) mechanisms. In this paper, we demonstrate that existing power saving techniques are not effective for KV stores. This is because the high rate of traffic even under low load prevents the system from entering low power states for extended periods of time. To achieve power savings, we must unbalance the load among the CPU cores so that some of them can enter low power states during periods of low load. We accomplish this by introducing the notion of in-application CPU scheduling. Instead of relying on the kernel to schedule threads, we pin threads to bypass the kernel CPU scheduler and then perform the scheduling within the KV store application. Our design, Peafowl, is a KV store that features an in-application CPU scheduler that monitors the load to learn the workload characteristics and then scales the number of active CPU cores when the load drops, leading to notable power savings during low or medium periods of utilization. Our experiments demonstrate that Peafowl uses up to 40--54% lower power than state of the art approaches such as Rubik and μDPM.

Large-scale online services parallelize sub-operations of a user's request across a large number of physical machines (service components) so as to enhance the responsiveness. Even a temporary spike in latency of any service component can notably inflate the end-to-end delay; therefore, the tail of the latency distribution of service components has become a subject of intensive research. The key characteristics of clouds such as elasticity and on-demand resource provisioning have made clouds attractive for hosting large-scale online services wherein VMs are the building blocks of services. However, adherence to traditional hypervisor scheduling policies has led to unpredictable CPU access latencies for virtual CPUs (vCPUs) that are responsible for performing network IO processes. This has resulted in poor and unpredictable performance for network IO, exacerbating VMs' long tail latencies and discouraging the hosting of large-scale parallel web services on virtualized clouds. This paper presents TerrierTail, a hypervisor CPU scheduler whose primary goal is to trim the tail of the latency distribution of individual VMs in virtualized clouds. In TerrierTail, we have modified the network driver to identify vCPUs that are responsible for performing network IO processes. Leveraging this information, the TerrierTail scheduler mitigates the CPU access latencies of such vCPUs using novel scheduling policies, resulting in a higher and more predictable network IO performance and therefore lower tail latency. TerrierTail's gains come at no measurable negative impacts on other performance attributes (e.g., fairness) or on the performance of VMs running other types of workloads (e.g., CPU-intensive VMs). A prototype implementation of TerrierTail in the Xen hypervisor substantially outperforms the default Credit scheduler of Xen. For example, TerrierTail mitigates the tail latency of a Memcached server by up to 53 percent and an RPC server by up to 50 percent at 99.9th percentile.