Automatic configuration tuning of online services with dynamic workloads has attracted increasing interest. Effective online tuning ensures configurations adapt to workload changes over time to maintain optimal online service performance. To be practical, online tuning must satisfy the dynamicity, efficiency, and Quality of Service (QoS) requirements. However, existing online tuning approaches fail to meet these requirements due to the inability to eliminate negative effects from historical observations. In this paper, we propose A-Tune-Online, an online configuration tuning system that tackles dynamic workloads, delivering superior tuning efficiency, and QoS guarantee simultaneously to a wide range of online scenarios. We identify that restarting the optimization based on explicit workload shift detection is necessary and critical to eliminate negative historical observations. First, to invoke optimization restarts appropriately, we design a multi-stage multi-indicator detection strategy based on heuristic rules and configuration replays. Then, to avoid initial efficiency drop after re-optimization, A-Tune-Online utilizes a similarity-based dual warm start scheme that transfers knowledge from similar historical workloads effectively. Finally, to prevent transient performance degradation from violating QoS guarantee after optimization restart, we leverage lower confidence bound to construct a safety region where each configuration is expected to perform better than the QoS requirement. Empirical study on five tuning scenarios showcases the superiority of A-Tune-Online compared with state-of-art tuning systems. A-Tune-Online achieves an average speedup of 2.90x and 1.72x compared with OnlineTune and DDPG+, respectively. We provide a version of our system in https://github.com/PKU-DAIR/A-Tune-Online.

In serverless computing, an idle container is not recycled directly, in order to mitigate time-consuming cold container startup. These idle containers still occupy the memory, exasperating the memory shortage of today's data centers. By offloading their cold memory to remote memory pool could potentially resolve this problem. However, existing offloading policies either hurt the Quality of Service (QoS) or are too coarse-grained in serverless computing scenarios.

Energy consumption in cloud data centers has become an increasingly important contributor to greenhouse gas emissions and operation costs. To reduce energy-related costs and improve environmental sustainability, most modern data centers consolidate Virtual Machine (VM) workloads belonging to different application classes, some being latency-critical (LC) and others being more tolerant to performance changes, known as best-effort (BE). However, in public cloud scenarios, the real classes of applications are often opaque to data center operators. The heterogeneous applications from different cloud tenants are usually consolidated onto the same hosts to improve energy efficiency, but it is not trivial to guarantee decent performance isolation among colocated workloads. We tackle the above challenges by introducing Demeter, a QoS-aware power management controller for heterogeneous black-box workloads in public clouds. Demeter is designed to work without offline profiling or prior knowledge about black-box workloads. Through the correlation analysis between network throughput and CPU resource utilization, Demeter automatically classifies black-box workloads as either LC or BE. By provisioning differentiated CPU management strategies (including dynamic core allocation and frequency scaling) to LC and BE workloads, Demeter achieves considerable power savings together with a minimum impact on the performance of all workloads. We discuss the design and implementation of Demeter in this work, and conduct extensive experimental evaluations to reveal its effectiveness. Our results show that Demeter not only meets the performance demand of all workloads, but also responds quickly to dynamic load changes in our cloud environment. In addition, Demeter saves an average of 10.6% power consumption than state of the art mechanisms.

To reduce the increasing cost of building and operating cloud data centers, cloud providers are seeking various mechanisms to achieve higher resource effectiveness. For example, cloud operators are leveraging dynamic resource management techniques to consolidate a higher density of application workloads into commodity physical servers to maximize server resource utilization. However, higher workload density is a major source of performance interference problems in multi-tenant clouds. Existing performance isolation techniques such as dedicated CPU cores for specific workloads are not enough as there are still common resource (e.g., last-level cache and memory bandwidth in memory subsystem) on the processor that are shared among all CPUs on the same NUMA node. While prior work has proposed a variety of resource partitioning techniques, it still remains unexplored to characterize the impact of memory subsystem resource partitioning for the consolidated workloads with different priorities and investigate software support to dynamically manage memory subsystem resource sharing in a real-time manner. To bridge the gap, we propose Themis, a feedback-based controller that enables a priority-aware and fairness-aware memory subsystem resource management strategy to guarantee the performance of high-priority workloads while maintaining fairness across all colocated workloads in high-density clouds. Themis is evaluated with multiple typical cloud applications in our data center environment. The results show that Themis improves the performance of various workloads by up to 3.15%, and fairness by more than 70% in memory subsystem resource allocation compared to existing state-of-the-art work.

More and more complex applications are deployed in virtualized Cloud environments. Cloud applications usually consist of multiple virtual components and the data flow among these components tends to be highly complex and unpredictable. Cloud providers have no visibility into tenants' virtual components and rely on non-intrusive knowledge to infer the service performance. The scale, complexity, and heterogeneity make Cloud environments prone to different kinds of performance anomalies. An automatic anomaly detection mechanism is necessary to help avoid service failure at an early stage of anomalies. We propose DMFD, a decentralized and non-intrusive anomaly detection system for Cloud applications. DMFD considers each virtual component as a black box, and works without knowledge from underlying operating system or applications. DMFD infers the dependency model to characterize the complex interaction behavior among multiple components, and further builds an adaptive data flow ratio model for anomaly detection. The data flow model explores a simple and common relationship that the number of outgoing response data packets changes accordingly with the number of incoming request data packets at each component. We evaluate DMFD with several different kinds of Cloud applications in our data center. The results reveal that DMFD has high precision and low false positive/negative in detecting different kinds of performance anomalies under varying workload scenarios.

Fast detection of performance anomalies is critical in Cloud applications, but challenging to implement in a general and effective tool with low operational overload. We propose FSAD, a performance anomaly detection system based on the concept of flow similarity. It stems from the observation that, in general, the number of responses generated by a component closely follows the number of received requests, but this relation stops holding in presence of performance anomalies. In FSAD, components are regarded as black boxes, and time series of incoming and outgoing packets are fed to the flow similarity analysis for anomaly detection. The effectiveness of FSAD is demonstrated in experimental results.

Many complex application services are deployed in virtualized Cloud environments. Cloud applications consist of multiple components and the data flow among these components tends to be highly complex and unpredictable. The complexity and heterogeneity make anomaly detection challenging. We propose FlowBox, a distributed anomaly detection system for Cloud applications. FlowBox considers each server component as a black box and detects performance anomalies using the flow analysis. The black box model addresses the challenge of accurately describing the complex system model. The flow analysis is based on a simple relationship of data flow in any given component of Cloud applications. Between any two components, the number of requests should always be equal to the number of responses within a given time interval during normal operations. FlowBox monitors traffic flow in each component and continually builds flow signatures in order to describe the normal application behavior. Using the flow signatures, FlowBox detects performance anomalies in Cloud applications. We evaluate FlowBox with several different kinds of Cloud applications in our datacenter. Experimental results show that FlowBox achieves 96.02% detection precision, 3.98% false positive, and 3.5% false negative in detecting various kinds of anomalies.