Improving resource utilization in datacenters is vital for reducing costs for cloud service providers (CSPs). Increasing resource utilization must be balanced with maintaining quality of service (QoS) for latency-critical applications. In cloud environments, users often request excessive resources for applications to ensure QoS. To address this issue, CSPs use resource overcommitment - offering users resources that exceed the actual capacity of physical infrastructure. However, if not properly managed, such strategies may result in performance degradation or even request failure. Therefore, to achieve optimal resource utilization while maintaining QoS to applications, it is critical to implement a fine-grained overcommitment strategy.We propose HAPPIES, a History-aware management system with a precise prediction for machine resource demand. HAPPIES uses historical usage to extract resource characteristics and build application portraits that describe their resource demands. Compared to the existing strategy, this is a more aggressive overcommitment strategy that achieves higher resource utilization. We simulated experiments on 3,021 nodes and deployed over 14,000 applications on them. Results show that HAPPIES significantly outperforms Kubernetes Least Request and Peak Oracle in load balancing. Not only does it reduce the number of nodes experiencing high utilization, but it also decreases the peak usage of the most heavily utilized nodes. Therefore, HAPPIES scheduling reduces the risk of a machine being used beyond capacity.

Identifying and managing performance interference in clouds has long been a critical and challenging task for cloud providers. They keep seeking useful performance indicators from underlying systems to monitor cloud applications accurately. However, state-of-the-art indicators are either sensitive to limited applications and resource contention or are unrobust to the continually changing production environments. There still lacks a practical and efficient indicator for production environments.

Cloud platform provides great flexibility and cost-efficiency for end-users and cloud operators. However, low resource utilization in modern datacenters brings huge wastes of hardware resources and infrastructure investment. To improve resource utilization, a straightforward way is co-locating different workloads on the same hardware. To figure out the resource efficiency and understand the key characteristics of workloads in co-located cluster, we analyze an 8-day trace from Alibaba's production trace. We reveal three key findings as follows. First, memory becomes the new bottleneck and limits the resource efficiency in Alibaba's datacenter. Second, in order to protect latency-critical applications, batch-processing applications are treated as second-class citizens and restricted to utilize limited resources. Third, more than 90% of latency-critical applications are written in Java applications. Massive self-contained JVMs further complicate resource management and limit the resource efficiency in datacenters.