With the widely adoption of microservice architecture in the cloud computing industry, accurate prediction of workloads, especially CPU cores, can support reasonable resource allocation, thereby optimizing the resource utilization of the system. However, workload prediction is challenging in two dimensions. In the temporal dimension, workload series 1) has non-stationary characteristics, leading to poor predictability; 2) has a multi-periodic nature with entangled temporal patterns; 3) may be influenced by dynamic system states like response time and number of requests. In the spatial dimension, when regarding microservices as nodes in a distributed system, there is no topology caused by physical connections, but exists complex similarity dependencies. Extracting robust spatial features from these dependencies presents difficulties. To address these, we propose STEAM, a Spatio Temporal Heterogenous Graph Contrastive Learning for Microservice Workload Prediction. STEAM leverages non-stationary decomposition self-attention to extract temporal features from non-stationary and multi-periodic workload series, while the decoupled embedding is used to capture system state information of microservices. By treating microservices as nodes and constructing a similarity graph, STEAM effectively models the similarity relationships between microservices. To reduce the prior interference caused by the similarity threshold and improve the robustness, STEAM constructs two heterogeneous augmentation views and uses contrastive learning to extract the shared consistent spatial features. The multi-scale learning is adopted to model the long- and short-term temporal features, forming a spatio-temporal stacking structure. Experiments on two datasets, including MS dataset obtained from Ant Group, which is one of the world’s largest cloud service providers, demonstrate the superiority of STEAM.

Microservice architecture has become a driving force in enhancing the modularity and scalability of web applications, as evidenced by the Alipay platform's operational success. However, a prevalent issue within such infrastructures is the suboptimal utilization of CPU resources due to inflexible resource allocation policies. This inefficiency necessitates the development of dynamic, accurate workload prediction methods to improve resource allocation. In response to this challenge, we present STAMP, a Spatio Temporal Graph Network for Microservice Workload Prediction. STAMP is designed to comprehensively address the multifaceted interdependencies between microservices, the temporal variability of workloads, and the critical role of system state in resource utilization. Through a graph-based representation, STAMP effectively maps the intricate network of microservice interactions. It employs time series analysis to capture the dynamic nature of workload changes and integrates system state insights to enhance prediction accuracy. Our empirical analysis, using three distinct real-world datasets, establishes that STAMP exceeds baselines by achieving an average boost of 5.72% in prediction precision, as measured by RMSE. Upon deployment in Alipay's microservice environment, STAMP achieves a 33.10% reduction in resource consumption, significantly outperforming existing online methods. This research solidifies STAMP as a validated framework, offering meaningful contributions to the field of resource management in microservice architecture-based applications.

In the cloud environment, different kinds of jobs (Flink, PyTorch, TensorFlow, AI-Serving) are running in the same cluster with different service-level agreements (SLA). To manage large amounts of jobs in a cloud environment efficiently, it is critical to build a system to optimize the job performance in consideration of multiple predefined objectives. For example, one kind of optimization target is improving the resource utilization of jobs, other kinds of objectives are to guarantee the system SLA (e.g., system throughput, response time, and so on). Currently, most of the existing frameworks are working on one aspect of optimization, and can not support different kinds of optimization targets via a uniform framework or system. In Antgroup, we have designed and implemented a general framework (named Cougar) to improve jobs and cluster performance to meet such requirements. Cougar provides the ability to support different optimization scenarios like the initial and runtime optimization for one job, and cross-job optimization for multiple jobs. Nowadays, Cougar has widely used in the production environment of Antgroup including 110,000 jobs and 800,000 Pods daily, and has successfully improved the CPU/Memory/GPU utilization by more than 20% and performance (i.e., throughput or completion time or latency) by around 10%. In the end, we also like to share our best practice on how to tune Flink and Deep Learning Job (GPU collocate) in the production environment.