Microservices are widely shared in production user-facing applications. These shared microservices have various resource usage patterns when queries from different call graphs of different services access them. However, existing microservice management works fail to efficiently scale resources for them, mainly due to the lack of fine-grained scheduling of diverse queries. We therefore propose Delphinus , a runtime system that efficiently manages resources for shared microservices while ensuring the Quality-of-Service (QoS). Delphinus comprises a group-oriented query scheduler and a borrowing-based load adapter . The query scheduler identifies diverse queries, groups the containers of shared microservices, and schedules the queries into separate groups. The load adapter efficiently scales resources for shared microservices, and fully utilizes the idle containers among groups when the loads of diverse queries change. Results show that Delphinus reduces CPU and memory usage by 40.1% and 36.4% for shared microservices, respectively, compared to state-of-the-art works.

Distributed training of large language models demands efficient gradient synchronization strategies. Existing All Reduce algorithms often fail to fully leverage underlying topological properties, resulting in low edge utilization and unbalanced load in hypercube networks. This paper proposes a Redundant Hierarchical Ring All-Reduce (RHRA) algorithm tailored for hypercube topologies. The algorithm first designs a hierarchical Ring All-Reduce structure with high edge utilization by exploiting the hypercube's inherent high parallelism and symmetry, proving that the optimal number of hierarchical layers is n 2. It then introduces controlled redundancy, dynamically adjusting transmission strategies based on the relationship between the value of bandwidth B and the value of data volume D to reduce communication hops and transmission latency while ensuring reliability. Experiments demonstrate that our algorithm achieves the minimum number of gradient synchronization hops under varying bandwidth constraints, with end-to-end transmission time significantly outperforming existing schemes. Further reliability simulations verify its effectiveness in balancing redundancy to prevent resource waste while maintaining high reliability.

While GNN-based services often experience load fluctuation, applying serverless computing to serve GNN inference reduces the cost and allows elastic resource scaling. However, GNN serverless shows poor performance due to heavy data fetching latency and long cold startup overhead, and our observation indicates opportunities for reducing data redundancy and mitigating cold startup latency. In this paper, we present FaaSGNN, a serverless GNN inference framework that enables low latency and memory efficient GNN serving through three key designs: (i) serverless-native on-demand graph fetching strategy that enables lightweight in-container graph sampling with full dataset resides in remote; (ii) memory-aware adaptive feature caching policy, which facilitates data reuse between requests to reduce redundant fetching; and (iii) load-aware request scheduler, which reschedules requests to bypass cold start and achieve load balance between containers. Experimental results show that FaaSGNN achieves a 5.6x lower end-to-end latency and 57.1% less memory usage on average compared to state-of-the-art works.

The secure container that hosts a single container in a micro virtual machine (VM) is now used in serverless computing, as the containers are isolated through the microVMs. There are high demands on the high-density container deployment and high-concurrency container startup to improve both the resource utilization and user experience, as user functions are fine-grained in serverless platforms. Our investigation shows that the entire software stacks, containing the cgroups in the host operating system, the guest operating system, and the containerrootfsfor the function workload, together result in low deployment density and slow startup performance at high-concurrency.We propose a lightweight and holistic-scalable secure container runtime, named RunD-V, to resolve above problems in serverless computing. RunD-V proposes a guest-to-host runtime template for microVM scaling-out, and CR-bind feature in guest kernel for microVM scaling-up. Using guest-to-host runtime template, over 200 secure containers can be launched within 1son a node equipped with 104 vCPUs. It also enables more than 2,500 secure containers to be deployed on a node with 384GB of memory. The vertical scaling mechanism CR-bind further enhances both startup concurrency and deployment density.