Keeping runtime alive is critical for mitigating cold start issues in Function-as-a-Service (FaaS) platforms. State-of-the-Art (SOTA) keep-alive policies often draw an analogy to data caching, adapting classic replacement algorithms to manage runtime pools. Our analysis reveals that this analogy is fundamentally flawed: 1) they violate their own eviction priorities due to conflicts with running containers, and 2) they ignore the exorbitant memory-time resource cost of runtime replacement. To address these challenges, we present FaaShadow, a lightweight keep-alive policy that recasts the problem from simple caching to cost-aware resource allocation. FaaShadow introduces the concept of a shadow pool, a per-function data structure that enables online estimation of the marginal utility of memory adjustments. By quantifying both the potential performance gain from allocating new containers and the performance loss from removing existing ones, FaaShadow makes data-driven reallocation decisions that maximize the global warm start rate. Experimental results show that FaaShadow achieves a 95% warm start rate using only 60% of the memory required by the best-in-class baseline. When paired with our dynamic scaling mechanism, FaaShadow reduces average memory consumption by a staggering 81.71% compared to the SOTA predictive scaler, while upholding performance targets.

In the rapidly evolving landscape of cloud computing, serverless architectures offer a paradigm shift towards fine-grained function deployment and meticulous resource auto-scaling. Despite its growing popularity, existing systems often struggle to ensure the stability of function execution due to frequent cold starts and high concurrency demands. Our observations reveal a critical issue where a few hotspot functions excessively create new containers, resulting in substantial response latency fluctuations. To address this challenge, we propose Eunomia, a SLO-aware (Service Level Objective-aware) serverless framework. Eunomia introduces an optimized Poisson model with dynamic, sliding windows to accurately capture the arrival patterns of hotspot functions. Based on the optimized Poisson model, it proposes a queuing-based delay execution approach to mitigate initialization overhead by promoting instance reuse. Additionally, Eunomia designs flexible instance orchestration, providing dedicated concurrency pools for hotspot functions and dynamically adjusting the number of active instances. Experimental results demonstrate that Eunomia ensures 97% tail latency under a 100 ms response latency SLO, and outperforms the second-best baseline by 46% when memory is limited.

In serverless computing, cold start results in long response latency. Existing approaches strive to alleviate the issue by reducing the number of cold starts. However, our measurement based on real-world production traces shows that the minimum number of cold starts does not equate to the minimum response latency, and solely focusing on optimizing the number of cold starts will lead to sub-optimal performance. The root cause is that functions have different priorities in terms of latency benefits by transferring a cold start to a warm start. In this paper, we proposeIncendio, a serverless computing framework exploiting priority-based scheduling to minimize the overall response latency from the perspective of cloud providers. We reveal the priority of a function is correlated to multiple factors and design a priority model based on Spearman’s rank correlation coefficient. We integrate a hybrid Prophet-LightGBM prediction model to dynamically manage runtime pools, which enables the system to prewarm containers in advance and terminate containers at the appropriate time. Furthermore, to satisfy the low-cost and high-accuracy requirements in serverless computing, we propose a Clustered Reinforcement Learning-based function scheduling strategy. The evaluations show that Incendio speeds up the native system by 1.4×, and achieves 23% and 14.8% latency reductions compared to two state-of-the-art approaches.

Function-as-a-Service (FaaS) is a promising cloud computing model known for its scalability and elasticity. In various application domains, FaaS workflows have been widely adopted to manage user requests and complete computational tasks efficiently. Motivated by the fact that function containers collaboratively use the image layer's memory, co-placing functions would leverage memory sharing to reduce cluster memory footprint, this paper studies layer- wise memory sharing for serverless functions. We find that overwhelming memory sharing by placing containers in the same cluster machine may lead to performance deterioration and Service Level Objective (SLO) violations due to the increased CPU pressure. We investigate how to maximally reduce cluster memory footprint via layer- wise memory sharing for serverless workflows while guaranteeing their SLO. First, we study the container memory sharing problem under serverless workflows with a static Directed Acyclic Graph (DAG) structure. We prove it is NP-Hard and propose a 2-approximation algorithm, namely MDP. Then we consider workflows with dynamic DAG structure scenarios, where the memory sharing problem is also NP-Hard. We design a Greedy-based algorithm called GSP to address this issue. We implement a carefully designed prototype on the OpenWhisk platform, and our evaluation results demonstrate that both MDP and GSP achieve a balanced and satisfying state, effectively reducing up to 63% of cache memory usage while guaranteeing serverless workflow SLO.