Edge computing is a promising paradigm for deploying latency-sensitive applications (Apps) as it brings resources closer to end users. Edge Apps often adopt a microservice (MS) architecture, breaking monolithic Apps into lightweight, containerized MSs that can be dynamically and independently deployed. However, managing such Apps involves three key challenges: (i) optimizing the placement of MSs to reduce both response time and resource overhead, (ii) handling MS migration or relocation as users move while minimizing App service disruption (App downtime), and (iii) enabling MS sharing across Apps while ensuring performance guarantees. We formulate this as an optimization problem, named Multi-microservice Application Placement (MAP), prove its NP-hardness, and introduce STEP (State and Topology-aware Edge-MS Placement), a polynomial-time heuristic. STEP distinguishes itself from prior work by: (i) jointly considering stateful and stateless MS characteristics in deployment decisions, (ii) exploiting MS shareability to reduce resource usage, (iii) balancing response latency, App downtime, and resource utilization, and (iv) leveraging multiple versions of the same MS to adapt quality of service to available edge resources. Our results in a small-scale scenario show that STEP achieves near-optimal performance with only 7% higher CPU cost than the optimal solution. Large-scale real-time experiments on a Kubernetes cluster demonstrate that STEP consistently outperforms competing methods, achieving up to 50% lower deployment costs while delivering 50% gain in app quality and saving 15% in radio resources with over 90% request success rates.

Edge computing reduces latency for mobile applications (Apps) by processing data closer to users, while containerized microservices (MSs) enable their modular deployment. Managing such Apps involves three key challenges: (i) strategically placing MSs to minimize response latency and resource consumption, (ii) managing MS migration/relocation during user mobility or traffic load changes while limiting App downtime, and (iii) enabling MS sharing across Apps while ensuring target performance. We formulate this as an optimization problem (proven to be NP-hard) and propose STEP, a polynomial-time heuristic. In contrast to prior art, STEP (i) jointly considers stateful and stateless MSs in its decisions, (ii) exploits MS shareability to reduce resource usage, (iii) balances response latency, App downtime, and resource utilization, and (iv) leverages multiple versions of the same MS to adapt QoS to available edge resources. Results show that STEP achieves near-optimal performance with only 1.6% higher deployment cost while reducing CPU usage by 42% compared to baselines. Also, it enables real-time App deployment in a large-scale scenario on a Kubernetes cluster with sub-second order execution time and reduced deployment cost by 16-17% compared to its benchmarks.

In resource-constrained cloud systems, e.g., at the network edge or in private clouds, serverless computing is increasingly adopted to deploy microservices-based applications, leveraging its promised high resource efficiency. Provisioning resources to serverless services, however, poses several challenges, due to the high cold-start latency of containers and stringent Service Level Agreement (SLA) requirements of the microservices. In response, we investigate the behavior of containers in different states (i.e., running, warm, or cold) and exploit our experimental observations to formulate an optimization problem that minimizes the energy consumption of the active servers while reducing SLA violations. In light of the problem complexity, we propose a low-complexity algorithm, named AiW, which utilizes a multi-queueing approach to balance energy consumption and system performance by reusing containers effectively and invoking cold-starts only when necessary. To further minimize the energy consumption of data centers, we introduce the two-timescale COmputing resource Management at the Edge (COME) framework, comprising an orchestrator running our proposed AiW algorithm for container provisioning and Dynamic Server Provisioner (DSP) for dynamically activating/deactivating servers in response to AiW’s decisions on request scheduling. COME addresses the mismatch in timescales for resource provisioning decisions at the container and server levels. Extensive performance evaluation through simulation shows AiW’s close match to the optimum and COME’s significant reduction in power consumption by 22–64% compared state-of-the-art alternatives.

Serverless edge computing allows for highly efficient resource utilization, reducing the energy footprint of edge data centers. Indeed, the containers can be dynamically created and destroyed, allowing to adapt the workload to the available resources. Creating containers upon arrivals of service requests entails, however, a high start-up latency, which may be unsuitable for time-critical services. As alternative solution, pre-started containers (“warm containers”) are used to decrease start-up latency, but incurring in higher resource costs. In this work, we minimize the energy consumption of the active servers in the data center by optimally managing the various container states while meeting the target delay of the requested services. Further, in light of the problem complexity, we investigate how a simple threshold-based algorithm performs and show that it can closely match the optimum.

In resource-constrained cloud systems, e.g., at the network edge or in private clouds, it is essential to deploy microservices (MSs) efficiently. Unlike most of the existing approaches, we tackle this issue by accounting for two important facts: (i) the interference that arises when MSs compete for the same resources and degrades their performance, and (ii) the MSs’ deployment time. In particular, we first present some experiments highlighting the impact of interference on the throughput of MSs co-located in the same server, as well as the benefits of MSs’ parallel deployment. Then, we formulate an optimization problem that minimizes the number of used servers while meeting the MSs’ performance requirements. In light of the problem complexity, we design a low-complexity heuristic, called iPlace, that clusters together MSs competing for resources as diverse as possible and, hence, interfering as little as possible. Importantly, clustering MSs also allows us to exploit the benefit of parallel deployment, which greatly reduces the deployment time as compared to the sequential approach applied in prior art and by default in state-of-the-art orchestrators. Our numerical results show that iPlace closely matches the optimum and uses 21-92% fewer servers compared to alternative schemes while proving to be highly scalable. Further, by deploying MSs in parallel using Kubernetes, iPlace reduces the deployment time by 69% compared to state-of-the-art solutions.

Network Functions Virtualization (NFV) promises programmability and cost savings by replacing hardware middleboxes with more flexible Virtual Network Functions (VNFs) on commodity servers. But, the current virtualization technologies do not fully isolate the system resources like Last Level Cache (LLC) and Memory Bandwidth (MB); therefore, co-location of VNFs on the same commodity server causes interference effects which might severely impact the performance of VNFs in terms of throughput, latency, etc. Contention at LLC is one of the root causes of this performance degradation and it is addressed by LLC resource partitioning. But, it remains unexplored the impact of both LLC and MB on VNF performance. In this work, we investigate the importance of MB partitioning along with LLC partitioning to achieve performance isolation in NFV-based systems. Allocating these system resources among co-located VNFs to meet Service Level Agreements (SLAs) is challenging due to the dynamic nature of traffic and varying functionality of VNFs. In this work, we formulate the resource allocation problem as an Integer Linear Programming (ILP) for maximizing the number of accepted VNF requests with SLA guarantees. Since the problem is NP-hard, we present a polynomial time$\epsilon $-approximation scheme. Further, we propose a heuristic approach namedNFVPermit, a resource manager for NFV-based systems that tries to ensure performance isolation among co-located VNFs based on their current traffic rates and SLA requirements. Through extensive experiments, we show howNFVPermitoutperforms state-of-the-art and baseline approaches.

Efficiently deploying microservices (MSs) is critical, especially in data centers at the edge of the network infrastructure where computing resources are precious. Unlike most of the existing approaches, we tackle this issue by accounting for the interference that arises when MSs compete for the same resources and degrades their performance. In particular, we first present some experiments highlighting the impact of interference on the throughput of co-located MSs. Then, we formulate an optimization problem that minimizes the number of used servers while meeting the MSs’ performance requirements. In light of the problem complexity, we design a low-complexity heuristic, called iPlace, that clusters together MSs competing for resources as diverse as possible and, hence, interfering as little as possible. Importantly, the choice of clustering MSs allows us to exploit the benefit of parallel MSs deployment, which, as shown by experimental evidence, greatly reduces the deployment time as compared to the sequential approach applied in prior art. Our numerical results show that iPlace closely matches the optimum and uses 10-63% fewer servers compared to alternative schemes, while proving to be highly scalable.