Service meshes are becoming pivotal software frameworks for managing communication among microservices in distributed applications. Each microservice in a service mesh is paired with an L7 sidecar proxy, which intercepts incoming and outgoing requests to provide enhanced observability, traffic management, and security. These sidecar proxies use application-level load balancing policies to route outgoing requests to available replicas of destination microservices. A widely adopted policy is the Least Outstanding Request (LOR), which directs requests to the replica with the fewest outstanding requests. While LOR effectively reduces latency in applications with a small number of replicas, our comprehensive investigation– combining analytical, simulation, and experimental methods– uncovers a novel and critical issue for large-scale microservice applications: the performance of LOR significantly degrades as the number of microservice replicas increases, eventually converging to the performance of a random load balancing policy. To recover LOR performance at scale, we propose an opensource solution named Proxy-Service, tailored for microservice applications where load balancing incurs significantly lower resource demands than microservice execution. The core idea is to consolidate load balancing decisions per microservice into one or a few reverse proxies, transparently injected into the application.

Container technology plays an important role in the virtualization landscape today. A container uses its own file system consisting of a stack of layers, which are stored on the execution server's disk. Containers running on the same server share the layers they have in common, and this sharing results in valuable savings in server storage space. Many containers can run on a single server, but when their resource demands grow enough, they are distributed across a cluster of nodes/servers by orchestration systems, such as Kubernetes. In this work, we found that for the same amount of containers to run, the storage required is higher for clusters consisting of a larger number of nodes. The severity of this storage overhead depends on the scheduling policy used to select the nodes that run the containers. By comparing different storage-saving scheduling policies that differ from each other in the depth of storage knowledge they leverage to make decisions, our analysis reveals that only deep, layer-level knowledge can effectively counter the growth in storage demand as the cluster size increases. Policies with coarser-grained knowledge achieve limited benefit because they achieve performance that is nearly equal to that of a random, zero-knowledge policy.

$\mu$Bench is an open-source tool for benchmarking cloud/edge computing platforms that run microservice applications. The tool creates dummy microservice applications that can be customized and executed on a Kubernetes cluster.$\mu$Bench allows users to control fundamental properties of the microservice applications it creates, such as service mesh topology, microservices’ behaviors using a portfolio of stress functions (e.g., for CPU, memory, I/O, network) or implementing new ones, microservice-to-microservice API (HTTP or gRPC), etc. Application performance can be evaluated by stochastic or trace-driven workloads. μBench is aimed at researchers and cloud platform developers who lack real microservice applications to benchmark their findings (e.g., new resource control mechanisms, artificial intelligence-driven orchestration, etc.) or wish to thoroughly evaluate their proposals versus a broad set of heterogeneous applications that μBench can create. In addition to the description of μBench, in this article, we show one possible use of it. We compared advantages and disadvantages of microservice architectures versus monolithic ones, and analyzed the performance impact of key architectural choices, such as service mesh topology and the use of replication. For this analysis, we generated several microservice applications with different properties, and two of them are derived from a real cloud dataset.

Message Queuing Telemetry Transport (MQTT) is a widespread protocol for topic-based publish-subscribe architectures supporting IoT and social networks applications. MQTT brokers are logical entities that couple publishers and subscribers and play a critical role in such architectures. MQTT brokers can be implemented either as standalone servers or in a cluster configuration. Clusters of brokers increase reliability, availability and overall performance, since operations can be highly parallelized among the brokers that form the cluster. The load-balancing strategy in a cluster usually consists in connecting an incoming client to a randomly selected broker. This random-attach strategy, it is very simple, but generates a significant amount of inter-broker traffic, as we demonstrate through theoretical and experimental evaluations. Inter-broker traffic is an overhead for the system and it increases the CPU load of the brokers, compromising the scaling behavior of the whole cluster. Indeed, we found that a linear increase of the number of brokers forming a cluster does not necessarily provide an equivalent linear gain in performance, and such a scaling penalty can be surprisingly significant, in the order of 40%. To solve this issue and improve performance, we propose a novel load-balancing strategy that envisages the use of multiple MQTT sessions per client to reduce inter-broker traffic and that can be implemented by means of a greedy algorithm. We show feasibility and effectiveness of our strategy for IoT and social-network applications by means of simulations and real measurements. The resulting scaling penalty is reduced to 10%.