Containers are a mainstream virtualization technique for running stateful workloads over persistent storage. In highly utilized multitenant hosts, resource contention at the system kernel leads to inefficient container input/output (I/O) handling. Although there are interesting techniques to address this issue, they incur high implementation complexity and execution overhead. As a cost-effective alternative, we introduce the Diciclo architecture with our assumptions, goals, and principles. For each tenant, Diciclo isolates the control and data I/O path at user level and runs dedicated storage systems. Diciclo includes the libservice unified user-level abstraction of system services and the node structure design pattern for the application and server side. We prototyped a toolkit of user-level components that comprise the library to invoke the standard I/O calls, the I/O communication mechanism, and the I/O services. Based on Diciclo, we built Danaus, a filesystem client that integrates a union filesystem with a Ceph distributed filesystem client and configurable shared cache. Across different host configurations, workloads, and systems, Danaus achieves improved performance stability, because it handles I/O with reserved per-tenant resources and avoids intensive kernel locking. Based on having built and evaluated Danaus, we share valuable lessons about resource contention, file management, service separation, and performance stability in multitenant systems.

Containers are a mainstream virtualization technique commonly used to run stateful workloads over persistent storage. In multi-tenant hosts with high utilization, resource contention at the system kernel often leads to inefficient handling of the container I/O. Assuming a distributed storage architecture for scalability, resource sharing is particularly problematic at the client hosts serving the applications of competing tenants. Although increasing the scalability of a system kernel can improve resource efficiency, it is highly challenging to refactor the kernel for fair access to system services. As a realistic alternative, we isolate the storage I/O paths of different tenants by serving them with distinct clients running at user level. We introduce the Danaus client architecture to let each tenant access the container root and application filesystems over a private host path. We developed a Danaus prototype that integrates a union filesystem with a Ceph distributed filesystem client and a configurable shared cache. Across different host configurations, workloads and systems, Danaus achieves improved performance stability because it handles I/O with reserved per-tenant resources and avoids intensive kernel locking. Danaus offers up to 14.4x higher throughput than a popular kernel-based client under conditions of I/O contention. In comparison to a FUSE-based user-level client, Danaus also reduces by 14.2x the time to start 256 high-performance webservers. Based on our extensive experience from building and evaluating Danaus, we share several valuable lessons that we learned about resource contention, file management, service separation and performance stability.

The operation of modern systems requires the low latency and high throughput of producer-consumer communication over shared memory. In order to achieve fast communication at high concurrency, we define a relaxed ordering model that splits the queue operations into two stages, the sequential assignment to queue slots and their subsequent concurrent execution. Based on this model, we design and implement the linearizable and lock-free algorithm called Relaxed Concurrent Queue Single (RCQS). We experimentally show that RCQS achieves factors to orders of magnitude advantage over the state-of-the-art queue algorithms in operation latency and item transfer speed.

Critical operations are often implemented in roughly the same way across multiple platforms, but differently by software systems running on the same platform. This observation is arguably justified by the potential restrictions of each software system, but it is surprising given the operation sensitivity to numerous platform-specific software and hardware parameters. With initial focus on the memory copy operation (memcpy), we introduce a methodology based on exhaustive search to optimize the performance across different platforms. We design and implement the Asterope algorithm to experimentally generate optimal memcpy parameters for two x86-64 processor models from different vendors. With experiments on microbenchmarks and two production systems, we demonstrate that Asterope respectively achieves up to 2.4x and 1.9x higher function and system performance in comparison to using the Linux kernel memcpy.

Software containers limit the performance of data-intensive applications due to storage I/O contention in the system kernel of the host. Although kernel partitioning has been claimed as a promising approach, it is impractical due to implementation complexity. As a pragmatic alternative, we suggest that the tenants of a host should run their own user-level filesystems. We introduce the Polytropon toolkit as a collection of user-level components configurable to build several types of filesystems. The toolkit provides an application library to invoke the standard I/O calls, a user-level path to isolate the tenant I/O traffic to private host resources, and user-level filesystem services distinct per tenant. Furthermore, we introduce the RCQB concurrent queue that relaxes operation ordering for improved communication throughput, and we provide the SMO pipelined memory copy that is faster than standard methods. We use Polytropon to build the client of a distributed filesystem optionally combined with a union filesystem. Polytropon serves 32 tenants with RocksDB over network storage with 7.2--14x lower latency than kernel systems, and reduces up to 2.9x the timespan to execute source-code processing in the containers of 32 tenants. RCQB achieves higher performance than the state-of-the-art queues by 4--52x, while SMO achieves 29--66% higher data transfer throughput than existing methods.

In a virtualization environment that serves multiple tenants (independent organizations), storage consolidation at the filesystem level is desirable because it enables data sharing, administration efficiency, and performance optimizations. The scalable deployment of filesystems in such environments is challenging due to intermediate translation layers required for networked file access or identity management. First we define the entities involved in a multitenant filesystem and present relevant security requirements. Then we introduce the design of the Dike authorization architecture. It combines native access control with tenant namespace isolation and compatibility to object-based filesystems. We introduce secure protocols to authenticate the participating entities and authorize the data access over the network. We alternatively use a local cluster and a public cloud to experimentally evaluate a Dike prototype implementation that we developed. At several thousand tenants, our prototype incurs limited performance overhead below 21 percent, unlike a solution from industry whose multitenancy overhead approaches 84 percent in some cases.

In a virtualization environment that serves multiple tenants, storage consolidation at the filesystem level is desirable because it enables data sharing, administration efficiency, and performance optimizations. The scalable deployment of filesystems in such environments is challenging due to intermediate translation layers required for networked file access or identity management. First we present several security requirements in multitenant filesystems. Then we introduce the design of the Dike authorization architecture. It combines native access control with tenant namespace isolation and compatibility to object-based filesystems. We use a public cloud to experimentally evaluate a prototype implementation of Dike that we developed. At several thousand tenants, our prototype incurs limited performance overhead up to 16%, unlike an existing solution whose multitenancy overhead approaches 84% in some cases.