Current cloud environments provide numerous storage functions to virtual machines such as disk encryption, snapshotting, compression and so on. These functions are implemented using software stacks inside the hypervisor’s kernel, emulator, or as a userspace polling driver like SPDK. However, each stack brings its own limitations: Linux’s kernel I/O stack cannot easily integrate proprietary technologies such as Intel SGX, while SPDK requires significant changes in software development and tooling yet lacks the rich feature set of existing solutions like Linux LVM. To remedy these limitations, we introduce NVMetro, a high-performance storage framework for virtual machines based on the NVMe protocol. NVMetro provides multiple I/O paths that can be dynamically combined to fit the needs of each storage function. It links these paths together with an eBPF-based I/O router/classifier framework, as well as a userspace software stack for out-of-kernel I/O processing. We implemented three different storage functions with NVMetro and evaluated them under various workloads. Our results show that NVMetro approaches the performance of kernel-bypass solutions like SPDK while maintaining the compatibility and ease of use of in-kernel storage stacks.

Nowadays, most services are delivered through the web and thus heavily rely on JavaScript (JS). To accommodate the need for more performance, JS runtimes integrated Just-In-Time (JIT) compilation engines, which compile frequently-called portions of code for faster execution. To produce efficient machine code, the JIT applies complex optimization passes on the code in question. However, inadequate modeling of the side effects of these optimizations can introduce vulnerabilities in certain optimization passes. Such vulnerabilities are regularly discovered, and often have a high impact. Once a vulnerability is identified, it is eventually patched, but not without involving several steps (development, testing, release, user consent), leaving the system vulnerable for a relatively long period: the vulnerability window. We propose JITBULL, a solution that secures the JIT engines of JS runtimes during the vulnerability window by leveraging a vulnerability's demonstrator codes. To that end, JITBULL extracts the effects of JIT compiler optimization passes on said vulnerability demonstrator codes. For every subsequent JITed code, JITBULL compares the effects of its optimization passes with those on the demonstrator codes. If similarities are detected, JITBULL assumes that the currently executing script may be malicious and disables the related optimization passes, or if that's not possible, the whole JIT engine. We implemented JITBULL in Firefox's JS runtime (SpiderMonkey) and tested it against several known vulnerabilities with public demonstrator codes. Our results demonstrate that JITBULL consistently safeguards the JIT engine against exploitation by a variant of a known vulnerability. Moreover, we show that JITBULL exhibits a false positive rate of less than 5 % on the JS Octane benchmark suite, while causing an acceptable overhead of less than 20 %.

Recent advances in storage hardware have resulted in massive improvements in both I/O latency and throughput. However, existing storage virtualization tools either depend on a heavy and inefficient I/O stack that is not optimized for parallelism, or require a separate API that is difficult to manage and monitor. In this work, we introduce NVMetro, a solution based on the NVMe protocol that proposes a flexible choice between multiple I/O paths to ease the development of adaptive and performant virtual storage. NVMetro provides two components: (1) an intelligent I/O classification and routing framework powered by eBPF; and (2) an easy-to-use and performant API to assist the creation of userspace I/O functions within our framework. We demonstrate the benefits of NVMetro by implementing two virtual storage functions, and we evaluate them using various benchmarks. The obtained results show that NVMetro achieves a performance and scalability comparable to bleeding-edge, kernel-bypass technologies while retaining the flexibility of traditional OS-based storage APIs.

The reliability of virtualization infrastructures in the face of availability issues is a long-standing problem. Current fault tolerance approaches such as live VM replication are effective at addressing external, accidental issues (e.g. hardware failures, power cuts, environmental disasters); however, against an active attacker exploiting zero-day denial-of-service (DoS) vulnerabilities in the hypervisor itself, these approaches do not address the root cause of said vulnerabilities, and therefore cannot protect against these issues. This is made more relevant by the prevalence of DoS vulnerabilities among many widely used hypervisors.

Summary Nowadays, datacenters lean on a computer‐centric approach based on monolithic servers which include all necessary hardware resources (mainly CPU, RAM, network, and disks) to run applications. Such an architecture comes with two main limitations: (1) difficulty to achieve full resource utilization and (2) coarse granularity for hardware maintenance. Recently, many works investigated a resource‐centric approach called disaggregated architecture where the datacenter is composed of self‐content resource boards interconnected using fast interconnection technologies, each resource board including instances of one resource type. The resource‐centric architecture allows each resource to be managed (maintenance, allocation) independently. LegoOS is the first work which studied the implications of disaggregation on the operating system, proposing to disaggregate the operating system itself. They demonstrated the suitability of this approach, considering mainly CPU and RAM resources. However, they did not study the implication of disaggregation on network resources. We reproduced a LegoOS infrastructure and extended it to support disaggregated networking. We show that networking can be disaggregated following the same principles, and that classical networking optimizations such as DMA, DDIO, or loopback can be reproduced in such an environment. Our evaluations show the viability of the approach and the potential of future disaggregated infrastructures.

Function as a Service (FaaS) has become a key service in the cloud. It enables customers to conceive their appli-cation as a collection of minimal serverless functions interacting with each other. FaaS platforms abstract all the management complexity to the client. This emerging paradigm is also attractive because of its billing model. Clients are charged based on the execution time of functions, allowing finer-grained pricing. There-fore, executing functions as fast as possible is very important to lower the cost. Several research studies have investigated runtime optimization in FaaS environments, but none have explored CPU cache allocation. Indeed, CPU cache contention is a well-known issue in software and FaaS is not exempt from this issue. Various hardware improvements have been made to address the CPU cache partitioning problem. Among other things, Intel has implemented a new technology in their new processors that allows cache partitioning: Cache Allocation Technology (CAT). This technology allows allocating cache ways to processes, and the usage of the cache by each process will be limited to the allocated amount. In this paper, we propose CASY (CPU Cache Allocation SYstem), a system that performs CPU cache allocation for serverless functions using the Intel CAT technology. CASY uses machine learning to build a cache usage profile for functions and uses this profile to predict the cache requirements based on the function's input data. Because the CPU's cache size is small, CASY integrates an allocation algorithm which ensures that the cache loads are balanced on all cache ways. We implemented our system and integrated it into the Open Whisk FaaS platform. Our evaluations show a 11 % decrease in execution time for some serverless functions without degrading the performance of other functions.

Nowadays, virtualization is a central element in data centers as it allows sharing server resources among multiple users across virtual machines (VM). These servers often follow a Non-Uniform Memory Access (NUMA) architecture, consisting of independent nodes with their own cache hierarchies and I/O controllers. In this work, we investigate the impact of such an architecture on network access. As network devices are typically connected to one particular NUMA node, this leads to a situation where device access on one node is faster than another. This phenomenon is called Non-Uniform I/O Access (NUIOA). This non-uniformity impacts the performance of I/O applications that are not executed on the correct NUMA node. In this paper, we are interested in NUIOA effects in virtualized environments. Our contribution in this work is twofold: 1) we thoroughly study the impact of NUIOA on application performance in VMs, and 2) we propose a resource allocation strategy for VMs that reduces the impact of NUIOA. We implemented our allocation strategy on the Xen hypervisor and carried out evaluations with well-known benchmarks to validate our strategy. The obtained results show that with our NUIOA allocation scheme, we can improve the performance of application in VMs by up to 20 % compared to common allocation strategies.

Resource management is a critical issue in today’s virtualized computing infrastructures. Consolidation is the main technique used to optimize such infrastructure. Regarding memory management, it allows gathering overloaded and underloaded VM on the same server so that memory can be mutualized. However, because of infrastructures constraints and complexity of managing multiple resources, consolidation can hardly optimize memory management. In this article, we propose to rely on a remote memory sharing for mutualizing memory. We implemented a system which monitors the working set of virtual machines, reclaims unused memory and makes it available (as a remote swap device) for virtual machines which need memory. Our evaluations with HPC and Big Data benchmarks demonstrate the effectiveness of this approach. We show that remote memory can improve the performance of a standard Spark benchmark by up the 17 percent with an average performance degradation of 1.5 percent (for the providing application).

The vulnerability window of a hypervisor regarding a given security flaw is the time between the identification of the flaw and the integration of a correction/patch in the running hypervisor. Most vulnerability windows, regardless of severity, are long enough (several days) that attackers have time to perform exploits. Nevertheless, the number of critical vulnerabilities per year is low enough to allow an exceptional solution. This paper introduces hypervisor transplant, a solution for addressing vulnerability window of critical flaws. It involves temporarily replacing the current datacenter hypervisor (e.g., Xen) which is subject to a critical security flaw, by a different hypervisor (e.g., KVM) which is not subject to the same vulnerability.

Nested/Extended Page Table (EPT) is the current hardware solution for virtualizing memory in virtualized systems. It induces a significant performance overhead due to the 2D page walk it requires, thus 24 memory accesses on a TLB miss (instead of 4 memory accesses in a native system). This 2D page walk constraint comes from the utilization of paging for managing virtual machine (VM) memory. This paper shows that paging is not necessary in the hypervisor. Our solution Compromis, a novel Memory Management Unit, uses direct segments for VM memory management combined with paging for VM's processes. This is the first time that a direct segment based solution is shown to be applicable to the entire VM memory while keeping applications unchanged. Relying on the 310 studied datacenter traces, the paper shows that it is possible to provision up to 99.99% of the VMs using a single memory segment. The paper presents a systematic methodology for implementing Compromis in the hardware, the hypervisor and the datacenter scheduler. Evaluation results show that Compromis outperforms the two popular memory virtualization solutions: shadow paging and EPT by up to 30% and 370% respectively.

This paper addresses the problem of efficiently virtualizing NUMA architectures. The major challenge comes from the fact that the hypervisor regularly reconfigures the placement of a virtual machine (VM) over the NUMA topology. However, neither guest operating systems (OSes) nor system runtime libraries (e.g., Hotspot) are designed to consider NUMA topology changes at runtime, leading end user applications to unpredictable performance. This paper presents eXtended Para-Virtualization (XPV), a new principle to efficiently virtualize a NUMA architecture. XPV consists in revisiting the interface between the hypervisor and the guest OS, and between the guest OS and system runtime libraries (SRL) so that they can dynamically take into account NUMA topology changes. The paper presents a methodology for systematically adapting legacy hypervisors, OSes, and SRLs. We have applied our approach with less than 2k line of codes in two legacy hypervisors (Xen and KVM), two legacy guest OSes (Linux and FreeBSD), and three legacy SRLs (Hotspot, TCMalloc, and jemalloc). The evaluation results showed that XPV outperforms all existing solutions by up to 304%.

vNUMA is the most recent technology used by hypervisors to deal with Non Uniform Memory Access (NUMA) machines, which currently composed most datacenters. vNUMA consists in presenting to the virtual machine (VM) the initial mapping (at boot time) of its virtual resources to physical resources. By this way, all NUMA optimizations implemented by almost all VM’s OS (e.g. Linux) can become effective. However, in order to be effective itself, vNUMA imposes that the initial resource mapping of the VM should remain unchanged during the VM lifetime. Current hypervisors enforce this requirement by avoiding virtual resource migration (between different NUMA nodes, in the same machine), VM migration (between different machines), and memory ballooning.However, we found that memory flipping the most efficient network virtualization approach violates the above requirement. In other words, a VM which performs network operations leads the hypervisor implicitly performs memory page migrations. In this paper, we show that violating this requirement can degrade performance by up to 18%. We present two solutions which mitigate the issue. We prototype these solutions in Xen hypervisor, a popular open source hypervisor, which is widely used by Amazon Web Services. The evaluation results, performed with well known benchmarks, show that our two solutions are able to almost cancel the issue, while keeping memory flipping effective.

In most of today's virtualized systems (e.g., Xen), the hypervisor relies on a privileged virtual machine (pVM). The pVM accomplishes work both for the hypervisor (e.g., VM life cycle management) and for client VMs (I/O management). Usually, the pVM is based on a standard OS (Linux). This is source of performance unpredictability, low performance, resource waste, and vulnerabilities. This paper presents Closer, a principle for designing a suitable OS for the pVM. Closer consists in respectively scheduling and allocating pVM's tasks and memory as close to the involved client VM as possible. By revisiting Linux and Xen hypervisor, we present a functioning implementation of Closer. The evaluation results of our implementation show that Closer outperforms standard implementations.

In this paper, we propose an effortless way for disaggregating the CPU-memory couple, two of the most important resources in cloud computing. Instead of redesigning each resource board, the disaggregation is done at the power supply domain level. In other words, CPU and memory still share the same board, but their power supply domains are separated. Besides this disaggregation, we make the two following contributions: (1) the prototyping of a new ACPI sleep state (called zombie and noted Sz) which allows to suspend a server (thus save energy) while making its memory remotely accessible; and (2) the prototyping of a rack-level system software which allows the transparent utilization of the entire rack resources (avoiding resource waste). We experimentally evaluate the effectiveness of our solution and show that it can improve the energy efficiency of state-of-the-art consolidation techniques by up to 86%, with minimal additional complexity.

In native Linux systems, spinlock's implementation relies on the assumption that both the lock holder thread and lock waiter threads cannot be preempted. However, in a virtualized environment, these threads are scheduled on top of virtual CPUs (vCPU) that can be preempted by the hypervisor at any time, thus forcing lock waiter threads on other vCPUs to busy wait and to waste CPU cycles. This leads to the well-known Lock Holder Preemption (LHP) and Lock Waiter Preemption (LWP) issues.

Summary Virtualized cloud infrastructures (also known as IaaS platforms) generally rely on a server consolidation system to pack virtual machines (VMs) on as few servers as possible. However, an important limitation of consolidation is not addressed by such systems. Because the managed VMs may be of various sizes (small, medium, large, etc.), VM packing may be obstructed when VMs do not fit available spaces. This phenomenon leaves servers with a set of unused resources (‘holes’). It is similar to memory fragmentation, a well‐known problem in operating system domain. In this paper, we propose a solution which consists in resizing VMs so that they can fit with holes. This operation leads to the management of what we call elastic VMs and requires cooperation between the application level and the IaaS level, because it impacts management at both levels. To this end, we propose a new resource negotiation and allocation model in the IaaS, calledHRNM. We demonstrate HRNM's applicability through the implementation of a prototype compatible with two main IaaS managers (OpenStack and OpenNebula). By performing thorough experiments with SPECvirt_sc2010 (a reference benchmark for server consolidation), we show that the impact of HRNM on customer's application is negligible. Finally, using Google data center traces, we show an improvement of about 62.5% for the traditional consolidation engines. Copyright © 2017 John Wiley & Sons, Ltd.

In virtualized cloud hosting centers, a virtual machine (VM) is generally allocated a fixed computing capacity. The virtualization system schedules the VMs and guarantees that each VM capacity is provided and respected. However, a significant amount of CPU time is consumed by the underlying virtualization system, which generally includes device drivers (mainly network and disk drivers). In today's virtualization systems, this CPU time consumed is difficult to monitor and it is not charged to VMs. Such a situation can have important consequences for both clients and provider: performance isolation and predictability for the former and resource management (and especially consolidation) for the latter. In this paper, we propose a virtualization system mechanism which allows estimating the CPU time used by the virtualization system on behalf of VMs. Subsequently, this CPU time is charged to VMs, thus removing the two previous side effects. This mechanism has been implemented in Xen. Its benefits have been evaluated using reference benchmarks.

Scheduling has a significant influence on application performance. Deciding on a quantum length can be very tricky, especially when concurrent applications have various characteristics. This is actually the case in virtualized cloud computing environments where virtual machines from different users are colocated on the same physical machine. We claim that in a multi-core virtualized platform, different quantum lengths should be associated with different application types. We apply this principle in a new scheduler called AQL_Sched. We identified 5 main application types and experimentally found the best quantum length for each of them. Dynamically, AQL_Sched associates an application type with each virtual CPU (vCPU) and schedules vCPUs according to their type on physical CPU (pCPU) pools with the best quantum length. Therefore, each vCPU is scheduled on a pCPU with the best quantum length. We implemented a prototype of AQL_Sched in Xen and we evaluated it with various reference benchmarks (SPECweb2009, SPECmail2009, SPEC CPU2006, and PARSEC). The evaluation results show that AQL_Sched outperforms Xen's credit scheduler. For instance, up to 20%, 10% and 15% of performance improvements have been obtained with SPECweb2009, SPEC CPU2006 and PARSEC, respectively.

Designing a cloud infrastructure for HPC applications requires to correlate design choices for virtualization solutions, resources management strategies, and their implementation on specific hardware platforms. Such investigations can hardly be conducted without accurate simulation tools. This paper introduces VMcSim, the first micro architecture and virtualized many core simulation framework. VMcSim models a x86-based asymmetric many core micro architecture, including a virtualization layer which allows to model a hyper visor, a set of virtual machines with their virtual resources (CPU and memory), their operating system, and their applications. By simulating all the involved hardware and software layers, VMcSim outperforms other simulators. Simulation accuracy is evaluated through several benchmark (SPLASH-2). The simulator is demonstrated with the evaluation of virtual resources placement policies in multicore systems.