Multi-access Edge Computing (MEC) in Edge-cloud computing continuum environments is an emerging technology that enables locality-aware low-latency processing for requests from edge devices. Confidential Containers (CoCo) are an emerging confidential computing technology designed for protecting in-use data in cloud datacenters. Although containers in a MEC layer can help reduce latency for requests from edge devices, the current increasing demands for security features such as confidential computing and memory integrity protection will seriously affect expected end-to-end latencies of microservice applications. In this paper, we build a CoCo environment using Kata containers on the MEC layer and attempt to evaluate microservice-level application behavior for the advanced security features. Especially, we focus on breaking down the latency into memory access level, microservice component level, and the end-to-end latency level. From the results, we observe that 9% overhead is incurred for each memory access when applying confidential computing with memory integrity protection. At the individual microservices level, we observed the latency for gRPC communication and memcached access are increased up to 40 %. We also reveal that the latency overheads in microservice-level and application-level are from twice to 4 times larger than those observed in memory access level.

Time-Sensitive Network (TSN) has been considered one of the most viable solutions to meet the increasing demands for ultra-low latency in the 5G and post-5G eras. Due to the importance of timing constraints, network measurement tools are necessary for TSN research and operations to evaluate the performance limits of proofs-of-concept, troubleshoot failures, etc. However, current measurement solutions struggle to achieve important aspects of TSN networks, such as precise control of the transmission frame intervals and burst sizes. Additionally, they cannot generate a mix of multiple flows of various characteristics. In this paper, we present Ethernet Frame Crafter & Capture (EFCC), an FPGA-based network measurement tool to address the aforementioned issues. The proposed system allows the frame generation/capture functions to be configured independently for each port at a line rate of 1GbE. Furthermore, EFCC is open-source and can be ported to most commodity FPGA boards available in the market. The evaluation results demonstrate that the proposed frame generator allows the mixing of multiple flows of various characteristics and that the frame capture module can record the transmit and receive timestamps of frames with a precision of 8-ns/6.4-ns.

Time Sensitive Network (TSN) is one of the most auspicious solutions to respond to the increasing demands in ultra-low latency in the 5G and post-5G eras. It comes as an extension to the conventional IEEE 802.3 Ethernet networks by adding a set of novel open standards that aims to provide deterministic, reliable, high-bandwidth, and low-latency communication. In this paper, we present an open-source and light-weight FPGA-based network switch design and implementation supporting Credit Based Shaper (CBS) for Time Sensitive Networks. We present the key design components and implementation aspects of the proposed switch and discuss the preliminary evaluation results in a fair amount of detail to validate our proposal. The conducted experiments show that the proposed switch properly shapes the traffic by eliminating bursts in irregular traffic and efficiently forwards prioritized traffic as stipulated by the TSN requirements. In addition, we demonstrate that our hardware latency evaluation results conform with the CBS theoretical model and that the proposed switch consumes a very reasonable portion of the hardware resources on an affordable FPGA.

Deep Neural Networks (DNNs) have become increasingly computationally intensive and have larger parameters, requiring efficient parallelization or distribution using multiple accelerators. Pipeline parallelism has been proposed as an effective way to distribute models and improve hardware utilization. However, the problem with pipeline parallelism is the trade-off between speedup and accuracy: synchronous approaches do not provide sufficient speedup, while asynchronous approaches suffer from accuracy degradation due to a different scheme from a single worker. In this paper, we propose AshPipe, a hybrid parallel framework that combines data parallelism and asynchronous pipeline parallelism to achieve efficient speedup for training. The proposed runtime uses the 1F1B schedule and data parallelism, with non-uniform numbers of workers and identical global batch sizes across stages. A Switch Parallelism (SP) mechanism is also proposed as an option to mitigate accuracy degradation, which switches over from data parallelism to hybrid parallelism in the course of training. Experimental results show that AshPipe achieves 1.844× the throughput of data parallelism for ViT-H/14 whose parameter size is 632M. With the SP mechanism, AshPipe achieved a 30.2% reduction in training time with comparable accuracy compared to data parallelism when training on the CIFAR100 dataset.

Computed Tomography (CT) is a key 3D imaging technology that fundamentally relies on the compute-intense back-projection operation to generate 3D volumes. GPUs are typically used for back-projection in production CT devices. However, with the rise of power-constrained micro-CT devices, and also the emergence of CPUs comparable in performance to GPUs, back-projection for CPUs could become favorable. Unlike GPUs, extracting parallelism for back-projection algorithms on CPUs is complex given that parallelism and locality are not explicitly defined and controlled by the programmer, as is the case when using CUDA for instance. We propose a collection of novel back-projection algorithms that reduce the arithmetic computation, robustly enable vectorization, enforce a regular memory access pattern, and maximize the data locality. We also implement the novel algorithms as efficient back-projection kernels that are performance portable over a wide range of CPUs. Performance evaluation using a variety of CPUs from different vendors and generations demonstrates that our back-projection implementation achieves on average 5.2 times speedup over the multi-threaded implementation of the most widely used, and optimized, open library. With a state‐of‐the‐art CPU, we reach performance that rivals top-performing GPUs.

Filtered Back-Projection (FBP) is a fundamental compute intense algorithm used in tomographic image reconstruction. Cone-Beam Computed Tomography (CBCT) devices use a cone-shaped X-ray beam, in comparison to the parallel beam used in older CT generations. Distributed image reconstruction of cone-beam datasets typically relies on dividing batches of images into different nodes. This simple input decomposition, however, introduces limits on input/output sizes and scalability.

As real systems become larger and more complex, the use of simulator frameworks grows in our research community. By leveraging them, users can focus on the major aspects of their algorithm, run in-siclo experiments (i.e., simulations), and thoroughly analyze results, even for a large-scale environment without facing the complexity of conducting in-vivo studies (i.e., on real testbeds). Since nowadays the virtual machine (VM) technology has become a fundamental building block of distributed computing environments, in particular in cloud infrastructures, our community needs a full-fledged simulation framework that enables us to investigate large-scale virtualized environments through accurate simulations. To be adopted, such a framework should provide easy-to-use APIs as well as accurate simulation results. In this paper, we present a highly-scalable and versatile simulation framework supporting VM environments. By leveraging SimGrid, a widely-used open-source simulation toolkit, our simulation framework allows users to launch hundreds of thousands of VMs on their simulation programs and control VMs in the same manner as in the real world (e.g., suspend/resume and migrate). Users can execute computation and communication tasks on physical machines (PMs) and VMs through the same SimGrid API, which will provide a seamless migration path to IaaS simulations for hundreds of SimGrid users. Moreover, SimGrid VM includes a live migration model implementing the precopy migration algorithm. This model correctly calculates the migration time as well as the migration traffic, taking account of resource contention caused by other computations and data exchanges within the whole system. This allows user to obtain accurate results of dynamic virtualized systems. We confirmed accuracy of both the VM and the live migration models by conducting several micro-benchmarks under various conditions. Finally, we conclude the article by presenting a first use-case of one consolidation algorithm dealing with a significant number of VMs/PMs. In addition to confirming the accuracy and scalability of our framework, this first scenario illustrates the main interest of SimGrid VM: investigating through in-siclo experiments pros/cons of new algorithms in order to limit expensive in-vivo experiments only to the most promising ones.

A network latency emulator allows IT architects to thoroughly investigate how network latencies impact workload performance. Software-based emulation tools have been widely used by researchers and engineers. It is possible to use commodity server computers for emulation and set up an emulation environment quickly without outstanding hardware cost. However, existing software-based tools built in the network stack of an operating system are not capable of supporting the bandwidth of today's standard interconnects (e.g., 10GbE) and emulating sub-milliseconds latencies likely caused by network virtualization in a datacenter. In this paper, we propose a network latency emulator (DEMU) supporting broad bandwidth traffic with sub-milliseconds accuracy, which is based on an emerging packet processing framework, DPDK. It avoids the overhead of the network stack by directly interacting with NIC hardware. Through experiments, we confirmed that DEMU can emulate latencies on the order of 10 μs for short-packet traffic at the line rate of 10GbE. The standard deviation of inserted delays was only 2-3 μs. This is a significant improvement from a network emulator built in the Linux Kernel (i.e., NetEm), which loses more than 50% of its packets for the same 10GbE traffic. For 1 Gbps traffic, the latency deviation of NetEm was approximately 20 μs, while that of our mechanism was 2 orders of magnitude smaller (i.e., only 0.3 μs).

In the future, STT-MRAM will achieve larger capacity and comparable read/write performance, but incur orders of magnitude greater write energy than DRAM. To achieve large capacity as well as energy-efficiency, it is necessary to use both DRAM and STT-MRAM for the main memory of a computer. In this paper, we propose a hypervisor-based hybrid memory mechanism (RAMinate) that reduces write traffic to STT-MRAM by optimizing page locations between DRAM and STT-MRAM. In contrast to past studies, our mechanism works at the hypervisor level, not at the hardware or operating system level. It does not require any special program at the operating system level nor any design changes of the current memory controller at the hardware level. We developed a prototype of the proposed system by extending Qemu/KVM and conducted experiments with application benchmarks. We confirmed that our page replacement mechanism successfully worked for unmodified operating systems and dynamically diverted memory write traffic to DRAM. Our experiments also confirmed that our system successfully reduced write traffic to STT-MRAM by approximately 70% for tested workloads, which results in a 50% reduction in energy consumption in comparison to a DRAM-only system.

Predicting performance of a given program on a given machine is highly important because the environment where the program is developed and the one where it is actually executed are often different. However, this prediction is also difficult because the performance of the same program on different machines is not the same, due to the different balances in performance of the various computer components (e.g. CPU, memory, etc.). Although many studies tackle this problem by modelling the target program and/or the target machine, model-based techniques can only provide what they model and cannot leverage existing performance analysis tools. In this paper, we tackle this problem by actually executing the target program in an emulated environment, where the performance balance of the CPU and the memory subsystem is virtually tweaked using a dynamic binary instrumentation technique. We show that this approach can emulate the total execution time of a memory-access-intensive application on different machines, and provide a vision of the future, showing how our approach can outperform existing model-based approaches.

Virtualization techniques greatly benefit cloud computing. Live migration enables a datacenter to dynamically replace virtual machines (VMs) without disrupting services running on them. Efficient live migration is the key to improve the energy efficiency and resource utilization of a datacenter through dynamic placement of VMs. Recent studies have achieved efficient live migration by deleting the page cache of the guest OS to shrink the memory size of it before a migration. However, these studies do not solve the problem of IO performance penalty after a migration due to the loss of page cache. We propose an advanced memory transfer mechanism for live migration, which skips transferring the page cache to shorten total migration time while restoring it transparently from the guest OS via the SAN to prevent IO performance penalty. To start a migration, our mechanism collects the mapping information between page cache and disk blocks. During a migration, the source host skips transferring the page cache but transfers other memory content, while the destination host transfers the same data as the page cache from the disk blocks via the SAN. Experiments with web server and database workloads showed that our mechanism reduced total migration time with significantly small IO performance penalty.

Energy efficiency of cloud data centers is of great concern today and has been tackled by many researchers. Dynamic VM placement is a well-known strategy to improve energy efficiency of a data center. Virtual machines (VMs) under light load are consolidated into a small number of physical machines (PMs) to turn idle PMs into low-power states. Although live migration is essential for dynamic VM placement, former studies have not yet revealed how energy overhead of live migration has impact on energy efficiency of dynamic VM placement. To tackle this problem, we conducted integrated simulation of energy overhead of live migration and dynamic VM placement sing Sim Grid. We used three dynamic VM placement policies and two live migration mechanisms (existing pre-copy and an accelerated mechanism invented by us) to thoroughly evaluate the energy overhead. The results showed that in the worst case energy overhead of live migration occupies 5.8% of total energy consumption of a data center.

Live migration of virtual machines over a wide area network has many use cases such as cross-data center load balancing, low carbon virtual private clouds, and disaster recovery of IT systems. An efficient wide area live migration method is required because cross-data center connections have a narrow bandwidth. Page cache occupies a large portion of the memory of a Virtual Machine (VM) when it executes data-intensive workloads. We propose a new live migration technique, page cache teleportation, which reduces the total migration time of wide area live migration and has a low overhead. It detects the restorable page cache in the guest memory that has the same contents as the corresponding disk blocks. The restorable page cache is not transferred via the WAN but is restored from the disk image before the VM resumes. In this way, the IO performance degradation reduces after the migration. Evaluations show that page cache teleportation reduces the total migration time of wide area live migration and has a lower performance overhead than existing approaches.

Although virtual machine (VM) placement problem has been an active research area over the past decade, the research community is still looking for an open simulation framework that can simulate in an accurate as well as scalable manner VM operations including live migrations. Existing frameworks, however, leverage a naive migration model that considers neither memory update operations nor resource sharing contention, resulting in an underestimate of both the duration of a live migration and the size of migration traffic. In this paper, we propose a simulation framework of virtualized distributed systems with the first class support of live migration operations. We developed a resource share calculation mechanism for VMs and a live migration model implementing the precopy migration algorithm of Qemu/KVM. We extended a widely used simulation toolkit, SimGrid, which allows users to simulate large-scale distributed systems by using user-friendly programming API. Through experiments, we confirmed that our simulation framework correctly reproduced live migration behaviors of the real world under various conditions. Through a first use case, we also confirmed that it is possible to conduct large-scale simulations of complex virtualized workloads upon hundred thousands of VMs upon thousands of physical machines (PMs).

This paper describes EXECO, a library that provides easy and efficient control of local or remote, standalone or parallel, processes execution, as well as tools designed for scripting distributed computing experiments on any computing platform. After discussing the EXECO internals, we illustrate its interest by presenting two experiments dealing with virtualization technologies on the Grid'5000 testbed.

In Infrastructure-as-a-Service datacenters, the placement of Virtual Machines (VMs) on physical hosts are dynamically optimized in response to resource utilization of the hosts. However, existing live migration techniques, used to move VMs between hosts, need to involve large data transfer and prevents dynamic consolidation systems from optimizing VM placements efficiently. In this paper, we propose a technique called “memory reusing” that reduces the amount of transferred memory of live migration. When a VM migrates to another host, the memory image of the VM is kept in the source host. When the VM migrates back to the original host later, the kept memory image will be “reused”, i.e. memory pages which are identical to the kept pages will not be transferred. We implemented a system named MiyakoDori that uses memory reusing in live migrations. Evaluations show that MiyakoDori significantly reduced the amount of transferred memory of live migrations and reduced 87% of unnecessary energy consumption when integrated with our dynamic VM consolidation system.

An HPC cloud, a flexible and robust cloud computing service specially dedicated to high performance computing, is a promising future e-Science platform. In cloud computing, virtualization is widely used to achieve flexibility and security. Virtualization makes migration or checkpoint/restart of computing elements (virtual machines) easy, and such features are useful for realizing fault tolerance and server consolidations. However, in widely used virtualization schemes, I/O devices are also virtualized, and thus I/O performance is severely degraded. To cope with this problem, VMM-bypass I/O technologies, including PCI passthrough and SR-IOV, in which the I/O overhead can be significantly reduced, have been introduced. However, such VMM-bypass I/O technologies make it impossible to migrate or checkpoint/restart virtual machines, since virtual machines are directly attached to hardware devices. This paper proposes a novel and practical mechanism, called Symbiotic Virtualization (SymVirt), for enabling migration and checkpoint/restart on a virtualized cluster with VMM-bypass I/O devices, without the virtualization overhead during normal operations. SymVirt allows a VMM to cooperate with a message passing layer on the guest OS, then it realizes VM-level migration and checkpoint/restart by using a combination of a PCI hotplug and coordination of distributed VMMs. We have implemented the proposed mechanism on top of QEMU/KVM and the Open MPI system. All PCI devices, including Infiniband and Myrinet, are supported without implementing specific para-virtualized drivers; and it is not necessary to modify either of the MPI runtime and applications. Using the proposed mechanism, we demonstrate reactive and proactive FT mechanisms on a virtualized Infiniband cluster. We have confirmed the effectiveness using both a memory intensive micro benchmark and the NAS parallel benchmark. Moreover, we also show that postcopy live migration enables us to reduce the down time of an application as the memory footprint increases.

Virtualized IT infrastructures combined with virtual machine migration technologies have a potential to support IT services that are resilient to partial physical infrastructure failures caused by extreme events. This paper experimentally evaluates the migration of multiple VMs across long geographical distances - an activity that is required to move virtualized IT systems from a disaster site to a safe location. Taking into account the resource availability parameters observed after the Great East Japan Earthquake, experimental results show that if (1) service downtime in the order of minutes is acceptable, (2) VMs can be kept with small storage footprint, and (3) power and network are available for tens of minutes, it is possible to migrate tens of VMs from damaged sites to a very distant stable location.

Wide-area live migration of virtual machines (VMs) is a key to advanced cloud federation, allowing dynamic and transparent load balancing among data centers. Although Mobile IPv6 (MIPv6) provides strong network infrastructure for mobile nodes, there still exists a missing link to the achievement of MIPv6-based VM migration. Real-world IaaS datacenters require guest-transparent and flexible tunneling mechanisms, which are not provided by existing MIPv6 programs. In this paper, we propose a guest-transparent MIPv6 tunneling mechanism (Kagemusha), that performs Client MIPv6 signaling and tunneling on a host operating system. No MIPv6 program is required to be installed into a guest operating system. The proposed system is fully compatible with existing home agents. It basically works with most virtual machine monitors, including Qemu/KVM and Xen. We have developed the first proof-of-concept prototype of the proposed mechanism. Our experiments showed that our prototype system successfully created MIPv6 tunnels with existing home agents. Its performance overhead was negligible for normal use cases. We also confirmed that the prototype system successfully supported live migration, transparently achieving continuous network reachability for migrated VMs. The downtime of migration increased only by several hundred milliseconds.

We are developing an efficient resource management system with aggressive virtual machine (VM) relocation among physical nodes in a data center. Existing live migration technology, however, requires a long time to change the execution host of a VM, it is difficult to optimize VM packing on physical nodes dynamically, corresponding to ever-changing resource usage. In this paper, we propose an advanced live migration mechanism enabling instantaneous relocation of VMs. To minimize the time needed for switching the execution host, memory pages are transferred after a VM resumes at a destination host. A special character device driver allows transparent memory page retrievals from a source host for the running VM at the destination. In comparison with related work, the proposed mechanism supports guest operating systems without any modifications to them (i.e, no special device drivers and programs are needed in VMs). It is implemented as a lightweight extension to KVM (Kernel-based Virtual Machine Monitor). It is not required to modify critical parts of the VMM code. Experiments were conducted using the SPECweb2005 benchmark. A running VM with heavily-loaded web servers was successfully relocated to a destination within one second. Temporal performance degradation after relocation was resolved by means of a precaching mechanism for memory pages. In addition, for memory intensive workloads, our migration mechanism moved all the states of a VM faster than existing migration technology.

IaaS (Infrastructure-as-a-Service) is an emerging concept of cloud computing, which allows users to obtain hardware resources from virtualized data centers. Although many commercial IaaS clouds have recently been launched, dynamic virtual machine (VM) migration is not possible among service providers; users are locked into a particular provider, and cannot transparently relocate their VMs to another one for the best cost-effectiveness. In this paper, we propose an advanced storage access mechanism that strongly supports live VM migration over WAN. It rapidly relocates VM disks between source and destination sites with the minimum impact on I/O performance. The proposed mechanism addresses I/O consistency of virtual disks before/after migration, which is the major issue regarding wide-area live migration. The proposed mechanism works as a storage server of a block-level storage I/O protocol (e.g.,iSCSI and NBD). Two key techniques (on-demand fetching and background copying) move on-line virtual disks among remote sites, transparently and efficiently. Our prototype system works perfectly for Xen and KVM without any modification to them. Experiments showed the prototype system also worked successfully for an emulated WAN environment.