Failures in a large distributed storage system are often critical, leading to unexpected I/Os that are required to restore the system's health and ensure availability. With the advent of NVMe-oF, the disaggregation of compute and storage resources presents an opportunity to minimize the negative impact of the compute failure by reattaching the storage resources. However, despite advances in hardware, modern distributed storage systems have not yet fully adapted to the disaggregated architecture. There are four main reasons: (1) lack of awareness of recoverable failure events in the disaggregated architecture, (2) incorrect availability management with respect to the NVMe-oF fault domains, (3) unnecessary data rebalance I/Os for uniform distribution triggered even after the failure is recovered, (4) load imbalance caused by asymmetric deployment of compute resources after blind relocation for recovery. To address these challenges, we introduceNVMe-oF-R, a resilient disaggregated distributed storage architecture for fast recovery.NVMe-oF-Rcomprises three techniques: (1)NVMe-oF adapter, which detects recoverable failure events and orchestrates relocation; (2)DCRUSH, a data placement strategy that considers the NVMe-oF based disaggregation architecture; and (3)Relocater, which efficiently relocates failed compute resources and fixes stragglers that arise after recovery. We implementNVMe-oF-Ratop the storage orchestration layer in a CRUSH-based distributed storage system, Ceph. Our experimental results demonstrate thatNVMe-oF-Rcan eliminate unnecessary recovery traffic and reduce recovery time by more than 50%.

Memory contention among consolidated virtual machines (VMs) creates the need for a memory balancing operation. In an attempt to provide a prompt memory balancing mechanism, we found problems with the retardation of memory transfer by the reclamation delay. The scheduling of the VMs generates the delay, and a conflicts of two reclamation policies between the guest OS and the hypervisor deteriorates it. As a remedy to these problems, we propose HyperDealer, which selects the victim page by applying reference patterns, reclaims the pages with hypervisor-level paging, and transfers those pages with ballooning of the guest OS. Our scheme eliminates the involvement of the victim VM in memory balancing and extends the dwell time of reclaimed pages in the reclaimed state. Consequently, HyperDealer significantly reduces the time taken to transfer memory with a low overhead and enhances the value of additional memory for the recipient VM. The experimental results of our scheme show that the application performance in the recipient VM is 11% more time-efficient and has a penalty which is 50% less than previous approaches.

Providing variable granularities is an attractive way to achieve good speedups for various classes of parallel applications. A few systems achieve this goal by instrumenting an application with the checking code for the state of shared data. Although these systems can provide arbitrary granularities flexibly, they have severe race conditions inherent to software-only approaches as well as the run-time overhead of the instrumentation. In this paper, we propose a new mechanism, which has low overhead and incurs no race conditions while providing variable granularities in software. The unique idea of our mechanism is to delegate the state checks to the segmentation hardware of the Intel X86. The instrumented code only maintains the state of shared data to use the segmentation hardware. Because the hardware atomically performs the required state checks and corresponding references, our mechanism is free from difficult race conditions. This feature efficiently enhances the response time to remote requests via an interrupt mechanism without additional synchronization overheads for avoiding race conditions. The run-time overhead further decreases owing to the reduced works to be done by software. The evaluation results show that our mechanism exhibits sufficiently low overhead even without any optimization.

While superpage is an efficient solution to increase TLB reach, its limited flexibility for address mapping is still a hard issue. Our proposed mechanism has been developed for taking advantage of two previous approaches which resolve the issue partially: the partial-subblock TLB and the shadow memory. Through integration of them, our mechanism enjoys various benefits inherited from the both sides. By adopting Memory Controller TLB (MTLB) from the shadow memory, it allows superpages to be composed of arbitrary physical pages. The entry structure of the partial-subblock TLB applied for the processor TLB enables all invalid address mappings to be identified inside CPU, which reduces the overhead of handling invalid mappings. In addition, cache flushing which is required when a mapping of shadow address to physical address is destroyed (e.g. due to paging) can be replaced just by resetting the corresponding valid bit in the processor TLB. At last, the per-base-page reference bits in the processor TLB make the page replacement policy of the operating system more efficient.In simulation with six benchmarks, our mechanism generates only 27% of TLB misses compared to the single-page-size TLB. With a detailed analysis, it is shown to be evident that the efficiency of our mechanism is magnified in real computing environment where multitasking and applications of large sizes are ordinary cases.