Nested virtualization is now widely supported by major cloud vendors, allowing users to leverage virtualization-based technologies in the cloud. However, supporting nested virtualization significantly increases host hypervisor complexity and introduces a new attack surface in cloud platforms. While many prior studies have explored hypervisor fuzzing, none has explicitly addressed nested virtualization due to the challenge of generating effective virtual machine (VM) instances with a vast state space as fuzzing inputs.

Fuzzing is an effective technique to discover software vulnerabilities by automatically generating large volumes of inputs that may trigger crashes. However, post-fuzzing root cause analysis (RCA) to identify the true problems of the crashes requires substantial manual efforts, leaving many crashes discovered through fuzzing unresolved. Previous studies based on statistical approaches assist human analysts by suggesting candidate root causes inferred from large volumes of fuzzing results, but they often fail to reveal the causal relationship between the root cause and the crash site, as the two are typically distant in the code and connected only indirectly. We propose Sagitta, an assistive tool to facilitate post-fuzzing RCA by highlighting highly probable root causes based on differences in data flow between crashing and non-crashing inputs. Sagitta uses dynamic taint analysis on a crashing input and its pre-mutation parent to identify diverging points in data flow as well as control flow, and visualizes the results as localization graphs enriched with lexical information to aid analyst comprehension. We evaluated Sagitta on 14 known vulnerabilities in large-scale software and found that it captures causal relationships with only 1 to 10 branches across five bug categories. The results demonstrate that visualizing data flow differences with Sagitta's localization graphs significantly reduces manual effort in RCA.

Confidential virtual machines (CVMs) are an emerging form of trusted execution environment that enable existing operating systems (OSs) to run securely without trusting cloud providers. To this end, CVMs employ hardware-based memory encryption for runtime confidentiality and cryptographic attestation to verify memory integrity at startup. However, we reveal a previously overlooked attack vector that allows malicious cloud providers to bypass CVM attestation and execute arbitrary code within users' CVMs regardless of specific CVM configurations. Our attack, BadAML, exploits the Advanced Configuration and Power Interface (ACPI), a legacy yet widely adopted firmware interface for machine configuration. Specifically, BadAML leverages ACPI Machine Language (AML) to inject arbitrary binary code into the guest OS kernel without affecting CVM attestation. Because ACPI remains an essential component even in virtualized environments, BadAML constitutes a powerful and portable attack vector independent of guest OS type and CVM technology. We demonstrate proof-of-concept exploits of BadAML in both Linux and Windows CVM environments. We then analyze possible mitigation measures, discussing their effectiveness and limitations. Finally, we introduce AML sandboxing, a practical defense that restricts memory access to safe regions under the CVM threat model; we present its design, implementation, and evaluation, demonstrating its effectiveness across 18 real-world cloud CVM instances.

A sandbox isolates program execution in a restricted environment to reduce the risk of compromise. In application sandboxes, reference monitors mediate system calls issued by application processes and enforce predefined security policies through access control. However, guaranteeing that reference monitors are robust against all attacks is challenging, as attackers who compromise sandboxed applications can issue arbitrarily complex system call sequences. Formal verification is a promising approach for ensuring software security, but modeling all access patterns a reference monitor must handle remains difficult. In this paper, we propose vRM, a verification scheme for reference monitors based on systematic access pattern generation. We leverage SMT solvers to systematically generate concurrent access patterns that could lead to unauthorized accesses, using OS resource models and predefined access control policies. We then use concurrent model checkers to verify whether reference monitors prevent all such unauthorized accesses. To demonstrate feasibility, we implement a proof-of-concept verification for Time-Of-Check to Time-Of-Use (TOCTTOU) vulnerabilities. Our implementation uses the Z3 SMT solver for exhaustive access pattern generation and applies CDSChecker to explore concurrent access patterns. Our evaluation confirms that vRM effectively identifies known TOCTTOU vulnerabilities in the reference monitor of Apache 2.4.58. Furthermore, we formally verify our implementation of a known TOCTTOU-resistant algorithm.

Fast Identity Online 2 (FIDO2) employs public-key authentication to mitigate the weaknesses of password-based authentication. FIDO2 traditionally assumes that credentials are non-exportable and bound to hardware authenticators for strong security. However, modern users increasingly demand credential sharing across devices for better usability. Passkeys offer a choice between synchronized and device-bound credentials, but this creates a trade-off between usability and security. We propose a hardware authenticator binding (HAB) scheme that enables virtual synchronization of device-bound FIDO2 credentials, achieving the best of both worlds. A user-selectable, cloud-based HAB service manages authenticator binding, facilitating hardware authenticator registration, revocation, and recovery in case of loss. To ensure security, the HAB service incorporates decentralized key management, unlinkability through binding certificates, and attestability via trusted execution environments. We implemented the HAB service using AMD SEV-SNP, leveraging the virtual machine privilege level for trusted attestation. Our security analysis and performance evaluation demonstrate the feasibility of the HAB scheme.

Encrypted domain name resolution can reduce the risk of privacy leakage for Internet users, but it may also prevent network administrators from detecting suspicious communications. Since operating systems supporting DNS over HTTPS (DoH) have increased in recent years, malware that uses Domain Generation Algorithm (DGA) can exploit it to hide the generated domain names. In this paper, we propose a system that detects DGA-based malware communications from DoH traffic. Based on the concept of hierarchical machine learning analysis, the proposed system classifies network traffic with Gradient Boosting Decision Tree (GBDT) and tree-ensemble models. The evaluation confirmed that the system was able to detect DoH traffic generated by PadCrypt, Sisron, Tinba, and Zloader with 99.12% accuracy. The results indicate that the system has the ability to detect different DGA-based malware communications from DoH traffic with sufficient accuracy to support network administrators.

DNS over HTTPS (Do) can mitigate the risk of privacy breaches but makes it difficult to control network security services due to the DNS traffic encryption. However, since malicious DNS tunnel tools for the DoH protocol pose network security threats, network administrators need to recognize malicious communications even after the DNS traffic encryption has become widespread. In this paper, we propose a malicious DNS tunnel tool recognition system using persistent DoH traffic analysis based on machine learning. The proposed system can accomplish continuous knowledge updates for emerging malicious DNS tunnel tools on the machine learning model. The system is based on hierarchical machine learning classification and focuses on DoH traffic analysis. The evaluation results confirm that the proposed system is able to recognize the six malicious DNS tunnel tools in total, not only well-known ones, including dns2tcp, dnscat2, and iodine, but also the emerging ones such as dnstt, tcp-over-dns, and tuns with 98.02% classification accuracy.

Analyzing a large amount of malware is a major burden for security analysts. Since emerging malware is often a variant of existing malware, automatically classifying malware into known families greatly reduces a part of their burden. Image-based malware classification with deep learning is an attractive approach due to its simplicity, versatility, and affinity with the latest technologies. However, the impact of differences in deep learning models and the degree of transfer learning on the classification accuracy of malware variants has not been fully studied. In this paper, we conducted an exhaustive study of deep learning models using 24 ImageNet pre-trained models and five fine-tuning parameters, totaling 120 combinations, on two platforms. As a result, we found that the highest classification accuracy was obtained by fine-tuning one of the latest deep learning models with a relatively low degree of transfer learning, and we achieved the highest classification accuracy ever in cross-validation on the Malimg and Drebin datasets. We also confirmed that this trend is true for recent mal ware variants using the VirusTotal 2020 Windows and Android datasets.

Virtual machine (VM) live migration is an essential feature for cloud vendors. Since VM live migration involves transferring a large amount of memory between VMs, reducing memory transfer is critical to successful and efficient migration. However, software-based approaches consume considerable CPU and memory resources, resulting in degraded performance under heavy load. We propose to exploit a new CPU feature, sub-page write protection, to reduce memory transfer in VM live migration. To mitigate the overhead in naive implementations, we identified its leading cause and introduced an optimization. Emulator-based experiments demonstrated that our approach reduced memory transfer as much as software-based approaches while consuming less CPU and memory. Our optimization reduced the maximum CPU overhead by 25.5 percentage points.

Device drivers tend to be vulnerable to errant/malicious devices because many of them assume that devices always operate correctly. If a device driver is compromised either deliberately or accidentally, this can lead to system failure or give adversaries entire system access. Therefore, testing whether device drivers can handle compromised I/O correctly is important. There are several studies on testing device drivers against I/O attacks or device failures. Previous studies, however, either require source code for testing, lack test efficiency, only support a specific OS, or only target MMIO accesses. In this paper, we present a novel testing framework of device drivers' I/O boundaries. By combining a hypervisor-based fault injection mechanism and coverage-guided fuzzing scheme, our testing framework is not only OS-independent but also efficient and can test closed-source drivers. To get the information needed to test without OS cooperation, we use IOMMU to detect DMA regions and a hardware tracing mechanism to get coverage. We describe the detailed design and the current status.

Containers are an attractive platform for developing and deploying applications. Unfortunately, in order to start up a new or updated container, the entire container image must first be pulled from a registry, increasing the waiting time for containers to actually become available. Various methods have been proposed to accelerate container startup with lazy pull of container images. However, these methods have compatibility problems with existing container ecosystems, which is a major obstacle to their deployment in production environments. In this paper, we propose a fast container startup system with lazy layer pull that is highly compatible with existing container ecosystems. Our approach is to add a boot layer with only the minimum files required for booting on top of the existing container image, and to introduce a carefully designed overlay file system called LlpFS that allows for lazy pull of layers while conforming to the specifications and de facto standards of the container ecosystem. Evaluation in practical applications showed that our system made container startup about 4.9× faster under 1 Gbps bandwidth with little performance overhead.

Live migration allows a running operating system (OS) to be moved to another physical machine with negligible downtime. Unfortunately, live migration is not supported in bare-metal clouds, which lease physical machines rather than virtual machines to offer maximum hardware performance. Since bare-metal clouds have no virtualization software, implementing live migration is difficult. Previous studies have proposed OS-level live migration; however, to prevent user intervention and broaden OS choices, live migration should be OS-independent. In addition, the overhead of live migration mechanisms should be as low as possible. This paper introduces BLMVisor, a live migration scheme for bare-metal clouds. To achieve OS-independent and lightweight live migration, BLMVisor utilizes a very thin hypervisor that exposes physical hardware devices to the guest OS directly rather than virtualizing the devices. The hypervisor captures, transfers, and reconstructs physical device states by monitoring access from the guest OS and controlling the physical devices with effective techniques. To minimize performance degradation, the hypervisor is mostly idle after completing the live migration. A performance evaluation confirmed that the OS performance with BLMVisor is comparable to that of a bare-metal machine.

Cluster scheduling based on the prior estimation of job execution time is vulnerable to inaccurate estimates. To avoid performance degradation due to this misestimation, recent studies have proposed cluster schedulers that do not rely on prior estimation. However, they do not assume tasks with multitype heterogeneous computing resource demands, resulting in high job latency in real environments. Unfortunately, the optimal scheduling of such tasks is inherently difficult. In this paper, we present a cluster scheduler that heuristically handles multi-type heterogeneous resource demands without prior estimation. To reduce job latency, especially that of short jobs, our scheduler employs two techniques: (1) distributing tasks to nodes based on the similarity between resource demands and availability to simultaneously run as many tasks as possible, and (2) finding a suitable set of tasks for preemption in a node to minimize the number of task preemptions. Experimental evaluations using a real cluster and practical workloads confirm that our scheduler reduced the 90th percentile of slowdown rates by 6.4% and the 99th percentile by 29% compared to a naive extension of Kairos, an existing non-estimation-based scheduler. The experimental results also demonstrate that our scheduler is more effective when workloads have higher heterogeneity in resource demands.

Running applications for one operating system (OS) on another OS is useful in many cases. However, porting applications requires high development costs and running applications in a virtual machine poses resource sharing problems. A promising approach is to use an OS compatibility layer that converts the guest application's interface into the host one. Unfortunately, existing OS compatibility layers sacrifice either robustness or flexibility due to in-kernel subsystems or user-space-only implementations. This paper proposes a new architecture of OS compatibility layers that achieves robustness with almost user-level implementations while improving flexibility by exploiting standardized virtualization interfaces supported by most modern OSs. Our implementation of a Linux compatibility layer for macOS called Noah can run many of Ubuntu Linux binary applications, and a prototype implementation of a Linux compatibility layer for Windows confirmed the generality of our approach. Our experimental results demonstrated that the overhead of Linux kernel build time on Noah was 16%.

Regular file server upgrades are indispensable to improve performance, robustness, and power consumption. In upgrading file servers, it is crucial to quickly migrate file-sharing services between heterogeneous servers with little downtime while minimizing performance interference. We present a practical quick file server migration scheme based on the postcopy approach that defers file copy until after switching servers. This scheme can (1) reduce downtime with on-demand file migration, (2) avoid performance interference using background migration, and (3) support heterogeneous servers with stub-based file management. We discuss several practical issues, such as intermittent crawling and traversal strategy, and present the solutions in our scheme. We also address several protocol-specific issues to achieve a smooth migration. This scheme is good enough to be adopted in production systems, as it has been demonstrated for several years in real operational environments. The performance evaluation demonstrates that the downtime is less than 3 seconds, and the first file access after switching servers does not cause a timeout in the default timeout settings; it takes less than 10 seconds in most cases and up to 84.55 seconds even in a large directory tree with a depth of 16 and a width of 1,000. Although the total migration time is approximately 3 times longer than the traditional precopy approach that copies all files in advance, our scheme allows the clients to keep accessing files with acceptable overhead. We also show that appropriate selection of traversal strategy reduces tail latency by 88%, and the overhead after the migration is negligible.

Live migration, underpinned by virtualisation technologies, has enabled improved manageability and fault tolerance for servers. However, virtualised server infrastructures suffer from significant processing overheads, system inconsistencies, security issues and unpredictable performance which makes them unsuitable for low-power and resource-constraint computing devices that processing latency-sensitive, “Big-data”-type data. Consequently, we ask: “How do we eliminate the overhead of virtualisation whilst still retaining its benefits?” Motivated by this question, we investigate a practical approach for a bare-metal live migration scheme for ARM-based instances low-power servers and edge devices. In this paper, we position ARM-based bare-metal live migration as a technique that will underpin the efficiency on edge-computing and on Micro-datacentres. We also introduce our early work on identifying three key technical challenges and discuss their solutions.

Hardware failures are inevitable, especially in cloud environments where there are many hardware devices. To improve the hypervisor's reliability, hypervisor device drivers must handle hardware failures appropriately. Our goal is to allow cloud vendors to test closed-source hypervisor device drivers against failures of their real hardware. Previous studies either require source code, can only test against virtual hardware, or cannot be applied to hypervisors. In this paper, we propose FaultVisor2, a hypervisor device driver testing framework that combines fault injection and nested virtualization. To test closed-source hypervisor device drivers, we inject pseudo faults to the I/O data returned from hardware to hypervisor device drivers. To test against real hardware, we allow the target hypervisors pass-through access to the physical hardware and manipulate I/O data of the target devices by intercepting I/O access. To apply to hypervisors, we exploit nested virtualization and run a small hypervisor underneath the target hypervisor to inject pseudo faults. We omit some nested virtualization functions, including nested paging virtualization, to achieve a close to real execution environment and reduce runtime overhead. In our experiment using the VMWare ESXi hypervisor, we found three types of errors which led to critical system failures.

Upgrading file servers is indispensable for improving the performance, reducing the possibility of failures, and reducing the power consumption. To upgrade file servers, files must be migrated from the old to new servers, which poses three challenges: reducing the downtime during migration, reducing the migration overhead, and supporting the migration between heterogeneous servers. Existing technologies are difficult to achieve all of the three challenges. We propose a quick file migration scheme for heterogeneous servers. To reduce the downtime, we exploit the post-copy approach and introduce on-demand migration that allows file access before completing the migration. To reduce the overhead, we introduce background migration that migrates files as soon as possible without affecting the performance and incurs no overhead after the migration. To support heterogeneity, we introduce stub-based file management that requires no internal states of the old server. We implemented our scheme for Linux and supported the NFS and SMB protocols. The experimental results depict that the downtime was a maximum of 23 s in a 4-level 1000-file directory and the migration time was 70 min in NFS and 204 min in SMB with 242 GiB of data.

Traditional infrastructure-as-a-service (IaaS) clouds provide virtual machines as servers. However, virtualization incurs a performance overhead and prevents maximum utilization of hardware functions, so several IaaS vendors have started new services called bare-metal clouds that provide physical rather than virtual machines, allowing users to have direct access to physical hardware in the cloud. Unfortunately, exposing physical hardware to users causes a hardware protection issue for cloud vendors. Since physical hardware uses non-volatile memory (NVM) to store firmware code and configuration data, this is also exposed to users. If the NVM is modified by malicious users, the hardware could be permanently corrupted or infected by malware without being noticed. This is difficult for cloud vendors to prevent because bare-metal clouds have no virtualization layer to protect their hardware. In this paper, we describe the types of attacks that are possible for bare-metal clouds and propose BMCArmor, a hardware protection scheme for baremetal clouds. BMCArmor uses a thin hypervisor that does not virtualize the hardware, just preventing access to NVM. Our experiments show that BMCArmor can successfully protect hardware while incurring little performance overhead.

Storage cache prefetching is an effective technique for reducing the access latency in hierarchical storage systems when the access pattern is predictable based on access locality.In Infrastructure-as-a-Service (IaaS) clouds, however, storage virtualization significantly rearranges data placement, thereby reducing the spatial locality observed in the host operating system (OS). Moreover, IaaS clouds consolidate applications with various workloads that may change over time.Therefore, the access pattern changes both spatially and temporally.This paper proposes an adaptive storage cache prefetching scheme that uses structural and statistical information inside virtual machines (VMs). Observation of the application's file usage and internal file-layout information in the guest OS allows the host OS to capture spatial and temporal locality during storage access.In addition, application-level performance statistics allow the host OS to tune the prefetch speed adaptively to prevent performance degradation due to excessive prefetching.We implemented a prototype cache prefetching system that cooperates with Linux and PostgreSQL in a VM.Experiments using the TPCx-V benchmark showed that VM-awareness improved the performance by 17.1% compared with traditional prefetching.Our system achieved 3.15 times better performance than an existing non-prefetching caching system.

Bare-metal clouds are an emerging infrastructure-as-a-service (IaaS) that leases physical machines (bare-metal instances) rather than virtual machines, allowing resource-intensive applications to have exclusive access to physical hardware. Unfortunately, bare-metal instances require time-consuming or OS-specific tasks for deployment due to the lack of virtualization layers, thereby sacrificing several beneficial features of traditional IaaS clouds such as agility, elasticity, and OS transparency. We present BMcast, an OS deployment system with a special-purpose de-virtualizable virtual machine monitor (VMM) that supports quick and OS-transparent startup of bare-metal instances. BMcast performs streaming OS deployment while allowing direct access to physical hardware from the guest OS, and then disappears after completing the deployment. Quick startup of instances improves agility and elasticity significantly, and OS transparency greatly simplifies management tasks for cloud customers. Experimental results have confirmed that BMcast initiated a bare-metal instance 8.6 times faster than image copying, and database performance on BMcast during streaming OS deployment was comparable to that on a state-of-the-art VMM without performing deployment. BMcast incurred zero overhead after de-virtualization.

We propose HyperShield, which is a hypervisor that can be inserted into and removed from a running operating system, for improving security. While many existing security-oriented hypervisors require modifying or rebooting an overlying operating system, HyperShield does not require this. HyperShield is intended to be a general framework for various security mechanisms. The current implementation provides two mechanisms for preventing kernel-level buffer overflow. One detects the execution of user code with the kernel privilege, and the other detects malicious modification of a return address in a control stack. HyperShield is implemented on Linux as a loadable kernel module. When the module is inserted, it places itself under the operating system and executes as a hypervisor. The operating system is migrated into a virtual machine and managed by the hypervisor. HyperShield detects attacks by combining virtualization of memory management with a hardware-assisted execution-bit feature. We have confirmed through experiments that HyperShield successfully prevented kernel-level buffer overflow attacks.

Virtual machine monitors (VMMs), including hypervisors, are a popular platform for implementing various security functionalities. However, traditional VMMs require numerous components for providing virtual hardware devices and for sharing and protecting system resources among virtual machines (VMs), enlarging the code size of and reducing the reliability of the VMMs.This paper introduces a hypervisor architecture, called parapass-through, designed to minimize the code size of hypervisors by allowing most of the I/O access from the guest operating system (OS) to pass-through the hypervisor, while the minimum access necessary to implement security functionalities is completely mediated by the hypervisor. This architecture uses device drivers of the guest OS to handle devices, thereby reducing the size of components in the hypervisor to provide virtual devices. This architecture also allows to run only single VM on it, eliminating the components for sharing and protecting system resources among VMs.We implemented a hypervisor called BitVisor and a parapass-through driver for enforcing storage encryption of ATA devices based on the parapass-through architecture. The experimental result reveals that the hypervisor and ATA driver require approximately 20 kilo lines of code (KLOC) and 1.4 KLOC respectively.

In recent years, as the data processed by governmental or commercial organizations increases, cases involving information leak have risen. It is difficult to control information on many distributed end-point computers using conventional security mechanisms. Therefore, we have been proposed a novel secure VMM (Virtual Machine Monitor) architecture which is used as a foundation of security policy enforcement on distributed computers. This paper especially introduces Role-based Access Control (RBAC) to theID management framework in a secure VMM system. Our proposal will reduce costs for distributed policies updates. Proposed RBAC mechanism employs attribute certificates (ACs) to handle user’s roles. This paper shows design and prototype implementation based on PKI-based ID card and proven open source VMM software, QEMU.

This paper presents a strong and efficient scheme for protecting against buffer overflow attacks. The basic approach of this scheme is pointer copying: copies of code pointers are stored in a safe memory area to detect and prevent the manipulation of code pointers. In order to protect the copied code pointers from data-pointer modification attacks, this scheme exploits the segmentation hardware of IA-32 (Intel x86) processors. This scheme provides as strong protection as write-protecting the memory area via system calls. On the other hand, this scheme involves a modest overhead because copying a code pointer requires only a few user-level instructions and there is no penalty of entering the kernel. The experimental results show that the performance overhead in OpenSSL ranges from 0.9% to 4.3%