Given a crash dump or a kernel memory snapshot, it is often desirable to have a capability that can traverse its pointers to locate the root cause of the crash, or check their integrity to detect the control flow hijacks. To achieve this, one key challenge lies in how to locate where the pointers are. While locating a pointer usually requires the data structure knowledge of the corresponding program, an important advance made by this work is that we show a technique of extracting address-independent data reference expressions for pointers through dynamic binary analysis. This novel pointer reference expression encodes how a pointer is accessed through the combination of a base address (usually a global variable) with certain offset and further pointer dereferences. We have applied our techniques to OS kernels, and our experimental results with a number of real world kernel malware show that we can correctly identify the hijacked kernel function pointers by locating them using the extracted pointer reference expressions when only given a memory snapshot.

Over the past 20 years, we have witnessed a widespread adoption of dynamic binary instrumentation (DBI) for numerous program analyses and security applications including program debugging, profiling, reverse engineering, and malware analysis. To date, there are many DBI platforms, and the most popular one is Pin, which provides various instrumentation APIs for process instrumentation. However, Pin does not support the instrumentation of OS kernels. In addition, the execution of the instrumentation and analysis routine is always inside the virtual machine (VM). Consequently, it cannot support any out-of-VM introspection that requires strong isolation. Therefore, this paper presents PEMU, a new open source DBI framework that is compatible with Pin-APIs, but supports out-of-VM introspection for both user level processes and OS kernels. Unlike in-VM instrumentation in which there is no semantic gap, for out-of-VM introspection we have to bridge the semantic gap and provide abstractions (i.e., APIs) for programmers. One important feature of PEMU is its API compatibility with Pin. As such, many Pin plugins are able to execute atop PEMU without any source code modification. We have implemented PEMU, and our experimental results with the SPEC 2006 benchmarks show that PEMU introduces reasonable overhead.

Precise fingerprinting of an operating system (OS) is critical to many security and forensics applications in the cloud, such as virtual machine (VM) introspection, penetration testing, guest OS administration, kernel dump analysis, and memory forensics. The existing OS fingerprinting techniques primarily inspect network packets or CPU states, and they all fall short in precision and usability. As the physical memory of a VM always exists in all these applications, in this article, we present OS-SOMMELIER+, a multi-aspect, memory exclusive approach for precise and robust guest OS fingerprinting in the cloud. It works as follows: given a physical memory dump of a guest OS, OS-SOMMELIER+first uses a code hash based approach from kernel code aspect to determine the guest OS version. If code hash approach fails, OS-SOMMELIER+then uses a kernel data signature based approach from kernel data aspect to determine the version. We have implemented a prototype system, and tested it with a number of Linux kernels. Our evaluation results show that the code hash approach is faster but can only fingerprint the known kernels, and data signature approach complements the code signature approach and can fingerprint even unknown kernels.

Recent advances in bridging the semantic gap between virtual machines (VMs) and their guest processes have a dark side: They can be abused to subvert and compromise VM file system images and process images. To demonstrate this alarming capability, a context-aware, reactive VM Introspection (VMI) instrument is presented and leveraged to automatically break the authentication mechanisms of both Linux and Windows operating systems. By bridging the semantic gap, the attack is able to automatically identify critical decision points where authentication succeeds or fails at the binary level. It can then leverage the VMI to transparently corrupt the control-flow or data-flow of the victim OS at that point, resulting in successful authentication without any password-guessing or encryption-cracking. The approach is highly flexible (threatening a broad class of authentication implementations), practical (realizable against real-world OSes and VM images), and useful for both malicious attacks and forensics analysis of virtualized systems and software.

With the wide existence of binary code, it is desirable to reuse it in many security applications, such as malware analysis and software patching. While prior approaches have shown that binary code can be extracted and reused, they are often based on static analysis and face challenges when coping with obfuscated binaries. This paper introduces trace-oriented programming (TOP), a general framework for generating new software from existing binary code by elevating the low-level binary code to C code with templates and inlined assembly. Different from existing work, TOP gains benefits from dynamic analysis such as resilience against obfuscation and avoidance of points-to analysis. Thus, TOP can be used for malware analysis, especially for malware function analysis and identification. We have implemented a proof-of-concept of TOP and our evaluation results with a range of benign and malicious software indicate that TOP is able to reconstruct source code from binary execution traces in malware analysis and identification, and binary function transplanting.

This paper presents EXTERIOR, a dual-VM architecture based external shell that can be used for trusted, timely out-of-VM management of guest-OS such as introspection, configuration, and recovery. Inspired by recent advances in virtual machine introspection (VMI), EXTERIOR leverages an isolated, secure virtual machine (SVM) to introspect the kernel state of a guest virtual machine (GVM). However, it goes far beyond the read-only capability of the traditional VMI, and can perform automatic, fine-grained guest-OS writable operations. The key idea of EXTERIOR is to use a dual-VM architecture in which a SVM runs a kernel identical to that of the GVM to create the necessary environment for a running process (e.g., rmmod, kill), and dynamically and transparently redirect and update the memory state at the VMM layer from SVM to GVM, thereby achieving the same effect in terms of kernel state updates of running the same trusted in-VM program inside the shell of GVM. A proof-of-concept EXTERIOR has been implemented. The experimental results show that EXTERIOR can be used for a timely administration of guest-OS, including introspection and (re)configuration of the guest-OS state and timely response of kernel malware intrusions, without any user account in the guest-OS.

It is generally believed to be a tedious, time-consuming, and error-prone process to develop a virtual machine introspection (VMI) tool because of the semantic gap. Recent advance shows that the semantic-gap can be largely narrowed by reusing the executed code from a trusted OS kernel. However, the limitation for such an approach is that it only reuses the exercised code through a training process, which suffers the code coverage issues. Thus, in this article, we present Vmst, a new technique that can seamlessly bridge the semantic gap and automatically generate the VMI tools. The key idea is that, through system wide instruction monitoring, Vmst automatically identifies the introspection related data from a secure-VM and online redirects these data accesses to the kernel memory of a product-VM, without any training. Vmst offers a number of new features and capabilities. Particularly, it enables an in-VM inspection program (e.g., ps) to automatically become an out-of-VM introspection program. We have tested Vmst with over 25 commonly used utilities on top of a number of different OS kernels including Linux and Microsoft Windows. The experimental results show that our technique is general (largely OS-independent), and it introduces 9.3X overhead for Linux utilities and 19.6X overhead for Windows utilities on average for the introspected program compared to the native in-VM execution without data redirection.

Precise fingerprinting of an operating system (OS) is critical to many security and virtual machine (VM) management applications in the cloud, such as VM introspection, penetration testing, guest OS administration (e.g., kernel update), kernel dump analysis, and memory forensics. The existing OS fingerprinting techniques primarily inspect network packets or CPU states, and they all fall short in precision and usability. As the physical memory of a VM is always present in all these applications, in this paper, we present OS-Sommelier, a memory-only approach for precise and efficient cloud guest OS fingerprinting. Given a physical memory dump of a guest OS, the key idea of OS-Sommelier is to compute the kernel code hash for the precise fingerprinting. To achieve this goal, we face two major challenges: (1) how to differentiate the main kernel code from the rest of code and data in the physical memory, and (2) how to normalize the kernel code to deal with practical issues such as address space layout randomization. We have designed and implemented a prototype system to address these challenges. Our experimental results with over 45 OS kernels, including Linux, Windows, FreeBSD, OpenBSD and NetBSD, show that our OS-Sommelier can precisely fingerprint all the tested OSes without any false positives or false negatives, and do so within only 2 seconds on average.

It is generally believed to be a tedious, time consuming, and error-prone process to develop a virtual machine introspection (VMI) tool manually because of the semantic gap. Recent advances in Virtuoso show that we can largely narrow the semantic gap. But it still cannot completely automate the VMI tool generation. In this paper, we present VMST, an entirely new technique that can automatically bridge the semantic gap and generate the VMI tools. The key idea is that, through system wide instruction monitoring, we can automatically identify the introspection related data and redirect these data accesses to the in-guest kernel memory. VMST offers a number of new features and capabilities. Particularly, it automatically enables an in-guest inspection program to become an introspection program. We have tested VMST over 15 commonly used utilities on top of 20 different Linux kernels. The experimental results show that our technique is general (largely OS-agnostic), and it introduces 9.3X overhead on average for the introspected program compared to the native non-redirected one.