The similarity of the user interfaces (UIs) among Android apps is an important factor to indicate abnormal behaviors in Android apps. For example, phishing apps use UIs similar to those of their target apps to lure users to input sensitive information. As another example, repackaged apps preserve the UIs of the original apps while adding malicious code. In this paper, we propose a novel solution, GeminiScope, to robustly detect similar UIs among Android apps. Our approach analyzes the UI layouts of Android apps, extracts the fundamental features of the UIs' visual appearance, and rates the similarity among the UIs of apps. We evaluated GeminiScope using a few sets of apps from various sources. It reliably detected apps with similar UIs, showing the potential of using visual similarity as a unique feature to detect malicious apps.

Mobile apps continue to consume increasing amounts of sensitive data, such as banking credentials and classified documents. At the same time, the number of smartphone thefts is increasing at a rapid speed. As a result, there is an imperative need to protect sensitive data on lost or stolen mobile devices. In this work, we develop a practical solution to protect sensitive data on mobile devices. Our solution enables adaptive protection by pro-actively stepping up or stepping down data security based on perceived contextual risk of the device. We realize our solution for the Android platform in the form of a system called AppShell. AppShell does not require root privilege, nor need any modification to the underlying framework, and hence is a ready-to-deploy solution. It supports both in-memory and on-disk data protection by transparently encrypting the data, and discarding the encryption key, when required, for enhanced protection. We implement a working prototype of AppShell and evaluate it against several popular Android apps. Our results show that AppShell can successfully protect sensitive data in the lost devices with a reasonable performance overhead.

Users of Android phones increasingly entrust personal information to third-party apps. However, recent studies reveal that many apps, even benign ones, could leak sensitive information without user awareness or consent. Previous solutions either require to modify the Android framework thus significantly impairing their practical deployment, or could be easily defeated by malicious apps using a native library.

Developers often integrate third-party services into their apps. To access a service, an app must authenticate itself to the service with a credential. However, credentials in apps are often not properly or adequately protected, and might be easily extracted by attackers. A leaked credential could pose serious privacy and security threats to both the app developer and app users.

As mobile computing becomes an integral part of the modern user experience, malicious applications have infiltrated open marketplaces for mobile platforms. Malware apps stealthily launch operations to retrieve sensitive user or device data or abuse system resources. We describe a highly accurate classification approach for detecting malicious Android apps. Our method statically extracts a data-flow feature on how user inputs trigger sensitive API invocations, a property referred to as the user-trigger dependence. Our evaluation with 1433 malware apps and 2684 free popular apps gives a classification accuracy (2.1% false negative rate and 2.0% false positive rate) that is better than, or at least competitive against, the state-of-the-art. Our method also discovers new malicious apps in the Google Play market that cannot be detected by virus scanning tools. Our thesis in this mobile app classification work is to advocate the approach of benign property enforcement, i.e., extracting unique behavioral properties from benign programs and designing corresponding classification policies.

Android has a dominating share in the mobile market and there is a significant rise of mobile malware targeting Android devices. Android malware accounted for 97% of all mobile threats in 2013 [26]. To protect smartphones and prevent privacy leakage, companies have implemented various host-based intrusion prevention systems (HIPS) on their Android devices. In this paper, we first analyze the implementations, strengths and weaknesses of three popular HIPS architectures. We demonstrate a severe loophole and weakness of an existing popular HIPS product in which hackers can readily exploit. Then we present a design and implementation of a secure and extensible HIPS platform---"Patronus." Patronus not only provides intrusion prevention without the need to modify the Android system, it can also dynamically detect existing malware based on runtime information. We propose a two-phase dynamic detection algorithm for detecting running malware. Our experiments show that Patronus can prevent the intrusive behaviors efficiently and detect malware accurately with a very low performance overhead and power consumption.

App repackaging remains a serious threat to the emerging mobile app ecosystem. Previous solutions have mostly focused on the postmortem detection of repackaged apps by measuring similarity among apps. In this paper, we propose DIVILAR, a virtualization-based protection scheme to enable self-defense of Android apps against app repackaging. Specifically, it re-encodes an Android app in a diversified virtual instruction set and uses a specialized execute engine for these virtual instructions to run the protected app. However, this extra layer of execution may cause significant performance overhead, rendering the solution unacceptable for daily use. To address this challenge, we leverage a light-weight hooking mechanism to hook into Dalvik VM, the execution engine for Dalvik bytecode, and piggy-back the decoding of virtual instructions to that of Dalvik bytecode. By compositing virtual and Dalvik instruction execution, we can effectively eliminate this extra layer of execution and significantly reduce the performance overhead. We have implemented a prototype of DIVILAR. Our evaluation shows that DIVILAR is resilient against existing static and dynamic analysis, including these specific to VM-based protection. Further performance evaluation demonstrates its efficiency for daily use (an average of 16.2 and 8.9 increase to the start time and run time, respectively).

Mobile malware threats (e.g., on Android) have recently become a real concern. In this paper, we evaluate the state-of-the-art commercial mobile anti-malware products for Android and test how resistant they are against various common obfuscation techniques (even with known malware). Such an evaluation is important for not only measuring the available defense against mobile malware threats, but also proposing effective, next-generation solutions. We developed DroidChameleon, a systematic framework with various transformation techniques, and used it for our study. Our results on 10 popular commercial anti-malware applications for Android are worrisome: none of these tools is resistant against common malware transformation techniques. In addition, a majority of them can be trivially defeated by applying slight transformation over known malware with little effort for malware authors. Finally, in light of our results, we propose possible remedies for improving the current state of malware detection on mobile devices.

Traditional malware detection and analysis approaches have been focusing on code-centric aspects of malicious programs, such as detection of the injection of malicious code or matching malicious code sequences. However, modern malware has been employing advanced strategies, such as reusing legitimate code or obfuscating malware code to circumvent the detection. As a new perspective to complement code-centric approaches, we propose a data-centric OS kernel malware characterization architecture that detects and characterizes malware attacks based on the properties of data objects manipulated during the attacks. This framework consists of two system components with novel features: First, a runtime kernel object mapping system which has an un-tampered view of kernel data objects resistant to manipulation by malware. This view is effective at detecting a class of malware that hides dynamic data objects. Second, this framework consists of a new kernel malware detection approach that generates malware signatures based on the data access patterns specific to malware attacks. This approach has an extended coverage that detects not only the malware with the signatures, but also the malware variants that share the attack patterns by modeling the low level data access behaviors as signatures. Our experiments against a variety of real-world kernel rootkits demonstrate the effectiveness of data-centric malware signatures.

Mobile malware threats have recently become a real concern. In this paper, we evaluate the state-of-the-art commercial mobile antimalware products for Android and test how resistant they are against various common obfuscation techniques (even with known malware). Such an evaluation is important for not only measuring the available defense against mobile malware threats but also proposing effective, next-generation solutions. We developed DroidChameleon, a systematic framework with various transformation techniques, and used it for our study. Our results on ten popular commercial anti-malware applications for Android are worrisome: none of these tools is resistant against common malware transformation techniques. Moreover, the transformations are simple in most cases and anti-malware tools make little effort to provide transformation-resilient detection. Finally, in the light of our results, we propose possible remedies for improving the current state of malware detection on mobile devices.

The smartphone market has grown explosively in recent years, as more and more consumers are attracted to the sensor-studded multipurpose devices. Android is particularly ascendant; as an open platform, smartphone manufacturers are free to extend and modify it, allowing them to differentiate themselves from their competitors. However, vendor customizations will inherently impact overall Android security and such impact is still largely unknown.

With increased popularity and wide adoption of smartphones and mobile devices, recent years have seen a new burgeoning economy model centered around mobile apps. However, app repackaging, among many other threats, brings tremendous risk to the ecosystem, including app developers, app market operators, and end users. To mitigate such threat, we propose and develop a watermarking mechanism for Android apps. First, towards automatic watermark embedding and extraction, we introduce the novel concept of manifest app, which is a companion of a target Android app under protection. We then design and develop a tool named AppInk, which takes the source code of an app as input to automatically generate a new app with a transparently-embedded watermark and the associated manifest app. The manifest app can be later used to reliably recognize embedded watermark with zero user intervention. To demonstrate the effectiveness of AppInk in preventing app repackaging, we analyze its robustness in defending against distortive, subtractive, and additive attacks, and then evaluate its resistance against two open source repackaging tools. Our results show that AppInk is easy to use, effective in defending against current known repackaging threats on Android platform, and introduces small performance overhead.

Mobile applications (or apps) are rapidly growing in number and variety. These apps provide useful features, but also bring certain privacy and security risks. For example, malicious authors may attach destructive payloads to legitimate apps to create so-called "piggybacked" apps and advertise them in various app markets to infect unsuspecting users. To detect them, existing approaches typically employ pair-wise comparison, which unfortunately has limited scalability. In this paper, we present a fast and scalable approach to detect these apps in existing Android markets. Based on the fact that the attached payload is not an integral part of a given app's primary functionality, we propose a module decoupling technique to partition an app's code into primary and non-primary modules. Also, noticing that piggybacked apps share the same primary modules as the original apps, we develop a feature fingerprint technique to extract various semantic features (from primary modules) and convert them into feature vectors. We then construct a metric space and propose a linearithmic search algorithm (with O(n log n) time complexity) to efficiently and scalably detect piggybacked apps. We have implemented a prototype and used it to study 84,767 apps collected from various Android markets in 2011. Our results show that the processing of these apps takes less than nine hours on a single machine. In addition, among these markets, piggybacked apps range from 0.97% to 2.7% (the official Android Market has 1%). Further investigation shows that they are mainly used to steal ad revenue from the original developers and implant malicious payloads (e.g., for remote bot control). These results demonstrate the effectiveness and scalability of our approach.

Mobile devices such as smartphones and Internet-capable tablets have achieved computing and networking capabilities comparable to traditional personal computers. Their successful consumerization has also become a source of pain for adopting users and organizations. The operating systems supporting these new devices have both advantages and disadvantages with respect to offered security. On one hand, they use application sandboxing to contain exploits and limit privileges given to malware. On the other hand, they collect and organize many forms of security- and privacy-sensitive information simply as a matter of operation, and make that information easily accessible to downloaded third-party applications.

Recent years have witnessed incredible popularity and adoption of smartphones and mobile devices, which is accompanied by large amount and wide variety of feature-rich smartphone applications. These smartphone applications (or apps), typically organized in different application marketplaces, can be conveniently browsed by mobile users and then simply clicked to install on a variety of mobile devices. In practice, besides the official marketplaces from platform vendors (e.g., Google and Apple), a number of third-party alternative marketplaces have also been created to host thousands of apps (e.g., to meet regional or localization needs). To maintain and foster a hygienic smartphone app ecosystem, there is a need for each third-party marketplace to offer quality apps to mobile users.

Hosted hypervisors (e.g., KVM) are being widely deployed. One key reason is that they can effectively take advantage of the mature features and broad user bases of commodity operating systems. However, they are not immune to exploitable software bugs. Particularly, due to the close integration with the host and the unique presence underneath guest virtual machines, a hosted hypervisor -- if compromised -- can also jeopardize the host system and completely take over all guests in the same physical machine.

Smartphone sales have recently experienced explosive growth. Their popularity also encourages malware authors to penetrate various mobile marketplaces with malicious applications (or apps). These malicious apps hide in the sheer number of other normal apps, which makes their detection challenging. Existing mobile anti-virus software are inadequate in their reactive nature by relying on known malware samples for signature extraction. In this paper, we propose a proactive scheme to spot zero-day Android malware. Without relying on malware samples and their signatures, our scheme is motivated to assess potential security risks posed by these untrusted apps. Specifically, we have developed an automated system called RiskRanker to scalably analyze whether a particular app exhibits dangerous behavior (e.g., launching a root exploit or sending background SMS messages). The output is then used to produce a prioritized list of reduced apps that merit further investigation. When applied to examine 118,318 total apps collected from various Android markets over September and October 2011, our system takes less than four days to process all of them and effectively reports 3281 risky apps. Among these reported apps, we successfully uncovered 718 malware samples (in 29 families) and 322 of them are zero-day (in 11 families). These results demonstrate the efficacy and scalability of RiskRanker to police Android markets of all stripes.

The popularity and adoption of smart phones has greatly stimulated the spread of mobile malware, especially on the popular platforms such as Android. In light of their rapid growth, there is a pressing need to develop effective solutions. However, our defense capability is largely constrained by the limited understanding of these emerging mobile malware and the lack of timely access to related samples. In this paper, we focus on the Android platform and aim to systematize or characterize existing Android malware. Particularly, with more than one year effort, we have managed to collect more than 1,200 malware samples that cover the majority of existing Android malware families, ranging from their debut in August 2010 to recent ones in October 2011. In addition, we systematically characterize them from various aspects, including their installation methods, activation mechanisms as well as the nature of carried malicious payloads. The characterization and a subsequent evolution-based study of representative families reveal that they are evolving rapidly to circumvent the detection from existing mobile anti-virus software. Based on the evaluation with four representative mobile security software, our experiments show that the best case detects 79.6% of them while the worst case detects only 20.2% in our dataset. These results clearly call for the need to better develop next-generation anti-mobile-malware solutions.

In recent years, there has been explosive growth in smartphone sales, which is accompanied with the availability of a huge number of smartphone applications (or simply apps). End users or consumers are attracted by the many interesting features offered by these devices and the associated apps. The developers of these apps are also benefited by the prospect of financial compensation, either by selling their apps directly or by embedding one of the many ad libraries available on smartphone platforms. In this paper, we focus on potential privacy and security risks posed by these embedded or in-app advertisement libraries (henceforth "ad libraries," for brevity). To this end, we study the popular Android platform and collect 100,000 apps from the official Android Market in March-May, 2011. Among these apps, we identify 100 representative in-app ad libraries (embedded in 52.1% of them) and further develop a system called AdRisk to systematically identify potential risks. In particular, we first decouple the embedded ad libraries from host apps and then apply our system to statically examine the ad libraries, ranging from whether they will upload privacy-sensitive information to remote (ad) servers or whether they will download untrusted code from remote servers. Our results show that most existing ad libraries collect private information: some of them may be used for legitimate targeting purposes (i.e., the user's location) while others are hard to justify by invasively collecting the information such as the user's call logs, phone number, browser bookmarks, or even the list of installed apps on the phone. Moreover, additional ones go a step further by making use of an unsafe mechanism to directly fetch and run code from the Internet, which immediately leads to serious security risks. Our investigation indicates the symbiotic relationship between embedded ad libraries and host apps is one main reason behind these exposed risks. These results clearly show the need for better regulating the way ad libraries are integrated in Android apps.

Code-reuse attacks are software exploits in which an attacker directs control flow through existing code with a malicious result. One such technique, return-oriented programming, is based on "gadgets" (short pre-existing sequences of code ending in a ret instruction) being executed in arbitrary order as a result of a stack corruption exploit. Many existing codereuse defenses have relied upon a particular attribute of the attack in question (e.g., the frequency of ret instructions in a return-oriented attack), which leads to an incomplete protection, while a smaller number of efforts in protecting all exploitable control flow transfers suffer from limited deploy-ability due to high performance overhead. In this paper, we present a novel cost-effective defense technique called control flow locking, which allows for effective enforcement of control flow integrity with a small performance overhead. Specifically, instead of immediately determining whether a control flow violation happens before the control flow transfer takes place, control flow locking lazily detects the violation after the transfer. To still restrict attackers' capability, our scheme guarantees that the deviation of the normal control flow graph will only occur at most once. Further, our scheme ensures that this deviation cannot be used to craft a malicious system call, which denies any potential gains an attacker might obtain from what is permitted in the threat model. We have developed a proof-of-concept prototype in Linux and our evaluation demonstrates desirable effectiveness and competitive performance overhead with existing techniques. In several benchmarks, our scheme is able to achieve significant gains.

Internet advertising is one of the most popular online business models. JavaScript-based advertisements (ads) are often directly embedded in a web publisher's page to display ads relevant to users (e.g., by checking the user's browser environment and page content). However, as third-party code, the ads pose a significant threat to user privacy. Worse, malicious ads can exploit browser vulnerabilities to compromise users' machines and install malware. To protect users from these threats, we propose AdSentry, a comprehensive confinement solution for JavaScript-based advertisements. The crux of our approach is to use a shadow JavaScript engine to sandbox untrusted ads. In addition, AdSentry enables flexible regulation on ad script behaviors by completely mediating its access to the web page (including its DOM) without limiting the JavaScript functionality exposed to the ads. Our solution allows both web publishers and end users to specify access control policies to confine ads' behaviors. We have implemented a proof-of-concept prototype of AdSentry that transparently supports the Mozilla Firefox browser. Our experiments with a number of ads-related attacks successfully demonstrate its practicality and effectiveness. The performance measurement indicates that our system incurs a small performance overhead.

Return-oriented programming is an effective code-reuse attack in which short code sequences ending in a ret instruction are found within existing binaries and executed in arbitrary order by taking control of the stack. This allows for Turing-complete behavior in the target program without the need for injecting attack code, thus significantly negating current code injection defense efforts (e.g., W⊕X). On the other hand, its inherent characteristics, such as the reliance on the stack and the consecutive execution of return-oriented gadgets, have prompted a variety of defenses to detect or prevent it from happening.

Recent rapid malware growth has exposed the limitations of traditional in-host malware-defense systems and motivated the development of secure virtualization-based out-of-VM solutions. By running vulnerable systems as virtual machines (VMs) and moving security software from inside the VMs to outside, the out-of-VM solutions securely isolate the anti-malware software from the vulnerable system. However, the presence of semantic gap also leads to the compatibility problem in not supporting existing defense software. In this paper, we present process out-grafting, an architectural approach to address both isolation and compatibility challenges in out-of-VM approaches for fine-grained process-level execution monitoring. Specifically, by relocating a suspect process from inside a VM to run side-by-side with the out-of-VM security tool, our technique effectively removes the semantic gap and supports existing user-mode process monitoring tools without any modification. Moreover, by forwarding the system calls back to the VM, we can smoothly continue the execution of the out-grafted process without weakening the isolation of the monitoring tool. We have developed a KVM-based prototype and used it to natively support a number of existing tools without any modification. The evaluation results including measurement with benchmark programs show it is effective and practical with a small performance overhead.

A typical Web 2.0 application usually includes JavaScript from various sources with different trust. It is critical to properly regulate JavaScript's access to web application resources. Unfortunately, existing protection mechanisms in web browsers do not provide enough granularity in JavaScript access control. Specifically, existing solutions partially mitigate this sort of threat by only providing access control for certain types of JavaScript objects, or by unnecessarily restricting the functionality of untrusted JavaScript. In this paper, we systematically analyze the complete access control requirements in a web browser's JavaScript environment and identify the fundamental lack of fine-grained JavaScript access control mechanisms in modern web browsers. As our solution, we propose a reference monitor called JCShadow that enables fine-grained access control in JavaScript contexts without unnecessarily restricting the functionality of JavaScript. We have developed a proof-of-concept prototype in the Mozilla Firefox browser and the evaluation with real-world attacks indicates that JCShadow effectively prevents such attacks with low performance overhead.

Previous research on virtual machine introspection proposed "out-of-box" approach by moving out security tools from the guest operating system. However, compared to the traditional "in-the-box" approach, it remains a challenge to obtain a complete semantic view due to the semantic gap between the guest VM and the hyper visor. In this paper, we present Process Implanting, a new active VM introspection framework, to narrow the semantic gap by implanting a process from the host into the guest VM and executing it under the cover of an existing running process. With the protection and coordination from the hyper visor, the implanted process can run with a degree of stealthiest and exit gracefully without leaving negative impact on the guest operating system. We have designed and implemented a proof-of-concept prototype on KVM which leverages hardware virtualization. We also propose and demonstrate application scenarios for Process Implanting in the area of VM security.

Protecting kernel control data (e.g., function pointers and return addresses) has been a serious issue plaguing rootkit defenders. In particular, rootkit authors only need to compromise one piece of control data to launch their attacks, while defenders need to protect thousands of such values widely scattered across kernel memory space. Worse, some of this data (e.g., return addresses) is volatile and can be dynamically generated at run time. Existing solutions, however, offer either incomplete protection or excessive performance overhead. To overcome these limitations, we present indexed hooks, a scheme that greatly facilitates kernel control-flow enforcement by thoroughly transforming and restricting kernel control data to take only legal jump targets (allowed by the kernel's control-flow graph). By doing so, we can severely limit the attackers' possibility of exploiting them as an infection vector to launch rootkit attacks. To validate our approach, we have developed a compiler-based prototype that implements this technique in the FreeBSD 8.0 kernel, transforming 49 025 control transfer instructions (~7.25% of the code base) to use indexed hooks instead of direct pointers. Our evaluation results indicate that our approach is generic, effective, and can be implemented on commodity hardware with a low performance overhead (<;5% based on benchmarks).

Discretionary Access Control (DAC) is the primary access control mechanism in today’s major operating systems. It is, however, vulnerable to Trojan Horse attacks and attacks exploiting buggy software. We propose to combine the discretionary policy in DAC with the dynamic information flow techniques in MAC, therefore achieving the best of both worlds, that is, the DAC’s easy-to-use discretionary policy specification and MAC’s defense against threats caused by Trojan Horses and buggy programs. We propose the Information Flow Enhanced Discretionary Access Control (IFEDAC) model that implements this design philosophy. We describe our design of IFEDAC, and discuss its relationship with the Usable Mandatory Integrity Protection (UMIP) model proposed earlier by us. In addition, we analyze their security property and their relationships with other protection systems. We also describe our implementations of IFEDAC in Linux and the evaluation results and deployment experiences of the systems.

Code injection continues to pose a serious threat to computer systems. Among existing solutions, W ⊕ X is a notable approach to prevent the execution of injected code. In this paper, we focus on the Linux kernel memory protection and systematically check for possible W ⊕ X violations in the Linux kernel design and implementation. In particular, we have developed a Murphi-based abstract model and used it to discover several serious shortcomings in the current Linux kernel that violate the W ⊕ X property. We have confirmed with the Linux community the presence of these problems and accordingly developed five Linux kernel patches. (Four of them are in the process of being integrated into the mainline Linux kernel.) Our evaluation with these patches indicate that they involve only minimal changes to the existing code base and incur negligible performance overhead.

This paper presents HyperSentry, a novel framework to enable integrity measurement of a running hypervisor (or any other highest privileged software layer on a system). Unlike existing solutions for protecting privileged software, HyperSentry does not introduce a higher privileged software layer below the integrity measurement target, which could start another race with malicious attackers in obtaining the highest privilege in the system. Instead, HyperSentry introduces a software component that is properly isolated from the hypervisor to enable stealthy and in-context measurement of the runtime integrity of the hypervisor. While stealthiness is necessary to ensure that a compromised hypervisor does not have a chance to hide the attack traces upon detecting an up-coming measurement, in-context measurement is necessary to retrieve all the needed inputs for a successful integrity measurement.

Targeting the operating system (OS) kernel, kernel rootkits pose a formidable threat to computer systems and their users. Recent efforts have made significant progress in blocking them from injecting malicious code into the OS kernel for execution. Unfortunately, they cannot block the emerging so-called return-oriented rootkits (RORs). Without the need of injecting their own malicious code, these rootkits can discover and chain together "return-oriented gadgets" (that consist of only legitimate kernel code) for rootkit computation.

Virtualization is being widely adopted in today's computing systems. Its unique security advantages in isolating and introspecting commodity OSes as virtual machines (VMs) have enabled a wide spectrum of applications. However, a common, fundamental assumption is the presence of a trustworthy hypervisor. Unfortunately, the large code base of commodity hypervisors and recent successful hypervisor attacks (e.g., VM escape) seriously question the validity of this assumption. In this paper, we present HyperSafe, a lightweight approach that endows existing Type-I bare-metal hypervisors with a unique self-protection capability to provide lifetime control flow integrity. Specifically, we propose two key techniques. The first one, non-bypassable memory lockdown, reliably protects the hypervisor's code and static data from being compromised even in the presence of exploitable memory corruption bugs (e.g., buffer overflows), therefore successfully providing hypervisor code integrity. The second one, restricted pointer indexing, introduces one layer of indirection to convert the control data into pointer indexes. These pointer indexes are restricted such that the corresponding call/return targets strictly follow the hypervisor control flow graph, hence expanding protection to control-flow integrity. We have built a prototype and used it to protect two open-source Type-I hypervisors: BitVisor and Xen. The experimental results with synthetic hypervisor exploits and benchmarking programs show HyperSafe can reliably enable the hypervisor self-protection and provide the integrity guarantee with a small performance overhead.

Virtual machine (VM) introspection is a powerful technique for determining the specific aspects of guest VM execution from outside the VM. Unfortunately, existing introspection solutions share a common questionable assumption. This assumption is embodied in the expectation that original kernel data structures are respected by the untrusted guest and thus can be directly used to bridge the well-known semantic gap. In this paper, we assume the perspective of the attacker, and exploit this questionable assumption to subvert VM introspection. In particular, we present an attack called DKSM (Direct Kernel Structure Manipulation), and show that it can effectively foil existing VM introspection solutions into providing false information. By assuming this perspective, we hope to better understand the challenges and opportunities for the development of future reliable VM introspection solutions that are not vulnerable to the proposed attack.

Code injection attacks, despite being well researched, continue to be a problem today. Modern architectural solutions such as the execute-disable bit and PaX have been useful in limiting the attacks; however, they enforce program layout restrictions and can oftentimes still be circumvented by a determined attacker. We propose a change to the memory architecture of modern processors that addresses the code injection problem at its very root by virtually splitting memory into code memory and data memory such that a processor will never be able to fetch injected code for execution. This virtual split memory system can be implemented as a software-only patch to an operating system and can be used to supplement existing schemes for improved protection. Furthermore, our system is able to accommodate a number of response modes when a code injection attack occurs. Our experiments with both benchmarks and real-world attacks show the system is effective in preventing a wide range of code injection attacks while incurring reasonable overhead.

An alarming trend in recent malware incidents is that they are armed with stealthy techniques to detect, evade, and subvert malware detection facilities of the victim. On the defensive side, a fundamental limitation of traditional host-based antimalware systems is that they run inside the very hosts they are protecting (“in-the-box”), making them vulnerable to counter detection and subversion by malware. To address this limitation, recent solutions based on virtual machine (VM) technologies advocate placing the malware detection facilities outside of the protected VM (“out-of-the-box”). However, they gain tamper resistance at the cost of losing the internal semantic view of the host, which is enjoyed by “in-the-box” approaches. This poses a technical challenge known as the semantic gap. In this article, we present the design, implementation, and evaluation of VMwatcher —an “out-of-the-box” approach that overcomes the semantic gap challenge. A new technique called guest view casting is developed to reconstruct internal semantic views (e.g., files, processes, and kernel modules) of a VM nonintrusively from the outside. More specifically, the new technique casts semantic definitions of guest OS data structures and functions on virtual machine monitor (VMM)-level VM states, so that the semantic view can be reconstructed. Furthermore, we extend guest view casting to reconstruct details of system call events (e.g., the process that makes the system call as well as the system call number, parameters, and return value) in the VM, enriching the semantic view. With the semantic gap effectively narrowed, we identify three unique malware detection and monitoring capabilities: (i) view comparison-based malware detection and its demonstration in rootkit detection; (ii) “out-of-the-box” deployment of off-the-shelf anti malware software with improved detection accuracy and tamper-resistance; and (iii) nonintrusive system call monitoring for malware and intrusion behavior observation. We have implemented a proof-of-concept VMwatcher prototype on a number of VMM platforms. Our evaluation experiments with real-world malware, including elusive kernel-level rootkits, demonstrate VMwatcher's practicality and effectiveness.

Targeting the operating system kernel, the core of trust in a system, kernel rootkits are able to compromise the entire system, placing it under malicious control, while eluding detection efforts. Within the realm of kernel rootkits, dynamic data rootkits are particularly elusive due to the fact that they attack only data targets. Dynamic data rootkits avoid code injection and instead use existing kernel code to manipulate kernel data. Because they do not execute any new code, they are able to complete their attacks without violating kernel code integrity. We propose a prevention solution that blocks dynamic data kernel rootkit attacks by monitoring kernel memory access using virtual machine monitor (VMM) policies. Although the VMM is an external monitor, our system preemptively detects changes to monitored kernel data states and enables fine-grained inspection of memory accesses on dynamically changing kernel data. In addition, readable and writable kernel data can be protected by exposing the illegal use of existing code by dynamic data kernel rootkits. We have implemented a prototype of our system using the QEMU VMM. Our experiments show that it successfully defeats synthesized dynamic data kernel rootkits in real-time, demonstrating its effectiveness and practicality.

Dynamic kernel data have become an attractive target for kernel-mode malware. However, previous solutions for checking kernel integrity either limit themselves to code and static data or can only inspect a fraction of dynamic data, resulting in limited protection. Our study shows that previous solutions may reach only 28% of the dynamic kernel data and thus may fail to identify function pointers manipulated by many kernel-mode malware.

Kernel rootkits have posed serious security threats due to their stealthy manner. To hide their presence and activities, many rootkits hijack control flows by modifying control data or hooks in the kernel space. A critical step towards eliminating rootkits is to protect such hooks from being hijacked. However, it remains a challenge because there exist a large number of widely-scattered kernel hooks and many of them could be dynamically allocated from kernel heap and co-located together with other kernel data. In addition, there is a lack of flexible commodity hardware support, leading to the socalled protection granularity gap -- kernel hook protection requires byte-level granularity but commodity hardware only provides page level protection.

The man-in-the-middle (MITM) attack has been shown to be one of the most serious threats to the security and trust of existing VoIP protocols and systems. For example, the MITM who is in the VoIP signaling and/or media path can easily wiretap, divert and even hijack selected VoIP calls by tempering with the VoIP signaling and/or media traffic. Since all previously identified MITM attacks on VoIP require the adversary initially in the VoIP signaling and/or media path, there is a common belief that it is infeasible for a remote attacker, who is not initially in the VoIP path, to launch any MITM attack on VoIP. This makes people think that securing all the nodes along the normal path of VoIP traffic is sufficient to prevent MITM attacks on VoIP.

Kernel rootkits, malicious software designed to compromise a running operating system kernel, are difficult to analyze and profile due to their elusive nature, the variety and complexity of their behavior, and the privilege level at which they run. However, a comprehensive kernel rootkit profile that reveals key aspects of the rootkit's behavior is helpful in aiding a detailed manual analysis by a human expert. In this paper we present PoKeR, a kernel rootkit profiler capable of producing multi-aspect rootkit profiles which include the revelation of rootkit hooking behavior, the exposure of targeted kernel objects (both static and dynamic), assessment of user-level impacts, as well as the extraction of kernel rootkit code. The system is designed to be deployed in scenarios which can tolerate high overheads, such as honeypots. Our evaluation results with a number of real-world kernel rootkits show that PoKeR is able to accurately profile a variety of rootkits ranging from traditional ones with system call hooking to more advanced ones with direct kernel object manipulation. The obtained profiles lead to unique insights into the rootkits' characteristics and demonstrate PoKeR's usefulness as a tool for rootkit investigators.

Modern operating systems primarily use Discretionary Access Control (DAC) to protect files and other operating system resources. DAC mechanisms are more user-friendly than Mandatory Access Control (MAC) systems, but are vulnerable to attacks that use trojan horses or exploit buggy software. We show that it is possible to have the best of both worlds: DAC's easy-to-use discretionary policy and MAC's defense against trojan horses and buggy programs. This is made possible by a key new insight that DAC has weaknesses not because it uses the discretionary principle, but because existing DAC enforcement mechanisms assume that a single principal is responsible for any request, whereas in reality a request may be influenced by multiple principals; thus these mechanisms cannot correctly identify the true origin(s) of a request and fall prey to trojan horses. We propose to solve this problem by combining DAC's policy specification with new enforcement techniques that use ideas from MAC's information flow tracking. Our model, called Information Flow Enhanced DAC (IFEDAC), significantly strengthens end host security, while preserving to a large degree DAC's ease of use. In this paper, we present the IFEDAC model, analyze its security properties, and discuss our implementation for Linux.

Voice communication is fundamental to the normal operation of our society. The general public have put a lot of trust in voice communication and they have been relying on it for many critical and sensitive information exchange (e.g., emergency 911 calls, calls to customer service of financial institutions). Now more and more voice calls are carried, at least partially, over the public Internet rather than traditional Public Switched Telephone Network (PSTN). The security ramifications of using VoIP, however, have not been fully recognized. It is not clear how secure and trustworthy the currently deployed VoIP systems are, and there exists a substantial gap in the understanding of the potential impact of VoIP exploits on the VoIP users. In this paper, we seek to fill this gap by investigating the trust issues of currently deployed VoIP systems and their implications to the VoIP users.

To detect and investigate self-propagating worm attacks against networked servers, the following capabilities are desirable: 1) raising timely alerts to trigger a worm investigation, 2) determining the break-in point of a worm, i.e., the vulnerable service from which the worm infiltrates the victim, and 3) identifying all contaminations inflicted by the worm during its residence in the victim. In this paper, we argue that the worm break-in provenance information has not been exploited in achieving these capabilities and thus propose process coloring, a new approach that preserves worm break-in provenance information and propagates it along operating- system-level information flows. More specifically, process coloring assigns a "color," a unique systemwide identifier, to each remotely accessible server process. The color will be either inherited by spawned child processes or diffused transitively through process actions. Process coloring achieves three new capabilities: color-based worm warning generation, break-in point identification, and log file partitioning. The virtualization-based implementation enables more tamper-resistant log collection, storage, and real-time monitoring. Beyond the overhead introduced by virtualization, process coloring only incurs very small additional system overhead. Experiments with real-world worms demonstrate the advantages of processing coloring over non-provenance-preserving tools.

An alarming trend in malware attacks is that they are armed with stealthy techniques to detect, evade, and subvert malware detection facilities of the victim. On the defensive side, a fundamental limitation of traditional host-based anti-malware systems is that they run inside the very hosts they are protecting ("in the box"), making them vulnerable to counter-detection and subversion by malware. To address this limitation, recent solutions based on virtual machine (VM) technologies advocate placing the malware detection facilities outside of the protected VM ("out of the box"). However, they gain tamper resistance at the cost of losing the native, semantic view of the host which is enjoyed by the "in the box" approach, thus leading to a technical challenge known as the semantic gap.

Code injection attacks, despite being well researched, continue to be a problem today. Modern architectural solutions such as the NX-bit and PaX have been useful in limiting the attacks, however they enforce program layout restrictions and can often times still be circumvented by a determined attacker. We propose a change to the memory architecture of modern processors that addresses the code injection problem at its very root by virtually splitting memory into code memory and data memory such that a processor will never be able to fetch injected code for execution. This virtual split memory system can be implemented as a software only patch to an operating system, and can be used to supplement existing schemes for improved protection. Our experimental results show the system is effective in preventing a wide range of code injection attacks while incurring acceptable overhead.

Protecting kernel integrity is one of the fundamental security objectives in building a trustworthy operating system (OS). For this end, a variety of approaches and systems have been proposed and developed. However, access control models used in most of these systems are not expressive enough to capture important security requirements such as continuous policy enforcement and mutable process and object attributes. Even worse, most existing protection mechanisms in these systems reside in the same space as the running OS, which unfortunately can be disabled or subverted after an attacker successfully exploits kernel-level vulnerabilities (or features) to compromise the OS kernel. The increasing number of kernel-level root kit attacks clearly demonstrates this threat.

Code injection attacks are a top threat to today's Internet. With zero-day attacks on the rise, randomization techniques have been introduced to diversify software and operation systems of networked hosts so that attacks that succeed on one host cannot succeed on others. Two most notable system-wide randomization techniques are Instruction Set Randomization (ISR) and Address Space Layout Randomization (ASLR). The former randomizes instruction set for each process, while the latter randomizes the memory address space layout. Both suffer from a number of attacks. In this paper, we advocate and demonstrate that by combining ISR and ASLR effectively, we can offer much more robust protection than each of them individually. However, trivial combination of both schemes is not sufficient. To this end, we make the key observation that system call instructions matter the most to attackers for code injection. Our system, RandSys, uses system call instruction randomization and the general technique of ASLR along with a number of new enhancements to thwart code injection attacks. We have built a prototype for both Linux and Windows platforms. Our experiments show that RandSys can effectively thwart a wide variety of code injection attacks with a small overhead.

To investigate the exploitation and contamination by self-propagating Internet worms, a provenance-aware tracing mechanism is highly desirable. Provenance unawareness causes difficulties in fast, accurate identification of a worm’s break-in point, and incurs significant log inspection overhead. This paper presents the design, implementation, and evaluation of process coloring, an efficient provenance-aware approach to worm break-in and contamination tracing. More specifically, process coloring assigns a "color", a unique system-wide identifier, to each remotely-accessible server or process. The color will then be either inherited by spawned child processes or diffused indirectly through process actions (e.g., read/write operations). Process coloring brings two major advantages: (1) It enables fast color-based identification of a worm’s break-in point even before detailed log analysis; (2) It naturally partitions log data based on their colors, effectively reducing the volume of log data that need to be examined for worm investigation. A tamper-resistant log collection method is developed based on the virtual machine introspection technique. Our experiments with a number of real-world worms demonstrate the advantages of processing coloring.

The application service hosting platform (ASHP) has recently received tremendous attention from both industry and academia. An ASHP provides a shared high-performance infrastructure to host different application services (AS), outsourced by application service providers (ASP). In this paper, we focus on the protection of ASHP which has inherent requirement of sharing, openness, and mutual isolation. In contrast to a dedicated server platform, which is analogous with a private house, an ASHP is like an apartment building, involving the 'host' - the ASHP infrastructure, and the 'tenants' - the AS. Strong protection and isolation must be provided between the host and the tenants, as well as between different tenants. Unfortunately, traditional OS architecture and mechanisms are not adequate to provide strong ASHP protection. In this paper we first make the case for a new OS architecture based on the virtual OS technology. We then present three protection mechanisms we have developed in SODA, our ASHP architecture. The mechanisms include: (1) resource isolation between AS; (2) virtual switching and firewalling between AS; and (3) kernelized intrusion detection and logging for each AS. For (3), we have developed a system called Kernort inside the virtual OS kernel. Kernort detects network intrusions in real-time and logs AS activities even when the AS has been compromised. Moreover for the privacy of AS, logs are encrypted by Kernort so that the 'landlord' (namely ASHP owner) cannot view them without authorization. We are applying SODA to iShare, an Internet-based distributed resource sharing platform.

With the proliferation of multimedia data sources on the Internet, we envision an increasing demand for value-added and function-rich multimedia services that transport, process, and analyze multimedia data on behalf of end users. More importantly, multimedia services are expected to be easily accessible and composable by users. In this paper, we propose MSODA, a service-oriented platform that hosts a wide spectrum of media services provided by different parties. From the user's point of view, MSODA is a shared "market" for media service access and composition. For a media service provider, MSODA creates a virtual dedicated environment for service deployment and management. Finally, the underlying MSODA middleware performs the key functions of service composition, configuration, and mapping for users. We discuss key challenges in the design of MSODA and present preliminary results towards its full realization.

The grid is realizing the vision of providing computation as utility: computational jobs can be scheduled on-demand at grid hosts based on available computational capacity. In this project, we study another emerging usage of grid utility: the hosting of application services. Different from a computational job, an application service such as an e-Laboratory or an on-line business has longer lifetime, and performs multiple jobs requested by its clients. A service hosting utility platform (HUP) is formed by a set of hosts in the grid, and multiple application services will be hosted on the HUP. SODA is a service-on-demand architecture that enables on-demand creation of application services on a HUP. With SODA, an application service will be created in the form of a set of virtual service nodes; each node is a virtual machine which is physically a 'slice' of a real host in the HUP. SODA involves both OS and middleware techniques, and has the following salient capabilities: (1) on-demand service priming: the image of an application service as well as the OS on which it runs will be created on-demand and bootstrapped automatically; (2) better service isolation: services sharing the same HUP host are isolated with respect to administration, faults, intrusion, and resources; (3) integrated service load management: for each service, a service switch will be created to direct client requests to appropriate virtual service nodes. Moreover, the application service provider can replace the default request switching policy with a service-specific policy.

We present the design and prototype of GnuStream, a peer- to-peer (P2P) and receiver-driven media streaming system. GnuStream is built on top of Gnutella, and it integrates dynamic peer location and streaming capacity aggregation. Each GnuStream streaming session is controlled by the receiver peer and involves a dynamic set of peer senders instead of one fixed sender. The receiver aggregates streaming bandwidth from the multiple senders, achieving load distribution and fast reaction to sender capacity and on/off-line status changes. The effectiveness of GnuStream is demonstrated by our experiments with its prototype, which serves as the basis for real-world development and evaluation of resilient P2P media streaming services.