Kernel heap buffer overflow vulnerabilities have been exposed for decades, but there are few practical countermeasures that can be applied to OS kernels. Previous solutions either suffer from high performance overhead or compatibility problems with mainstream kernels and hardware. In this article, we presentKruiser, a concurrent kernel heap buffer overflow monitor. Unlike conventional methods, the security enforcement of which is usually inlined into the kernel execution, Kruiser migrates security enforcement from the kernel’s normal execution to a concurrent monitor process, leveraging the increasingly popular multi-core architectures. To reduce the synchronization overhead between the monitor process and the running kernel, we design a novel semi-synchronized non-blocking monitoring algorithm, which enables efficient runtime detection on live memory without incurring false positives. To prevent the monitor process from being tampered and provide guaranteed performance isolation, we utilize the virtualization technology to run the monitor process out of the monitored VM, while heap memory allocation information is collected inside the monitored VM in a secure and efficient way. The hybrid VM monitoring technique combined with the secure canary that cannot be counterfeited by attackers provides guaranteed overflow detection with high efficiency. We have implemented a prototype ofKruiserbased on Linux and the Xen/KVM hypervisor. The evaluation shows that Kruiser can detect realistic kernel heap buffer overflow attacks in cloud environment effectively with minimal cost.

Summary Return‐Oriented Programming (ROP) attacks become very popular in recent years as these attacks can bypass traditional defense mechanisms such as data execution prevention (DEP) effectively. Previous solutions suffer from limitations in that: 1) some methods need to modify the target programs; 2) some methods introduce considerable performance cost; 3) some methods rely on the special hardware; and 4) ,most of existing methods could not provide an online protection for the target processes. In this paper, we present OnRop, an on‐the‐fly ROP attack protection system by using the commodity hardware features and OS internal facilities. Our system is compatible with the existing programs, and its protection layer can be added on demand. The experiments show that OnRop can detect ROP attacks effectively with moderate performance cost.

Summary Loadable kernel modules (LKMs) that contain vulnerabilities are a big threat to modern operating systems (OSs). The primary reason is that there is no protection mechanism inside the kernel space when the LKM is executed. As a result, kernel module exploitation can seriously affect the OS kernel security. Although many protection systems have been developed to address this problem in the past few years, there still remain some challenges: (1) How to automatically generate a security policy before the kernel module is enforced? (2) How to properly mediate the interactions between the kernel module and the OS kernel without modifications on the existing OS, hardware, and kernel module structure? To address these challenges, we present LKM guard (LKMG), a policy‐centric system that can protect commodity OS kernel from vulnerable LKMs. Compared with previous systems, LKMG is able to generate a security policy from a kernel module and then enforce the policy during the run time. Generally, the working process of LKMG can be divided into 2 stages. First, we utilize static analysis to extract the kernel code and data access patterns from a kernel module's source code and then combine these patterns with the related memory address information to generate a security policy. Second, by leveraging the hardware‐assisted virtualization technology, LKMG isolates the kernel module from the rest of the kernel and then enforces the kernel module's execution to obey the derived policy. The experiments show that our system can defend against various attacks launched by the compromised kernel module effectively with moderate performance cost.

ABSTRACT Web services have been greatly threatened by remote exploit code attacks, where maliciously crafted HTTP requests are used to inject binary code to compromise web servers and web applications. In practice, besides detection of such attacks, attack attribution analysis (i.e., to automatically categorize exploits or determine whether an exploit is a variant of an attack from the past) is also very important. In this paper, we present SA3, a novel exploit code attribution analysis that combines semantics‐based analysis and statistical modeling to automatically categorize given exploit code. SA3 extracts semantic features from exploit code through data anomaly analysis and then attributes the exploit to an appropriate class on the basis of our statistical model derived from a Markov model. We evaluate SA3 over a comprehensive set of shellcode collected from Metasploit and other polymorphic engines. Experimental results show that SA3 is effective and efficient. The attribution analysis accuracy can be over 90% in different parameter settings with false positive rate no more than 4.5%. The novelty of SA3 is that it combines semantic analysis with statistical modeling for exploit code attribution analysis. Copyright © 2012 John Wiley & Sons, Ltd.

Alert correlation is a promising technique in intrusion detection. It takes the alerts produced by intrusion detection systems and produces compact reports which provide a more succinct and high-level view of occurring or attempted intrusions and highly improve security expert's work efficiency. Traditional alert correlation system adopts a centralized architecture which can be easily over flooded by the raw alarms. To address this issue, a distributed alert correlation model based on hierarchical architecture is proposed. This model greatly improves the performance of alert correlation through integrating three novel methods. The experiments show effectiveness of this alert correlation model on 2000 DARPA intrusion detection scenario specific datasets.