In recent years, malware has emerged as a critical security threat. In addition, malware authors continue to embed numerous anti-detection features to evade the existing malware detection approaches. Against this advanced class of malicious programs, dynamic behavior-based malware detection approaches outperform the traditional signature-based approaches by neutralizing the effects of obfuscation and morphing techniques. The majority of dynamic behavior detectors rely on system-calls to model the infection and propagation dynamics of malware. However, these approaches do not account an important anti-detection feature of modern malware, i.e., systemcall injection attack. This attack allows the malicious binaries to inject irrelevant and independent system-calls during the program execution thus modifying the execution sequences defeating the existing system-call-based detection. To address this problem, we propose an evasion-proof solution that is not vulnerable to system-call injection attacks. Our proposed approach characterizes program semantics using asymptotic equipartition property (AEP) mainly applied in information theoretic domain. The AEP allows us to extract information-rich call sequences that are further quantified to detect the malicious binaries. Furthermore, the proposed detection model is less vulnerable to call-injection attacks as the discriminating components are not directly visible to malware authors. We run a thorough set of experiments to evaluate our solution and compare it with the existing system-call-based malware detection techniques. The results demonstrate that the proposed solution is effective in identifying real malware instances.

Packed malware imposes negative impact on the accuracy of AV scanners. It is essential for a security researcher to nullify the effects of packing tools, prior to malware detection. Numerous open and commercial packers are available to facilitate unwelcome intentions of malware authors. Thus, identification of packers becomes necessary phase prior to malware scanning. In this paper, we have proposed a GPGPU based approach for accelerating our previous signature based packer detection (SPADE) [1] method. SPADE generates packer signature by utilizing the intra-family malware alignments. It makes use of Smith-Waterman algorithm to reveal the actual relationship among the packer family samples and achieves high detection rate as compared to other packer detection tools. The use of Smith-Waterman comes with a trade off between accuracy and high computational complexity. So, we have implemented a parallel version of Smith-Waterman to improve the signature generation phase of SPADE. Our GPU based approach (O(m+n)) produces 14.89X to 49.91X speedup over CPU based implementation of SPADE preserving detection accuracy. Moreover, the proposed approach opens up new domain of applying GPUs to the existing signature based approaches for malware detection where signature database updation is done on daily basis.

Worms are becoming a potential threat to Internet users across the globe. The financial damages due to computer worms increased significantly in past few years. Analyzing these hazardous worm attacks has become a crucial issue to be addressed. Given the fact that worm analysts would prefer to analyze classes of worms rather than individual files, their task will be significantly reduced. In this paper, we have proposed a dynamic host--based worm categorization approach to segregate worms. These groups indicate that worm samples constitute different behavior according to their infection and anti--detection vectors. Our proposed approach utilizes system--call traces and computes a distance matrix using Dynamic Time Warping (DTW) algorithm to form these groups. In conjunction to that, the proposed approach also discriminates worm and benign executables. The constructed model is further evaluated with unknown instances of real--world worms.

Malware developers hide the malicious payload of malware binary by employing various obfuscation techniques. One such technique commonly applied is packing. Packer transforms the original bytes so it is difficult to recognize the behaviour of any executable. Although the contents of a file is changed, some byte patterns may be preserved across different packed executables. Malware detectors need to apply unpacking mechanism prior to any detection or analysis to every sample under consideration. In this paper, we have proposed a method that discriminate packed binaries from the native files to minimize the processing time of AV scanners. We have used the blockwise entropy score of byte features of the executable. Experimental results show that the proposed method is capable of identifying packed and native executable which are packed using different malware packers.