In recent times malware attacks on government and private organizations are rising. These attacks are carried out to steal confidential information which leads to loss of privacy, intellectual property issues and loss of revenue. These attacks are sophisticated and described as Advanced Persistent Threats(APT). The payloads used in this type of attacks are polymorphic and metamorphic in nature and contains stealth and root-kit components. As a result the conventional defence mechanisms like rule-based and signature-based methods fail to detect these malware. So modern approaches rely on static and dynamic analysis to detect sophisticated malware. However this process generates huge log files. The domain expert needs to review these logs to classify whether the binary is malicious or benign which is tedious, time consuming and expensive. Our work uses machine learning models trained on the datasets, created using the analysis logs, to overcome these problems. In this paper a number of supervised machine learning models are presented to classify the binary as malicious or benign. In this work we have used automated malware analysis framework to collect run time behavioural artefacts. Static analysis mainly focuses on collecting binary meta information, import functions and opcode sequences. The dataset is created by collecting malware from online sources and benign files from windows operating system and third party software. © 2021 by SCITEPRESS - Science and Technology Publications, Lda.

Modern day cyberattacks are complex in nature. These attacks have adverse effects like loss of privacy, intellectual property and revenue on the victim institutions. These attacks have sophisticated payloads like ransom-ware for money extortion, distributed denial of service(DDOS) malware for service disruptions and advanced persistent threat(APT) malware to posses complete control over the victims computing resources. These malware are metamorphic and polymorphic in nature and contains root-kit components to maintain stealth and hide their malicious activity. So conventional defence mechanisms like rule-based and signature based mechanisms fail to detect these malware. Modern approaches use behavioural analysis(static analysis, dynamic analysis) to identity this kind of malware. However behavioural analysis process is hindered by factors like execution environment detection, code obfuscation, anti virtualization, anti-debugging, analysis environment detection etc. Behavioural analysis also requires domain expert to review the large amount of logs produced by it to decide on the nature of the binary which is complex, time consuming and expensive. To deal with these problems we proposed deep learning methods, where convolutional neural network model is trained on the image representation of the binary to decide the binary nature as malicious or benign. In this work we have encoded the binaries into images in a unique way. Deep convolution neural network is trained on these images to learn the features to identify the binary as malicious or normal. The malware and benign samples for the dataset creation are collected from online sources and windows operating system along with compatible third party application software respectively.