Cybersecurity and software research have crossed paths with modern deep learning research for a few years. The power of large language models (LLMs) in particular has intrigued us to apply them to understanding binary code. In this paper, we investigate some of the many ways LLMs can be applied to binary code similarity detection, as it is a significantly more difficult task compared to source code similarity detection due to the sparsity of information and less meaningful syntax. It also has great practical implications, such as vulnerability and malware detection. We find that pretrained LLMs are mostly capable of detecting similar binary code, even with a zero-shot setting. Our main contributions and findings are to provide several supervised fine-tuning methods that, when combined, significantly surpass zero-shot LLMs and state-of-the-art binary code similarity detection methods. Specifically, we up-train the model through data augmentation, translation-style causal learning, LLM2Vec, and cumulative GTE loss. With a complete ablation study, we show that our training method can transform a generic language model into a powerful binary similarity expert, and is also robust and general enough for cross-optimization, cross-architecture, and cross-obfuscation detection.

Embedded PowerShell commands or scripts are among the most popular malware payloads. For malware that prioritizes stealthiness, such as fileless malware, PowerShell’s access to Windows API functions without additional libraries makes it useful for evading detection. Detecting malicious PowerShell scripts and commands is an open challenge for proactive endpoint protection due to three major issues: (1) The malicious commands are usually hidden in a long script beyond the processing limit of typical machine learning models. (2) They are usually mixed with bulky benign scripts. (3) Script obfuscation can easily conceal their potential matching signatures. In this article, we introduce a novel model addressing these challenges. It incorporates similarity learning, sentence transformer, sliding window method, and stochastic gradient descent (SGD) classifier. Our key insight is that malicious PowerShell code, particularly when obfuscated, exhibits semantic and statistical deviations from benign administrative usage, and these deviations can be captured by contrastive sentence embeddings without the need for de-obfuscation or handcrafted features. We operate this insight through a Siamese similarity learning framework that improves robustness against Out-of-Vocabulary tokens due to unseen code obfuscation methods. The sliding window method enables the model to handle long scripts, and the SGD classifier evaluates segment-level maliciousness. Our model achieves accuracies of 99.01%, 97.59%, 98.70%, and 99.73% across multiple obfuscated and mixed script benchmarks, outperforming existing baselines by over 30% in all cases. This work demonstrates a scalable and effective strategy for robust PowerShell malware detection in real-world scenarios.

Finding similar code is important for software engineering, defense of intellectual property, and security, and one of the increasingly common ways adversaries use to defeat the detection of similar code is through obfuscations such as code transformation and scattering the code they wish to hide among long sequences. Moving code far enough apart poses a specific challenge for solutions with localized features (e.g., n-grams), or attention mechanisms as the code parts are distributed beyond the local context window. We introduce a neural network solution pattern called “Cybertron” that addresses this problem by utilizing reinforcement learning to train a code abstraction and summarization function; this converts arbitrarily long code into fixed-length real vectors in a way that is optimized for similarity search. The key to the design is the smart selection of important elements of the code and abstraction to preserve semantic function while minimizing syntactic feature information. We evaluated the approach on a three-challenge benchmark of obfuscated JavaScript, a scripting language that is commonly obfuscated and for which code-mixing is a rising challenge. The evaluation shows our approach identifies obfuscated code within even large scripts with an AUC of 78%, which outperforms current state-of-the-art sequence models by 7–35%.

Software vulnerabilities have been posing tremendous reliability threats to the general public as well as critical infrastructures, and there have been many studies aiming to detect and mitigate software defects at the binary level. Most of the standard practices leverage both static and dynamic analysis, which have several drawbacks like heavy manual workload and high complexity. Existing deep learning-based solutions not only suffer to capture the complex relationships among different variables from raw binary code but also lack the explainability required for humans to verify, evaluate, and patch the detected bugs. We propose VulANalyzeR, a deep learning-based model, for automated binary vulnerability detection, Common Weakness Enumeration-type classification, and root cause analysis to enhance safety and security. VulANalyzeR features sequential and topological learning through recurrent units and graph convolution to simulate how a program is executed. The attention mechanism is integrated throughout the model, which shows how different instructions and the corresponding states contribute to the final classification. It also classifies the specific vulnerability type through multi-task learning as this not only provides further explanation but also allows faster patching for zero-day vulnerabilities. We show that VulANalyzeR achieves better performance for vulnerability detection over the state-of-the-art baselines. Additionally, a Common Vulnerability Exposure dataset is used to evaluate real complex vulnerabilities. We conduct case studies to show that VulANalyzeR is able to accurately identify the instructions and basic blocks that cause the vulnerability even without given any prior knowledge related to the locations during the training phase.

In recent years, the number of anonymous script-based fileless malware attacks, software copyright disputes, and code plagiarism issues has increased rapidly. In the literature, automated Code Authorship Analysis (CAA) techniques have been proposed to reduce the manual effort in identifying those attacks and issues. Most CAA techniques aim to solve the task of Authorship Attribution (AA), i.e., identifying the actual author of a source code fragment from a given set of candidate authors. However, in many real-world scenarios, investigators do not have a predefined set of authors containing the actual author at the time of investigation, i.e., contradicting AA's assumption. Additionally, existing AA techniques ignore the influence of code functionality when identifying the authorship, which leads to biased matching simply based on code functionality. Different from AA, the task of (extreme) Authorship Verification (AV) is to decide if two texts were written by the same person or not. AV techniques do not need a predefined author set and thus could be applied in more code authorship-related applications than AA. To our knowledge, there is no previous work attempting to solve the AV problem for the source code. To fill the gap, we propose a novel adversarial neural network, namely SCS-Gan, that can learn a stylometric representation of code for automated AV. With the multi-head attention mechanism, SCS-Gan focuses on the code parts that are most informative regarding personal styles and generates functionality-agnostic stylometric representations through adversarial training. We benchmark SCS-Gan and two state-of-the-art code representation models on four out-of-sample datasets collected from a real-world programming competition. Our experiment results show that SCS-Gan outperforms the baselines on all four out-of-sample datasets.

The impact of crisis events can be devastating in a multitude of ways, many of which are unpredictable due to the suddenness in which they occur. The evolution of social media (for example Twitter) has given directly affected individuals or those with valuable information a platform to effectively share their stories to the masses. As a result, these platforms have become vast repositories of helpful information for emergency organizations. However, different crisis events often contain event-specific keywords, which results in the difficult extraction of useful information with a single model. In this paper, we put forward TASR, which stands for Topic-Agnostic Stylometric Representations, a novice deep learning architecture that uses stylometric and adversarial learning to remove topical bias to better manage the unknown surrounding unseen events. As an alternative to domain adaptive approaches requiring data from the unseen event, it reduces the work for those responding to the onset of a crisis. Overall, we conduct a comprehensive study of the situational properties of TASR, the benefits of its architecture including its topic-agnostic and explainable properties, and how it improves upon comparable models in past research. From two experiments, on average, TASR is able to outperform state-of-the-art methods such as transfer learning and domain adoption by 11% in AUC. The ablation study illustrates how different architecture choices of TASR impact the results and that TASR has been optimized for this task. Finally, we conduct a case study to show that explainable results from our model can be used to help guide human analysts through crisis information extraction.