Named Entity Recognition (NER) from open-source security reports has become a crucial task in Cyber Threat Intelligence (CTI) to enable knowledge extraction and support proactive cyber defence. However, existing fully supervised NER approaches struggle to adapt to the dynamic and linguistically complex nature of CTI text. To address these challenges, we propose Label-Efficient Named Entity Recognition (LETNER), a model designed to handle multi-token, sparsely distributed, and fine-grained CTI entity patterns while maintaining low annotation demand. LETNER leverages Convolutional Neural Networks (CNNs) and a gating mechanism to learn dynamic span-based representations, and introduces an orthogonal regularisation to align entity-span relationships in a shared metric space for effective similarity-based inference. Furthermore, a cost-aware evaluation framework is presented to jointly quantify annotation effort and model performance, providing practical insights for decision-making in low-resource settings. Experimental results on a complex CTI dataset containing 22 fine-grained entity classes show that LETNER significantly outperforms baseline models, achieving high performance using only 10% of the annotated data.

• This paper proposes Latent Space Refinement (LSR), a novel unsupervised classification framework that integrates metric learning with clustering-based representation refinement, addressing the critical challenge of label scarcity in cyber threat intelligence (CTI). • LSR introduces a posterior regularisation strategy that aligns latent representations from Pretrained Language Models (PLMs) with an auxiliary TF-IDF-based distribution. This guides unsupervised adaptation to the target domain without any PLM fine-tuning, ensuring scalability and efficiency. • Extensive experiments on three CTI benchmarks demonstrate that LSR consistently outperforms state-of-the-art unsupervised and few-shot baselines in Accuracy and F1 score. • By enabling lightweight unsupervised domain adaptation, LSR offers a plug-and-play solution applicable to CTI, including other resource-constrained domains such as health and legal text classification. Text classification plays a critical role in Cyber Threat Intelligence (CTI) applications, where open-source text data is mined to identify patterns such as Indicators of Compromise (IoC), Tactics, Techniques and Procedures (TTPs), Named Entities and more. However, the dynamic nature of CTI makes traditional supervised machine learning classifiers impractical due to their reliance on large number of labelled training datasets. To address this, we propose Latent Space Refinement (LSR), an unsupervised method designed for CTI text classification. LSR introduces a posterior regularisation strategy where an auxiliary distribution derived from a TF-IDF feature space serves as signals to refine latent representations derrived from Pretrained Language Models (PLMs). By iteratively refining this latent space with clustering signals, LSR enables efficient similarity-based classification using only a few user-provided seed keywords. Extensive experiments on diverse CTI tasks, including both binary and multi-class classification, demonstrate that LSR consistently outperforms state-of-the-art unsupervised and zero-shot/few-shot methods in Accuracy and Weighted F1 score, all without tuning internal PLM parameters. This makes LSR a lightweight and PLM-agnostic solution for real-world CTI applications.

Different from source code clone detection, clone detection (similar code search) in binary executables faces big challenges due to the gigantic differences in the syntax and the structure of binary code that result from different configurations of compilers, architectures and OSs. Existing studies have proposed different categories of features for detecting binary code clones, including CFG structures, n-gram in CFG, input/output values, etc. In our previous study and the tool BinGo, to mitigate the huge gaps in CFG structures due to different compilation scenarios, we propose a selective inlining technique to capture the complete function semantics by inlining relevant library and user-defined functions. However, only features of input/output values are considered in BinGo. In this study, we propose to incorporate features from different categories (e.g., structural features and high-level semantic features) for accuracy improvement and emulation for efficiency improvement. We empirically compare our tool, BinGo-E, with the pervious tool BinGo and the available state-of-the-art tools of binary code search in terms of search accuracy and performance. Results show that BinGo-E achieves significantly better accuracies than BinGo for cross-architecture matching, cross-OS matching, cross-compiler matching and intra-compiler matching. Additionally, in the new task of matching binaries of forked projects, BinGo-E also exhibits a better accuracy than the existing benchmark tool. Meanwhile, BinGo-E takes less time than BinGo during the process of matching.

Software vulnerability is one of the major threats to software security. Once discovered, vulnerabilities are often fixed by applying security patches. In that sense, security patches carry valuable information about vulnerabilities, which could be used to discover, understand and fix (similar) vulnerabilities. However, most existing patch analysis approaches work at the source code level, while binary-level patch analysis often heavily relies on a lot of human efforts and expertise. Even worse, some vulnerabilities may be secretly patched without applying CVE numbers, or only the patched binary programs are available while the patches are not publicly released. These practices greatly hinder patch analysis and vulnerability analysis. In this paper, we propose a scalable binary-level patch analysis framework, named SPAIN, which can automatically identify security patches and summarize patch patterns and their corresponding vulnerability patterns. Specifically, given the original and patched versions of a binary program, we locate the patched functions and identify the changed traces (i.e., a sequence of basic blocks) that may contain security or non-security patches. Then we identify security patches through a semantic analysis of these traces and summarize the patterns through a taint analysis on the patched functions. The summarized patterns can be used to search similar patches or vulnerabilities in binary programs. Our experimental results on several real-world projects have shown that: i) SPAIN identified security patches with high accuracy and high scalability, ii) SPAIN summarized 5 patch patterns and their corresponding vulnerability patterns for 5 vulnerability types, and iii) SPAIN discovered security patches that were not documented, and discovered 3 zero-day vulnerabilities.

Coverage-based fuzzing is one of the most effective techniques to find vulnerabilities, bugs or crashes. However, existing techniques suffer from the difficulty in exercising the paths that are protected by magic bytes comparisons (e.g., string equality comparisons). Several approaches have been proposed to use heavy-weight program analysis to break through magic bytes comparisons, and hence are less scalable. In this paper, we propose a program-state based binary fuzzing approach, named Steelix, which improves the penetration power of a fuzzer at the cost of an acceptable slow down of the execution speed. In particular, we use light-weight static analysis and binary instrumentation to provide not only coverage information but also comparison progress information to a fuzzer. Such program state information informs a fuzzer about where the magic bytes are located in the test input and how to perform mutations to match the magic bytes efficiently. We have implemented Steelix and evaluated it on three datasets: LAVA-M dataset, DARPA CGC sample binaries and five real-life programs. The results show that Steelix has better code coverage and bug detection capability than the state-of-the-art fuzzers. Moreover, we found one CVE and nine new bugs.

In the arms race of attackers and defenders, the defense is usually more challenging than the attack due to the unpredicted vulnerabilities and newly emerging attacks every day. Currently, most of existing malware detection solutions are individually proposed to address certain types of attacks or certain evasion techniques. Thus, it is desired to conduct a systematic investigation and evaluation of anti-malware solutions and tools based on different attacks and evasion techniques. In this paper, we first propose a meta model for Android malware to capture the common attack features and evasion features in the malware. Based on this model, we develop a framework, MYSTIQUE, to automatically generate malware covering four attack features and two evasion features, by adopting the software product line engineering approach. With the help of MYSTIQUE, we conduct experiments to 1) understand Android malware and the associated attack features as well as evasion techniques; 2) evaluate and compare the 57 off-the-shelf anti-malware tools, 9 academic solutions and 4 App market vetting processes in terms of accuracy in detecting attack features and capability in addressing evasion. Last but not least, we provide a benchmark of Android malware with proper labeling of contained attack and evasion features.

Binary code search has received much attention recently due to its impactful applications, e.g., plagiarism detection, malware detection and software vulnerability auditing. However, developing an effective binary code search tool is challenging due to the gigantic syntax and structural differences in binaries resulted from different compilers, architectures and OSs. In this paper, we propose BINGO — a scalable and robust binary search engine supporting various architectures and OSs. The key contribution is a selective inlining technique to capture the complete function semantics by inlining relevant library and user-defined functions. In addition, architecture and OS neutral function filtering is proposed to dramatically reduce the irrelevant target functions. Besides, we introduce length variant partial traces to model binary functions in a program structure agnostic fashion. The experimental results show that BINGO can find semantic similar functions across architecture and OS boundaries, even with the presence of program structure distortion, in a scalable manner. Using BINGO, we also discovered a zero-day vulnerability in Adobe PDF Reader, a COTS binary.

Recently, malware has increasingly become a critical threat to embedded systems, while the conventional software solutions, such as antivirus and patches, have not been so successful in defending the ever-evolving and advanced malicious programs. In this paper, we propose a hardware-enhanced architecture, GuardOL, to perform online malware detection. GuardOL is a combined approach using processor and field-programmable gate array (FPGA). Our approach aims to capture the malicious behavior (i.e., high-level semantics) of malware. To this end, we first propose the frequency-centric model for feature construction using system call patterns of known malware and benign samples. We then develop a machine learning approach (using multilayer perceptron) in FPGA to train classifier using these features. At runtime, the trained classifier is used to classify the unknown samples as malware or benign, with early prediction. The experimental results show that our solution can achieve high classification accuracy, fast detection, low power consumption, and flexibility for easy functionality upgrade to adapt to new malware samples. One of the main advantages of our design is the support of early prediction-detecting 46% of malware within first 30% of their execution, while 97% of the samples at 100% of their execution, with <;3% false positives.

Network monitoring is an important way to ensure the security of hosts from being attacked by malicious attackers. One challenging problem for network operators is how to distribute the limited monitoring resources (e.g., intrusion detectors) among the network to detect attacks in a cost-effective manner, especially when the attacking strategies can be changing dynamically and unpredictable. To this end, we adopt Markov game to model the interactions between the network operator and the attacker and propose an adaptive Markov strategy (AMS) to determine how the detectors should be placed on the network against possible attacks to minimize the network's accumulated cost over time. The AMS is guaranteed to converge to the best response strategy when the attacker's strategy is fixed (rationality), converge to a fixed strategy under self-play (convergence) and obtain a payoff no less than that under the precomputed Nash equilibrium strategy of the Markov game (safety). The experimental results show that the AMS can achieve better protection for the network compared with both previous approaches based on the prediction of attack paths (equivalent to a graph coloring problem) and Nash equilibrium strategy.

Existing malicious JavaScript (JS) detection tools and commercial anti-virus tools mostly use feature-based or signature-based approaches to detect JS malware. These tools are weak in resistance to obfuscation and JS malware variants, not mentioning about providing detailed information of attack behaviors. Such limitations root in the incapability of capturing attack behaviors in these approches. In this paper, we propose to use Deterministic Finite Automaton (DFA) to abstract and summarize common behaviors of malicious JS of the same attack type. We propose an automatic behavior learning framework, named JS*, to learn DFAs from dynamic execution traces of JS malware, where we implement an effective online teacher by combining data dependency analysis, defense rules and trace replay mechanism. We evaluate JS* using real world data of 10000 benign and 276 malicious JS samples to cover 8 most-infectious attack types. The results demonstrate the scalability and effectiveness of our approach in the malware detection and classification, compared with commercial anti-virus tools. We also show how to use our DFAs to detect variants and new attacks.

In recent years, malware (malicious software) has greatly evolved and has become very sophisticated. The evolution of malware makes it difficult to detect using traditional signature-based malware detectors. Thus, researchers have proposed various behavior-based malware detection techniques to mitigate this problem. However, there are still serious shortcomings, related to scalability and computational complexity, in existing malware behavior modeling techniques. This raises questions about the practical applicability of these techniques. This paper proposes and evaluates a bounded feature space behavior modeling (BOFM) framework for scalable malware detection. BOFM models the interactions between software (which can be malware or benign) and security-critical OS resources in a scalable manner. Information collected at run-time according to this model is then used by machine learning algorithms to learn how to accurately classify software as malware or benign. One of the key problems with simple malware behavior modeling (e.g., n-gram model) is that the number of malware features (i.e., signatures) grows proportional to the size of execution traces, with a resulting malware feature space that is so large that it makes the detection process very challenging. On the other hand, in BOFM, the malware feature space is bounded by an upper limit N, a constant, and the results of our experiments show that its computation time and memory usage are vastly lower than in currently reported, malware detection techniques, while preserving or even improving their high detection accuracy.

Anti-malware vendors receive several thousand new malware (malicious software) variants per day. Due to large volume of malware samples, it has become extremely important to group them based on their malicious characteristics. Grouping of malware variants that exhibit similar behavior helps to generate malware signatures more efficiently. Unfortunately, exponential growth of new malware variants and huge-dimensional feature space, as used in existing approaches, make the clustering task very challenging and difficult to scale. Furthermore, malware behavior modeling techniques proposed in the literature do not scale well, where malware feature space grows in proportion with the number of samples under examination.

Bug reporting is an uncoordinated process that is often the cause of redundant workload in triaging and fixing bugs due to many duplicated bug reports. Furthermore, quite often, same bugs are repeatedly reported as users or testers are unaware of whether they have been reported from the search query results. In order to reduce both the users and developers' efforts, the quality of search in a bug tracking system is crucial. However, all existing search functions in a bug tracking system produce results with undesired relevance and ranking. Hence, it is essential to provide an effective search function to any bug tracking system.

An Autonomous Bee Colony Optimization (A-BCO) algorithm for solving multi-objective numerical problems is proposed. In contrast with previous Bee Colony algorithms, A-BCO utilizes a diversity-based performance metric to dynamically assess the archive set. This assessment is employed to adapt the bee colony structures and flying patterns. This self-adaptation feature is introduced to optimize the balance between exploration and exploitation during the search process. Moreover, the total number of search iterations is also determined/optimized by A-BCO, according to user pre-specified conditions, during the search process. We evaluate A-BCO upon numerical benchmark problems and the experimental results demonstrate the effectiveness and robustness of the proposed algorithm when compared with the Non-dominated Sorting Genetic Algorithm II and the latest Multi-objective Bee Colony Algorithm proposed to date.