The rapid advancement of pre-trained language models (PLMs) has demonstrated promising results for various code-related tasks. However, their effectiveness in detecting real-world vulnerabilities remains a critical challenge. While existing empirical studies evaluate PLMs for vulnerability detection (VD), they suffer from data leakage, limited scope, and superficial analysis, hindering the accuracy and comprehensiveness of evaluations. This paper begins by revisiting the common issues in existing research on PLMs for VD through the evaluation pipeline. It then proceeds with an accurate and extensive evaluation of 18 PLMs, spanning model parameters from millions to billions, on high-quality datasets that feature accurate labeling, diverse vulnerability types, and various projects. Specifically, we compare the performance of PLMs under both fine-tuning and prompt engineering, assess their effectiveness and generalizability across various training and testing settings, and analyze their robustness to perturbations such as code normalization, abstraction, and semantic-preserving transformations.

Fuzzing is a prevalent technology for identifying software vulnerabilities. Existing fuzzing techniques predominantly focus on maximizing code coverage to unearth potential security issues. However, the mere expansion of explored code does not necessarily correlate with an increased discovery of vulnerabilities. Additionally, existing fuzzers often neglect comprehensive execution path information in code exploration. Consequently, potential vulnerabilities may be delayed or overlooked in the fuzzing process. To address this, we propose VPGFUZZ, a vulnerable path-guided fuzzer that can not only explore new code but also exploit known vulnerability path knowledge for vulnerability discovery. It employs a vulnerable path recognition model to identify test cases with potentially vulnerable paths. This model is trained with various execution paths derived from real-world vulnerability PoCs (Proof of Concepts). Based on this model, VPGFUZZ applies an explore-exploit seed selection strategy to effectively choose test cases for testing. Unlike traditional seed selection methods that maintain a single queue for exploring new code, this strategy includes a separate queue for retaining test cases identified as potentially vulnerable, allowing for more thorough testing. Experimental results demonstrate that VPGFUZZ discovers 24 zero-day vulnerabilities, with 18 receiving vulnerability identifiers from third-party organizations such as CVE. Our evaluation also shows VPGFUZZ’s superior efficiency by uncovering the first vulnerability approximately 1.2 to 70 times faster than popular fuzzers in most programs.

Federated learning (FL), an emerging distributed machine learning paradigm, has been applied to various privacy-preserving scenarios. However, due to its distributed nature, FL faces two key issues: the non-independent and identical distribution (non-IID) of user data and vulnerability to Byzantine threats. To address these challenges, in this paper, we propose FedCAP, a robust FL framework against both data heterogeneity and Byzantine attacks. The core of FedCAP is a model update calibration mechanism to help a server capture the differences in the direction and magnitude of model updates among clients. Furthermore, we design a customized model aggregation rule that facilitates collaborative training among similar clients while accelerating the model deterioration of malicious clients. With a Euclidean norm-based anomaly detection mechanism, the server can quickly identify and permanently remove malicious clients. Moreover, the impact of data heterogeneity and Byzantine attacks can be further mitigated through personalization on the client side. We conduct extensive experiments, comparing multiple state-of-the-art baselines, to demonstrate that FedCAP performs well in several non-IID settings and shows strong robustness under a series of poisoning attacks.

A timely software update is vital to combat the increasing security vulnerabilities. However, some software vendors may secretly patch their vulnerabilities without creating CVE entries or even describing the security issue in their change log. Thus, it is critical to identify these hidden security patches and defeat potential N-day attacks. Researchers have employed various machine learning techniques to identify security patches in open-source software, leveraging the syntax and semantic features of the software changes and commit messages. However, all these solutions cannot be directly applied to the binary code, whose instructions and program flow may dramatically vary due to different compilation configurations. In this paper, we propose BinGo, a new security patch detection system for binary code. The main idea is to present the binary code as code property graphs to enable a comprehensive understanding of program flow and perform a language model over each basic block of binary code to catch the instruction semantics. BinGo consists of four phases, namely, patch data pre-processing, graph extraction, embedding generation, and graph representation learning. Due to the lack of an existing binary security patch dataset, we construct such a dataset by compiling the pre-patch and post-patch source code of the Linux kernel. Our experimental results show BinGo can achieve up to 80.77% accuracy in identifying security patches between two neighboring versions of binary code. Moreover, BinGo can effectively reduce the false positives and false negatives caused by the different compilers and optimization levels.

Network protocol fingerprinting is a critical technique for identifying various implementations of network protocols, which is essential for vulnerability assessment and security management. However, current fingerprinting methods such as Nmap still heavily rely on manual probe crafting, requiring experts with domain knowledge and leading to inefficiencies and potential oversights. This paper introduces pFuzz, an automatic network protocol fingerprint discovery system utilizing difference-guided fuzzing, to address the challenge of the vast search space inherent in fingerprinting. We propose a difference tree to model the nested recursive condition structure of network protocols and a packet oracle map to capture and utilize multifield relationships revealed by value co-occurrence. Our evaluation of pFuzz on the widely used TCP/IP protocol demonstrates its effectiveness and efficiency on discovering fingerprints.

Over-privileged Linux containers might put the underlying OS at risk by permitting pointless system calls that could be exploited as entry points to the kernel. However, finding such security profiles is a difficult task as it demands examining the implementation/operation of containers in the absence of knowledge regarding its required system calls. In this article, we propose a hybrid approach to limit the system call usage during the execution of containers. Specifically, given an application container, we maintain an initial fine-grained whitelist by dynamic tracking to control the run-time security along with a complementary whitelist extracted via static analysis to maintain container's functionality while addressing the coverage limitation of dynamic analysis. Our method automatically analyzes the container behavior to identify three execution phases and dynamically enforce the corresponding fine-grained system call whitelists. The invoked system call will be compared with both whitelists to decide if it should be killed to guarantee the container security or logged for further analysis. Our evaluation results with 193 Docker images demonstrate the effectiveness of our approach in significantly reducing the required system calls during the applications' life-cycle. Furthermore, we discuss the reduced attack surface and demonstrate the efficiency of our approach through empirical analysis results.

System logs record rich information for system events. Practical anomaly detection from system logs should be able to address three challenges: 1) understanding complicated attributes in event logs; 2) extracting complex context relations among events; and 3) providing concrete explanations to human analysts. In this paper, we develop an attention-equipped encoder-decoder system to capture context from system logs for explainable anomaly detection. For each target event, we collect its nearby events in chronological order as its context events. Instead of using a recurrent neural network-based encoder like previous works, we adopt a Transformer-based encoder to extract complex relations among context events and their attributes. Then, a context vector is generated and passed to the decoder, where an attention matrix is learned and used to weigh the context events for detecting the anomalies. Evaluation on the large-scale real-world Los Alamos National Laboratory dataset shows that, compared with existing works, our methods can provide fine-grained one-to-one attention to help explain the importance of each attribute in the context events to the prediction, without sacrificing detection performance.

Python has become the most popular programming language as it is friendly to work with for beginners. However, a recent study has found that most security issues in Python have not been indexed by CVE and may only be fixed by "silent" security commits, which pose a threat to software security and hinder the security fixes to downstream software. It is critical to identify the hidden security commits; however, the existing datasets and methods are insufficient for security commit detection in Python, due to the limited data variety, non-comprehensive code semantics, and uninterpretable learned features. In this paper, we construct the first security commit dataset in Python, namely PySecDB, which consists of three subsets including a base dataset, a pilot dataset, and an augmented dataset. The base dataset contains the security commits associated with CVE records provided by MITRE. To increase the variety of security commits, we build the pilot dataset from GitHub by filtering keywords within the commit messages. Since not all commits provide commit messages, we further construct the augmented dataset by understanding the semantics of code changes. To build the augmented dataset, we propose a new graph representation named CommitCPG and a multi-attributed graph learning model named SCOPY to identify the security commit candidates through both sequential and structural code semantics. The evaluation shows our proposed algorithms can improve the data collection efficiency by up to 40 percentage points. After manual verification by three security experts, PySecDB consists of 1,258 security commits and 2,791 non-security commits. Furthermore, we conduct an extensive case study on PySecDB and discover four common security fix patterns that cover over 85% of security commits in Python, providing insight into secure software maintenance, vulnerability detection, and automated program repair.

With the increasing popularity of open-source software, embedded vulnerabilities have been widely propagating to downstream software. Due to different maintenance policies, software vendors may silently release security patches without providing sufficient advisories (e.g., CVE). This leaves users unaware of security patches and provides attackers good chances to exploit unpatched vulnerabilities. Thus, detecting those silent security patches becomes imperative for secure software maintenance. In this paper, we propose a graph neural network based security patch detection system named GraphSPD, which represents patches as graphs with richer semantics and utilizes a patch-tailored graph model for detection. We first develop a novel graph structure called PatchCPG to represent software patches by merging two code property graphs (CPGs) for the pre-patch and post-patch source code as well as retaining the context, deleted, and added components for the patch. By applying a slicing technique, we retain the most relevant context and reduce the size of PatchCPG. Then, we develop the first end-to-end deep learning model called PatchGNN to determine if a patch is security-related directly from its graph-structured PatchCPG. PatchGNN includes a new embedding process to convert PatchCPG into a numeric format and a new multi-attributed graph convolution mechanism to adapt diverse relationships in PatchCPG. The experimental results show GraphSPD can significantly outperform the state-of-the-art approaches on security patch detection.

Security patches, embedding both vulnerable code and the corresponding fixes, are of great significance to vulnerability detection and software maintenance. However, the existing patch datasets suffer from insufficient samples and low varieties. In this paper, we construct a large-scale patch dataset called PatchDB that consists of three components, namely, NVD-based dataset, wild-based dataset, and synthetic dataset. The NVD-based dataset is extracted from the patch hyperlinks indexed by the NVD. The wild-based dataset includes security patches that we collect from the commits on GitHub. To improve the efficiency of data collection and reduce the effort on manual verification, we develop a new nearest link search method to help find the most promising security patch candidates. Moreover, we provide a synthetic dataset that uses a new oversampling method to synthesize patches at the source code level by enriching the control flow variants of original patches. We conduct a set of studies to investigate the effectiveness of the proposed algorithms and evaluate the properties of the collected dataset. The experimental results show that PatchDB can help improve the performance of security patch identification.

Security patches in open source software (OSS) not only provide security fixes to identified vulnerabilities, but also make the vulnerable code public to the attackers. Therefore, armored attackers may misuse this information to launch N-day attacks on unpatched OSS versions. The best practice for preventing this type of N-day attacks is to keep upgrading the software to the latest version in no time. However, due to the concerns on reputation and easy software development management, software vendors may choose to secretly patch their vulnerabilities in a new version without reporting them to CVE or even providing any explicit description in their change logs. When those secretly patched vulnerabilities are being identified by armored attackers, they can be turned into powerful "0-day" attacks, which can be exploited to compromise not only unpatched version of the same software, but also similar types of OSS (e.g., SSL libraries) that may contain the same vulnerability due to code clone or similar design/implementation logic. Therefore, it is critical to identify secret security patches and downgrade the risk of those "0-day" attacks to at least "n-day" attacks. In this paper, we develop a defense system and implement a toolset to automatically identify secret security patches in open source software. To distinguish security patches from other patches, we first build a security patch database that contains more than 4700 security patches mapping to the records in CVE list. Next, we identify a set of features to help distinguish security patches from non-security ones using machine learning approaches. Finally, we use code clone identification mechanisms to discover similar patches or vulnerabilities in similar types of OSS. The experimental results show our approach can achieve good detection performance. A case study on OpenSSL, LibreSSL, and BoringSSL discovers 12 secret security patches.