Abstract As software systems grow in complexity, the importance of efficient defect detection escalates, becoming vital to maintain software quality. In recent years, artificial intelligence technology has boomed. In particular, with the proposal of Large Language Models (LLMs), researchers have found the huge potential of LLMs to enhance the performance of software defect detection. This review aims to elucidate the relationship between LLMs and software defect detection. We categorize and summarize existing research based on the distinct applications of LLMs in dynamic and static detection scenarios. Dynamic detection methods are categorized based on the different phases in which they employ LLMs, such as using them for test case generation, providing feedback guidance, and conducting output assessment. Static detection methods are classified according to whether they analyze the source code or the binary of the software under test. Furthermore, we investigate the prompt engineering and model fine-tuning strategies adopted within these studies. Finally, we summarize the emerging trend of integrating LLMs into software defect detection, identify challenges to be addressed and prospect for some potential research directions.

IoT devices, now integral to our daily routines, offer unparalleled convenience but also face mounting security threats. Embedded web services, prevalent in public networks, pose a major risk to these devices. While research has focused on detecting vulnerabilities in IoT embedded web services, it has overlooked the presence of invisible interfaces, which have emerged as significant security threats. In this paper, we propose InvRadar, a novel framework for detecting vulnerabilities in invisible interfaces of embedded web services in IoT devices. Specifically, InvRadar identifies invisible interfaces by analyzing the differences between the front-end visible interface keywords and the back-end interface keywords through a correlation analysis method. Subsequently, InvRadar uses a static taint analysis method to detect the vulnerabilities that can be triggered by the invisible interfaces. To validate the performance of InvRadar, we conduct extensive experiments and compare InvRadar with the state-of-the-art methods. In testing 13 device firmware, InvRadar identifies 1,793 invisible interfaces and detects 124 vulnerabilities, including 53 newly discovered ones, with 34 receiving new CVE/CNVD IDs. Additionally, InvRadar outperforms the state-of-the-art methods in interface keyword extraction, border binary and data ingestion function identification.

Large language models (LLMs) have demonstrated significant utility in a wide range of applications; however, their deployment is plagued by security vulnerabilities, notably jailbreak attacks. These attacks manipulate LLMs to generate harmful or unethical content by crafting adversarial prompts. While much of the current research on jailbreak attacks has focused on single-turn interactions, it has largely overlooked the impact of historical dialogues on model behavior. Although recent studies have explored multi-turn jailbreak attacks, they generally assume that the attacker can only manipulate the user prompt. In contrast, we highlight that an attacker can also control the model’s previous outputs. To this end, we introduce DIA, a new paradigm that leverages fabricated dialogue history to enhance jailbreak effectiveness. DIA operates in a black-box setting, requiring only access to the chat API or knowledge of the LLM’s chat template. We propose two methods for constructing adversarial historical dialogues: one adapts gray-box prefilling attacks, and the other exploits deferred responses. Our experiments demonstrate that DIA achieves state-of-the-art attack success rates on recent LLMs, including Llama-3.1 and GPT-4o. Additionally, we show that DIA can bypass 6 different defense mechanisms, highlighting its robustness.

Deep learning (DL) frameworks serve as the backbone for a wide range of artificial intelligence applications. However, bugs within DL frameworks can cascade into critical issues in higher-level applications, jeopardizing reliability and security. While numerous techniques have been proposed to detect bugs in DL frameworks, research exploring common API patterns across frameworks and the potential risks they entail remains limited. Notably, many DL frameworks expose similar APIs with overlapping input parameters and functionalities, rendering them vulnerable to shared bugs, where a flaw in one API may extend to analogous APIs in other frameworks. To address this challenge, we propose MirrorFuzz, an automated API fuzzing solution to discover shared bugs in DL frameworks. MirrorFuzz operates in three stages: First, MirrorFuzz collects historical bug data for each API within a DL framework to identify potentially buggy APIs. Second, it matches each buggy API in a specific framework with similar APIs within and across other DL frameworks. Third, it employs large language models (LLMs) to synthesize code for the API under test, leveraging the historical bug data of similar APIs to trigger analogous bugs across APIs. We implement MirrorFuzz and evaluate it on four popular DL frameworks (TensorFlow, PyTorch, OneFlow, and Jittor). Extensive evaluation demonstrates that MirrorFuzz improves code coverage by 39.92% and 98.20% compared to state-of-the-art methods on TensorFlow and PyTorch, respectively. Moreover, MirrorFuzz discovers 315 bugs, 262 of which are newly found, and 80 bugs are fixed, with 52 of these bugs assigned CNVD IDs.

Directed greybox fuzzing has become a crucial technique for discovering vulnerabilities in software. The seed mutation plays an important role in fuzzing by generating new inputs that explore diverse program states and find the target vulnerability. While seed mutation is critical to the effectiveness of fuzzing, most existing mutation strategies are designed for coverage-based fuzzing and lack the guidance required in directed scenarios. This limits the quality of generated testcases and reduces fuzzing efficiency in directed greybox fuzzing.In this paper, we propose DMut, a novel seed mutation strategy based on a multi-population genetic algorithm, designed to address these limitations. DMut models the seed mutation process using a genetic algorithm, optimizing the seed mutation probability distribution and iteratively evolving it to generate higher-quality testcases. The approach incorporates a well-designed fitness function and selection strategy that aligns with directed fuzzing scenarios to guide the evolution of the mutation strategy. Through comprehensive experiments on real-world CVEs, we demonstrate that DMut significantly improves the effectiveness of directed fuzzing. Compared to the widely adopted directed greybox fuzzing tool, AFLGo, DMut reduces the time to expose the target vulnerability by 41% on average. Additionally, DMut improves path exploration efficiency, covering more unique execution paths and speeding up the exploration process. In summary, the experimental results show that DMut provides a robust, efficient method for improving directed fuzzing performance, offering a significant advancement over existing approaches.

As an extension to the Git version control system that optimizes the handling of large files and binary content, Git Large File Storage (LFS) has been widely adopted by nearly all Git platforms. While Git LFS offers significant improvements in managing large files, it introduces new security implications that remain largely unexplored. This paper presents the first comprehensive security analysis of Git LFS, identifying 11 critical security properties that LFS servers must uphold. Building on our analysis of these property violations, we propose four new attack vectors: Private LFS File Leakage, LFS File Replacement, Quota-based Denial of Service (DoS), and Quota Escape. These attacks exploit weaknesses in practical LFS server implementations and can lead to serious consequences, including unauthorized access to sensitive files, malware injection, denial of service affecting all public repositories, and resource abuse. To evaluate the security of LFS implementations, we develop a semi-automated black-box testing tool and apply it to 14 major Git platforms. We uncover 36 previously unknown vulnerabilities and have responsibly disclosed them to the respective platform maintainers, receiving positive feedback and over $1800 in bug bounty rewards.

Internationalized Domain Names (IDNs) allow users to access the internet using domain names in their native languages. This technology provides significant convenience for non-English speaking users. However, despite the widespread acceptance and use of IDNs, the risks associated with using IDNs remain unclear in practice, such as the IDN homograph problem. To address this issue, we conduct a systematic analysis of the IDN homograph problem and explore the adoption characteristics of IDNs in practice. Specifically, we design and implement an effective IDN analysis framework, named as IDNMon. We perform a large-scale measurement study covering 863 top-level domain zone files and historical top lists based on IDNMon. Our findings indicate that the IDN registration and usage in Europe exceeds that in East Asia. Our results confirm that the IDN homograph problem is universal (12.32% of 2,623,161 IDNs face this problem), which raises serious challenges when designing protection strategies for browsers. Our work provides new insights into the adoption of IDNs in practice, contributes to a better understanding, and promotes the development of IDNs.

With the rising demand for website data storage, leveraging cloud storage services for vast user file storage has become prevalent. Nowadays, a new file upload scenario has been introduced, allowing web users to upload files directly to the cloud storage service. This new scenario offers convenience but involves more roles (i.e., web users, web servers, and cloud storage services) and their interactions, bringing new security threats. In this paper, we perform the first systematic security study in this scenario. With in-depth analysis, we identify six new types of vulnerabilities and conduct large-scale real-world measurements on the top 500 Alexa Rank websites. Among these websites, 182 (36.4%) use cloud storage services, illustrating the widespread use of the cloud. Then, we perform a detailed analysis of 28 popular websites that allow user upload. Surprisingly, they all have at least one of the six vulnerabilities. Totally, we discover 79 new vulnerabilities and responsibly report them to the websites. Many popular websites respond positively, including Google, Reddit, and CSDN. We discuss the root causes of these vulnerabilities and propose possible mitigation methods. In summary, our work offers significant value in understanding the security risks of cloud storage services for websites and facilitating future research.

Directed grey-box fuzzing (DGF) aims to discover vulnerabilities in specific code areas efficiently. Distance metric, which is used to measure the quality of seed in DGF, is a crucial factor in affecting the fuzzing performance. Despite distance metrics being widely applied in existing DGF frameworks, it remains opaque about how different distance metrics guide the fuzzing process and affect the fuzzing result in practice. In this paper, we conduct the first empirical study to explore how different distance metrics perform in guiding DGFs. Specifically, we systematically discuss different distance metrics in the aspect of calculation method and granularity. Then, we implement different distance metrics based on AFLGo. On this basis, we conduct comprehensive experiments to evaluate the performance of these distance metrics on the benchmarks widely used in existing DGF-related work. The experimental results demonstrate the following insights. First, the difference among different distance metrics with varying methods of calculation and granularities is not significant. Second, the distance metrics may not be effective in describing the difficulty of triggering the target vulnerability. In addition, by scrutinizing the quality of testcases, our research highlights the inherent limitation of existing mutation strategies in generating high-quality testcases, calling for designing effective mutation strategies for directed fuzzing. We open-source the implementation code and experiment dataset to facilitate future research in DGF.

gVisor is a Google-published application-level kernel for containers. As gVisor is lightweight and has sound isolation, it has been widely used in many IT enterprises [1],[2],[3]. When a new vulnerability of the upstream gVisor is found, it is important for the downstream developers to test the corresponding code to maintain the security. To achieve this aim, directed fuzzing is promising. Nevertheless, there are many challenges in applying existing directed fuzzing methods for gVisor. The core reason is that existing directed fuzzers are mainly for general C/C++ applications, while gVisor is an OS kernel written in the Go language. To address the above challenges, we propose G-Fuzz, a directed fuzzing framework for gVisor. There are three core methods in G-Fuzz, including lightweight and fine-grained distance calculation, target related syscall inference and utilization, and exploration and exploitation dynamic switch. Note that the methods of G-Fuzz are general and can be transferred to other OS kernels. We conduct extensive experiments to evaluate the performance of G-Fuzz. Compared to Syzkaller, the state-of-the-art kernel fuzzer, G-Fuzz outperforms it significantly. Furthermore, we have rigorously evaluated the importance for each core method of G-Fuzz. G-Fuzz has been deployed in industry and has detected multiple serious vulnerabilities.

Unrestricted file upload (UFU) vulnerabilities, especially unrestricted executable file upload (UEFU) vulnerabilities, pose severe security risks to web servers. For instance, attackers can leverage such vulnerabilities to execute arbitrary code to gain the control of a whole web server. Therefore, it is significant to develop effective and efficient methods to detect UFU and UEFU vulnerabilities. Towards this, most state-of-the-art methods are designed based on dynamic testing. Nevertheless, they still entail two critical limitations. 1) They heavily rely on manual efforts, which are error-prone and have poor adaptability. 2) They seldom leverage effective information to guide the testing, resulting in generating a large number of invalid test cases. Such limitations severely hinder the performance of UFU vulnerability detection. In this paper, we propose URadar, an adaptive dynamic testing-based method for detecting UFU and UEFU vulnerabilities. There are three core designs in URadar, including file upload interface identification, file type restriction inference, and invalid mutation combination filtration, which can effectively solve the two limitations of existing methods. To evaluate the performance of URadar, we conduct extensive experiments and compare URadar with state-of-the-art methods (e.g., FUSE, RIPS). In testing 18 web applications, URadar discovers 26 UEFU vulnerabilities, where 8 are new, and 6 have been assigned new CVE/CNNVD IDs. By contrast, FUSE and RIPS find 14 and 2 UEFU vulnerabilities, respectively. To discover the same number of UFU vulnerabilities, FUSE needs to send 73,261 request packets with a time cost of 2,791.1s on average, 23.43 and 20.53 times of the requirements for URadar. The above results demonstrate that URadar significantly outperforms the state-of-the-art methods. In addition, we have open-sourced URadar to facilitate future research on UFU vulnerability detection.

Fuzzing is a technique of finding bugs by executing a target program recurrently with a large number of abnormal inputs. Most of the coverage-based fuzzers consider all parts of a program equally and pay too much attention to how to improve the code coverage. It is inefficient as the vulnerable code only takes a tiny fraction of the entire code. In this article, we design and implement an evolutionary fuzzing framework called V-Fuzz, which aims to find bugs efficiently and quickly in limited time for binary programs. V-Fuzz consists of two main components: 1) a vulnerability prediction model and 2) a vulnerability-oriented evolutionary fuzzer. Given a binary program to V-Fuzz, the vulnerability prediction model will give a prior estimation on which parts of a program are more likely to be vulnerable. Then, the fuzzer leverages an evolutionary algorithm to generate inputs which are more likely to arrive at the vulnerable locations, guided by the vulnerability prediction result. The experimental results demonstrate that V-Fuzz can find bugs efficiently with the assistance of vulnerability prediction. Moreover, V-Fuzz has discovered ten common vulnerabilities and exposures (CVEs), and three of them are newly discovered.