Abstract As software systems grow in complexity, the importance of efficient defect detection escalates, becoming vital to maintain software quality. In recent years, artificial intelligence technology has boomed. In particular, with the proposal of Large Language Models (LLMs), researchers have found the huge potential of LLMs to enhance the performance of software defect detection. This review aims to elucidate the relationship between LLMs and software defect detection. We categorize and summarize existing research based on the distinct applications of LLMs in dynamic and static detection scenarios. Dynamic detection methods are categorized based on the different phases in which they employ LLMs, such as using them for test case generation, providing feedback guidance, and conducting output assessment. Static detection methods are classified according to whether they analyze the source code or the binary of the software under test. Furthermore, we investigate the prompt engineering and model fine-tuning strategies adopted within these studies. Finally, we summarize the emerging trend of integrating LLMs into software defect detection, identify challenges to be addressed and prospect for some potential research directions.

Deep learning (DL) frameworks serve as the backbone for a wide range of artificial intelligence applications. However, bugs within DL frameworks can cascade into critical issues in higher-level applications, jeopardizing reliability and security. While numerous techniques have been proposed to detect bugs in DL frameworks, research exploring common API patterns across frameworks and the potential risks they entail remains limited. Notably, many DL frameworks expose similar APIs with overlapping input parameters and functionalities, rendering them vulnerable to shared bugs, where a flaw in one API may extend to analogous APIs in other frameworks. To address this challenge, we propose MirrorFuzz, an automated API fuzzing solution to discover shared bugs in DL frameworks. MirrorFuzz operates in three stages: First, MirrorFuzz collects historical bug data for each API within a DL framework to identify potentially buggy APIs. Second, it matches each buggy API in a specific framework with similar APIs within and across other DL frameworks. Third, it employs large language models (LLMs) to synthesize code for the API under test, leveraging the historical bug data of similar APIs to trigger analogous bugs across APIs. We implement MirrorFuzz and evaluate it on four popular DL frameworks (TensorFlow, PyTorch, OneFlow, and Jittor). Extensive evaluation demonstrates that MirrorFuzz improves code coverage by 39.92% and 98.20% compared to state-of-the-art methods on TensorFlow and PyTorch, respectively. Moreover, MirrorFuzz discovers 315 bugs, 262 of which are newly found, and 80 bugs are fixed, with 52 of these bugs assigned CNVD IDs.

The security of deep learning (DL) libraries is crucial due to their central role in developing AI applications. Fuzzing has become a key technique for discovering bugs in DL libraries, but generating high-quality seeds still poses a significant challenge. Although large language models (LLMs) offer promising opportunities, current methods that utilize single-agent frameworks for seed generation often produce invalid seeds. This issue arises from the complexities of DL API structures and the inherent randomness of LLMs, which ultimately reduce fuzzing efficiency. To overcome these challenges, we propose a collaborative multi-agent framework that utilizes specialized LLM-driven agents for iterative seed refinement. This framework consists of three components: (1) a coding agent that incorporates historical bug knowledge to produce the initial bug-prone seeds,(2) a repair agent performs static analysis on the initial seeds and repairs the invalid seeds, and (3) a mutation agent leverages four mutation operators to explore the test space thoroughly.Our framework improves seed validity and functional diversity through ongoing collaboration among agents and established feedback loops. Experimental evaluations show that our approach achieves a 20% increase in seed validity compared to state-of-the-art methods and uncovers 12 previously unknown bugs in popular DL libraries.