Abstract As software systems grow in complexity, the importance of efficient defect detection escalates, becoming vital to maintain software quality. In recent years, artificial intelligence technology has boomed. In particular, with the proposal of Large Language Models (LLMs), researchers have found the huge potential of LLMs to enhance the performance of software defect detection. This review aims to elucidate the relationship between LLMs and software defect detection. We categorize and summarize existing research based on the distinct applications of LLMs in dynamic and static detection scenarios. Dynamic detection methods are categorized based on the different phases in which they employ LLMs, such as using them for test case generation, providing feedback guidance, and conducting output assessment. Static detection methods are classified according to whether they analyze the source code or the binary of the software under test. Furthermore, we investigate the prompt engineering and model fine-tuning strategies adopted within these studies. Finally, we summarize the emerging trend of integrating LLMs into software defect detection, identify challenges to be addressed and prospect for some potential research directions.

IoT devices, now integral to our daily routines, offer unparalleled convenience but also face mounting security threats. Embedded web services, prevalent in public networks, pose a major risk to these devices. While research has focused on detecting vulnerabilities in IoT embedded web services, it has overlooked the presence of invisible interfaces, which have emerged as significant security threats. In this paper, we propose InvRadar, a novel framework for detecting vulnerabilities in invisible interfaces of embedded web services in IoT devices. Specifically, InvRadar identifies invisible interfaces by analyzing the differences between the front-end visible interface keywords and the back-end interface keywords through a correlation analysis method. Subsequently, InvRadar uses a static taint analysis method to detect the vulnerabilities that can be triggered by the invisible interfaces. To validate the performance of InvRadar, we conduct extensive experiments and compare InvRadar with the state-of-the-art methods. In testing 13 device firmware, InvRadar identifies 1,793 invisible interfaces and detects 124 vulnerabilities, including 53 newly discovered ones, with 34 receiving new CVE/CNVD IDs. Additionally, InvRadar outperforms the state-of-the-art methods in interface keyword extraction, border binary and data ingestion function identification.

Deep learning (DL) frameworks serve as the backbone for a wide range of artificial intelligence applications. However, bugs within DL frameworks can cascade into critical issues in higher-level applications, jeopardizing reliability and security. While numerous techniques have been proposed to detect bugs in DL frameworks, research exploring common API patterns across frameworks and the potential risks they entail remains limited. Notably, many DL frameworks expose similar APIs with overlapping input parameters and functionalities, rendering them vulnerable to shared bugs, where a flaw in one API may extend to analogous APIs in other frameworks. To address this challenge, we propose MirrorFuzz, an automated API fuzzing solution to discover shared bugs in DL frameworks. MirrorFuzz operates in three stages: First, MirrorFuzz collects historical bug data for each API within a DL framework to identify potentially buggy APIs. Second, it matches each buggy API in a specific framework with similar APIs within and across other DL frameworks. Third, it employs large language models (LLMs) to synthesize code for the API under test, leveraging the historical bug data of similar APIs to trigger analogous bugs across APIs. We implement MirrorFuzz and evaluate it on four popular DL frameworks (TensorFlow, PyTorch, OneFlow, and Jittor). Extensive evaluation demonstrates that MirrorFuzz improves code coverage by 39.92% and 98.20% compared to state-of-the-art methods on TensorFlow and PyTorch, respectively. Moreover, MirrorFuzz discovers 315 bugs, 262 of which are newly found, and 80 bugs are fixed, with 52 of these bugs assigned CNVD IDs.

As software reuse has become increasingly prevalent in the modern era, binary code similarity detection plays a critical role in program analysis. Numerous machine learning methods have been introduced to this field, with the primary challenge lying in the effective representation of binary code. While existing approaches leverage multimodal features for embedding, they still adopt relatively simple fusion methods to combine different modalities. Therefore, their performance can be limited by inadequate modality interaction during the feature fusion step and an over-reliance on a single modality in the final embedding step. To address these issues, we propose FuseBinRepr, a method that fuses text modality and graph modality representation techniques using a fusion model architecture and three specialized learning tasks to enhance binary code similarity detection. We design a fusion architecture integrating both text and graph embeddings via cross-attention and self-attention mechanisms. For model training, we developed three tasks: text-graph alignment (TGA), graph masking recovery (GMR), and contrastive learning (CL), to capture and align high-level semantics across modalities. Through evaluation, FuseBinRepr demonstrates improvements in Mean Reciprocal Rank (MRR) and Recall compared to state-of-the-art baseline methods, achieving increases of up to 40.8 % and 42 %, respectively. Ablation studies confirm the robustness of our model design, as well as the effectiveness of pretraining tasks. In the downstream software vulnerability detection task, FuseBinRepr achieves the best MRR and Recall performance in ranking CVE binary functions.

Mini-apps, which run on super-apps, have attracted a large number of users due to their lightweight nature and the convenience of supporting the authorized use of super-app user information. Super-apps employ encryption to protect the transmission of sensitive identity information authorized by users to the mini-app, using the session key as the key. However, we have identified a risk of session key leakage, which could be exploited to maliciously manipulate sensitive user identity information, thereby posing a significant threat to user data security. To reveal this damage, we explore potential business scenarios of session key leakage in detail. Nevertheless, the diversity in design among various mini-apps makes automated testing of these business scenarios at a large scale challenging. This diversity is reflected in the inconsistent naming of identical types of controls and the disparate execution orders of controls within the same business scenarios across different mini-apps. To overcome these challenges, we propose Whiskey, which can adaptively and intelligently optimize dynamic testing strategies for mini-apps with diverse designs using large language models to detect session key leakage at scale. We evaluated Whiskey on 157,063 WeChat mini-apps and 10,000 TikTok mini-apps, and found that 15,712 of WeChat mini-apps and 678 of TikTok mini-apps had session key leakage vulnerabilities. Further analysis showed that this leakage could lead to account takeover and promotion abuse attacks. We responsibly reported the detection results to Tencent and the mini-app vendors. At the time of submission, 17 reported issues had been assigned CNVD IDs.

With the rising demand for website data storage, leveraging cloud storage services for vast user file storage has become prevalent. Nowadays, a new file upload scenario has been introduced, allowing web users to upload files directly to the cloud storage service. This new scenario offers convenience but involves more roles (i.e., web users, web servers, and cloud storage services) and their interactions, bringing new security threats. In this paper, we perform the first systematic security study in this scenario. With in-depth analysis, we identify six new types of vulnerabilities and conduct large-scale real-world measurements on the top 500 Alexa Rank websites. Among these websites, 182 (36.4%) use cloud storage services, illustrating the widespread use of the cloud. Then, we perform a detailed analysis of 28 popular websites that allow user upload. Surprisingly, they all have at least one of the six vulnerabilities. Totally, we discover 79 new vulnerabilities and responsibly report them to the websites. Many popular websites respond positively, including Google, Reddit, and CSDN. We discuss the root causes of these vulnerabilities and propose possible mitigation methods. In summary, our work offers significant value in understanding the security risks of cloud storage services for websites and facilitating future research.

Since the compiler acts as a core component in software building, it is essential to ensure its availability and reliability through software testing and security analysis. Most research has focused on compiler robustness when compiling various test cases, while the reliability of compiler options lacks attention, especially since each option can activate a specific compiler function. Although some researchers have made efforts in testing it, the insufficient utilization of compiler command feedback messages leads to the poor efficiency, which hinders more diverse and in-depth testing.In this paper, we propose a novel solution to enhance black-box compiler option fuzzing by utilizing command feedback, such as error messages, standard output and compiled files, to guide the error fixing and option pruning via prompting large language models for suggestions. We have implemented the prototype and evaluated it on 4 versions of LLVM. Experiments show that our method significantly improves the detection of crashes, reduces false negatives, and even increase the success rate of compilation when compared to the baseline. To date, our method has identified hundreds of unique bugs, and 9 of them are previously unknown. Among these, 8 have been assigned CVE numbers, and 1 has been fixed following our report.

A webshell is a malicious backdoor that allows remote access and control to a web server by executing arbitrary commands. The wide use of obfuscation and encryption technologies has greatly increased the difficulty of webshell detection. To this end, we propose a novel webshell detection model leveraging the grammatical features extracted from the PHP code. The key idea is to combine the executable data characteristics of the PHP code with static text features for webshell classification. To verify the proposed model, we construct a cleaned data set of webshell consisting of 2,917 samples from 17 webshell collection projects and conduct extensive experiments. We have designed three sets of controlled experiments, the results of which show that the accuracy of the three algorithms has reached more than 99.40%, the highest reached 99.66%, the recall rate has been increased by at least 1.8%, the most increased by 6.75%, and the F1 value has increased by 2.02% on average. It not only confirms the efficiency of the grammatical features in webshell detection but also shows that our system significantly outperforms several state‐of‐the‐art rivals in terms of detection accuracy and recall rate.