Large Language Models (LLMs) have been found to show over-refusal problems—erroneously rejecting benign queries due to overly conservative safety measures—a critical functional flaw that undermines their reliability and usability. Current methods for testing this behavior are demonstrably inadequate, suffering from flawed benchmarks and limited test generation capabilities, as highlighted by our empirical user study. To the best of our knowledge, this paper introduces the first evolutionary testing framework, ORFuzz, for the systematic detection and analysis of LLM over-refusals. ORFuzz uniquely integrates three core components: (1) safety category-aware seed selection for comprehensive test coverage, (2) adaptive mutator optimization using reasoning LLMs to generate effective test cases, and (3) OR-Judge, a human-aligned judge model validated to accurately reflect user perception of toxicity and refusal. Our extensive evaluations demonstrate that ORFuzz generates diverse, validated over-refusal instances at a rate (6.98% average) more than double that of leading baselines, effectively uncovering vulnerabilities. Furthermore, ORFuzz’s outputs form the basis of ORFuzzSet, a new benchmark of 1,786 highly transferable test cases that achieves a superior 57.37% average over-refusal rate across 14 diverse LLMs, significantly outperforming existing datasets. ORFuzz and ORFuzzSet provide a robust automated testing framework and a valuable community resource, paving the way for developing more reliable and trustworthy LLM-based software systems. The code of this paper is available at: https://github.com/HotBento/ORFuzz.

Implementing upgradeable smart contracts (USCs) has become a trend in Decentralized applications. Due to blockchain immutability, ensuring the upgradeability of smart contracts requires specialized implementation strategies. A systematic study of developers’ concerns regarding USC development can provide insights to reduce development costs and increase software robustness. In this work, we propose the first empirical study on exploring developers’ concerns over USCs. We first extract 2,224 USC-related posts based on an iterative process combining keyword filtering and manual filtering of posts from theEthereum StackExchangecommunity. Following open card-sorting practice, we propose 13 common development concerns based on the extracted posts, as well as the causes of these concerns. Furthermore, we analyze the frequency of these concerns within developer discussions. We highlight the most significant concerns of developers, where the top 5 most frequently discussed concerns are notably absent in existing research, e.g., code bugs originating from interacting and testing USCs in specific development frameworks. Additionally, we examine the real-world impact of these concerns by analyzing on-chain smart contracts and security reports from two widely referred databases, i.e.,RektandSlowmist. Based on case studies of the USC-related security reports in the past two years, we found that the causes of most USC security incidents are related to the identified concerns. Besides, we proposed a semi-automatic tool based on static analysis to detect related bugs and found 26 bugs in real-world smart contracts, which have involved over 0.3 million transactions. Based on these findings, we provide suggestions on the less-solved-yet-prevalent concerns regarding usability and security of USC development, such as facilitating the testing on USCs under existing development frameworks.

WebAssembly (WASM) has rapidly emerged as a ubiquitous target for web browsers, server-side applications, and blockchain platforms, with promising performance and portability. As WASM grows in popularity, ensuring its security and resilience becomes paramount. However, traditional fuzzing approaches struggle to detect potential security vulnerabilities in existing WebAssembly runtimes due to their lack of perception of the WASM file structure. In this paper, we introduce Sw Fuzz, a dedicated fuzzing framework tailored for WASM binaries. SwFuzz integrates comprehensive structure-sensitive policies that capture the nuances and intricacies within the WASM binaries. Our proposed fuzzing framework not only identifies vulnerabilities present in conventional binaries but also emphasizes the detection of WASM-specific bugs that have previously gone unnoticed. Experimental results demonstrate that Sw Fuzz has discovered numerous new bugs, with 17 CCVEs being assigned, underscoring the importance of a specialized fuzzing framework for evolving platforms like WASM. Our findings also highlight the critical requirement for a proactive approach to securing the WASM landscape.

Browsers are responsible for managing and interpreting the diverse data coming from the web. Despite the considerable efforts of developers, however, it is nearly impossible to completely eliminate potential vulnerabilities in such complicated software. While a family of fuzzing techniques has been proposed to detect flaws in web browsers, they still face the inherent challenge of generating test inputs with low semantic correctness and poor diversity. In this paper, we propose Tacoma, a novel fuzzing framework tailored for web browsers. Tacoma comprises three main modules: a semantic parser, a semantic aligner, and an input generator. By taking advantage of fine-grained semantic alignment techniques, Tacoma is capable of generating semantically correct test inputs, which significantly improve the probability of a fuzzer in triggering a deep browser state. In particular, by integrating a scope-aware strategy into input generation, Tacoma is able to deal with asynchronous code generation, thereby substantially increasing the diversity of the generated test inputs. We conduct extensive experiments to evaluate Tacoma on three production-level browsers, i.e., Chromium, Safari, and Firefox. Empirical results demonstrate that Tacoma outperforms state-of-the-art browser fuzzers in both achieving code coverage and detecting unique crashes. So far, Tacoma has identified 32 previously unknown bugs, 10 of which have been assigned CVEs. It is worth noting that Tacoma unearthed two bugs in Chromium that have remained undetected for ten years.

With the wide application and deployment of cloud computing in enterprises, virtualization developers and security researchers are paying more attention to cloud computing security. The core component of cloud computing products is the hypervisor, which is also known as the virtual machine monitor (VMM) that can isolate multiple virtual machines in one host machine. However, compromising the hypervisor can lead to virtual machine escape and the elevation of privilege, allowing attackers to gain the permission of code execution in the host. Therefore, the security analysis and vulnerability detection of the hypervisor are critical for cloud computing enterprises. Importantly, virtual devices expose many interfaces to a guest user for communication, making virtual devices the most vulnerable part of a hypervisor. However, applying fuzzing to the virtual devices of a hypervisor is challenging because the data structures transferred by DMA are constructed in a nested form according to protocol specifications. Failure to understand the protocol of the virtual devices will make the fuzzing process stuck in the initial fuzzing stage, resulting in inefficient fuzzing.