WebAssembly (Wasm) is an emerging binary format that serves as a compilation target for over 40 programming languages. Wasm runtimes provide execution environments that enhance portability by abstracting away operating systems and hardware details. A key component in these runtimes is the WebAssembly System Interface (WASI), which manages interactions with operating systems, like file operations. Considering the critical role of Wasm runtimes, the community has aimed to detect their implementation bugs. However, no work has focused on WASI-specific bugs that can affect the original functionalities of running Wasm binaries and cause unexpected results. To fill the void, we present DrWASI , the first general-purpose differential testing framework for WASI implementations. Our approach uses a large language model to generate seeds and applies variant and environment mutation strategies to expand and enrich the test case corpus. We then perform differential testing across major Wasm runtimes. By leveraging dynamic and static information collected during and after the execution, DrWASI can identify bugs. Our evaluation shows that DrWASI uncovered 33 unique bugs, with all confirmed and 7 fixed by developers. This research represents a pioneering step in exploring a promising yet under-explored area of the Wasm ecosystem, providing valuable insights for stakeholders.

A fundamental component of the Wasm ecosystem is the Wasm runtime, as it directly impacts whether Wasm applications can be executed as expected. Bugs in Wasm runtimes are frequently reported, so the research community has made a few attempts to design automated testing frameworks to detect bugs in Wasm runtimes. However, existing testing frameworks are limited by the quality of test cases, i.e., they face challenges in generating Wasm binaries that are both semantically rich and syntactically correct. As a result, complicated bugs cannot be triggered effectively. In this work, we present WASMaker, a novel differential testing framework that can generate complicated Wasm test cases by disassembling and assembling real-world Wasm binaries, which can trigger hidden inconsistencies among Wasm runtimes. To further pinpoint the root causes of unexpected behaviors, we design a runtime-agnostic root cause location method to locate bugs accurately. Extensive evaluation suggests that WASMaker outperforms state-of-the-art techniques in terms of both efficiency and effectiveness. We have uncovered 33 unique bugs in popular Wasm runtimes, among which 25 have been confirmed.

WebAssembly (abbreviated WASM) has emerged as a promising language of the Web and also been used for a wide spectrum of software applications such as mobile applications and desktop applications. These applications, named WASM applications, commonly run in WASM runtimes. Bugs in WASM runtimes are frequently reported by developers and cause the crash of WASM applications. However, these bugs have not been well studied. To fill in the knowledge gap, we present a systematic study to characterize and detect bugs in WASM runtimes. We first harvest a dataset of 311 real-world bugs from hundreds of related posts on GitHub. Based on the collected high-quality bug reports, we distill 31 bug categories of WASM runtimes and summarize their common fix strategies. Furthermore, we develop a pattern-based bug detection framework to automatically detect bugs in WASM runtimes. We apply the detection framework to seven popular WASM runtimes and successfully uncover 60 bugs that have never been reported previously, among which 13 have been confirmed and 9 have been fixed by runtime developers.