We report on our experience at Google deploying a variety of coverage-guided and black-box fuzzers to find bugs in compilers for two graphics shading languages: the WebGPU Shading Language (WGSL) and Standard Portable Intermediate Representation (SPIR-V). We discuss the deployment of coverage-guided fuzzing on ClusterFuzz and OSS-Fuzz using libFuzzer’s built-in mutators, and a number of custom mutators that exploit knowledge of the syntax and semantics of WGSL and SPIR-V. We also discuss the deployment of several black box fuzzers on ClusterFuzz, including two that are based on new randomised program generators for WGSL, which we have also used in a targeted fashion for end-to-end testing of WebGPU implementations. We discuss a series of insights arising from our experience that we hope will be valuable to practitioners and researchers interested in applying fuzzing to industrial problems.

We introduce program reconditioning, a method for allowing program generation and differential testing to be used to find miscompilation bugs, and test-case reduction to be used to simplify bug-triggering programs, even when (a) the programming language of interest features undefined behaviour (UB) and (b) no tools exist to detect and avoid this UB. We present two program generation tools based on our reconditioning idea: GLSLsmith for the OpenGL Shading Language (GLSL), a widely-used language for graphics programming, and WGSLsmith for the WebGPU Shading Language (WGSL), a new language for web-based graphics rendering. GLSL features many UBs, but unlike for languages such as C and C++ no tools exist to detect them automatically. While the WGSL language specification features very limited UB, early WGSL implementations do exhibit UB, for reasons of initial implementation simplicity, making it challenging to test them to quickly detect and eliminate unrelated miscompilation bugs. Thanks to reconditioning, we show that GLSLsmith and WGSLsmith allow differential testing and test-case reduction to be applied to compilers for GLSL and WGSL for the first time, despite the unavailability of UB detection techniques for these languages. Through a large testing campaign, we have found 24 and 33 bugs in GLSL and WGSL compilers, respectively. We present experiments showing that when reconditioning is disabled, compiler testing leads to a high rate of test programs that appear to trigger miscompilation bugs, but actually just feature UB. We also present a novel approach to managing floating-point roundoff error using reconditioning, implemented for both GLSL and WGSL.