Software testing is a critical stage in the software development lifecycle, ensuring that programs behave correctly and reliably across diverse environments. Unlike general software testing, compiler testing requires well-structured input programs that systematically exercise the compiler’s various optimization passes, since incomplete coverage can easily mask critical issues. Currently, a range of techniques exists for compiler testing, including differential testing, fuzzing, slicing, sanitizers and formal methods based validation tools, While these methods are effective at detecting compiler crashes, undefined behavior, or general correctness bugs, they are largely inadequate for systematically evaluating compiler optimizations. Tools like Csmith have pioneered random program generation for compiler testing, but they lack precise control over program structure, which is critical when the goal is to stress specific optimization passes or identify performance regressions. To address this limitation, we developed CPerf- Smith, an extension of Csmith that provides fine-grained control over code constructs, enabling targeted generation of programs that exercise particular compiler optimizations. Through our experiments, CPerfSmith was able to generate over 100 programs exhibiting significant runtime differences across industry-standard C compilers (GCC, Clang, and AOCC). Furthermore, we successfully created more than 30 test cases that revealed significant performance regressions between two subsequent release versions of Clang, demonstrating the tool’s ability to detect missing or improperly applied optimizations.

Programming environments for sequential programs provide strong debugging support. However, concurrent programs, especially under relaxed memory models, lack powerful interactive debugging tools. In this work, we present Gambit, an interactive debugging environment that uses gdb to run a concrete debugging session on a concurrent program, while employing a symbolic execution on the program trace in the background simultaneously. The symbolic execution is analysed by a theorem prover to answer queries from the programmer on possible scenarios resulting from alternate thread interleavings or due to reorderings on other relaxed memory models.

Concretization is an effective weapon in the armory of symbolic execution engines. However, concretization can lead to loss in coverage, path divergence, and generation of test-cases on which the intended bugs are not reproduced. In this paper, we propose an algorithm, Deferred Concretization, that uses a new category for values within symbolic execution (referred to as the symcrete values) to pend concretization till they are actually needed. Our tool, COLOSSUS, built around these ideas, was able to gain an average coverage improvement of 66.94% and reduce divergence by more than 55% relative to the state-of-the-art symbolic execution engine, KLEE. Moreover, we found that KLEE loses about 38.60% of the states in the symbolic execution tree that COLOSSUS is able to recover, showing that COLOSSUS is capable of covering a much larger coverage space.

In spite of decades of research in bug detection tools, there is a surprising dearth of ground-truth corpora that can be used to evaluate the efficacy of such tools. Recently, systems such as LAVA and EvilCoder have been proposed to automatically inject bugs into software to quickly generate large bug corpora, but the bugs created so far differ from naturally occurring bugs in a number of ways. In this work, we propose a new automated bug injection system, Apocalypse, that uses formal techniques—symbolic execution, constraint-based program synthesis and model counting—to automatically inject fair (can potentially be discovered by current bug-detection tools), deep (requiring a long sequence of dependencies to be satisfied to fire), uncorrelated (each bug behaving independent of others), reproducible (a trigger input being available) and rare (can be triggered by only a few program inputs) bugs in large software code bases. In our evaluation, we inject bugs into thirty Coreutils programs as well as the TCAS test suite. We find that bugs synthesized by Apocalypse are highly realistic under a variety of metrics, that they do not favor a particular bug-finding strategy (unlike bugs produced by LAVA), and that they are more difficult to find than manually injected bugs, requiring up around 240× more tests to discover with a state-of-the-art symbolic execution tool.

Bounded model checking based debugging solutions search for mutations of program expressions that produce the expected output for a currently failing test. However, the current localization tools are not regression aware: they do not use information from the passing tests in their localization formula. On the other hand, the current repair tools attempt to guarantee regression freedom: when provided with a set of passing tests, they guarantee that none of these tests can break due to the suggested repair patch, thereby constructing a large repair formula.