Abstract As software projects evolve and grow in size and complexity, so do their test suites. Test suite reduction (TSR) aims at reducing the size of a test suite by removing redundant and obsolete test cases based on a coverage metric while preserving its fault detection capabilities. The contributions of this paper are twofold: (1) We examine a lesser‐known coverage criterion, that is, checked coverage. Checked coverage not only investigates if a part of the code was executed but also if it was checked by a test oracle. In an empirical evaluation, we performed TSR based on different reduction algorithms, coverage metrics, and open‐source Java projects with our own TSR tool to determine the most effective and efficient combination of metric and method. (2) Given the results of the first evaluation, we further investigate the potential of parameter optimization in regard to a genetic reduction algorithm. In particular, we focus on finding a general setting for the parameters crossover rate and mutation rate such that test suites can be reduced in a reasonable time while maintaining a high fault detection power.

As the size of software projects increases, their test suites usually grow accordingly. Test suite size, however, has a direct impact on the efficiency of software testing. Hence, test suite reduction (TSR) procedures aim at removing redundant test cases while maintaining the suites fault detection capabilities (FDC). This paper explores checked coverage as a coverage metric for TSR; checked coverage not only investigates if a part of code was executed but also if it was checked by a test oracle. Previously, this metric has been applied successfully as an indicator for oracle quality. To assess how suitable checked coverage is in comparison to traditional metrics, such as line or method coverage, we developed a TSR tool for Java programs. In an empirical evaluation, we performed TSR based on different reduction algorithms, coverage metrics, and open-source Java projects. Our study investigates both the efficiency of the TSR as well as effectiveness in regard to the FDC and size of the reduced test suites.