Abstract The minimizing Delta Debugging (DDMIN) was among the first algorithms designed to automate the task of reducing test cases. Its popularity is based on the characteristics that it works on any kind of input, without knowledge about the input structure. Several studies proved that smaller outputs can be produced faster with more advanced techniques (e.g., building a tree representation of the input and reducing that data structure); however, if the structure is unknown or changing frequently, maintaining the descriptors might not be resource‐efficient. Therefore, in this paper, we focus on the evaluation of the novel fixed‐point iteration of minimizing Delta Debugging (DDMIN*) on publicly available test suites related to software engineering. Our experiments show that DDMIN* can help reduce inputs further by 48.08% on average compared to DDMIN (using lines as the units of the reduction). Although the effectiveness of the algorithm improved, it comes with the cost of additional testing steps. This study shows how the characteristics of the input affect the results and when it pays off using DDMIN*.

Finding the relevant part of failure-inducing inputs is an important first step on the path of debugging. If much of a test case that triggers a bug does not contribute to the actual failure, then the time required to fix the bug can increase considerably. In this paper, we focus on the memory requirements of automatic test case reduction. During minimization, the same test case might be tested multiple times, and determining the outcome of an input may take time, therefore, different caching solutions were proposed to avoid re-testing previously seen inputs. We investigated the caching solutions of DDMIN and HDD, and found that their scaling is suboptimal. We propose three optimizations for one of the state-of-the-art caching solutions: with the optimizations combined, DDMIN requires 96% and HDD requires 85% less memory compared to the baseline implementation. Furthermore, as a side effect, the reduction becomes faster by 9.9% with DDMIN.

Spectrum-Based Fault Localization (SBFL) is based on risk formulas to rank program elements, which work generally well in various situations. However, it cannot be ruled out that zero division might happen during score calculation, which has negative consequences, e.g., essential elements will not be in the top part of the rank list. The literature has given several strategies to tackle the problem, although there is little knowledge on which one to use. In our work, we performed mathematical analysis and an empirical study to find out how this phenomenon affects SBFL. Results show that division by zero happens in many cases, and the strategies can mitigate their consequences with varying success. Thus, we propose a combined method to avoid the threat of division by zero and improve the trustworthiness of SBFL. Our proposals should be taken into consideration whenever a formula is being used or a new one is proposed.

Abstract Minimizing failing test cases is an important preprocessing step on the path of debugging. If much of a test case that triggered a bug does not contribute to the actual failure, then the time required to fix the bug can increase considerably. However, test case reduction itself can be a time‐consuming task, especially if done manually. Therefore, automated minimization techniques have been proposed, the minimizing Delta Debugging and the Hierarchical Delta Debugging (HDD) algorithms being the most well known. In this paper, we investigated the input format of HDD, searching for structures that the algorithm cannot reduce. Motivated by the findings, we have created an algorithmic framework that enabled the use of transformations other than pruning. Furthermore, with the Transformation‐based Minimization framework, we propose to extend HDD and its coarse and recursive variants with a reduction method that does not prune subtrees but replaces them with compatible subtrees further down the hierarchy, called hoisting. We have evaluated various combinations of pruning and hoisting on multiple test suites and found that hoisting can help to further reduce the size of test cases by 27% on average and by 80% as best case compared with the baseline algorithm.

Minimizing failing test cases is an important pre-processing step on the path of debugging. If much of a test case that triggered a bug does not contribute to the actual failure, then the time required to fix the bug can increase considerably. However, test case reduction itself can be a time consuming task, especially if done manually. Therefore, automated minimization techniques have been proposed, the minimizing Delta Debugging (DDMIN) and the Hierarchical Delta Debugging (HDD) algorithms being the most well known. DDMIN does not need any information about the structure of the test case, thus it works for any kind of input. If the structure is known, however, it can be utilized to create smaller test cases faster. This is exemplified by HDD, which works on tree-structured inputs, pruning subtrees at each level of the tree with the help of DDMIN.In this paper, we propose to extend HDD with a reduction method that does not prune subtrees, but replaces them with compatible subtrees further down the hierarchy, called hoisting. We have evaluated various combinations of pruning and hoisting on multiple test suites and found that hoisting can help to further reduce the size of test cases by as much as 80% compared to the baseline HDD. We have also compared our results to other state-of-the-art test case reduction algorithms and found that HDD extended with hoisting can produce smaller output in most of the cases.