SUMMARY Dynamic slicing is a promising trace based technique that helps programmers in the process of debugging. In order to debug a failed run, dynamic slicing requires the dynamic dependence graph (DDG) information for that particular run. The two major challenges involved in utilizing dynamic slicing as a debugging technique are the efficient computation of the DDG and the efficient computation of the dynamic slice, given the DDG. In this paper, we present an efficient debugger, which first computes the DDG efficiently while the program is executing; dynamic slicing is later performed efficiently on the computed DDG, on demand. To minimize program slowdown during the online computation of DDG, we make the design decision of not outputting the computed dependencies to a file, instead, storing them in memory in a specially allocated fixed size circular buffer. The size of the buffer limits the length of the execution history that can be stored. To maximize the execution history that can be maintained, we introduce optimizations to eliminate the storage of most of the generated dependencies, at the same time ensuring that those that are stored are sufficient to capture the bug. Experiments conducted on CPU‐intensive programs show that our optimizations are able to reduce the trace rate from 16 to 0.8 bytes per executed instruction. This enables us to store the dependence trace history for a window of 20 million executed instructions in a 16‐MB buffer. Our debugger is also very efficient, yielding slicing times of around a second, and only slowing down the execution of the program by a factor of 19 during the online tracing step. Using recently proposed architectural support for monitoring, we are also able to handle multithreaded programs running on multicore processors. Copyright © 2011 John Wiley & Sons, Ltd.

The above article (DOI: 10.1002/spe.1105) was published in print in Volume 42, Issue 8 (August 2012), for which errors in the arrangement of the affiliation links have subsequently been identified. This article has been corrected in the online version of the issue. The affiliation links have been corrected as follows:Vijay Nagarajan1, Dennis Jeffrey2, Rajiv Gupta3 and Neelam Gupta1School of Informatics, University of Edinburgh, Edinburgh, EH8 9AB, UK2Google Inc., 1600 Amphitheatre Parkway, Mountain View, CA 94043, USA3CSE Department, University of California at Riverside, Riverside, CA 92521, USA

By studying the behavior of several programs that crash due to memory errors, we observed that locating the errors can be challenging because significant propagation of corrupt memory values can occur prior to the point of the crash. In this article, we present an automated approach for locating memory errors in the presence of memory corruption propagation. Our approach leverages the information revealed by a program crash: when a crash occurs, this reveals a subset of the memory corruption that exists in the execution. By suppressing (nullifying) the effect of this known corruption during execution, the crash is avoided and any remaining (hidden) corruption may then be exposed by subsequent crashes. The newly exposed corruption can then be suppressed in turn. By iterating this process until no further crashes occur, the first point of memory corruption—and the likely root cause of the program failure—can be identified. However, this iterative approach may terminate prematurely, since programs may not crash even when memory corruption is present during execution. To address this, we show how crashes can be exposed in an execution by manipulating the relative ordering of particular variables within memory. By revealing crashes through this variable re-ordering, the effectiveness and applicability of the execution suppression approach can be improved. We describe a set of experiments illustrating the effectiveness of our approach in consistently and precisely identifying the first points of memory corruption in executions that fail due to memory errors. We also discuss a baseline software implementation of execution suppression that incurs an average overhead of 7.2x, and describe how to reduce this overhead to 1.8x through hardware support.

We previously presented a fault localization technique called value replacement that repeatedly alters the state of an executing program to locate a faulty statement [9]. The technique searches for program statements involving values that can be altered during runtime to cause the incorrect output of a failing run to become correct. We showed that highly effective fault localization results could be achieved by the technique on programs containing single faults. In the current work, we generalize value replacement so that it can also perform effectively in the presence of multiple faults. We improve scalability by describing two techniques that significantly improve the efficiency of value replacement. In our experimental study, our generalized technique effectively isolates multiple simultaneous faults in time on the order of minutes in each case, whereas in , the technique had sometimes required time on the order of hours to isolate only single faults.

We present a tool called BugFix that can assist developers in fixing program bugs. Our tool automatically analyzes the debugging situation at a statement and reports a prioritized list of relevant bug-fix suggestions that are likely to guide the developer to an appropriate fix at that statement. BugFix incorporates ideas from machine learning to automatically learn from new debugging situations and bug fixes over time. This enables more effective prediction of the most relevant bug-fix suggestions for newly-encountered debugging situations. The tool takes into account the static structure of a statement, the dynamic values used at that statement by both passing and failing runs, and the interesting value mapping pairs [17] associated with that statement. We present a case study illustrating the efficacy of BugFix in helping developers to fix bugs.

We present a general approach for automatically isolating the root causes of memory-related bugs in software. Our approach is based on the observation that most memory bugs involve uses of corrupted memory locations. By iteratively suppressing (nullifying) the effects of these corrupted memory locations during program execution, our approach gradually isolates the root cause of a memory bug. Our approach can work for common memory bugs such as buffer overflows, uninitialized reads, and double frees. However, our approach is particularly effective in finding root causes for memory bugs in which memory corruption propagates during execution until an observable failure such as a program crash occurs.

We are designing scalable dynamic information flow tracking techniques and employing them to carry out tasks related to debugging (bug location and fault avoidance), security (software attack detection), and data validation (lineage tracing of scientific data). The focus of our ongoing work is on developing online dynamic analysis techniques for long running multithreaded programs that may be executed on a single core or on multiple cores to exploit thread level parallelism.

We present a value profile based approach for ranking program statements according to their likelihood of being faulty. The key idea is to see which program statements exercised during a failing run use values that can be altered so that the execution instead produces correct output. Our approach is effective in locating statements that are either faulty or directly linked to a faulty statement. We present experimental results showing the effectiveness and efficiency of our approach. Our approach outperforms Tarantula which, to our knowledge, is the most effective prior approach for statement ranking based fault localization using the benchmark programs we studied.

Dynamic slicing is a promising trace based technique that helps programmers in the process of debugging. In order to debug a failed run, dynamic slicing requires the dynamic dependence graph (DDG) information for that particular run. In prior work, address and control-flow traces are collected online and then extensively post-processed offline to yield the DDG, upon which slicing is performed. Unfortunately, the offline post-processing step can be extremely time consuming [21], impeding the use of dynamic slicing as a debugging technique. In this paper, we present ONTRAC, an efficient online tracing system, that directly computes the dynamic dependences online, thus eliminating the expensive offline postprocessing step. To minimize the program slowdown, we make the design decision of not outputting the computed dependences to a file, instead storing them in memory in a specially allocated fixed size circular buffer. The size of the buffer limits the length of the execution history that can be stored. To maximize the execution history that can be maintained, we introduce optimizations to eliminate the storage of most of the generated dependences, at the same time ensuring that those that are stored are sufficient to capture the bug. Our experiments conducted with real bugs confirm the above fact. Other experiments conducted on cpu-intensive programs show that our optimizations are able to reduce the trace-rate from 16 bytes to 0.8 bytes per executed instruction. This enables us to store the dependence trace history for a window of 20 million executed instructions in a 16 MB buffer. ONTRAC is also very efficient, only slowing down the execution by a factor of 19, eliminating the slowdown by a factor of 540 due to post-processing.

We are designing dynamic analysis techniques to identify executed program statements where a fault lies, i.e. the fault candidate set. To narrow the set of statements in the fault candidate set, automated dynamic analyses are being developed which consider not only a failed run of a program but also execution perturbations of the failed run. The goal of this work is to focus the users attention on a small subset of statements in the fault candidate set.

Execution omission errors are known to be difficult to locate using dynamic analysis. These errors lead to a failure at runtime because of the omission of execution of some statements that would have been executed if the program had no errors. Since dynamic analysis is typically designed to focus on dynamic information arising from executed statements, and statements whose execution is omitted do not produce dynamic information, detection of execution omission errors becomes a challenging task. For example, while dynamic slices are very effective in capturing faulty code for other types of errors, they fail to capture faulty code in presence of execution omission errors. To address this issue relevant slices have been defined to consider certain static dependences (called potential dependences) in addition to dynamic dependences. However, due to the conservative nature of static analysis, overly large slices are produced. In this paper, we propose a fully dynamic solution to locating execution omission errors using dynamic slices. We introduce the notion of implicit dependences which are dependences that are normally invisible to dynamic slicing due to the omission of execution of some statements. We design a dynamic method that forces the execution of the omitted code by switching outcomes of relevant predicates such that those implicit dependences are exposed and become available for dynamic slicing. Dynamic slices can be computed and effectively pruned to produce fault candidate sets containing the execution omission errors. We solve two main problems: verifying the existence of a single implicit dependence through predicate switching, and recovering the implicit dependences in a demand driven manner such that a small number of verifications are required before the root cause is captured. Our experiments show that the proposed technique is highly effective in capturing execution omission errors.

Abstract Dynamic slicing has long been considered as a useful tool for debugging programs as it effectively identifies a reduced fault candidate set which captures the faulty code in the program. Traditionally, a backward dynamic slice is computed starting from an incorrect value observed by the programmer during a failed program run. This incorrect value is either an incorrect output value or an incorrect address whose dereferencing causes the program to crash. Recently we proposed two additional types of dynamic slices, a forward dynamic slice of a minimal failure inducing input difference and a bidirectional dynamic slice of a critical predicate. We have built a dynamic slicing tool that computes dynamic slices by instrumenting program binaries and executing them to build dynamic dependence graphs. In this paper, through experiments, we demonstrate that supporting three different types of dynamic slices has the following advantages. First, we observe that for each type of dynamic slice there are distinct situations in which it is not applicable. Therefore, we should support multiple types of slices to handle a wide range of situations. Second, supporting multiple types of dynamic slices enables us to compute a multiple points dynamic slice which is the intersection of different type of available slices. Our experiments show that multiple points dynamic slices are significantly smaller than any of the three kinds of individual dynamic slices. Copyright © 2006 John Wiley & Sons, Ltd.

Software testing is a critical part of software development. As new test cases are generated over time due to software modifications, test suite sizes may grow significantly. Because of time and resource constraints for testing, test suite minimization techniques are needed to remove those test cases from a suite that, due to code modifications over time, have become redundant with respect to the coverage of testing requirements for which they were generated. Prior work has shown that test suite minimization with respect to a given testing criterion can significantly diminish the fault detection effectiveness (FDE) of suites. We present a new approach for test suite reduction that attempts to use additional coverage information of test cases to selectively keep some additional test cases in the reduced suites that are redundant with respect to the testing criteria used for suite minimization, with the goal of improving the FDE retention of the reduced suites. We implemented our approach by modifying an existing heuristic for test suite minimization. Our experiments show that our approach can significantly improve the FDE of reduced test suites without severely affecting the extent of suite size reduction

Software testing and retesting occurs continuously during the software development lifecycle to detect errors as early as possible. The sizes of test suites grow as software evolves. Due to resource constraints, it is important to prioritize the execution of test cases so as to increase chances of early detection of faults. Prior techniques for test case prioritization are based on the total number of coverage requirements exercised by the test cases. In this paper, we present a new approach to prioritize test cases based on the coverage requirements present in the relevant slices of the outputs of test cases. We present experimental results comparing the effectiveness of our prioritization approach with that of existing techniques that only account for total requirement coverage, in terms of ability to achieve high rate of fault detection. Our results present interesting insights into the effectiveness of using relevant slices for test case prioritization

Dynamic analysis techniques reason over program executions and deal with data produced at program execution time. Dynamic analysis and static analysis techniques complement each other. Hence, a key focus of the workshop is dynamic analysis of software systems with an emphasis on research that integrates static and dynamic analyses.

Typically debugging begins when during a program execution a point is reached at which an obviously incorrect value is observed. A general and powerful approach to automated debugging can be based upon identifying modifications to the program state that will bring the execution to a successful conclusion. However, searching for arbitrary changes to the program state is difficult due to the extremely large search space. In this paper we demonstrate that by forcibly switching a predicate's outcome at runtime and altering the control flow, the program state can not only be inexpensively modified, but in addition it is often possible to bring the program execution to a successful completion (i.e., program produces the desired output). By examining the switched predicate, also called the critical predicate, the cause of the bug can then be identified. Since the outcome of a branch can only be either true or false, the number of modified states resulting by predicate switching is far less than those possible through arbitrary state changes. Thus, it is possible to automatically search through modified states to find one that leads to the correct output. We have developed an implementation based upon dynamic instrumentation to perform this search through program re-execution -- the program is executed from the beginning and a predicate's outcome is switched to produce the desired change in control flow. To evaluate our approach, we tried our technique on several reported bugs for a number of UNIX utility programs. Our technique was found to be practical (i.e., acceptable in time taken) and effective (i.e., we were able to automatically identify critical predicates). Moreover we show that bidirectional dynamic slices of critical predicates capture the faulty code.

Given an incorrect value produced during a failed program run (e.g., a wrong output value or a value that causes the program to crash), the backward dynamic slice of the value very frequently captures the faulty code responsible for producing the incorrect value. Although the dynamic slice often contains only a small percentage of the statements executed during the failed program run, the dynamic slice can still be large and thus considerable effort may be required by the programmer to locate the faulty code.In this paper we develop a strategy for pruning the dynamic slice to identify a subset of statements in the dynamic slice that are likely responsible for producing the incorrect value. We observe that some of the statements used in computing the incorrect value may also have been involved in computing correct values (e.g., a value produced by a statement in the dynamic slice of the incorrect value may also have been used in computing a correct output value prior to the incorrect value). For each such executed statement in the dynamic slice, using the value profiles of the executed statements, we compute a confidence value ranging from 0 to 1 - a higher confidence value corresponds to greater likelihood that the execution of the statement produced a correct value. Given a failed run involving execution of a single error, we demonstrate that the pruning of a dynamic slice by excluding only the statements with the confidence value of 1 is highly effective in reducing the size of the dynamic slice while retaining the faulty code in the slice. Our experiments show that the number of distinct statements in a pruned dynamic slice are 1.79 to 190.57 times less than the full dynamic slice. Confidence values also prioritize the statements in the dynamic slice according to the likelihood of them being faulty. We show that examining the statements in the order of increasing confidence values is an effective strategy for reducing the effort of fault location.

Software testing is a critical part of software development. Test suite sizes may grow significantly with subsequent modifications to the software over time. Due to time and resource constraints for testing, test suite minimization techniques attempt to remove those test cases from the test suite that have become redundant over time since the requirements covered by them are also covered by other test cases in the test suite. Prior work has shown that test suite minimization techniques can severely compromise the fault detection effectiveness of test suites. In this paper, we present a novel approach to test suite reduction that attempts to selectively keep redundant tests in the reduced suites. We implemented our technique by modifying an existing heuristic for test suite minimization. Our experiments show that our approach can significantly improve the fault detection effectiveness of reduced suites without severely affecting the extent of test suite size reduction.

Software debugging is the process of locating and correcting faulty code. Prior techniques to locate faulty code either use program analysis techniques such as backward dynamic program slicing or exclusively use delta debugging to analyze the state changes during program execution. In this paper, we present a new approach that integrates the potential of delta debugging algorithm with the benefit of forward and backward dynamic program slicing to narrow down the search for faulty code. Our approach is to use delta debugging algorithm to identify a minimal failure-inducing input, use this input to compute a forward dynamic slice and then intersect the statements in this forward dynamic slice with the statements in the backward dynamic slice of the erroneous output to compute a failure-inducing chop. We implemented our technique and conducted experiments with faulty versions of several programs from the Siemens suite to evaluate our technique. Our experiments show that failure-inducing chops can greatly reduce the size of search space compared to the dynamic slices without significantly compromising the capability to locate the faulty code. We also applied our technique to several programs with known memory related bugs such as buffer overflow bugs. The failure-inducing chop in several of these cases contained only 2 to 4 statements which included the code causing memory corruption.

Software testing and retesting occurs continuously during the software development lifecycle to detect errors as early as possible and to ensure that changes to existing software do not break the software. Test suites once developed are reused and updated frequently as the software evolves. As a result, some test cases in the test suite may become redundant as the software is modified over time since the requirements covered by them are also covered by other test cases. Due to the resource and time constraints for re-executing large test suites, it is important to develop techniques to minimize available test suites by removing redundant test cases. In general, the test suite minimization problem is NP complete. In this paper, we present a new greedy heuristic algorithm for selecting a minimal subset of a test suite T that covers all the requirements covered by T. We show how our algorithm was inspired by the concept analysis framework. We conducted experiments to measure the extent of test suite reduction obtained by our algorithm and prior heuristics for test suite minimization. In our experiments, our algorithm always selected same size or smaller size test suite than that selected by prior heuristics and had comparable time performance.

Dynamic detection of program invariants is emerging as an important research area with many challenging problems. Generating suitable test cases that support accurate detection of program invariants is crucial to the dynamic approach of program invariant detection. In this paper, we propose a new structural coverage criterion called invariant-coverage criterion for dynamic detection of program invariants. We also show how the invariant-coverage criterion can be used to improve the accuracy of dynamically detected invariants. We first used the Daikon tool to report likely program invariants using the branch coverage and all definition-use pair coverage test suites for several programs. We then generated invariant-coverage suites for these likely invariants. When Daikon was run with the invariant-coverage suites, several spurious invariants reported earlier by the branch coverage and definition-use pair coverage test suites were removed from the reported invariants. Our approach also produced more meaningful invariants than randomly generated test suites.

Generating test inputs for a path in a function with integer and real parameters is an important but difficult problem. The problem becomes more difficult when pointers are passed as inputs to a function. In this case, the shape of the input data structure as well as the data values in the fields of this data structure need to be determined for traversal of the given path. The existing techniques to address this problem are inefficient since they use backtracking to simultaneously satisfy the constraints on the pointer variables and the data values used along the path. In this paper, we develop a novel approach that allows the generation of the shape of an input data structure to be done independently of the generation of its data values so as to force the control flow of a function along a given path. We also present a new technique that generates the shape of the input data structure by solving a set of pointer constraints derived in a single pass of the statements along the path. Although simple, our approach is powerful in handling pointer aliasing. It is efficient and provides a practical solution to generating test data for functions with pointer inputs.

Module cohesion describes the degree to which different actions performed by a module contribute towards a unified function. High module cohesion is a desirable property of a program. The program modifications during successive maintenance interventions can have negative effect on the structure of the program resulting in less cohesive modules. Therefore, metrics that measure module cohesion are important for software restructuring during maintenance. The existing static slice based module cohesion metrics significantly overestimate cohesion due to the limitations of static slicing. In this paper, we present a novel program execution based approach to measure module cohesion of legacy software. We define cohesion metrics based on definition-use pairs in the dynamic slices of the outputs. Our approach significantly improves the accuracy of cohesion measurement. We implemented our technique and measured module cohesion for several programs. Cohesion measurements using our technique were found to be more insightful than static slice based measurements.

Branch coverage is an important criteria used during the structural testing of programs. We present a new program execution based approach to generate input data that exercises a selected branch in a program. The test data generation is initiated with an arbitrarily chosen input from the input domain of the program. A new input is derived from the initial input in an attempt to force execution through any of the paths through the selected branch. The method dynamically switches among the paths that reach the branch by refining the input. Using a numerical iterative technique that attempts to generate an input to exercise the branch, it dynamically selects a path that offers less resistance. We have implemented the technique and present experimental results of its performance for some programs. Our results show that our method is feasible and practical.

A number of approaches have been proposed to automatically generate test data to traverse a given path in a program. We present a program execution based approach to generate test data for a given path. The technique derives a desired input for a test path by iteratively refining an arbitrarily chosen input. A set of linear constraints on the increments to the input are derived to refine the input. We solve this constraint set using a Unified Numerical Approach (UNA) developed in this paper. Our technique can generate both integer and floating point inputs as well as handle arrays and loops. We determine a basis set of paths for a program and use our technique to generate test data for this set. We implemented and experimentally evaluated our technique. We present results of generating input for scientific programs. The experimental results show that the technique is effective in that it generates input for most of the paths in the basis sets and also efficiently detects linear infeasible paths. Our experiments also show that our technique is efficient in the number of iterations required to generate test data. The time performance shows that it provides a practical method to automatically generate test data for scientific programs.

An important problem that arises in path oriented testing is the generation of test data that causes a program to follow a given path. In this paper, we present a novel program execution based approach using an iterative relaxation method to address the above problem. In this method, test data generation is initiated with an arbitrarily chosen input from a given domain. This input is then iteratively refined to obtain an input on which all the branch predicates on the given path evaluate to the desired outcome. In each iteration the program statements relevant to the evaluation of each branch predicate on the path are executed, and a set of linear constraints is derived. The constraints are then solved to obtain the increments for the input. These increments are added to the current input to obtain the input for the next iteration. The relaxation technique used in deriving the constraints provides feedback on the amount by which each input variable should be adjusted for the branches on the path to evaluate to the desired outcome.When the branch conditions on a path are linear functions of input variables, our technique either finds a solution for such paths in one iteration or it guarantees that the path is infeasible. In contrast, existing execution based approaches may require an unacceptably large number of iterations for relatively long paths because they consider only one input variable and one branch predicate at a time and use backtracking. When the branch conditions on a path are nonlinear functions of input variables, though it may take more then one iteration to derive a desired input, the set of constraints to be solved in each iteration is linear and is solved using Gaussian elimination. This makes our technique practical and suitable for automation.