Statistical fault localization (SFL) techniques use execution profiles and success/failure information from software executions, in conjunction with statistical inference, to automatically score program elements based on how likely they are to be faulty. SFL techniques typically employ one type of profile data: either coverage data, predicate outcomes, or variable values. Most SFL techniques actually measure correlation, not causation, between profile values and success/failure, and so they are subject to confounding bias that distorts the scores they produce. This paper presents a new SFL technique, named UniVal, that uses causal inference techniques and machine learning to integrate information about both predicate outcomes and variable values to more accurately estimate the true failure-causing effect of program statements. UniVal was empirically compared to several coverage-based, predicate-based, and value-based SFL techniques on 800 program versions with real faults.

This paper presents a new, flexible approach to automatically localizing faults in software, named CounterFault. It uses a form of causal inference called counterfactual prediction to predict the effect, on the success or failure of an execution Ex, of intervening at a statement s to set an assignment target A to a value a that is not actually assigned to A in Ex but that could be if s or Ex was modified. CounterFault generates this prediction without actually modifying s or Ex, by employing a very flexible non-parametric statistical or machine learning model (e.g., a random forest). CounterFault applies this basic idea to estimate, with minimal confounding bias, the average causal effects on program failures of different changes in the values assigned to program variables, and these estimates are then employed to derive suspiciousness scores, which are used to assist developers in localizing faults. This paper also reports on an empirical evaluation of CounterFault involving the widely used Defects4J evaluation framework, which contains real software faults, as well as several other Java numerical programs. CounterFault is compared empirically with two other value-based fault localization techniques and four of the best performing coverage-based techniques. The results indicate that CounterFault is more effective than the competing techniques.

Statistical Fault Localization (SFL) uses coverage profiles (or "spectra") collected from passing and failing tests, together with statistical metrics, which are typically composed of simple estimators, to identify which elements of a program are most likely to have caused observed failures. Previous SFL research has not thoroughly examined how the effectiveness of SFL metrics is related to the proportion of failures in test suites and related quantities. To address this issue, we studied the Defects4J benchmark suite of programs and test suites and found that if a test suite has very few failures, SFL performs poorly. To better understand this phenomenon, we investigated the precision of some statistical estimators of which SFL metrics are composed, as measured by their coefficients of variation. The precision of an embedded estimator, which depends on the dataset, was found to correlate with the effectiveness of a metric containing it: low precision is associated with poor effectiveness. Boosting precision by adding test cases was found to improve overall SFL effectiveness. We present our findings and discuss their implications for the evaluation and use of SFL metrics.

Statistical fault localization (SFL) techniques are commonly compared and evaluated using a measure known as "Rank Score" and its associated evaluation process. In the latter process each SFL technique under comparison is used to produce a list of program locations, ranked by their suspiciousness scores. Each technique then receives a Rank Score for each faulty program it is applied to, which is equal to the rank of the first faulty location in the corresponding list. The SFL technique whose average Rank Score is lowest is judged the best overall, based on the assumption that a programmer will examine each location in rank order until a fault is found. However, this assumption oversimplifies how an SFL technique would be used in practice. Programmers are likely to regard suspiciousness ranks as just one source of information among several that are relevant to locating faults. This paper provides a new evaluation approach using first-order Markov models of debugging processes, which can incorporate multiple additional kinds of information, e.g., about code locality, dependences, or even intuition. Our approach, RT_rank, scores SFL techniques based on the expected number of steps a programmer would take through the Markov model before reaching a faulty location. Unlike previous evaluation methods, HT_rank can compare techniques even when they produce fault localization reports differing in structure or information granularity. To illustrate the approach, we present a case study comparing two existing fault localization techniques that produce results varying in form and granularity.

Fault localization is a well-received technique for helping developers to identify faulty statements of a program. Research has shown that the coverages of faulty statements and its predecessors in program dependence graph are important for effective fault localization. However, app executions in Android split into segments in different components, i.e., methods, threads, and processes, posing challenges for traditional program dependence computation, and in turn rendering fault localization less effective. We present RunDroid, a tool for recovering the dynamic call graphs of app executions in Android, assisting existing tools for more precise program dependence computation. For each exectuion, RunDroid captures and recovers method calls from not only the application layer, but also between applications and the Android framework. Moreover, to deal with the widely adopted multi-threaded communications in Android applications, RunDroid also captures methods calls that are split among threads. Demo : https://github.com/MiJack/RunDroid Video : https://youtu.be/EM7TJbE-Oaw

Summary This paper presents NUMFL, a value‐based causal inference technique for localizing faults in numerical software. NUMFL combines causal and statistical analyses to characterize the causal effects of individual numerical expressions on output errors. Given value‐profiles for an expression's variables, NUMFL uses generalized propensity scores or covariate balancing propensity scores to reduce confounding bias caused by evaluation of other, faulty expressions. It estimates the average failure‐causing effect of an expression using statistical regression models fit within generalized propensity score or covariate balancing propensity score subclasses (strata). This paper also reports on an empirical evaluation of NUMFL involving components from four Java numerical libraries, in which it was compared with five alternative statistical fault localization metrics. The results indicate that NUMFL is more effective than competitive statistical fault localization techniques. The results also indicate NUMFL that works surprisingly well with data from failing runs alone. Copyright © 2016 John Wiley & Sons, Ltd.

In this paper, we investigate several coverage-based statistical fault localization metrics that have performed well in recent comparisons of many metrics, in order to better understand the properties of effective metrics. We first algebraically and probabilistically analyze the metrics to identify their key elements. Then we report on an empirical study we conducted to assess the relative importance of those elements. The results suggest that the most effective metrics contain a product of two terms: one that estimates the failure-causing effect of a program element (possibly with confounding bias) and one that weights the first term based on the evidence for the existence of faults in other program elements.

We present NUMFL, a value-based causal inference model for localizing faults in numerical software. NUMFL combines causal and statistical analyses to characterize the causal effects of individual numerical expressions on failures. Given value-profiles for an expression's variables, NUMFL uses generalized propensity scores (GPSs) to reduce confounding bias caused by evaluation of other, faulty expressions. It estimates the average failure-causing effect of an expression using quadratic regression models fit within GPS subclasses. We report on an evaluation of NUMFL with components from four Java numerical libraries, in which it was compared to five alternative statistical fault localization metrics. The results indicate that NUMFL is the most effective technique overall.

Adverse and anomalous (A&A) events are a serious concern in medical robots. We describe a system that can rapidly detect such events and predict their occurrence. As part of this system, we describe simulation, data collection and user interface tools we build for a robot for small animal biopsies. The data we collect consists of both the hardware state of the robot and variables in the software controller. We use this data to train dynamic Bayesian network models of the joint hardware-software state-space dynamics of the robot. Our empirical evaluation shows that (i) our models can accurately model normal behavior of the robot, (ii) they can rapidly detect anomalous behavior once it starts, (iii) they can accurately predict a future A&A event within a time window of it starting and (iv) the use of additional software variables beyond the hardware state of the robot is important in being able to detect and predict certain kinds of events.

Dependence analysis is a fundamental technique for program understanding and is widely used in software testing and debugging. However, there are a limited number of analysis tools available despite a wide range of research work in this field. In this paper, we present JavaPDG1, a static analyzer for Java bytecode, which is capable of producing various graphical representations such as the system dependence graph, procedure dependence graph, control flow graph and call graph. As a program-dependence-graph based analyzer, JavaPDG performs both intra- and inter-procedural dependence analysis, and enables researchers to apply a wide range of program analysis techniques that rely on dependence analysis. JavaPDG provides a graphical viewer to browse and analyze the various graphs and a convenient JSON based serialization format.

Recent studies have shown that use of causal inference techniques for reducing confounding bias improves the effectiveness of statistical fault localization (SFL) at the level of program statements. However, with very large programs and test suites, the overhead of statement-level causal SFL may be excessive. Moreover cost evaluations of statement-level SFL techniques generally are based on a questionable assumption-that software developers can consistently recognize faults when examining statements in isolation. To address these issues, we propose and evaluate a novel method-level SFL technique called MFL, which is based on causal inference methodology. In addition to reframing SFL at the method level, our technique incorporates a new algorithm for selecting covariates to use in adjusting for confounding bias. This algorithm attempts to ensure that such covariates satisfy the conditional exchangeability and positivity properties required for identifying causal effects with observational data. We present empirical results indicating that our approach is more effective than four method-level versions of well-known SFL techniques and that our confounder selection algorithm is superior to two alternatives.

Commercial static program analysis tools can be used to detect many defects that are common across applications. However, such tools currently have limited ability to reveal defects that are specific to individual projects, unless specialized checkers are devised and implemented by tool users. Developers do not typically exploit this capability. By contrast, defect mining tools developed by researchers can discover project-specific defects, but they require specialized expertise to employ and they may not be robust enough for general use. We present a hybrid approach in which a sophisticated dependence-based rule mining tool is used to discover project-specific programming rules, which are then transformed automatically into checkers that a commercial static analysis tool can run against a code base to reveal defects. We also present the results of an empirical study in which this approach was applied successfully to two large industrial code bases. Finally, we analyze the potential implications of this approach for software development practice.

In the context of field testing (operational testing) of software, we address the problem of balancing the potential reduction in failure risk that developers may achieve by reviewing captured test executions to identify failures (and by successfully debugging their causes) against the cost of reviewing the tests. To achieve a desirable balance, we propose a cost-sensitive active learning strategy. Our approach guides developers in selecting a sample of test executions to review and label, and it calls for them to profile execution dynamics and characterize the symptoms and relative severity levels of failures. Profiles, labels, failure symptoms, and severity levels are used by the active learner to construct and refine a mapping between examined and unexamined tests, on one hand, and possible defects, on the other hand. This mapping is used together with estimates of test review costs to guide the selection of additional tests. We evaluate our approach on three subject programs and show that it (1) produces reasonable predictions of risk reduction and (2) significantly improves severity-weighted reliability for each subject program, with relatively low developer effort.

SUMMARY This paper presents a novel approach to discovering implicit programming rules and rule violations in a code base, which integrates static interprocedural analysis and graph mining techniques to identify both function‐call ordering rules and conditional rules that check input parameters or return values of functions. The approach discovers rules even when rule instances cross function boundaries. Rules are modeled as graph minors of dependence graphs augmented with edges indicating shared data dependences. The approach employs two innovative algorithms: a greedy one for mining maximal frequent minors from a set of interprocedural dependence spheres and a heuristic minor‐matching algorithm for discovering rule violations. We evaluated our approach on the latest versions of three applications: net‐snmp, openssl, and the Apache HTTP server. It detected 62 new bugs (24 involving rules with interprocedural instances), 35 of which have been confirmed and fixed recently by developers based on our reports. Copyright © 2011 John Wiley & Sons, Ltd.

This paper presents a new, statistical approach to evaluating software products that transform complex inputs into complex outputs. This approach, called multistage stratified input/output (MSIO) sampling, combines automatic clustering of multidimensional I/O data with multistage sampling and manual examination of data elements, in order to accurately and economically estimate summary measures of output data quality. We report results of two case studies in which MSIO sampling was successfully applied to evaluating complex graphical outputs.

Dynamic program dependences are recognized as important factors in software debugging because they contribute to triggering the effects of faults and propagating the effects to a program's output. The effects of dynamic dependences also produce significant confounding bias when statistically estimating the causal effect of a statement on the occurrence of program failures, which leads to poor fault localization results. This paper presents a novel causal-inference technique for fault localization that accounts for the effects of dynamic data and control dependences and thus, significantly reduces confounding bias during fault localization. The technique employs a new dependence-based causal model together with matching of test executions based on their dynamic dependences. The paper also presents empirical results indicating that the new technique performs significantly better than existing statistical fault-localization techniques as well as our previous fault localization technique based on causal-inference methodology.

Previous work has shown that application of graph mining techniques to system dependence graphs improves the precision of automatic defect discovery by revealing subgraphs corresponding to implicit programming rules and to rule violations. However, developers must still confirm, edit, or discard reported rules and violations, which is both costly and error-prone. In order to reduce developer effort and further improve precision, we investigate the use of supervised learning models for classifying and ranking rule and violation subgraphs. In particular, we present and evaluate logistic regression models for rules and violations, respectively, which are based on general dependence-graph features. Our empirical results indicate that (i) use of these models can significantly improve the precision and recall of defect discovery, and (ii) our approach is superior to existing heuristic approaches to rule and violation ranking and to an existing static-warning classifier, and (iii) accurate models can be learned using only a few labeled examples.

We present a powerful and efficient approach to the problem of propagating a bug fix to all the locations in a code base to which it applies. Our approach represents bug and fix patterns as subgraphs of a system dependence graph, and it employs a fast, index-based subgraph matching algorithm to discover unfixed bug-pattern instances remaining in a code base. We have also developed a graphical tool to help programmers specify bug patterns and fix patterns easily. We evaluated our approach by applying it to bug fixes in four large open-source projects. The results indicate that the approach exhibits good recall and precision and excellent efficiency.

This paper investigates the application of causal inference methodology for observational studies to software fault localization based on test outcomes and profiles. This methodology combines statistical techniques for counterfactual inference with causal graphical models to obtain causal-effect estimates that are not subject to severe confounding bias. The methodology applies Pearl's Back-Door Criterion to program dependence graphs to justify a linear model for estimating the causal effect of covering a given statement on the occurrence of failures. The paper also presents the analysis of several proposed-fault localization metrics and their relationships to our causal estimator. Finally, the paper presents empirical results demonstrating that our model significantly improves the effectiveness of fault localization.

This paper presents an innovative model of a program's internal behavior over a set of test inputs, called the probabilistic program dependence graph (PPDG), which facilitates probabilistic analysis and reasoning about uncertain program behavior, particularly that associated with faults. The PPDG construction augments the structural dependences represented by a program dependence graph with estimates of statistical dependences between node states, which are computed from the test set. The PPDG is based on the established framework of probabilistic graphical models, which are used widely in a variety of applications. This paper presents algorithms for constructing PPDGs and applying them to fault diagnosis. The paper also presents preliminary evidence indicating that a PPDG-based fault localization technique compares favorably with existing techniques. The paper also presents evidence indicating that PPDGs can be useful for fault comprehension.

Dynamic information flow analysis (DIFA) was devised to enable the flow of information among variables in an executing program to be monitored and possibly regulated. It is related to techniques like dynamic slicing and dynamic impact analysis . To better understand the basis for DIFA, we conducted an empirical study in which we measured the strength of information flows identified by DIFA, using information theoretic and correlation-based methods. The results indicate that in most cases the occurrence of a chain of dynamic program dependences between two variables does not indicate a measurable information flow between them. We also explored the relationship between the strength of an information flow and the length of the corresponding dependence chain, and we obtained results indicating that no consistent relationship exists between the length of an information flow and its strength. Finally, we investigated whether data dependence and control dependence makes equal or unequal contributions to flow strength. The results indicate that flows due to data dependences alone are stronger, on average, than flows due to control dependences alone. We present the details of our study and consider the implications of the results for applications of DIFA and related techniques.

We present empirical results indicating that when programmers fix bugs, they often fail to propagate the fixes to all of the locations in a code base where they are applicable, thereby leaving instances of the bugs in the code. We propose a practical approach to help programmers to propagate many bug fixes completely. This entails first extracting a programming rule from a bug fix, in the form of a graph minor of an enhanced procedure dependence graph. Our approach assists the programmer in specifying rules by automatically matching simple rule templates; the programmer may also edit rules or compose them from scratch. A graph matching algorithm for detecting rule violations is then used to locate the places in the code base where the bug fix is applicable. Our approach does not require that rules occur repeatedly in the code base. We present empirical results indicating that the approach nevertheless exhibits good precision.

This paper presents an innovative model of a program's internal behavior over a set of test inputs, called the probabilistic program dependence graph (PPDG), that facilitates probabilistic analysis and reasoning about uncertain program behavior, particularly that associated with faults. The PPDG is an augmentation of the structural dependences represented by a program dependence graph with estimates of statistical dependences between node states, which are computed from the test set. The PPDG is based on the established framework of probabilistic graphical models, which are widely used in applications such as medical diagnosis. This paper presents algorithms for constructing PPDGs and applying the PPDG to fault diagnosis. This paper also presents preliminary evidence indicating that PPDGs can facilitate fault localization and fault comprehension.

Neglected conditions are an important but difficult-to-find class of software defects. This paper presents a novel approach to revealing neglected conditions that integrates static program analysis and advanced data mining techniques to discover implicit conditional rules in a code base and to discover rule violations that indicate neglected conditions. The approach requires the user to indicate minimal constraints on the context of the rules to be sought, rather than specific rule templates. To permit this generality, rules are modeled as graph minors of enhanced procedure dependence graphs (EPDGs), in which control and data dependence edges are augmented by edges representing shared data dependences. A heuristic maximal frequent subgraph mining algorithm is used to extract candidate rules from EPDGs, and a heuristic graph matching algorithm is used to identify rule violations. We also report the results of an empirical study in which the approach was applied to four open source projects (openssl, make, procmail, amaya). These results indicate that the approach is effective and reasonably efficient.

Operational or "beta" testing of software has a number of benefits for software vendors and has become common industry practice. However, ordinary users are more likely to overlook or misreport software problems than experienced software testers are. To compensate for this shortcoming, we present a technique called corroboration-based filtering for corroborating user assessments of individual operational executions for which audit information has been captured for possible offline review. Independent assessments concerning similar executions are pooled by automatically clustering together executions with similar execution profiles. Executions are chosen for review based on their user assessments, the size of the cluster each execution belongs to, and whether the cluster has already been confirmed by developers to contain an actual failure. We explain the rationale for this technique, analyze it probabilistically, and present the results of empirically comparing it to alternative techniques.

Neglected conditions are an important but difficult-to-find class of software defects. This paper presents a novel approach to revealing neglected conditions that integrates static program analysis and advanced data mining techniques to discover implicit conditional rules in a code base and to discover rule violations that indicate neglected conditions. The approach requires the user to indicate minimal constraints on the context of the rules to be sought, rather than specific rule templates. To permit this generality, rules are modeled as graph minors of program dependence graphs, and both frequent itemset mining and frequent subgraph mining algorithms are employed to identify candidate rules. We report the results of an empirical evaluation of the approach in which it was used to discover conditional rules and neglected conditions in ~25,000 lines of source code.

Some software defects trigger failures only when certain local or nonlocal program interactions occur. Such interactions are modeled by the closely related concepts of information flows, program dependences, and program slices. The latter concepts underlie a 78variety of proposed test data adequacy criteria, and they form a potentially important basis for filtering existing test cases. We report the results of an empirical study of several test case filtering techniques that are based on exercising information flows. Both coverage-based and profile-distribution-based filtering techniques are considered. They are compared to filtering techniques based on exercising simpler program elements, such as basic blocks, branches, function calls, and call pairs, with respect to their effectiveness for revealing defects.

Dynamic analysis techniques reason over program executions and deal with data produced at program execution time. Dynamic analysis and static analysis techniques complement each other. Hence, a key focus of the workshop is dynamic analysis of software systems with an emphasis on research that integrates static and dynamic analyses.

Forward computation of dynamic slices is necessary to support interactive debugging and online analysis of long running programs. However, the overhead of existing forward computing algorithms limits their use to non-processing intensive applications. Recent empirical studies have shown that slices tend to reoccur often during execution. This paper presents a new forward computing algorithm for dynamic slicing, which is based on the stronger assumption that the same set union operations need to be performed repeatedly during slice computation. We present the results of an empirical study contrasting the performance of our new algorithm to the performance of a basic forward computing algorithm that unconditionally merges slices influencing an executing statement. The results indicate that the new algorithm is substantially faster than the basic algorithm and often requires significantly less memory

Some software defects trigger failures only when certain complex information flows occur within the software. Profiling and analyzing such flows therefore provides a potentially important basis for filtering test cases. We report the results of an empirical evaluation of several test case filtering techniques that are based on exercising complex information flows. Both coverage-based and profile-distribution-based filtering techniques are considered. They are compared to filtering techniques based on exercising basic blocks, branches, function calls, and def-use pairs, with respect to their effectiveness for revealing defects.

Multidimensional scaling (MDS) is a technique for visualizing multidimensional data points as a 2D scatter plot. It can be applied to execution profiles of software to reveal how similar executions are to one another. This is useful for certain software engineering applications, which require accurate representation of small dissimilarities and nearest neighbor relationships. However, the high-dimensionality of profiles can cause MDS techniques to represent small dissimilarities poorly. We evaluate several variants of MDS on large sets of profiles, to see which techniques produce the most accurate displays. These include four previously proposed techniques -classical scaling followed by iterative majorization, energy minimization, ordinal AIDS, and cluster differences scaling - and two techniques of our invention - hierarchical MDS and sparse region scaling. The results suggest that each technique except ordinal MDS can significantly improve the representation of small dissimilarities between program executions and that hierarchical MDS and sparse region scaling perform best overall

This paper describes an automated tool called Dex (difference extractor) for analyzing syntactic and semantic changes in large C-language code bases. It is applied to patches obtained from a source code repository, each of which comprises the code changes made to accomplish a particular task. Dex produces summary statistics characterizing these changes for all of the patches that are analyzed. Dex applies a graph differencing algorithm to abstract semantic graphs (ASGs) representing each version. The differences are then analyzed to identify higher-level program changes. We describe the design of Dex, its potential applications, and the results of applying it to analyze bug fixes from the Apache and GCC projects. The results include detailed information about the nature and frequency of missing condition defects in these projects.

Recent research has addressed the problem of providing automated assistance to software developers in classifying reported instances of software failures so that failures with the same cause are grouped together. In this paper, two new tree-based techniques are presented for refining an initial classification of failures. One of these techniques is based on the use of dendrograms, which are rooted trees used to represent the results of hierarchical cluster analysis. The second technique employs a classification tree constructed to recognize failed executions. With both techniques, the tree representation is used to guide the refinement process. We also report the results of experimentally evaluating these techniques on several subject programs.

A new approach to dynamic information flow analysis is presented that can be used to detect and debug insecure flows in programs. It can be applied offline to validate and debug a program against an information flow policy, or, when fast response is not critical, it can be applied online to prevent illegal flows in deployed programs. Since dynamic analysis alone is inherently unable to detect implicit information flows, our approach incorporates a static preprocessing phase that permits detection of most implicit flows at runtime, in addition to explicit ones. To support interactive debugging of insecure flows, it also incorporates a new forward computing algorithm for dynamic slicing, which is more precise than previous forward computing algorithms and is not restricted to programs with structured control flow. A prototype tool implementing the proposed approach has been developed for Java byte code programs. Case studies in which this tool was applied to several subject programs are described.

This paper proposes automated support for classifying reported software failures in order to facilitate prioritizing them and diagnosing their causes. A classification strategy is presented that involves the use of supervised and unsupervised pattern classification and multivariate visualization. These techniques are applied to profiles of failed executions in order to group together failures with the same or similar causes. The resulting classification is then used to assess the frequency and severity of failures caused by particular defects and to help diagnose those defects. The results of applying the proposed classification strategy to failures of three large subject programs are reported These results indicate that the strategy can be effective.

This paper presents an empirical comparison of four different techniques for filtering large test suites: test suite minimization, prioritization by additional coverage, cluster filtering with one-per-cluster sampling, and failure pursuit sampling. The first two techniques are based on selecting subsets that maximize code coverage as quickly as possible, while the latter two are based on analyzing the distribution of the tests' execution profiles. These techniques were compared with data sets obtained from three large subject programs: the GCC, Jikes, and javac compilers. The results indicate that distribution-based techniques can be as efficient or more efficient for revealing defects than coverage-based techniques, but that the two kinds of techniques are also complementary in the sense that they find different defects. Accordingly, some simple combinations of these techniques were evaluated for use in test case prioritization. The results indicate that these techniques can create more efficient prioritizations than those generated using prioritization by additional coverage.

We experimentally evaluate the effectiveness of using cluster analysis of execution profiles to find failures among the executions induced by a set of potential test cases. We compare several filtering procedures for selecting executions to evaluate for conformance to requirements. Each filtering procedure involves a choice of a sampling strategy and a clustering metric. The results suggest that filtering procedures based on clustering are more effective than simple random sampling for identifying failures in populations of operational executions, with adaptive sampling from clusters being the most effective sampling strategy. The results also suggest that clustering metrics that give extra weight to industrial profile features are most effective. Scatter plots of execution populations, produced by multidimensional scaling, are used to provide intuition for these results.

Observation-based testing calls for analyzing profiles of executions induced by potential test cases, in order to select a subset of executions to be checked for conformance to requirements. A family of techniques for selecting such a subset is evaluated experimentally. These techniques employ automatic cluster analysis to partition executions, and they use various sampling techniques to select executions from clusters. The experimental results support the hypothesis that with appropriate profiling, failures often have unusual profiles that are revealed by cluster analysis. The results also suggest that failures often form small clusters or chains in sparsely-populated areas of the profile space. A form of adaptive sampling called failure-pursuit sampling is proposed for revealing failures in such regions, and this sampling method is evaluated experimentally. The results suggest that failure-pursuit sampling is effective.

We explore the use of multivariate visualization techniques to support a new approach to test data selection, called observation-based testing. Applications of multivariate visualization are described, including: evaluating and improving synthetic tests; filtering regression test suites; filtering captured operational executions; comparing test suites; and assessing bug reports. These applications are illustrated by the use of correspondence analysis to analyze test inputs for the GNU GCC compiler.

We describe the design of jRapture: a tool for capturing and replaying Java program executions in the field. jRapture works with Java binaries (byte code) and any compliant implementation of the Java virtual machine. It employs a lightweight, transparent capture process that permits unobtrusive capture of a Java programs executions. jRapture captures interactions between a Java program and the system, including GUI, file, and console inputs, among other types, and on replay it presents each thread with exactly the same input sequence it saw during capture. In addition, jRapture has a profiling interface that permits a Java program to be instrumented for profiling ó after its executions have been captured. Using an XML-based profiling specification language a tester can specify various forms of profiling to be carried out during replay.

Summarizes results of a five-year, multi-disciplinary, university-industry collaborative effort investigating design issues in agile manufacturing. The focus of the project is specifically on light mechanical assembly, with the demand that new assembly tasks be implementable quickly, economically, and effectively. Key to achieving these goals is the ease of equipment and software reuse. Design choices for both hardware and software must strike a balance between the inflexibility of special-purpose designs and the impracticality of overly general designs. We review both our physical and software design choices and make recommendations for the design of agile manufacturing systems.

Assembly tasks involving large position uncertainties are unsuitable for use of position-controlled robots. To automate such tasks, the assembly system must be responsive to contact forces. Issues in addressing force-responsive automated assembly include contact stability, the degree of force responsiveness required for success, the speed of a successful implementation, and the means to program a force-responsive system to perform a given assembly task. We examine these issues for robotic assembly in the context of automotive transmission components. We report on an impedance-based low-level algorithm and its interface to higher-level strategies that exhibits gentle, fast and reliable assembly of our example components.

A new approach to software reliability estimation is presented that combines operational testing with stratified sampling in order to reduce the number of program executions that must be checked manually for conformance to requirements. Automatic cluster analysis is applied to execution profiles in order to stratify captured operational executions. Experimental results are reported that suggest this approach can significantly reduce the cost of estimating reliability.

The need to rapidly reconfigure an agile manufacturing system makes it especially difficult to test the system's control software. Although thorough testing is essential for system reliability, the time available for testing may be short. With a simulator, however, the software can be developed and tested independently from the actual workcell, while production continues or the workcell is reconfigured for the next target product. To facilitate testing of agile manufacturing software, a 3D graphical simulator has been developed at Case Western Reserve University. It permits workcell control software to be tested with a virtual workcell, which exhibits much of the behavior of the real workcell. The simulator has been extensively used in the Agile Manufacturing Project at Case Western Reserve University (CWRU), and most of the actual control software was tested and debugged with it. Considerable time and effort have been saved by simulating various workcell scenarios, some of which are difficult to create in a real workcell, e.g., device failure. In this paper, the architecture of the graphical simulator is described, and a framework for virtual testing is presented. Actual errors found during virtual testing are also described.

The flexibility required of an agile manufacturing system must be achieved largely through computer software. The system's control software must be adaptable to new products and to new system components without becoming unreliable or difficult to maintain. This requires designing the software specifically to facilitate future changes. As part of the Agile Manufacturing Project at Case Western Reserve University, we have developed a software architecture for control of an agile manufacturing workcell, and we have demonstrated its flexibility with rapid changeover and introduction of new products. In this paper, we describe the requirements for agile manufacturing software and how our software architecture addresses them.

An agile workcell has been developed for light mechanical assembly in collaboration with industrial sponsors. The workcell includes multiple Adept robots, a Bosch conveyor system, multiple flexible parts feeders at each robot's workstation, CCD cameras for parts feeding and hardware registration, and a dual VMEbus control system. Our flexible pairs feeder design uses multiple conveyors to singulate the parts and machine vision to locate them. Specialized hardware is encapsulated on modular grippers and modular worktables which can be quickly interchanged for assembly of different products. Object-oriented software (C++) running under VxWorks, a real-time operating system, is used for workcell control. An agile software architecture was developed for rapid introduction of new assemblies through code re-use. A simulation of the workcell was developed so that controller software could be written and tested off-line, enabling the rapid introduction of new products.

This paper introduces a design for agile manufacturing workcells intended for light mechanical assembly of products made from similar components (i.e. parts families). We define agile manufacturing as the ability to accomplish rapid changeover from the assembly of one product to the assembly of another product. Rapid hardware changeover is made possible through the use of robots, flexible part feeders, modular grippers and modular assembly hardware. The flexible feeders rely on belt feeding and binary computer vision for Dose estimation. This has a distinct advantage over non-flexible feeding schemes such as bowl feeders which require considerable adjustment to changeover from one part to another. Rapid software changeover is being facilitated by the use of a real-time, object-oriented software environment, modular software, graphical simulations for off-line software development, and an innovative dual VMEbus controller architecture. These agile features permit new products to be introduced with minimal downtime and system reconfiguration.

We present a new approach to reducing the manual labor required to estimate software reliability. It combines the ideas of partition testing methods with those of stratified sampling to reduce the sample size necessary to estimate reliability with a given degree of precision. Program executions are stratified by using automatic cluster analysis to group those with similar features. We describe the conditions under which stratification is effective for estimating software reliability, and we present preliminary experimental results suggesting that our approach may work well in practice.

A new method, called behavior sampling , is proposed for automated retrieval of reusable components from software libraries. Behavior sampling exploits the property of software that distinguished it from other forms of test: executability. Basic behavior sampling identifies relevant routines by executing candidates on a searcher-supplied sample of operational inputs and by comparing their output to output provided by the searcher. The probabilistic basis for behavior sampling is described, and experimental results are reported that suggest that basic behavior sampling exhibits high precision when used with small samples. Extensions to basic behavior sampling are proposed to improve its recall and to make it applicable to the retrieval of abstract data types and object classes.

Article Free Access Share on Behavior sampling: a technique for automated retrieval of reusable components Authors: Andy Podgurski View Profile , Lynn Pierce View Profile Authors Info & Claims ICSE '92: Proceedings of the 14th international conference on Software engineeringJune 1992 Pages 349–361https://doi.org/10.1145/143062.143152Published:01 June 1992Publication History 14citation415DownloadsMetricsTotal Citations14Total Downloads415Last 12 Months20Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

A formal, general model of program dependences is presented and used to evaluate several dependence-based software testing, debugging, and maintenance techniques. Two generalizations of control and data flow dependence, called weak and strong syntactic dependence, are introduced and related to a concept called semantic dependence. Semantic dependence models the ability of a program statement to affect the execution behavior of other statements. It is shown that weak syntactic dependence is a necessary but not sufficient condition for semantic dependence and that strong syntactic dependence is necessary but not sufficient condition for a restricted form of semantic dependence that is finitely demonstrated. These results are used to support some proposed uses of program dependences, to controvert others, and to suggest new uses.>

The authors report on the results of their evaluation of path-selection criteria based on data-flow relationships. They show how these criteria relate to each other, thereby demonstrating some of their strengths and weaknesses. A subsumption hierarchy showing their relationship is presented. It is shown that one of the major weaknesses of all the criteria is that they are based solely on syntactic information and do not consider semantic issues such as infeasible paths. The authors discuss the infeasible-path problem as well as other issues that must be considered in order to evaluate these criteria more meaningfully and to formulate a more effective path-selection criterion.>