Some software defects trigger failures only when certain local or nonlocal program interactions occur. Such interactions are modeled by the closely related concepts of information flows, program dependences, and program slices. The latter concepts underlie a 78variety of proposed test data adequacy criteria, and they form a potentially important basis for filtering existing test cases. We report the results of an empirical study of several test case filtering techniques that are based on exercising information flows. Both coverage-based and profile-distribution-based filtering techniques are considered. They are compared to filtering techniques based on exercising simpler program elements, such as basic blocks, branches, function calls, and call pairs, with respect to their effectiveness for revealing defects.

Streaming is one of the key 3G mobile multimedia services provided by network operators. Traditional streaming applications need to be redesigned in order to work well over wireless. 3GPP has standardized streaming services for Release 4, 5 and 6 specifications. The last release overcomes the technical challenges posed by wireless and best effort networks. This paper describes the solution standardized in Release 6 for adaptive streaming.

Some software defects trigger failures only when certain complex information flows occur within the software. Profiling and analyzing such flows therefore provides a potentially important basis for filtering test cases. We report the results of an empirical evaluation of several test case filtering techniques that are based on exercising complex information flows. Both coverage-based and profile-distribution-based filtering techniques are considered. They are compared to filtering techniques based on exercising basic blocks, branches, function calls, and def-use pairs, with respect to their effectiveness for revealing defects.

Multidimensional scaling (MDS) is a technique for visualizing multidimensional data points as a 2D scatter plot. It can be applied to execution profiles of software to reveal how similar executions are to one another. This is useful for certain software engineering applications, which require accurate representation of small dissimilarities and nearest neighbor relationships. However, the high-dimensionality of profiles can cause MDS techniques to represent small dissimilarities poorly. We evaluate several variants of MDS on large sets of profiles, to see which techniques produce the most accurate displays. These include four previously proposed techniques -classical scaling followed by iterative majorization, energy minimization, ordinal AIDS, and cluster differences scaling - and two techniques of our invention - hierarchical MDS and sparse region scaling. The results suggest that each technique except ordinal MDS can significantly improve the representation of small dissimilarities between program executions and that hierarchical MDS and sparse region scaling perform best overall

As the adoption of 2.5G and 3G systems grows, multimedia streaming is one of the services that operators will increasingly seek to provide. The multimedia streaming solutions that have successfully been deployed commercially on the Internet in the past are all based on proprietary technologies. However, existing streaming applications need to be considerably rethought to work well over wireless. The 3GPP standardization body has standardized streaming services for Release 4, 5 and 6 specifications. The Release 6 advanced streaming service overcomes the technical challenges posed by wireless and best effort networks. A standard solution for advanced multimedia streaming will ultimately drive adoption by operators and users. The paper describes the solution standardized in 3GPP(2) for adaptive streaming.

This paper describes an automated tool called Dex (difference extractor) for analyzing syntactic and semantic changes in large C-language code bases. It is applied to patches obtained from a source code repository, each of which comprises the code changes made to accomplish a particular task. Dex produces summary statistics characterizing these changes for all of the patches that are analyzed. Dex applies a graph differencing algorithm to abstract semantic graphs (ASGs) representing each version. The differences are then analyzed to identify higher-level program changes. We describe the design of Dex, its potential applications, and the results of applying it to analyze bug fixes from the Apache and GCC projects. The results include detailed information about the nature and frequency of missing condition defects in these projects.

Recent research has addressed the problem of providing automated assistance to software developers in classifying reported instances of software failures so that failures with the same cause are grouped together. In this paper, two new tree-based techniques are presented for refining an initial classification of failures. One of these techniques is based on the use of dendrograms, which are rooted trees used to represent the results of hierarchical cluster analysis. The second technique employs a classification tree constructed to recognize failed executions. With both techniques, the tree representation is used to guide the refinement process. We also report the results of experimentally evaluating these techniques on several subject programs.

A new approach to dynamic information flow analysis is presented that can be used to detect and debug insecure flows in programs. It can be applied offline to validate and debug a program against an information flow policy, or, when fast response is not critical, it can be applied online to prevent illegal flows in deployed programs. Since dynamic analysis alone is inherently unable to detect implicit information flows, our approach incorporates a static preprocessing phase that permits detection of most implicit flows at runtime, in addition to explicit ones. To support interactive debugging of insecure flows, it also incorporates a new forward computing algorithm for dynamic slicing, which is more precise than previous forward computing algorithms and is not restricted to programs with structured control flow. A prototype tool implementing the proposed approach has been developed for Java byte code programs. Case studies in which this tool was applied to several subject programs are described.

This paper proposes automated support for classifying reported software failures in order to facilitate prioritizing them and diagnosing their causes. A classification strategy is presented that involves the use of supervised and unsupervised pattern classification and multivariate visualization. These techniques are applied to profiles of failed executions in order to group together failures with the same or similar causes. The resulting classification is then used to assess the frequency and severity of failures caused by particular defects and to help diagnose those defects. The results of applying the proposed classification strategy to failures of three large subject programs are reported These results indicate that the strategy can be effective.

This paper presents an empirical comparison of four different techniques for filtering large test suites: test suite minimization, prioritization by additional coverage, cluster filtering with one-per-cluster sampling, and failure pursuit sampling. The first two techniques are based on selecting subsets that maximize code coverage as quickly as possible, while the latter two are based on analyzing the distribution of the tests' execution profiles. These techniques were compared with data sets obtained from three large subject programs: the GCC, Jikes, and javac compilers. The results indicate that distribution-based techniques can be as efficient or more efficient for revealing defects than coverage-based techniques, but that the two kinds of techniques are also complementary in the sense that they find different defects. Accordingly, some simple combinations of these techniques were evaluated for use in test case prioritization. The results indicate that these techniques can create more efficient prioritizations than those generated using prioritization by additional coverage.

We experimentally evaluate the effectiveness of using cluster analysis of execution profiles to find failures among the executions induced by a set of potential test cases. We compare several filtering procedures for selecting executions to evaluate for conformance to requirements. Each filtering procedure involves a choice of a sampling strategy and a clustering metric. The results suggest that filtering procedures based on clustering are more effective than simple random sampling for identifying failures in populations of operational executions, with adaptive sampling from clusters being the most effective sampling strategy. The results also suggest that clustering metrics that give extra weight to industrial profile features are most effective. Scatter plots of execution populations, produced by multidimensional scaling, are used to provide intuition for these results.

Observation-based testing calls for analyzing profiles of executions induced by potential test cases, in order to select a subset of executions to be checked for conformance to requirements. A family of techniques for selecting such a subset is evaluated experimentally. These techniques employ automatic cluster analysis to partition executions, and they use various sampling techniques to select executions from clusters. The experimental results support the hypothesis that with appropriate profiling, failures often have unusual profiles that are revealed by cluster analysis. The results also suggest that failures often form small clusters or chains in sparsely-populated areas of the profile space. A form of adaptive sampling called failure-pursuit sampling is proposed for revealing failures in such regions, and this sampling method is evaluated experimentally. The results suggest that failure-pursuit sampling is effective.

We explore the use of multivariate visualization techniques to support a new approach to test data selection, called observation-based testing. Applications of multivariate visualization are described, including: evaluating and improving synthetic tests; filtering regression test suites; filtering captured operational executions; comparing test suites; and assessing bug reports. These applications are illustrated by the use of correspondence analysis to analyze test inputs for the GNU GCC compiler.