Given a test set T to test a program P, there are at least two attributes of T that determine its fault detection effectiveness. One attribute is the size of T measured as the number of test cases in T. Another attribute is the code coverage measured when P is executed on all elements of T. The fault detection effectiveness of T is the ratio of the number of faults guaranteed to result in program failure when P is executed on T to the total number of faults present in P. An empirical study was conducted to determine the relative importance of the size and coverage attributes in affecting the fault detection effectiveness of a randomly selected test set for some program P. Results from this study indicate that as the size of a test set is reduced, while the code coverage is kept constant, there is little or no reduction in the fault detection effectiveness of the new test set so generated. For the study reported, of the two attributes mentioned above, the code coverage attribute of a test set is more important than its size attribute. © 1998 John Wiley & Sons, Ltd.

The purpose of regression testing is to ensure that changes made to software, such as adding new features or modifying existing features, have not adversely affected features of the software that should not change. Regression testing is usually performed by running some, or all, of the test cases created to test modifications in previous versions of the software. Many techniques have been reported on how to select regression tests so that the number of test cases does not grow too large as the software evolves. Our proposed hybrid technique combines modification, minimization and prioritization-based selection using a list of source code changes and the execution traces from test cases run on previous versions. This technique seeks to identify a representative subset of all test cases that may result in different output behavior on the new software version. We report our experience with a tool called ATAC (Automatic Testing Analysis tool in C) which implements this technique.

Article Free AccessEffect of test set minimization on fault detection effectiveness Authors: W. Eric Wong Bell Communications Research, Morristown, NJ Bell Communications Research, Morristown, NJView Profile , Joseph R. Horgan Bell Communications Research, Morristown, NJ Bell Communications Research, Morristown, NJView Profile , Saul London Bell Communications Research, Morristown, NJ Bell Communications Research, Morristown, NJView Profile , Aditya P. Mathur Software Engineering Research Center, Department of Computer Sciences, Purdue University, W. Lafayette, IN Software Engineering Research Center, Department of Computer Sciences, Purdue University, W. Lafayette, INView Profile Authors Info & Claims ICSE '95: Proceedings of the 17th international conference on Software engineeringApril 1995Pages 41–50https://doi.org/10.1145/225014.225018Published:23 April 1995Publication History 135citation967DownloadsMetricsTotal Citations135Total Downloads967Last 12 Months94Last 6 weeks14 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Finding a fault in a program is a complex process which involves understanding the program's purpose, structure, semantics, and the relevant characteristics of failure producing tests. We describe a tool which supports execution slicing and dicing based on test cases. We report the results of an experiment that uses heuristic techniques in fault localization.

Size and code coverage are two important attributes that characterize a set of tests. When a program P is executed on elements of a test set T, we can observe the fault-detecting capacity of T for P. We can also observe the degree to which T induces code coverage on P according to some coverage criterion. We would like to know whether it is the size of T or the coverage of T on P which determines the fault detection effectiveness (FDE) of T for P. We found that there is little or no reduction in the FDE of a test set when its size is reduced while the all-uses coverage is kept constant. These data suggest, indirectly, that coverage is more correlated than the size with the FDE. To further investigate this suggestion, we report an empirical study to compare the statistical correlation between (1) FDE and coverage, and (2) FDE and the size. Results from our experiments indicate that the correlation between FDE and block coverage is higher than that between FDE and size.>

The purpose of regression testing is to ensure that bug fixes and new functionality introduced in a new version of a software do not adversely affect the correct functionality inherited from the previous version. Efficient methods of selecting small subsets of regression test sets that can be used to ensure correct functionality are explored.>

We describe a software testing and analysis tool, called ATAC (Automatic Test Analysis for C), which is developed as a research instrument at Bellcore to measure the effectiveness of testing data. The design, functionality, and usage of ATAC are presented in this paper. Furthermore, to demonstrate the capability and applicability of ATAC, we obtain the 12 program versions of a critical industrial application developed in a recent university/industry N-Version Software project, and use the ATAC tool to analyze and compare coverage of the testing conducted in the program versions. Preliminary results from this investigation show that ATAC could be a powerful testing tool to provide testing metrics and quality control guidance for the certification of high quality software components or systems. It can also assist software reliability researchers and practitioners in searching for the missing link between structure-based testing schemes and software reliability.