This review examines recent feature-based transfer learning techniques for Cross-Project Software Defect Prediction. We summarize representative approaches in five categories-feature selection, feature mapping/alignment, deep/adversarial learning, semantics-enhanced transfer, and hybrid/multi-source designs-and discuss their reported effectiveness across common benchmark datasets. The review highlights trade-offs between predictive accuracy, computational cost, and model interpretability, and concludes with open challenges and directions for future work.

The convergence of autonomous driving and deep learning technologies has brought unprecedented convenience to future mobility while also introducing new security challenges. Adversarial attacks can exploit minor perturbations to deceive vehicle perception and decision-making processes, thereby posing potential dangers to both passengers and pedestrians. Although considerable progress has been made in developing adversarial detection and defense mechanisms, significant challenges remain, including high computational overhead, limited real-time performance, incomplete multi-modal integration, and insufficient understanding of black-box attacks and cross-scenario transfer. To comprehensively enhance the security and robustness of autonomous driving systems, it is necessary to further expand the data and model scales of adversarial examples, promote multi-modal fusion, improve the generalizability and real-time performance of adversarial defenses, and conduct additional validation under realistic and complex environments. Based on these considerations, this paper systematically reviews recent advances and gaps in adversarial research for autonomous driving. Furthermore, it explore future research directions from the perspectives of multi-modal fusion, dataset scale expansion, black-box defense, and the emergent role of large language models.

Effective software defect prediction (SDP) is important for software quality assurance. Numerous advanced SDP methods have been proposed recently. However, how to consider the task correlations and achieve multi-objective SDP accurately and efficiently still remains to be further explored. In this paper, we propose a novel multi-objective SDP method via multi-source uncertain information fusion and multi-task multi-view learning (MTMV) to accurately and efficiently predict the proneness, location, and type of defects. Firstly, multi-view features are extracted from multi-source static analysis results, reflecting uncertain defect location distribution and semantic information. Then, a novel MTMV model is proposed to fully fuse the uncertain defect information in multi-view features and realize effective multi-objective SDP. Specifically, the convolutional GRU encoders capture the consistency and complementarity of multi-source defect information to automatically filter the noise of false and missed alarms, and reduce location and type uncertainty of static analysis results. A global attention mechanism combined with the hard parameter sharing in MTMV fuse features according to their global importance of all tasks for balanced learning. Then, considering the latent task and feature correlations, multiple task-specific decoders jointly optimize all SDP tasks by sharing the learning experience. Through the extensive experiments on 14 datasets, the proposed method significantly improves the prediction performance over 12 baseline methods for all SDP objectives. The average improvements are 30.7%, 31.2%, and 32.4% for defect proneness, location, and type prediction, respectively. Therefore, the proposed multi-objective SDP method can provide more sufficient and precise insights for developers to significantly improve the efficiency of software analysis and testing.

The decentralization mechanism provides manufacturers and distributors with greater customization and flexibility they need through Internet of Things (IoT)-based industrial collaboration systems (IoT-ICS), but it has brought forward security concerns about the shared data-processing tasks and IoT-based access to services and resources. To address them, we propose a practical blockchain solution to achieve decentralized policy management and evaluation on attribute-based access control (ABAC). By offloading the responsibility of ABAC policy administration and decision making to blockchain nodes, a blockchain-based access control framework, called Policychain, is presented to ensure policy with high availability, autonomy, and traceability. To deliver a solid design, we first present a transaction-oriented policy expression scheme with a well-defined syntax and semantics. The scheme can translate ABAC policies into the blockchain transactions with JavaScript object notation (JSON) syntax and script-based logical expression. We further realize a script-driven policy evaluation by extending blockchain inherent scripting instructions to support attribute acquisition of ABAC entities. Furthermore, we propose a policy lifecycle management scheme from policy creation, renovation, to revocation, in which policies are verified by three validation principles at the transaction level. Finally, we provide sophisticated analysis and experiments to show that our framework is secure and practical for decentralized policy management on ABAC in IoT-ICS.

In practice, a program may contain multiple bugs. The simultaneous presence of these bugs may deteriorate the effectiveness of existing fault-localization techniques to locate program bugs. While it is acceptable to use all failed and successful tests to identify suspicious code for programs with exactly one bug, it is not appropriate to use the same approach for programs with multiple bugs because the due-to relationship between failed tests and underlying bugs cannot be easily identified. One solution is to generate fault-focused clusters by grouping failed tests caused by the same bug into the same clusters. We propose MSeer-an advanced fault localization technique for locating multiple bugs in parallel. Our major contributions include the use of (1) a revised Kendall tau distance to measure the distance between two failed tests, (2) an innovative approach to simultaneously estimate the number of clusters and assign initial medoids to these clusters, and (3) an improved K-medoids clustering algorithm to better identify the due-to relationship between failed tests and their corresponding bugs. Case studies on 840 multiple-bug versions of seven programs suggest that MSeer performs better in terms of effectiveness and efficiency than two other techniques for locating multiple bugs in parallel.

This Innovative Practice, Work in Progress Paper presents how we improve software testing education via industry sponsored contests. Over the past decades, we have built software to improve our efficiency, reliability, and safety in production, business, daily life, etc. These goals, however, cannot be accomplished if the software is not properly tested. Some universities provide classes to teach students the fundamental knowledge and techniques of software testing. However, these classes often ignore industry practices and can hardly offer real-world testing experiences to students. To address this, we partnered with industry sponsors to design and host several software testing contests along with software testing tutorials. Through the contests and tutorials, we brought real-world testing and tool experience to the students and provided excellent opportunities for them to practice their learned testing techniques to overcome industry testing challenges.

In practice, a program may contain multiple bugs. The simultaneous presence of these bugs may deteriorate the effectiveness of existing fault-localization techniques to locate program bugs. While it is acceptable to use all failed and successful tests to identify suspicious code for programs with exactly one bug, it is not appropriate to use the same approach for programs with multiple bugs because the due-to relationship between failed tests and underlying bugs cannot be easily identified. One solution is to generate fault-focused clusters by grouping failed tests caused by the same bug into the same clusters. We propose MSeer - an advanced fault localization technique for locating multiple bugs in parallel. Our major contributions include the use of (1) a revised Kendall tau distance to measure the distance between two failed tests, (2) an innovative approach to simultaneously estimate the number of clusters and assign initial medoids to these clusters, and (3) an improved K-medoids clustering algorithm to better identify the due-to relationship between failed tests and their corresponding bugs. Case studies on 840 multiple-bug versions of seven programs suggest that MSeer performs better in terms of effectiveness and efficiency than two other techniques for locating multiple bugs in parallel.

Accidents caused by defective software systems have long been a nightmare. Though engineers utilize advanced techniques and rigorous quality control procedures, we still have to admit that the increasing complexity and expanding scale of software systems make it extremely difficult to guarantee high quality deliverables. Since large-scale software systems exhibit the characteristics of complex networks, applying the principles of complex networks to evaluate the quality of software systems has attracted attention from both academia and industry. Unfortunately, most current research studies focus only on one or a limited number of attributes of software structures which makes them ineffective in providing comprehensive and insightful quality evaluation for software structures. To overcome this problem, we propose an approach based on various software structural characteristics to evaluate software structures from modularity, hierarchy, complexity, and fault propagation points of view. A model based on these four aspects is proposed to better understand software structural quality. A prediction model is also proposed to provide insights on the nature of software evolution and its current status. Experiments using two software projects were performed against the thresholds obtained by evaluating more than 5,000 versions of open source projects. Our results suggest that the approach described in this paper can help us analyze real-world software projects for better quality evaluation.

Combinatorial testing (CT) is a black-box-based technique to generate a small number of test cases with a focus on covering various interactions among input parameters of a software system. How effective is CT in practice? What are the challenges and issues presented by the process of applying CT? How can these challenges and issues be overcome to improve the application of CT? Although CT has attracted attention from both academia and industry, these questions have not been addressed or researched in depth based on an empirical study in industrial settings. From January 2016 to February 2016, we have worked with CEPREI, an authoritative software testing company in China, on testing three real-life software systems using CT. Throughout the process, we have generated 601 test cases and found 33 bugs in total. The results demonstrate that CT not only detects more bugs but also requires less time in designing test cases for a specific system comparing to traditional function coverage-based testing. This paper will present our empirical study thoroughly and give an insightful analysis to investigate the advantages as well as the possible challenges of applying CT.

This paper rigorously introduces the concept of model-based mutation testing (MBMT) and positions it in the landscape of mutation testing. Two elementary mutation operators, insertion and omission , are exemplarily applied to a hierarchy of graph-based models of increasing expressive power including directed graphs, event sequence graphs, finite-state machines and statecharts. Test cases generated based on the mutated models ( mutants ) are used to determine not only whether each mutant can be killed but also whether there are any faults in the corresponding system under consideration (SUC) developed based on the original model. Novelties of our approach are: (1) evaluation of the fault detection capability (in terms of revealing faults in the SUC) of test sets generated based on the mutated models, and (2) superseding of the great variety of existing mutation operators by iterations and combinations of the two proposed elementary operators. Three case studies were conducted on industrial and commercial real-life systems to demonstrate the feasibility of using the proposed MBMT approach in detecting faults in SUC, and to analyze its characteristic features. Our experimental data suggest that test sets generated based on the mutated models created by insertion operators are more effective in revealing faults in SUC than those generated by omission operators. Worth noting is that test sets following the MBMT approach were able to detect faults in the systems that were tested by manufacturers and independent testing organizations before they were released.

The papers in this special section are devoted to the topic of software quality assurance. As software is integrated more frequently into every aspect of our lives, as it grows more quickly in size and function, as its failure in operations causes increasingly devastating consequences, and as schedules and budgets are continually reduced despite the need for high-quality, reliable, and secure software, advanced and innovative technologies must be developed to achieve software quality assurance more effectively and efficiently. It is also critical for the industry and academia to work together to conduct cooperative research to reduce the gap between state-of-the-art analyses and practice applications.

Software fault localization, the act of identifying the locations of faults in a program, is widely recognized to be one of the most tedious, time consuming, and expensive - yet equally critical - activities in program debugging. Due to the increasing scale and complexity of software today, manually locating faults when failures occur is rapidly becoming infeasible, and consequently, there is a strong demand for techniques that can guide software developers to the locations of faults in a program with minimal human intervention. This demand in turn has fueled the proposal and development of a broad spectrum of fault localization techniques, each of which aims to streamline the fault localization process and make it more effective by attacking the problem in a unique way. In this article, we catalog and provide a comprehensive overview of such techniques and discuss key issues and concerns that are pertinent to software fault localization as a whole.

Automated program repair can save tremendous manual efforts in software debugging. Therefore, a huge body of research efforts have been dedicated to design and implement automated program repair techniques. Among the existing program repair techniques, genetic-programming-based techniques have shown promising results. Recently, researchers found that random-search-based and adaptive program repair techniques can also produce effective results. In this work, we performed an extensive study for four program repair techniques, including genetic-programming-based, random-search-based, brute-force-based and adaptive program repair techniques. Due to the extremely large time cost of the studied techniques, the study was performed on 153 bugs from 9 small to medium sized programs. In the study, we further investigated the impacts of different programs and test suites on effectiveness and efficiency of program repair techniques. We found that techniques that work well with small programs become too costly or ineffective when applied to medium sized programs. We also computed the false positive rates and discussed the ratio of the explored search space to the whole search space for each studied technique. Surprisingly, all the studied techniques except the random-search-based technique are consistent with the 80/20 rule, i.e., about 80% of successful patches are found within the first 20% of search space.

Universities have created programs such as the Executive Master's Degree in Software Engineering (EMSE) to enable busy professionals to achieve a balance between work and life while pursuing a graduate degree. In this talk, I will address several important issues based on my experience of teaching the same course for both the EMSE program and the regular Master's program.

Effective debugging is crucial to producing reliable software. Manual debugging is becoming prohibitively expensive, especially due to the growing size and complexity of programs. Given that fault localization is one of the most expensive activities in program debugging, there has been a great demand for fault localization techniques that can help guide programmers to the locations of faults. In this paper, a technique named DStar (D*) is proposed which can suggest suspicious locations for fault localization automatically without requiring any prior information on program structure or semantics. D* is evaluated across 24 programs, and is compared to 38 different fault localization techniques. Both single-fault and multi-fault programs are used. Results indicate that D* is more effective at locating faults than all the other techniques it is compared to. An empirical evaluation is also conducted to illustrate how the effectiveness of D* increases as the exponent * grows, and then levels off when the exponent * exceeds a critical value. Discussions are presented to support such observations.

In this panel, we will address questions regarding the development and execution of one type of industry-academic collaboration - corporate-sponsored projects. As representatives of programs at both public and private institutions, we will address a set of questions related to these programs. We have prepared an initial list that will be augmented with questions submitted and voted on by conference participants in advance of the session. After we have presented our prepared responses, the balance of the session will be dedicated to open discussion with audience members.

At last year's conference, we organized a panel “Involving Undergraduates in Research: Motivations and Challenges” which was a great success with many interesting discussions. This has motivated us to develop a second iteration of this panel with an additional dimension - the Path Forward. We expect to have more discussions on how to extend what we have already learned from the past experiences to further enhance the way we involve undergraduates in doing research. Panelists will deliver a 5-minute overview of their research experiences with undergraduate students, including challenges they faced, lessons learned, and areas for improvement. The floor will then be opened for audience members to voice any concerns, questions, or comments. Student panel attendees will be given special consideration when presenting observations from their own perspective.

SUMMARY A vast number of software fault localization techniques have been proposed recently with the growing realization that manual debugging is time‐consuming, tedious, and error‐prone, and fault localization is one of the most expensive debugging activities. Although some techniques perform better than others on a large number of data sets, they do not do so on all data sets and therefore, the actual quality of fault localization can vary considerably by using just one technique. This paper proposes the use of a consensus‐based strategy that combines the results of multiple fault localization techniques to consistently provide high quality performance, irrespective of data set. Empirical evidence based on case studies conducted on six sets of programs (seven programs of the Siemens suite, and the gzip, grep, make, space, and Ant programs) and three different fault localization techniques (Tarantula, Ochiai, and H3) suggests that the consensus‐based strategy holds merit and generally provides close to the best, if not the best, results. Empirically, we show that this is true of both single‐fault and multifault programs. Additionally, the consensus‐based strategy makes use of techniques that all operate on the same set of input data, minimizing the overhead. It is also simple to include or exclude techniques from consensus, making it an easily extensible or tractable strategy. Copyright © 2011 John Wiley & Sons, Ltd.

Random testing is a basic software testing technique that can be used to assess the software reliability as well as to detect software failures. Adaptive random testing has been proposed to enhance the failure-detection capability of random testing. Previous studies have shown that adaptive random testing can use fewer test cases than random testing to detect the first software failure. In this paper, we evaluate and compare the performance of adaptive random testing and random testing from another perspective, that of code coverage. As shown in various investigations, a higher code coverage not only brings a higher failure-detection capability, but also improves the effectiveness of software reliability estimation. We conduct a series of experiments based on two categories of code coverage criteria: structure-based coverage, and fault-based coverage. Adaptive random testing can achieve higher code coverage than random testing with the same number of test cases. Our experimental results imply that, in addition to having a better failure-detection capability than random testing, adaptive random testing also delivers a higher effectiveness in assessing software reliability, and a higher confidence in the reliability of the software under test even when no failure is detected.

In the United States, one of the main objectives of the National Science Foundation (NSF) is the encouragement and support of undergraduate involvement in hands-on research projects. An early exposure to actual research practice can help these students more quickly and effectively grasp the principles and goals of research in general.

We propose the application of a modified radial basis function neural network in the context of software fault localization, to assist programmers in locating bugs effectively. This neural network is trained to learn the relationship between the statement coverage information of a test case and its corresponding execution result, success or failure. The trained network is then given as input a set of virtual test cases, each covering a single statement. The output of the network, for each virtual test case, is considered to be the suspiciousness of the corresponding covered statement. A statement with a higher suspiciousness has a higher likelihood of containing a bug, and thus statements can be ranked in descending order of their suspiciousness. The ranking can then be examined one by one, starting from the top, until a bug is located. Case studies on 15 different programs were conducted, and the results clearly show that our proposed technique is more effective than several other popular, state of the art fault localization techniques. Further studies investigate the robustness of the proposed technique, and illustrate how it can easily be applied to programs with multiple bugs as well.

It is becoming prohibitively expensive and time consuming, as well as tedious and error-prone, to perform debugging manually. Among the debugging activities, fault localization has been one of the most expensive, and therefore, a large number of fault-localization techniques have been proposed over the recent years. This paper presents a crosstab-based statistical technique that makes use of the coverage information of each executable statement and the execution result (success or failure) with respect to each test case to localize faults in an effective and efficient manner. A crosstab is constructed for each executable statement, and a statistic is computed to determine the suspiciousness of the corresponding statement. Statements with a higher suspiciousness are more likely to contain bugs and should be examined before those with a lower suspiciousness. Case studies are performed on both small- (the Siemens and Unix suites) and large-sized programs (space, grep, gzip, and make), and results suggest that the crosstab-based technique (CBT) is more effective (in terms of a smaller percentage of executable statements that have to be examined until the first statement containing the fault is reached) than other techniques, such as Tarantula. Further studies using the Siemens suite reveal that the proposed technique is also more effective at locating faults than other statistically oriented techniques, such as SOBER and Liblit05. Additional experiments evaluate the CBT from other perspectives, such as its efficiency in terms of time taken, its applicability to object-oriented languages (on a very large Java program: Ant), and its sensitivity to test suite size, and demonstrate its superior performance.

According to a study commissioned by the National Institute of Standards and Technology in 2002, software bugs cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the nation's gross domestic product (GDP). The same study also found that more than one-third of these costs, or an estimated $22.2 billion, could be eliminated by an improved testing infrastructure. These numbers would be significantly higher if the study were conducted today.

Software fault localization techniques typically rank program components, such as statements or predicates, in descending order of their suspiciousness (likelihood of being faulty). During debugging, programmers may examine these components, starting from the top of the ranking, in order to locate faults. However, the assigned suspiciousness to each component may not always be unique, and thus some of them may be tied for the same position in the ranking. In such a scenario, the total number of components that a programmer needs to examine in order to find the faults may vary considerably. The greater the variability, the harder it is for a programmer to decide which component to examine first, and the harder it is to accurately compute the expected effectiveness of a fault localization technique. In this paper, we first conduct a case study, based on three fault localization techniques across four sets of programs, which reveals that the phenomenon of assigning the same suspiciousness to multiple components is not limited to any technique or program in particular. Thus, to reduce variability and alleviate this problem, four tie-breaking strategies are discussed and evaluated empirically in our second case study. Results indicate that the strategies can not only reduce the number of ties in the rankings, but also maintain the effectiveness of the fault localization techniques. We also propose a new metric for evaluating fault localization techniques called CScore, which takes the notion of ties into account. Finally, an additional slicing-based approach to breaking ties is discussed briefly, which aims to provide further insights into tie-breaking and stimulate further research in the area.

Functional modeling and safety analysis are two important aspects of safety-critical embedded systems. However, they are often conducted separately. In this paper, we present an approach for integrating fault-tree-based safety analysis into statechart-based functional modeling. The proposed approach uses systematic transformation steps that maintain the semantics of both the fault tree and the statechart. It also provides a set of conversion rules that transform the gates of fault trees into statechart notations. The resultant model shows how the system behaves when a failure condition occurs and acts as a basis model that ensures safety through requirement validation. Using the gas burner case study, we demonstrate the advantages of the integrated model over the use of separate models, such as the lack of ambiguities, separation of concerns, and taking the order of the occurrence of faults into consideration.

Poorly designed software systems are one of main causes of accidents in safety-critical systems, and thus, the importance of safety analysis for software has greatly increased over the recent years. Software safety can be improved by analyzing both its desired and undesired behaviors, and this in turn requires expressive power such that both can be modeled. However, there is a considerable gap between modeling methods for desired and undesired behaviors. Therefore, we propose a method to bridge the gap between fault trees (for undesired behavior) and UML state machine diagrams (for desired behavior). More specifically, we present rules and algorithms that facilitate the transformation of a hazard (in the context of fault trees) to a UML state machine diagram. We illustrate our proposed approach via an example on a microwave-oven system. Our proposed transformation can help engineers identify how the hazards may occur, thereby allowing them to prevent the hazard from occurring.

This paper shares the experiences and lessons learned from conducting an NSF-sponsored eightweek summer research program for ten undergraduate students from multiple universities. The focus of the research was on ¿Verification and Validation for Software Safety¿, and emphasized a strong fundamental knowledge of software safety while maintaining a close collaboration with industry. The program included lectures and special tutorials from guest speakers, field trips, review sessions, as well as posters and final presentations by the students.

This paper proposes a strategy for automatically fixing faults in a program by combining the processes of mutation and fault localization. Statements that are ranked in order of their suspiciousness of containing faults can then be mutated in the same order to produce possible fixes for the faulty program. The proposed strategy is evaluated against the seven benchmark programs of the Siemens suite and the Ant program. Results indicate that the strategy is effective at automatically suggesting fixes for faults without any human intervention.

The Annual Technical Report this year is focused on infrastructure reliability. Infrastructure constitutes those things that are apparent only in their absence. We take the infrastructure for granted, assuming it will always be there. We turn on our water facet, and drinkable water has always flowed out, for most of us, most of the time. Our infrastructure is subject to environment breakages (e.g., earthquakes), accidents (e.g., dig ups of cables), sabotage, intrusion, and compromise. Also everyday component, software or system failures can bring our infrastructure down. Our global connectivity and communications, as well as our world wide distributed development and maintenance systems, increase our productivity and efficiency, but can also increase our vulnerabilities. Our critical infrastructures can be found in many places.

Multiple faults in a program may interact with each other in a variety of ways. A test case that fails due to a fault may not fail when another fault is added, because the second fault may mask the failure-causing effect of the first fault. Multiple faults may also collectively cause failure on a test case that does not fail due to any single fault alone. Many studies try to perform fault localization on multi-fault programs and several of them seek to match a failed test to its causative fault. It is therefore, important to better understand the interference between faults in a multi-fault program, as an improper assumption about test case failure may lead to an incorrect matching of failed test to fault, which may in turn result in poor fault localization. This paper investigates such interference and examines if one form of interference holds more often than another, and uniformly across all conditions. Empirical studies on the Siemens suite suggest that no one form of interference holds unconditionally and that observation of failure masking is a more frequent event than observation of a new test case failure.

Component-based software development techniques are being adopted to rapidly deploy complex, high quality systems. One of its aspects is the selection of components that realize the specified requirements. In addition to the functional requirements, the selection must be done taking into account some non-functional requirements such as performance, reliability, and usability. Hence, data that characterize the non-functional behavior of the components is needed; a test set is needed to collect this data for each component under consideration. This set may be large, which results in a considerable increase in the cost of the development process. Here, a process is proposed to considerably reduce the number of test cases used in the performance evaluation of components. The process is based on sequential curve fittings from an incremental number of test cases until a minimal pre-specified residual error is achieved. The incremental selection of test cases is done in two different ways: randomly and adaptively. The accuracy and performance of the proposed approach are dependent on the values of the desired residual error. The smaller the residual error, the higher the accuracy. However, performance has an opposite behavior. The smaller the error, the larger the number of test cases needed. The results from experiments with image compression components are a clear indication that a reduction in the number of test cases can be achieved while maintaining reasonable accuracy when using the proposed approach.

In program debugging, fault localization identifies the exact locations of program faults. Finding these faults using an ad-hoc approach or based only on programmers' intuitive guesswork can be very time consuming. A better way is to use a well-justified method, supported by case studies for its effectiveness, to automatically identify and prioritize suspicious code for an examination of possible fault locations. To do so, we propose the use of a back-propagation (BP) neural network, a machine learning model which has been successfully applied to software risk analysis, cost prediction, and reliability estimation, to help programmers effectively locate program faults. A BP neural network is suitable for learning the input-output relationship from a set of data, such as the inputs and the corresponding outputs of a program. We first train a BP neural network with the coverage data (statement coverage in our case) and the execution result (success or failure) collected from executing a program, and then we use the trained network to compute the suspiciousness of each executable statement, in terms of its likelihood of containing faults. Suspicious code is ranked in descending order based on its suspiciousness. Programmers will examine such code from the top of the rank to identify faults. Four case studies on different programs (the Siemens suite, the Unix suite, grep and gzip) are conducted. Our results suggest that a BP neural network-based fault localization method is effective in locating program faults.

This paper presents an approach to automated generation of executable test code from class models represented by the UML protocol state machines. It supports several coverage criteria for state models, including state coverage, transition coverage, and basic and extended round-trip coverage. It allows the tester to add and modify detailed test parameters (e.g., actual arguments for method invocations and implementation-specific environments) if necessary. When the state model is modified due to requirements change, the hand-crafted test parameters, if still valid, are automatically reused. This reduces the working load for regeneration of tests for modified models. In addition to test code, we also automatically generate state wrapper aspects in AspectJ, which facilitates comparing actual object states to expected states during test execution. This enables the automated verdict of pass/failure for test cases without the need to modify the source code of the class under test. We present two examples for which the executable test code is generated. They demonstrate the reuse of test parameters and testing of object interactions, respectively.

An aspect-oriented design model is composed of a primary design model and an aspect model, each of which has a different origin and pursues different goals. How to ensure the correctness of the composition of a primary design model and an aspect model has become an important issue. In this paper, we validate an aspect-oriented design model using the UML profile extension mechanism. With this mechanism, engineers can utilize the ICER tool to find errors in an aspect-oriented design model using the instance-of relationship. Moreover, the profile extension mechanism also provides engineers with an ability to introduce their own validation of an aspect-oriented design model. This flexibility allows engineers to express the special requirements in a particular development environment. We also illustrate how ICER can find errors in a real aspect-oriented design model composed by the UML metamodel and a creation aspect model.

As software systems are encompassing a wide range of fields and applications, software reliability becomes a crucial step. The need for safety analysis and test cases that have high probability to uncover plausible faults are necessities in proving software quality. System models that represent only the operational behavioral of a system are incomplete sources for deriving test cases and performing safety analysis before the implementation process. Therefore, a system model that encompasses faults is required. This paper presents a technique that formalizes a safety model through the incorporation of faults with system specifications. The technique focuses on introducing semantic faults through the integration of fault trees with system specifications or statechart. The method uses a set of systematic transformation rules that tries to maintain the semantics of both fault trees and statechart representations during the transformation of fault trees into statechart notations.

Fault localization is the most expensive activity in program debugging. Traditional ad-hoc methods can be time-consuming and ineffective because they rely on programmers’ intuitive guesswork, which may neither be accurate nor reliable. A better solution is to utilize a systematic and statistically well-defined method to automatically identify suspicious code that should be examined for possible fault locations. We present a crosstab-based statistical method using the coverage information of each executable statement and the execution result (success or failure) with respect to each test case. A crosstab is constructed for each executable statement and a statistic is computed to determine the suspiciousness of the corresponding statement. Statements with a higher suspiciousness are more likely to contain bugs and should be examined before those with a lower suspiciousness. Three case studies using the Siemens suite, the Space program, and the Unix suite, respectively, are conducted. Our results suggest that the crosstab-based method is effective in fault localization and performs better (in terms of a smaller percentage of executable statements that have to be examined until the first statement containing the fault is reached) than other methods such as Tarantula. The difference in efficiency (computational time) between these two methods is very small.

Recent experience and feedback from panels indicates that what the audience likes best is the chance to ask questions, particularly regarding things that might solve problems on their development projects or in their research studies. This panel has been held at every ISSRE since 1997, and has been highly popular. Originally the panel was chaired by John Musa, but Mike Hinchey took over that role last year. The panel has no presentations, only questions. The questions can be on anything, ranging from theory to details of application. The panelists have been selected to make available wide and extensive experience in the field.

We propose an RBF (radial basis function) neural network-based fault localization method to help programmers locate bugs in a more effective way. An RBF neural network with a three-layer feed-forward structure is employed to learn the relationship between the statement coverage of a test case and its corresponding execution result. The trained network is then given as input a set of virtual test cases, each covering only a single statement. The output of the network for each test case is considered to be the suspiciousness of the corresponding statement; a statement with a higher suspiciousness has a higher likelihood of containing a bug. The set of statements ranked in descending order by their suspiciousness are then examined by programmers one by one until a bug is located. Three case studies on different programs (space, grep and make) were conducted with each faulty version having exactly one bug. An additional program gcc was also used to demonstrate the concept of extending the proposed method to programs with multiple bugs. Our experimental data suggest that an RBF neural network-based fault localization method is more effective in locating a program bug (by examining less code before the first faulty statement containing the bug is identified) than another popular method, Tarantula, which also uses the coverage and execution results to compute the suspiciousness of each statement.

The application of specification-based program verification techniques (e.g., black-box testing, formal proof) faces strong challenges in practice when the gap between the structure of a specification and that of its program is large. This paper describes a view-based program review approach to addressing these challenges. The essential idea of the approach is first to derive comparable views from the specification and program, and then detect and eliminate the violations of structural consistency in the program views on the basis of a set of criteria. We also developed a prototype tool to support the review approach, and conducted a case study to assess the effectiveness of the approach.

One common approach to test sequence generation for structurally testing concurrent programs involves constructing a reachability graph (RG) and selecting a set of paths from the graph to satisfy some coverage criterion. It is often suggested that test sequence generation methods for testing sequential programs based on a control flow graph (CFG) can also be used to select paths from an RG for testing concurrent programs. However, there is a major difference between these two, as the former suffers from a feasibility problem (i.e., some paths in a CFG may not be feasible at run-time) and the latter does not. As a result, even though test sequence generation methods for sequential programs can be applied to concurrent programs, they may not be efficient. We propose four methods — two based on hot spot prioritization and two based on topological sort — to effectively generate a small set of test sequences that covers all the nodes in an RG. The same methods are also applied to the corresponding dual graph for generating test sequences to cover all the edges. A case study was conducted to demonstrate the use of our methods.

The new constructs in aspect-oriented programming bring new types of programming faults with respect to crosscutting concerns, such as incorrect pointcuts and advice. This paper presents a UML-based approach to testing whether or not an aspect-oriented program conforms to its expected crosscutting behavior. We explore aspect-oriented UML design models to derive tests for exercising interactions between aspects and classes. Each aspect-oriented model consists of class diagrams, aspect diagrams, and sequence diagrams. For a method under test, we weave the sequence diagrams of the advice on the method into the method's sequence diagram. Based on the woven sequence diagram and class/aspect diagrams, we then generate an AOF (Aspect-Object Flow) tree by applying coverage criteria such as condition coverage, polymorphic coverage, and loop coverage to woven sequence diagrams. In the AOF tree, each path from the root to a leaf is an abstract message sequence, indicating a template of test cases. A concrete test case is obtained by creating objects that satisfy the collective constraints in the template. Our empirical study shows that the model-based testing approach is capable of revealing several types of aspect-specific faults, including incorrect advice type, incorrect (weaker or stronger) pointcut strengths, and incorrect aspect precedence.

Autonomous adaptive systems (AAS) have been proposed as a solution to effectively (re)design software so that it can respond to changes in execution environments, without human intervention. In the software engineering community, alternative approaches to the design of AAS have been proposed including solutions based on component technology, design patterns, and resource allocation techniques. A key limitation of the currently available approaches is that they detect constraint violations, but they do not support the prediction of constraint violations. In this work we propose an architectural framework for the design and analysis of autonomous adaptive systems, hereafter referred to as KAROO, which provides a key, new contribution: the capability to predict when a system needs to adapt itself. The results of extensive experimental evaluation of a KAROO-based system are excellent: 100% of the violations are predicted; the system is able to avoid the violations by adapting itself almost 98% of the time. The framework is a novel integration of control-theory-based adaptation, multi-criteria decision making and component-based software engineering techniques.

In data stream management systems (DSMSs), request handling has to meet various QoS requirements. Traditional open-loop DSMS ignores system status information and run-time history in decision making and is unable to achieve stable QoS performance over run-time as the workload or environment changes. This paper discusses the design of a close-loop DSMS that applies adaptive control in control theory to adapt itself to the changing environment and workloads in order to optimize the QoS performance over run-time. The proposed adaptive DSMS is built on system identification and successive approximation knowledge in control theory and uses a least square approach for parameter estimation. Experimental data shows that the proposed adaptive DSMS outperforms the traditional open-loop DSMS in overall QoS among various network conditions and workload settings.

Localizing a bug in a program can be a complex and timeconsuming process. In this paper we propose a code coveragebased fault localization method to prioritize suspicious code in terms of its likelihood of containing program bugs. Code with a higher risk should be examined before that with a lower risk, as the former is more suspicious (i.e., more likely to contain program bugs) than the latter. We also answer a very important question: How can each additional test case that executes the program successfully help locate program bugs? We propose that with respect to a piece of code, the aid introduced by the first successful test that executes it in computing its likelihood of containing a bug is larger than or equal to that of the second successful test that executes it, which is larger than or equal to that of the third successful test that executes it, etc. A tool, ÷Debug, was implemented to automate the computation of the risk of the code and the subsequent prioritization of suspicious code for locating program bugs. A case study using the Siemens suite was also conducted. Data collected from our study support the proposal described above. They also indicate that our method (in particular Heuristics III (c), (d), and (e)) can effectively reduce the search domain for locating program bugs.

Software metric models predict the target software metric(s), e.g., the development work effort or defect rates, for any future software project based on the project's predictor software metric(s), e.g., the project team size. Obviously, the construction of such a software metric model makes use of a data sample of such metrics from analogous past projects. However, incomplete data often appear in such data samples. Moreover, the decision on whether a particular predictor metric should be included is most likely based on an intuitive or experience-based assumption that the predictor metric has an impact on the target metric with a statistical significance. However, this assumption is usually not verifiable "retrospectively" after the model is constructed, leading to redundant predictor metric(s) and/or unnecessary predictor metric complexity. To solve all these problems, we derived a methodology consisting of the k-nearest neighbors (k-NN) imputation method, statistical hypothesis testing, and a "goodness-of-fit" criterion. This methodology was tested on software effort metric models and software quality metric models, the latter usually suffers from far more serious incomplete data. This paper documents this methodology and the tests on these two types of software metric models.

Much attention has been focused on the problem of effectively designing software architecture to meet non-functional requirements (NFRs). The significant benefits of such work include detecting and removing defects earlier, which reduces development time and cost while improving the quality of the design; in turn the improved quality of the design makes it easier to maintain as the system evolves. The Formal Design Analysis Framework (FDAF) is an aspect-oriented approach proposed to support the design and analysis of multiple NFRs for distributed, concurrent, and real-time systems. In FDAF, non-functional requirements are realized and represented as reusable aspects in the repository; designs are captured in an extended version of the Unified Modeling Language (UML). FDAF supports the automated translation of extended UML designs into existing formal notations. Subsequently, the analysis of an aspect design is achieved using existing formal analysis tools, which leverages a large body of work in the research community. This paper focuses on the modeling and analysis of the response time performance aspect. The algorithms for translating extended UML diagrams into Rapide, the proofs of correctness for the algorithms, and an illustration of the FDAF approach using the Domain Name System are presented.

Abstract This paper describes intra‐method control‐flow and data‐flow testing criteria for the Java bytecode language. Six testing criteria are considered for the generation of testing requirements: four control‐flow and two data‐flow based. The main reason to work at a lower level is that, even when there is no source code, structural testing requirements can still be derived and used to assess the quality of a given test set. It can be used, for instance, to perform structural testing on third‐party Java components. In addition, the bytecode can be seen as an intermediate language, so the analysis performed at this level can be mapped back to the original high‐level language that generated the bytecode. To support the application of the testing criteria, we have implemented a tool named JaBUTi (Java Bytecode Understanding and Testing). JaBUTi is used to illustrate the application of the ideas developed in this paper. Copyright © 2006 John Wiley & Sons, Ltd.

Testing a large-scale, real-life commercial software application is a very challenging task due to the constant changes in the software, the involvement of multiple programmers and testers, and a large amount of code. Integrating testing with development can help find program bugs at an earlier stage and hence reduce the overall cost. In this paper, we report our experience on how to apply eXVantage (a tool suite for code coverage testing, debugging, performance profiling, etc.) to a large, complex Java application at the implementation and unit testing phases in Avaya. Our results suggest that programmers and testers can benefit from using eXVantage to monitor the testing process, gain confidence on the quality of their software, detect bugs which are otherwise difficult to reveal, and identify performance bottlenecks in terms of which part of code is most frequently executed.

A software metric model can be used to predict a target metric (e.g., the development work effort) for a future release of a software system based on the project's predictor metrics (e.g., the project team size). However, missing or incomplete data often appear in the data samples used to construct the model. So far, the least biased and thus the most recommended software metric models for dealing with the missing/incomplete data are those constructed by using the maximum likelihood methods. It is true that the inclusion of a particular predictor metric in the model construction is initially based on an intuitive or experience-based assumption that the predictor metric impacts significantly the target metric. Nevertheless, this assumption has to be verified. Previous research on metric models constructed by using the maximum likelihood methods simply took this verification for granted. This can result in probable inclusion of superfluous predictor metric(s) and/or unnecessary predictor metric complexity. In this paper, we propose a methodology to optimize and simplify such models based on the results of appropriate hypothesis tests. An experiment is also reported to demonstrate the use of our methodology in trimming redundant predictor metric(s) and/or unnecessary predictor metric complexity.

In recent years, Web applications have grown so quickly that they have already become crucial to the success of businesses. However, since they are built on Internet and open standard technologies, Web applications bring new challenges to researchers, such as dynamic behaviors, heterogeneous representations, novel control flow and data flow mechanisms, etc. In this paper, we propose an agent-based approach for Web application testing. While the agent-based framework greatly reduces the complexity of Web applications, a four-level dataflow test approach can be employed to perform structure testing on them. In this approach, data flow analysis is performed as function level testing, function cluster level testing, object level testing, and Web application level testing, from low abstract level to high abstract level. Each test agent in the framework takes charge of the testing in an abstract level for a particular type of Web document or object.

One common approach to test sequence generation for structurally testing concurrent programs involves constructing a reachability graph (RG) and selecting a set of paths from the graph to satisfy some coverage criterion. It is often suggested that test sequence generation methods for testing sequential programs based on a control flow graph (CFG) can also be used to select paths from a RG for testing concurrent programs. However, there is a major difference between these two, as the former suffers from a feasibility problem (i.e., some paths in a CFG may not be feasible at run-time) and the latter does not. As a result, even though test sequence generation methods for sequential programs can be applied to concurrent programs, they may not be efficient. Moreover, in order to reduce testing effort and costs, it is important to reduce the number of test sequences being generated. Stated differently, we need methods which can generate efficient test sequences to increase the coverage in an effective way. We propose four different methods - two based on hot spot prioritization and two based on topological sort -to effectively generate a small set of test sequences that cover all the nodes in a RG. The same methods are also applied to the corresponding dual graph for generating test sequences to cover all the edges. A case study was conducted to demonstrate the use of our methods.

Localizing a fault in a program is a complex and time-consuming process. In this paper we present a novel approach using execution slice and inter-block data dependency to effectively identify the locations of program faults. An execution slice with respect to a given test case is the set of code executed by this test, and two blocks are data dependent if one block contains a definition that is used by another block or vice versa. Not only can our approach reduce the search domain for program debugging, but also prioritize suspicious locations in the reduced domain based on their likelihood of containing faults. More specifically, the likelihood of a piece of code containing a specific fault is inversely proportional to the number of successful tests that execute it. In addition, the likelihood also depends on whether this piece of code is data dependent on other suspicious code. A debugging tool, DESiD, was developed to support our method. A case study that shows the effectiveness of our method in locating faults on an application developed for the European Space Agency is also reported.

We present an experimental report of a multimedia wireless system with secure delivery of multimodal services to end-users over voice and data converged networks. Our design leads to an easily adaptable and extensible system architecture, in which system components interact with each other through loosely coupled interfaces based on asynchronous exchange of messages. It effectively separates service logic from service delivery, through the introduction of a secure broker between the end user devices and the enterprise level application that contains the service logic. This separation also provides additional levels of security and reliability at the application level. This work describes our system implementation and its performance results.

Object-oriented languages support many modern programming concepts such as information hiding, inheritance, polymorphism, and dynamic binding. As a result, software systems implemented in OO languages are in general more reusable and reliable than others. Many legacy software systems, created before OO programming became popular, need to be redesigned and updated to OO programs. The process of abstracting OO designs from the procedural source code has often been done with limited assistance from program structural diagrams. Most reengineering focuses on the functionality of the original program, and the OO redesign often results in a completely new design based on the designers' understanding of the original program. Such an approach is not sufficient because it may take a significant amount of time and effort for designers to comprehend the original program. This paper presents a computer-aided semi-automatic method that abstracts OO designs from the original procedural source code. More specifically, it is a method for OO redesign based on program structural diagrams, visualization, and execution slices. We conducted a case study by applying this method to an inventory management software system. Results indicate that our method can effectively and efficiently abstract an appropriate OO design out of the original C code. In addition, some of the code from the original system can be automatically identified and reused in the new OO system.

Statistical data show that it is much less expensive to correct software bugs at the early design stage rather than the late stage of the development process when the final system has already been implemented and integrated together. The use of slicing and execution histories as an aid in software debugging is well established for programming languages like C and C++; however, it is rarely applied in the field of software design specification. We propose a solution by applying the source code level technologies to debugging software designs represented in a high-level specification and description language such as SDL. More specifically, we extend execution slice-based heuristics from source code-based debugging to the software design specification level. Suspicious locations in an SDL specification are prioritized by their likelihood of containing faults. Locations with a higher priority should be examined first rather than those with a lower priority as the former are more likely to contain the faults. A debugging tool, SmartD/sub DSL/, with user-friendly interfaces was developed to support our method. An illustration is provided to demonstrate the feasibility of using our method to effectively debug an architectural design.

In this paper, we present an approach of integrating SIP (session initiation protocol) in converged multimodal/multimedia communication services. An architecture of multimedia technology integration platform (MTIP) in converged communication environment with SIP is described. This distributed dialogue system infrastructure is based on the concept of dialogue system and web convergence, where applications and communication services can be enabled through task specific markup languages based on XML. The loosely coupled open architecture in our approach is highly extensible. It allows an easy extension to include SIP in converged multimodal/multimedia communication services. The concept and structure of VoIPTeleServer is described, which interfaces to the VoIP world and works as a broker between the SIP environment and MTIP to deliver converged communication services. A prototype of VoIPTeleServer was implemented, and services and applications based on SIP and MTIP convergence are constructed.

We report a study on a predictive handoff scheme in cellular networks. We present a new channel resource allocation strategy, which combines predictive channel reservation based on real-time positioning of mobile users and dynamic channel borrowing among cells in time of congestion, to provide services to handoff requests with higher priority over new call requests. We also propose a different criterion, threshold time, to improve the accuracy of channel reservation by better determining the right time to send a reservation request, if necessary, for a potential handoff call. Our simulation results indicate that the new scheme outperforms other handoff schemes including guard channel, the original predictive channel reservation, and channel borrowing by significantly reducing the handoff blocking rates with little impact on the throughput of cellular networks. In addition, our data suggest that initiating reservation requests based on an appropriate threshold time introduces fewer false reservations than initiating reservation requests based on a threshold distance.

We present an approach for secure delivery of multimodal services to end users over heterogeneous converged networks. Our approach leads to an easily adaptable and extensible system architecture, in which system components interact with each other through loosely coupled interfaces based on asynchronous exchange of messages. It also effectively separates service logic from service delivery, through the introduction of a secure broker between the end user devices and the enterprise level application that contains the service logic. This separation also provides additional levels of security at the application level. This paper describes how our approach provides the features mentioned above. We have also validated our approach through the implementation of a prototype system, a description of which is also provided.

Object-oriented/sup 1/ languages support many modern programming paradigms such as information hiding, inheritance, polymorphism, and dynamic binding. As a result, software systems implemented in OO languages are more reusable and reliable than those in non-OO. Many legacy software systems were created before OO programming became popular and needed to be redesigned and updated to OO programs. The process of abstracting OO designs from the procedural source code was often done manually or with limited assistance from program structural diagrams. Most reengineering focuses on the functionality of the original program, and the OO redesign often results in a completely new design based on the designers' understanding of the original program. Such an approach is not sufficient, for it not only takes time and effort for designers to become familiar with the original program, but the approach itself is also mistake-prone due to the human involvement. This paper presents a computer-aided semi-automatic method that abstracts OO designs from the original procedural source code. More specifically, it is a method for OO redesign based on program structural diagrams, visualization, and execution slice. We conducted a case study by applying this method to an inventory management software system. Results indicate that our method can effectively and efficiently abstract an appropriate OO design out of the original C code as well as re-generate part of the system in C++ code based on this new OO design.

formerly part of Lucent Bell Labs), Telcordia Technologies (formerly Bellcore), Nokia

We present a metric to determine the distance between the features of a software system. Such a measurement can elucidate how features of the system being examined are close to each other. We first use an execution slice-based technique to identify a set of code (basic blocks in our case) that is used to implement each feature. Then, depending on whether the execution frequency of each block is considered during the construction of such sets of code, a static as well as a dynamic distance is computed for each pair of features. These two types of distance differ in that the former computes the distance between two features only by how these features are implemented in the system, while the latter also takes into account how each feature is executed based on a user's operational profile. In other words, the static distance quantitatively gives the closeness of two features from the system implementation point of view, whereas the dynamic distance presents such closeness from the users' execution point of view. To illustrate the use of our metric, we report a case study on a Symbolic Hierarchical Automated Reliability and Performance Evaluator (SHARPE). The results of our study suggest that the distance metric discussed in this paper can provide a good measurement, in a quantitative way, of how close two program features are. Such information can also serve as a good start to understanding how a modification made to one feature is likely to affect other features.

The rapid growth of the Internet has spurred development of new protocols and algorithms to meet changing operational requirements such as security, multicast transport, mobile networking, policy management, and quality-of-service support. Development and evaluation of these operational tools requires answering many design questions. Building testbeds and labs is expensive, reconfiguring and sharing them are difficult, and they are relatively inflexible. Networking research increasingly depends on simulation to investigate new network design, protocol behavior, performance, and interactions. Simulation relies on network models. This paper provides an automatic method for constructing simulation models from network scenario workflows. Our experiments show that automatic construction of simulation models allows online simulation that can be adjusted dynamically and promptly.

Mutation 2000-A Symposium on Mutation TestingSoftware testing involves, among other activities, construction of test cases, execution of the program being tested against these test cases and observation of program behaviour to determine its acceptability.Although its use is hardly pervasive, mutation-based testing is one of the most fascinating and powerful techniques for software testing.It achieves these goals by requiring the tester to construct test cases that will distinguish the program under test from associated mutant versions, each of which contains a small syntactic deviation, representing a specific type of fault, from the program under test.Sponsored by SERC (Software Engineering Research Center: http://www.serc.net)and the IEEE Reliability Society, as well as having financial support from Telcordia Technologies (formerly Bellcore) and the National Science Foundation, Mutation 2000 was held in San Jose, California on the 6-7 October 2000.It was co-located with the International Conference on Software Maintenance (ICSM 2000) and the International Symposium on Software Reliability Engineering (ISSRE 2000).Mutation 2000 was the first event of its kind to bring together researchers and practitioners of mutation testing from all over the world.These individuals shared their experiences and insights on various practical and theoretical aspects of mutation testing.Of all the papers presented, three were selected for publication in this special issue.The first paper, by Kim, Clark and McDermid, investigates the effectiveness of object-oriented (OO) testing strategies using the mutation method.Test cases are generated by using three different OO test methods to validate an application that is a beta version of an IBM product written in Java.The fault detection effectiveness of these tests is compared in terms of their capability for killing traditional mutants generated by a version of the mutation tool Mothra for Java, as well as OO-specific mutants generated by class mutation.The next paper, by Ghosh and Mathur, describes a method called interface mutation, for testing software components using the information available from the description of a component's interface.The test adequacy criterion based on this interface mutation is compared with control flow-based coverage criteria for their relative effectiveness in revealing faults and the cost incurred in developing test cases.The third paper, by Vincenzi, Maldonado, Barbosa and Delamaro, is an empirical study on unit and integration testing strategies for C programs using mutation-based criteria.This paper describes an

Existing software reliability-growth models often over-estimate the reliability of a given program. Empirical studies suggest that the over-estimations exist because the models do not account for the nature of the testing. Every testing technique has a limit to its ability to reveal faults in a given system. Thus, as testing continues in its region of saturation, no more faults are discovered and inaccurate reliability-growth phenomena are predicted from the models. This paper presents a technique intended to solve this problem, using both time and code coverage measures for the prediction of software failures in operation. Coverage information collected during testing is used only to consider the effective portion of the test data. Execution time between test cases, which neither increases code coverage nor causes a failure, is reduced by a parameterized factor. Experiments were conducted to evaluate this technique, on a program created in a simulated environment with simulated faults, and on two industrial systems that contained tenths of ordinary faults. Two well-known reliability models, Goel-Okumoto and Musa-Okumoto, were applied to both the raw data and to the data adjusted using this technique. Results show that over-estimation of reliability is properly corrected in the cases studied. This new approach has potential, not only to achieve more accurate applications of software reliability models, but to reveal effective ways of conducting software testing.

Software testing and maintenance are the two most expensive phases of the software life cycle. Why, in general, are customers not satisfied with the quality of software? Studies have shown that approximately 20% of a software system is responsible for 80% of the faults and over the lifetime of an average software system as much as 70% of the cost is estimated to be dedicated to testing and maintenance. Clearly an appropriate methodology ought to be employed to address these high-leverage concerns. With the increased emphasis on software quality and the pressure to deliver software faster and cheaper such a methodology should encompass software design, architecture, development, testing, and maintenance as a whole and provide an integrated solution that leads to the creation of more dependable software. In addition, it is also crucial to have a set of effective tools with user-friendly interfaces available to software practitioners to apply this methodology in real-life contexts. We explain how to take advantage of sophisticated analysis of the dynamic behavior of the software being developed to help programmers, as well as testers, accomplish their assignments more effectively and efficiently.

The purpose of this study was to identify fault-prone functions that are likely to contain faults in a given software system. Five metrics were used: Di, an internal design metric which incorporates factors related to a function's internal structure; De, an external design metric which focuses on a function's external relationships to the rest of the software system; D(G), a composite design metric which is a linear combination of Di and De; and the union and intersection of Di, De, and D(G). Since the system being considered was already developed, a very important aspect of our study was to extract the design information directly from the source code rather than from the corresponding design documentation which may not exist or, if it does exist, it may be incomplete, difficult to understand, or not updated. To make the analysis more accurate and efficient, a metric analysis tool (χMetrics) was implemented. We conducted experiments using χMetrics on part of a distributed software system, written in C, with a client–server architecture, and identified a small percentage of its functions as good candidates for fault-proneness. Files containing these functions were then validated by the real defect data collected between a recent major release and its subsequent release for their fault-proneness. The results indicate that our metrics are good indicators of fault-prone functions. Two extra experiments were also conducted to show that function size cannot replace any of our metrics; and where the function size was factored out our metrics performed better than the normalized metrics. The important benefit of our metrics is that they help project managers determine where additional testing effort should be spent and possibly which fault-prone functions should be assigned to more experienced programmers if modifications are required. Copyright © 2000 John Wiley & Sons, Ltd.

Programmers working on the year 2000 problem need to locate and understand date sensitive code, that is, code whose execution depends on date inputs. This paper presents several dynamic analysis methods for addressing this problem. Date sensitive code can be located by running many test cases that are identical except for a change in one date value. An advanced test coverage tool can recover each test's execution count vector giving the number of times that each basic block was executed. Comparison of the vectors reveals the blocks whose execution is affected by the input date values. We present several methods of analysing the execution count data to locate date algorithms. One method identifies subdomains, that is, ranges of dates that are processed similarly. Another method involves graphical or Fourier analysis to identify common programming patterns such as leap-year computations. The last method exploits the 28-year regularity in dates to look for anomalies in processing. The methods are illustrated using Bellcore's ATAC testing tool on five C programs that use some of the date encodings that year 2000 maintainers may need to locate and understand. Copyright © 1999 John Wiley & Sons, Ltd.

Three metrics were used to extract design information from existing code to identify structural stress points in a software system being analyzed: D/sub i/, an internal design metric which incorporates factors related to a module's internal structure; D/sub e/, an external design metric which focuses on a module's external relationships to other modules in the software system; and D(G), a composite design metric which is the sum of D/sub i/ and D/sub e/. Since stress point modules generally have a high probability for being fault-prone, project managers can use the information to determine where additional testing effort should be spent and assign these modules to more experienced programmers if modifications are needed. To make the analysis more accurate and efficient, a design metrics analyzer (/sub /spl chi//Metrics) was implemented. We conducted experiments using /sub /spl chi//Metrics on part of a distributed software system, written in C, with a client-server architecture, and identified a small percentage of its functions as good candidates for fault proneness. Files containing these functions were then validated by the real defect data collected from a recent major release to its next release for their fault proneness. Normalized metrics values were also computed by dividing the D/sub i/, D/sub e/, and D(G) values by the corresponding function size determined by non-blank and non-comment lines of code to study the possible impact of function size on these metrics. Results indicate that function size has little impact on the predictive quality of our design metrics in identifying fault-prone functions.

The purpose of the article is to implement the idea of using testability to estimate software reliability. The basic steps involve estimating testability, evaluating how well software was written, and assessing the effectiveness of testing. Results from these steps along with operational profiles are used to estimate software reliability. The paper describes an application of this method to evaluate the reliability of a real software system of about 6000 lines of executable code and discusses the results of such an estimation. The results are also compared with those obtained by using two reliability growth models.

Given a test set T to test a program P, there are at least two attributes of T that determine its fault detection effectiveness. One attribute is the size of T measured as the number of test cases in T. Another attribute is the code coverage measured when P is executed on all elements of T. The fault detection effectiveness of T is the ratio of the number of faults guaranteed to result in program failure when P is executed on T to the total number of faults present in P. An empirical study was conducted to determine the relative importance of the size and coverage attributes in affecting the fault detection effectiveness of a randomly selected test set for some program P. Results from this study indicate that as the size of a test set is reduced, while the code coverage is kept constant, there is little or no reduction in the fault detection effectiveness of the new test set so generated. For the study reported, of the two attributes mentioned above, the code coverage attribute of a test set is more important than its size attribute. © 1998 John Wiley & Sons, Ltd.

An important question in software testing is whether it is reasonable to apply coverage based criteria as a filter to reduce the size of a test set. An empirical study was conducted using a test set minimization technique to explore the effect of reducing the size of a test set, while keeping block coverage constant, on the fault detection strength of the resulting minimized test set. Two types of test sets were examined. For those with respect to a fixed size, no test case screening was conducted during the generation, whereas for those with respect to a fixed coverage, each subsequent test case had to improve the overall coverage in order to be included. The study reveals that no matter how a test set is generated (with or without any test case screening) block minimized test sets have a size/effectiveness advantage, in terms of a significant reduction in test set size but with almost the same fault detection effectiveness, over the original non-minimized test sets.

The purpose of regression testing is to ensure that changes made to software, such as adding new features or modifying existing features, have not adversely affected features of the software that should not change. Regression testing is usually performed by running some, or all, of the test cases created to test modifications in previous versions of the software. Many techniques have been reported on how to select regression tests so that the number of test cases does not grow too large as the software evolves. Our proposed hybrid technique combines modification, minimization and prioritization-based selection using a list of source code changes and the execution traces from test cases run on previous versions. This technique seeks to identify a representative subset of all test cases that may result in different output behavior on the new software version. We report our experience with a tool called ATAC (Automatic Testing Analysis tool in C) which implements this technique.

Presents a technique that uses coverage measures in reliability estimation for fault-tolerant programs, particularly N-version software. This technique exploits both coverage and time measures collected during testing phases for the individual program versions and the N-version software system for reliability prediction. The application of this technique to single-version software was presented in our previous research (IEEE 3rd Int. Symp. on Software Metrics, Berlin, Germany, March 1996). In this paper, we extend this technique and apply it on the N-version programs. The results obtained from the experiment conducted on an industrial project demonstrate that our technique significantly reduces the hazard of reliability overestimation for both single-version and multi-version fault-tolerant software systems.

Article Free AccessEffect of test set minimization on fault detection effectiveness Authors: W. Eric Wong Bell Communications Research, Morristown, NJ Bell Communications Research, Morristown, NJView Profile , Joseph R. Horgan Bell Communications Research, Morristown, NJ Bell Communications Research, Morristown, NJView Profile , Saul London Bell Communications Research, Morristown, NJ Bell Communications Research, Morristown, NJView Profile , Aditya P. Mathur Software Engineering Research Center, Department of Computer Sciences, Purdue University, W. Lafayette, IN Software Engineering Research Center, Department of Computer Sciences, Purdue University, W. Lafayette, INView Profile Authors Info & Claims ICSE '95: Proceedings of the 17th international conference on Software engineeringApril 1995Pages 41–50https://doi.org/10.1145/225014.225018Published:23 April 1995Publication History 135citation967DownloadsMetricsTotal Citations135Total Downloads967Last 12 Months94Last 6 weeks14 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Finding a fault in a program is a complex process which involves understanding the program's purpose, structure, semantics, and the relevant characteristics of failure producing tests. We describe a tool which supports execution slicing and dicing based on test cases. We report the results of an experiment that uses heuristic techniques in fault localization.

In a recent article by P.G. Frankl and E.J. Weyuker (see ibid., vol.19, no.3, p.962-75, 1993), results are reported that appear to establish a hierarchy of software test methods based on their respective abilities to detect faults. The methods used by Frankl and Weyuker to obtain this hierarchy constitute a new and important addition to their arsenal of tools. These tools were developed specifically to establish simple, useful comparisons of test data generation methods. This is the latest step in an ambitious test method classification program undertaken by the Frankl and Weyuker and their collaborators. The article discusses the method and goes on to present a reply to the critique.>

Size and code coverage are two important attributes that characterize a set of tests. When a program P is executed on elements of a test set T, we can observe the fault-detecting capacity of T for P. We can also observe the degree to which T induces code coverage on P according to some coverage criterion. We would like to know whether it is the size of T or the coverage of T on P which determines the fault detection effectiveness (FDE) of T for P. We found that there is little or no reduction in the FDE of a test set when its size is reduced while the all-uses coverage is kept constant. These data suggest, indirectly, that coverage is more correlated than the size with the FDE. To further investigate this suggestion, we report an empirical study to compare the statistical correlation between (1) FDE and coverage, and (2) FDE and the size. Results from our experiments indicate that the correlation between FDE and block coverage is higher than that between FDE and size.>

Abstract Evaluation of the adequacy of a test set consisting of one or more test cases is a problem oftes encountered in software testing environments. Two test adequacy criiteria are considered, namely the data flow based all‐uses criterion and a mutation based criterion. An empirical study was conducted to compare the ‘difficulty’ of satisfying the two criteria and their costs. Similar studies conducted in the past are discussed in the light of this study. A discussion is also presented of how and why the results of this study, when viewed in conjunction with the results of earlier comparisons of testing methods, are useful to a software test team.