Mutation testing realises the idea of fault-based testing, i.e., using artificial defects to guide the testing process. It is used to evaluate the adequacy of test suites and to guide test case generation. It is a potentially powerful form of testing, but it is well-known that its effectiveness is inhibited by the presence of equivalent mutants. We recently studied Trivial Compiler Equivalence (TCE) as a simple, fast and readily applicable technique for identifying equivalent mutants for C programs. In the present work, we augment our findings with further results for the Java programming language. TCE can remove a large portion of all mutants because they are determined to be either equivalent or duplicates of other mutants. In particular, TCE equivalent mutants account for 7.4 and 5.7 percent of all C and Java mutants, while duplicated mutants account for a further 21 percent of all C mutants and 5.4 percent Java mutants, on average. With respect to a benchmark ground truth suite (of known equivalent mutants), approximately 30 percent (for C) and 54 percent (for Java) are TCE equivalent. It is unsurprising that results differ between languages, since mutation characteristics are language-dependent. In the case of Java, our new results suggest that TCE may be particularly effective, finding almost half of all equivalent mutants.

Mutation testing is considered as one of the most powerful testing methods. It operates by asking testers to design tests that reveal a set of mutants, which are purpose-made injected defects. Evidently, the strength of the method strongly depends on the used mutants. However, this dependence raises concerns regarding the mutation testing practice that is implemented by existing tools. Thus, it is probable that implementation inadequacies can lead to incompetent results. In this paper, we cross-evaluate three popular mutation testing tools for Java, namely MUJAVA, MAJOR and PIT, with respect to their effectiveness. We perform an empirical study of 3,324 manually analysed mutants from real-world projects and we find that there are large differences between the tools' effectiveness, ranging from 76% to 88%, with MUJAVA achieving the best results. We also demonstrate that no tool is able to subsume the others and provide practical recommendations on how to strengthen each one of the studied tools. Finally, our analysis shows that 11%, 12% and 7% of the mutants generated by MUJAVA, MAJOR and PIT are equivalent, respectively.

Summary The equivalent mutant problem is a major hindrance to mutation testing. Being undecidable in general, it is only susceptible to partial solutions. In this paper, mutant classification is utilised for isolating likely to be first‐order equivalent mutants. A new classification technique,Isolating Equivalent Mutants (I‐EQM), is introduced and empirically investigated. The proposed approach employs a dynamic execution scheme that integrates theimpacton the program execution of first‐order mutants with the impact on the output of second‐order mutants. An experimental study, conducted using two independently created sets of manually classified mutants selected from real‐world programs revalidates previously published results and provides evidence for the effectiveness of the proposed technique. Overall, the study shows that I‐EQM substantially improves previous methods by retrieving a considerably higher number of killable mutants, thus, amplifying the quality of the testing process. Copyright © 2014 John Wiley & Sons, Ltd.

Equivalent mutants are one of the major costs of mutation testing. The undecidable nature of this problem makes a fully automated solution unattainable and necessitates the manual analysis of live mutants. This paper introduces the concept of mirrored mutants, ones that affect similar code fragments. It is argued that mirrored mutants exhibit analogous behavior with respect to their equivalence. Thus, if one of them is equivalent, then the other mirrored mutants should be too. An empirical study, conducted on real world programs, investigates this argument, focusing on both intra-method and inter-method mirrored mutants. The obtained results suggest that mirrored mutants indeed exhibit this kind of behavior and thus can be utilized to ameliorate the adverse effects of the equivalent mutant problem.

In this paper, a technique named I-EQM, able to dynamically isolate first order equivalent mutants, is proposed. I-EQM works by employing a novel dynamic execution scheme that integrates both first and second order mutation. The proposed approach combines the "impact" on the program execution of the first order mutants with the "impact" on the output of second order ones, to isolate likely to be first order equivalent mutants. Experimental results on a benchmark set of manually classified mutants, selected from real word programs, reveals that I-EQM achieves to classify equivalent mutants with a 71% and 82% classification precision and recall respectively. These results improve the previously proposed approaches by selecting (retrieving) a considerably higher number of killable mutants with only a limited loss on the classification precision.

Mutation testing while being a successful fault revealing technique for unit testing, it is a rather expensive one for practical use. To bridge these two aspects there is a need to establish approximation techniques able to reduce its expenses while maintaining its effectiveness. In this paper several second order mutation testing strategies are introduced, assessed and compared along with weak mutation against strong. The experimental results suggest that they both constitute viable alternatives for mutation as they establish considerable effort reductions without greatly affecting the test effectiveness. The experimental assessment of weak mutation suggests that it reduces significantly the number of the produced equivalent mutants on the one hand and that the test criterion it provides is not as weak as is thought to be on the other. Finally, an approximation of the number of first order mutants needed to be killed in order to also kill the original mutant set is presented. The findings indicate that only a small portion of a set of mutants needs to be targeted in order to be killed while the rest can be killed collaterally.

Software testing forms a substantial activity of the software development cycle. Although important, it lacks from being automated mainly because of the various undecidable problems that it encounters. To this extend efficient heuristics have been proposed in order to bypass this problem. One such approach, called symbolic execution, is usually used for automating the test data generation activity. In this paper, an automated symbolic execution tool is proposed. The tool employs an efficient path heuristic, integrated with random testing for producing test cases. The tool handles the path explosion and constraint solving problems efficiently. This is achieved by targeting on specific likely to be feasible paths and by using a linear programming approach for the determination of their feasibility. Preliminary results are very encouraging as they show that a high coverage can be achieved within a limited amount of time-effort.

The automatic test case generation is the principal issue of the software testing activity. Dynamic symbolic execution appears to be a promising approach to this matter as it has been shown to be quite powerful in producing the sought tests. Despite its power, it has only been effectively applied to the entry level criteria of the structural criteria hierarchy such as branch testing. In this paper an extension of this technique is proposed in order to effectively generate test data based on mutation testing. The proposed approach conjoins program transformation and dynamic symbolic execution techniques in order to successfully automate the test generation process. The propositions made in this paper have been incorporated into an automated framework for producing mutation based test cases. Its evaluation on a set of benchmark programs suggests that it is able to produce tests capable of killing most of the non equivalent introduced mutants. The same study also provides some evidence that by employing efficient heuristics it can be possible to perform mutation with reasonable resources.

Mutation testing has been identified as one of the most effective techniques, in detecting faults. However, because of the large number of test elements that it introduces, it is regarded as rather expensive for practical use. Therefore, there is a need for testing strategies that will alleviate this drawback by selecting effective test data that will make the technique more practical. Such a strategy based on path selection is reported in this paper. A significant influence on the efficiency associated with path selection strategies is the number of test paths that must be generated in order to achieve a specified level of coverage, and it is determined by the number of paths that are found to be feasible. Specifically, a path selection strategy is proposed that aims at reducing the effects of infeasible paths and conversely developing effective and efficient mutation based tests. The results obtained from applying the method to a set of program units are reported and analysed presenting the flexibility, feasibility and practicality of the proposed approach.

Abstract Introduced in 1976 by Hennell, Woodward, and Hedley, the Linear Code Sequence and Jump (LCSAJ) has since been named the jump‐to‐jump path (JJ‐path). If all JJ‐paths in a piece of code have been tested, then it is guaranteed, for example, that all of the code's branches and all of its statements will likewise have been tested. JJ‐path testing is thus said to include both branch and statement testing. Over the years, much work has been carried out on the inclusion relation, and this is also true of a closely‐linked relation that is known as subsumption. Not surprisingly, some of the work in the literature has focussed on the criteria of executing all JJ‐paths and sequences of such, and how these criteria relate to other path coverage and test data criteria. Unfortunately, certain results involving JJ‐paths in the context of inclusion, as portrayed in a widely referenced and influential paper, are in error. Consequently, the main purpose of this paper is to rectify this situation. Copyright © 2008 John Wiley & Sons, Ltd.

Testing multi-tier database applications can be viewed as a distributed task performed at each tier by a number of agents. From this point of view, an agent based approach, originally introduced for the software maintenance of such applications, is extended to provide an effective, intelligent and extensible solution suitable for their testing.

Database systems lie at the core of almost every modern software application. The interaction between the application source code and the underlying database schema results in a dependency relationship that affects the application 's maintainability by raising a number of additional maintenance issues. To assess this effect and facilitate the maintenance process, a software engineering approach based on software agents is introduced. The distributed and cooperative nature of a software agent system provides the flexibility required to analyze modern multi-tier database applications such as web-based applications. A prototype system, which employs agent architecture in order to satisfy the requirements of the suggested approach, is presented.

In the light of the extraordinary and continuous growth of the Web, there is a significant development shift from traditional software applications to web applications. The spread of this type of applications raises a demand for new approaches towards their testing and maintenance. Contrary to most previous proposals our interest focuses on the interaction between a web application and its underlying database. This paper describes an approach for the estimation of the impact that database schema modifications imply to the operability of a web application. In addition, a software tool called DaSIAn that is based upon this proposal is presented and its experimental results are being discussed.

Test data generation is one of the hardest tasks in the software life-cycle. Many testing methods try to answer this question, all in a heuristic way. Symbolic execution is one such software testing method that can be used either for program evaluation or in order to assist the automated test data generation process. A number of systems employing symbolic execution for test data generation have already been built. In this paper, a new symbolic execution system is presented, which can be used regardless of the language in which the program under test is written. The system is called VOLCANO and the scripts are written in SYMEXLAN (SYMbolic EXecution LANguage), a scripting language that can be either an intermediate representation for many other languages or a symbolic execution language that facilitates the symbolic execution process.