Summary Mutation testing is a very effective testing technique that creates mutants in order to design test cases that will kill the mutants. One problem of mutation testing is high costs: creating mutants, executing mutants and calculating the mutation score. This paper empirically analyzes the effect of covering mutants through the mutant schema improved with extra code (MUSIC) technique. This technique annotates the statements covered by the tests in the original system in order to filter the mutant executions, because tests are only executed against the mutants whose mutated statement is covered by the tests. Therefore, MUSIC is meant to reduce the number of required executions and identify infinite loops at a reduced cost. Besides, an experiment was performed to evaluate the advantages and disadvantages of analyzing the covered mutants. As a result, we conclude that analyzing the covered mutants reduces the execution cost of mutation testing and its application is therefore recommended. Copyright © 2014 John Wiley & Sons, Ltd.

We study the simultaneous test effectiveness and efficiency improvement achievable by Strongly Subsuming Higher Order Mutants (SSHOMs), constructed from 15,792 first order mutants in four Java programs. Using SSHOMs in place of the first order mutants they subsume yielded a 35%-45% reduction in the number of mutants required, while simultaneously improving test efficiency by 15% and effectiveness by between 5.6% and 12%. Trivial first order faults often combine to form exceptionally non-trivial higher order faults; apparently innocuous angels can combine to breed monsters. Nevertheless, these same monsters can be recruited to improve automated test effectiveness and efficiency.

SUMMARY Despite the existing techniques to reduce the costs of mutation analysis, the computational cost to apply mutation testing with large applications can be very high. One effective technique to improve the efficiency of mutation without losing effectiveness is parallel execution, where mutants and tests are executed in parallel processors, reducing the total time needed to perform mutation analysis. This paper presents a study of this technique adapted to current technologies. Five algorithms to execute mutants in parallel are analysed with three studies that use different network configurations and different number of processors with diverse characteristics. The experiments are performed with BacterioP, a tool that is also presented. Unlike previous studies about parallel mutant execution, which date from the mid‐1990s, in the studies in this paper, the communication time in parallel systems no longer acts as a bottleneck. Thus, dynamic strategies, which require more communication, combined with other mutant cost reduction techniques, are the best strategies to run mutants in parallel.Copyright © 2012 John Wiley & Sons, Ltd.

Mutation has been recognized to be an effective software testing technique. It is based on the insertion of artificial faults in the system under test (SUT) by means of a set of mutation operators. Different operators can mutate each program statement in several ways, which may produce a huge number of mutants. This leads to very high costs for test case execution and result analysis. Several works have approached techniques for cost reduction in mutation testing, like n-order mutation where each mutant contains n artificial faults instead of one. There are two approaches to n-order mutation: increasing the effectiveness of mutation by searching for good n-order mutants, and decreasing the costs of mutation testing by reducing the mutants set through the combination of the first-order mutants into n-order mutants. This paper is focused on the second approach. However, this second use entails a risk: the possibility of leaving undiscovered faults in the SUT, which may distort the perception of the test suite quality. This paper describes an empirical study of different combination strategies to compose second-order mutants at system level as well as a cost-risk analysis of n-order mutation at system level.

Mutation testing is one of the most effective techniques to evaluate the quality of a test suites. Due to its tasks and its costs, a tool to automate and reduce costs of mutation testing is required, in order to perform mutation analyses. Bacterio is a Java mutation testing tool that automates the tasks to perform mutation analyses and that implements a set of mutation techniques that reduce the costs of mutation and the execution mutant time drastically. Thus, this tool helps testers to evaluate the quality of their tests, keeping mutation cheap and fast.

Mutation testing is a very effective testing technique that creates mutants (copies of the original system with small syntactic changes) in order to design test cases that kill the mutants (identifying the syntactic changes). The main disadvantage of mutation testing is its high costs: creating mutants, executing mutants and calculating the mutation score. This paper describes the MUSIC technique, an improvement of the cost reduction technique Mutant Schema, which is meant to reduce the number of required executions and identify infinite loops at a reduced cost. Besides, an experiment was performed to evaluate the advantages and disadvantages of MUSIC and identify possible ways to improved it. As a result, we conclude that MUSIC reduces the execution cost of mutation testing and its application is therefore recommended.