SUMMARY Mutation testing measures the adequacy of a test suite by seeding artificial defects (mutations) into a program. If a test suite fails to detect a mutation, it may also fail to detect real defects—and hence should be improved. However, there are also mutations that keep the program semantics unchanged and thus cannot be detected by any test suite. Such equivalent mutants must be weeded outmanually, which is a tedious task. In this paper, we examine whetherchanges in coveragecan be used to detect non‐equivalent mutants: If a mutant changes the coverage of a run, it is more likely to be non‐equivalent. In a sample of 140 manually classified mutations of seven Java programs with 5000 to 100 000 lines of code, we found that (i) the problem is serious and widespread—about 45% of all undetected mutants turned out to be equivalent; (ii) manual classification takes time—about 15 min per mutation; (iii) coverage is a simple, efficient and effective means to identify equivalent mutants—with a classification precision of 75% and a recall of 56%; and (iv) coverage as an equivalence detector is superior to the state of the art, in particular violations of dynamic invariants. Our detectors have been released as part of the open‐source JAVALANCHE framework; the data set is publicly available for replication and extension of experiments. Copyright © 2012 John Wiley & Sons, Ltd.

SUMMARY A known problem of traditional coverage metrics is that they do not assessoracle quality—that is, whether the computation result is actually checked against expectations. In this paper, we introduce the concept ofchecked coverage—the dynamic slice of covered statements that actually influence an oracle. Our experiments on seven open‐source projects show that checked coverage is a sure indicator for oracle quality and even more sensitive than mutation testing. Copyright © 2013 John Wiley & Sons, Ltd.

A known problem of traditional coverage metrics is that they do not assess oracle quality - that is, whether the computation result is actually checked against expectations. In this paper, we introduce the concept of checked coverage - the dynamic slice of covered statements that actually influence an oracle. Our experiments on seven open-source projects show that checked coverage is a sure indicator for oracle quality - and even more sensitive than mutation testing, its much more demanding alternative.

Mutation testing measures the adequacy of a test suite by seeding artificial defects (mutations) into a program. If a test suite fails to detect a mutation, it may also fail to detect real defects-and hence should be improved. However, there also are mutations which keep the program semantics unchanged and thus cannot be detected by any test suite. Such equivalent mutants must be weeded out manually, which is a tedious task. In this paper, we examine whether changes in coverage can be used to detect non-equivalent mutants: If a mutant changes the coverage of a run, it is more likely to be non-equivalent. In a sample of 140 manually classified mutations of seven Java programs with 5,000 to 100,000 lines of code, we found that: (a) the problem is serious and widespread-about 45% of all undetected mutants turned out to be equivalent; (b) manual classification takes time-about 15 minutes per mutation; (c) coverage is a simple, efficient, and effective means to identify equivalent mutants-with a classification precision of 75% and a recall of 56%; and (d) coverage as an equivalence detector is superior to the state of the art, in particular violations of dynamic invariants. Our detectors have been released as part of the open source JAVALANCHE framework; the data set is publicly available for replication and extension of experiments.

Global and distributed software development increases the need to find and connect developers with relevant expertise. Existing recommendation systems typically model expertise based on file changes (implementation expertise). While these approaches have shown success, they require a substantial recorded history of development for a project. Previously, we have proposed the concept of usage expertise, i.e., expertise manifested through the act of calling (using) a method. In this paper, we assess the viability of this concept by evaluating expert recommendations for the ASPECTJ and ECLIPSE projects. We find that both usage and implementation expertise have comparable levels of accuracy, which suggests that usage expertise may be used as a substitute measure. We also find a notable overlap of method calls across both projects, which suggests that usage expertise can be leveraged to recommend experts from different projects and thus for projects with little or no history.

Mutation testing measures the adequacy of a test suite by seeding artificial defects (mutations) into a program. If a mutation is not detected by the test suite, this usually means that the test suite is not adequate. However, it may also be that the mutant keeps the program's semantics unchanged-and thus cannot be detected by any test. Such equivalent mutants have to be eliminated manually, which is tedious.

To assess the quality of a test suite, one can use mutation testing - seeding artificial defects (mutations) into the program and checking whether the test suite finds them. Javalanche is an open source framework for mutation testing Java programs with a special focus on automation, efficiency, and effectiveness. In particular, Javalanche assesses the impact of individual mutations to effectively weed out equivalent mutants; it has been demonstrated to work on programs with up to 100,000 lines of code.

In software development, there is an increasing need to find and connect developers with relevant expertise. Existing expertise recommendation systems are mostly based on variations of the Line 10 Rule: developers who changed a file most often have the most implementation expertise. In this paper, we introduce the concept of usage expertise, which manifests itself whenever developers are using functionality, e.g., by calling API methods. We present preliminary results for the ECLIPSE project that demonstrate that our technique allows to recommend experts for files with no or little history, identify developers with similar expertise, and measure the usage of API methods.

Code theft is a threat for companies that consider code asa core asset. A birthmark can help them to prove codetheft by identifying intrinsic properties of a program. Twoprograms with the same birthmark are likely to share a com-mon origin. Birthmarking works in particular for code thatwas not protected by tamper-resistant copyright notices thatotherwise could prove ownership.We propose a dynamic birthmark for Java that observes howa program uses objects provided by the Java Standard API.Such a birthmark is difficult to foil because it captures the observable semantics of a program. In an evaluation, ourAPI Birthmark reliably identified XML parsers and PNGreaders before and after obfuscating them with state-of-the-art obfuscation tools. These rendered existing birthmarksineffective, such as the Whole-Program-Path Birthmark byMyles and Collberg