Many software production processes advocate rigorous development testing alongside functional code writing, which implies that both test code and production code should co-evolve. To gain insight in the nature of this co-evolution, this paper proposes three views (realized by a tool called TeMo) that combine information from a software project’s versioning system, the size of the various artifacts and the test coverage reports. We validate these views against two open source and one industrial software project and evaluate our results both with the help of log messages, code inspections and the original developers of the software system. With these views we could recognize different co-evolution scenarios (i.e., synchronous and phased) and make relevant observations for both developers as well as test engineers.

Understanding software through dynamic analysis has been a popular activity in the past decades. One of the most common approaches in this respect is execution trace analysis: among our own efforts in this context is EXTRAVIS, a tool for the visualization of large traces. Similar to other trace visualization techniques, our tool has been validated through anecdotal evidence, but should also be quantitatively evaluated to assess its usefulness for program comprehension. This paper reports on a first controlled experiment concerning trace visualization for program comprehension. We designed eight typical tasks aimed at gaining an understanding of a representative subject system, and measured how a control group (using the Eclipse IDE) and an experimental group (using both Eclipse and EXTRAVIS) performed in terms of correctness and time spent. The results are statistically significant in both regards, showing a 21% decrease in time and a 43% increase in correctness for the latter group.

Engineering software systems is a multidisciplinary activity, whereby a number of artifacts must be created - and maintained - synchronously. In this paper we investigate whether production code and the accompanying tests co- evolve by exploring a project's versioning system, code coverage reports and size-metrics. Our main aim for studying this co-evolution is to create awareness with developers and managers alike about the testing process that is followed. We explore the possibilities of our technique through two open source case studies and observe a number of different co-evolution scenarios. We evaluate our results both with the help of log-messages and the original developers of the software system.

As a fine-grained defect detection technique, unit testing introduces a strong dependency on the structure of the code. Accordingly, test coevolution forms an additional burden on the software developer which can be tempered by writing tests in a manner that makes them easier to change. Fortunately, we are able to concretely express what a good test is by exploiting the specific principles underlying unit testing. Analogous to the concept of code smells, violations of these principles are termed test smells. In this paper, we clarify the structural deficiencies encapsulated in test smells by formalizing core test concepts and their characteristics. To support the detection of two such test smells, General Fixture and Eager Test, we propose a set of metrics defined in terms of unit test concepts. We compare their detection effectiveness using manual inspection and through a comparison with human reviewing. Although the latter is the traditional means for test quality assurance, our results indicate it is not a reliable means for test smell detection. This work thus stresses the need for a more reliable detection mechanism and provides an initial contribution through the validation of test smell metrics.

Test code, just like any other code we write, erodes when frequently changed. As such, refactoring, which has been shown to impact maintainability and comprehensibility, can be part of a solution to counter this erosion. We propose a metric-based heuristical approach, which allows to rank occurrences of so-called test smells (i.e. symptoms of poorly designed tests) according to their relative significance. This ranking can subsequently be used to start refactoring. Through an open-source case study, ArgoUML, we demonstrate that we are able to identify those test cases who violate unit test criteria