In a test suite, all the test cases should be independent: no test should affect any other test’s result, and running the tests in any order should produce the same test results. Techniques such as test prioritization generally assume that the tests in a suite are independent. Test dependence is a little-studied phenomenon. This paper presents five results related to test dependence.

Debugging and isolating changes responsible for regression test failures are some of the most challenging aspects of modern software development. Automatic bug localization techniques reduce the manual effort developers spend examining code, for example, by focusing attention on the minimal subset of recent changes that results in the test failure, or on changes to components with most dependencies or highest churn. We observe that another subset of changes is worth the developers' attention: the complement of the maximal set of changes that does not produce the failure. While for simple, independent source-code changes, existing techniques localize the failure cause to a small subset of those changes, we find that when changes interact, the failure cause is often in our proposed subset and not in the subset existing techniques identify. In studying 45 regression failures in a large, open-source project, we find that for 87% of those failures, the complement of the maximal passing set of changes is different from the minimal failing set of changes, and that for 78% of the failures, our technique identifies relevant changes ignored by existing work. These preliminary results suggest that combining our ideas with existing techniques, as opposed to using either in isolation, can improve the effectiveness of bug localization tools.

Unit testing aims to ensure that methods correctly implement the specified and implied pre- and post-conditions, while integration testing ensures that modules correctly follow interaction protocols. While the generation of unit test cases has been explored extensively in the literature, there is still little work on the generation of integration test cases. In this paper we present a new technique to generate integration test cases that leverages existing unit test cases. Our key observation is that both, unit and integration testing, use method calls as the atoms to construct test cases from. Unit tests contain information on how to instantiate classes in meaningful ways, how to construct arguments for method calls, and what the resulting system state should be after calling methods with those arguments. We use this information to construct more complex test cases that focus on class interactions rather than on individual state transformations caused by single method calls. This paper presents the approach and shows that the generated test cases can find interesting faults, compared to test suites generated with state of the art approaches.

We present an approach to augment test suites with automatically generated integration test cases. Our approach utilizes existing test cases to direct generation towards producing complex object interactions and execution sequences that have not been observed before.

Even in simple programs there are hidden assumptions and dependencies between units that are not immediately visible in each involved unit. These dependencies are generally hard to identify and locate, and can lead to subtle faults that are often missed, even by extensive test suites.