Recently, tools for the analysis and visualization of code coverage have become widely available. At first glance, their value in assessing and improving the quality of automated test suites seems to be obvious. Yet, experimental studies as well as experience from projects in industry indicate that their use is not without pitfalls. We found these tools in a number of recent projects quite beneficial. Therefore, we set out to gather code coverage information from one of these projects. In this experience report, first the system under scrutiny as well as our methodology is described. Then, four major questions concerning the impact and benefits of using these tools are discussed. Furthermore, a list of ten lessons learned is derived. The list may help developers judiciously use code coverage tools, in order to reap a maximum of benefits.

This report addresses some of our observations made in a dozen of projects in the area of software testing, and more specifically, in automated testing. It documents, analyzes and consolidates what we consider to be of interest to the community. The major findings can be summarized in a number of lessons learned, covering test strategy, testability, daily integration, and best practices.The report starts with a brief description of five sample projects. Then, we discuss our observations and experiences and illustrate them with the sample projects. The report concludes with a synopsis of these experiences and with suggestions for future test automation endeavors.