Continuous integration (CI) is a software engineering practice that advocates the frequent integration of software through an automated build process. Existing research has explored the benefits of CI, such as detecting errors earlier in the software life-cycle. Although CI heavily focuses on automated tests, it is still unclear whether CI is associated with better code coverage, which could be a major benefit of using CI. To investigate whether CI is associated with an improvement in code coverage, our work compares 30 projects that adopted CI (CI projects) and 30 projects that have never adopted CI (NOCI projects). In total, we studied 1,440 versions from different projects to analyze trends in code coverage related to CI. While evaluating trends of code coverage within CI and NOCI projects, we observe more projects with rising trends of code coverage in CI projects (50%) than NOCI projects (10%). Moreover, the maintaining trends are different, as CI projects tend to stabilize at a higher code coverage rate than NOCI projects. Investigating the CI projects alone, the statistical evidences indicate that the adoption of CI is associated with the increase in code coverage. The findings of this study, therefore, reveal a positive association between CI and a higher code coverage rate.

Continuous Integration (CI) is the practice of automating and improving the frequency of code integration. CI has been widely adopted by software development teams and has brought the attention of researchers to study its benefits. Existing research shows that CI can improve software quality by identifying the errors earlier in the software development life-cycle. One question that remains open, however, is whether CI increases the adoption of testing practices in software projects. The goal of our work is to investigate the evolution of software tests and its relationship with the adoption of Continuous Integration. We set out to compare 82 projects that adopted CI (CI projects) and 82 projects that have never adopted CI (NOCI projects). In total, we studied 3,936 versions of our studied projects to investigate trends on the test code ratio and coverage. We observe that 40.2% of the CI projects have a rising test-code ratio trend while only 17% of the NOCI projects have a rising trend. Additionally, we find evidences that CI projects have improved the overall test coverage while NOCI projects do not experience the same growth. Finally, we build a mixed-effects model to study software development factors than can possibly explain the test ratio. Our models reveal that test ratio is largely explained by the project inherent context rather than code or process factors. In overall, our work demonstrates that Continuous Integration can be empirically associated with a healthier test code evolution.