Regression testing is a practice that ensures a System Under Test (SUT) still works as expected after changes have been implemented. The simplest approach for regression testing is Retest-all, which consists of re-executing the entire Test Suite (TS) on the changed version of the SUT. Retest-all could be expensive in case a SUT and its TS grow in size and, if resources are insufficient, its application could be impracticable. A Test Suite Reduction (TSR) approach aims to overcome these issues by reducing the size of TSs, while preserving their fault-detection capability. In this paper, we introduce and validate an approach for TSR based on a multi-objective evolutionary algorithm, namely, Non-dominated Sorting Genetic Algorithm II (NSGA-II). This approach seeks to reduce TSs by maximizing both statement coverage and diversity of test cases of the reduced TSs, while minimizing the size of the reduced TSs. We named this approach Genetic Algorithm for teSt SuitE Reduction (GASSER). To assess GASSER, we conducted an experiment on 19 versions of four software systems from a public dataset—i.e. Software-artifact Infrastructure Repository (SIR). We compared GASSER with nine baseline approaches. The comparison was based on the size of the reduced TSs and their fault-detection capability. The most important take-away result is that GASSER, as compared with the baseline approaches, reduces more the size of the TSs with a non-significant effect on their fault-detection capability. The results of our empirical assessment suggest that the application of multi-objective evolutionary algorithms and, in particular, NSGA-II might represent a viable means to deal with TSR.

Regression testing is an important activity that ensures a System Under Test (SUT) still works as expected after changes. Regression testing can be expensive in case of large Test Suites (TSs). Test Suite Reduction (TSR) approaches speed up regression testing by removing redundant test cases. These approaches can be classified as adequate or inadequate. Adequate approaches reduce TSs so that they completely preserve the test requirements (e.g., statement coverage) of the original TSs. Inadequate approaches produce reduced TSs that only partially preserve test requirements. An inadequate TSR approach is appealing when it leads to a higher reduction in TS size at the expense of a negligible loss in fault-detection capability. We defined an inadequate approach for TSR named GASSER (Genetic Algorithm for teSt SuitE Reduction). It is based on a multi-objective evolutionary algorithm, NSGA-II (Non-dominated Sorting Genetic Algorithm II). GASSER seeks to reduce TSs by maximizing both the statement coverage and diversity of test cases, and minimizing the size of the reduced TSs. We implemented GASSER in a Java prototype of a supporting tool and named it as the approach, namely GASSER. In this tooldemo paper, we present such a tool prototype as well as the results of a preliminary empirical study to assess the validity of both the approach and the tool prototype. A screen-cast of GASSER in action is available at https://youtu.be/20Uf1ugEvAQ.

Background. Regression testing is a practice that ensures a System Under Test (SUT) still works as expected after changes. The simplest regression testing approach is Retest-all, which consists of re-executing the entire Test Suite (TS) on the new version of the SUT. When SUT and its TS grow in size, applying Retest-all could be expensive. Test Suite Reduction (TSR) approaches would allow overcoming the above-mentioned issues by reducing TSs while preserving their fault-detection capability.

During the evolution of software, it could happen that the information in the comments and in the associated source code are not aligned, so hampering the execution of software evolution and maintenance tasks. This kind of misalignment is known as lack of coherence and it can happen for several reasons, e.g., programmers modify the intent of source code while executing a maintenance task without updating its comment accordingly. We study the problem of detecting a lack of coherence between comments and source code by exploiting Word Embeddings (WEs). We present four models based on WE and tested these models using six different WE variants through an experiment conducted on a publicly available dataset. Results are compared against a baseline. The most important outcome is: the considered models and WE variants are more efficient in terms of execution time while maintaining performance very close to the baseline. The explanation for such an improvement is that WEs are able to concentrate the important information in a more compact input representation.

I present here a part of the research conducted in my Ph.D. course. In particular, I focus on my ongoing work on how to support testing in the context of Continuous Integration (CI) development by distributing the execution of test cases (TCs) on geographically dispersed servers. I show how to find a trade-off between the cost of leased servers and the time to execute a given test suite (TS). The distribution and the execution of TCs on servers is modeled as a multi-objective optimization problem, where the goal is to balance the cost to lease servers and the time to execute TCs. The preliminary results : (i) show evidence of the existence of a Pareto Front (trade-off between costs to lease servers and TCs time) and (ii) suggest that the found solutions are worthwhile as compared to a traditional non-distributed TS execution (i.e., a single server/PC). Although the obtained results cannot be considered conclusive, it seems that the solutions are worth to speed up the testing activities in the context of CI.

Background. Regression testing is conducted after changes are made to a system in order to ensure that these changes did not alter its expected behavior. The problem with regression testing is that it can require too much time and/or too many resources. This is why researchers have defined a number of regression testing approaches. Among these, Test Suite Reduction (TSR) approaches reduce the size of the original test suites, while preserving their capability to detect faults. TSR approaches can be classified as adequate or inadequate. Adequate approaches reduce test suites so that they completely preserve the test requirements (e.g., statement coverage) of the original test suite, while inadequate ones produce reduced test suites that partially preserve these test requirements.

Regression testing is an important activity that can be expensive (e.g., for large test suites). Test suite reduction approaches speed up regression testing by removing redundant test cases. These approaches can be classified as adequate or inadequate. Adequate approaches reduce test suites so that they completely preserve the test requirements (e.g., code coverage) of the original test suites. Inadequate approaches produce reduced test suites that only partially preserve the test requirements. An inadequate approach is appealing when it leads to a greater reduction in test suite size at the expense of a small loss in fault-detection capability. We investigate a clustering-based approach for inadequate test suite reduction and compare it with well-known adequate approaches. Our investigation is founded on a public dataset and allows an exploration of trade-offs in test suite reduction. Results help a more informed decision, using guidelines defined in this research, to balance size, coverage, and fault-detection loss of reduced test suites when using clustering.