Researchers in the microservices community often resort to demonstrating the impact of their proposed advancements on custom-made microservices projects. This is a possible source of bias that can reduce the trustworthiness of the results. Moreover, it is hard to compare advances in small projects, often developed due to lack of time. It is common across disciplines to recognize benchmarks that mitigate bias and unify the advancements' impact. To facilitate the identification of available open-source microservice projects (OSS-MS), we performed a comprehensive study to identify, curate, and catalog OSS-MS. We started with 389559 projects and filtered them down to 3804 projects that we manually labeled. After manual labeling, our dataset contains 378 projects with three or more microservices and with over 100 commits. We document the projects from many perspectives, including project size, platform, number of contributors, project purpose, and foundation support. This dataset can serve researchers as a roadmap to identify benchmarks, as our dataset can be used to answer questions such as whether the number of services impacts the issue count.

Test smells are symptoms of sub-optimal design choices adopted when developing test cases. Previous studies have proved their harmfulness for test code maintainability and effectiveness. Therefore, researchers have been proposing automated, heuristic-based techniques to detect them. However, the performance of these detectors is still limited and dependent on tunable thresholds. We design and experiment with a novel test smell detection approach based on machine learning to detect four test smells. First, we develop the largest dataset of manually-validated test smells to enable experimentation. Afterward, we train six machine learners and assess their capabilities in within- and cross-project scenarios. Finally, we compare the ML-based approach with state-of-the-art heuristic-based techniques. The key findings of the study report a negative result. The performance of the machine learning-based detector is significantly better than heuristic-based techniques, but none of the learners able to overcome an average F-Measure of 51%. We further elaborate and discuss the reasons behind this negative result through a qualitative investigation into the current issues and challenges that prevent the appropriate detection of test smells, which allowed us to catalog the next steps that the research community may pursue to improve test smell detection techniques.

Technical debt management is a critical activity that is gaining the attention of both practitioners and researchers. Several tools providing automatic support for technical debt management have been introduced over the last years. SonarQube is one of the most widely applied tools to automatically measure technical debt in software systems. SonarQube has been adopted to quantify the diffuseness of technical debt in projects of the Apache Software Foundation ecosystem. Lenarduzzi et al. [1] found that the vast majority of technical debt issues in the code are code smells and that, surprisingly, developers tend to take more time to remove severe issues than the less-severe ones. While this study provides very interesting insights both for researchers and practitioners interested in technical debt management, we identified some major limitations that could have led to results that do not perfectly reflect reality. This study aims to address such limitations by presenting a differentiated replication study. Our findings have pointed out significant differences with the reference work. The results show that technical debt issues appear much more rarely than what the reference work reported.In this study, we implemented a new methodology to calculate the diffuseness of SonarQube issues at project and commit level, based on the reconstruction of the SonarQube quality profile in order to understand how the quality profile has evolved and to compare the number of active rules per category and severity level with the respective number of issues found. The results show that over 50% of rules active in the quality profile, are Code Smell rules and that over 90% of the issues belong to Code Smell category. Furthermore, analyzing the life span of the issues, we found that developers take into account the level of severity of the issues only for the Bug category, thus fixing the issues starting from the most severe, which is not the case for the other categories.

Regression testing wants to prevent that errors, which have already been corrected once, creep back into a system that has been updated. A naïve approach consists of re-running the entire test suite (TS) against the changed version of the software under test (SUT). However, this might result in a time-and resource-consuming process; e.g., when dealing with large and/or complex SUTs and TSs. To avoid this problem, Test Case Selection (TCS) approaches can be used. This kind of approaches build a temporary TS comprising only those test cases (TCs) that are relevant to the changes made to the SUT, so avoiding executing unnecessary TCs. In this paper, we introduce CATTO (Commit Adaptive Tool for Test suite Optimization), a tool implementing a TCS strategy for SUTs written in Java as well as a wrapper to allow developers to use CATTO within IntelliJ IDEA and to execute CATTO just-in-time before committing changes to the repository. We conducted a preliminary evaluation of CATTO on seven open-source Java projects to evaluate the reduction of the test-suite size, the loss of fault-revealing TCs, and the loss of fault-detection capability. The results suggest that CATTO can be of help to developers when performing TCS. The video demo and the documentation of the tool is available at: https://catto-tool.github.io/