Test smells can pose difficulties during testing activities, such as poor maintainability, non-deterministic behavior, and incomplete verification. Existing research has extensively addressed test smells in automated software tests but little attention has been given to smells in natural language tests. While some research has identified and catalogued such smells, there is a lack of systematic approaches for their removal. Consequently, there is also a lack of tools to automatically identify and remove natural language test smells. This paper introduces a catalog of transformations designed to remove seven natural language test smells and a companion tool implemented using Natural Language Processing (NLP) techniques. Our work aims to enhance the quality and reliability of natural language tests during software development. The research employs a two-fold empirical strategy to evaluate its contributions. First, a survey involving 15 software testing professionals assesses the acceptance and usefulness of the catalog’s transformations. Second, an empirical study evaluates our tool to remove natural language test smells by analyzing a sample of real-practice tests from the Ubuntu OS. The results indicate that software testing professionals find the transformations valuable. Additionally, the automated tool demonstrates a good level of precision, as evidenced by a F-Measure rate of 83.70%.

Background: Test smells indicate potential problems in the design and implementation of automated software tests that may negatively impact test code maintainability, coverage, and reliability. When poorly described, manual tests written in natural language may suffer from related problems, which enable their analysis from the point of view of test smells. Despite the possible prejudice to manually tested software products, little is known about test smells in manual tests, which results in many open questions regarding their types, frequency, and harm to tests written in natural language. Aims: Therefore, this study aims to contribute to a catalog of test smells for manual tests. Method: We perform a two-fold empirical strategy. First, an exploratory study in manual tests of three systems: the Ubuntu Operational System, the Brazilian Electronic Voting Machine, and the User Interface of a large smartphone manufacturer. We use our findings to propose a catalog of eight test smells and identification rules based on syntactical and morphological text analysis, validating our catalog with 24 in-company test engineers. Second, using our proposals, we create a tool based on Natural Language Processing (NLP) to analyze the subject systems' tests, validating the results. Results: We observed the occurrence of eight test smells. A survey of 24 in-company test professionals showed that 80.7% agreed with our catalog definitions and examples. Our NLP-based tool achieved a precision of 92%, recall of 95%, and f-measure of 93.5%, and its execution evidenced 13,169 occurrences of our cataloged test smells in the analyzed systems. Conclusion: We contribute with a catalog of natural language test smells and novel detection strategies that better explore the capabilities of current NLP mechanisms with promising results and reduced effort to analyze tests written in different idioms.

Test smells are symptoms in the test code that indicate possible design or implementation problems. Previous research demonstrated their harmfulness and the developers’ acknowledgment of test smells’ effects, prevention, and refactoring strategies. Test automation frameworks are constantly evolving, and the JUnit, one of the most used ones for Java projects, has its version 5 available since late 2017. However, we do not know the extent to which developers use the newly introduced features and whether such features indeed help refactor existing test code to remove test smells. This article conducts a mixed-method study investigation to minimize these knowledge gaps. Our study consists of three parts. First, we evaluate the usage of this framework and its features by analyzing the source code of 485 popular Java open-source projects on GitHub that use JUnit. We found that 15.9% of these projects use the JUnit 5 library. We also found that, from 17 new features detected in use, only 3 (i.e., 17.6%) are responsible for more than 70% of usages, limiting optimized propositions to test code creation and maintenance. Second, after identifying features in the JUnit 5 framework that could be considered to test smells removal and prevention, we use these features to propose novel refactorings. In particular, we present refactorings based on 7 introduced JUnit 5 features that help to remove 13 test smells, such as Assertion Roulette, Test Code Duplication, and Conditional Test Logic. Third, to evaluate our refactorings with the opinions of experienced developers, we (i) survey 212 developers for their preferences and comments about our refactorings, corroborating the benefits of our proposals and raising community feedback on JUnit 5 features, and (ii) we refactor actual test code from popular GitHub Java projects and submit 38 Pull Requests, reaching a 94% acceptance rate among respondents. As implications of our study, we alert the software testing community (i.e., practitioners and researchers) to the need to study the JUnit 5 features to effectively remove and prevent test smells. To better assist this process, we give directions on how test smells can be refactored using such features.

Applying Machine learning algorithms to data streams is a challenging task because traditional strategies suppose datasets to be labeled, finite and stationary. In the context of data streams, where the data is generated in real-time and the labels may be missing due to the high cost of the labeling process, the proposal of semi-supervised learning (SSL) strategies to learn from labeled and unlabeled data at the same time seems to be a viable solution, despite also being challenging. In this paper, we present a novel approach to handle missing labels for classification learning in data streams, named co-op training, which is based on self-training incremental and co-training. In a controlled experiment, we execute the proposed algorithm, along with most well-known semi-supervised learning strategies, in 11 artificial and real-world datasets, and compare the results. We found our strategy to be more accurate than the other SSL algorithms in most datasets, also presenting better run-times when accuracies were similar. These methods are implemented in the Massive Online Analysis (MOA) open-source software as an internal benchmark component, to help researchers to run experimental comparisons on semi-supervised learning on data streams easily.