Background: Mocking is a common technique for isolating test dependencies, yet duplicated mock setup code-what we call “mock clones”-can increase test maintenance overhead and reduce clarity. While code clone detection is a mature research area, the prevalence, characteristics, and refactoring of mock clones remain insufficiently understood.Aims: This study investigates the prevalence, detection challenges, and practical elimination of mock clones in real-world Java test suites, aiming to identify systematic patterns and assess the feasibility and value of mock clone refactoring.Method: We analyzed 698 mock clone instances across six open-source Java projects. We evaluated the effectiveness of existing code clone detection tools in capturing mock clones, manually refactored a large subset of clones to assess behavior preservation, and measured the structural impact of mock clone elimination.Results: Mock clones are prevalent, with frequently mocked classes often exhibiting extensive duplication. Existing clone detection tools failed to reliably detect mock clones due to scattered, reordered, and semantically varied mocking logic. Manual refactoring successfully eliminated $60 \%$ of identified mock clones while preserving test behavior, resulting in up to $64 \%$ reduction in mock objects and up to $61 \%$ reduction in mocking-related LOC. Targeting only the top $\mathbf{5 - 1 0}$ high-impact clones still achieved substantial simplification.Conclusions: Mock clone elimination is both feasible and highly valuable for improving test code maintainability. Systematic refactoring strategies can address common patterns, and prioritizing high-impact clones enables efficient gains with minimal effort. These findings motivate the need for mock-aware detection and automated refactoring tools to unlock broader maintainability improvements in testing practices.

The AAA (Arrange, Act, Assert) pattern provides a unified structure for unit test cases, potentially benefiting comprehension and maintenance. However, its adoption and implementation in practice remain insufficiently understood. This study investigates the prevalence of AAA pattern usage, identifies recurring deviations and design issues within AAA structures, and assesses developers’ receptiveness to AAA-based improvements. We conducted an empirical study on 735 real-life unit test cases randomly selected from seven open-source projects. We manually analyzed these test cases, identified AAA-related issues, and proposed fixes to developers. Our analysis found that 77% of test cases follow the AAA structure. We identified three recurring patterns deviating from AAA and four design issues within A blocks. Comparison with classic test smells revealed unique insights provided by AAA analysis. Of 27 improvement proposals sent to developers, 78% received positive feedback. These findings show that the AAA pattern is widely adopted in practice, but deviations from and design issues within AAA patterns are common. Our analysis provides a novel perspective on test case quality, complementing traditional test smell analysis. The high acceptance rate of our improvement proposals suggests that developers value AAA-based enhancements. These findings can guide the development of tools for improving AAA practice in unit tests.

Software performance is critical for system efficiency, with performance issues potentially resulting in budget overruns, project delays, and market losses. Such problems are reported to developers through issue tracking systems, which are often under-tagged, as the manual tagging process is voluntary and time-consuming. Existing automated performance issue tagging techniques, such as keyword matching and machine/deep learning models, struggle due to imbalanced datasets and a high degree of variance. This paper presents a novel hybrid classification approach, combining Heuristic Linguistic Patterns (HLPs) with machine/deep learning models to enable practitioners to automatically identify performance-related issues. The proposed approach works across three progressive levels:HLPtagging, sentence tagging, and issue tagging, with a focus on linguistic analysis of issue descriptions. The authors evaluate the approach on three different datasets collected from different projects and issue-tracking platforms to prove that the proposed framework is accurate, project- and platform-agnostic, and robust to imbalanced datasets. Furthermore, this study also examined how the two unique techniques of the framework, including the fuzzyHLPmatching and theIssue HLP Matrix, contribute to the accuracy. Finally, the study explored the effectiveness and impact of two off-the-shelf feature selection techniques,BorutaandRFE, with the proposed framework. The results showed that the proposed framework has great potential for practitioners to accurately (with up to 100% precision, 66% recall, and 79%F1-score) identify performance issues, with robustness to imbalanced data and good transferability to new projects and issue tracking platforms.

Unit testing focuses on verifying the functions of individual units of a software system. It is challenging due to the high inter dependencies among software units. Developers address this by mocking—replacing the dependency by a “fake” object. Despite the existence of powerful, dedicated mocking frameworks, developers often turn to a “hand-rolled” approach—inheritance. That is, they create a subclass of the dependent class and mock its behavior through method overriding. However, this requires tedious implementation and compromises the design quality of unit tests. This work contributes a fully automated refactoring framework to identify and replace the usage of inheritance by using Mockito—a well received mocking framework. Our approach is built upon the empirical experience fromfiveopen source projects that use inheritance for mocking. We evaluate our approach onnineother projects. Results show that our framework is efficient, generally applicable to new datasets, mostly preserves test case behaviors in detecting defects (in the form of mutants), and decouples test code from production code. The qualitative evaluation by experienced developers suggests that the auto-refactoring solutions generated by our framework improve the quality of the unit test cases in various aspects, such as making test conditions more explicit, as well as improved cohesion, readability, understandability, and maintainability with test cases. Finally, we submit 23 pull requests containing our refactoring solutions to the open source projects. It turns our that, 9 requests are accepted/merged, 6 requests are rejected, the remaining requests are pending (5 requests), with unexpected exceptions (2 requests), or undecided (1 request). In particular, among the 21 open source developers that are involved in the reviewing process, 81% give positive votes. This indicates that our refactoring solutions are quite well received by the open source projects and developers.

TheAAApattern (i.e.,Arrange-Act-Assert) is a common and natural layout to create a test case. Following this pattern in test cases may benefit comprehension, debugging, and maintenance. TheAAAstructure of real-life test cases, however, may not be clear due to their high complexity. Manually labelingAAAstatements in test cases is tedious. Thus, we envision that an automated approach for labelingAAAstatements in existing test cases could benefit new developers and projects that practice collective code ownership and test-driven development. This paper contributes an automatic approach based on machine learning models. The “secret sauce” of this approach is a set of three learning features that are based on the semantic, syntax, and context information in test cases, derived from the manual tagging process. Thus, our approach mimics how developers may manually tag theAAApattern of a test case. We assess the precision, recall, and F-1 score of our approach based on 449 test cases, containing about 16,612 statements, across 4 Apache open source projects. To achieve the best performance in our approach, we explore the usage of six machine learning models; the contribution of the SMOTE data balancing technique; the comparison of the three learning features; and the comparison of five different methods for calculating the semantic feature. The results show our approach is able to identifyArrangement,Action, andAssertionstatements with a precision upwards of 92%, and recall up to 74%. We also summarize some experience based on our experiments—regarding the choice of machine learning models, data balancing algorithm, and feature engineering methods—which could potentially provide some reference to related future research.

The AAA pattern, i.e. the Arrangement, Action, and Assertion, is a common and nature layout to create a test case. Following this pattern in test cases may benefit comprehension, debugging, and maintenance. The AAA structure of real-life test cases may not be explicit due to its high complexity. Manually labeling AAA statements in test cases is tedious. Thus, an automated approach for labeling AAA statements in existing test cases could benefit new developers and projects that practice collective code ownership and test driven development.

Unit testing focuses on verifying the functions of individual units of a software system. It is challenging due to the high inter-dependencies among software units. Developers address this by mocking-replacing the dependency by a "faked" object. Despite the existence of powerful, dedicated mocking frameworks, developers often turn to a "hand-rolled" approach-inheritance. That is, they create a subclass of the dependent class and mock its behavior through method overriding. However, this requires tedious implementation and compromises the design quality of unit tests. This work contributes a fully automated refactoring framework to identify and replace the usage of inheritance by using Mockito-a well received mocking framework. Our approach is built upon the empirical experience from five open source projects that use inheritance for mocking. We evaluate our approach on four other projects. Results show that our framework is efficient, generally applicable to new datasets, mostly preserves test case behaviors in detecting defects (in the form of mutants), and decouples test code from production code. The qualitative evaluation by experienced developers suggests that the auto-refactoring solutions generated by our framework improve the quality of the unit test cases in various aspects, such as making test conditions more explicit, as well as improved cohesion, readability, understandability, and maintainability with test cases.

Performance issues compromise the response time and resource consumption of a software system. Modern software systems use issue tracking systems to manage all kinds of issue reports, including performance issues. The problem is that performance issues are often not explicitly tagged. The tagging mechanism, if exists, is completely voluntary, depending on the project’s convention and on submitters’ discipline. For example, the performance tag rate in Apache’s Jira system is below 1%. This paper contributes a hybrid classification approach that combines linguistic patterns and machine/deep learning techniques to automatically detect performance issue reports. We manually analyzed 980 real-life performance issue reports and derived 80 project-agnostic linguistic patterns that recur in the reports. Our approach uses these linguistic patterns to construct the sentence-level and issue-level learning features for training effective machine/deep learning classifiers. We test our approach on two separate datasets, each consisting of 980 unclassified issue reports, and compare the results with 31 baseline methods. Our approach can reach up to 83% precision and up to 59% recall. The only comparable baseline method is BERT, which is still 25% lower in the F1-score.

While existing research has explored the trade-off between security and performance, these efforts primarily focus on software consumers and often overlook the effectiveness and productivity of software producers. In this paper, we highlight an established security practice, air-gap isolation, and some challenges it uniquely instigates. To better understand and start quantifying the impacts of air-gap isolation on software development productivity, we conducted a survey at a commercial software company: Analytical Graphics, Inc. Based on our insights of dealing with air-gap isolation daily, we suggest some possible directions for future research. Our goal is to bring attention to this neglected area of research and to start a discussion in the SE community about the struggles faced by many commercial and governmental organizations.

Important software design concepts, such as information hiding and separation of concerns, are often conveyed to students informally. The modularity and hence maintainability of student software is difficult to assess. In this paper, we report our study of using design structure matrix (DSM) to assess the modularity of student software by comparing the differences between the DSM representing the intended design and the DSMs representing the software implemented by the students. We applied this approach to a software design class at Drexel University. We found that even though the lab and homework assignments were of small scale, and in many cases, detailed designs were given to the students in the form of UML class diagrams, 74% of the 85 student submissions, although fulfilled the required functionality, introduced unexpected dependencies so that the modules that designed to be independent are actually coupled. These design problems can only be revealed during software evolution, which is usually not possible for student projects. The results show the necessity and benefits of applying DSM modeling to make such design problems explicit to the students.

This paper presents Clio, an approach that detects modularity violations, which can cause software defects, modularity decay, or expensive refactorings. Clio computes the discrepancies between how components should change together based on the modular structure, and how components actually change together as revealed in version history. We evaluated Clio using 15 releases of Hadoop Common and 10 releases of Eclipse JDT. The results show that hundreds of violations identified using Clio were indeed recognized as design problems or refactored by the developers in later versions. The identified violations exhibit multiple symptoms of poor design, some of which are not easily detectable using existing approaches.

Researchers have leveraged evolutionary coupling derived from revision history to conduct various software analyses, such as software change impact analysis (IA). The problem is that the validity of historical data depends on the recency of changes and varies with different evolution paths-thus, influencing the accuracy of analysis results. In this paper, we formalize evolutionary coupling as a stochastic process using a Markov chain model. By varying the parameters of this model, we define a family of stochastic dependencies that accounts for different types of evolution paths. Each member of this family weighs historical data differently according to their recency and frequency. To assess the utility of this model, we conduct IA on 78 releases of five open source systems, using 16 stochastic dependency types, and compare with the results of several existing approaches. The results show that our stochastic-based IA technique can provide more accurate results than these existing techniques.

Empirical studies show that mismatches between design and organizational structures may cause expensive inter-team communication costs. However, prevailing design models and metrics are not designed for identifying independent, globally distributable task assignments, and do not account for organizational structure. Our research objectives are to develop design-centric theories, models, and measures to: (1) minimize the communication needs among developers, and (2) to accurately predict communication needs for performing maintenance tasks. To reach these objectives, we propose to develop design-centric theories and models to identify independent tasks from a design, and to predict coordination requirements from historical change-coupling.

To improve the ability of predicting the impact scope of a given change, we present two approaches applicable to the maintenance of object-oriented software systems. Our first approach exclusively uses a logical model extracted from UML relations among classes, and our other, hybrid approach additionally considers information mined from version histories. Using the open source Hadoop system, we evaluate our approaches by comparing our impact predictions with predictions generated using existing data mining techniques, and with actual change sets obtained from bug reports. We show that both our approaches produce better predictions when the system is immature and the version history is not well-established, and our hybrid approach produces comparable results with data mining as the system evolves.

To address the problem that existing software dependency extraction methods do not work on higher-level software artifacts, do not express decisions explicitly, and do not reveal implicit or indirect dependencies, our recent work explored the possibility of formally defining and automatically deriving a pairwise dependence relation from an augmented constraint networks (ACN) that models the assumption relation among design decisions. The current approach is difficult to scale, requiring constraint solving and solution enumeration. We observe that the assumption relation among design decisions for most software systems can be abstractly modeled using a special form of ACN. For these more restrictive, but highly representative models, we present an O(n3) algorithm to derive the dependency relation without solving the constraints. We evaluate our approach by computing design structure matrices for existing ACNs that model multiple versions of heterogenous real software designs, often reducing the running time from hours to seconds.

As software projects continue to grow in scale, being able to maximize the work that developers can carry out in parallel as a set of concurrent development tasks, without incurring excessive coordination overhead, becomes increasingly important. Prevailing design models, however, are not explicitly conceived to suggest how development tasks on the software modules they describe can be effectively parallelized. In this paper, we present a design rule hierarchy based on the assumption relations among design decisions. Software modules located within the same layer of the hierarchy suggest independent, hence parallelizable, tasks. Dependencies between layers or within a module suggest the need for coordination during concurrent work. We evaluate our approach by investigating the source code and mailing list of Apache Ant. We observe that technical communication between developers working on different modules within the same hierarchy layer, as predicted, is significantly less than communication between developers working across layers.

According to Parnas’s information hiding principle and Baldwin and Clark’s design rule theory, the key step to decomposing a system into modules is to determine the design rules (or in Parnas’s terms, interfaces) that decouple otherwise coupled design decisions and to hide decisions that are likely to change in independent modules. Given a modular design, it is often difficult to determine whether and how its implementation realizes the designed modularity. Manually comparing code with abstract design is tedious and error-prone. We present an automated approach to check the conformance of implemented modularity to designed modularity, using design structure matrices as a uniform representation for both. Our experiments suggest that our approach has the potential to manifest the decoupling effects of design rules in code, and to detect modularity deviation caused by implementation faults. We also show that design and implementation models together provide a comprehensive view of modular structure that makes certain implicit dependencies within code explicit.

Modularity is one of the most important properties of a software design, with significant impact on changeability and evolvability. However, a formalized and automated approach is lacking to test and verify software design models against their modularity properties, in particular, their ability to accommodate potential changes. In this paper, we propose a novel framework for testing design modularity. The software artifact under test is a software design. A test input is a potential change to the design. The test output is a modularity vector, which precisely captures quantitative capability extents of the design for accommodating the test input (the potential change). Both the design and the test input are represented as formal computable models to enable automatic testing. The modularity vector integrates the net option value analysis with well-known design principles. We have implemented the framework with tool supports and tested aspect-oriented and object-oriented design patterns in terms of their ability to accommodate sequences of possible changes. The results showed that previous informal, implementation-based analysis can be conducted by our framework automatically and quantitatively at the design level. This framework also opens the opportunities of applying testing techniques, such as coverage criteria, on software designs