Background: Mocking is a common technique for isolating test dependencies, yet duplicated mock setup code-what we call “mock clones”-can increase test maintenance overhead and reduce clarity. While code clone detection is a mature research area, the prevalence, characteristics, and refactoring of mock clones remain insufficiently understood.Aims: This study investigates the prevalence, detection challenges, and practical elimination of mock clones in real-world Java test suites, aiming to identify systematic patterns and assess the feasibility and value of mock clone refactoring.Method: We analyzed 698 mock clone instances across six open-source Java projects. We evaluated the effectiveness of existing code clone detection tools in capturing mock clones, manually refactored a large subset of clones to assess behavior preservation, and measured the structural impact of mock clone elimination.Results: Mock clones are prevalent, with frequently mocked classes often exhibiting extensive duplication. Existing clone detection tools failed to reliably detect mock clones due to scattered, reordered, and semantically varied mocking logic. Manual refactoring successfully eliminated $60 \%$ of identified mock clones while preserving test behavior, resulting in up to $64 \%$ reduction in mock objects and up to $61 \%$ reduction in mocking-related LOC. Targeting only the top $\mathbf{5 - 1 0}$ high-impact clones still achieved substantial simplification.Conclusions: Mock clone elimination is both feasible and highly valuable for improving test code maintainability. Systematic refactoring strategies can address common patterns, and prioritizing high-impact clones enables efficient gains with minimal effort. These findings motivate the need for mock-aware detection and automated refactoring tools to unlock broader maintainability improvements in testing practices.

The co-evolution of production and test code (PT co-evolution) has received increasing attention in recent years. However, we found that existing work did not comprehensively study various PT co-evolution scenarios, such as the qualification and persistence of their effects on software. Inspired by technical debt (TD), we refer to TD generated during the co-evolution between production and test code as PT co-evolution technical debt (PTCoTD). To better understand PT co-evolution, we first conducted an exploratory study on its characteristics on 15 open-source projects, finding unbalanced PT co-evolution prevalent and summarizing five potential PT flaws. Then we proposed an approach to identify and quantify PTCoTDs of these flaw patterns, considering evolutionary and structural relationships. We also built prediction models to describe cost trajectories and rank all PTCoTDs to prioritize expensive ones. The evaluation on the 15 projects shows that our approach can identify PTCoTDs that deserve attention. The identified PTCoTDs account for about half of the project's total maintenance costs, and the cost proportion of the expensive Top-5 is 1.8x more than the file proportion they contain. Almost all covered maintenance costs persist as PTCoTD in the future, with an average increase of 6.8% between the last two releases. Our approach also accurately predicts the costs of PTCoTD with an average prediction deviation of only 8.3%. Our study provides valuable insights into PT co-evolution scenarios and their effects, which can guide practices and inspire future work on software testing and maintenance.

The AAA (Arrange, Act, Assert) pattern provides a unified structure for unit test cases, potentially benefiting comprehension and maintenance. However, its adoption and implementation in practice remain insufficiently understood. This study investigates the prevalence of AAA pattern usage, identifies recurring deviations and design issues within AAA structures, and assesses developers’ receptiveness to AAA-based improvements. We conducted an empirical study on 735 real-life unit test cases randomly selected from seven open-source projects. We manually analyzed these test cases, identified AAA-related issues, and proposed fixes to developers. Our analysis found that 77% of test cases follow the AAA structure. We identified three recurring patterns deviating from AAA and four design issues within A blocks. Comparison with classic test smells revealed unique insights provided by AAA analysis. Of 27 improvement proposals sent to developers, 78% received positive feedback. These findings show that the AAA pattern is widely adopted in practice, but deviations from and design issues within AAA patterns are common. Our analysis provides a novel perspective on test case quality, complementing traditional test smell analysis. The high acceptance rate of our improvement proposals suggests that developers value AAA-based enhancements. These findings can guide the development of tools for improving AAA practice in unit tests.

Abstract Mocking frameworks provide convenient APIs, which create mock objects, manipulate their behavior, and verify their execution, for the purpose of isolating test dependencies in unit testing. This study contributes an in-depth empirical study of whether and how mocking frameworks are used in Apache projects. The key findings and insights of this study include: First, mocking frameworks are widely used in 66% of Apache Java projects, with Mockito, EasyMock, and PowerMock being the top three most popular frameworks. Larger-scale and more recent projects tend to observe a stronger need to use mocking frameworks. This underscores the importance of mocking in practice and related future research. Second, mocking is overall practiced quite selectively in software projects—not all test files use mocking, nor all dependencies of a test target are mocked. It calls for more future research to gain a more systematic understanding of when and what to mock to provide formal guidance to practitioners. On top of this, the intensity of mocking in different projects shows different trends in the projects’ evolution history—implying the compound effects of various factors, such as the pace of a project’s growth, the available resources, time pressure, and priority, etc. This points to an important future research direction in facilitating best mocking practices in software evolution. Furthermore, we revealed the most frequently used APIs in the three most popular frameworks, organized based on the function types. The top five APIs in each functional type of the three mocking frameworks usually take the majority (78% to 100%) of usage in Apache projects. This indicates that developers can focus on these APIs to quickly learn the common usage of these mocking frameworks. We further investigated informal methods of mocking, which do not rely on any mocking framework. These informal mocking methods point to potential sub-optimal mocking practices that could be improved, as well as limitations of existing mocking frameworks. Finally, we conducted a developer survey to collect additional insights regarding the above analysis based on their experience, which complements our analysis based on repository mining. Overall, this study offers practitioners profound empirical knowledge of how mocking frameworks are used in practice and sheds light on future research directions to enhancing mocking in practice.

Software performance is critical for system efficiency, with performance issues potentially resulting in budget overruns, project delays, and market losses. Such problems are reported to developers through issue tracking systems, which are often under-tagged, as the manual tagging process is voluntary and time-consuming. Existing automated performance issue tagging techniques, such as keyword matching and machine/deep learning models, struggle due to imbalanced datasets and a high degree of variance. This paper presents a novel hybrid classification approach, combining Heuristic Linguistic Patterns (HLPs) with machine/deep learning models to enable practitioners to automatically identify performance-related issues. The proposed approach works across three progressive levels:HLPtagging, sentence tagging, and issue tagging, with a focus on linguistic analysis of issue descriptions. The authors evaluate the approach on three different datasets collected from different projects and issue-tracking platforms to prove that the proposed framework is accurate, project- and platform-agnostic, and robust to imbalanced datasets. Furthermore, this study also examined how the two unique techniques of the framework, including the fuzzyHLPmatching and theIssue HLP Matrix, contribute to the accuracy. Finally, the study explored the effectiveness and impact of two off-the-shelf feature selection techniques,BorutaandRFE, with the proposed framework. The results showed that the proposed framework has great potential for practitioners to accurately (with up to 100% precision, 66% recall, and 79%F1-score) identify performance issues, with robustness to imbalanced data and good transferability to new projects and issue tracking platforms.

Continuous integration (CI) has become a popular method for automating code changes, testing, and software project delivery. However, sufficient testing prior to code submission is crucial to prevent build breaks. Additionally, testing must provide developers with quick feedback on code changes, which requires fast testing times. While regression test selection (RTS) has been studied to improve the cost-effectiveness of regression testing for lower-level tests (i.e., unit tests), it has not been applied to the testing of user interfaces (UI) in application domains such as mobile apps. UI testing at the UI level requires different techniques such as impact analysis and automated test execution. In this paper, we examine the use of RTS in CI settings for UI testing across various open-source mobile apps. Our analysis focuses on using Frequency Analysis to understand the need for RTS, Cost Analysis to evaluate the cost of impact analysis and test case selection algorithms, and Test Reuse Analysis to determine the reusability of UI test sequences for automation. The insights from this study will guide practitioners and researchers in developing advanced RTS techniques that can be adapted to CI environments for mobile apps.

Unit testing focuses on verifying the functions of individual units of a software system. It is challenging due to the high inter dependencies among software units. Developers address this by mocking—replacing the dependency by a “fake” object. Despite the existence of powerful, dedicated mocking frameworks, developers often turn to a “hand-rolled” approach—inheritance. That is, they create a subclass of the dependent class and mock its behavior through method overriding. However, this requires tedious implementation and compromises the design quality of unit tests. This work contributes a fully automated refactoring framework to identify and replace the usage of inheritance by using Mockito—a well received mocking framework. Our approach is built upon the empirical experience fromfiveopen source projects that use inheritance for mocking. We evaluate our approach onnineother projects. Results show that our framework is efficient, generally applicable to new datasets, mostly preserves test case behaviors in detecting defects (in the form of mutants), and decouples test code from production code. The qualitative evaluation by experienced developers suggests that the auto-refactoring solutions generated by our framework improve the quality of the unit test cases in various aspects, such as making test conditions more explicit, as well as improved cohesion, readability, understandability, and maintainability with test cases. Finally, we submit 23 pull requests containing our refactoring solutions to the open source projects. It turns our that, 9 requests are accepted/merged, 6 requests are rejected, the remaining requests are pending (5 requests), with unexpected exceptions (2 requests), or undecided (1 request). In particular, among the 21 open source developers that are involved in the reviewing process, 81% give positive votes. This indicates that our refactoring solutions are quite well received by the open source projects and developers.

TheAAApattern (i.e.,Arrange-Act-Assert) is a common and natural layout to create a test case. Following this pattern in test cases may benefit comprehension, debugging, and maintenance. TheAAAstructure of real-life test cases, however, may not be clear due to their high complexity. Manually labelingAAAstatements in test cases is tedious. Thus, we envision that an automated approach for labelingAAAstatements in existing test cases could benefit new developers and projects that practice collective code ownership and test-driven development. This paper contributes an automatic approach based on machine learning models. The “secret sauce” of this approach is a set of three learning features that are based on the semantic, syntax, and context information in test cases, derived from the manual tagging process. Thus, our approach mimics how developers may manually tag theAAApattern of a test case. We assess the precision, recall, and F-1 score of our approach based on 449 test cases, containing about 16,612 statements, across 4 Apache open source projects. To achieve the best performance in our approach, we explore the usage of six machine learning models; the contribution of the SMOTE data balancing technique; the comparison of the three learning features; and the comparison of five different methods for calculating the semantic feature. The results show our approach is able to identifyArrangement,Action, andAssertionstatements with a precision upwards of 92%, and recall up to 74%. We also summarize some experience based on our experiments—regarding the choice of machine learning models, data balancing algorithm, and feature engineering methods—which could potentially provide some reference to related future research.

Software performance is a critical quality attribute that determines the success of a software system. However, practitioners lack comprehensive and holistic understanding of how real-life performance issues are caused and resolved in practice from the technical, engineering, and economic perspectives. This paper presents a large-scale empirical study of 570 real-life performance issues from 13 open source projects from various problem domains, and implemented in three popular programming languages, Java (192 issues), C/C++ (162 issues), and Python (216 issues). From the technical perspective, we summarizeeightgeneral types of performance issues with corresponding root causes and resolutions that apply for all three languages. We also identify available tools for detecting and resolving different types of issues from the literature. In addition, we found that 27% of the 570 issues are resolved by design-level optimization—coordinated revision of a group of related source files and their design structure. We reveal four typical design-level optimization patterns, includingclassic design patterns,change propagation,optimization clone, andparallel optimizationthat practitioners should be aware of in resolving performance issues. From the engineering perspective, this study analyzes how test code changes in performance optimization. We found that only 15% of the 570 performance issues involve revision of test code. In most cases, the revised test cases focus on the functional logic of the performance optimization, rather than directly evaluate the performance improvement. This finding points to the potential lack of engineering standard for formally verifying performance optimization in regression testing. Finally, from the economic perspective, we analyze the“Return On Investment”of performance optimization. We found that design-level optimization usually requires more investment, but not always yields to higher performance improvement. However, developers tend to use design-level optimization when they concern about other quality attributes, such as maintainability and readability.

The AAA pattern, i.e. the Arrangement, Action, and Assertion, is a common and nature layout to create a test case. Following this pattern in test cases may benefit comprehension, debugging, and maintenance. The AAA structure of real-life test cases may not be explicit due to its high complexity. Manually labeling AAA statements in test cases is tedious. Thus, an automated approach for labeling AAA statements in existing test cases could benefit new developers and projects that practice collective code ownership and test driven development.

Architectural Technical Debt (ATD) refers to sub-optimal architectural design in a software system that incurs high maintenance “interest” over time. Previous research revealed thatATDhas significant negative impact on daily development. This paper contributes an approach to enable an architect to precisely locateATDs, as well as capture the trajectory of maintenance cost on each debt, based on which, predict the cost of the debt in a future release. TheATDsare expressed in four typical patterns, which entail the core of each debt. Furthermore, we aggregate compoundATDsto capture the complicated relationship among multipleATDinstances, which should be examined together for effective refactoring solutions. We evaluate our approach on 18 real-world projects. We identifiedATDsthatpersistentlyincur significant (up to 95 percent of) maintenance costs in most projects. The maintenance costs on the majority of debts fit into a linear regression model—indicating stable “interest” rate. In five projects, 12.1 to 27.6 percent of debts fit into an exponential model, indicating increasing “interest” rate, which deserve higher priority from architects. The regression models can accurately predict the costs of the majority of (82 to 100 percent) debts in the next release of a system. By aggregating relatedATDs, architects can focus on a small number of cost-effective compound debts, which contain a relatively small number of source files, but account for a large portion of maintenance costs in their projects. With these capabilities, our approach can help architects make informed decisions regarding whether, where, and how to refactor for eliminatingATDsin their systems.

Over the past three decades software engineering researchers have produced a wide range of techniques and tools for understanding the architectures of large, complex systems. However, these have tended to be one-off research projects, and their idiosyncratic natures have hampered research collaboration, extension and combination of the tools, and technology transfer. The area of software architecture is rich with disjoint research and development infrastructures, and datasets that are either proprietary or captured in proprietary formats. This paper describes a concerted effort to reverse these trends. We have designed and implemented a flexible and extensible infrastructure (SAIN) with the goal of sharing, replicating, and advancing software architecture research. We have demonstrated that SAIN is capable of incorporating the constituent tools extracted from three independently developed, large, long-lived software architecture research environments. We discuss SAIN's ambitious goals, the challenges we have faced in achieving those goals, the key decisions made in SAIN's design and implementation, the lessons learned from our experience to date, and our ongoing and future work.

Unit testing focuses on verifying the functions of individual units of a software system. It is challenging due to the high inter-dependencies among software units. Developers address this by mocking-replacing the dependency by a "faked" object. Despite the existence of powerful, dedicated mocking frameworks, developers often turn to a "hand-rolled" approach-inheritance. That is, they create a subclass of the dependent class and mock its behavior through method overriding. However, this requires tedious implementation and compromises the design quality of unit tests. This work contributes a fully automated refactoring framework to identify and replace the usage of inheritance by using Mockito-a well received mocking framework. Our approach is built upon the empirical experience from five open source projects that use inheritance for mocking. We evaluate our approach on four other projects. Results show that our framework is efficient, generally applicable to new datasets, mostly preserves test case behaviors in detecting defects (in the form of mutants), and decouples test code from production code. The qualitative evaluation by experienced developers suggests that the auto-refactoring solutions generated by our framework improve the quality of the unit test cases in various aspects, such as making test conditions more explicit, as well as improved cohesion, readability, understandability, and maintainability with test cases.

In large-scale software systems, error-prone or change-prone files rarely stand alone. They are typically architecturally connected and their connections usually exhibit architecture problems causing the propagation of error-proneness or change-proneness. In this paper, we propose and empirically validate a suite of architecture anti-patterns that occur in all large-scale software systems and are involved in high maintenance costs. We define these architecture anti-patterns based on fundamental design principles and Baldwin and Clark's design rule theory. We can automatically detect these anti-patterns by analyzing a project's structural relationships and revision history. Through our analyses of 19 large-scale software projects, we demonstrate that these architecture anti-patterns have significant impact on files' bug-proneness and change-proneness. In particular, we show that 1) files involved in these architecture anti-patterns are more error-prone and change-prone; 2) the more anti-patterns a file is involved in, the more error-prone and change-prone it is; and 3) while all of our defined architecture anti-patterns contribute to file's error-proneness and change-proneness, Unstable Interface and Crossing contribute the most by far.

The design structure of a system continuously evolves as the consequence of fast-paced code revisions. Agile techniques, such as continuous testing, ensures the function goals of a system with every code revision. However, there lacks an efficient approach that can continuously model the design difference resulting from every single code revision to facilitate comprehension and ensure the design quality. This paper contributes a novel design modeling approach, called Design Differencing (DESIGNDIFF), that models and visualizes the highlevel design differences resulting from every code revision. This paper defines a complete and general set of 17 design change operators to capture the design difference from any code revision. We evaluated the potential of DESIGNDIFF in three aspects. First, a user study of 10 developers indicated that DESIGNDIFF can help practitioners to faster and better understand high-level design differences from real-life software commits. Second, DESIGNDIFF analyzed 14,832 real-life commits in five real-life projects: finding 4,189 commits altered the software design, 855 commits introduced and 337 commits eliminated design flaws. The latency between the flaw introduction and elimination is on average 2 months to 2 years! With an affordable performance overhead, DESIGNDIFF has great potential to benefit practitioners in more applications.

Performance issues widely exist in modern software systems. Existing performance optimization approaches, such as dynamic profiling, usually fail to consider the impacts of architectural connections among methods on performance issues. This paper contributes an architectural approach, Butterfly Space modeling, to investigate performance issues. Each Butterfly Space is composed of 1) a seed method; 2) methods in the "upper wing" that call the seed directly or transitively; and 3) methods in the "lower wing" that are called by the seed, directly or transitively. The rationale is that the performance of the seed method impacts and is impacted by all the other methods in the space because of the call relationship. As such, developers can more efficiently investigate groups of connected performance improvement opportunities in Butterfly Spaces. We studied three real-world open source Java projects to evaluate such potential. Our findings are three-fold: 1) If the seed method of a Butterfly Space contains performance problems, up to 60% of the methods in the space also contain performance problems; 2) Butterfly Spaces can potentially help to non-trivially increase the precision/recall and reduce the costs in identifying performance improvement opportunities, compared to dynamic profiling; and 3) Visualizing dynamic profiling metrics with Butterfly Spaces simultaneously help to reveal two typical patterns, namely Expensive Callee and Inefficient Caller, that are responsible for performance problems and provide insights on where to improve next. We believe that Butterfly Space modeling has great potential for investigating performance issues.

Performance issues compromise the response time and resource consumption of a software system. Modern software systems use issue tracking systems to manage all kinds of issue reports, including performance issues. The problem is that performance issues are often not explicitly tagged. The tagging mechanism, if exists, is completely voluntary, depending on the project’s convention and on submitters’ discipline. For example, the performance tag rate in Apache’s Jira system is below 1%. This paper contributes a hybrid classification approach that combines linguistic patterns and machine/deep learning techniques to automatically detect performance issue reports. We manually analyzed 980 real-life performance issue reports and derived 80 project-agnostic linguistic patterns that recur in the reports. Our approach uses these linguistic patterns to construct the sentence-level and issue-level learning features for training effective machine/deep learning classifiers. We test our approach on two separate datasets, each consisting of 980 unclassified issue reports, and compare the results with 31 baseline methods. Our approach can reach up to 83% precision and up to 59% recall. The only comparable baseline method is BERT, which is still 25% lower in the F1-score.

Empirical experience regarding how real-life performance issues are caused and resolved can provide valuable insights for practitioners to effectively and efficiently prevent, detect, and fix performance issues. Prior work shows that most performance issues have their roots in poor architectural decisions. This paper contributes a large scale empirical study of 192 real-life performance issues, with an emphasis on software design. First, this paper contributes a holistic view of eight common root causes and typical resolutions that recur in different projects, and surveyed existing literature, in particular, tools, that can detect and fix each type of performance issue. Second, this study is first-of-its-kind to investigate performance issues from a design perspective. In the 192 issues, 33% required design-level optimization, i.e. simultaneously revising a group of related source files for resolving the issues. We reveal four design-level optimization patterns, which have shown different prevalence in resolving different root causes. Finally, this study investigated the Return on Investment for addressing performance issues, to help practitioners choose between localized or design-level optimization resolutions, and to prioritize issues due to different root causes.

Frequent co-changes to multiple files, i.e., evolutionary coupling, can demonstrate active relations among files, explicit or implicit. Although evolutionary coupling has been used to analyze software quality, there is no systematic study on the categorization of frequent co-changes between files which may used for characterizing various quality problems. In this paper, we report an empirical study on 27,087 co-change commits of 6 open-source systems with the purpose of understanding the observed evolutionary coupling. We extracted fine-grained change information from version control system to investigate whether two files exhibit particular kinds of co-change relationships. We consider code changes on 5 types of program entities (i.e., field, method, control statement, non-control statement, and class) and identified 6 types of dominating co-change relationships. Our manual analysis showed that each of the 6 types can be explained by structural coupling, semantic coupling, or implicit dependencies. Temporal analysis further shows that files may exhibit different co-change relationships at different phases in the evolution history. Finally, we investigated co-changes among multiple files by combining co-change relationships between related file pairs and showed with live examples that rich information embedded in the fine-grained co-change relationships may help developers to change code at multiple locations. Moreover, we analyzed how these co-change relationship types can be used to facilitate change impact analysis and to pinpoint design problems.

In this paper, we propose an architecture model called Design Rule Space (DRSpace). We model the architecture of a software system as multiple overlapping DRSpaces, reflecting the fact that any complex software system must contain multiple aspects, features, patterns, etc. We show that this model provides new ways to analyze software quality. In particular, we introduce an Architecture Root detection algorithm that captures DRSpaces containing large numbers of a project's bug-prone files, which are called Architecture Roots (ArchRoots). After investigating ArchRoots calculated from 15 open source projects, the following observations become clear: from 35 to 91 percent of a project's most bug-prone files can be captured by just 5 ArchRoots, meaning that bug-prone files are likely to be architecturally connected. Furthermore, these ArchRoots tend to live in the system for significant periods of time, serving as the major source of bug-proneness and high maintainability costs. Moreover, each ArchRoot reveals multiple architectural flaws that propagate bugs among files and this will incur high maintenance costs over time. The implication of our study is that the quality, in terms of bug-proneness, of a large, complex software project cannot be fundamentally improved without first fixing its architectural flaws.

Performance problems widely exist in modern software systems. Existing performance optimization techniques, including profiling-based and pattern-based techniques, usually fail to consider the architectural impacts among methods that easily slow down the overall system performance. This paper contributes a new approach, named Speedoo, to identify groups of methods that should be treated together and deserve high priorities for performance optimization. The uniqueness of Speedoo is to measure and rank the performance optimization opportunities of a method based on 1) the architectural impact and 2) the optimization potential. For each highly ranked method, we locate a respective Optimization Space based on 5 performance patterns generalized from empirical observations. The top ranked optimization spaces are suggested to developers as potential optimization opportunities. Our evaluation on three real-life projects has demonstrated that 18.52% to 42.86% of methods in the top ranked optimization spaces indeed undertook performance optimization in the projects. This outperforms one of the state-of-the-art profiling tools YourKit by 2 to 3 times. An important implication of this study is that developers should treat methods in an optimization space together as a group rather than as individuals in performance optimization. The proposed approach can provide guidelines and reduce developers' manual effort.

Despite decades of research on software metrics, we still cannot reliably measure if one design is more maintainable than another. Software managers and architects need to understand whether their software architecture is "good enough", whether it is decaying over time and, if so, by how much. In this paper, we contribute a new architecture maintainability metric---Decoupling Level (DL)---derived from Baldwin and Clark's option theory. Instead of measuring how coupled an architecture is, we measure how well the software can be decoupled into small and independently replaceable modules. We measured the DL for 108 open source projects and 21 industrial projects, each of which has multiple releases. Our main result shows that the larger the DL, the better the architecture. By "better" we mean: the more likely bugs and changes can be localized and separated, and the more likely that developers can make changes independently. The DL metric also opens the possibility of quantifying canonical principles of single responsibility and separation of concerns, aiding cross-project comparison and architecture decay monitoring.

Our prior work showed that the majority of error-prone source files in a software system are architecturally connected. Flawed architectural relations propagate defects among these files and accumulate high maintenance costs over time, just like debts accumulate interest. We model groups of architecturally connected files that accumulate high maintenance costs as architectural debts. To quantify such debts, we formally define architectural debt, and show how to automatically identify debts, quantify their maintenance costs, and model these costs over time. We describe a novel history coupling probability matrix for this purpose, and identify architecture debts using 4 patterns of architectural flaws shown to correlate with reduced software quality. We evaluate our approach on 7 large-scale open source projects, and show that a significant portion of total project maintenance effort is consumed by paying interest on architectural debts. The top 5 architectural debts, covering a small portion (8% to 25%) of each project's error-prone files, capture a significant portion (20% to 61%) of each project's maintenance effort. Finally, we show that our approach reveals how architectural issues evolve into debts over time.

Recently there has been increased attention to the consequences of architecture design decisions and their impact on security. Architectural design decisions have been identified as being critical for achieving high levels of software system security. However the majority of this research has been anecdotal and there are few tools or methods for understanding the architectural relations among files, and their impact on security. In this paper we employ a DRSpace-based analysis approach to identify architectural design flaws and we show, via an empirical study of 10 open source projects, that areas of a software architecture that suffer from greater numbers of design flaws are highly correlated with security bugs, and high levels of churn associated with those security bugs. Finally, we show that a specific type of design flaw -- unstable interface -- is correlated with the greatest increase in software security bugs.

Our recent research has shown that, in large-scale software systems, defective files seldom exist alone. They are usually architecturally connected, and their architectural structures exhibit significant design flaws which propagate bugginess among files. We call these flawed structures the architecture roots, a type of technical debt that incurs high maintenance penalties. Removing the architecture roots of bugginess requires refactoring, but the benefits of refactoring have historically been difficult for architects to quantify or justify. In this paper, we present a case study of identifying and quantifying such architecture debts in a large-scale industrial software project. Our approach is to model and analyze software architecture as a set of design rule spaces (DRSpaces). Using data extracted from the project's development artifacts, we were able to identify the files implicated in architecture flaws and suggest refactorings based on removing these flaws. Then we built economic models of the before and (predicted) after states, which gave the organization confidence that doing the refactorings made business sense, in terms of a handsome return on investment.

Conventional wisdom suggests that a software system's architecture has a significant impact on its evolution. Prior research has studied the evolution of software using the information of how its files have changed together in their revision history. No prior study, however, has investigated the impact of architecture on the evolution of software from its change history. This is mainly because most open-source software systems do not document their architectures. We have overcome this challenge using several architecture recovery techniques. We used the recovered models to examine if co-changes spanning multiple architecture modules are more likely to introduce bugs than co-changes that are within modules. The results show that the co-changes that cross architectural module boundaries are more correlated with defects than co-changes within modules, implying that, to improve accuracy, bug predictors should also take the software architecture of the system into consideration.

In our prior research, we found that problematic architectural connections can propagate errors. We also found that among multiple files, the architectural connections that violate common design principles strongly correlate with the error-proneness of files. The flawed architectural connections, if not fixed properly and timely, can become debts that accumulate high interest in terms of maintenance costs over time. In this paper, we define architectural debts as clusters of files with problematic architectural connections among them, and their connections incur high maintenance costs over time. Our goal is to 1) precisely identify which and how many files are involved in architectural debts; 2) quantify the penalties of architectural debts in terms of maintenance costs; and 3) model the growth trend of penalties---maintenance costs---that accumulate due to architectural debts. We plan to provide a quantitative model for project managers and stakeholders as a reference in making decisions of whether, when and where to invest in refactoring.

In this paper, we propose and empirically validate a suite of hotspot patterns: recurring architecture problems that occur in most complex systems and incur high maintenance costs. In particular, we introduce two novel hotspot patterns, Unstable Interface and Implicit Cross-module Dependency. These patterns are defined based on Baldwin and Clark's design rule theory, and detected by the combination of history and architecture information. Through our tool-supported evaluations, we show that these patterns not only identify the most error-prone and change-prone files, they also pinpoint specific architecture problems that may be the root causes of bug-proneness and change-proneness. Significantly, we show that 1) these structure-history integrated patterns contribute more to error- and change-proneness than other hotspot patterns, and 2) the more hotspot patterns a file is involved in, the more error- and change-prone it is. Finally, we report on an industrial case study to demonstrate the practicality of these hotspot patterns. The architect and developers confirmed that our hotspot detector discovered the majority of the architecture problems causing maintenance pain, and they have started to improve the system's maintainability by refactoring and fixing the identified architecture issues.

Context: Long-term software management decisions are directly impacted by the quality of the software's architecture. Goal: Herein, we present a replication case study where structural information about a commercial software system is used in conjunction with bug-related change frequencies to measure and predict architecture quality. Method: Metrics describing history and structure were gathered and then correlated with future bug-related issues; the worst of which were visualized and presented to developers. Results: We identified dependencies between components that change together even though they belong to different architectural modules, and as a consequence are more prone to bugs. We validated these dependencies by presenting our results back to the developers. The developers did not identify any of these dependencies as unexpected, but rather architectural necessities. Conclusions: This replication study adds to the knowledge base of CLIO (a tool that detects architectural degradations) by incorporating a new programming language (C++) and by externally replicating a previous case study on a separate commercial code base. Additionally, we provide lessons learned and suggestions for future applications of CLIO.

In this paper, we investigate software architecture as a set of overlapping design rule spaces, formed by one or more structural or evolutionary relationships and clustered using our design rule hierarchy algorithm. Considering evolutionary coupling as a special type of relationship, we investigated (1) whether design rule spaces can reveal structural relations among error-prone files; (2) whether design rule spaces can reveal structural problems contributing to error-proneness.We studied three large-scale open source projects and found that error-prone files can be captured by just a few design rule sub-spaces. Supported by our tool, Titan, we are able to flexibly visualize design rule spaces formed by different types of relationships, including evolutionary dependencies. This way, we are not only able to visualize which error-prone files belong to which design rule spaces, but also to visualize the structural problems that give insight into why these files are error prone. Design rule spaces provide valuable direction on which parts of the architecture are problematic, and on why, when, and how to refactor.

Numerous techniques have been proposed to locate buggy files in a code base, but the problem of fixing one bug unexpectedly affecting other files is persistent and prevailing. Our recent study revealed that buggy files are usually architecturally connected by architecture issues such as unstable interfaces and modularity violations. We aim to detect and prevent these architecture issues that are the root causes of defects. Our contributions include (1) a new architecture model, Design Rule Space (DRSpace), that can express structural relations, quality, and evolutionary information simultaneously; (2) a method of automatically extracting defect-prone architecture roots by combining static architecture analysis with software revision history data mining. The preliminary application of our approach to dozens of open source and industry projects has demonstrated its significant potential to inform developers about how software defects should be discovered, examined, and handled.

In this tool demo, we will illustrate our tool---Titan---that supports a new architecture model: design rule spaces (DRSpaces). We will show how Titan can capture both architecture and evolutionary structure and help to bridge the gap between architecture and defect prediction. We will demo how to use our toolset to capture hundreds of buggy files into just a few architecturally related groups, and to reveal architecture issues that contribute to the error-proneness and change-proneness of these groups. Our tool has been used to analyze dozens of large-scale industrial projects, and has demonstrated its ability to provide valuable direction on which parts of the architecture are problematic, and on why, when, and how to refactor. The video demo of Titan can be found at https://art.cs.drexel.edu/~lx52/titan.mp4

This case study combines known software structure and revision history analysis techniques, in known and new ways, to predict bug-related change frequency, and uncover architecture-related risks in an agile industrial software development project. We applied a suite of structure and history measures and statistically analyzed the correlations between them. We detected architecture issues by identifying outliers in the distributions of measured values and investigating the architectural significance of the associated classes. We used a clustering method to identify sets of files that often change together without being structurally close together, investigating whether architecture issues were among the root causes. The development team confirmed that the identified clusters reflected significant architectural violations, unstable key interfaces, and important undocumented assumptions shared between modules. The combined structure diagrams and history data justified a refactoring proposal that was accepted by the project manager and implemented.

With the growing use of location-based services, location privacy attracts increasing attention from users, industry, and the research community. While considerable effort has been devoted to inventing techniques that prevent service providers from knowing a user's exact location, relatively little attention has been paid to enabling so-called peer-wise privacy---the protection of a user's location from unauthorized peer users. This paper identifies an important efficiency problem in existing peer-privacy approaches that simply apply a filtering step to identify users that are located in a query range, but that do not want to disclose their location to the querying peer. To solve this problem, we propose a novel, privacy-policy enabled index called the PEB-tree that seamlessly integrates location proximity and policy compatibility. We propose efficient algorithms that use the PEB-tree for processing privacy-aware range and k NN queries. Extensive experiments suggest that the PEB-tree enables efficient query processing.