Throughout the years, the attention on technical debt experienced a steady growth, and can now boast to be a consolidated concept within the software engineering field. Despite the growing academic and industrial interest in the topic, a category of technical debt appears to date to be unexplored. In this paper, we introduce the concept of turnover technical debt, i.e., technical debt that arises when developers leave projects with software artifacts other developers will struggle to work with. Our contribution presents an initial formulation of the phenomenon, which intuitively revolves around two co-occurring properties in software artifacts, namely (i) centralized ownership and (ii) low understandability and suboptimal documentation. Based on these two properties, we present an approach outlining a conceptual basis to quantitatively observe turnover technical debt via a mix of repository mining and static analysis. We complement our contribution with a mixed-method case study conducted with the twofold goal of assessing the viability of the proposed measurement approach and collecting initial insights from practitioners on the phenomenon. The gathered results point to the relevance of turnover technical debt in practice, and to promising avenues to more precisely measure the phenomenon.

Context: Microservices are gaining significant traction in academic research and industry due to their advantages, and technical debt has long been a heavily researched metric in software quality context. However, to date, no study has attempted to understand how code technical debt evolves in such architectures. Aim: This research aims to understand how technical debt evolves over time in microservice architectures by investigating its trends, patterns, and potential relations with microservices number. Method: We analyze the technical debt evolution of 13 open-source projects. We collect data from systems through automated source code analysis , statistically analyze results to identify technical debt trends and correlations with microservices number, and conduct a subsequent manual commit inspection. Results: Technical debt increases over time, with periods of stability. The growth is related to microservices number, but its rate is not. The analysis revealed trend differences during initial development phases and later stages. Different activities can introduce technical debt, while its removal relies mainly on refactoring. Conclusions: Microservices independence is fundamental to maintain the technical debt under control, keeping it compartmentalized. The findings underscore the importance of technical debt management strategies to support the long-term success of microservices.

The ever-increasing complexity of modern software architectures has exacerbated the need for advanced software tools able to track software execution traces to improve software reliability. In this paper, we present OREO, a tool for offline and run-time monitoring and fault localization. The tool implements a novel method enabling to trace software executions to discover the run-time status, dependencies, and interactions among software components. OREO is based on a timeline extractor, i.e., an abstraction of component lifecycles and their interactions. The timeline extractor enables the tool to perform a runtime health state examination of the software under analysis. The profiler is then used to analyze the error propagation originated during the running states among software components. In so doing, the possible fault-error-failure chains are identified. To showcase the capabilities of OREO and its flexibility, we report the execution of the tool on three software projects of different nature, sizes, and architectures. The analysis results in the localization of fault-error-failure chains and safe components of the three software projects. A discussion of the versatility, scalability, and applicability of the proposed tool to a rich variety of application contexts is provided.

The substantial increase in AI model training has considerable environmental implications, requiring energy-efficient and sustainable AI practices. On one hand, data-centric approaches show great potential towards training energy-efficient AI models. On the other hand, instance selection methods demonstrate the capability of training AI models with minimised training sets and negligible performance degradation. Despite the growing interest in both topics, the impact of data-centric training set selection on energy efficiency remains to date unexplored. This paper presents an evolutionary-based sampling framework aimed at (i) identifying elite training samples tailored for datasets and model pairs, (ii) comparing model performance and energy efficiency gains against typical model training practice, and (iii) investigating the feasibility of this framework for fostering sustainable model training practices. To evaluate the proposed framework, we conducted an empirical experiment including 8 commonly used AI classification models and 25 publicly available datasets. The results showcase that by considering 10% elite training samples, the models’ performance can show a 50% improvement and remarkable energy savings of 98% compared to the common training practice. In essence, this study establishes a new benchmark for AI researchers and practitioners interested in improving the environmental sustainability of AI model training via data-centric approaches.

Background: Over the past decade, microservices have surged in popularity within software engineering. From a research viewpoint, mining studies are frequently employed to assess the evolution of diverse microservice properties. Despite the growing need, a validated static method to swiftly identify microservices seems to be currently missing in the literature.

Context: With the ever-evolving software landscape, methods to train software programmers are continuously advancing and evolving. In this investigation, we study the case of 42, a programming school with over 50 campuses worldwide. 42’s pedagogical method blends elements of problem-based learning, peer pedagogy, community building, and gamification. Objectives: The goal of the research is twofold: On one hand, to gain a deep understanding of the pedagogical method itself, and on the other hand, to study how its different components affect learning. Method: We adopt an ethnographic qualitative inquiry, with two academic researchers conducting participant observation over a period of six months by using activity theory as theoretical underpinning. Results: Problems of incremental difficulty, albeit challenging, foster virtuous cycles of reinforcing feedback and community building. Gamification and peer learning elements, which are deeply rooted in the carefully crafted educational receipt, further support the pedagogical method. Conclusions: The characteristic nature of 42 positions it as an outlier compared to the recurrent academic setting of frontal lectures followed by a final exam, making it a valuable case study to understand how various pedagogical components may function, interact, and affect student learning.

Background: In recent years, a discourse on how to systematically consider and report threats to validity started to gain momentum within the empirical software engineering community. Aims: With this study, we aim to systematically underpin the current state of threats to validity practices in software engineering research. Method: We conduct a literature review comprising 91 papers awarded with the ACM SIGSOFT Distinguished Paper Award at the ACM/IEEE International Conference on Software Engineering. Data is extracted and analyzed by considering six main facets of threats to validity, e.g., their explicit documentation, categorization, discussion of limitations, and trade-offs. Results: Results corroborate current critiques to the threats management state of the art. Threats result to be seldom discussed in depth, and are mostly considered as an enforced afterthought rather than an active concern of the research design and execution. Conclusions: To improve the observed practice, we derived items to consider for researchers, reviewers and readers, and call for a community action to increase the understanding of knowledge creation in empirical software engineering research.

When testing three-tiered architectures, strategies often rely on superficial information, e.g., black-box input. However, the correct behavior of software-intensive systems based on such architectural pattern also depends on the logic hidden behind the interface. Verifying the response process is thus often complex and requires ad-hoc strategies. We propose an approach to identify faults hidden behind the presentation layer. The model-based approach uses an architectural abstraction called managed component Data Flow Graph (mcDFG). The mcDFG is aware of the interactions between all layers of the architecture and guides the generation of tests based on different mcDFG coverage criteria to identify faults in the business logic. To evaluate the approach viability, we consider a three-tiered web application and 32 faults. The fault detection capability is assessed by comparing a set of test suites created by following our method and a set of test suites developed by utilizing traditional testing strategies. The collected data show that the proposed model-based approach is a viable option to identify faults hidden in the logic layer, as it can outperform standard strategies based solely on the presentation layer while keeping the number of test cases and number of interactions per test case low.

Background: Software security is crucial to ensure that the users are protected from undesirable consequences such as malware attacks which can result in loss of data and, subsequently, financial loss. Technical Debt (TD) is a metaphor incurred by suboptimal decisions resulting in long-term consequences such as increased defects and vulnerabilities if not managed. Although previous studies have studied the relationship between security and TD, examining their intersection in developers' discussion on Stack Overflow (SO) is still unexplored. Aims: This study investigates the characteristics of security-related TD questions on SO. More specifically, we explore the prevalence of TD in security-related queries, identify the security tags most prone to TD, and investigate which user groups are more aware of TD. Method: We mined 117,233 security-related questions on SO and used a deep-learning approach to identify 45,078 security-related TD questions. Subsequently, we conducted quantitative and qualitative analyses of the collected security-related TD questions, including sentiment analysis. Results: Our analysis revealed that 38% of the security questions on SO are security-related TD questions. The most recurrent tags among the security-related TD questions emerged as “security” and “encryption.” The latter typically have a neutral sentiment, are lengthier, and are posed by users with higher reputation scores. Conclusions: Our findings reveal that developers implicitly discuss TD, suggesting developers have a potential knowledge gap regarding the TD metaphor in the security domain. Moreover, we identified the most common security topics mentioned in TD-related posts, providing valuable insights for developers and researchers to assist developers in prioritizing security concerns in order to minimize TD and enhance software security.

We propose a novel scalable approach for testing non-testable programs denoted as ARMED testing. The approach leverages efficient Association Rules Mining algorithms to determine relevant implication relations among features and actions observed while the system is in operation. These relations are used as the specification of positive and negative tests, allowing for identifying plausible or suspicious behaviors: for those cases when oracles are inherently unknownable, such as in social testing, ARMED testing introduces the novel concept of testing for plausibility. To illustrate the approach we walk-through an application example.

Background: Q&A sites allow to study how users reference and request support on technical debt. To date only few studies, focusing on narrow aspects, investigate technical debt on Stack Overflow.

Abstract Context Tangled commits are changes to software that address multiple concerns at once. For researchers interested in bugs, tangled commits mean that they actually study not only bugs, but also other concerns irrelevant for the study of bugs. Objective We want to improve our understanding of the prevalence of tangling and the types of changes that are tangled within bug fixing commits. Methods We use a crowd sourcing approach for manual labeling to validate which changes contribute to bug fixes for each line in bug fixing commits. Each line is labeled by four participants. If at least three participants agree on the same label, we have consensus. Results We estimate that between 17% and 32% of all changes in bug fixing commits modify the source code to fix the underlying problem. However, when we only consider changes to the production code files this ratio increases to 66% to 87%. We find that about 11% of lines are hard to label leading to active disagreements between participants. Due to confirmed tangling and the uncertainty in our data, we estimate that 3% to 47% of data is noisy without manual untangling, depending on the use case. Conclusion Tangled commits have a high prevalence in bug fixes and can lead to a large amount of noise in the data. Prior research indicates that this noise may alter results. As researchers, we should be skeptics and assume that unvalidated data is likely very noisy, until proven otherwise.

Background: With the rising popularity of Artificial Intelligence (AI), there is a growing need to build large and complex AI-based systems in a cost-effective and manageable way. Like with traditional software, Technical Debt (TD) will emerge naturally over time in these systems, therefore leading to challenges and risks if not managed appropriately. The influence of data science and the stochastic nature of AI-based systems may also lead to new types of TD or antipatterns, which are not yet fully understood by researchers and practitioners. Objective: The goal of our study is to provide a clear overview and characterization of the types of TD (both established and new ones) that appear in AI-based systems, as well as the antipatterns and related solutions that have been proposed. Method: Following the process of a systematic mapping study, 21 primary studies are identified and analyzed. Results: Our results show that (i) established TD types, variations of them, and four new TD types (data, model, configuration, and ethics debt) are present in AI-based systems, (ii) 72 antipatterns are discussed in the literature, the majority related to data and model deficiencies, and (iii) 46 solutions have been proposed, either to address specific TD types, antipatterns, or TD in general. Conclusions: Our results can support AI professionals with reasoning about and communicating aspects of TD present in their systems. Additionally, they can serve as a foundation for future research to further our understanding of TD in AI-based systems.

Architectural technical debt in software-intensive systems is a metaphor used to describe the “big” design decisions (e.g., choices regarding structure, frameworks, technologies, languages, etc.) that, while being suitable or even optimal when made, significantly hinder progress in the future. While other types of debt, such as code-level technical debt, can be readily detected by static analyzers, and often be refactored with minimal or only incremental efforts, architectural debt is hard to be identified, of wide-ranging remediation cost, daunting, and often avoided. In this study, we aim at developing a better understanding of how software development organizations conceptualize architectural debt, and how they deal with it. In order to do so, in this investigation we apply a mixed empirical method, constituted by a grounded theory study followed by focus groups. With the grounded theory method we construct a theory on architectural technical debt by eliciting qualitative data from software architects and senior technical staff from a wide range of heterogeneous software development organizations. We applied the focus group method to evaluate the emerging theory and refine it according to the new data collected. The result of the study, i.e., a theory emerging from the gathered data, constitutes an encompassing conceptual model of architectural technical debt, identifying and relating concepts such as its symptoms, causes, consequences, management strategies, and communication problems. From the conducted focus groups, we assessed that the theory adheres to the four evaluation criteria of classic grounded theory, i.e., the theory fits its underlying data, is able to work, has relevance, and is modifiable as new data appears. By grounding the findings in empirical evidence, the theory provides researchers and practitioners with novel knowledge on the crucial factors of architectural technical debt experienced in industrial contexts.

Architectural technical debt (ATD) in software-intensive systems refers to the architecture design decisions which work as expedient in the short term, but later negatively impact system evolvability and maintainability. Over the years numerous approaches have been proposed to detect particular types of ATD at a refined level of granularity via source code analysis. Nevertheless, how to gain an encompassing overview of the ATD present in a software-intensive system is still an open question. In this study, we present a multi-step approach designed to build an ATD index (ATDx), which provides insights into a set of ATD dimensions building upon existing architectural rules by leveraging statistical analysis. The ATDx approach can be adopted by researchers and practitioners alike in order to gain a better understanding of the nature of the ATD present in software-intensive systems, and provides a systematic framework to implement concrete instances of ATDx according to specific project and organizational needs.

The recent push towards test automation and test-driven development continues to scale up the dimensions of test code that needs to be maintained, analysed, and processed side-by-side with production code. As a consequence, on the one side regression testing techniques, e.g., for test suite prioritization or test case selection, capable to handle such large-scale test suites become indispensable; on the other side, as test code exposes own characteristics, specific techniques for its analysis and refactoring are actively sought. We present JTeC, a large-scale dataset of test cases that researchers can use for benchmarking the above techniques or any other type of tool expressly targeting test code. JTeC collects more than 2.5M test classes belonging to 31K+ GitHub projects and summing up to more than 430 Million SLOCs of ready-to-use real-world test code.

For surviving in the highly competitive market of Android apps, it is fundamental for app developers to deliver apps of high quality and with short release times. A well-architected Android app is beneficial for developers, e.g. in terms of maintainability, testability, performance, and avoidance of resource leaks. However, how to properly architect Android apps is still debated and subject to conflicting opinions usually influenced by technological hypes rather than objective evidence. In this paper we present an empirical study on how developers architect Android apps, what architectural patterns and practices Android apps are based on, and their potential impact on quality. We apply a mixed-method empirical research design that combines (i) semi-structured interviews with Android practitioners in the field and (ii) a systematic analysis of both the grey (i.e., websites, Online blogs) and white literature (i.e., academic studies) on the architecture of Android apps. Based on the analysis of the state of the art and practice about architecting Android apps, we systematically extract a set of 42 evidence-based guidelines supporting developers when architecting their Android apps.

Test suite reduction approaches aim at decreasing software regression testing costs by selecting a representative subset from large-size test suites. Most existing techniques are too expensive for handling modern massive systems and moreover depend on artifacts, such as code coverage metrics or specification models, that are not commonly available at large scale. We present a family of novel very efficient approaches for similarity-based test suite reduction that apply algorithms borrowed from the big data domain together with smart heuristics for finding an evenly spread subset of test cases. The approaches are very general since they only use as input the test cases themselves (test source code or command line input). We evaluate four approaches in a version that selects a fixed budget B of test cases, and also in an adequate version that does the reduction guaranteeing some fixed coverage. The results show that the approaches yield a fault detection loss comparable to state-of-the-art techniques, while providing huge gains in terms of efficiency. When applied to a suite of more than 500K real world test cases, the most efficient of the four approaches could select B test cases (for varying B values) in less than 10 seconds.

Many test case prioritization criteria have been proposed for speeding up fault detection. Among them, similarity-based approaches give priority to the test cases that are the most dissimilar from those already selected. However, the proposed criteria do not scale up to handle the many thousands or even some millions test suite sizes of modern industrial systems and simple heuristics are used instead. We introduce the FAST family of test case prioritization techniques that radically changes this landscape by borrowing algorithms commonly exploited in the big data domain to find similar items. FAST techniques provide scalable similarity-based test case prioritization in both white-box and black-box fashion. The results from experimentation on real world C and Java subjects show that the fastest members of the family outperform other black-box approaches in efficiency with no significant impact on effectiveness, and also outperform white-box approaches, including greedy ones, if preparation time is not counted. A simulation study of scalability shows that one FAST technique can prioritize a million test cases in less than 20 minutes.

Architectural Technical Debt (ATD) regards sub-optimal design decisions that bring short-term benefits to the cost of long-term gradual deterioration of the quality of the architecture of a software system. The identification of ATD strongly influences the technical and economic sustainability of software systems and is attracting growing interest in the scientific community. During the years several approaches for ATD identification have been conceived, each of them addressing ATD from different perspectives and with heterogeneous characteristics.

Context. Android is the largest mobile platform today, with thousands of apps published and updated in the Google Play store everyday. Maintenance is an important factor in Android apps lifecycle, as it allows developers to constantly improve their apps and better tailor them to their user base. Goal. In this paper we investigate the evolution of various maintainability issues along the lifetime of Android apps. Method. We designed and conducted an empirical study on 434 GitHub repositories containing open, real (i.e., published in the Google Play store), and actively maintained Android apps. We statically analyzed 9,945 weekly snapshots of all apps for identifying their maintainability issues over time. We also identified maintainability hotspots along the lifetime of Android apps according to how their density of maintainability issues evolves over time. More than 2,000 GitHub commits belonging to identified hotspots have been manually categorized to understand the context in which maintainability hotspots occur. Results. Our results shed light on (i) how often various types of maintainability issues occur over the lifetime of Android apps, (ii) the evolution trends of the density of maintainability issues in Android apps, and (iii) an in-depth characterization of development activities related to maintainability hotspots. Together, these results can help Android developers in (i) better planning code refactoring sessions, (ii) better planning their code review sessions (e.g., steering the assignment of code reviews), and (iii) taking special care of their code quality when performing tasks belonging to activities highly correlated with maintainability issues. We also support researchers by objectively characterizing the state of the practice about maintainability of Android apps. Conclusions. Independently from the type of development activity, maintainability issues grow until they stabilize, but are never fully resolved.

Background. Often motivated by optimization objectives, software products are characterized by different subsequent releases and deployed through different strategies. The impact of these two aspects of software on energy consumption has still to be completely understood and can be improved by carrying out ad-hoc analyses for specific software products. Aims. In this research we report on an industrial collaboration aiming at assessing the different impact that releases and deployment strategies of a software product can have on the energy consumption of its underlying hardware infrastructure. Method. We designed and performed an empirical experiment in a controlled environment. Deployment strategies, releases and use case scenarios of an industrial third-party software product were adopted as experimental factors. The use case scenarios were used as a blocking factor and adopted to dynamically load-test the software product. Power consumption and execution time were selected as response variables to measure the energy consumption. Results. We observed that both deployment strategies and software releases significantly influence the energy consumption of the hardware infrastructure. A strong interaction between the two factors was identified. The impact of such interaction highly varied depending on which use case scenario was considered, making the identification of the most frequently adopted use case scenario critical for energy optimisation. The collaboration between industry and academia has been productive for both parties, even if some practitioners manifested low interest/awareness on software energy efficiency. Conclusions. For the software product considered there is no absolute preferable release or deployment strategy with respect to energy efficiency, as the interaction of these factors has to be considered. The number of machines involved in a software deployment strategy does not simply constitute an additive effect of the energy consumption of the underlying hardware infrastructure.