Managing technical debt (TD) is critical to ensure the sustainability of long-term software projects. However, the time and cost involved in technical debt management (TDM) often discourage practitioners from performing this activity consistently. Continuous Integration and Continuous Delivery (CI/CD) pipelines offer an opportunity to support TDM by embedding automated practices directly into the development workflow. Despite this potential, it remains unclear how TDM tools could be integrated into CI/CD pipelines, and we still lack established best practices for this process. To address this problem, the objective of this study is to understand how TDM tools have been used in CI/CD pipelines and also identify potential configuration anti-patterns. To this end, we conducted a large-scale mining software repository (MSR) study on GitHub. In total, we collected around 600,000 Travis CI configuration files and 50,000 supporting scripts, and identified 3,684 pipelines that contain at least one TDM tool. We applied descriptive statistics to analyze the prevalence of tools and anti-patterns, and our findings show that most tools are executed and integrated using an external script; in addition, Absent Feedback is the most common configuration anti-pattern. We believe that researchers and practitioners can use the evidence of this study to further investigate how to improve both the tools that are integrated in CI/CD and the integration practices.

Abstract Context Technical debt (TD) is a widely studied metaphor that helps to explain how sub-optimal decisions, which usually have short-term benefits, can harm software maintainability over time. Although incurring TD is not intrinsically bad, tracking and managing TD are crucial to avoid its negative effects. Hence, researchers and practitioners have proposed and developed diverse approaches and tools for managing TD. However, we are still lacking specialized tools for technical debt management (TDM), specifically ones that can be easily integrated into existing development workflows. Objective We present and evaluate TagDebt, a bot that can be integrated within GitHub repositories and automatically assign labels to issues (i.e., SATD or non-SATD). TagDebt helps in the identification of TD (i.e., by looking for self-admitted technical debt (SATD)), leading to more efficient TDM. Methods We carried out a Design Science Research study to design and implement TagDebt. For its evaluation, we executed a Technology Acceptance Model (TAM) study through interviews with 16 practitioners, to check the bot’s usefulness, ease of use, and contextual factors that might impact the bot’s usage (such as team size and practitioners’ roles). Results Overall, practitioners found that TagDebt is useful, especially for organizing issues and reducing manual work. Furthermore, they pointed out that the bot is overall easy to use, and its documentation is clear. The analysis also revealed that contextual factors, such as team and codebase size, impact the decision to adopt TagDebt. Finally, several improvements were suggested, such as including features to check and update the source code. Conclusion TagDebt is a proof-of-concept for the development and usage of more specialized tools for TDM. It helps to make TD visible without disrupting existing workflows, which could lead to increased adoption of TDM tools and, consequently, help practitioners avoid the risks of unmanaged TD.

Abstract Self-Admitted Technical Debt (SATD) refers to technical compromises explicitly admitted by developers in natural language artifacts, such as code comments, commit messages, and issue trackers. Among its types, Architecture Technical Debt (ATD) is particularly difficult to detect due to its abstract and context-dependent nature. Manual annotation of ATD is costly, time-consuming, and challenging to scale. To reduce labeling effort, this study combines keyword-based filtering, active learning, and explainable AI for ATD detection. We refined an existing dataset of ATD-related Jira issues to obtain an expert-validated seed set used to extract representative keywords. These keywords were then applied to identify more than 103k candidate issues across 10 open-source projects. To assess the reliability of keyword-based filtering, we qualitatively evaluated a statistically representative sample of labeled issues. Building on the resulting dataset, we applied active learning with multiple query strategies to prioritize informative samples for annotation. The results show that Breaking Ties achieved the best performance, with an F1-score of 0.72 and a 49% reduction in annotation effort. To improve transparency, we used SHAP and LIME to explain ATD classification results. Expert evaluation showed that both methods provided useful explanations, with LIME generally preferred for its clarity and ease of use.

Architectural technical debt (ATD) represents trade-offs in software architecture that accelerate initial development but create long-term maintenance challenges. ATD, in particular when self-admitted, impacts the foundational structure of software, making it difficult to detect and resolve. This study investigates the lifecycle of ATD, focusing on how it affects i) the connectivity between classes and ii) the frequency of file modifications. We aim to understand how ATD evolves from introduction to repayment and its implications on software architectures. Our empirical approach was applied to a dataset of SATD items extracted from various software artifacts. We isolated ATD instances, filtered for architectural indicators, and calculated dependencies at different lifecycle stages using FAN-IN and FAN-OUT metrics. Statistical analyses, including the Mann-Whitney U test and Cliff's Delta, were used to assess the significance and effect size of connectivity and dependency changes over time. We observed that ATD repayment increased class connectivity, with FAN-IN increasing by 57.5% on average and FAN-OUT by 26.7%, suggesting a shift toward centralization and increased architectural complexity after repayment. Moreover, ATD files were modified less frequently than Non-ATD files, with changes accumulated in high-dependency portions of the code. Our study shows that resolving ATD improves software quality in the short-term, but can make the architecture more complex by centralizing dependencies. Also, even if dependency metrics (like FAN-IN and FAN-OUT) can help understand the impact of ATD, they should be combined with other measures to capture other effects of ATD on software maintainability.

Enhancing the modular structure of existing systems has attracted substantial research interest, focusing on two main methods: (1) software modularization and (2) identifying design issues (e.g., smells) as refactoring opportunities. However, remodularization solutions often require extensive modifications to the original modules, and the design issues identified are generally too coarse to guide refactoring strategies. Combining the above two methods, this paper introduces a novel concept, PairSmell, which exploits modularization to pinpoint design issues necessitating refactoring. We concentrate on a granular but fundamental aspect of modularity principles-modular relation (MR), i.e., whether a pair of entities are separated or collocated. The main assumption is that, if the actual MR of a pair violates its ‘apt MR’, i.e., an MR agreed on by multiple modularization tools (as raters), it can be deemed likely a flawed architectural decision that necessitates further examination. To quantify and evaluate PairSmell, we conduct an empirical study on 20 C/C++ and Java projects, using 4 established modularization tools to identify two forms of PairSmell: inapt separated pairs$InSep$and inapt collocated pairs$InCol$. Our study on 260,003 instances reveals that their architectural impacts are substantial: (1) on average, 14.60 % and 20.44 % of software entities are involved in$InSep$and$InCol$MRs respectively; (2)$InSep$pairs are associated with 190 % more co-changes than properly separated pairs, while$InCol$pairs are associated with 35% fewer co-changes than properly collocated pairs, both indicating a successful identification of modular structures detrimental to software quality; and (3) both forms of PairSmell persist across software evolution. This evidence strongly suggests that PairSmell can provide meaningful insights for inspecting modular structure, with the identified issues being both granular and fundamental, making the enhancement of modular design more efficient.

Managing technical debt (TD) is essential for maintaining long-term software projects. Nonetheless, the time and cost involved in technical debt management (TDM) are often high, which may lead practitioners to omit TDM tasks. The adoption of tools, and particularly the usage of automated solutions, can potentially reduce the time, cost, and effort involved. However, the adoption of tools remains low, indicating the need for further research on TDM automation. To address this problem, this study aims at understanding which TDM activities practitioners are discussing with respect to automation in TDM, what tools they report for automating TDM, and the challenges they face that require automated solutions. To this end, we conducted a mining software repositories (MSR) study on three websites of Stack Exchange (Stack Overflow, Project Management, and Software Engineering) and collected 216 discussions, which were analyzed using both thematic synthesis and descriptive statistics. We found that identification and measurement are the most cited activities. Furthermore, 51 tools were reported as potential alternatives for TDM automation. Finally, a set of nine main challenges were identified and clustered into two main categories: challenges driving TDM automation and challenges related to tool usage. These findings highlight that tools for automating TDM are being discussed and used; however, several significant barriers persist, such as tool errors and poor explainability, hindering the adoption of these tools. Moreover, further research is needed to investigate the automation of other TDM activities such as TD prioritization.

Abstract Context Maintaining software projects over the long term requires controlling the accumulation of technical debt (TD). However, the time and cost associated with technical debt management (TDM) are often high, hindering practitioners from performing TDM tasks. Using tools for TDM has the potential to reduce the effort involved. Despite this, the adoption of such tools remains low, indicating a need for more efficient tool support. Objective This study aims to understand practitioners’ perspectives on tool support for TDM, specifically regarding the selection and use of these tools. Additionally, we identified potential requirements that could be implemented into existing or new TD tools. Method We surveyed practitioners and received 103 answers, from which 89 valid answers were analyzed using thematic synthesis and descriptive statistics. Results Practitioners’ decision-making processes regarding adopting tools are primarily driven by ten main concerns identified from practitioners’ responses (e.g., the load of information provided by tools). Additionally, we elicited 46 requirements and classified them into two main categories (“Information to be provided” and “Tool Usage”). Conclusion Practitioners aim to maintain control over tool execution and outputs. Our study then highlights the necessity of human-centered approaches for TDM automation, i.e., not only tools are essential, but the interaction between tools and practitioners is critical for a more efficient TDM.

Self-adaptation equips a software system with a feedback loop that resolves uncertainties during operation and adapts the system to deal with them when necessary. Most self-adaptation approaches today use decision-making mechanisms that select for execution the adaptation option with the best-estimated benefit expressed as a set of adaptation goals. A few approaches also consider the estimated (one-off) cost of executing the candidate adaptation options. We argue that besides benefit and cost, decision-making in self-adaptive systems should also consider the estimated risk the system or its users would be exposed to if an adaptation option were selected for execution. Balancing all three concerns when evaluating the options for adaptation to mitigate uncertainty is essential for satisfying stakeholders’ concerns and ensuring the safety and public acceptance of self-adaptive systems. In this article, we present a reference model for decision-making in self-adaptation that considers the estimated benefit, cost, and risk as core concerns of each adaptation option. Leveraging this model, we then present an ISO/IEC/IEEE 42010 compatible architectural viewpoint that aims at supporting software architects responsible for designing robust decision-making mechanisms for self-adaptive systems. We demonstrate the applicability, usefulness, and understandability of the viewpoint through a case study where participants with experience in the engineering of self-adaptive systems performed a set of design tasks in DeltaIoT, an Internet-of-Things exemplar for research on self-adaptive systems.

Self-Admitted Technical Debt (SATD) refers to circumstances where developers use textual artifacts to explain why the existing implementation is not optimal. Past research in detecting SATD has focused on either identifying SATD (classifying SATD items as SATD or not) or categorizing SATD (labeling instances as SATD that pertain to requirement, design, code, test debt, etc.). However, the performance of these approaches remains suboptimal, particularly for specific types of SATD, such as test and requirement debt, primarily due to extremely imbalanced datasets. To address these challenges, we build on earlier research by utilizing BiLSTM architecture for the binary identification of SATD and BERT architecture for categorizing different types of SATD. Despite their effectiveness, both architectures struggle with imbalanced data. Therefore, we employ a large language model data augmentation strategy to mitigate this issue. Furthermore, we introduce a two-step approach to identify and categorize SATD across various datasets derived from different artifacts. Our contributions include providing a balanced dataset for future SATD researchers and demonstrating that our approach significantly improves SATD identification and categorization performance compared to baseline methods.

Technical debt (TD) refers to non-optimal decisions made in software projects that may lead to short-term benefits, but potentially harm the system’s maintenance in the long-term. Technical debt management (TDM) refers to a set of activities that are performed to handle TD, e.g., identification or measurement of TD. These activities typically entail tasks such as code and architectural analysis, which can be time-consuming if done manually. Thus, substantial research work has focused on automating TDM tasks (e.g., automatic identification of code smells). However, there is a lack of studies that summarize current approaches in TDM automation. This can hinder practitioners in selecting optimal automation strategies to efficiently manage TD. It can also prevent researchers from understanding the research landscape and addressing the research problems that matter the most. The main objective of this study is to provide an overview of the state of the art in TDM automation, analyzing the available tools, their use, and the challenges in automating TDM. We conducted a systematic mapping study (SMS), following the guidelines proposed by Kitchenham et al. From an initial set of 1086 primary studies, 178 were selected to answer three research questions covering different facets of TDM automation. We found 121 automation artifacts that can be used to automate TDM activities. The artifacts were classified in 4 different types (i.e., tools, plugins, scripts, and bots); the inputs/outputs and interfaces were also collected and reported. Finally, a conceptual model is proposed that synthesizes the results and allows to discuss the current state of TDM automation and related challenges. The research community has investigated to a large extent how to perform various TDM activities automatically, considering the number of studies and automation artifacts we identified. Nonetheless, more research is needed towards fully automated TDM, specially concerning the integration of the automation artifacts.

Code review is a common practice in software development and often conducted before code changes are merged into the code repository. A number of approaches for automatically recommending appropriate reviewers have been proposed to match such code changes to pertinent reviewers. However, such approaches are generic, i.e., they do not focus on specific types of issues during code reviews. In this paper, we propose an approach that focuses on architecture violations, one of the most critical type of issues identified during code review. Specifically, we aim at automating the recommendation of code reviewers, who are potentially qualified to review architecture violations, based on reviews of code changes. To this end, we selected three common similarity detection methods to measure the file path similarity of code commits and the semantic similarity of review comments. We conducted a series of experiments on finding the appropriate reviewers through evaluating and comparing these similarity detection methods in separate and combined ways with the baseline reviewer recommendation approach, RevFinder. The results show that the common similarity detection methods can produce acceptable performance scores and achieve a better performance than RevFinder. The sampling techniques used in recommending code reviewers can impact the performance of reviewer recommendation approaches. We also discuss the potential implications of our findings for both researchers and practitioners.

Software engineers share their architectural knowledge (AK) in different places on the Web. Recent studies show that architectural blogs contain the most relevant AK, which can help software engineers to make design steps. Nevertheless, we know little about blogs, and specifically architectural blogs, where software engineers share their AK. In this paper, we conduct an exploratory study on architectural blogs to explore their types, topics, and their AK. Moreover, we determine the relevance of architectural blogs to make design steps. Our results support researchers and practitioners to find and re-use AK from blogs.

Technical debt, specifically Self-Admitted Technical Debt (SATD), remains a significant challenge for software developers and managers due to its potential to adversely affect long-term software maintainability. Although various approaches exist to identify SATD, tools for its comprehensive management are notably lacking. This paper presents DebtViz, an innovative SATD tool designed to automatically detect, classify, visualize and monitor various types of SATD in source code comments and issue tracking systems. DebtViz employs a Convolutional Neural Network-based approach for detection and a deconvolution technique for keyword extraction. The tool is structured into a back-end service for data collection and pre-processing, a SATD classifier for data categorization, and a front-end module for user interaction. DebtViz not only makes the management of SATD more efficient but also provides in-depth insights into the state of SATD within software systems, fostering informed decision-making on managing it. The scalability and deployability of DebtViz also make it a practical tool for both developers and managers in diverse software development environments. The source code of DebtViz is available at https://github.com/yikun-li/visdom-satd-management-system and the demo of DebtViz is at https://youtu.be/QXH6Bj0HQew.

Self-Admitted Technical Debt or SATD can be found in various sources, such as source code comments, commit messages, issue tracking systems, and pull requests. Previous research has established the existence of relations between SATD items in different sources; such relations can be useful for investigating and improving SATD management. However, there is currently a lack of approaches for automatically detecting these SATD relations. To address this, we proposed and evaluated approaches for automatically identifying SATD relations across different sources. Our findings show that our approach outperforms baseline approaches by a large margin, achieving an average F1-score of 0.829 in identifying relations between SATD items. Moreover, we explored the characteristics of SATD relations in 103 open-source projects and describe nine major cases in which related SATD is documented in a second source, and give a quantitative overview of 26 kinds of relations.

Abstract Context The management of Interface Control Documents (ICDs) has shown to be a major pain point in the architecting processes of Systems of Systems (SoS). Objective This work aims to improve on previously identified ICD management issues using the documentation-as-code philosophy as a potential basis for a treatment, and in collaboration with practitioners. Method We conducted a Technical Action Research (TAR) study with a group of engineers at the Netherlands Radio Astronomy Institute (ASTRON), in the context of the LOFAR radio telescope. An additional research instrument, in the form of an expert panel, was used to evaluate the transferability of the proposed treatment to alternative domains. Results In-depth insights on previously identified interface management issues were gained. Based on these insights a functional proof-of-concept was developed aimed at addressing these issues following the documentation-as-code principles. In addition to receiving overall positive reviews from practitioners and experts, further areas of improvement and transferability considerations for future work were identified. Conclusions The proposed approach, which to our knowledge has not been explored before in this context, is promising to address some of the recurring interfacing-related issues with directed SoS in multiple engineering domains. This could be done mainly by enforcing consistency and completeness on both text-based and formal elements of the ICDs, and turning ICDs into single sources of truth for the architecting processes of large scale SoS.

Abstract Technical debt refers to taking shortcuts to achieve short-term goals while sacrificing the long-term maintainability and evolvability of software systems. A large part of technical debt is explicitly reported by the developers themselves; this is commonly referred to as Self-Admitted Technical Debt or SATD. Previous work has focused on identifying SATD from source code comments and issue trackers. However, there are no approaches available for automatically identifying SATD from other sources such as commit messages and pull requests, or by combining multiple sources. Therefore, we propose and evaluate an approach for automated SATD identification that integrates four sources: source code comments, commit messages, pull requests, and issue tracking systems. Our findings show that our approach outperforms baseline approaches and achieves an average F1-score of 0.611 when detecting four types of SATD (i.e., code/design debt, requirement debt, documentation debt, and test debt) from the four aforementioned sources. Thereafter, we analyze 23.6M code comments, 1.3M commit messages, 3.7M issue sections, and 1.7M pull request sections to characterize SATD in 103 open-source projects. Furthermore, we investigate the SATD keywords and relations between SATD in different sources. The findings indicate, among others, that: 1) SATD is evenly spread among all sources; 2) issues and pull requests are the two most similar sources regarding the number of shared SATD keywords, followed by commit messages, and then followed by code comments; 3) there are four kinds of relations between SATD items in the different sources.

Context: As a software system evolves, its architecture tends to degrade, and gradually impedes software maintenance and evolution activities and negatively impacts the quality attributes of the system. The main root cause behind architecture erosion phenomenon derives from violation symptoms (i.e., various architecturally-relevant violations, such as violations of architecture pattern). Previous studies focus on detecting violations in software systems using architecture conformance checking approaches. However, code review comments are also rich sources that may contain extensive discussions regarding architecture violations, while there is a limited understanding of violation symptoms from the viewpoint of developers. Objective: In this work, we investigated the characteristics of architecture violation symptoms in code review comments from the developers’ perspective. Methods: We employed a set of keywords Related to violation symptoms to collect 606 (out of 21,583) code review comments from four popular OSS projects in the openStack and qt communities. We manually analyzed the collected 606 review comments to provide the categories and linguistic patterns of violation symptoms, as well as the reactions how developers addressed them. Results: Our findings show that: (1) three main categories of violation symptoms are discussed by developers during the code review process ; (2) The frequently-used terms of expressing violation symptoms are “ inconsistent ” and “ violate ”, and the most common linguistic pattern is Problem Discovery ; (3) Refactoring and removing code are the major measures (90%) to tackle violation symptoms, while a few violation symptoms were ignored by developers. Conclusions: Our findings suggest that the investigation of violation symptoms can help researchers better understand the characteristics of architecture erosion and facilitate the development and maintenance activities, and developers should explicitly manage violation symptoms, not only for addressing the existing architecture violations but also preventing future violations.

Context: Although Technical Debt (TD) has increasingly gained attention in recent years, most studies exploring TD are based on a single source (e.g., source code, code comments or issue trackers). Objective: Investigating information combined from different sources may yield insight that is more than the sum of its parts. In particular, we argue that exploring how TD items are managed in both issue trackers and software repositories (including source code and commit messages) can shed some light on what happens between the commits that incur TD and those that pay it back. Method: To this end, we randomly selected 3,000 issues from the trackers of five projects, manually analyzed 300 issues that contained TD information, and identified and investigated the lifecycle of 312 TD items. Results: The results indicate that most of the TD items marked as resolved in issue trackers are also paid back in source code, although many are not discussed after being identified in the issue tracker. Test Debt items are the least likely to be paid back in source code. We also learned that although TD items may be resolved a few days after being identified, it often takes a long time to be identified (around one year). In general, time is reduced if the same developer is involved in consecutive moments (i.e., introduction, identification, repayment decision-making and remediation), but whether the developer who paid back the item is involved in discussing the TD item does not seem to affect how quickly it is resolved. Conclusions: Investigating how developers manage TD across both source code repositories and issue trackers can lead to a more comprehensive oversight of this activity and support efforts to shorten the lifecycle of undesirable debt.

Identifying self-admitted technical debt (SATD) plays an important role in maintaining software stability and improving software quality. Although existing methods can detect SATD and researchers have identified design debt and requirement debt, an approach to realize multiple classification of SATD, including defect, test, and documentation, is still lacking. In this paper, we combine text generation oversampling and the Convolutional Neural Networks-Gated Recurrent Unit (CNNGRU) model, and propose an approach called SCGRU to classify multiple debt, including defect, test, documentation, design, and requirement. First, SeqGAN-based text generation is employed to generate new samples by learning the original SATD data, thereby increasing the number of SATD samples such as defect debt and reducing data imbalance. Then, we apply the CNNGRU model to refine SATD into multiple classes. An experiment with cross-project identification of 10 projects shows that our approach is more effective than existing methods such as CNN and text mining. The proposed SCGRU approach has strong advantages especially in cases of flawed debt with very unbalanced data such as test debt and documention debt.

Technical debt denotes shortcuts taken during software development, mostly for the sake of expedience. When such shortcuts are admitted explicitly by developers (e.g., writing a TODO/Fixme comment), they are termed asSelf-Admitted Technical DebtorSATD. There has been a fair amount of work studying SATD management in Open Source projects, but SATD in industry is relatively unexplored. At the same time, there is no work focusing on developers’ perspectives towards SATD and its management. To address this, we conducted an exploratory case study in cooperation with an industrial partner to study how they think of SATD and how they manage it. Specifically, we collected data by identifying and characterizing SATD in different sources (issues, source code comments, and commits) and carried out a series of interviews with 12 software practitioners. The results show: 1) the core characteristics of SATD in industrial projects; 2) developers’ attitudes towards identified SATD and statistics; 3) triggers for practitioners to introduce and repay SATD; 4) relations between SATD in different sources; 5) practices used to manage SATD; 6) challenges and tooling ideas for SATD management.

A key aspect of technical debt (TD) management is the ability to measure the amount of principal accumulated in a system. The current literature contains an array of approaches to estimate TD principal, however, only a few of them focus specifically on architectural TD, but none of them satisfies all three of the following criteria: being fully automated, freely available, and thoroughly validated. Moreover, a recent study has shown that many of the current approaches suffer from certain shortcomings, such as relying on hand-picked thresholds. In this paper, we propose a novel approach to estimate architectural technical debt principal based on machine learning and architectural smells to address such shortcomings. Our approach can estimate the amount of technical debt principal generated by a single architectural smell instance. To do so, we adopt novel techniques from Information Retrieval to train a learning-to-rank machine learning model (more specifically, a gradient boosting machine) that estimates the severity of an architectural smell and ensure the transparency of the predictions. Then, for each instance, we statically analyse the source code to calculate the exact number of lines of code creating the smell. Finally, we combine these two values to calculate the technical debt principal. To validate the approach, we conducted a case study and interviewed 16 practitioners, from both open source and industry, and asked them about their opinions on the TD principal estimations for several smells detected in their projects. The results show that for 71% of instances, practitioners agreed that the estimations provided wererepresentativeof the effort necessary to refactor the smell.

The phenomenon of architecture erosion can negatively impact the maintenance and evolution of software systems, and manifest in a variety of symptoms during software development. While erosion is often considered rather late, its symptoms can act as early warnings to software developers, if detected in time. In addition to static source code analysis, code reviews can be a source of detecting erosion symptoms and subsequently taking action. In this study, we investigate the erosion symptoms discussed in code reviews, as well as their trends, and the actions taken by developers. Specifically, we conducted an empirical study with the two most active Open Source Software (OSS) projects in the OpenStack community (i.e., Nova and Neutron). We manually checked 21,274 code review comments retrieved by keyword search and random selection, and identified 502 code review comments (from 472 discussion threads) that discuss erosion. Our findings show that (1) the proportion of erosion symptoms is rather low, yet notable in code reviews and the most frequently identified erosion symptoms are architectural violation, duplicate functionality, and cyclic dependency; (2) the declining trend of the identified erosion symptoms in the two OSS projects indicates that the architecture tends to stabilize over time; and (3) most code reviews that identify erosion symptoms have a positive impact on removing erosion symptoms, but a few symptoms still remain and are ignored by developers. The results suggest that (1) code review provides a practical way to reduce erosion symptoms; and (2) analyzing the trend of erosion symptoms can help get an insight about the erosion status of software systems, and subsequently avoid the potential risk of architecture erosion.

Abstract Technical debt is a metaphor indicating sub-optimal solutions implemented for short-term benefits by sacrificing the long-term maintainability and evolvability of software. A special type of technical debt is explicitly admitted by software engineers (e.g. using a TODO comment); this is called Self-Admitted Technical Debt or SATD. Most work on automatically identifying SATD focuses on source code comments. In addition to source code comments, issue tracking systems have shown to be another rich source of SATD, but there are no approaches specifically for automatically identifying SATD in issues. In this paper, we first create a training dataset by collecting and manually analyzing 4,200 issues (that break down to 23,180 sections of issues) from seven open-source projects (i.e., Camel, Chromium, Gerrit, Hadoop, HBase, Impala, and Thrift) using two popular issue tracking systems (i.e., Jira and Google Monorail). We then propose and optimize an approach for automatically identifying SATD in issue tracking systems using machine learning. Our findings indicate that: 1) our approach outperforms baseline approaches by a wide margin with regard to the F1-score; 2) transferring knowledge from suitable datasets can improve the predictive performance of our approach; 3) extracted SATD keywords are intuitive and potentially indicating types and indicators of SATD; 4) projects using different issue tracking systems have less common SATD keywords compared to projects using the same issue tracking system; 5) a small amount of training data is needed to achieve good accuracy.

Abstract Architectural smells (AS) are notorious for their long-term impact on the Maintainability and Evolvability of software systems. The majority of research work has investigated this topic by mining software repositories of open source Java systems, making it hard to generalise and apply them to an industrial context and other programming languages. To address this research gap, we conducted an embedded multiple-case case study, in collaboration with a large industry partner, to study how AS evolve in industrial embedded systems. We detect and track AS in 9 C/C++ projects with over 30 releases for each project that span over two years of development, with over 20 millions lines of code in the last release only. In addition to these quantitative results, we also interview 12 among the developers and architects working on these projects, collecting over six hours of qualitative data about the usefulness of AS analysis and the issues they experienced while maintaining and evolving artefacts affected by AS. Our quantitative findings show how individual smell instances evolve over time, how long they typically survive within the system, how they overlap with instances of other smell types, and finally what the introduction order of smell types is when they overlap. Our qualitative findings, instead, provide insights on the effects of AS on the long-term maintainability and evolvability of the system, supported by several excerpts from our interviews. Practitioners also mention what parts of the AS analysis actually provide actionable insights that they can use to plan refactoring activities.

The challenges posed by the architecting of System of Systems (SoS) has motivated a significant number of research efforts in the area. However, literature is lacking when it comes to the interplay between the disciplines involved in the architecting process, a key factor in addressing these challenges. This paper aims to contribute to this line of research by confirming and extending previously characterized architecting harmonization practices from Systems and Software Engineering, adopted in an ultra-large-scale SoS. We conducted a confirmatory case study on the Square-Kilometre Array (SKA) project to evaluate and extend the findings of our exploratory case on the LOFAR/LOFAR2.0 radio-telescope projects. In doing so, a pre-study was conducted to map the findings of the previous study with respect to the SKA context. A survey was then designed, through which the views of 46 SKA engineers were collected and analyzed. The study confirmed in various degrees the four practices identified in the exploratory case, and provided further insights about them: (1) the friction between disciplines caused by long-term system requirements, and how they can be ameliorated through intermediate, short-term requirements; (2) the way design choices with a cross-cutting impact on multiple agile teams have an indirect impact on the system architecture; (3) how these design choices are often caused by the criteria that guided early system decomposition; (4) the seemingly recurrent issue with the lack of details about the dynamic elements of the interfaces; and (5) the use of machine-readable interface specifications for aligning hardware/software development processes. The findings of this study and its predecessor support the importance of a cross-disciplinary view in the Software Engineering research agenda in SoS as a whole, not to mention their value as a convergence point for research on SoS architecting from the Systems and Software Engineering standpoints.

Technical Debt (TD) can be paid back either by those that incurred it or by others. We call the former self-fixed TD, and it can be particularly effective, as developers are experts in their own code and are well-suited to fix the corresponding TD issues. The goal of our study is to investigate self-fixed technical debt, especially the extent in which TD is self-fixed, which types of TD are more likely to be self-fixed, whether the remediation time of self-fixed TD is shorter than non-self-fixed TD and how development behaviors are related to self-fixed TD. We report on an empirical study that analyzes the self-fixed issues of five types of TD (i.e., Code, Defect, Design, Documentation and Test), captured via static analysis, in more than 44,000 commits obtained from 20 Python and 16 Java projects of the Apache Software Foundation. The results show that about half of the fixed issues are self-fixed and that the likelihood of contained TD issues being self-fixed is negatively correlated with project size, the number of developers and total issues. Moreover, there is no significant difference of the survival time between self-fixed and non-self-fixed issues. Furthermore, developers are more keen to pay back their own TD when it is related to lower code level issues, e.g., Defect Debt and Code Debt. Finally, developers who are more dedicated to or knowledgeable about the project contribute to a higher chance of self-fixing TD. These results can benefit both researchers and practitioners by aiding the prioritization of TD remediation activities and refining strategies within development teams, and by informing the development of TD management tools.

With an increase in Agile, Lean, and DevOps software methodologies over the last years (collectively referred to as Continuous Software Development (CSD)), we have observed that documentation is often poor. This work aims at collecting studies on documentation challenges, documentation practices, and tools that can support documentation in CSD. A systematic mapping study was conducted to identify and analyze research on documentation in CSD, covering publications between 2001 and 2019. A total of 63 studies were selected. We found 40 studies related to documentation practices and challenges, and 23 studies related to tools used in CSD. The challenges include: informal documentation is hard to understand, documentation is considered as waste, productivity is measured by working software only, documentation is out-of-sync with the software and there is a short-term focus. The practices include: non-written and informal communication, the usage of development artifacts for documentation, and the use of architecture frameworks. We also made an inventory of numerous tools that can be used for documentation purposes in CSD. Overall, we recommend the usage of executable documentation, modern tools and technologies to retrieve information and transform it into documentation, and the practice of minimal documentation upfront combined with detailed design for knowledge transfer afterwards. It is of paramount importance to increase the quantity and quality of documentation in CSD. While this remains challenging, practitioners will benefit from applying the identified practices and tools in order to mitigate the stated challenges.

Abstract Architecture erosion (AEr) can adversely affect software development and has received significant attention in the last decade. However, there is an absence of a comprehensive understanding of the state of research about the reasons and consequences of AEr, and the countermeasures to address AEr. This work aims at systematically investigating, identifying, and analyzing the reasons, consequences, and ways of detecting and handling AEr. With 73 studies included, the main results are as follows: (1) AEr manifests not only through architectural violations and structural issues but also causing problems in software quality and during software evolution; (2) non‐technical reasons that cause AEr should receive the same attention as technical reasons, and practitioners should raise awareness of the grave consequences of AEr, thereby taking actions to tackle AEr‐related issues; (3) a spectrum of approaches, tools, and measures has been proposed and employed to detect and tackle AEr; and (4) three categories of difficulties and five categories of lessons learned on tackling AEr were identified. The results can provide researchers a comprehensive understanding of AEr and help practitioners handle AEr and improve the sustainability of their architecture. More empirical studies are required to investigate the practices of detecting and addressing AEr in industrial settings.

Abstract Although architectural smells are one of the most studied type of architectural technical debt, their impact on maintenance effort has not been thoroughly investigated. Studying this impact would help to understand how much technical debt interest is being paid due to the existence of architecture smells and how this interest can be calculated. This work is a first attempt to address this issue by investigating the relation between architecture smells and source code changes. Specifically, we study whether thefrequencyandsizeof changes are correlated with the presence of a selected set of architectural smells. We detect architectural smells using the Arcan tool, which detects architectural smells by building a dependency graph of the system analyzed and then looking for the typical structures of the architectural smells. The findings, based on a case study of 31 open‐source Java systems, show that 87% of the analyzed commits present more changes in artifacts with at least one smell, and the likelihood of changing increases with the number of smells. Moreover, there is also evidence to confirm that change frequency increases after the introduction of a smell and that the size of changes is also larger in smelly artifacts. These findings hold true especially in Medium–Large and Large artifacts.

While technical debt grows in absolute numbers as software systems evolve over time, the density of technical debt (technical debt divided by lines of code) is reduced in some cases. This can be explained by either the application of refactorings or the development of new artifacts with limited Technical Debt. In this paper we explore the second explanation, by investigating the relation between the amount of Technical Debt in new code and the evolution of Technical Debt in the system. To this end, we compare the Technical Debt Density of new code with existing code, and we investigate which of the three major types of code changes (additions, deletions and modifications) is primarily responsible for changes in the evolution of Technical Debt density. Furthermore, we study whether there is a relation between code quality practices and the ‘cleanness’ of new code. To obtain the required data, we have performed a large-scale case study on twenty-seven open-source software projects by the Apache Software Foundation, analyzing 66,661 classes and 56,890 commits. The results suggest that writing “clean” (or at least “cleaner”) new code can be an efficient strategy for reducing Technical Debt Density, and thus preventing software decay over time. The findings also suggest that projects adopting an explicit policy for quality improvement, e.g., through discussions on code quality in board meetings, are associated with a higher frequency of cleaner new code commits. Therefore, we champion the establishment of processes that monitor the density of Technical Debt of new code to control the accumulation of Technical Debt in a software system.

The problems caused by the gap between system- and software-level architecting practices, especially in the context of Systems of Systems where the two disciplines inexorably meet, is a well known issue with a disappointingly low amount of works in the literature dedicated to it. At the same time, organizations working on Systems of Systems have been developing solutions for closing this gap for many years now. This work aims to extract such knowledge from practitioners by studying the case of a large-scale scientific instrument, a geographically distributed radio telescope to be more specific, developed as a sequence of projects during the last two decades. As the means for collecting data for this study we combine online interviews with a virtual focus group of practitioners from the organization responsible for building the instrument. Through this process, we identify persisting problems and the best practices that have been developed to deal with them, together with the perceived benefits and drawbacks of applying the latter in practice. Some of our major findings include the need to avoid over-reliance on the flexibility of software to compensate for incomplete requirements, hidden assumptions, as well as late involvement of system architecting, and to facilitate the cooperation between the involved disciplines through dedicated architecting roles and the adoption of unifying practices and standards.

Software engineers need relevant and up-to-date architectural knowledge (AK), in order to make well-founded design decisions. However, finding such AK is quite challenging. One pragmatic approach is to search for AK on the web using traditional search engines (e.g. Google); this is common practice among software engineers. Still, we know very little about what AK is retrieved, from where, and how useful it is. In this paper, we conduct an empirical study with 53 software engineers, who used Google to make design decisions using the Attribute-Driven-Design method. Based on how the subjects assessed the nature and relevance of the retrieved results, we determined how effective web search engines are to find relevant architectural information. Moreover, we identified the different sources of AK on the web and their associated AK concepts.

The impact of Technical Debt (TD) on software maintenance and evolution is of great concern, but recent evidence shows that a considerable amount of TD is fixed by the same developers who introduced it; this is termed self-fixed TD. This characteristic of TD management can potentially impact team dynamics and practices in managing TD. However, the initial evidence is based on low-level source code analysis; this casts some doubt whether practitioners repay their own debt intentionally and under what circumstances. To address this gap, we conducted an online survey on 17 well-known Java and Python open-source software communities to investigate practitioners' intent and rationale for self-fixing technical debt. We also investigate the relationship between human-related factors (e.g., experience) and self-fixing. The results, derived from the responses of 181 participants, show that a majority addresses their own debt consciously and often. Moreover, those with a higher level of involvement (e.g., more experience in the project and number of contributions) tend to be more concerned about self-fixing TD. We also learned that the sense of responsibility is a common self-fixing driver and that decisions to fix TD are not superficial but consider balancing costs and benefits, among other factors. The findings in this paper can lead to improving TD prevention and management strategies.

As software systems evolve, their architecture is meant to adapt accordingly by following the changes in requirements, the environment, and the implementation. However, in practice, the evolving system often deviates from the architecture, causing severe consequences to system maintenance and evolution. This phenomenon of architecture erosion has been studied extensively in research, but not yet been examined from the point of view of developers. In this exploratory study, we look into how developers perceive the notion of architecture erosion, its causes and consequences, as well as tools and practices to identify and control architecture erosion. To this end, we searched through several popular online developer communities for collecting data of discussions related to architecture erosion. Besides, we identified developers involved in these discussions and conducted a survey with 10 participants and held interviews with 4 participants. Our findings show that: (1) developers either focus on the structural manifestation of architecture erosion or on its effect on run-time qualities, maintenance and evolution; (2) alongside technical factors, architecture erosion is caused to a large extent by non-technical factors; (3) despite the lack of dedicated tools for detecting architecture erosion, developers usually identify erosion through a number of symptoms; and (4) there are effective measures that can help to alleviate the impact of architecture erosion.

Context: Making architectural decisions is a crucial task but also very difficult, considering the scope of the decisions and their impact on quality attributes.To make matters worse, architectural decisions need to combine both technical and business factors, which are very dissimilar by nature.Objectives: We provide a cost-benefit approach and supporting tooling that treats architectural decisions as financial investments by: (a) combining both technical and business factors; and (b) transforming the involved factors into currency, allowing their uniform aggregation.Apart from illustrating the method, we validate both the proposed approach and the tool, in terms of fitness for purpose, usability, and potential limitations.Method: To validate the approach, we have performed a case study in a software development company, in the domain of low-energy embedded systems.We employed triangulation in the data collection phase of the case study, by performing interviews, focus groups, an observational session, and questionnaires.Results: The results of the study suggested that the proposed approach: (a) provides a structured process for systematizing decision-making; (b) enables the involvement of multiple stakeholders, distributing the decision-making responsibility to more knowledgeable people; (c) uses monetized representations that are important for assessing decisions in a unified manner; and (d) enables decision reuse and documentation. Conclusions:The results of the study suggest that architectural decision-making can benefit from treating this activity as a financial investment.The various benefits that have been identified from mixing financial and technological aspects are well-accepted from industrial stakeholders.

During software development, some architectural design decisions incur technical debt, either deliberately or inadvertently. These have serious impact on the quality of a software system, and can cost significant time and effort to be changed. While current research efforts have explored general concepts of architectural design decisions and technical debt separately, debt-incurring architectural design decisions have not been specifically explored in practice. In this case study, we explore debt-incurring architectural design decisions (DADDs) in practice. Specifically, we explore the main types of DADDs, why and how they are incurred in a software system, and how practitioners deal with these types of design decisions. We performed interviews and a focus group with practitioners working in embedded and enterprise software companies, discussing their concrete experience with such architectural design decisions. We provide the following contributions: 1) A categorization for the types of DADDs, which extend a current ontology on architectural design decisions. 2) A process on how deliberate DADDs are made in practice. 3) A conceptual model which shows the relationships between the causes and triggers of inadvertent DADDs. 4) The main factors that influence the way of dealing with DADDs. The results can support the development of new approaches and tools for Architecture Technical Debt management from the perspective of Design Decisions. Moreover, they support future research to capture architecture knowledge related to DADDs.

Abstract During the last decades, software traceability has been studied in a large number of studies, from different perspectives (e.g., how to create traces and what are its benefits). This large body of knowledge needs to be better explored and exploited by both practitioners and researchers: We need an overview of different aspects of traceability and a structured way to assess and compare existing work in order to extend it with new research or apply it in practice, Thus, we have conducted a secondary study on this large corpus of primary studies, focusing on empirical studies on software traceability, without setting any further restrictions in terms of investigating a specific domain or concrete artifacts. The study explores the goals of existing approaches and the empirical methods used for their evaluation. Its main contributions are the investigation of (a) the type of artifacts linked through traceability approaches; (b) the benefits of using artifact traceability approaches; (c) the ways of measuring their benefit; and (d) the research methods used. The results of the study suggest that (i) requirements artifacts are dominant in traceability; (ii) the research corpus focuses on the proposal of novel techniques for establishing traceability; and (iii) the main benefits are the improvement of software correctness and maintainability. Finally, although many studies include some empirical validation, there is still room for improvements and research methods that can be used more extensively. The obtained results are discussed under the prism of both researchers and practitioners and are compared against the state‐of‐the‐art.

Abstract In recent years, the evolution of software ecosystems and the detection of technical debt received significant attention by researchers from both industry and academia. While a few studies that analyze various aspects of technical debt evolution already exist, to the best of our knowledge, there is no large‐scale study that focuses on the remediation of technical debt over time in Python projects—that is, one of the most popular programming languages at the moment. In this paper, we analyze the evolution of technical debt in 44 Python open‐source software projects belonging to the Apache Software Foundation. We focus on the type and amount of technical debt that is paid back. The study required the mining of over 60K commits, detailed code analysis on 3.7K system versions, and the analysis of almost 43K fixed issues. The findings show that most of the repayment effort goes into testing, documentation, complexity, and duplication removal. Moreover, more than half of the Python technical debt is short term being repaid in less than 2 months. In particular, the observations that a minority of rules account for the majority of issues fixed and spent effort suggest that addressing those kinds of debt in the future is important for research and practice.

Unix has evolved for almost five decades, shaping modern operating systems, key software technologies, and development practices. Studying the evolution of this remarkable system from an architectural perspective can provide insights on how to manage the growth of large, complex, and long-lived software systems. Along main Unix releases leading to the FreeBSD lineage we examine core architectural design decisions, the number of features, and code complexity, based on the analysis of source code, reference documentation, and related publications. We report that the growth in size has been uniform, with some notable outliers, while cyclomatic complexity has been religiously safeguarded. A large number of Unix-defining design decisions were implemented right from the very early beginning, with most of them still playing a major role. Unix continues to evolve from an architectural perspective, but the rate of architectural innovation has slowed down over the system's lifetime. Architectural technical debt has accrued in the forms of functionality duplication and unused facilities, but in terms of cyclomatic complexity it is systematically being paid back through what appears to be a self-correcting process. Some unsung architectural forces that shaped Unix are the emphasis on conventions over rigid enforcement, the drive for portability, a sophisticated ecosystem of other operating systems and development organizations, and the emergence of a federated architecture, often through the adoption of third-party subsystems. These findings have led us to form an initial theory on the architecture evolution of large, complex operating system software.

Background: The Systems Engineering and Software Engineering disciplines are highly intertwined in most modern Systems of Systems (SoS), and particularly so in industries such as defense, transportation, energy and health care. However, the combination of these disciplines during the architecting of SoS seems to be especially challenging; the literature suggests that major integration and operational issues are often linked to ambiguities and gaps between system-level and software-level architectures.

Technical debt refers to taking shortcuts to achieve short-term goals, which might negatively influence software maintenance in the long-term. There is increasing attention on technical debt that is admitted by developers in source code comments (termed as self-admitted technical debt or SATD). But SATD in issue trackers is relatively unexplored. We performed a case study, where we manually examined 500 issues from two open source projects (i.e. Hadoop and Camel), which contained 152 SATD items. We found that: 1) eight types of technical debt are identified in issues, namely architecture, build, code, defect, design, documentation, requirement, and test debt; 2) developers identify technical debt in issues in three different points in time, and a small part is identified by its creators; 3) the majority of technical debt is paid off, 4) mostly by those who identified it or created it; 5) the median time and average time to repay technical debt are 872.3 and 25.0 hours respectively.

Technical debt (TD) reflects issues that may negatively affect software maintenance and evolution. There is currently little evidence on how the different types of TD co-occur; for example, how code smells and design smells affect the same part of the system. This paper investigates how different types of TD co-occur, as well as the time period of the co-occurrence. To that end, we analyzed the co-occurring associations between five types of TD, captured in 42 SonarQube rules, in 3862 files of 20 Python projects from the Apache Software Foundation. We found that this phenomenon is dominant, affecting more than 90% of Python files. We also found that Documentation Debt and Test Debt appear in the majority of the files, although it seems to be mostly by coincidence. Finally, we noticed that co-occurrence of TD seems to happen very quickly: co-occurring issues tend to be introduced within the same week. But once it does happen, it is hard to get rid of. These results can benefit both researchers and practitioners by: aiding the prioritization of TD remediation; leading to novel tools for detecting co-occurring TD and warning potential issues; shedding further light on the explanation of how TD is introduced and can be mitigated.

Technical Debt (TD) can be paid back either by those that incurred it or by others. We call the former self-fixed TD, and it is particularly effective, as developers are experts in their own code and are best-suited to fix the corresponding TD issues. To what extent is TD self-fixed, which types of TD are more likely to be self-fixed and is the remediation time of self-fixed TD shorter than non-self-fixed TD? This paper attempts to answer these questions. It reports on an empirical study that analyzes the self-fixed issues of five types of TD (i.e., Code, Defect, Design, Documentation and Test), captured via static analysis, in more than 17,000 commits from 20 Python projects of the Apache Software Foundation. The results show that more than two thirds of the issues are self-fixed and that the self-fixing rate is negatively correlated with the number of commits, developers and project size. Furthermore, the survival time of self-fixed issues is generally shorter than non-self-fixed issues. Moreover, the majority of Defect Debt tends to be self-fixed and has a shorter survival time, while Test Debt and Design Debt are likely to be fixed by other developers. These results can benefit both researchers and practitioners by aiding the prioritization of TD remediation activities within development teams, and by informing the development of TD management tools.

The cornerstones of technical debt (TD) are two concepts borrowed from economics: principal and interest. Although in economics the two terms are related, in TD there is no study on this direction so as to validate the strength of the metaphor. We study the relation between Principal and Interest, and subsequently dig further into the ‘ingredients’ of each concept (since they are multi-faceted). In particular, we investigate if artifacts with similar levels of TD Principal exhibit a similar amount of TD Interest, and vice-versa. To achieve this goal, we performed an empirical study, analyzing the dataset using the Mantel test. Through the Mantel test, we examined the relation between TD Principal and Interest, and identified aspects that are able to denote proximity of artifacts, with respect to TD. Next, through Linear Mixed Effects (LME) modelling we studied the generalizability of the results. The results of the study suggest that TD Principal and Interest are related, in the sense that classes with similar levels of TD Principal tend to have similar levels of Interest. Additionally, we have reached the conclusion that aggregated measures of TD Principal or Interest are more capable of identifying proximate artifacts, compared to isolated metrics. Finally, we have provided empirical evidence on the fact that improving certain quality properties (e.g., size and coupling) should be prioritized while ranking refactoring opportunities in the sense that high values of these properties are in most of the cases related to artifacts with higher levels of TD Principal. The findings shed light on the relations between the two concepts, and can be useful for both researchers and practitioners: the former can get a deeper understanding of the concepts, whereas the latter can use our findings to guide their TD management processes such as prioritization and repayment.

Context: The term System of Systems (SoS) has increasingly been used in a wide variety of domains to describe those systems composed of independent constituent systems that collaborate towards a mission that they could not accomplish on their own. There is a significant volume of research by the software architecture community that aims to overcome the challenges involved in architecting SoS, as evidenced by the number of secondary studies in the field published so far. However, the boundaries of such research do not seem to be well defined, at least partially, due to the emergence of SoS-adjacent areas of interest like the Internet of Things . Objective: This paper aims to investigate the current state of research on SoS architecting by synthesizing the demographic data, assessing the quality and the coverage of architecting activities and software quality attributes by the research, and distilling a concept map that reflects a community-wide understanding of the concept of SoS. Method: We conduct what is, to the best of our understanding, the first tertiary study on SoS architecting. Such tertiary study was based on five research questions, and was performed by following the guidelines of Kitchenham et al. In all, 19 secondary studies were evaluated, which is comparable to other tertiary studies. Results: The study illustrates a state of disconnection in the research community, with research gaps in the coverage of particular phases and quality attributes. Furthermore, a more effective approach in classifying systems as SoS is required, as the means of resolving conceptual and terminological overlaps with the related domains. Conclusions: Despite the amount of research in the area of SoS architecting, more coordinated and systematic targeted efforts are required in order to address the identified issues with the current state of research.

Abstract The embedded systems domain has grown exponentially over the past years. The industry is forced by the market to rapidly improve and release new products to beat the competition. Frenetic development rhythms thus shape this domain and give rise to several new challenges for software design and development. One of them is dealing with trade-offs between run-time and design-time quality attributes. To study practices, processes and tools concerning the management of run-time and design-time quality attributes as well as the trade-offs among them from the perspective of embedded systems software engineers. An exploratory case study with two qualitative data collection steps, namely interviews and a focus group, involving six different companies from the embedded systems domain with a total of twenty participants. The interviewed subjects showed a preference for run-time over design-time qualities. Trade-offs between design-time and run-time qualities are very common, but they are often implicit, due to the lack of adequate monitoring tools and practices. Practitioners prefer to deal with trade-offs in the most lightweight way possible, by applying ad-hoc practices, thus avoiding any overhead incurred. Finally, practitioners have elaborated on how they envision the ideal tool support for dealing with trade-offs. Although it is notoriously difficult to deal with trade-offs, constantly monitoring the quality attributes of interest with automated tools is key in making explicit and prudent trade-offs and mitigating the risk of incurring technical debt.

One of the primary drivers for self-adaptation is ensuring that systems achieve their goals regardless of the uncertainties they face during operation. Nevertheless, the concept of uncertainty in self-adaptive systems is still insufficiently understood. Several taxonomies of uncertainty have been proposed, and a substantial body of work exists on methods to tame uncertainty. Yet, these taxonomies and methods do not fully convey the research community’s perception on what constitutes uncertainty in self-adaptive systems and on the key characteristics of the approaches needed to tackle uncertainty. To understand this perception and learn from it, we conducted a survey comprising two complementary stages in which we collected the views of 54 and 51 participants, respectively. In the first stage, we focused on current research and development, exploring how the concept of uncertainty is understood in the community and how uncertainty is currently handled in the engineering of self-adaptive systems. In the second stage, we focused on directions for future research to identify potential approaches to dealing with unanticipated changes and other open challenges in handling uncertainty in self-adaptive systems. The key findings of the first stage are: (a) an overview of uncertainty sources considered in self-adaptive systems, (b) an overview of existing methods used to tackle uncertainty in concrete applications, (c) insights into the impact of uncertainty on non-functional requirements, (d) insights into different opinions in the perception of uncertainty within the community and the need for standardised uncertainty-handling processes to facilitate uncertainty management in self-adaptive systems. The key findings of the second stage are: (a) the insight that over 70% of the participants believe that self-adaptive systems can be engineered to cope with unanticipated change, (b) a set of potential approaches for dealing with unanticipated change, (c) a set of open challenges in mitigating uncertainty in self-adaptive systems, in particular in those with safety-critical requirements. From these findings, we outline an initial reference process to manage uncertainty in self-adaptive systems. We anticipate that the insights on uncertainty obtained from the community and our proposed reference process will inspire valuable future research on self-adaptive systems.

Although managing architectural assumptions can benefit software development in several aspects (e.g., reducing architectural misunderstanding and mismatch), the effort required is a key obstacle towards employing architectural assumption management in practice. One potential solution is to apply agile practices in order to reduce this effort. To this end, we conducted a survey with 91 practitioners to investigate the possibility of integrating agile practices into architectural assumption management in industrial practice. The results offer an overview of which agile practices can be integrated in architectural assumption management and how. Six agile practices were selected by more than half of the subjects: "Backlog", "Iterative and Incremental Development", "Refactoring", "Continuous Integration", "Effective Communication", and "Just Enough Work". Twelve agile practices were further elaborated by the subjects regarding how they can be used in architectural assumption management. Based on the survey results, we developed a classification of agile practices for agile architectural assumption management, which can act as a reference for researchers and practitioners to employ certain agile practices in architectural assumption management.

Architectural smells can be detrimental to the system maintainability, evolvability and represent a source of architectural debt. Thus, it is very important to be able to understand how they evolved in the past and to predict their future evolution. In this paper, we evaluate if the existence of architectural smells in the past versions of a project can be used to predict their presence in the future. We analyzed four Java projects in 295 Github releases and we applied for the prediction four different supervised learning models in a repeated cross-validation setting. We found that historical architectural smell information can be used to predict the presence of architectural smells in the future. Hence, practitioners should carefully monitor the evolution of architectural smells and take preventative actions to avoid introducing them and stave off their progressive growth.

Architectural smells may substantially increase maintenance effort and thus require extra attention for potential refactoring. While we currently understand this concept and have identified different types of such smells, we have not yet studied their evolution in depth. This is necessary to inform their prioritisation and refactoring. This study analyses the evolution of individual architectural smell instances over time, and the characteristics that define these instances. Three different types of architectural smells are taken into consideration and mined from a total of 524 versions across 14 different projects. The results show how different smell types differ in multiple aspects, such as their growth rate, the importance of the affected elements over time in the dependency network of the system, and the time each instance affects the system. They also cast valuable insights on what aspects are the most important to consider during prioritisation and refactoring activities.

Secondary studies are vulnerable to threats to validity. Although, mitigating these threats is crucial for the credibility of these studies, we currently lack a systematic approach to identify, categorize and mitigate threats to validity for secondary studies. In this paper, we review the corpus of secondary studies, with the aim to identify: (a) the trend of reporting threats to validity, (b) the most common threats to validity and corresponding mitigation actions, and (c) possible categories in which threats to validity can be classified. To achieve this goal we employ the tertiary study research method that is used for synthesizing knowledge from existing secondary studies. In particular, we collected data from more than 100 studies, published until December 2016 in top quality software engineering venues (both journals and conference). Our results suggest that in recent years, secondary studies are more likely to report their threats to validity. However, the presentation of such threats is rather ad hoc, e.g., the same threat may be presented with a different name, or under a different category. To alleviate this problem, we propose a classification schema for reporting threats to validity and possible mitigation actions. Both the classification of threats and the associated mitigation actions have been validated by an empirical study, i.e., Delphi rounds with experts. Based on the proposed schema, we provide a checklist, which authors of secondary studies can use for identifying and categorizing threats to validity and corresponding mitigation actions, while readers of secondary studies can use the checklist for assessing the validity of the reported results.

Abstract To capitalize upon the benefits of software reuse, an efficient selection among candidate reusable assets should be performed in terms of functional fitness and adaptability. The reusability of assets is usually measured through reusability indices. However, these do not capture all facets of reusability, such as structural characteristics, external quality attributes, and documentation. In this paper, we propose a reusability index (REI) as a synthesis of various software metrics and evaluate its ability to quantify reuse, based on IEEE Standard on Software Metrics Validity. The proposed index is compared with existing ones through a case study on 80 reusable open‐source assets. To illustrate the applicability of the proposed index, we performed a pilot study, where real‐world reuse decisions have been compared with decisions imposed by the use of metrics (including REI). The results of the study suggest that the proposed index presents the highest predictive and discriminative power; it is the most consistent in ranking reusable assets and the most strongly correlated to their levels of reuse. The findings of the paper are discussed to understand the most important aspects in reusability assessment (interpretation of results), and interesting implications for research and practice are provided.

Empirical evidence has pointed out that Extract Method refactorings are among the most commonly applied refactorings by software developers. The identification of Long Method code smells and the ranking of the associated refactoring opportunities is largely based on the use of metrics, primarily with measures of cohesion, size and coupling. Despite the relevance of these proper-ties to the presence of large, complex and non-cohesive pieces of code, the empirical validation of these metrics has exhibited relatively low accuracy (max precision: 66%) regarding their predictive power for long methods or extract method opportunities. In this work we perform an empirical validation of the ability of cohesion, coupling and size metrics to predict the existence and the intensity of long method occurrences. According to the statistical analysis, the existence and the intensity of the Long Method smell can be effectively predicted by two size (LoC and NoLV), two coupling (MPC and RFC), and four cohesion (LCOM1, LCOM2, Coh, and CC) metrics. Furthermore, the integration of these metrics into a multiple logistic regression model can predict whether a method should be refactored with a precision of 89% and a recall of 91%. The model yields suggestions whose ranking is strongly correlated to the ranking based on the effect of the corresponding refactorings on source code (correl. coef. 0.520). The results are discussed by providing interpretations and implications for research and practice.

Technical debt management entails the quantification of principal and interest. In our previous work we had introduced a framework for calculating the Technical Debt Breaking Point (TD-BP), which is a point in time where the accumulated interest becomes larger than the principal; thus the debt of the company is no longer sustainable after this point in time. In this paper, we instantiate this framework and validate its ability to assess the breaking point of source code modules in an industrial setting. The results of the validation suggest that the calculated TD-BP is strongly correlated to experts' opinion on the sustainability of modules, and that it can accurately rank components, based on their maintenance difficulty.

Monitoring software quality is a non-trivial task, since it requires the selection of: (a) quality attributes, based on application domain and development phase, and (b) appropriate metrics to quantify them. We aim to aid this process by reviewing the state-of-research on design-time qualities and metrics.

During software evolution technical debt (TD) follows a constant ebb and flow, being incurred and paid back, sometimes in the same day and sometimes ten years later. There have been several studies in the literature investigating how technical debt in source code accumulates during time and the consequences of this accumulation for software maintenance. However, to the best of our knowledge there are no large scale studies that focus on the types of issues that are fixed and the amount of TD that is paid back during software evolution. In this paper we present the results of a case study, in which we analyzed the evolution of fifty-seven Java open-source software projects by the Apache Software Foundation at the temporal granularity level of weekly snapshots. In particular, we focus on the amount of technical debt that is paid back and the types of issues that are fixed. The findings reveal that a small subset of all issue types is responsible for the largest percentage of TD repayment and thus, targeting particular violations the development team can achieve higher benefits.

Change proneness is a quality characteristic of software artifacts that represents their probability to change in the future due to: (a) evolving requirements, (b) bug fixing, or (c) ripple effects. In the literature, change proneness has been associated with many negative consequences along software evolution. For example, artifacts that are change-prone tend to produce more defects, and accumulate more technical debt. Therefore, identifying and monitoring modules of the system that are change-prone is of paramount importance. Assessing change proneness requires information from two sources: (a) the history of changes in the artifact as a proxy of how frequently the artifact itself is changing, and (b) the source code structure that affects the probability of a change being propagated among artifacts. In this paper, we propose a method for assessing the change proneness of classes based on the two aforementioned information sources. To validate the proposed approach, we performed a case study on five open-source projects. Specifically, we compared the accuracy of the proposed approach to the use of other software metrics and change history to assess change proneness, based on the 1061-1998 IEEE Standard on Software Measurement. The results of the case study suggest that the proposed method is the most accurate and reliable assessor of change proneness. The high accuracy of the method suggests that the method and accompanying tool can effectively aid practitioners during software maintenance and evolution.

Abstract Gang of Four (GoF) patterns are well‐known best practices for the design of object‐oriented systems. In this paper, we aim at empirically assessing their relationship to energy consumption, ie, a performance indicator that has recently attracted the attention of both researchers and practitioners. To achieve this goal, we investigate pattern‐participating methods (ie, those that play a role within the pattern) and compare their energy consumption to the consumption of functionally equivalent alternative (nonpattern) solutions. We obtained the alternative solution by refactoring the pattern instances using well‐known transformations (eg, replace polymorphism with conditional statements). The comparison is performed on 169 methods of 2 GoF patterns (namely, State/Strategy and Template Method), retrieved from 2 well‐known open source projects. The results suggest that for the majority of cases the alternative design excels in terms of energy consumption. However, in some cases (eg, when the method is large in size or invokes many methods) the pattern solution presents similar or lower energy consumption. The outcome of our study can be useful to both researchers and practitioners, because we: (1) provide evidence on a possible negative effect of GoF patterns, and (2) can provide guidance on which cases the use of the pattern is not hurting energy consumption.

`Extract Method' is considered one of the most frequently applied and beneficial refactorings, since the corresponding Long Method smell is among the most common and persistent ones. Although Long Method is conceptually related to the implementation of diverse functionalities within a method, until now, this relationship has not been utilized while identifying refactoring opportunities. In this paper we introduce an approach (accompanied by a tool) that aims at identifying source code chunks that collaborate to provide a specific functionality, and propose their extraction as separate methods. The accuracy of the proposed approach has been empirically validated both in an industrial and an open-source setting. In the former case, the approach was capable of identifying functionally related statements within two industrial long methods (approx. 500 LoC each), with a recall rate of 93 percent. In the latter case, based on a comparative study on open-source data, our approach ranks better compared to two well-known techniques of the literature. To assist software engineers in the prioritization of the suggested refactoring opportunities the approach ranks them based on an estimate of their fitness for extraction. The provided ranking has been validated in both settings and proved to be strongly correlated with experts' opinion.

Cyber-foraging is a technique to enable mobile devices to extend their computing power and storage by offloading computation or data to more powerful servers located in the cloud or in single-hop proximity. While there is a large amount of research in this area, the reality is that there are not many deployed, operational cyber-foraging systems. As these systems become more prevalent due to their proven benefits, combined with the emergence of micro data centers and edge clouds, a need will arise for guidance on their architecture and development. To provide this guidance, we present a decision model based on a mapping of functional and non-functional requirements for cyber-foraging systems to a set of architectural tactics. The decision model was validated by developers to obtain an expert opinion on its correctness and usefulness for guiding the architecture, design, and evolution of cyber-foraging systems that meet their intended functional and non-functional requirements, while understanding the effects of decisions.

Software quality attributes are assessed by employing appropriate metrics. However, the choice of such metrics is not always obvious and is further complicated by the multitude of available metrics. To assist metrics selection, several properties have been proposed. However, although metrics are often used to assess successive software versions, there is no property that assesses their ability to capture structural changes along evolution. We introduce a property, Software Metric Fluctuation (SMF), which quantifies the degree to which a metric score varies, due to changes occurring between successive system's versions. Regarding SMF, metrics can be characterized as sensitive (changes induce high variation on the metric score) or stable (changes induce low variation on the metric score). SMF property has been evaluated by: (a) a case study on 20 OSS projects to assess the ability of SMF to differently characterize different metrics, and (b) a case study on 10 software engineers to assess SMF's usefulness in the metric selection process. The results of the first case study suggest that different metrics that quantify the same quality attributes present differences in their fluctuation. We also provide evidence that an additional factor that is related to metrics’ fluctuation is the function that is used for aggregating metric from the micro to the macro level. In addition, the outcome of the second case study suggested that SMF is capable of helping practitioners in metric selection, since: (a) different practitioners have different perception of metric fluctuation, and (b) this perception is less accurate than the systematic approach that SMF offers. SMF is a useful metric property that can improve the accuracy of metrics selection. Based on SMF, we can differentiate metrics, based on their degree of fluctuation. Such results can provide input to researchers and practitioners in their metric selection processes.

The importance of architectural knowledge (AK) management for software development has been highlighted over the past ten years, where a significant amount of research has been done. Since the first systems using design rationale in the seventies and eighties to the more modern approaches using AK for designing software architectures, a variety of models, approaches, and research tools have leveraged the interests of researchers and practitioners in AK management (AKM). Capturing, sharing, and using AK has many benefits for software designers and maintainers, but the cost to capture this relevant knowledge hampers a widespread use by software companies. However, as the improvements made over the last decade didn't boost a wider adoption of AKM approaches, there is a need to identify the successes and shortcomings of current AK approaches and know what industry needs from AK. Therefore, as researchers and promoters of many of the AK research tools in the early stages where AK became relevant for the software architecture community, and based on our experience and observations, we provide in this research an informal retrospective analysis of what has been done and the challenges and trends for a future research agenda to promote AK use in modern software development practices.

For technical reasons the number of authors shown on this cover page is limited to 10 maximum.

Combining software architecture and agile development has received significant attention in recent years. However, there exists no comprehensive overview of the state of research on the architecture-agility combination. This work aims to analyze the combination of architecture and agile methods for the purpose of exploration and analysis with respect to architecting activities and approaches, agile methods and practices, costs, benefits, challenges, factors, tools, and lessons learned concerning the combination. A systematic mapping study (SMS) was conducted, covering the literature on the architecture-agility combination published between February 2001 and January 2014. Fifty-four studies were finally included in this SMS. Some of the highlights: (1) a significant difference exists in the proportion of various architecting activities, agile methods, and agile practices employed in the combination. (2) none of the architecting approaches has been widely used in the combination. (3) there is a lack of description and analysis regarding the costs and failure stories of the combination. (4) twenty challenges, twenty-nine factors, and twenty-five lessons learned were identified. The results of this SMS help the software engineering community to reflect on the past thirteen years of research and practice on the architecture-agility combination with a number of implications.

Managing architectural assumptions (AA) during the software lifecycle, as an important type of architecture knowledge, is critical to the success of projects. However, little empirical evidence exists on the understanding, identification, and recording of AA from the practitioners’ perspective. We investigated the current situation on (1) how practitioners understand AA and its importance, and (2) whether and how practitioners identify and record AA in software development. A web-based survey was conducted with 112 practitioners, who use Chinese as native language and are engaged in software development in China. The main findings are: (1) AA are important in both software architecting and development. However, practitioners understand AA in different ways; (2) only a few respondents identified and recorded AA in their projects, and very few approaches and tools were used for identifying and recording AA; (3) the lack of specific approaches and tools is the major challenge (reason) of (not) identifying and recording AA. The results emphasize the need for a widely accepted understanding of the AA concept in software development, and specific approaches, tools, and guidelines to support AA identification and recording.

Population aging has brought important challenges at social, economical, and healthcare issues, mainly due to the increasing number of people that need intensive care. Ambient Assisted Living (AAL) systems that aim at assisting people in their health-related activities have emerged. In another perspective, Reference Architectures (RAs) are a special type of software architecture that promotes reuse of design expertise and facilitates the development, standardization, and evolution of software systems. During the last years, important RAs for AAL systems have been created. However, there is a lack of studies that compare RAs for healthcare systems in AAL domain, making the selection among RAs a rather difficult task. In this paper, we present a comparative analysis and evaluate the completeness of that healthcare RAs. Specifically, we intend to offer: (i) a guide to select the most complete and adequate RA for software projects, and (ii) to describe research directions to improve existing and future RAs for that systems.

Ambient Assisted Living (AAL) has become focus of attention for governmental organisations, and healthcare and ICT researchers. AAL includes methods, products, services, and software systems to support the everyday lives of elders. To develop AAL software systems, reference architectures (i.e., a special software architecture that guides the development, standardization, and evolution of systems architectures) have been proposed. Despite of the existing reference architectures, their use is a difficult task, due their high level of abstraction. Moreover, these architectures do not support the development of systems for telemonitoring and self-management of chronic diseases at home. The main objective of this project is to propose a reference architecture for Healthcare Supportive Home (HSH) systems. With this project we will contribute in promoting and reusing the knowledge of expertise and in supporting the development, standardization, evolution, and quality of HSH software systems.

Context: Change impact analysis investigates the negative consequence of system changes, i.e., the propagation of changes to other parts of the system (also known as the ripple effect). Identifying modules of the system that will be affected by the ripple effect is an important activity, before and after the application of any change. Goal: However, in the literature, there is only a limited set of studies that investigate the probability of a random change occurring in one class, to propagate to another. In this paper we discuss and evaluate the Ripple Effect Measure (in short REM), a metric that can be used to assess the aforementioned probability. Method: To evaluate the capacity of REM as an assessor of the prob-ability of a class to change due to the ripple effect, we: (a) mathematically validate it against established metric properties (e.g., non-negativity, monotonicity, etc.), proposed by Briand et al., and (b) empirically investigate its validity as an assessor of class proneness to the ripple effect, based on the 1061-1998 IEEE Standard on Software Measurement (e.g., correlation, predictive power, etc.). To apply the empirical validation process, we conducted a holistic multiple-case study on java open-source classes. Results: The results of REM validation (both mathematical and empirical) suggest that REM is a theoretically sound measure that is the most valid assessor of the probability of a class to change due to the ripple effect, compared to other existing metrics.

Systems-of-Systems (SoS) refer to a new class of software-intensive systems, where their constituent systems work cooperatively in order to fulfill specific missions. Characterized by managerial and operational independence, geographic distribution, evolutionary development, and emergent behavior, SoS bring substantial challenges to the software engineering area. SESoS 2015, held in Florence, Italy, on May 17, 2015, as a joint workshop of the 37th International Conference on Software Engineering (ICSE), provided a forum to exchange ideas and experiences, analyze current research and development issues, discuss promising solutions, and to explore inspiring visions for the future of Software Engineering (SE) for SoS.

Software engineers and architects of complex software systems need to balance hard quality attribute requirements while at the same time manage risks and make decisions with a system-wide and long-lasting impact. To achieve these tasks efficiently, they need quantitative information about design-time and run-time system aspects through usable and quick tools. While there is body of work focusing on code quality and metrics, their applicability at the design and architecture level and at scale are inconsistent and not proven. We are interested in exploring whether architecture can assist with better contextualizing existing system and code quality and metrics approaches. Furthermore, we ask whether we need additional architecture-level metrics to make progress and whether something as complex and subtle as software architecture can be quantified. The goal of this workshop is to discuss progress, gather empirical evidence, and identify priorities for a research agenda on architecture and metrics in the software engineering field.

Architectural technical debt (ATD) is incurred by design decisions that intentionally or unintentionally compromise system-wide quality attributes, particularly maintainability and evolvability. ATD is harmful to the system's long-term health, thus it needs to be identified for further management. However, existing ATD identification approaches are mainly based on source code analysis and thus suffer from certain shortcomings: they can only identify issues at the system implementation, they can only be employed after the systems is implemented in code, they lack a mechanism to confirm whether the potential ATD identified is real ATD or not. To address these issues, we proposed an ATD identification approach based on architecture decisions and change scenarios. To evaluate the effectiveness and usability of this approach, we conducted a case study with an information system in a large telecommunications company. The results show that the proposed approach is useful and easy to use, and it supports release planning and ATD interest measurement.

Context Technical debt (TD) is a metaphor reflecting technical compromises that can yield short-term benefit but may hurt the long-term health of a software system. Objective This work aims at collecting studies on TD and TD management (TDM), and making a classification and thematic analysis on these studies, to obtain a comprehensive understanding on the TD concept and an overview on the current state of research on TDM. Method A systematic mapping study was performed to identify and analyze research on TD and its management, covering publications between 1992 and 2013. Results Ninety-four studies were finally selected. TD was classified into 10 types, 8 TDM activities were identified, and 29 tools for TDM were collected. Conclusions The term “debt” has been used in different ways by different people, which leads to ambiguous interpretation of the term. Code-related TD and its management have gained the most attention. There is a need for more empirical studies with high-quality evidence on the whole TDM process and on the application of specific TDM approaches in industrial settings. Moreover, dedicated TDM tools are needed for managing various types of TD in the whole TDM process.

Stability refers to a software system's resistance to the “ripple effect”, i.e., propagation of changes. In this paper, we investigate the stability of classes that participate in instances/occurrences of GoF design patterns. We examine whether the stability of such classes is affected by (a) the pattern type, (b) the role that the class plays in the pattern, (c) the number of pattern occurrences in which the class participates, and (d) the application domain. To this end, we conducted a case study on about 65.000 Java open-source classes, where we performed change impact analysis on classes that participate in zero, one (single pattern), or more than one (coupled) pattern occurrences. The results suggest that, the application of design patterns can provide the expected “shielding” of certain pattern-participating classes against changes, depending on their role in the pattern. Moreover, classes that participate in coupled pattern occurrences appear to be the least stable. The results can be used for assessing the benefits and liabilities of the use of patterns and for testing and refactoring prioritization, because less stable classes are expected to require more effort while testing, and urge for refactoring activities that would make them more resistant to change propagation.

Architecture decisions are often not explicitly documented in practice but reside in the architect's mind as tacit knowledge, even though explicit capturing and documentation of architecture decisions has been associated with a multitude of benefits. As part of a research collaboration with ABB, we developed a tool to document architecture decisions. This tool is an add-in for Enterprise Architect and is an implementation of a viewpoint-based decision documentation framework. To validate the add-in, we conducted an exploratory case study with ABB architects. In the study, we assessed the status quo of architecture decision documentation, identified architects' expectations of the ideal decision documentation tool, and evaluated the new add-in. We found that although awareness of decision documentation is increasing at ABB, several barriers exist that limit the use of decisions in practice. Regarding their ideal tool, architects want a descriptive and efficient approach. Supplemental features like reporting or decision sharing are requested. The new add-in, was well-perceived by the architects. As a result of the study, we propose a clearer separation of problem, outcomes, and alternatives for the decision documentation framework.

Software architects make architectural decisions such as choosing architecture patterns or frameworks. Capturing architectural decisions reduces evolution costs for software systems. Existing approaches overlook the challenge of capturing tacit knowledge about architectural decisions. Previously, we proposed the REGAIN approach to capture tacit knowledge about architectural decisions. REGAIN is based on the Repertory Grid technique, a powerful knowledge acquisition approach from knowledge engineering. However, REGAIN lacked industrial validation. Therefore, in this paper, we initially present a study to ensure that REGAIN meets the needs of industrial architects. We interviewed 16 architects who indicated REGAIN advantages such as systematic decision-making support. Also, architects indicated improvement opportunities, in particular tool support and the possibility to prioritize concerns that are used to evaluate decision alternatives. Therefore, we conducted an additional study to evaluate two approaches for prioritizing concerns: pairwise-comparisons and the hundred-dollar approach. We conducted an experiment with 30 graduate students to compare the two prioritization approaches. Based on the results of the experiment, we added the hundred-dollar approach to REGAIN. Moreover, based on the study with architects and the experiment with students, we implemented open source tool support for REGAIN, which architects can use to capture architectural decisions.

A number of organizations have adopted Open Source Software (OSS) development practices to support or augment their software development processes, a phenomenon frequently referred to as Inner Source . However the adoption of Inner Source is not a straightforward issue. Many organizations are struggling with the question of whether Inner Source is an appropriate approach to software development for them in the first place. This article presents a framework derived from the literature on Inner Source, which identifies nine important factors that need to be considered when implementing Inner Source. The framework can be used as a probing instrument to assess an organization on these nine factors so as to gain an understanding of whether or not Inner Source is suitable. We applied the framework in three case studies at Philips Healthcare, Neopost Technologies, and Rolls-Royce, which are all large organizations that have either adopted Inner Source or were planning to do so. Based on the results presented in this article, we outline directions for future research.

Context: Variability (i.e., the ability of software systems or artifacts to be adjusted for different contexts) became a key property of many systems. Objective: We analyze existing research on variability in software systems. We investigate variability handling in major software engineering phases (e.g., requirements engineering, architecting). Method: We performed a systematic literature review. A manual search covered 13 premium software engineering journals and 18 premium conferences, resulting in 15,430 papers searched and 196 papers considered for analysis. To improve reliability and to increase reproducibility, we complemented the manual search with a targeted automated search. Results: Software quality attributes have not received much attention in the context of variability. Variability is studied in all software engineering phases, but testing is underrepresented. Data to motivate the applicability of current approaches are often insufficient; research designs are vaguely described. Conclusions: Based on our findings we propose dimensions of variability in software engineering. This empirically grounded classification provides a step towards a unifying, integrated perspective of variability in software systems, spanning across disparate or loosely coupled research themes in the software engineering community. Finally, we provide recommendations to bridge the gap between research and practice and point to opportunities for future research.

Context: Investing on Open Source Software (OSS) as a "code reuser", involves certain risks, such as the difficulty in understanding the level of OSS design quality Aim: We investigate the levels of design quality of OSS projects, across different application domains. Method: We conducted a case study, which is the most fitting research method for observing a phenomenon in its real context, which is active for a long period of time, and for which variables cannot be controlled. Results: We present the values for seven design quality metrics of 546 OSS projects, as well as the statistically significant differences across application domains. Conclusions: The results of the study suggest that OSS application domains correlate with several design quality characteristics, in the sense that projects within one application domain appear to have similar levels of design quality. In addition to that, the results reveal application domains with high and low levels of design quality.

The disciplines of requirements engineering (RE) and software architecture (SA) are fundamental to the success of software projects. Even though RE and SA are often considered separately, it has been argued that drawing a line between RE and SA is neither feasible nor reasonable as requirements and architectural design processes impact each other. Requirements are constrained by what is feasible technically and also by time and budget restrictions. On the other hand, feedback from the architecture leads to renegotiating architecturally significant requirements with stakeholders. The topic of bridging RE and SA has been discussed in both the RE and SA communities, but mostly independently. Therefore, the motivation for this ICSE workshop is to bring both communities together in order to identify key issues, explore the state of the art in research and practice, identify emerging trends, and define challenges related to the transition and the relationship between RE and SA.

Software architecture design is challenging, especially for junior software designers. Lacking practice and experience, junior designers need process support in order to make rational architecture decisions. In this paper, we present the results of a comparative multiple-case study conducted to find out if decision viewpoints from van Heesch et al., 2012, van Heesch et al., in press can provide such a support. The case study was conducted with four teams of software engineering students working in industrial software projects. Two of the four teams were instructed to document their decisions using decision viewpoints; the other two teams were not instructed to do so. We observed the students for a period of seven weeks by conducting weekly focus groups and by analyzing their work artifacts and minutes. Our findings suggest that junior designers who use decision viewpoints are more systematic in exploring and evaluating solution options. However, the decision viewpoints did not help them in managing requirements and complexity.

SUMMARY This paper presents an approach to construct execution views, which are views that describe what the software of a software‐intensive system does at runtime and how it does it. The approach represents an architecture reconstruction solution based on a metamodel, a set of viewpoints, and a dynamic analysis technique. The metamodel and viewpoints capture the conventions that can be used to describe the runtime of a system developed by a particular organization. The dynamic analysis technique is used to extract and abstract runtime information from a combination of system logging and runtime measurements in a top‐down fashion. The approach was developed and validated constructing execution views for a magnetic resonance imaging scanner developed by Philips Healthcare. Therefore, the approach represents a solution that can be applied at similar large and complex software‐intensive systems. Copyright © 2011 John Wiley & Sons, Ltd.

SUMMARY Architectural patterns are often applied in combination with related patterns within software architectures. The relationships among architectural patterns must be considered when applying a combination of patterns into a system; for example the way the Model‐View‐Controller uses the Observer pattern to implement the change propagation mechanism needs to be carefully designed. However, effective integration of architectural patterns within software architectures remains a challenging task. This is because the integration of any two architectural patterns can take several forms. Furthermore, existing pattern languages define generic and abstract relationships between architectural patterns without going into detail about associations among the participants of architectural patterns. In this paper, we propose to address the pattern integration issue by discovering and defining a set of pattern participants relationships that serve the purpose of effectively integrating architectural patterns. Our findings are validated through a controlled experiment, which provides significant evidence that the proposed relationships support inexperienced designers in integrating patterns. Copyright © 2011 John Wiley & Sons, Ltd.

Background: Many empirical software engineering studies use students as subj ects and are conducted as part of university courses.Aim: We aim at reporting our experiences with using guidelines for integrating empirical studies with our research and teaching goals.Method: We document our experience from conducting three studies with graduate students in two software architecture cou rses.Results: Our results show some problems that we faced when following the guidelines and deviations we made from the original guidelines.Conclusions: Based on our results we propose recommendations for empirical software engineering studies that are integrated in university courses.

In this paper, we introduce a documentation framework for architecture decisions. This framework consists of four viewpoint definitions using the conventions of ISO/IEC/IEEE 42010, the new international standard for the description of system and software architectures. The four viewpoints, a Decision Detail viewpoint, a Decision Relationship viewpoint, a Decision Chronology viewpoint, and a Decision Stakeholder Involvement viewpoint satisfy several stakeholder concerns related to architecture decision management. With the exception of the Decision Stakeholder Involvement viewpoint, the framework was evaluated in an industrial case study. The results are promising, as they show that decision views can be created with reasonable effort while satisfying many of the stakeholder concerns in decision documentation.

Abstract — Background: Case studies and experiments are research methods frequently applied in empirical software engineering. Experiments are well-understood and their value as an empirical method is recognized. On the other hand, there seem to be different opinions on what constitutes a case study, and about the value of case studies as a thorough research method. Aim: We aim at exploring the attitudes of software engineering researchers on case studies and experiments. Furthermore, we investigate how the perceptions of researchers vary along their views on what constitutes a case study. Method: We performed an exploratory survey involving 26 software engineering researchers. We collected data using a paper-based questionnaire. Results: We found that participants slightly prefer experiments over case studies. Moreover, participants believe there is more useful literature on experiments, than on case studies. By analyzing two different views on the nature of case studies, we found differences in the perceived validity of case studies. Conclusions: The survey provided insights into the perceptions of researchers on case studies and experiments. Moreover, the results help reconcile different views on case studies. Keywords- empirical software engineering; case studies; experiments; survey I.

Architectural Knowledge (AK) is defined as the integrated representation of the software architecture of a software-intensive system or family of systems along with architectural decisions and their rationale, external influence and the development environment. The SHARK workshop series focuses on current methods, languages, and tools that can be used to extract, represent, share, apply, and reuse AK, and the experimentation and/or exploitation thereof. This sixth edition of SHARK will discuss, among other topics, the approaches for AK personalization, where knowledge is not codified through templates or annotations, but it is exchanged through the discussion between the different stakeholders.

Knowledge about the architecture of a software-intensive system tends to vaporize easily. This leads to increased maintenance costs. We explore a new idea: utilizing the repertory grid technique to capture tacit architectural knowledge. Particularly, we investigate the elicitation of design decision alternatives and their characteristics. To study the applicability of this idea, we performed an exploratory study. Seven independent subjects applied the repertory grid technique to document a design decision they had to take in previous projects. Then, we interviewed each subject to understand their perception about the technique. We identified advantages and disadvantages of using the technique. The main advantage is the reasoning support it provides; the main disadvantage is the additional effort it requires. Also, applying the technique depends on the context of the project. Using the repertory grid technique is a promising approach for fighting architectural knowledge vaporization.

Variability helps manage differences and commonalities within and across software systems. As variability is reflected in and facilitated through the software architecture, it is important to understand the problems that architects face when carrying out their tasks. This would help us improve methods for architecting variability-intensive software systems. In this paper, we therefore present an exploratory study to identify problems that occur when performing variability-related tasks during software architecting. Our subjects were 27 graduate students. We identified eleven problems as experienced by the subjects of our study. The paper also presents implications of the findings for the software architecture field. In particular, we discuss implications for architecture description, methods and tools, and the training of architects.

Variability is the ability of a software artifact to be changed for a specific context. Mechanisms to accommodate variability include software product lines, configuration wizards and tools in commercial software, configuration interfaces of software components, or the dynamic runtime composition of web services. Variability is primarily reflected in and facilitated through the software architecture. Also, the software architecture is the centerpiece of software systems and acts as reference point for many development activities, and many of today's software systems are built to accommodate variability. Thus, variability in software architecture should be well-understood and be treated as a first-class concern. The software architecture community acknowledges that variability is a concern of different stakeholders, and in turn affects other concerns. Nevertheless, treating variability related to the architecture and all architecture aspects, as a cross-cutting concern, is currently not well understood. Therefore, VARSA 2011 aims at identifying critical challenges and progressing the state-of-the-art on variability in software architecture.

Architecting is to a large extent a decision-making process. While many approaches and tools exist to support architects during the various activities of architecting, little guidance exists to support the reasoning part of decision-making. This is partly due to our limited understanding of how professional architects make decisions. We report on findings of a survey that we have conducted with 53 industrial software architects to find out how they reason in real projects. The results of the survey are interpreted with respect to the industrial context and the architecture literature. We derive reasoning best practices that can support especially inexperienced architects with optimizing their decision-making process.

When following architecture-driven strategies to develop large software-intensive systems, the analysis of the dependencies is not an easy task. In this paper, we report a systematic literature review on dependency analysis solutions. Dependency analysis concerns making dependencies due to interconnections between programs or system components explicit. The review is practice-driven because its research questions, execution, and reporting were influenced by the practice of a group of software architects at Philips Healthcare MRI. The review results in an overview and assessment of the state-of-the-art and applicability of dependency analysis. The overview provides insights about definitions related to dependency analysis, the sort of development activities that need dependency analysis, and the classification and description of a number of dependency analysis solutions. The contribution of this paper is for both practitioners and researchers. They can take it as a reference to learn about dependency analysis, match their own practice to the presented results, and to build similar overviews of other techniques and methods for other domains or types of systems.

Architectural Knowledge (AK) is defined as the integrated representation of the software architecture of a software-intensive system or family of systems along with architectural decisions and their rationale, external influence and the development environment. The SHARK workshop series focuses on current methods, languages, and tools that can be used to extract, represent, share, apply, and reuse AK, and the experimentation and/or exploitation thereof. This fifth edition of SHARK will discuss, among other topics, the contributions of this community to a Body of Knowledge on software architecture.

Software architecture designers inevitably work with both architecture patterns and tactics. Architecture patterns describe the high-level structure and behavior of software systems as the solution to multiple system requirements, whereas tactics are design decisions that improve individual quality attribute concerns. Tactics that are implemented in existing architectures can have significant impact on the architecture patterns in the system. Similarly, tactics that are selected during initial architecture design significantly impact the architecture of the system to be designed: which patterns to use, and how they must be changed to accommodate the tactics. However, little is understood about how patterns and tactics interact. In this paper, we develop a model for the interaction of patterns and tactics that enables software architects to annotate architecture diagrams with information about the tactics used and their impact on the overall structure. This model is based on our in-depth analysis of the types of interactions involved, and we show several examples of how the model can be used to annotate different kinds of architecture diagrams. We illustrate the model and annotation by showing examples taken from real systems, and describe how the annotation was used in architecture reviews. Tactics and patterns are known architectural concepts; this work provides more specific and in-depth understanding of how they interact. Its other key contribution is that it explores the larger problem of understanding the relation between strategic decisions and how they need to be tailored in light of more tactical decisions.

In large-scale collaborative software projects, thousands of requirements with complex interdependencies and different granularity spreading in different levels are elicited, documented, and evolved during the project lifecycle. Non-technical stakeholders involved in requirements engineering activities rarely apply formal techniques; therefore it is infeasible to automatically detect problems in requirements. This situation becomes even worse in a distributed context when all sites are responsible to maintain their own requirements list using various requirements models and management tools, and the detection of requirements problems across multiple sites is error-prone, and un-affordable if performed manually. This paper proposes an integrated approach of basing distributed requirements analysis on semantic Wiki by requirements reasoning. First, the functions concerning reasoning support provided by semantic Wiki for requirements analysis are proposed. Second, the underlying requirements rationale model for requirements reasoning is presented with sample reasoning rules. Third, our rationale model is mapped to the WinWin requirements negotiation model which further adds to its credibility.

Architecture patterns are an important tool in architectural design. However, while many architecture patterns have been identified, there is little in-depth understanding of their actual use in software architectures. For instance, there is no overview of how many patterns are used per system or which patterns are the most common or most important for particular domains. In addition, little is known of how architecture patterns ay interact with each other. We studied architecture documentation of 47 systems to learn about their architecture patterns. Most systems had two or more architecture patterns, and certain patterns were prominent in different application domains. We identified several patterns that are commonly used together, and are beginning to learn how such combinations may impact system quality attributes. This information can be used to help designers select architecture patterns, can help people learn both architectures and patterns, and can be useful in architectural reviews.

Software architecting is a highly knowledge-intensive process demanding and producing a large and rich amount of information. To remain competitive, companies and organizations working in the IT sector must be able to manage this knowledge portfolio and effectively exploit and reuse it. In the era of Web 2.0, knowledge grids, social networking, global development and semantic Web, this working session addresses the problem of building a knowledge community in the field of software architecture. To this end, we aim at exploring the wishes of academics and industrial organizations, on the one hand, and their boundaries on he other. Our goal is to compare and contrast the inputs from academia and industry, and gain a shared understanding about what can be done now, and in the near future.

In contrast to software architecture models, architectural decisions are often not explicitly documented, and therefore eventually lost. This contributes to major problems such as high-cost system evolution, stakeholders mis-communication, and limited reusability of core system assets. An approach is outlined that systematically and semi-automatically documents architectural decisions and allows them to be effectively shared by the stakeholders. A first attempt is presented that partially implements the approach by binding architectural decisions, models and the system implementation. The approach is demonstrated with an example demonstrating its usefulness with regards to some industrial use cases.

Architectural patterns are a key point in architectural documentation. Regrettably, there is poor support for modeling architectural patterns, because the pattern elements are not directly matched by elements in modeling languages, and, at the same time, patterns support an inherent variability that is hard to model using a single modeling solution. This paper proposes tackling this problem by finding and representing architectural primitives, as the participants in the solutions that patterns convey. In particular, we examine a number of architectural patterns to discover those primitive abstractions that are common among the patterns, and at the same time demonstrate a degree of variability in each pattern. These abstractions belong in the components and connectors architectural view, though more abstractions can be found in other views. We have selected UML 2 as the language for representing these primitive abstractions as extensions of the standard UML elements. The added value of this approach is twofold: it proposes a generic and extensible approach for modeling architectural patterns by means of architectural primitives; it demonstrates an initial set of primitives that participate in several well-known architectural patterns.

There exists an urgent demand on defining architectures for learning technology systems (LTS), so that high-level frameworks for understanding these systems can be discovered, portability, interoperability and reusability can be achieved and adaptability over time can be accomplished. The authors propose an architecting process for only the software subsystem of an LTS. We base our work upon the LTSA working standard of IEEE LTSC, which serves as a business model and on the practices of a well-established software engineering process. Special emphasis is granted on imposing a component based nature on the produced architecture.