Background: A commit message describes the main code changes in a commit and plays a crucial role in software maintenance. Existing commit message generation (CMG) approaches typically frame it as a direct mapping which inputs a code diff and produces a brief descriptive sentence as output. Aims: Since the raw code diff lacks the context related to the code itself, we intend to supplement the relevant code as input of CMG to generate high-quality and informative commit messages. Method: We propose a contextual code retrieval-based method called C3Gen to enhance CMG by retrieving commit-relevant code snippets from the repository and incorporating them into the model input to provide richer contextual information at the repository scope. In the experiments, we evaluated the effectiveness of C3Gen across various models using four objective and three subjective metrics. Meanwhile, we design and conduct a human evaluation to investigate how C3Gen-generated commit messages are perceived by human developers. Results & Conclusions: By incorporating contextual code into the input, C3Gen enables models to leverage additional information to generate more comprehensive and informative commit messages with greater practical value in real-world development scenarios. Further analysis underscores concerns about the reliability of similarity-based metrics and provides empirical insights for CMG.

Abstract Context: Managing data related to a software product and its development poses significant challenges for software projects and agile development teams. These include integrating data from diverse sources and ensuring data quality amidst continuous change and adaptation. Objective: The paper systematically explores data management challenges and potential solutions in agile projects, aiming to provide insights into data management challenges and solutions for both researchers and practitioners. Method: We employed a mixed-methods approach, including a systematic literature review (SLR) to understand the state-of-research followed by a survey with practitioners to reflect on the state-of-practice. The SLR reviewed 45 studies, identifying and categorizing data management aspects along with their associated challenges and solutions. The practitioner survey captured practical experiences and solutions from 32 industry practitioners who were significantly involved in data management to complement the findings from the SLR. Results: Our findings identified major data management challenges in practice, such as managing data integration processes, capturing diverse data, automating data collection, and meeting real-time analysis requirements. To address these challenges, solutions such as automation tools, decentralized data management practices, and ontology-based approaches have been identified. These solutions enhance data integration, improve data quality, and enable real-time decision-making by providing flexible frameworks tailored to agile project needs. Conclusion: The study pinpointed significant challenges and actionable solutions in data management for agile software development. Our findings provide practical implications for practitioners and researchers, emphasizing the development of effective data management practices and tools to address those challenges and improve project success.

Containerization in multi-cloud environments has received significant attention in recent years both from academic research and industrial development perspectives. However, there exists no effort to systematically investigate the state of research on this topic. The aim of this research is to systematically identify and categorize the multiple aspects of containerization in multi-cloud environment. We conducted the Systematic Mapping Study (SMS) on the literature published between January 2013 and July 2024. One hundred twenty one studies were selected and the key results are: (1) Four leading themes on containerization in multi-cloud environment are identified: ‘Scalability and High Availability’, ‘Performance and Optimization’, ‘Security and Privacy’, and ‘Multi-Cloud Container Monitoring and Adaptation’. (2) Ninety-eight patterns and strategies for containerization in multi-cloud environment were classified across 10 subcategories and 4 categories. (3) Ten quality attributes considered were identified with 47 associated tactics. (4) Four catalogs consisting of challenges and solutions related to security, automation, deployment, and monitoring were introduced. The results of this SMS will assist researchers and practitioners in pursuing further studies on containerization in multi-cloud environment and developing specialized solutions for containerization applications in multi-cloud environment.

Compilers play a central role in translating high-level code into executable programs, making their correctness essential for ensuring code safety and reliability. While extensive research has focused on verifying the correctness of compilers for single-language compilation, the correctness of cross-language compilation — which involves the interaction between two languages and their respective compilers — remains largely unexplored. To fill this research gap, we propose CrossLangFuzzer , a novel framework that introduces a universal intermediate representation (IR) for JVM-based languages and automatically generates cross-language test programs with diverse type parameters and complex inheritance structures. After generating the initial IR, CrossLangFuzzer applies three mutation techniques — LangShuffler, FunctionRemoval , and TypeChanger — to enhance program diversity. By evaluating both the original and mutated programs across multiple compiler versions, CrossLangFuzzer successfully uncovered 10 confirmed bugs in the Kotlin compiler, 4 confirmed bugs in the Groovy compiler, 7 confirmed bugs in the Scala 3 compiler, 2 confirmed bugs in the Scala 2 compiler, and 1 confirmed bug in the Java compiler. Among all mutators, TypeChanger is the most effective, detecting 11 of the 24 compiler bugs. Furthermore, we analyze the symptoms and root causes of cross-compilation bugs, examining the respective responsibilities of language compilers when incorrect behavior occurs during cross-language compilation. To the best of our knowledge, this is the first work specifically focused on identifying and diagnosing compiler bugs in cross-language compilation scenarios. Our research helps to understand these challenges and contributes to improving compiler correctness in multi-language environments.

Modern code generation tools utilizing AI models like Large Language Models have gained increased popularity due to their ability to produce functional code. However, their usage presents security challenges, often resulting in insecure code merging into the code base. Thus, evaluating the quality of generated code, especially its security, is crucial. While prior research explored various aspects of code generation, the focus on security has been limited, mostly examining code produced in controlled environments rather than open source development scenarios. To address this gap, we conducted an empirical study, analyzing code snippets generated by GitHub Copilot and two other AI code generation tools (i.e., CodeWhisperer and Codeium) from GitHub projects. Our analysis identified 733 snippets, revealing a high likelihood of security weaknesses, with 29.5% of Python and 24.2% of JavaScript snippets affected. These issues span 43 Common Weakness Enumeration (CWE) categories, including significant ones like CWE-330: Use of Insufficiently Random Values , CWE-94: Improper Control of Generation of Code , and CWE-79: Cross-site Scripting . Notably, eight of those CWEs are among the 2023 CWE Top-25, highlighting their severity. We further examined using Copilot Chat to fix security issues in Copilot-generated code by providing Copilot Chat with warning messages from the static analysis tools, and up to 55.5% of the security issues can be fixed. We finally provide the suggestions for mitigating security issues in generated code.

Fairness is one of the socio-technical concerns that must be addressed in software systems. Considering the popularity of mobile software applications (apps) among a wide range of individuals worldwide, mobile apps with unfair behaviors and outcomes can affect a significant proportion of the global population, potentially more than any other type of software system. Users express a wide range of socio-technical concerns in mobile app reviews. This research aims to investigate fairness concerns raised in mobile app reviews. Our research focuses on AI-based mobile app reviews as the chance of unfair behaviors and outcomes in AI-based mobile apps may be higher than in non-AI-based apps. To this end, we first manually constructed a ground-truth dataset, including 1,132 fairness and 1,473 non-fairness reviews. Leveraging the ground-truth dataset, we developed and evaluated a set of machine learning and deep learning models that distinguish fairness reviews from non-fairness reviews. Our experiments show that our best-performing model can detect fairness reviews with a precision of 94%. We then applied the best-performing model on approximately 9.5M reviews collected from 108 AI-based apps and identified around 92K fairness reviews. Next, applying the K-means clustering technique to the 92K fairness reviews, followed by manual analysis, led to the identification of six distinct types of fairness concerns (e.g., “receiving different quality of features and services in different platforms and devices” and “lack of transparency and fairness in dealing with user-generated content” ). Finally, the manual analysis of 2,248 app owners’ responses to the fairness reviews identified six root causes (e.g., “copyright issues”) that app owners report to justify fairness concerns.

Developers reuse programming-related knowledge (e.g., code snippets) on Q&A sites (e.g., Stack Overflow) that functionally matches the programming problems they encounter in their development. Despite extensive research on Q&A sites, being a high-level and important type of development-related knowledge, architectural solutions (e.g., architecture tactics) and their reuse are rarely explored. To fill this gap, we conducted a mixed-methods study that includes a mining study and a survey study. For the mining study, we mined 984 commits and issues (i.e., 821 commits and 163 issues) from 893 Open-Source Software (OSS) projects on GitHub that explicitly referenced architectural solutions from Stack Overflow (SO) and Software Engineering Stack Exchange (SWESE). For the survey study, we identified practitioners involved in the reuse of these architectural solutions and surveyed 227 of them to further understand how practitioners reuse architectural solutions from Q&A sites in their OSS development. Our main findings are that: (1) OSS practitioners reuse architectural solutions from Q&A sites to solve a large variety (15 categories) of architectural problems, whereinComponent design issue,Architectural anti-pattern, andSecurity issueare dominant; (2) Seven categories of architectural solutions from Q&A sites have been reused to solve those problems, among whichArchitectural refactoring,Use of frameworks, andArchitectural tacticare the three most reused architectural solutions; (3) OSS developers often rely on ad hoc ways (e.g., informal, improvised, or unstructured approaches) to reuse architectural solutions from SO, drawing on personal experience and intuition rather than standardized or systematic practices; (4) Reusing architectural solutions from SO comes with a variety of challenges, e.g., OSS practitioners complain that they need to spend significant time to adapt such architectural solutions to address design concerns raised in their OSS development, and it is challenging to reuse architectural solutions that are not tailored to the design context of their OSS projects. Our findings pave the way for future research directions, including the design and development of approaches and tools (such as IDE plugin tools) to facilitate the reuse of architectural solutions from Q&A sites, and could also be used to offer guidelines to practitioners when they contribute architectural solutions to Q&A sites. Our dataset is publicly available athttps://doi.org/10.5281/zenodo.10936098.

Since the emergence of GPT-3, Large Language Models (LLMs) have caught the eyes of researchers, practitioners, and educators in the field of software engineering. However, there has been relatively little investigation regarding the performance of LLMs in assisting with requirements analysis and UML modeling. This paper explores how LLMs can assist novice analysts in creating three types of typical UML models: use case models, class diagrams, and sequence diagrams. For this purpose, we designed the modeling tasks of these three UML models for 45 undergraduate students who participated in a requirements modeling course, with the help of LLMs. By analyzing their project reports, we found that LLMs can assist undergraduate students as novice analysts in UML modeling tasks, but LLMs also have shortcomings and limitations that should be considered when using them.

WebAssembly (Wasm) is a binary instruction format designed for secure and efficient execution within sandboxed environments - predominantly web apps and browsers - to facilitate performance, security, and flexibility of web programming languages. In recent years, Wasm has gained significant attention from the academic research community and industrial development projects to engineer high-performance web applications. Despite the offered benefits, developers encounter a multitude of issues rooted in Wasm (e.g., faults, errors, failures) and are often unaware of their root causes that impact the development of web applications. To this end, we conducted an empirical study that mines and documents practitioners’ knowledge expressed as 385 issues from 12 open-source Wasm projects deployed on GitHub and 354 question-answer posts via Stack Overflow. Overall, we identified 120 types of issues, which were categorized into 19 subcategories and 9 categories to create a taxonomical classification of issues encountered in Wasm-based applications. Furthermore, root cause analysis of the issues helped us identify 278 types of causes, which have been categorized into 29 subcategories and 10 categories as a taxonomy of causes. Our study led to first-of-its-kind taxonomies of the issues faced by developers and their underlying causes in Wasm-based applications. The issue-cause taxonomies - identified from GitHub and SO, offering empirically derived guidelines - can guide researchers and practitioners to design, develop, and refactor Wasm-based applications.

Artificial Intelligence has demonstrated its significance in software engineering through notable improvements in productivity, accuracy, collaboration, and learning outcomes.This study examines the impact of generative AI tools, specifically ChatGPT, on the software development experiences of undergraduate students. Over a three-month project with seven students, ChatGPT was used as a support tool. The research focused on assessing ChatGPT’s effectiveness, benefits, limitations, and its influence on learning. Results showed that ChatGPT significantly addresses skill gaps in software development education, enhancing efficiency, accuracy, and collaboration. It also improved participants’ fundamental understanding and soft skills. The study highlights the importance of incorporating AI tools like ChatGPT in education to bridge skill gaps and increase productivity, but stresses the need for a balanced approach to technology use. Future research should focus on optimizing ChatGPT’s appli cation in various development contexts to maximize learning and address specific challenges.

Background: Since Google introduced Kotlin as an official programming language for developing Android apps in 2017, Kotlin has gained widespread adoption in Android development. The interoperability of Java and Kotlin’s design nature allows them to coexist and interact with each other smoothly within a project. Aims: However, there is limited research on how Java and Kotlin interact with each other in real-world projects and what challenges are faced during these interactions. The answers to these questions are key to understanding these kinds of cross-language software systems. Methods: In this paper, we implemented a tool named DependExtractor, which can extract 11 kinds of Kotlin-Java dependencies, and conducted an empirical study of 23 Kotlin-Java real-world projects with 3,227 Java and 8,630 Kotlin source files. Results: Our findings revealed that Java and Kotlin frequently interact with each other in these cross-language projects, with access and call dependency types being the most dominant. Compared to files interacting with other files in the same language, Java/Kotlin source files, which participate in the cross-language interactions, undergo more commits. Additionally, among all Kotlin-Java problematic interactions, we identified seven common mistakes, along with their fixing strategies. Conclusions: The findings of this study can help developers understand and address the challenges in Kotlin-Java projects.

Since Google introduced Kotlin as an official programming language for developing Android apps in 2017, Kotlin has gained widespread adoption in Android development. However, compared to Java, there is limited support for Kotlin code dependency analysis, which is the foundation to software analysis. To bridge this gap, we develop Depends-Kotlin to extract entities and their dependencies in Kotlin source code. Not only does Depends-Kotlin support extracting entities' dependencies in Kotlin code, but it can also extract dependency relations between Kotlin and Java. Using three open-source Kotlin-Java mixing projects as our subjects, Depends-Kotlin demonstrates high accuracy and performance in resolving Kotlin-Kotlin and Kotlin-Java dependencies relations. The source code of Depends-Kotlin and the dataset used have been made available at https://github.com/XYZboom/depends-kotlin. We also provide a screen-cast presenting Depends-Kotlin at https://youtu.be/ZPq8SRhgXzM.

As one of the most popular dynamic languages, Python experiences a decrease in readability and maintainability when code smells are present. Recent advancements in Large Language Models have sparked growing interest in AI-enabled tools for both code generation and refactoring. GitHub Copilot is one such tool that has gained widespread usage. Copilot Chat, released in September 2023, functions as an interactive tool aimed at facilitating natural language-powered coding. However, limited attention has been given to understanding code smells in Copilot-generated Python code and Copilot Chat's ability to fix the code smells. To this end, we built a dataset comprising 102 code smells in Copilot-generated Python code. Our aim is to first explore the occurrence of code smells in Copilot-generated Python code and then evaluate the effectiveness of Copilot Chat in fixing these code smells employing different prompts. The results show that 8 out of 10 types of code smells can be detected in Copilot-generated Python code, among which Multiply-Nested Container is the most common one. For these code smells, Copilot Chat achieves a highest fixing rate of 87.1%, showing promise in fixing Python code smells generated by Copilot itself. In addition, the effectiveness of Copilot Chat in fixing these smells can be improved by providing more detailed prompts.

Code review is an essential component of software development, playing a vital role in ensuring a comprehensive check of code changes. However, the continuous influx of pull requests and the limited pool of available reviewer candidates pose a significant challenge to the review process, making the task of assigning suitable reviewers to each review request increasingly difficult. To tackle this issue, we present MIRRec, a novel code reviewer recommendation method that leverages a hypergraph with multiplex relationships. MIRRec encodes high-order correlations that go beyond traditional pairwise connections using degree-free hyperedges among pull requests and developers. This way, it can capture high-order implicit connectivity and identify potential reviewers. To validate the effectiveness of MIRRec, we conducted experiments using a dataset comprising 48,374 pull requests from ten popular open-source software projects hosted on GitHub. The experiment results demonstrate that MIRRec, especially without PR-Review Commenters relationship, outperforms existing state-of-the-art code reviewer recommendation methods in terms of ACC and MRR, highlighting its significance in improving the code review process.

A commit message is a textual description of the code changes in a commit, which is a key part of the Git version control system (VCS). It captures the essence of software updating. Therefore, it can help developers understand code evolution and facilitate efficient collaboration between developers. However, it is time-consuming and labor-intensive to write good and valuable commit messages. Some researchers have conducted extensive studies on the automatic generation of commit messages and proposed several methods for this purpose, such as generation-based and retrieval-based models. However, seldom studies explored whether large language models (LLMs) can be used to generate commit messages automatically and effectively. To this end, this paper designed and conducted a series of experiments to comprehensively evaluate the performance of popular open-source and closed-source LLMs, i.e., Llama 2 and ChatGPT, in commit message generation. The results indicate that considering the BLEU and Rouge-L metrics, LLMs surpass the existing methods in certain indicators but lag behind in others. After human evaluations, however, LLMs show a distinct advantage over all these existing methods. Especially, in 78 % of the 366 samples, the commit messages generated by LLMs were evaluated by humans as the best. This work not only reveals the promising potential of using LLMs to generate commit messages, but also explores the limitations of commonly used metrics in evaluating the quality of auto-generated commit messages.

Architecting software-intensive systems can be a complex process. It deals with the daunting tasks of unifying stakeholders’ perspectives, designers’ intellect, tool-based automation, pattern-driven reuse, and so on, to sketch a blueprint that guides software implementation and evaluation. Despite its benefits, architecture-centric software engineering (ACSE) suffers from a multitude of challenges. ACSE challenges could stem from a lack of standardized processes, socio-technical limitations, and scarcity of human expertise etc. that can impede the development of existing and emergent classes of software. Software Development Bots (DevBots) trained on large language models can help synergise architects’ knowledge with artificially intelligent decision support to enable rapid architecting in a human-bot collaborative ACSE. An emerging solution to enable this collaboration is ChatGPT, a disruptive technology not primarily introduced for software engineering, but is capable of articulating and refining architectural artifacts based on natural language processing. We detail a case study that involves collaboration between a novice software architect and ChatGPT to architect a service-based software. Future research focuses on harnessing empirical evidence about architects’ productivity and explores socio-technical aspects of architecting with ChatGPT to tackle challenges of ACSE.

Code review is a common practice in software development and often conducted before code changes are merged into the code repository. A number of approaches for automatically recommending appropriate reviewers have been proposed to match such code changes to pertinent reviewers. However, such approaches are generic, i.e., they do not focus on specific types of issues during code reviews. In this paper, we propose an approach that focuses on architecture violations, one of the most critical type of issues identified during code review. Specifically, we aim at automating the recommendation of code reviewers, who are potentially qualified to review architecture violations, based on reviews of code changes. To this end, we selected three common similarity detection methods to measure the file path similarity of code commits and the semantic similarity of review comments. We conducted a series of experiments on finding the appropriate reviewers through evaluating and comparing these similarity detection methods in separate and combined ways with the baseline reviewer recommendation approach, RevFinder. The results show that the common similarity detection methods can produce acceptable performance scores and achieve a better performance than RevFinder. The sampling techniques used in recommending code reviewers can impact the performance of reviewer recommendation approaches. We also discuss the potential implications of our findings for both researchers and practitioners.

Background: In modern software systems, more and more systems are written in multiple programming languages (PLs). There is no comprehensive investigation on the phenomenon of multi-programming-language (MPL) bugs, which resolution involves source files written in multiple PLs. Aim: This work investigated the characteristics of bug resolution in MPL software systems and explored the reasons why bug resolution involves multiple PLs. Method: We conducted an empirical study on 54 MPL projects selected from 655 Apache OSS projects, of which 66,932 bugs were analyzed. Results: (1) the percentage of MPL bugs (MPLBs) in the selected projects ranges from 0.17% to 42.26%, and the percentage of MPLBs for all projects as a whole is 10.01%; (2) 95.0% and 4.5% of all the MPLBs involve source files written in 2 and 3 PLs, respectively; (3) the change complexity resolution characteristics of MPLBs tend to be higher than those of single-programming-language bugs (SPLBs); (4) the open time for MPLBs is 19.52% to 529.57% significantly longer than SPLBs regarding 9 PL combinations; (5) the reopen rate of bugs involving the PL combination of JavaScript and Python reaches 20.66%; (6) we found 6 causes why the bug resolution involves multiple PLs and identified 5 cross-language calling mechanisms. Conclusion: MPLBs are related to increased development difficulty.

Background: Despite the widespread use of automated security defect detection tools, software projects still contain many security defects that could result in serious damage. Such tools are largely context-insensitive and may not cover all possible scenarios in testing potential issues, which makes them susceptible to missing complex security defects. Hence, thorough detection entails a synergistic cooperation between these tools and human-intensive detection techniques, including code review. Code review is widely recognized as a crucial and effective practice for identifying security defects. Aim: This work aims to empirically investigate security defect detection through code review. Method: To this end, we conducted an empirical study by analyzing code review comments derived from four projects in the OpenStack and Qt communities. Through manually checking 20,995 review comments obtained by keyword-based search, we identified 614 comments as security-related. Results: Our results show that (1) security defects are not prevalently discussed in code review, (2) more than half of the reviewers provided explicit fixing strategies/solutions to help developers fix security defects, (3) developers tend to follow reviewers' suggestions and action the changes, (4) Not worth fixing the defect now and Disagreement between the developer and the reviewer are the main causes for not resolving security defects. Conclusions: Our research results demonstrate that (1) software security practices should combine manual code review with automated detection tools, achieving a more comprehensive coverage to identifying and addressing security defects, and (2) promoting appropriate standardization of practitioners' behaviors during code review remains necessary for enhancing software security.

Deep learning frameworks (DLFs) have been playing an increasingly important role in this intelligence age since they act as a basic infrastructure for an increasingly wide range of AI-based applications. Meanwhile, as multi-programming-language (MPL) software systems, DLFs are inevitably suffering from bugs caused by the use of multiple programming languages (PLs). Hence, it is of paramount significance to understand the bugs (especially the bugs involving multiple PLs, i.e., MPL bugs) of DLFs, which can provide a foundation for preventing, detecting, and resolving bugs in the development of DLFs. To this end, we manually analyzed 1497 bugs in three MPL DLFs, namely MXNet, PyTorch, and TensorFlow. First, we classified bugs in these DLFs into 12 types (e.g., algorithm design bugs and memory bugs) according to their bug labels and characteristics. Second, we further explored the impacts of different bug types on the development of DLFs, and found that deployment bugs and memory bugs negatively impact the development of DLFs in different aspects the most. Third, we found that 28.6%, 31.4%, and 16.0% of bugs in MXNet, PyTorch, and TensorFlow are MPL bugs, respectively; the PL combination of Python and C/C++ is most used in fixing more than 92% MPL bugs in all DLFs. Finally, the code change complexity of MPL bug fixes is significantly greater than that of single-programming-language (SPL) bug fixes in all the three DLFs, while in PyTorch MPL bug fixes have longer open time and greater communication complexity than SPL bug fixes. These results provide insights for bug management in DLFs.

With the advances in machine learning, there is a growing interest in AI-enabled tools for autocompleting source code.GitHub Copilot, also referred to as the "AI Pair Programmer", has been trained on billions of lines of open source GitHub code, and is one of such tools that has been increasingly used since its launch in June 2021.However, little effort has been devoted to understanding the practices and challenges of using Copilot in programming with auto-completed source code.To this end, we conducted an empirical study by collecting and analyzing the data from Stack Overflow (SO) and GitHub Discussions.More specifically, we searched and manually collected 169 SO posts and 655 GitHub discussions related to the usage of Copilot.We identified the programming languages, IDEs, technologies used with Copilot, functions implemented, benefits, limitations, and challenges when using Copilot.The results show that when practitioners use Copilot: (1) The major programming languages used with Copilot are JavaScript and Python, (2) the main IDE used with Copilot is Visual Studio Code, (3) the most common used technology with Copilot is Node.js,(4) the leading function implemented by Copilot is data processing, (5) the significant benefit of using Copilot is useful code generation, and (6) the main limitation encountered by practitioners when using Copilot is difficulty of integration.Our results suggest that using Copilot is like a double-edged sword, which requires developers to carefully consider various aspects when deciding whether or not to use it.Our study provides empirically grounded foundations and basis for future research on the role of Copilot as an AI pair programmer in software development.

Artificial Intelligence (AI) technologies have been developed rapidly, and AI-based systems have been widely used in various application domains with opportunities and challenges. However, little is known about the architecture decisions made in AI-based systems development, which has a substantial impact on the success and sustainability of these systems. To this end, we conducted an empirical study by collecting and analyzing the data from Stack Overflow (SO) and GitHub. More specifically, we searched on SO with six sets of keywords and explored 32 AI-based projects on GitHub, and finally we collected 174 posts and 128 GitHub issues related to architecture decisions. The results show that in AI-based systems development (1) architecture decisions are expressed in six linguistic patterns, among which Solution Proposal and Information Giving are most frequently used, (2) Technology Decision, Component Decision, and Data Decision are the main types of architecture decisions made, (3) Game is the most common application domain among the eighteen application domains identified, (4) the dominant quality attribute considered in architecture decision-making is Performance, and (5) the main limitations and challenges encountered by practitioners in making architecture decisions are Design Issues and Data Issues. Our results suggest that the limitations and challenges when making architecture decisions in AI-based systems development are highly specific to the characteristics of AI-based systems and are mainly of technical nature, which need to be properly confronted.

With the advances in machine learning, there is a growing interest in AI-enabled tools for autocompleting source code. GitHub Copilot, also referred to as the “AI Pair Programmer”, has been trained on billions of lines of open source GitHub code, and is one of such tools that has been increasingly used since its launch in June 2021. However, little effort has been devoted to understanding the practices, challenges, and expected features of using Copilot in programming for auto-completed source code from the point of view of practitioners. To this end, we conducted an empirical study by collecting and analyzing the data from Stack Overflow (SO) and GitHub Discussions. More specifically, we searched and manually collected 303 SO posts and 927 GitHub discussions related to the usage of Copilot. We identified the programming languages, Integrated Development Environments (IDEs), technologies used with Copilot, functions implemented, benefits, limitations, and challenges when using Copilot. The results show that when practitioners use Copilot: (1) The major programming languages used with Copilot are JavaScript and Python, (2) the main IDE used with Copilot is Visual Studio Code, (3) the most common used technology with Copilot is Node.js, (4) the leading function implemented by Copilot is data processing, (5) the main purpose of users using Copilot is to help generate code, (6) the significant benefit of using Copilot is useful code generation, (7) the main limitation encountered by practitioners when using Copilot is difficulty of integration, and (8) the most common expected feature is that Copilot can be integrated with more IDEs. Our results suggest that using Copilot is like a double-edged sword, which requires developers to carefully consider various aspects when deciding whether or not to use it. Our study provides empirically grounded foundations that could inform software developers and practitioners, as well as provide a basis for future investigations on the role of Copilot as an AI pair programmer in software development.

Context: As a software system evolves, its architecture tends to degrade, and gradually impedes software maintenance and evolution activities and negatively impacts the quality attributes of the system. The main root cause behind architecture erosion phenomenon derives from violation symptoms (i.e., various architecturally-relevant violations, such as violations of architecture pattern). Previous studies focus on detecting violations in software systems using architecture conformance checking approaches. However, code review comments are also rich sources that may contain extensive discussions regarding architecture violations, while there is a limited understanding of violation symptoms from the viewpoint of developers. Objective: In this work, we investigated the characteristics of architecture violation symptoms in code review comments from the developers’ perspective. Methods: We employed a set of keywords Related to violation symptoms to collect 606 (out of 21,583) code review comments from four popular OSS projects in the openStack and qt communities. We manually analyzed the collected 606 review comments to provide the categories and linguistic patterns of violation symptoms, as well as the reactions how developers addressed them. Results: Our findings show that: (1) three main categories of violation symptoms are discussed by developers during the code review process ; (2) The frequently-used terms of expressing violation symptoms are “ inconsistent ” and “ violate ”, and the most common linguistic pattern is Problem Discovery ; (3) Refactoring and removing code are the major measures (90%) to tackle violation symptoms, while a few violation symptoms were ignored by developers. Conclusions: Our findings suggest that the investigation of violation symptoms can help researchers better understand the characteristics of architecture erosion and facilitate the development and maintenance activities, and developers should explicitly manage violation symptoms, not only for addressing the existing architecture violations but also preventing future violations.

Quantum computing systems rely on the principles of quantum mechanics to perform a multitude of computationally challenging tasks more efficiently than their classical counterparts. The architecture of software-intensive systems can empower architects who can leverage architecture-centric processes, practices, description languages to model, develop, and evolve quantum computing software (quantum software for short) at higher abstraction levels. We conducted a Systematic Literature Review (SLR) to investigate (i) architectural process, (ii) modelling notations, (iii) architecture design patterns, (iv) tool support, and (iv) challenging factors for quantum software architecture. Results of the SLR indicate that quantum software represents a new genre of software-intensive systems; however, existing processes and notations can be tailored to derive the architecting activities and develop modelling languages for quantum software. Quantum bits (Qubits) mapped to Quantum gates (Qugates) can be represented as architectural components and connectors that implement quantum software. Tool-chains can incorporate reusable knowledge and human roles (e.g., quantum domain engineers, quantum code developers) to automate and customise the architectural process. Results of this SLR can facilitate researchers and practitioners to develop new hypotheses to be tested, derive reference architectures, and leverage architecture-centric principles and practices to engineer emerging and next generations of quantum software.

Artificial intelligence (AI) solutions and technologies are being increasingly adopted in smart systems contexts; however, such technologies are concerned with ethical uncertainties. Various guidelines, principles, and regulatory frameworks are designed to ensure that AI technologies adhere to ethical well-being. However, the implications of AI ethics principles and guidelines are still being debated. To further explore the significance of AI ethics principles and relevant challenges, we conducted a survey of 99 randomly selected representative AI practitioners and lawmakers (e.g., AI engineers and lawyers) from 20 countries across five continents. To the best of our knowledge, this is the first empirical study that unveils the perceptions of two different types of population (AI practitioners and lawmakers) and the study findings confirm that transparency, accountability, and privacy are the most critical AI ethics principles. On the other hand, lack of ethical knowledge, no legal frameworks, and lacking monitoring bodies are found to be the most common AI ethics challenges. The impact analysis of the challenges across principles reveals that conflict in practice is a highly severe challenge. Moreover, the perceptions of practitioners and lawmakers are statistically correlated with significant differences for particular principles (e.g. fairness and freedom) and challenges (e.g. lacking monitoring bodies and machine distortion). Our findings stimulate further research, particularly empowering existing capability maturity models to support ethics-aware AI systems’ development and quality assessment.

Ethics in AI becomes a global topic of interest for both policymakers and academic researchers. In the last few years, various research organizations, lawyers, think tankers, and regulatory bodies get involved in developing AI ethics guidelines and principles. However, there is still debate about the implications of these principles. We conducted a systematic literature review (SLR) study to investigate the agreement on the significance of AI principles and identify the challenging factors that could negatively impact the adoption of AI ethics principles. The results reveal that the global convergence set consists of 22 ethical principles and 15 challenges. Transparency, privacy, accountability and fairness are identified as the most common AI ethics principles. Similarly, lack of ethical knowledge and vague principles are reported as the significant challenges for considering ethics in AI. The findings of this study are the preliminary inputs for proposing a maturity model that assesses the ethical capabilities of AI systems and provides best practices for further improvements.

Background: Technical Debt (TD) refers to the situation where developers make trade-offs to achieve short-term goals at the expense of long-term code quality, which can have a negative impact on the quality of software systems. In the context of code review, such sub-optimal implementations have chances to be timely resolved during the review process before the code is merged. Therefore, we could consider them as Potential Technical Debt (PTD) since PTD will evolve into TD when it is injected into software systems without being resolved. Aim: To date, little is known about the extent to which PTD is identified in code reviews. Many tools have been provided to detect TD, but these tools lack consensus and a large amount of PTD are undetectable by tools while code review could help verify the quality of code that has been committed by identifying issues, such as PTD. To this end, we conducted an exploratory study in an attempt to understand the nature of PTD in code reviews and track down the resolution of PTD after being identified. Method: We randomly collected 2,030 review comments from the Nova project of OpenStack and the Qt Base project of Qt. We then manually checked these review comments, and obtained 163 PTD-related review comments for further analysis. Results: Our results show that: (1) PTD can be identified in code reviews but is not prevalent. (2) Design, defect, documentation, requirement, test, and code PTD are identified in code reviews, in which code and documentation PTD are the dominant. (3) 81.0% of the PTD identified in code reviews has been resolved by developers, and 78.0% of the resolved TD was resolved by developers within a week. (4) Code refactoring is the main practice used by developers to resolve the PTD identified in code reviews. Conclusions: Our findings indicate that: (1) review-based detection of PTD is seen as one of the trustworthy mechanisms in development, and (2) there is still a significant proportion of PTD (19.0%) remaining unresolved when injected into the software systems. Practitioners and researchers should establish effective strategies to manage and resolve PTD in development.

Building software systems often requires knowledge and skills beyond what developers already possess. In such cases, developers have to leverage different sources of information to seek help. A growing number of researchers and practitioners have started investigating what programming-related information developers seek during software development. However, being a high level and a type of the most important development-related information, architectural information search activity is seldom explored. To fill this gap, we conducted an industrial survey completed by 103 participants to understand how developers search for architectural information to solve their architectural problems in development. Our main findings are: (1) searching for architectural information to learn about the pros and cons of certain architectural solutions (e.g., patterns, tactics) and to make an architecture decision among multiple choices are the most frequent purposes or tasks; (2) developers find difficulties mostly in getting relevant architectural information for addressing quality concerns and making design decisions among multiple choices when seeking architectural information; (3) taking too much time to go through architectural information retrieved from various sources and feeling overwhelmed due to the dispersion and abundance of architectural information in various sources are the top two major challenges developers face when searching for architectural information. Our findings (1) provide researchers with future directions, such as the design and development of approaches and tools for searching architectural information from multiple sources, and (2) can be used to provide guidelines for practitioners to refer to when seeking architectural information and providing architectural information that could be considered useful.

The phenomenon of architecture erosion can negatively impact the maintenance and evolution of software systems, and manifest in a variety of symptoms during software development. While erosion is often considered rather late, its symptoms can act as early warnings to software developers, if detected in time. In addition to static source code analysis, code reviews can be a source of detecting erosion symptoms and subsequently taking action. In this study, we investigate the erosion symptoms discussed in code reviews, as well as their trends, and the actions taken by developers. Specifically, we conducted an empirical study with the two most active Open Source Software (OSS) projects in the OpenStack community (i.e., Nova and Neutron). We manually checked 21,274 code review comments retrieved by keyword search and random selection, and identified 502 code review comments (from 472 discussion threads) that discuss erosion. Our findings show that (1) the proportion of erosion symptoms is rather low, yet notable in code reviews and the most frequently identified erosion symptoms are architectural violation, duplicate functionality, and cyclic dependency; (2) the declining trend of the identified erosion symptoms in the two OSS projects indicates that the architecture tends to stabilize over time; and (3) most code reviews that identify erosion symptoms have a positive impact on removing erosion symptoms, but a few symptoms still remain and are ignored by developers. The results suggest that (1) code review provides a practical way to reduce erosion symptoms; and (2) analyzing the trend of erosion symptoms can help get an insight about the erosion status of software systems, and subsequently avoid the potential risk of architecture erosion.

Code review is a mature practice for software quality assurance in software development with which reviewers check the code that has been committed by developers, and verify the quality of code. During the code review discussions, reviewers and developers might use code snippets to provide necessary information (e.g., suggestions or explanations). However, little is known about the intentions and impacts of code snippets in code reviews. To this end, we conducted a preliminary study to investigate the nature of code snippets and their purposes in code reviews. We manually collected and checked 10,790 review comments from the Nova and Neutron projects of the OpenStack community, and finally obtained 626 review comments that contain code snippets for further analysis. The results show that: (1) code snippets are not prevalently used in code reviews, and most of the code snippets are provided by reviewers. (2) We identified two high-level purposes of code snippets provided by reviewers (i.e., Suggestion and Citation) with six detailed purposes, among which, Improving Code Implementation is the most common purpose. (3) For the code snippets in code reviews with the aim of suggestion, around 68.1% was accepted by developers. The results highlight promising research directions on using code snippets in code reviews.

Bugs are usually in associations with other bugs in a software system, e.g., a bug may result from another bug.However, such bug associations are implicit and usually cannot be traced without a significant amount of effort.Intuitively, if a bug association is easier to trace, the involved bugs can be fixed in a cleaner way.However, there is little evidence on the explicitness of bug associations.In this paper, we aim to evaluate the explicitness of bug associations, so as to get a basic understanding on such associations.To this end, we defined a metric to quantify the explicitness of a bug association, and conducted an empirical study on 11 non-trivial Apache open source software systems.The main findings are summarized as follows: (1) From the perspective of code change history, around 29% of bug pairs are not explicitly associated, and about 71% are explicitly associated to some extent; (2) Bugs in the association of Container have relatively strong association explicitness, while bugs in the association of Blocked or Blocker, Cloners, and Dependent have relatively weak association explicitness.These findings provide insights on software analyzability to practitioners and researchers.

In recent years, Continuous Integration (CI) and Continuous Delivery (CD) has been heatedly discussed and widely used in part or all of the software development life cycle as the practices and pipeline to deliver software products in an efficient way.There are many tools, such as Travis CI, that offer various features to support the CI/CD pipeline, but there is a lack of understanding about what decisions are frequently made in CI/CD.In this work, we explored one popular open-source project on GitHub, Budibase, to provide insights on the types of decisions made in CI/CD from a practitioners' perspective.We first explored the GitHub Trending page, conducted a pilot repository extraction, and identified the Budibase repository as the case for our study.We then crawled all the closed issues from the repository and got 1,168 closed issues.Irrelevant issues were filtered out based on certain criteria, and 370 candidate issues that contain decisions were obtained for data extraction.We analyzed the issues using a hybrid approach combining predefined types and the Constant Comparison method to get the categories of decisions.The results show that the major type of decisions in the Budibase closed issues is Functional Requirement Decision (67.6%), followed by Architecture Decision (11.1%).Our findings encourage developers to put more effort on the issues and making decisions related to CI/CD, and provide researchers with a reference of decision classification made in CI/CD.

Abstract Public development projects (PDPs) and documented public development projects (DPDPs) are two types of projects that can provide valuable information on how developers and users participate in OSS projects. However, it is hard for researchers to effectively select PDPs and DPDPs due to the lack of specific project selection methods for these two types of projects. To address this problem, a standard dataset was labelled and the base line methods (i.e. selecting projects according to a single feature like star number) under 60 configurations and the machine learning methods under 18 configurations were tested to identify the best configurations in precision and F‐measure for selecting PDPs and DPDPs. The results show that (1) to select PDPs or DPDPs with a high precision, the base line method is the best with precision of 0.877 (PDPs) and 0.831 (DPDPs); (2) to select PDPs or DPDPs with a high F‐measure, the machine learning methods are the best, with F‐measure of 0.817 (PDPs) and 0.789 (DPDPs); (3) existing sample selection strategies can be combined with the machine learning methods, and the precision of selecting PDPs can be increased by 6.39%–41.33% and the precision of selecting DPDPs can be can be increased by 35.50%–269.02%.

Abstract Architecture erosion (AEr) can adversely affect software development and has received significant attention in the last decade. However, there is an absence of a comprehensive understanding of the state of research about the reasons and consequences of AEr, and the countermeasures to address AEr. This work aims at systematically investigating, identifying, and analyzing the reasons, consequences, and ways of detecting and handling AEr. With 73 studies included, the main results are as follows: (1) AEr manifests not only through architectural violations and structural issues but also causing problems in software quality and during software evolution; (2) non‐technical reasons that cause AEr should receive the same attention as technical reasons, and practitioners should raise awareness of the grave consequences of AEr, thereby taking actions to tackle AEr‐related issues; (3) a spectrum of approaches, tools, and measures has been proposed and employed to detect and tackle AEr; and (4) three categories of difficulties and five categories of lessons learned on tackling AEr were identified. The results can provide researchers a comprehensive understanding of AEr and help practitioners handle AEr and improve the sustainability of their architecture. More empirical studies are required to investigate the practices of detecting and addressing AEr in industrial settings.

Release planning for mobile apps has recently become an area of active research. Prior research in this area concentrated on the analysis of release notes and on tracking user reviews to support app evolution with issue trackers. However, little is known about the impact of user reviews on the evolution of mobile apps. Our work explores the role of user reviews in app updates based on release notes. For this purpose, we collected user reviews and release notes of Spotify, the ߢnumber one’ app in the ‘Music’ category in Apple App Store, as the research data. Then, we manually removed non-informative parts of each release note, and manually determined the relevance of the app reviews with respect to the release notes. We did this by using Word2Vec calculation techniques based on the top 80 app release notes with the highest similarities. Our empirical results show that more than 60 % of the matched reviews are actually irrelevant to the corresponding release notes. When zooming in at these relevant user reviews, we found that around half of them were posted before the new release and referred to requests, suggestions, and complaints. Whereas, the other half of the relevant user reviews were posted after updating the apps and concentrated more on bug reports and praise.

Deep learning (DL) frameworks have been extensively designed, implemented, and used in software projects across many domains. However, due to the lack of knowledge or information, time pressure, complex context, etc., various uncertainties emerge during the development, leading to assumptions made in DL frameworks. Though not all the assumptions are negative to the frameworks, being unaware of certain assumptions can result in critical problems (e.g., system vulnerability and failures). As the first step of addressing the critical problems, there is a need to explore and understand the assumptions made in DL frameworks. To this end, we conducted an exploratory study to understand self-claimed assumptions (SCAs) about their distribution, classification, and impacts using code comments from nine popular DL framework projects on GitHub. The results are that: (1) 3,084 SCAs are scattered across 1,775 files in the nine DL frameworks, ranging from 1,460 (TensorFlow) to 8 (Keras) SCAs. (2) There are four types of validity of SCAs: Valid SCA, Invalid SCA, Conditional SCA, and Unknown SCA, and four types of SCAs based on their content: Configuration and Context SCA, Design SCA, Tensor and Variable SCA, and Miscellaneous SCA. (3) Both valid and invalid SCAs may have an impact within a specific scope (e.g., in a function) on the DL frameworks. Certain technical debt is induced when making SCAs. There are source code written and decisions made based on SCAs. This is the first study on investigating SCAs in DL frameworks, which helps researchers and practitioners to get a comprehensive understanding on the assumptions made. We also provide the first dataset of SCAs for further research and practice in this area.

Stakeholders make various types of decisions with respect to requirements, design, management, and so on during the software development life cycle. Nevertheless, these decisions are typically not well documented and classified due to limited human resources, time, and budget. To this end, automatic approaches provide a promising way. In this paper, we aimed at automatically classifying decisions into five types to help stakeholders better document and understand decisions. First, we collected a dataset from the Hibernate developer mailing list. We then experimented and evaluated 270 configurations regarding feature selection, feature extraction techniques, and machine learning classifiers to seek the best configuration for classifying decisions. Especially, we applied an ensemble learning method and constructed ensemble classifiers to compare the performance between ensemble classifiers and base classifiers. Our experiment results show that (1) feature selection can decently improve the classification results; (2) ensemble classifiers can outperform base classifiers provided that ensemble classifiers are well constructed; (3) BoW + 50% features selected by feature selection with an ensemble classifier that combines Naïve Bayes (NB), Logistic Regression (LR), and Support Vector Machine (SVM) achieves the best classification result (with a weighted precision of 0.750, a weighted recall of 0.739, and a weighted F1-score of 0.727) among all the configurations. Our work can benefit various types of stakeholders in software development through providing an automatic approach for effectively classifying decisions into specific types that are relevant to their interests.

Due to its enormous benefits, the research and industry communities have shown an increasing interest in the Microservices Architecture (MSA) style over the last few years. Despite this, there is a limited evidence-based and thorough understanding of the types of issues (e.g., faults, errors, failures, mistakes) faced by microservices system developers and causes that trigger the issues. Such evidence-based understanding of issues and causes is vital for long-term, impactful, and quality research and practice in the MSA style. To that end, we conducted an empirical study on 1,345 issue discussions extracted from five open source microservices systems hosted on GitHub. Our analysis led to the first of its kind taxonomy of the types of issues in open source microservices systems, informing that the problems originating from Technical debt (321, 23.86%), Build (145, 10.78%), Security (137, 10.18%), and Service execution and communication (119, 8.84%) are prominent. We identified that “General programming errors”, “Poor security management”, “Invalid configuration and communication”, and “Legacy versions, compatibility and dependency” are the predominant causes for the leading four issue categories. Study results streamline a taxonomy of issues, their mapping with underlying causes, and present empirical findings that could facilitate research and development on emerging and next-generation microservices systems.

Software development processes play a key role in the software and system development life cycle. Processes are becoming complex and evolve rapidly due to the modern-day continuous software engineering (CSE) concepts, which are mainly based on continuous integration, continuous delivery, infrastructure-as-code, automation and more. The fast growing Chinese software development industry adopts various processes to achieve potential benefits offered in the international market. This study is conducted with the aim to investigate the trends of processes in practice in the Chinese industry. The survey questionnaire data is collected from 34 practitioners working in software development firms across the China and the results highlight that iterative and agile processes are extensively used in industrial setting. Furthermore, agile and traditional approaches are combined to develop the hybrid processes. Most of the participants are satisfied using the current development processes, however, they show interest to continuously improve the existing process models and methods. Finally, we noticed that majority of the software development organizations used the ISO 9001 standard for process assessment and improvement activities. The given results provide preliminary overview of processes deployed in the Chinese industry.

Background: In recent years, Low-code development (LCD) is growing rapidly, and Gartner and Forrester have predicted that the use of LCD is very promising. Giant companies, such as Microsoft, Mendix, and Outsystems have also launched their LCD platforms. Aim: In this work, we explored two popular online developer communities, Stack Overflow (SO) and Reddit, to provide insights on the characteristics and challenges of LCD from a practitioners' perspective. Method: We used two LCD related terms to search the relevant posts in SO and extracted 73 posts. Meanwhile, we explored three LCD related subreddits from Reddit and collected 228 posts. We extracted data from these posts and applied the Constant Comparison method to analyze the descriptions, benefits, and limitations and challenges of LCD. For platforms and programming languages used in LCD, implementation units in LCD, supporting technologies of LCD, types of applications developed by LCD, and domains that use LCD, we used descriptive statistics to analyze and present the results. Results: Our findings show that: (1) LCD may provide a graphical user interface for users to drag and drop with little or even no code; (2) the equipment of out-of-the-box units (e.g., APIs and components) in LCD platforms makes them easy to learn and use as well as speeds up the development; (3) LCD is particularly favored in the domains that have the need for automated processes and workflows; and (4) practitioners have conflicting views on the advantages and disadvantages of LCD. Conclusions: Our findings suggest that researchers should clearly define the terms when they refer to LCD, and developers should consider whether the characteristics of LCD are appropriate for their projects.

Code review plays an important role in software quality control. A typical review process would involve a careful check of a piece of code in an attempt to find defects and other quality issues/violations. One type of issues that may impact the quality of the software is code smells - i.e., bad programming practices that may lead to defects or maintenance issues. Yet, little is known about the extent to which code smells are identified during code reviews. To investigate the concept behind code smells identified in code reviews and what actions reviewers suggest and developers take in response to the identified smells, we conducted an empirical study of code smells in code reviews using the two most active OpenStack projects (Nova and Neutron). We manually checked 19,146 review comments obtained by keywords search and random selection, and got 1,190 smell-related reviews to study the causes of code smells and actions taken against the identified smells. Our analysis found that 1) code smells were not commonly identified in code reviews, 2) smells were usually caused by violation of coding conventions, 3) reviewers usually provided constructive feedback, including fixing (refactoring) recommendations to help developers remove smells, and 4) developers generally followed those recommendations and actioned the changes. Our results suggest that 1) developers should closely follow coding conventions in their projects to avoid introducing code smells, and 2) review-based detection of code smells is perceived to be a trustworthy approach by developers, mainly because reviews are context-sensitive (as reviewers are more aware of the context of the code given that they are part of the project's development team).

As software systems evolve, their architecture is meant to adapt accordingly by following the changes in requirements, the environment, and the implementation. However, in practice, the evolving system often deviates from the architecture, causing severe consequences to system maintenance and evolution. This phenomenon of architecture erosion has been studied extensively in research, but not yet been examined from the point of view of developers. In this exploratory study, we look into how developers perceive the notion of architecture erosion, its causes and consequences, as well as tools and practices to identify and control architecture erosion. To this end, we searched through several popular online developer communities for collecting data of discussions related to architecture erosion. Besides, we identified developers involved in these discussions and conducted a survey with 10 participants and held interviews with 4 participants. Our findings show that: (1) developers either focus on the structural manifestation of architecture erosion or on its effect on run-time qualities, maintenance and evolution; (2) alongside technical factors, architecture erosion is caused to a large extent by non-technical factors; (3) despite the lack of dedicated tools for detecting architecture erosion, developers usually identify erosion through a number of symptoms; and (4) there are effective measures that can help to alleviate the impact of architecture erosion.

Modern software systems, such as Spark, are usually written in multiple programming languages (PLs). Besides benefiting from code reuse, such systems can also take advantages of specific PLs to implement certain features, to meet various quality needs, and to improve development efficiency. In this context, a change to such systems may need to modify source files written in different PLs. We define a multi-programming-language commit (MPLC) in a version control system (e.g., Git) as a commit that involves modified source files written in two or more PLs. To our knowledge, the phenomenon of MPLCs in software development has not been explored yet. In light of the potential impact of MPLCs on development difficulty and software quality, we performed an empirical study to understand the state of MPLCs, their change complexity, as well as their impact on open time of issues and bug proneness of source files in real-life software projects. By exploring the MPLCs in 20 non-trivial Apache projects with 205,994 commits, we obtained the following findings: (1) 9% of the commits from all the projects are MPLCs, and the proportion of MPLCs in 80% of the projects goes to a relatively stable level; (2) more than 90% of the MPLCs from all the projects involve source files written in two PLs; (3) the change complexity of MPLCs is significantly higher than that of non-MPLCs in all projects; (4) issues fixed in MPLCs take significantly longer to be resolved than issues fixed in non-MPLCs in 80% of the projects; and (5) source files that have been modified in MPLCs tend to be more bug-prone than source files that have never been modified in MPLCs. These findings provide practitioners with useful insights on the architecture design and quality management of software systems written in multiple PLs.

UML diagrams are a recognized standard modelling language for representing design of software systems.For academic research, large cases containing UML diagrams are needed.One of the challenges in collecting such datasets is automatically determining whether an image is a UML diagram or not and what type of UML diagram an image contains.In this study, we collected UML diagrams from open datasets and manually labeled them into four types of UML diagrams (i.e., class diagrams, activity diagrams, sequence diagrams, and use case diagrams) and non-UML images.We evaluated the performance of five popular neural network architectures using transfer learning on the dataset of 3231 images that contains 700 class diagrams, 454 activity diagrams, 651 use case diagrams, 706 sequence diagrams, and 720 non-UML images, respectively.We also proposed our neural network architecture for multiclass classification of UML diagrams.The experiment results show that our proposed neural network architecture achieved the best performance amongst the algorithms we evaluated with an accuracy of 98.65%, a precision of 96.76%, a recall of 96.48%, and an F1-score of 96.62%.Moreover, among the neural network architectures that we have evaluated, our proposed architecture has the least parameters (around 2.4 millions) and spends the least time per image (0.0135 seconds per image using GPU) for classifying UML diagrams.

Decisions are an important type of artifacts in software development and maintenance, while decisions are not well-documented in projects due to limited human resources and budget. To this end, many studies focus on using automatic approaches to identify decisions from textual artifacts, e.g., mailing lists, issue tracking systems. In this paper, we present a replication study of our previous work (EASE2020), which conducted experiments to automatically identify decisions from the Hibernate developer mailing list. In addition, we utilized different datasets in the experiment with the aim of exploring the impact of the proprieties of dataset (i.e., the quality of positive samples, different negative samples in the dataset, and the size of the dataset) on classification results of decisions. The results show that (1) improving the quality of positive samples in the dataset can decently improve the classification results; (2) different negative samples in the dataset have an impact on the classification results; and (3) before the dataset size reaches 1200, increasing the size will improve the classification results.

Unified Modeling Language (UML) diagrams are a recognized standard modeling language for representing design of software systems. For academic research, large cases containing UML diagrams are needed. One of the challenges in collecting such datasets is automatically determining whether an image is a UML diagram or not and what type of UML diagram an image contains. In this work, we collected UML diagrams from open datasets and manually labeled them into 10 types of UML diagrams (i.e. class diagrams, activity diagrams, use case diagrams, sequence diagrams, communication diagrams, component diagrams, deployment diagrams, object diagrams, package diagrams, and state machine diagrams) and non-UML images. We evaluated the performance of seven popular neural network architectures using transfer learning on the dataset of 4706 images, including 700 class diagrams, 454 activity diagrams, 651 use case diagrams, 706 sequence diagrams, 204 communication diagrams, 208 component diagrams, 287 deployment diagrams, 207 object diagrams, 246 package diagrams, 323 state machine diagrams, and 720 non-UML images, respectively. We also proposed our neural network architecture for multiclass classification of UML diagrams. The experiment results show that Xception achieved the best performance amongst the algorithms we evaluated with a precision of 93.03%, a recall of 92.44%, and an F1-score of 92.73%. Moreover, it is possible to develop small and almost the same efficient neural network architectures, that our proposed architecture has the least parameters (around 2.4 millions) and spends the least time per image (0.0135[Formula: see text]s per image using graphics processing unit) for classifying UML diagrams with a precision of 91.25%, a recall of 90.34%, and an F1-score of 90.79%.

Abstract Software traceability plays a critical role in software maintenance and evolution. We conducted a systematic mapping study with six research questions to understand the benefits, costs, and challenges of using traceability in maintenance and evolution. We systematically selected, analyzed, and synthesized 63 studies published between January 2000 and May 2020, and the results show that traceability supports 11 maintenance and evolution activities, among which change management is the most frequently supported activity; strong empirical evidence from industry is needed to validate the impact of traceability on maintenance and evolution; easing the process of change management is the main benefit of deploying traceability practices; establishing and maintaining traceability links is the main cost of deploying traceability practices; 13 approaches and 32 tools that support traceability in maintenance and evolution were identified; improving the quality of traceability links , the performance of using traceability approaches , and tools are the main traceability challenges in maintenance and evolution. The findings of this study provide a comprehensive understanding of deploying traceability practices in software maintenance and evolution phase and can be used by researchers for future directions and practitioners for making informed decisions while using traceability in maintenance and evolution.

Microservices is an architectural style that provides several benefits to develop applications as small, independent, and modular services. Building Microservices Architecture (MSA)-based applications is immensely supported by using software testing fundamentals. With the increasing interest in the development of MSA-based applications, it is important to systematically identify, analyze, and classify the publication trends, research themes, approaches, tools, and challenges in the context of testing MSA-based applications. The search yielded 2,481 articles, and 33 articles were finally selected as the primary studies with snowballing. The key findings are that (i) 5 research themes characterize testing approaches in MSA-based applications; (ii) integration and unit testing are the most popular testing approaches; and (iii) addressing the challenges in automated and inter-communication testing is gaining the interest of the community. Additionally, it emerges that there is a lack of dedicated tools to support testing for MSA-based applications, and the reasons and solutions behind the challenges in testing MSA-based applications need to be further explored.

Decisions run through the whole software development and maintenance processes. Explicitly documenting these decisions helps to organize development knowledge and to reduce its vaporization, thereby controlling the development process and maintenance costs. It can also support the knowledge acquisition process for stakeholders of the project. Meanwhile, developers (e.g., architects) and managers will be able to rely on the decisions made in the past to solve the problems encountered in their current projects. However, identifying decisions from massive textual artifacts, which involves considerable human effort, time, and cost, is usually unaffordable due to limited resources. To address this problem, we conducted an experiment to automatically identify decisions from textual artifacts using machine learning techniques. We created a dataset of 1,300 sentences labelled from the Hibernate developer mailing list, containing 650 decision sentences and non-decision sentences respectively, and trained machine learning models using 160 configurations regarding text preprocessing, feature extraction, and classification algorithms. The results show that (1) the text preprocessing method with Including Stop Words, No Stemming and Lemmatization, and No Filtering Out Sentences performs best when preprocessing posts to identify decisions; (2) the simple Bag-of-Words (BoW) model works best when extracting features to identify decisions; (3) the Support Vector Machine (SVM) algorithm gets the best result when training classifiers to identify decisions; and (4) the SVM algorithm with Including Stop Words (ISW), No Stemming and Lemmatization (NSaL), Filtering Out Sentences by Length (FOSbL), and BoW achieves the best performance (with a precision of 0.640, a recall of 0.932, and an F1-score of 0.759), compared with other configurations when identifying decisions from the mailing list.

[Background] Publications on the professional occupation of Requirements Engineering (RE) reported on market demands for both RE and non-RE qualifications and indicated the state-of-the-practice of RE roles in industry. However, prior research was not from the perspective of the RE area in the Software Engineering Body of Knowledge (SWEBOK). Nor, they shed light on the industry needs of RE professionals in China. [Aims] This paper focused on RE-specific tasks and skills sought after in China, from the perspective of RE activities elaborated in SWEBOK. [Method] Using an empirical qualitative research method, we selected and analyzed 535 job ads from China's two largest job portals. Job titles and descriptions of these ads were analyzed to uncover RErelevant responsibilities in the categories of RE activities in SWEBOK as well as RE skills. [Results] We identified the qualifications, experience and skills demanded by Chinese employers. Specifically, we reported 23 RE tasks demanded in the 535 job ads, from the perspective of SWEBOK RE activities. [Conclusion] Our findings reveal that in China's job market, 'requirements engineer' is explicitly used as a title of job ads. Plus, around 78% of the selected job positions want the employees to perform tasks in requirements elicitation. Editing requirements specification is the most in-demand task in RE activities. In addition, employers placed more emphasis on both RE-specific and broad industry experience.

Background: Code smells indicate potential design or implementation problems that may have a negative impact on programs. Similar to other software artefacts, developers use Stack Overflow (SO) to ask questions about code smells. However, given the high number of questions asked on the platform, and the limitations of the default tagging system, it takes significant effort to extract knowledge about code smells by means of manual approaches. Aim: We utilized supervised machine learning techniques to automatically identify code-smell discussions from SO posts. Method: We conducted an experiment using a manually labeled dataset that contains 3000 code-smell and 3000 non-code-smell posts to evaluate the performance of different classifiers when automatically identifying code smell discussions. Results: Our results show that Logistic Regression (LR) with parameter C=20 (inverse of regularization strength) and Bag of Words (BoW) feature extraction technique achieved the best performance amongst the algorithms we evaluated with a precision of 0.978, a recall of 0.965, and an F1-score of 0.971. Conclusion: Our results show that machine learning approach can effectively locate code-smell posts even if posts' title and/or tags cannot be of help. The technique can be used to extract code smell discussions from other textual artefacts (e.g., code reviews), and promisingly to extract SO discussions of other topics.

Defect technical debt (defect debt) refers to known deferred bugs that have not been fixed. The interest of a defect debt item (i.e., bug) is the extra effort needed to fix the bug due to the delay of fixing. It is important to measure defect debt interest in a software system in order to be aware of how much the interest of certain bugs is and which bugs should be fixed first. Furthermore, it is valuable to understand the features of the bugs of high interest or no interest, so as to facilitate the identification of those kinds of bugs. In this work, we proposed three pairs of measures for quantifying defect debt interest at three granularities (i.e., lines of code, source file, and package) of software changes, and conducted an exploratory case study on 13,438 bugs collected from 59 non-trivial Apache open source software projects written mainly in Java. We have the following findings. (1) Each bug, with an average of 224 days delay of fixing, has interest of 660 to 845 lines of code, 0.57 to 1.05 source files, and 0.29 to 0.35 package on average in need for understanding or modifying when fixing the bug. (2) The average interest of a bug shows overall increasing trends over delayed time of bug fixing and bug priority levels from the lowest to highest. (3) Around 30% of the bugs under study did not incur interest at the granularity of lines of code, which means that the involved source files in the bug-fixing commits of such bugs were not modified at all during the delayed time of bug fixing. (4) The average code change size of bug-fixing commits of a bug without interest is much smaller than that of a bug with (high) interest; for a bug without interest at the granularity of lines of code, the source files involved in the bug-fixing commits of the bug were alternately modified by different committers much less frequently than those of the bug with (high) interest.

Architecture Smells (ASs), as one source of technical debt, indicate underlying problems at a high level of systems and negatively impact various system qualities, such as maintainability and evolvability. Detecting and refactoring ASs requires the relevant architectural knowledge and experience. Therefore, gathering the knowledge of ASs from various sources can facilitate ASs detecting and refactoring. However, manually identifying AS knowledge is time-consuming. Automatically and correctly identifying AS-related posts from Stack Overflow is a step toward utilizing the AS knowledge to help developers better maintain their systems. In this work, we propose an approach to automatically identify AS-related posts from Stack Overflow (SoF) by using machine learning algorithms. We evaluate the performance of 12 classifiers based on 3 feature extraction techniques and 4 classification algorithms with a created dataset of SoF posts (including 208 AS-related posts and 187 AS-unrelated posts). The results demonstrate that the SVM algorithm with Word2Vec achieved the best overall performance with an accuracy of 0.650, a precision of 0.613, a recall of 0.905, and an F1- score of 0.731. These results imply that the obtained model of the AS-related posts identification can be used to aid developers and researchers in collecting AS discussions from SoF.

Both complexity of code change for bug fixing and bug severity play an important role in release planning when considering which bugs should be fixed in a specific release under certain constraints. This work investigates whether there are significant differences between bugs of different severity levels regarding the complexity of code change for fixing the bugs. Code change complexity is measured by the number of modified lines of code, source files, and packages, as well as the entropy of code change. We performed a case study on 20 Apache open source software (OSS) projects using commit records and bug reports. The study results show that (1) for bugs of high severity levels (i.e. Blocker, Critical and Major in JIRA), there is no significant difference on the complexity of code change for fixing bugs of different severity levels for most projects, while (2) for bugs of low severity levels (i.e. Major, Minor and Trivial in JIRA), fixing bugs of a higher severity level needs significantly more complex code change than fixing bugs of a lower severity level for most projects. These findings provide useful and practical insights for effort estimation and release planning of OSS development.

During the software development life cycle, assumptions are an important type of software development knowledge that can be extracted from textual artifacts. Analyzing assumptions can help to, for example, comprehend software design and further facilitate software maintenance. Manual identification of assumptions by stakeholders is rather time-consuming, especially when analyzing a large dataset of textual artifacts. To address this problem, one promising way is to use automatic techniques for assumption identification. In this study, we conducted an experiment to evaluate the performance of existing machine learning classification algorithms for automatic assumption identification, through a dataset extracted from the Hibernate developer mailing list. The dataset is composed of 400 "Assumption" sentences and 400 "Non-Assumption" sentences. Seven classifiers using different machine learning algorithms were selected and evaluated. The experiment results show that the SVM algorithm achieved the best performance (with a precision of 0.829, a recall of 0.812, and an F1-score of 0.819). Additionally, according to the ROC curves and related AUC values, the SVM-based classifier comparatively performed better than other classifiers for the binary classification of assumptions.

Stakeholders make various decisions regarding requirements, architectural design, project management, etc. during the life cycle of software development. They have an interest in making high-quality and appropriate decisions that meet project objectives and maximize system benefits. However, decision-making in software development is not a trivial task and the decisions made are often not well documented. In this paper, we analyzed 9006 posts from the Hibernate developer mailing list, in order to explore (1) decision expression and (2) classification, (3) the rationale behind decision-making, (4) approaches employed in decision-making, (5) related software artifacts of decision-making, and (6) the trend of decision-making over time. The results show that (1) all decisions are expressed as Information Giving, Solution Proposal, and Feature Request; (2) the main categories of decisions are Design Decision and Requirement Decision; (3) the most common rationale behind decision-making is Non-functional Requirement; (4) the most commonly used decision-making approach is Problem Structuring; (5) the main categories of software artifacts related to decision-making are Design Document and Requirement Document; and (6) decision-making runs through the whole software development life cycle.

Although managing architectural assumptions can benefit software development in several aspects (e.g., reducing architectural misunderstanding and mismatch), the effort required is a key obstacle towards employing architectural assumption management in practice. One potential solution is to apply agile practices in order to reduce this effort. To this end, we conducted a survey with 91 practitioners to investigate the possibility of integrating agile practices into architectural assumption management in industrial practice. The results offer an overview of which agile practices can be integrated in architectural assumption management and how. Six agile practices were selected by more than half of the subjects: "Backlog", "Iterative and Incremental Development", "Refactoring", "Continuous Integration", "Effective Communication", and "Just Enough Work". Twelve agile practices were further elaborated by the subjects regarding how they can be used in architectural assumption management. Based on the survey results, we developed a classification of agile practices for agile architectural assumption management, which can act as a reference for researchers and practitioners to employ certain agile practices in architectural assumption management.

Architecture Smells (ASs) are design decisions that can have significant negative effects on a system's quality attributes such as reusability and testability. ASs are focused on higher level of software systems than code smells, which are implementation-level constructs. ASs can have much wider impact on a system than code smells. However, ASs usually receive less attention than code smells in both research and practice. We have conducted an exploratory study of developers' conception of ASs by analyzing related discussions in Stack Overflow. We used 14 ASs related terms to search the relevant posts in Stack Overflow and extracted 207 posts. We used Grounded Theory method for analyzing the extracted posts about developers' description of ASs, causes of ASs, approaches and tools for detecting and refactoring ASs, quality attributes affected by ASs, and difficulties in detecting and refactoring ASs. Our findings show that: (1) developers often describe ASs with some general terms; (2) ASs are mainly caused by violating architecture patterns, design principles, or misusing architecture antipatterns; (3) there is a lack of dedicated tools for detecting and refactoring ASs; (4) developers mainly concern about the maintainability and performance of systems affected by ASs; and (5) the inability to quantify the cost and benefit as well as the lack of approaches and tools makes detecting and refactoring ASs difficult.

Recent research on the automatic classification of app reviews either focused on grouping app reviews into categories relevant to software evolution, or employed app reviews as the only research data to improve app reviews classification.Although it was reported that app review classification can benefit from supplementing user reviews with the data from other sources, only a few studies employed app changelogs for this purpose.This paper explores how to augment app reviews with changelogs to improve the accuracy and performance of classifying functional and non-functional requirements in app reviews.Specifically, we propose AUG-AC as an approach to extract feature words from app changelogs and construct the augments for app reviews.Next, we designed a series of experiments to evaluate our approach, varying in the length of AC-based augments for app reviews.The results show that AUG-AC outperforms the existing method by using app changelogs as a source of data next to app reviews.

Abstract Crowdsourcing is an appealing concept for achieving good enough requirements and just‐in‐time requirements engineering (RE). A promising form of crowdsourcing in RE is the use of feedback on software systems, generated through a large network of anonymous users of these systems over a period of time. Prior research indicated implicit and explicit user feedback as key to RE‐practitioners to discover new and changed requirements and decide on software features to add, enhance, or abandon. However, a structured account on the types and characteristics of user feedback useful for RE purposes is still lacking. This research fills the gap by providing a mapping study of literature on crowdsourced user feedback employed for RE purposes. On the basis of the analysis of 44 selected papers, we found nine pieces of metadata that characterized crowdsourced user feedback and that were employed in seven specific RE activities. We also found that the published research has a strong focus on crowd‐generated comments (explicit feedback) to be used for RE purposes, rather than employing application logs or usage‐generated data (implicit feedback). Our findings suggest a need to broaden the scope of research effort in order to leverage the benefits of both explicit and implicit feedback in RE.

The use of architecture and design patterns have impacts on the quality attributes of a system, and the application of patterns depend on design contexts. There are complex interdependent relationships between them. In this study, we explore how developers use architecture and design patterns with respect to quality attributes concerns and design contexts. We extracted pattern related posts from Stack Overflow and analyzed the architecture discussions. Our study reveals what contexts and quality attributes developers consider when using architecture patterns, and we have identified new and previously unknown relationships between these design elements. These findings can enhance developers' knowledge when they design with architecture patterns, quality attributes, and design contexts.

App user reviews have been regarded as a valuable source to elicit user requirements. Existing research on eliciting requirements from user reviews mainly focuses on functional aspects. However, Non-Functional Requirements (NFRs) from diverse sources play a critical role during software development. In this paper, we conducted an exploratory and comparative study in order to gain a deeper understanding on the nature of NFRs in user reviews, and to further compare the difference on the distributions of various NFRs between user reviews and industrial requirements specifications with respect to the system properties that NFRs address. We used 1278 user review sentences from two popular Apps (iBooks and WhatsApp) classified as NFRs that address quality characteristics, and further classified these 1278 NFRs based on the system view and the behavior theory and compared the difference on the distributions of various NFRs between user reviews and industrial requirements specifications. The results show that in App user reviews, users primarily report quality aspects on Reliability and Usability, and over 50% NFRs address interface behavior of the systems. Overall the distributions of NFRs with respect to the system view and the behavior theory in user reviews and industrial requirements specifications are similar, but the distributions of NFRs classified as architecture and state, and the distributions of NFRs in certain quality characteristic classes (e.g., the Usability class) show some differences. We concluded that most NFRs in user reviews are essentially not non-functional since they describe behavior over the interface of the systems. Compared with NFRs in industrial requirements specifications, NFRs in user reviews report more state-related aspects of the systems and care less about the architecture of the systems.

Developers constantly make various assumptions regarding requirements, environment, design decisions, etc. during software development. However, these assumptions are usually implicit and undocumented and there is a lack of understanding regarding what assumptions have been made and discussed in software development. Open Source Software (OSS) is recently becoming an important part of software industry. To this end, we conducted an exploratory study on assumptions in OSS development. We extracted and analyzed 9006 posts from the developer mailing list of Hibernate (a popular OSS project), in order to explore (1) assumption expression and (2) classification, (3) the trend of assumptions over time, and (4) related software artifacts of assumptions in OSS development. We identified 832 assumptions from the Hibernate developer mailing list. The findings are: (1) most of the assumptions are expressed as "Feature Request" and "Solution Proposal"; (2) more than half of the identified assumptions are design assumptions and are made for software design; (3) assumptions exist in the whole OSS development lifecycle; and (4) the major category of related artifacts of assumptions is "Design Document".

[Background] Recent research on mining app reviews for software evolution indicated that the elicitation and analysis of user requirements can benefit from supplementing user reviews by data from other sources. However, only a few studies reported results of leveraging app changelogs together with app reviews. [Aims] Motivated by those findings, this exploratory experimental study looks into the role of app changelogs in the classification of requirements derived from app reviews. We aim at understanding if the use of app changelogs can lead to more accurate identification and classification of functional and non-functional requirements from app reviews. We also want to know which classification technique works better in this context. [Method] We did a case study on the effect of app changelogs on automatic classification of app reviews. Specifically, manual labeling, text preprocessing, and four supervised machine learning algorithms were applied to a series of experiments, varying in the number of app changelogs in the experimental data. [Results] We compared the accuracy of requirements classification from app reviews, by training the four classifiers with varying combinations of app reviews and changelogs. Among the four algorithms, Naïve Bayes was found to be more accurate for categorizing app reviews. [Conclusions] The results show that official app changelogs did not contribute to more accurate identification and classification of requirements from app reviews. In addition, Naïve Bayes seems to be more suitable for our further research on this topic.

 -Hosting over 10 million of software projects, GitHub is one of the most important data sources to study behavior of developers and software projects.However, with the increase of the size of open source datasets, the potential threats to mining these datasets have also grown.As the dataset grows, it becomes gradually unrealistic for human to confirm quality of all samples.Some studies have investigated this problem and provided solutions to avoid threats in sample selection, but some of these solutions (e.g., finding development projects) require human intervention.When the amount of data to be processed increases, these semi-automatic solutions become less useful since the effort in need for human intervention is far beyond affordable.To solve this problem, we investigated the GHTorrent dataset and proposed a method to detect public development projects.The results show that our method can effectively improve the sample selection process in two ways: (1) We provide a simple model to automatically select samples (with 0.827 precision and 0.947 recall); (2) We also offer a complex model to help researchers carefully screen samples (with 63.2% less effort than manually confirming all samples, and can achieve 0.926 precision and 0.959 recall).

 -In software development, developers commit code changes to the version control system.In a commit message, the committer may explicitly claim that the commit is a refactoring with the intention of code quality improvement.We defined such a commit as a self-admitted refactoring (SAR).Currently, there is little knowledge about the SAR phenomenon, and the impact of SARs on software projects is not clear.In this work, we performed a preliminary investigation on SARs with an emphasis on their impact on code quality using the assessment of code smells.We used two non-trivial open source software projects as cases and employed the PMD tool to detect code smells.The study results shows that: (1) SARs tend to improve code quality, though a small proportion of SARs introduced new code smells; and (2) projects that contain SARs have different results on frequently affected code smells.

Assigning responsibilities to classes is not only vital during initial software analysis/design phases in object-oriented analysis and design (OOAD), but also during maintenance and evolution phases, when new responsibilities have to be assigned to classes or existing responsibilities have to be changed. Class Responsibility Assignment (CRA) is one of the most complex tasks in OOAD as it heavily relies on designers' judgment and implicit design knowledge (DK) of design problems. Since CRA is highly dependent on the successful use of implicit DK, (semi)-automated approaches that help designers to assign responsibilities to classes should make implicit DK explicit and exploit the DK effectively. In this paper, we propose a learning based approach for the Class Responsibility Assignment (CRA) problem. A learning mechanism is introduced into Genetic Algorithm (GA) to extract the implicit DK about which responsibilities have a high probability to be assigned to the same class, and then the extracted DK is employed automatically to improve the design quality of the generated solutions. The proposed approach has been evaluated through an experimental study with three cases. By comparing the solutions obtained from the proposed approach and the existing approaches, the proposed approach can significantly improve the design quality of the generated solutions to the CRA problem, and the generated solutions by the proposed approach are more likely to be accepted by developers from the practical aspects.

Change-based code review plays an important role in open-source project development. Due to the large amount of human involvement and tight time schedule, tools that can facilitate this activity would be of great help. Current tools mainly focus on difference extraction, code style examination, static analysis, comment and discussion, etc. However, there is little support to change impact analysis for code change review. In this paper, we serve this purpose by providing a change review assistance tool, namely, MultiViewer, for the most popular OSS GitHub. We define metrics to characterize code changes from multiple perspectives. Specifically, these metrics mine coupling relations among related files in the changes, as well as estimate the change effort, risk and impact. Such information is visualized by MultiViewer in two formats. We demonstrate the helpfulness of MultiViewer by showing its ability as indicators to some important project features with real-life case studies.

Context: The leading App distribution platforms, Apple App Store, Google Play, and Windows Phone Store, have over 4 million Apps. Research shows that user reviews contain abundant useful information which may help developers to improve their Apps. Extracting and considering Non-Functional Requirements (NFRs), which describe a set of quality attributes wanted for an App and are hidden in user reviews, can help developers to deliver a product which meets users' expectations. Objective: Developers need to be aware of the NFRs from massive user reviews during software maintenance and evolution. Automatic user reviews classification based on an NFR standard provides a feasible way to achieve this goal. Method: In this paper, user reviews were automatically classified into four types of NFRs (reliability, usability, portability, and performance), Functional Requirements (FRs), and Others. We combined four classification techniques BoW, TF-IDF, CHI2, and AUR-BoW (proposed in this work) with three machine learning algorithms Naive Bayes, J48, and Bagging to classify user reviews. We conducted experiments to compare the F-measures of the classification results through all the combinations of the techniques and algorithms. Results: We found that the combination of AUR-BoW with Bagging achieves the best result (a precision of 71.4%, a recall of 72.3%, and an F-measure of 71.8%) among all the combinations. Conclusion: Our finding shows that augmented user reviews can lead to better classification results, and the machine learning algorithm Bagging is more suitable for NFRs classification from user reviews than Naïve Bayes and J48.

Combining software architecture and agile development has received significant attention in recent years. However, there exists no comprehensive overview of the state of research on the architecture-agility combination. This work aims to analyze the combination of architecture and agile methods for the purpose of exploration and analysis with respect to architecting activities and approaches, agile methods and practices, costs, benefits, challenges, factors, tools, and lessons learned concerning the combination. A systematic mapping study (SMS) was conducted, covering the literature on the architecture-agility combination published between February 2001 and January 2014. Fifty-four studies were finally included in this SMS. Some of the highlights: (1) a significant difference exists in the proportion of various architecting activities, agile methods, and agile practices employed in the combination. (2) none of the architecting approaches has been widely used in the combination. (3) there is a lack of description and analysis regarding the costs and failure stories of the combination. (4) twenty challenges, twenty-nine factors, and twenty-five lessons learned were identified. The results of this SMS help the software engineering community to reflect on the past thirteen years of research and practice on the architecture-agility combination with a number of implications.

Managing architectural assumptions (AA) during the software lifecycle, as an important type of architecture knowledge, is critical to the success of projects. However, little empirical evidence exists on the understanding, identification, and recording of AA from the practitioners’ perspective. We investigated the current situation on (1) how practitioners understand AA and its importance, and (2) whether and how practitioners identify and record AA in software development. A web-based survey was conducted with 112 practitioners, who use Chinese as native language and are engaged in software development in China. The main findings are: (1) AA are important in both software architecting and development. However, practitioners understand AA in different ways; (2) only a few respondents identified and recorded AA in their projects, and very few approaches and tools were used for identifying and recording AA; (3) the lack of specific approaches and tools is the major challenge (reason) of (not) identifying and recording AA. The results emphasize the need for a widely accepted understanding of the AA concept in software development, and specific approaches, tools, and guidelines to support AA identification and recording.

Context: Software architecture documentation is used to communicate architectural knowledge. It is often difficult for document users to find all the architectural knowledge they need to do their tasks, and this results in wasted time and mistakes during development. Objective: In this paper we investigate how ontology-based documentation may support users in finding the architectural knowledge they need. Method: We executed a controlled experiment to test for differences in knowledge retrieval efficiency and effectiveness between two groups of master students that used two ontologies built from different understandings of the architectural knowledge needs of document users. Results: Use of the ontology built based on a better understanding of architectural knowledge needs was significantly more efficient or effective for retrieving part of the knowledge needed by document users. We analysed participants' search actions and identified which organisation of knowledge in the ontologies resulted in efficient and effective knowledge retrieval. Conclusion: We found that an improved understanding of knowledge needs allows for the construction of an ontology from which document users retrieve knowledge more efficiently and effectively. In some cases we found that the ontology support for knowledge needs had to be traded off against ontology design criteria.

Understanding the causes of architecture changes allows us to devise means to prevent architecture knowledge vaporization and architecture degeneration.But the causes are not always known, especially in open source software (OSS) development.This makes it very hard to understand the underlying reasons for the architecture changes and design appropriate modifications.Architecture information is communicated in development mailing lists of OSS projects.To explore the possibility of identifying and understanding the causes of architecture changes, we conducted an empirical study to analyze architecture information (i.e., architectural threads) communicated in the development mailing lists of two popular OSS projects: Hibernate and ArgoUML, verified architecture changes with source code, and identified the causes of architecture changes from the communicated architecture information.The main findings of this study are: (1) architecture information communicated in OSS mailing lists does lead to architecture changes in code; (2) the major cause for architecture changes in both Hibernate and ArgoUML is preventative changes.(3) more than 45% of architecture changes in both projects happened before the first stable version was released, which indicates that the architectures of the investigated OSS projects are relatively stable after the first stable release.

Review function, as a feedback mechanism from users to developers and vendors, is provided by most APP distribution platforms that allow users to rate and comment an APP after using it.User reviews are recognized as a valuable source to improve APPs and increase the value for users.With the sharp increase in the amount of user reviews, how to effectively and efficiently analyze the user reviews and identify potential and critical user needs from them to improve the APPs becomes a challenge.In this paper, we propose an approach to automatically identify requirements information and further classify them into functional and non-functional requirements from user reviews, using a combination of information retrieval technique (TF-IDF) and NLP technique (regular expression) with human intervention in keywords selection for requirements identification and classification.We validated the proposed approach with the user reviews collected from a popular APP iBooks in English App Store, and further investigated the cost and return of our approach: how the size of sample reviews for keywords selection (cost) affects the classification results in precision, recall, and F-measure (return).The results show that when setting an appropriate size of sample reviews, our approach receives a relatively stable precision, recall, and F-measure of requirements classification, in particular for non-functional requirements, which is meaningful and practical for APP developers to elicit requirements from user reviews.

Architectural technical debt (ATD) is incurred by design decisions that intentionally or unintentionally compromise system-wide quality attributes, particularly maintainability and evolvability. ATD is harmful to the system's long-term health, thus it needs to be identified for further management. However, existing ATD identification approaches are mainly based on source code analysis and thus suffer from certain shortcomings: they can only identify issues at the system implementation, they can only be employed after the systems is implemented in code, they lack a mechanism to confirm whether the potential ATD identified is real ATD or not. To address these issues, we proposed an ATD identification approach based on architecture decisions and change scenarios. To evaluate the effectiveness and usability of this approach, we conducted a case study with an information system in a large telecommunications company. The results show that the proposed approach is useful and easy to use, and it supports release planning and ATD interest measurement.

The causes of architecture changes can tell about why architecture changes, and this knowledge can be captured to prevent architecture knowledge vaporization and architecture degeneration. But the causes are not always known, especially in open source software (OSS) development. This makes it very hard to understand the underlying reasons for the architecture changes and design appropriate modifications. Architecture information is communicated in development mailing lists of OSS projects. To explore the possibility of identifying and understanding the causes of architecture changes, we conducted an empirical study to analyze architecture information (i.e. architectural threads) communicated in the development mailing lists of two popular OSS projects: Hibernate and ArgoUML, verified architecture changes with source code, and identified the causes of architecture changes from the communicated architecture information. The main findings of this study are: (1) architecture information communicated in OSS mailing lists does lead to architecture changes in code; (2) the major cause for architecture changes in both Hibernate and ArgoUML is preventative changes, and the causes of architecture changes are further classified to functional requirement, external quality requirement, and internal quality requirement using the coding techniques of grounded theory; (3) more than 45% of architecture changes in both projects happened before the first stable version was released.

Context Technical debt (TD) is a metaphor reflecting technical compromises that can yield short-term benefit but may hurt the long-term health of a software system. Objective This work aims at collecting studies on TD and TD management (TDM), and making a classification and thematic analysis on these studies, to obtain a comprehensive understanding on the TD concept and an overview on the current state of research on TDM. Method A systematic mapping study was performed to identify and analyze research on TD and its management, covering publications between 1992 and 2013. Results Ninety-four studies were finally selected. TD was classified into 10 types, 8 TDM activities were identified, and 29 tools for TDM were collected. Conclusions The term “debt” has been used in different ways by different people, which leads to ambiguous interpretation of the term. Code-related TD and its management have gained the most attention. There is a need for more empirical studies with high-quality evidence on the whole TDM process and on the application of specific TDM approaches in industrial settings. Moreover, dedicated TDM tools are needed for managing various types of TD in the whole TDM process.

Software documents are used to capture and communicate knowledge in software projects. It is important that this knowledge can be retrieved efficiently and effectively, to prevent wasted time and errors that negatively affect the quality of software. In this paper we investigate how software professionals search for knowledge in documentation. We studied the search behaviour of professionals in industry. Prior knowledge helps professionals to search software documents efficiently and effectively. However, it can also misguide professionals to an incomplete search.

Software architecture (SA) documentation provides a blueprint of a software-intensive system for the communication between stakeholders about the high-level design of the system. In open source software (OSS) development, a lack of SA documentation may hinder the use and further development of OSS, but how much "architecture" documentation is enough and appropriate is largely dependent on the contextual factors of development. In order to understand the state of the practice of SA documentation in OSS projects, we conducted a documentation-based survey to explore how SA is documented in OSS projects. Out of 2,000 OSS projects from four major OSS sources, we found that 108 projects have some SA documentation, which shows that the SA documentation is scarce in OSS development. We analyzed these 108 projects to understand what SA information has been documented and how they have been described. We have found that frequently-documented architectural information is model, system, and mission, natural language is the most frequently-used architectural language for specifying architectural information in OSS SA documents. The results also show that the likelihood that an OSS project will document SA is increased when more developers are involved in the project, and industry and research OSS projects are more likely to create SA documents than freelance projects.

Architectural design decision (ADD) and its design rationale, as a paradigm shift on documenting and enriching architecture design description, is supposed to facilitate the understanding of architecture and the reasoning behind the design rationale, which consequently improves the architecting process and gets better architecture design results. But the lack of empirical evaluation that supports this statement is one of the major reasons that prevent industrial practitioners from using ADDs in their daily architecting activities. In this paper, we conducted two controlled experiments, as a family of experiments, to investigate how presence of ADDs can improve the understanding of architecture. The main results of our experiments are: (i) using ADDs and their rationale in architecture documentation does not affect the time needed for completing architecture design tasks; (ii) one experiment and the family of experiments achieved a significantly better understanding of architecture design when using ADDs; and (iii) with regard to the correctness of architecture understanding, more experienced participants benefited more from ADDs in comparison with less experienced ones.

To reuse successful experience in software architecture design, architects use architectural patterns as reusable architectural knowledge for architectural synthesis. However, it has been observed that the resulting architecture does not always conform to the initial architectural patterns employed. Architectural synthesis using architectural patterns is also recognized as a challenging task, especially for novice architects due to lack of experience. In this paper, we propose a cooperative coevolution approach to automate architectural synthesis using architectural patterns. We first analyze several common architectural patterns and the constraints when using them. We then extend existing architectural synthesis activity with patterns based on the results of this analysis. Finally, we map the extended architectural synthesis to a cooperative coevolution model, which can optimize the resulting architectural solutions and avoid the violations to the pattern constraints automatically. We evaluate the proposed approach through a case study: architecture design of a cinema booking system. The results show that the proposed approach can generate architectural solutions which are closer to the expert design.

With the increasing complexity and uncertainty in requirements engineering (RE), the impact of power relationships between stakeholders becomes critical to the success of requirements engineering process, especially in requirements negotiation to resolve conflicting requirements. In this paper, we make use of the basic principles of requirements negotiation and define reasoning rules to reason about stakeholder groups for requirements negotiation, based on an extended i modeling framework that represents the power relationships between stakeholders in RE. We derive decision-makers and supporter groups of conflicting requirements using the reasoning rules based on power relationships between stakeholders, and further reason about representative stakeholders from the supporter groups to participate in requirements negotiation activity. We describe the reasoning process and provide a concrete case of reasoning about stakeholder groups for requirements negotiation based on the extended i model.

Requirements is the formal expression of user's needs. Also, requirements elicitation is the process of activity focusing on requirements collection. Traditional acquisition methods, such as interview, observation and prototype, are unsuited for the service-oriented software development featuring in the distributed stakeholders, collective intelligence and behavioral emergence. In this paper, a collaborative requirements elicitation approach based on social intelligence for networked software is put forward, and requirements-semantics concept is defined as the formal requirements description generated by collective participation. Furthermore, semantic wikis technology is chosen as requirements authoring platform to adapt the distributed and collaborative features. Faced to the wide-area distributed Internet, it combines with the Web 2.0 and the semantic web to revise the experts requirements-semantics model through the social classification. At the same time, instantiation of requirements model is finished with semantic tagging and validation. Apart from the traditional documentary specification, requirements-semantics artifacts will be exported from the elicitation process to the subsequent software production process, i.e. services aggregation and services resource customization. Experiment and prototype have proved the feasibility and effectiveness of the proposed approach.

The basic format in which software requirements and architecture designs are documented is essentially file-based, and it has persisted for decades. Current indexing methods used in file-based documentation are not conducive to retrieving software knowledge. We propose to index software documents with a suitable lightweight ontology to improve the retrieval and traceability of software knowledge. Initial results from a prototype implementation have shown promising prospects.

In large-scale collaborative software projects, thousands of requirements with complex interdependencies and different granularity spreading in different levels are elicited, documented, and evolved during the project lifecycle. Non-technical stakeholders involved in requirements engineering activities rarely apply formal techniques; therefore it is infeasible to automatically detect problems in requirements. This situation becomes even worse in a distributed context when all sites are responsible to maintain their own requirements list using various requirements models and management tools, and the detection of requirements problems across multiple sites is error-prone, and un-affordable if performed manually. This paper proposes an integrated approach of basing distributed requirements analysis on semantic Wiki by requirements reasoning. First, the functions concerning reasoning support provided by semantic Wiki for requirements analysis are proposed. Second, the underlying requirements rationale model for requirements reasoning is presented with sample reasoning rules. Third, our rationale model is mapped to the WinWin requirements negotiation model which further adds to its credibility.