In recent years, Continuous Integration (CI) and Continuous Delivery (CD) has been heatedly discussed and widely used in part or all of the software development life cycle as the practices and pipeline to deliver software products in an efficient way.There are many tools, such as Travis CI, that offer various features to support the CI/CD pipeline, but there is a lack of understanding about what decisions are frequently made in CI/CD.In this work, we explored one popular open-source project on GitHub, Budibase, to provide insights on the types of decisions made in CI/CD from a practitioners' perspective.We first explored the GitHub Trending page, conducted a pilot repository extraction, and identified the Budibase repository as the case for our study.We then crawled all the closed issues from the repository and got 1,168 closed issues.Irrelevant issues were filtered out based on certain criteria, and 370 candidate issues that contain decisions were obtained for data extraction.We analyzed the issues using a hybrid approach combining predefined types and the Constant Comparison method to get the categories of decisions.The results show that the major type of decisions in the Budibase closed issues is Functional Requirement Decision (67.6%), followed by Architecture Decision (11.1%).Our findings encourage developers to put more effort on the issues and making decisions related to CI/CD, and provide researchers with a reference of decision classification made in CI/CD.

Deep learning (DL) frameworks have been extensively designed, implemented, and used in software projects across many domains. However, due to the lack of knowledge or information, time pressure, complex context, etc., various uncertainties emerge during the development, leading to assumptions made in DL frameworks. Though not all the assumptions are negative to the frameworks, being unaware of certain assumptions can result in critical problems (e.g., system vulnerability and failures). As the first step of addressing the critical problems, there is a need to explore and understand the assumptions made in DL frameworks. To this end, we conducted an exploratory study to understand self-claimed assumptions (SCAs) about their distribution, classification, and impacts using code comments from nine popular DL framework projects on GitHub. The results are that: (1) 3,084 SCAs are scattered across 1,775 files in the nine DL frameworks, ranging from 1,460 (TensorFlow) to 8 (Keras) SCAs. (2) There are four types of validity of SCAs: Valid SCA, Invalid SCA, Conditional SCA, and Unknown SCA, and four types of SCAs based on their content: Configuration and Context SCA, Design SCA, Tensor and Variable SCA, and Miscellaneous SCA. (3) Both valid and invalid SCAs may have an impact within a specific scope (e.g., in a function) on the DL frameworks. Certain technical debt is induced when making SCAs. There are source code written and decisions made based on SCAs. This is the first study on investigating SCAs in DL frameworks, which helps researchers and practitioners to get a comprehensive understanding on the assumptions made. We also provide the first dataset of SCAs for further research and practice in this area.

Stakeholders make various types of decisions with respect to requirements, design, management, and so on during the software development life cycle. Nevertheless, these decisions are typically not well documented and classified due to limited human resources, time, and budget. To this end, automatic approaches provide a promising way. In this paper, we aimed at automatically classifying decisions into five types to help stakeholders better document and understand decisions. First, we collected a dataset from the Hibernate developer mailing list. We then experimented and evaluated 270 configurations regarding feature selection, feature extraction techniques, and machine learning classifiers to seek the best configuration for classifying decisions. Especially, we applied an ensemble learning method and constructed ensemble classifiers to compare the performance between ensemble classifiers and base classifiers. Our experiment results show that (1) feature selection can decently improve the classification results; (2) ensemble classifiers can outperform base classifiers provided that ensemble classifiers are well constructed; (3) BoW + 50% features selected by feature selection with an ensemble classifier that combines Naïve Bayes (NB), Logistic Regression (LR), and Support Vector Machine (SVM) achieves the best classification result (with a weighted precision of 0.750, a weighted recall of 0.739, and a weighted F1-score of 0.727) among all the configurations. Our work can benefit various types of stakeholders in software development through providing an automatic approach for effectively classifying decisions into specific types that are relevant to their interests.

Modern software systems, such as Spark, are usually written in multiple programming languages (PLs). Besides benefiting from code reuse, such systems can also take advantages of specific PLs to implement certain features, to meet various quality needs, and to improve development efficiency. In this context, a change to such systems may need to modify source files written in different PLs. We define a multi-programming-language commit (MPLC) in a version control system (e.g., Git) as a commit that involves modified source files written in two or more PLs. To our knowledge, the phenomenon of MPLCs in software development has not been explored yet. In light of the potential impact of MPLCs on development difficulty and software quality, we performed an empirical study to understand the state of MPLCs, their change complexity, as well as their impact on open time of issues and bug proneness of source files in real-life software projects. By exploring the MPLCs in 20 non-trivial Apache projects with 205,994 commits, we obtained the following findings: (1) 9% of the commits from all the projects are MPLCs, and the proportion of MPLCs in 80% of the projects goes to a relatively stable level; (2) more than 90% of the MPLCs from all the projects involve source files written in two PLs; (3) the change complexity of MPLCs is significantly higher than that of non-MPLCs in all projects; (4) issues fixed in MPLCs take significantly longer to be resolved than issues fixed in non-MPLCs in 80% of the projects; and (5) source files that have been modified in MPLCs tend to be more bug-prone than source files that have never been modified in MPLCs. These findings provide practitioners with useful insights on the architecture design and quality management of software systems written in multiple PLs.

UML diagrams are a recognized standard modelling language for representing design of software systems.For academic research, large cases containing UML diagrams are needed.One of the challenges in collecting such datasets is automatically determining whether an image is a UML diagram or not and what type of UML diagram an image contains.In this study, we collected UML diagrams from open datasets and manually labeled them into four types of UML diagrams (i.e., class diagrams, activity diagrams, sequence diagrams, and use case diagrams) and non-UML images.We evaluated the performance of five popular neural network architectures using transfer learning on the dataset of 3231 images that contains 700 class diagrams, 454 activity diagrams, 651 use case diagrams, 706 sequence diagrams, and 720 non-UML images, respectively.We also proposed our neural network architecture for multiclass classification of UML diagrams.The experiment results show that our proposed neural network architecture achieved the best performance amongst the algorithms we evaluated with an accuracy of 98.65%, a precision of 96.76%, a recall of 96.48%, and an F1-score of 96.62%.Moreover, among the neural network architectures that we have evaluated, our proposed architecture has the least parameters (around 2.4 millions) and spends the least time per image (0.0135 seconds per image using GPU) for classifying UML diagrams.

Decisions are an important type of artifacts in software development and maintenance, while decisions are not well-documented in projects due to limited human resources and budget. To this end, many studies focus on using automatic approaches to identify decisions from textual artifacts, e.g., mailing lists, issue tracking systems. In this paper, we present a replication study of our previous work (EASE2020), which conducted experiments to automatically identify decisions from the Hibernate developer mailing list. In addition, we utilized different datasets in the experiment with the aim of exploring the impact of the proprieties of dataset (i.e., the quality of positive samples, different negative samples in the dataset, and the size of the dataset) on classification results of decisions. The results show that (1) improving the quality of positive samples in the dataset can decently improve the classification results; (2) different negative samples in the dataset have an impact on the classification results; and (3) before the dataset size reaches 1200, increasing the size will improve the classification results.

Unified Modeling Language (UML) diagrams are a recognized standard modeling language for representing design of software systems. For academic research, large cases containing UML diagrams are needed. One of the challenges in collecting such datasets is automatically determining whether an image is a UML diagram or not and what type of UML diagram an image contains. In this work, we collected UML diagrams from open datasets and manually labeled them into 10 types of UML diagrams (i.e. class diagrams, activity diagrams, use case diagrams, sequence diagrams, communication diagrams, component diagrams, deployment diagrams, object diagrams, package diagrams, and state machine diagrams) and non-UML images. We evaluated the performance of seven popular neural network architectures using transfer learning on the dataset of 4706 images, including 700 class diagrams, 454 activity diagrams, 651 use case diagrams, 706 sequence diagrams, 204 communication diagrams, 208 component diagrams, 287 deployment diagrams, 207 object diagrams, 246 package diagrams, 323 state machine diagrams, and 720 non-UML images, respectively. We also proposed our neural network architecture for multiclass classification of UML diagrams. The experiment results show that Xception achieved the best performance amongst the algorithms we evaluated with a precision of 93.03%, a recall of 92.44%, and an F1-score of 92.73%. Moreover, it is possible to develop small and almost the same efficient neural network architectures, that our proposed architecture has the least parameters (around 2.4 millions) and spends the least time per image (0.0135[Formula: see text]s per image using graphics processing unit) for classifying UML diagrams with a precision of 91.25%, a recall of 90.34%, and an F1-score of 90.79%.

Defect technical debt (defect debt) refers to known deferred bugs that have not been fixed. The interest of a defect debt item (i.e., bug) is the extra effort needed to fix the bug due to the delay of fixing. It is important to measure defect debt interest in a software system in order to be aware of how much the interest of certain bugs is and which bugs should be fixed first. Furthermore, it is valuable to understand the features of the bugs of high interest or no interest, so as to facilitate the identification of those kinds of bugs. In this work, we proposed three pairs of measures for quantifying defect debt interest at three granularities (i.e., lines of code, source file, and package) of software changes, and conducted an exploratory case study on 13,438 bugs collected from 59 non-trivial Apache open source software projects written mainly in Java. We have the following findings. (1) Each bug, with an average of 224 days delay of fixing, has interest of 660 to 845 lines of code, 0.57 to 1.05 source files, and 0.29 to 0.35 package on average in need for understanding or modifying when fixing the bug. (2) The average interest of a bug shows overall increasing trends over delayed time of bug fixing and bug priority levels from the lowest to highest. (3) Around 30% of the bugs under study did not incur interest at the granularity of lines of code, which means that the involved source files in the bug-fixing commits of such bugs were not modified at all during the delayed time of bug fixing. (4) The average code change size of bug-fixing commits of a bug without interest is much smaller than that of a bug with (high) interest; for a bug without interest at the granularity of lines of code, the source files involved in the bug-fixing commits of the bug were alternately modified by different committers much less frequently than those of the bug with (high) interest.

During the software development life cycle, assumptions are an important type of software development knowledge that can be extracted from textual artifacts. Analyzing assumptions can help to, for example, comprehend software design and further facilitate software maintenance. Manual identification of assumptions by stakeholders is rather time-consuming, especially when analyzing a large dataset of textual artifacts. To address this problem, one promising way is to use automatic techniques for assumption identification. In this study, we conducted an experiment to evaluate the performance of existing machine learning classification algorithms for automatic assumption identification, through a dataset extracted from the Hibernate developer mailing list. The dataset is composed of 400 "Assumption" sentences and 400 "Non-Assumption" sentences. Seven classifiers using different machine learning algorithms were selected and evaluated. The experiment results show that the SVM algorithm achieved the best performance (with a precision of 0.829, a recall of 0.812, and an F1-score of 0.819). Additionally, according to the ROC curves and related AUC values, the SVM-based classifier comparatively performed better than other classifiers for the binary classification of assumptions.

Although managing architectural assumptions can benefit software development in several aspects (e.g., reducing architectural misunderstanding and mismatch), the effort required is a key obstacle towards employing architectural assumption management in practice. One potential solution is to apply agile practices in order to reduce this effort. To this end, we conducted a survey with 91 practitioners to investigate the possibility of integrating agile practices into architectural assumption management in industrial practice. The results offer an overview of which agile practices can be integrated in architectural assumption management and how. Six agile practices were selected by more than half of the subjects: "Backlog", "Iterative and Incremental Development", "Refactoring", "Continuous Integration", "Effective Communication", and "Just Enough Work". Twelve agile practices were further elaborated by the subjects regarding how they can be used in architectural assumption management. Based on the survey results, we developed a classification of agile practices for agile architectural assumption management, which can act as a reference for researchers and practitioners to employ certain agile practices in architectural assumption management.

Developers constantly make various assumptions regarding requirements, environment, design decisions, etc. during software development. However, these assumptions are usually implicit and undocumented and there is a lack of understanding regarding what assumptions have been made and discussed in software development. Open Source Software (OSS) is recently becoming an important part of software industry. To this end, we conducted an exploratory study on assumptions in OSS development. We extracted and analyzed 9006 posts from the developer mailing list of Hibernate (a popular OSS project), in order to explore (1) assumption expression and (2) classification, (3) the trend of assumptions over time, and (4) related software artifacts of assumptions in OSS development. We identified 832 assumptions from the Hibernate developer mailing list. The findings are: (1) most of the assumptions are expressed as "Feature Request" and "Solution Proposal"; (2) more than half of the identified assumptions are design assumptions and are made for software design; (3) assumptions exist in the whole OSS development lifecycle; and (4) the major category of related artifacts of assumptions is "Design Document".

Combining software architecture and agile development has received significant attention in recent years. However, there exists no comprehensive overview of the state of research on the architecture-agility combination. This work aims to analyze the combination of architecture and agile methods for the purpose of exploration and analysis with respect to architecting activities and approaches, agile methods and practices, costs, benefits, challenges, factors, tools, and lessons learned concerning the combination. A systematic mapping study (SMS) was conducted, covering the literature on the architecture-agility combination published between February 2001 and January 2014. Fifty-four studies were finally included in this SMS. Some of the highlights: (1) a significant difference exists in the proportion of various architecting activities, agile methods, and agile practices employed in the combination. (2) none of the architecting approaches has been widely used in the combination. (3) there is a lack of description and analysis regarding the costs and failure stories of the combination. (4) twenty challenges, twenty-nine factors, and twenty-five lessons learned were identified. The results of this SMS help the software engineering community to reflect on the past thirteen years of research and practice on the architecture-agility combination with a number of implications.

Managing architectural assumptions (AA) during the software lifecycle, as an important type of architecture knowledge, is critical to the success of projects. However, little empirical evidence exists on the understanding, identification, and recording of AA from the practitioners’ perspective. We investigated the current situation on (1) how practitioners understand AA and its importance, and (2) whether and how practitioners identify and record AA in software development. A web-based survey was conducted with 112 practitioners, who use Chinese as native language and are engaged in software development in China. The main findings are: (1) AA are important in both software architecting and development. However, practitioners understand AA in different ways; (2) only a few respondents identified and recorded AA in their projects, and very few approaches and tools were used for identifying and recording AA; (3) the lack of specific approaches and tools is the major challenge (reason) of (not) identifying and recording AA. The results emphasize the need for a widely accepted understanding of the AA concept in software development, and specific approaches, tools, and guidelines to support AA identification and recording.