Hackathons are time-bound collaborative events that often target software creation. Although hackathons have been studied in the past, existing work focused on in-depth case studies limiting our understanding of hackathons as a software engineering activity.

Developers experience a wide range of emotions while creating software. Being able to identify the causes of one’s own and peers’ emotions can equip developers with the ability to regulate their behavior to restore positive moods and productivity. In this article, we investigate to what extent self-monitoring of emotions can enhance agile retrospective meetings by improving the emotion awareness of participants. To this aim, we conducted a controlled experiment involving three software development teams involving two student teams and one professional developers team. The experimental design involves the collection of biometrics and self-reported information about emotions, which are then visualized before the retrospective meetings to inform discussion using EmoVizPhy, a tool that we designed and implemented for this aim. While students found that self-monitoring helped them recall significant emotional episodes, leading to more meaningful contributions during retrospectives, professional developers perceived limited benefits from this practice. Furthermore, based on the analysis of corrective actions identified by the participants during the study, we hypothesize that self-monitoring of emotions through EmoVizPhy may play a valuable role in facilitating the consolidation of new agile teams for which roles and collaboration dynamics are still being defined.

Docker containers are widely used in modern enterprise applications and cloud environments for their efficiency, portability, and rapid deployment. As a leading containerization technology, Docker enables applications to be stored as images containing all required runtime dependencies. Nonetheless, the growing popularity of containers has also raised security concerns as the libraries and dependencies included in Docker images can contain security vulnerabilities. Previous research has predominantly focused on operating system vulnerabilities, with some investigations into application vulnerabilities originating from vulnerable dependencies. However, these studies have focused solely on the latest tag (version) of each Docker image repository, without offering insights into the prevalence and potential resolution of vulnerabilities across different releases. This limitation restricts our understanding of how effectively they are managed over time. In this study, we investigate the prevalence of vulnerable JavaScript packages within Docker containers across multiple release tags. Our time-based analysis enables us to assess the extent to which maintainers resolve these vulnerabilities in subsequent releases, as well as the time required to address them. We analyzed$\mathbf{6, 2 9 2}$unique images gathered from 1,573 active repositories. Our findings indicate that the majority of Docker images contain multiple vulnerabilities across various tags. Nearly 61 % of repositories have vulnerabilities in every examined tag. While some of these vulnerabilities are resolved by maintainers in subsequent releases, many remain unaddressed within our observation timeframe. Moreover, we found that only 10 % of vulnerabilities are typically addressed within the first 6 months, leaving many unattended for considerably longer durations. We also discovered that common repository attributes, including popularity, contributor count, and automation usage, have no significant effect on the timeliness of vulnerability resolution.

Abstract Self-Admitted Technical Debt, or SATD, is a self-admission of technical debt present in a software system. The presence of SATD in software systems negatively affects developers, therefore, managing and addressing SATD is crucial for software engineering. To effectively manage SATD, developers need to estimate its priority and assess the effort required to fix the described technical debt. About a quarter of descriptions of SATD in software systems express some form of negativity or negative emotions when describing technical debt. In this paper, we report on an experiment conducted with 59 respondents to study whether negativity expressed in the description of SATD actually affects the prioritization of SATD. The respondents are a mix of professional developers and students, and in the experiment, we asked participants to prioritize four vignettes: two expressing negativity and two expressing neutral sentiment. To ensure the vignettes were realistic, they were based on existing SATD extracted from a dataset. We find that negativity causes between one-third and half of developers to prioritize SATD in which negativity is expressed as having more priority. Developers affected by negativity when prioritizing SATD are twice as likely to increase their estimation of urgency and 1.5 times as likely to increase their estimation of importance and effort for SATD compared to the likelihood of decreasing these prioritization scores. Our findings show how developers actively use negativity in SATD to determine how urgently a particular instance of technical debt should be addressed. However, our study also describes a gap in the actions and belief of developers. Even if 33% to 50% use negativity to prioritize SATD, 67% of developers believe that using negativity as a proxy for priority is unacceptable. Therefore, we would not recommend using negativity as a proxy for priority. However, we also recognize it might be unavoidable that negativity is expressed by developers to describe technical debt.

Abstract The growing use of third-party libraries in software development poses a hidden security risk, as vulnerabilities in these libraries can easily spread to dependent applications. Project maintainers must remain vigilant regarding updates and patches for these external libraries, a responsibility that is facilitated by automated tools, also known as bots . This study centers on Dependabot, a widely adopted bot that offers security and version updates. We aim to scrutinize the impact of Dependabot on mitigating vulnerabilities arising from dependencies, preventing potential prolonged security issues in open-source software. We investigate how developers react to security updates provided by Dependabot within engineered and actively maintained JavaScript projects. We also delve into how project attributes, including the integration of tests and continuous integration (CI) tools, influence the acceptance rate of security updates. Additionally, we perform a detailed analysis of the lifespan of each vulnerability to demonstrate how they are dealt with when Dependabot is in use. Our findings reveal a significant reliance on Dependabot by developers for managing security vulnerabilities in dependencies, with most updates being merged swiftly within days. We find that projects equipped with tests and CI tools are more likely to merge security updates. Conversely, when developers opt not to merge a security update, they often manually address the identified vulnerability. This manual approach, however, could span over several months, potentially exposing projects to security risks. Crucially, in many instances, the manual fixes are potentially inspired by earlier security updates, underscoring Dependabot’s pivotal role in safeguarding dependencies.

Abstract The Coq Community Survey 2022 was an online public survey of users of the Coq proof assistant conducted during February 2022. Broadly, the survey asked about use of Coq features, user interfaces, libraries, plugins, and tools, views on renaming Coq and Coq improvements, and also demographic data such as education and experience with Coq and other proof assistants and programming languages. The survey received 466 submitted responses, making it the largest survey of users of an interactive theorem prover (ITP) so far. We present the design of the survey, a summary of key results, and analysis of answers relevant to ITP technology development and usage. In particular, we analyze user characteristics associated with adoption of tools and libraries and make comparisons to adjacent software communities. Notably, we find that experience has significant impact on Coq user behavior, including on usage of tools, libraries, and integrated development environments (IDEs).

Hybrid hackathons, which combine in-person and online participation, present unique challenges for organizers and participants. Although now widely practiced, research on them remains fragmented, with limited integration between hackathon studies and hybrid collaboration theories. Existing strategies for in-person or online-only events often fail to address hybrid-specific issues, such as managing communication across physical and virtual spaces and ensuring balanced participation. Our work examines hybrid hackathons through the lens of hybrid collaboration theories, focusing on how organizers structure these events and how participants navigate associated challenges. We frame our analysis using established theories of hybrid collaboration, focusing on four key dimensions - synchronicity, physical distribution, dynamic transitions, and technological infrastructure - that shape collaboration in hybrid events. Using an exploratory case study of three hybrid hackathons, involving observations and interviews with organizers and participants, we investigate how these dimensions are implemented and how they shape participant experiences. Our findings show differing organizer approaches to the hybrid dimensions, leading to varied participant experiences in access to resources, communication, and coordination. Implementation styles, favoring in-person, online, or balanced participation, produced disparities in engagement. Organizers often relied on technology to bridge physical and virtual spaces, but overlooked critical aspects such as time-zone management, dynamic transitions, and targeted hybrid team support. Participants, in turn, adapted collaboration tactics in response to gaps in event scaffolding, sometimes enabling collaboration, but at other times creating new challenges, underscoring that considerations for the hybrid format are still not fully integrated into hackathon planning. Learning from our findings, we offer practical recommendations for both organizers and participants to improve planning, participation, and collaboration in hybrid hackathon events.

Introduction : Digital products increasingly reshape industries, influencing human behavior and decision-making. However, the software development teams developing these systems often lack diversity, which may lead to designs that overlook the needs, equal treatment or safety of diverse user groups. These risks highlight the need for fostering diversity and inclusion in software development to create safer, more equitable technology. Method : This research is based on insights from an academic meeting in June 2023 involving 23 software engineering researchers and practitioners. We used the collaborative discussion method 1-2-4-ALL as a systematic research approach and identified six themes around the theme “challenges and opportunities to improve Software Developer Diversity and Inclusion (SDDI).” We identified benefits, harms, and future research directions for the four main themes. Then, we discuss the remaining two themes, AI & SDDI and AI & Computer Science education, which have a cross-cutting effect on the other themes. Results : This research explores the key challenges and research opportunities for promoting SDDI, providing a roadmap to guide both researchers and practitioners. We underline that research around SDDI requires a constant focus on maximizing benefits while minimizing harms, especially to vulnerable groups. As a research community, we must strike this balance in a responsible way.

Software developers often use logs to, e.g., investigate bugs, familiarize themselves with the underlying system, or improve performance. To do so, they commonly rely on text editors or their own scripts. This lack of appropriate tooling remains a primary challenge in the industrial application of log analysis (state-of-the-practice), despite many tools and techniques proposed by previous scientific studies (state-of-the-art). To aid in bridging this gap between industry and academia, between state-of-the-practice and state-of-the-art, we zoom in on the ways developers perform log analysis. In particular, we conduct an exploratory case study to understand what structures developers identify in logs and how they utilize their knowledge in this process. Based on the results of the case study, we identify two classes of features, one related to encoding domain knowledge and another one related to sharing domain knowledge. We implement two features from the first class in an open-source log analysis platform designed in collaboration with our industrial partner. To evaluate the impact of the implemented features on log analysis, we conduct a user evaluation with software developers from our industrial partner. During this evaluation developers complete several tasks using the features and complete a usability questionnaire. Results show that users are able to encode their domain knowledge about the logs during their analysis. Furthermore, we observe that participants value highly ease of use and indicate an interest in using the features in their current practice. This sentiment is reflected in the resulting scores of the usability questionnaire, indicating above-average usability. Our findings pave the way to bridge the gap between academia and industry and facilitate the application of advanced log analysis approaches in industry.

GitHub Copilot is an AI-enabled tool that automates program synthesis. It has gained significant attention since its launch in 2021. Recent studies have extensively examined Copilot's capabilities in various programming tasks, as well as its security issues. However, little is known about the effect of different natural languages on code suggestion. Natural language is considered a social bias in the field of NLP, and this bias could impact the diversity of software engineering. To address this gap, we conducted an empirical study to investigate the effect of three popular natural languages (English, Japanese, and Chinese) on Copilot. We used 756 questions of varying difficulty levels from AtCoder contests for evaluation purposes. The results highlight that the capability varies across natural languages, with Chinese achieving the worst performance. Furthermore, regardless of the type of natural language, the performance decreases significantly as the difficulty of questions increases. Our work represents the initial step in comprehending the significance of natural languages in Copilot's capability and introduces promising opportunities for future endeavors.

Technical debt (TD) refers to the accumulation of negative consequences resulting from sub-optimal solutions during software development. A recent paper by Edbert et al. studied the difference between security-related TD questions, and security-related non- TD questions on Stack Overflow (SO). One of the characteristics under investigation is the sentiment expressed in these two categories as sentiment provides insight into developers' attitudes and emotions toward security-related TD. To this end, Edbert et al. used a general-purpose, off-the-shelf, sentiment analysis tool. However, previous research has shown that general-purpose off-the-shelf sentiment tools are potentially unreliable when applied to software engineering texts. Therefore, we replicate the study by Edbert et al. using state-of-the-art sentiment analysis tools purpose-built and fine-tuned on SE data, to understand whether and how tool-choice influences the obtained results. We consider both shallow (Senti4SD) and deep learning (BERT4SentiSE) tools. To further understand the differences between shallow and deep-learning sentiment analysis tools, we perform a qualitative analysis into the underlying reasons for tools disagreement. We identify five categories of disagreements: misunderstanding context, courtesy phrases, subjective sentiment, brevity, and divergent examples. Our results are relevant to academics, reiterating the relevance of careful selection of tools used to perform sentiment analysis. Furthermore, the results are relevant to users and developers of sentiment analysis tools, as they inform tool selection dependent on the application domain, and provide insight into optimization of the pre-processing steps. Finally, our study shows that retraining sentiment analysis tools with identical data fails to resolve fundamental inconsis-tencies between how certain types of language, such as courtesy phrases, are classified.

Abstract Context EMF metamodels lie at the heart of model-based approaches for a variety of tasks, notably for defining the abstract syntax of modeling languages. The language design of EMF metamodels itself is part of a design process, where the needs of its specific range of users should be satisfied. Studying how people actually use the language in the wild would enable empirical feedback for improving the design of the EMF metamodeling language. Objective Our goal is to study the language usage of EMF metamodels in public engineered projects on GitHub. We aim to reveal information about the usage of specific language constructs, whether they match the language design. Based on our findings, we plan to suggest improvements in the EMF metamodelling language. Method We adopt a sample study research strategy and collect data from the EMF metamodels on GitHub. After a series of preprocessing steps including filtering out non-engineered projects and deduplication, we employ an analytics workflow on top of a graph database to formulate generalizing statements about the artifacts under study. Based on the results, we also give actionable suggestions for the EMF metamodeling language design. Results We have conducted various analyses on metaclass, attribute, feature/relationship usage as well as specific parts of the language: annotations and generics. Our findings reveal that the most used metaclasses are not the main building blocks of the language, but rather auxiliary ones. Some of the metaclasses, metaclass features and relations are almost never used. There are a few attributes which are almost exclusively used with a single value or illegal values. Some of the language features such as special forms of generics are very rarely used. Based on our findings, we provide suggestions to improve the EMF language, e.g. removing a language element, restricting its values or refining the metaclass hierarchy. Conclusions In this paper, we present an extensive empirical study into the language usage of EMF metamodels on GitHub. We believe this study fills a gap in the literature of model analytics and will hopefully help future improvement of the EMF metamodeling language.

Abstract Sentiment analysis has been used to study aspects of software engineering, such as issue resolution, toxicity, and self-admitted technical debt. To address the peculiarities of software engineering texts, sentiment analysis tools often consider the specific technical lingo practitioners use. To further improve the application of sentiment analysis, there have been two recommendations: Using pre-trained transformer models to classify sentiment and replacing non-natural language elements with meta-tokens. In this work, we benchmark five different sentiment analysis tools (two pre-trained transformer models and three machine learning tools) on 2 gold-standard sentiment analysis datasets. We find that pre-trained transformers outperform the best machine learning tool on only one of the two datasets, and that even on that dataset the performance difference is a few percentage points. Therefore, we recommend that software engineering researchers should not just consider predictive performance when selecting a sentiment analysis tool because the best-performing sentiment analysis tools perform very similarly to each other (within 4 percentage points). Meanwhile, we find that meta-tokenization does not improve the predictive performance of sentiment analysis tools. Both of our findings can be used by software engineering researchers who seek to apply sentiment analysis tools to software engineering data.

Abstract Context Finding code changes that introduced bugs is important both for practitioners and researchers, but doing it precisely is a manual, effort-intensive process. The perfect test method is a theoretical construct aimed at detecting Bug-Introducing Changes (BIC) through a theoretical perfect test. This perfect test always fails if the bug is present, and passes otherwise. Objective To explore a possible automatic operationalization of the perfect test method. Method To use regression tests as substitutes for the perfect test. For this, we transplant the regression tests to past snapshots of the code, and use them to identify the BIC, on a well-known collection of bugs from the Defects4J dataset. Results From 809 bugs in the dataset, when running our operationalization of the perfect test method, for 95 of them the BIC was identified precisely and in the remaining 4 cases, a list of candidates including the BIC was provided. Conclusions We demonstrate that the operationalization of the perfect test method through regression tests is feasible and can be completely automated in practice when tests can be transplanted and run in past snapshots of the code. Given that implementing regression tests when a bug is fixed is considered a good practice, when developers follow it, they can detect effortlessly bug-introducing changes by using our operationalization of the perfect test method.

Reflecting on the limitations of a study is a crucial part of the research process. In software engineering studies, this reflection is typically conveyed through discussions of study limitations or threats to validity. In current practice, such discussions seldom provide sufficient insight to understand the rationale for decisions taken before and during the study, and their implications. We revisit the practice of discussing study limitations and threats to validity and identify its weaknesses. We propose to refocus this practice of self-reflection to a discussion centered on the notion of trade-offs . We argue that documenting trade-offs allows researchers to clarify how the benefits of their study design decisions outweigh the costs of possible alternatives. We present guidelines for reporting trade-offs in a way that promotes a fair and dispassionate assessment of researchers’ work.

Advances in Artificial Intelligence (AI), and in particular in Machine Learning (ML), are introducing profound changes to scholarly submissions across publication venues, affecting in particular the contributions that are being submitted to Software Engineering (SE) conferences and journals. In this context, it is not always clear whether manuscripts submitted to SE venues under the umbrella term SE for AI are indeed relevant to SE, in the sense that they explicitly contain contributions to the SE body of knowledge. This leads to recurring discussions on whether certain AI-related submissions are appropriate to SE venues, or should instead be submitted to other journals and conferences, including AI or ML-specific ones. In this editorial, we discuss the kinds of AI-related contributions that are a better fit-and a less good fit-for publication in the IEEE Transactions on Software Engineering.

Scientific workflow has become essential in software engineering because it provides a structured approach to designing, executing, and analyzing scientific experiments. Software developers and researchers have developed hundreds of scientific workflow management systems so scientists in various domains can benefit from them by automating repetitive tasks, enhancing collaboration, and ensuring the reproducibility of their results. However, even for expert users, workflow creation is a complex task due to the dramatic growth of tools and data heterogeneity. Thus, scientists attempt to reuse existing workflows shared in workflow repositories. Unfortunately, several challenges prevent scientists from reusing those workflows. Thus, we first attempted to identify those reusability challenges in this study. We also offered an action list and evidence-based guidelines to promote the reusability of scientific workflows. Our intensive manual investigation examined the reusability of existing workflows and exposed several challenges. The challenges preventing reusability include tool upgrading, tool support unavailability, design flaws, incomplete workflows, failure to load a workflow, etc. Such challenges and our action list offered guidelines to future workflow composers to create better workflows with enhanced reusability. In the future, we plan to develop a recommender system using reusable workflows that can assist scientists in creating effective and error-free workflows.

The intersection of ageism and sexism can create a hostile environment for veteran software developers belonging to marginalized genders. In this study, we conducted 14 interviews to examine the experiences of people at this intersection, primar-ily women, in order to discover the strategies they employed in order to successfully remain in the field. We identified 283 codes, which fell into three main categories: Strategies, Experiences, and Perception. Several strategies we identified, such as (Deliberately) Not Trying to Look Younger, were not previously described in the software engineering literature. We found that, in some compa-nies, older women developers are recognized as having particular value, further strengthening the known benefits of diversity in the workforce. Based on the experiences and strategies, we suggest organizations employing software developers to consider the benefits of hiring veteran women software developers. For example, companies can draw upon the life experiences of older women developers in order to better understand the needs of customers from a similar demographic. While we recognize that many of the strategies employed by our study participants are a response to systemic issues, we still consider that, in the short-term, there is benefit in describing these strategies for developers who are experiencing such issues today.

Software bots fulfill an important role in collective software development, and their adoption by developers promises increased productivity. Past research has identified that bots that communicate too often can irritate developers, which affects the utility of the bot. However, it is not clear what other properties of human-bot collaboration affect developers' preferences, or what impact these properties might have. The main idea of this paper is to explore characteristics affecting developer preferences for interactions between humans and bots, in the context of GitHub pull requests. We carried out an exploratory sequential study with interviews and a subsequent vignette-based survey. We find developers generally prefer bots that are personable but show little autonomy, however, more experienced developers tend to prefer more autonomous bots. Based on this empirical evidence, we recommend bot developers increase configuration options for bots so that individual developers and projects can configure bots to best align with their own preferences and project cultures.

International audience

Modern software development practices increasingly rely on third-party libraries due to the inherent benefits of reuse. However, libraries may contain security vulnerabilities that can propagate to the dependent applications. To counter this, maintainers of dependent projects should monitor their dependencies and security reports to ensure that only patched releases of the upstream applications are in use. As manual maintenance of dependencies has shown to be ineffective, several automated tools (aka bots) have been proposed to assist developers in rapidly identifying and resolving vulnerable dependencies. In this work, we focus on Dependabot, a popular bot providing security and version updates, and study developers’ receptivity to its security updates in engineered and actively maintained JavaScript projects. Moreover, we carry out a fine-grained analysis of the lifecycle of every vulnerability to manifest how they are dealt with in the presence of Dependabot. Our findings show that the task of fixing vulnerable dependencies is, to a large extent, delegated to Dependabot and that developers merge the majority of security updates within several days. On the other hand, when developers do not merge a security update, they usually address the identified vulnerability manually. This approach, however, often takes up to several months which in turn could expose the projects to security issues.

Peer code review is a widely practiced software engineering process in which software developers collaboratively evaluate and improve source code quality. Whether developers can perform good reviews depends on whether they have sufficient competence and experience. However, the knowledge of what competencies developers need to execute code review is currently limited, thus hindering, for example, the creation of effective support tools and training strategies. To address this gap, we firstly identified 27 competencies relevant to performing code review through expert validation. Later, we conducted an online survey with 105 reviewers to rank these competencies along four dimensions: frequency of usage, importance, proficiency, and desire of reviewers to improve in that competency. The survey shows that technical competencies are considered essential to performing reviews and that respondents feel generally confident in their technical proficiency. Moreover, reviewers feel less confident in how to communicate clearly and give constructive feedback - competencies they consider like-wise an essential part of reviewing. Therefore, research and education should focus in more detail on how to support and develop reviewers' potential to communicate effectively during reviews. In the paper, we also discuss further implications for training, code review performance assessment, and reviewers of different experience level. Data and materials: https://doi.org/10.5281/zenodo.7401313

Early identification of emotions of software developers can enable timely intervention in order to support developers' well-being and prevent burnout. We present a machine learning experiment aimed at recognizing emotions during programming tasks using wearable biometric sensors, tracking electrodermal activity and heart-related metrics. As a gold standard for supervised learning, we rely on a state-of-the-art tool for emotion recognition based on facial expression analysis. We design, implement and evaluate an approach that combines the output of two classifiers for neutral valence recognition and positive/negative polarity classification. Our findings suggest that biometric sensors in a wristband can be used to identify emotions whose recognition would otherwise need an intrusive webcam.

Recently, the job market for Artificial Intelligence (AI) engineers has exploded. Since the role of AI engineer is relatively new, limited research has been done on the requirements as set by the industry. Moreover, the definition of an AI engineer is less established than for a data scientist or a software engineer. In this study we explore, based on job ads, the requirements from the job market for the position of AI engineer in The Netherlands. We retrieved job ad data between April 2018 and April 2021 from a large job ad database, Jobfeed from TextKernel. The job ads were selected with a process similar to the selection of primary studies in a literature review. We characterize the 367 resulting job ads based on meta-data such as publication date, industry/sector, educational background and job titles. To answer our research questions we have further coded 125 job ads manually.

Code review is a popular software engineering practice. Success of code reviews can be threatened by confusion experienced by code reviewers. For instance, on the one hand, research has studied the reasons for confusion in code reviews, and on the other hand, it also has analyzed source code patterns, so called "atoms of confusion", that have been shown to lead to misunderstanding in the lab setting. However, to the best of our knowledge, there is no research which tried to investigate the possible cause and effect relationship between atoms of confusion and confusion in code reviews. Another important aspect still not studied is how those atoms of confusion evolve across pull requests. In this emerging results paper, we report an exploratory case study to provide a deeper understanding of atoms of confusion, more specifically, whether atoms of confusion are related to confusion in code reviews and how they persist across pull requests. With the help of an existing tool for the detection of atoms of confusion, and a manual analysis of code reviews comments, we observed that statistical analysis did not show any relationship between atoms of confusion and presence of confusion comments in code reviews. Additionally, we found evidence that atoms of confusion are mostly not being removed in pull requests. Based on the results, we formulate hypotheses on atoms of confusion in the code review context, that should be confirmed or rejected by future studies.

Open-Source Software (OSS) projects have adopted various automations for repetitive tasks in recent years. One common type of automation in OSS is bots. In this exploratory case study, we seek to understand how the adoption of one particular bot (ASFBot) by the Apache Software Foundation (ASF) impacts the discussions in the issue-trackers of these projects. We use the SmartShark dataset to investigate whether the ASFBot affects (i) human comments mentioning pull requests and fixes in issue comments and (ii) the general human comment rate on issues. We apply a regression discontinuity design (RDD) on nine ASF projects that have been active both before and after the ASFBot adoption. Our results indicate (i) an immediate decrease in the number of median comments mentioning pull requests and fixes after the bot adoption, but the trend of a monthly decrease in this comment count is reversed, and (ii) no effect in the number of human comments after the bot adoption. We make an effort to gather first insights in understanding the impact of adopting the ASFBot on the commenting behavior of developers who are working on ASF projects.

Static analysis tools generate a large number of alarms that require manual inspection. In prior work, repositioning of alarms is proposed to (1) merge multiple similar alarms together and replace them by a fewer alarms, and (2) report alarms as close as possible to the causes for their generation. The premise is that the proposed merging and repositioning of alarms will reduce the manual inspection effort. To evaluate the premise, this paper presents an empirical study with 249 developers on the proposed merging and repositioning of static alarms. The study is conducted using static analysis alarms generated on$C$programs, where the alarms are representative of the merging vs. non-merging and repositioning vs. non-repositioning situations in real-life code. Developers were asked to manually inspect and determine whether assertions added corresponding to alarms in$C$code hold. Additionally, two spatial cognitive tests are also done to determine relationship in performance. The empirical evaluation results indicate that, in contrast to expectations, there was no evidence that merging and repositioning of alarms reduces manual inspection effort or improves the inspection accuracy (at times a negative impact was found). Results on cognitive abilities correlated with comprehension and alarm inspection accuracy.

Static analysis tools help to detect common pro-gramming errors but generate a large number of false positives. Moreover, when applied to evolving software systems, around 95 % of alarms generated on a version are repeated, i.e., they have also been generated on the previous version. Version-aware static analysis techniques (VSATs) have been proposed to suppress the repeated alarms that are not impacted by the code changes between the two versions. The alarms reported by VSATs after the suppression, called delta alarms, still constitute 63% of the tool-generated alarms. We observe that delta alarms can be further postprocessed using their corresponding code changes: the code changes due to which VSATs identify them as delta alarms. However, none of the existing VSATs or alarms postprocessing techniques postprocesses delta alarms using the corresponding code changes. Based on this observation, we use the code changes to classify delta alarms into six classes that have different priorities assigned to them. The assignment of priorities is based on the type of code changes and their likelihood of actually impacting the delta alarms. The ranking of alarms, obtained by prioritizing the classes, can help suppress alarms that are ranked lower, when resources to inspect all the tool-generated alarms are limited. We performed an empirical evaluation using 9789 alarms generated on 59 versions of seven open source C applications. The evaluation results indicate that the proposed classification and ranking of delta alarms help to identify, on average, 53 % of delta alarms as more likely to be false positives than the others.

Social debts in software teams are gaining increasing attention from the research community due to their potential adverse effects on software quality. For instance, community smells are indicators of sub-optimal organizational structures and may well lead to the emergence of social debt. Previous studies analyzed which factors influence the emergence/mitigation of such smells. In particular, studies by Catolino et al. showed how factors related to team composition, particularly gender diversity, correlated to the mitigation of community smells. However, a confirmation survey on 60 practitioners suggested that these results were not aligned with the experts' perceptions. In a separate survey, Catolino et al. collected the most common team refactoring strategies for those community smells. In this work we replicate two studies by those authors, focusing on the Brazilian software teams; culture-specific expectations on the behavior of people of different genders might have affected the perception of the importance of gender diversity and refactoring strategies when mitigating community smells. We translated the survey instrument used by Catolino et al. to Brazilian Portuguese and recruited 184 Brazilian developers. Re-sults did not show significant differences from the original study; indeed, participants perceived gender diversity as less valuable to mitigate community smells than such factors like experience or team size. Additionally, we performed a qualitative analysis of an open question within the questionnaire for the refactoring strategies. Brazilian developers agree with the original studies for most smells, mainly promoting restructuring communities, creating a communication plan and mentoring. We believe these results provide further evidence on the problem and its implications when managing software teams, avoiding technical debt and maintenance issues due to team communication and coordination problems.

Abstract Self-Admitted Technical Debt (SATD) consists of annotations—typically, but not only, source code comments—pointing out incomplete features, maintainability problems, or, in general, portions of a program not-ready yet. The way a SATD comment is written, and specifically its polarity, may be a proxy indicator of the severity of the problem and, to some extent, of the priority with which it should be addressed. In this paper, we study the relationship between different types of SATD comments in source code and their polarity, to understand in which circumstances (and why) developers use negative or rather neutral comments to highlight an SATD. To address this goal, we combine a manual analysis of 1038 SATD comments from a curated dataset with a survey involving 46 professional developers. First of all, we categorize SATD content into its types. Then, we study the extent to which developers express negative sentiment in different types of SATD as a proxy for priority, and whether they believe this can be considered as an acceptable practice. Finally, we look at whether such annotations contain additional details such as bug references and developers’ names/initials. Results of the study indicate that SATD comments are mainly used for annotating poor implementation choices ( $\simeq $ ≃ 41%) and partially implemented functionality ( $\simeq $ ≃ 22%). The latter may depend from “waiting” for other features being implemented, and this makes SATD comments more negatives than in other cases. Around 30% of the survey respondents agree on using/interpreting negative sentiment as a proxy for priority, while 50% of them indicate that it would be better to discuss SATD on issue trackers and not in the source code. However, while our study indicates that open-source developers use links to external systems, such as bug identifiers, to annotate high-priority SATD, better tool support is required for SATD management.

Abstract Context Tangled commits are changes to software that address multiple concerns at once. For researchers interested in bugs, tangled commits mean that they actually study not only bugs, but also other concerns irrelevant for the study of bugs. Objective We want to improve our understanding of the prevalence of tangling and the types of changes that are tangled within bug fixing commits. Methods We use a crowd sourcing approach for manual labeling to validate which changes contribute to bug fixes for each line in bug fixing commits. Each line is labeled by four participants. If at least three participants agree on the same label, we have consensus. Results We estimate that between 17% and 32% of all changes in bug fixing commits modify the source code to fix the underlying problem. However, when we only consider changes to the production code files this ratio increases to 66% to 87%. We find that about 11% of lines are hard to label leading to active disagreements between participants. Due to confirmed tangling and the uncertainty in our data, we estimate that 3% to 47% of data is noisy without manual untangling, depending on the use case. Conclusion Tangled commits have a high prevalence in bug fixes and can lead to a large amount of noise in the data. Prior research indicates that this noise may alter results. As researchers, we should be skeptics and assume that unvalidated data is likely very noisy, until proven otherwise.

Abstract Software bots have been facilitating several development activities in Open Source Software (OSS) projects, including code review. However, these bots may bring unexpected impacts to group dynamics, as frequently occurs with new technology adoption. Understanding and anticipating such effects is important for planning and management. To analyze these effects, we investigate how several activity indicators change after the adoption of a code review bot. We employed a regression discontinuity design on 1,194 software projects from GitHub. We also interviewed 12 practitioners, including open-source maintainers and contributors. Our results indicate that the adoption of code review bots increases the number of monthly merged pull requests, decreases monthly non-merged pull requests, and decreases communication among developers. From the developers’ perspective, these effects are explained by the transparency and confidence the bot comments introduce, in addition to the changes in the discussion focused on pull requests. Practitioners and maintainers may leverage our results to understand, or even predict, bot effects on their projects.

Opinion mining, sometimes referred to as sentiment analysis, has gained increasing attention in software engineering (SE) studies. SE researchers have applied opinion mining techniques in various contexts, such as identifying developers’ emotions expressed in code comments and extracting users’ critics toward mobile apps. Given the large amount of relevant studies available, it can take considerable time for researchers and developers to figure out which approaches they can adopt in their own studies and what perils these approaches entail. We conducted a systematic literature review involving 185 papers. More specifically, we present (1) well-defined categories of opinion mining-related software development activities, (2) available opinion mining approaches, whether they are evaluated when adopted in other studies, and how their performance is compared, (3) available datasets for performance evaluation and tool customization, and (4) concerns or limitations SE researchers might need to take into account when applying/customizing these opinion mining techniques. The results of our study serve as references to choose suitable opinion mining tools for software development activities and provide critical insights for the further development of opinion mining techniques in the SE domain.

Emotions are known to impact cognitive skills, thus influencing job performance. This is also true for software development, which requires creativity and problem-solving abilities. In this paper, we report the results of a field study involving professional developers from five different companies. We provide empirical evidence that a link exists between emotions and perceived productivity at the workplace. Furthermore, we present a taxonomy of triggers for developers’ positive and negative emotions, based on the qualitative analysis of participants’ self-reported answers collected through daily experience sampling. Finally, we experiment with a minimal set of non-invasive biometric sensors that we use as input for emotion detection. We found that positive emotional valence, neutral arousal, and high dominance are prevalent. We also found a positive correlation between emotional valence and perceived productivity, with a stronger correlation in the afternoon. Both social and individual breaks emerge as useful for restoring a positive mood. Furthermore, we found that a minimum set of non-invasive biometric sensors can be used as a predictor for emotions, provided that training is performed on an individual basis. While promising, our classifier performance is not yet robust enough for practical usage. Further data collection is required to strengthen the classifier, by also implementing individual fine-tuning of emotion models.

Complex embedded systems, such as robotics, automotive and high-tech manufacturing, are hard to maintain due to their complex nature. To advance our understanding of the software engineering practice for complex embedded systems, we conducted a series of empirical studies at ASML, a leading manufacturer of lithography machines for semi-conductor industry. We started with an interview study exploring how developers use execution logs, essential artifacts that capture the runtime behavior of software systems. The empirical insights obtained from this study led us to explore subtopics about model inference from logs, modeling practice and log comparison. Motivated by the observation that developers often manually sketch behavioral models based on logs, we propose a model inference technique that can extract models by combining log analysis, and analysis of a running system under stimuli. As observed in this model inference study, the transition from code to models requires developers to work with a hybrid system which consists of handwritten code and models. We then study modeling practices and the roles of model in such hybrid systems. Particularly, we study why developers violate modeling guidelines, providing implications for researchers and tool builders to support developers in modeling complex embedded systems. Another interesting observation from the interview study is that developers face challenges in comparing multiple logs generated from such systems. We therefore conduct a literature study to provide an overview of the existing techniques and identify the limitations of the existing techniques. In this project, we study logs and models in complex embedded systems, providing tool builders, researchers and practitioners with implications to facilitate log analysis, model inference, modeling practice and log comparison.

Several bot detection algorithms have recently been discussed in the literature, as software bots that perform maintenance tasks have become more popular in recent years. State-of-the-art techniques detect bots based on a binary classification, where a GitHub account is either a human or a bot. However, this conceptualisation of bot detection as an account-level binary classification problem fails to account for ‘mixed accounts’, accounts that are shared between a human and a bot, and that therefore exhibit both bot and human activity. By using binary classification models for bot detection, researchers might hence mischaracterize both human and bot behavior in software maintenance. This calls for conceptualisation of bot detection through a comment-level classification. However, the single such approach solely investigates a small number of mixed account comments. The nature of mixed accounts on GitHub is thus yet unknown, and the absence of appropriate datasets make this a difficult problem to study. In this paper, we investigate three comment-level classification models and we evaluate these classifiers on a manually labeled dataset of mixed accounts. We find that the best classifiers based on these classification models achieve a precision and recall between 88% and 96%. However, even the most accurate comment-level classifier cannot accurately detect mixed accounts; rather, we find that textual content alone, or textual content combined with templates used by bots, are very effective features for the detection of both bot and mixed accounts. Our study calls for more accurate bot detection techniques capable of identifying mixed accounts, and as such supporting more refined insights in software maintenance activities performed by humans and bots on social coding sites.

Self-Admitted Technical Debt (SATD) represents the admission, made through source code comments or other channels, of portions of a program being poorly implemented, containing provisional solutions or, in general, simply being not ready yet. To better understand developers' habits in SATD annotation, and possibly support their exploitation in tool support, this paper provides an in-depth analysis of the content provided in SATD comments, and the expressed sentiment. We manually inspect and classify 1038 instances from an existing dataset, grouping them along a taxonomy composed of 41 categories (of which 9 top-level ones), identifying their sentiment, and the presence of external references such as author names or issue IDs. Results of our study indicate that (i) the SATD content is crosscutting along life-cycle dimensions identified in previous work, (ii) comments related to functional problems or on-hold SATD are generally more negative than poor implementation choices or partially implemented functionality, and (iii) despite observations from previous literature, only a minority of SATD comments leverage external references.

Abstract Context Code review is a widely used technique of systematic examination of code changes which aims at increasing software quality. Code reviews provide several benefits for the project, including finding bugs, knowledge transfer, and assurance of adherence to project guidelines and coding style. However, code reviews have a major cost: they can delay the merge of the code change, and thus, impact the overall development process. This cost can be even higher if developers do not understand something, i.e., when developers faceconfusionduring the code review. Objective This paper studies the phenomenon ofconfusionin code reviews. Understanding confusion is an important starting point to help reducing the cost of code reviews and enhance the effectiveness of this practice, and hence, improve the development process. Method We conducted two complementary studies. The first one aimed at identifying the reasons for confusion in code reviews, its impacts, and the coping strategies developers use to deal with it. Then, we surveyed developers to identify the most frequently experienced reasons for confusion, and conducted a systematic mapping study of solutions proposed for those reasons in the scientific literature. Results From the first study, we build a framework with 30 reasons for confusion, 14 impacts, and 13 coping strategies. The results of the systematic mapping study shows 38 articles addressing the most frequent reasons for confusion. From those articles, we found 13 different solutions for confusion proposed in the literature, and five impacts were established related to the most frequent reasons for confusion. Conclusions Based on the solutions identified in the mapping study, or the lack of them, we propose an actionable guideline for developers on how to cope with confusion during code reviews; we also make several suggestions how tool builders can support code reviews. Additionally, we propose a research agenda for researchers studying code reviews.

Abstract Sentiment analysis methods have become popular for investigating human communication, including discussions related to software projects. Since general-purpose sentiment analysis tools do not fit well with the information exchanged by software developers, new tools, specific for software engineering (SE), have been developed. We investigate to what extent off-the-shelf SE-specific tools for sentiment analysis mitigate the threats to conclusion validity of empirical studies in software engineering, highlighted by previous research. First, we replicate two studies addressing the role of sentiment in security discussions on GitHub and in question-writing on Stack Overflow. Then, we extend the previous studies by assessing to what extent the tools agree with each other and with the manual annotation on a gold standard of 600 documents. We find that different SE-specific sentiment analysis tools might lead to contradictory results at a fine-grain level, when used off-the-shelf. Conversely, platform-specific tuning or retraining might be needed to take into account differences in platform conventions, jargon, or document lengths.

Abstract Context Models, as the main artifact in model-driven engineering, have been extensively used in the area of embedded systems for code generation and verification. One of the most popular behavioral modeling techniques is the state machine. Many state machine modeling guidelines recommend that a state machine should have more than one state in order to be meaningful. However, single-state state machines (SSSMs) violating this recommendation have been used in modeling cases reported in the literature. Objective We aim for understanding the phenomenon of using SSSMs in practice as understanding why developers violate the modeling guidelines is the first step towards improvement of modeling tools and practice. Method To study the phenomenon, we conducted an exploratory study which consists of two complementary studies. The first study investigated the prevalence and role of SSSMs in the domain of embedded systems, as well as the reasons why developers use them and their perceived advantages and disadvantages. We employed the sequential explanatory strategy, including repository mining and interview, to study 1500 state machines from 26 components at ASML, a leading company in manufacturing lithography machines from the semiconductor industry. In the second study, we investigated the evolutionary aspects of SSSMs, exploring when SSSMs are introduced to the systems and how developers modify them by mining the largest state-machine-based component from the company. Results We observe that 25 out of 26 components contain SSSMs. Our interviews suggest that SSSMs are used to interface with the existing code, to deal with tool limitations, to facilitate maintenance and to ease verification. Our study on the evolutionary aspects of SSSMs reveals that the need for SSSMs to deal with tool limitations grew continuously over the years. Moreover, only a minority of SSSMs have been changed between SSSM and multiple-state state machine (MSSM) during their evolution. The most frequent modifications developers made to SSSMs is inserting events with constraints on the execution of the events. Conclusions Based on our results, we provide implications for developers and tool builders. Furthermore, we formulate hypotheses about the effectiveness of SSSMs, the impacts of SSSMs on development, maintenance and verification as well as the evolution of SSSMs.

Abstract Objective The goal of this study is to identify gaps and challenges related to cross-domain model management focusing on consistency checking. Method We conducted a systematic literature review. We used the keyword-based search on Google Scholar, and we identified 618 potentially relevant studies; after applying inclusion and exclusion criteria, 96 papers were selected for further analysis. Results The main findings/contributions are: (i) a list of available tools used to support model management; (ii) 40% of the tools can provide consistency checking on models of different domains and 25% on models of the same domain, and 35% do not provide any consistency checking; (iii) available strategies to keep the consistency between models of different domains are not mature enough; (iv) most of the tools that provide consistency checking on models of different domains can only capture up to two inconsistency types; (v) the main challenges associated with tools that manage models on different domains are related to interoperability between tools and the consistency maintenance . Conclusion The results presented in this study can be used to guide new research on maintaining the consistency between models of different domains. Example of further research is to investigate how to capture the Behavioral and Refinement inconsistency types. This study also indicates that the tools should be improved in order to address, for example, more kinds of consistency check.

Code smells are poor implementation choices applied by developers during software evolution that often lead to critical flaws or failure. Much in the same way, community smells reflect the presence of organizational and socio-technical issues within a software community that may lead to additional project costs. Recent empirical studies provide evidence that community smells are often-if not always-connected to circumstances such as code smells. In this paper we look deeper into this connection by conducting a mixed-methods empirical study of 117 releases from 9 open-source systems. The qualitative and quantitative sides of our mixed-methods study were run in parallel and assume a mutually-confirmative connotation. On the one hand, we survey 162 developers of the 9 considered systems to investigate whether developers perceive relationship between community smells and the code smells found in those projects. On the other hand, we perform a fine-grained analysis into the 117 releases of our dataset to measure the extent to which community smells impact code smell intensity (i.e., criticality). We then propose a code smell intensity prediction model that relies on both technical and community-related aspects. The results of both sides of our mixed-methods study lead to one conclusion: community-related factors contribute to the intensity of code smells. This conclusion supports the joint use of community and code smells detection as a mechanism for the joint management of technical and social problems around software development communities.

Background. The effect of gender diversity in open source communities has gained increasing attention from practitioners and researchers. For instance, organizations such as the Python Software Foundation and the OpenStack Foundation started actions to increase gender diversity and promote women to top positions in the communities. Problem. Although the general underrepresentation of women (a.k.a. horizontal segregation) in open source communities has been explored in a number of research studies, little is known about the vertical segregation in open source communities---which occurs when there are fewer women in high level positions. Aims. To address this research gap, in this paper we present the results of a mixed-methods study on gender diversity and work practices of core developers contributing to open-source communities. Method. In the first study, we used mining-software repositories procedures to identify the core developers of 711 open source projects, in order to understand how common are women core developers in open source communities and characterize their work practices. In the second study, we surveyed the women core developers we identified in the first study to collect their perceptions of gender diversity and gender bias they might have observed while contributing to open source systems. Results. Our findings show that open source communities present both horizontal and vertical segregation (only 2.3% of the core developers are women). Nevertheless, differently from previous studies, most of the women core developers (65.7%) report never having experienced gender discrimination when contributing to an open source project. Finally, we did not note substantial differences between the work practices among women and men core developers. Conclusions. We reflect on these findings and present some ideas that might increase the participation of women in open source communities.

Self-Admitted Technical Debt (SATD) are comments, left by developers in the source code or elsewhere, aimed at describing the presence of TD, i.e., source code "not ready yet". Although this was never stated in the original paper by Potdar and Shihab, the term SATD might suggest that it refers to a "self-admission" by whoever has written or changed the source code. This paper empirically investigates, using a curated SATD dataset from five Java open-source projects, (i) the extent to which SATD comments are introduced by authors different from those who have done last changes to the related source code, and (ii) when this happens, what is the level of ownership those developers have about the commented source code. Results of the study indicate that, depending on the project, the percentage of SATD admissions introduced or changed without modifying the related source code varies between 0% and 16%, and therefore represent a small, yet not negligible, phenomenon. The level of ownership of those developers is not particularly low, with a median value per project between 10% and 42%. This indicates the possible use of SATD as a different way to perform code review, although this behavior should be considered sub-optimal to the use of more traditional tools, which entail suitable notification mechanisms.

The process of developing complex systems often involves knowledge of engineers from multiple domains: e.g., to develop a robot one needs to combine expertise about mechanics, electronics, and software. Such domain-specific knowledge is often represented in a form of interdependent models, consequently a change in a model of one domain might impact a model from a different domain. Thus, identifying which models are affected due to a change is an important problem, which is further exacerbated due to heterogeneity of modeling notations used.The aim of this PhD research project is to facilitate model management in a multi-domain setting. In the earlier stage of this study, we investigated the available approaches used to manage models from different domains. We concluded that the available approaches are tool-dependent, and do not fully support co-evolution of the models. Additionally, previous research recommends to explicitly indicate the dependency between models in order to support the co-evolution of models from different domains. Since these models are created using different modeling notations we believe that it is not reasonable to develop a tool to parse every notation. Furthermore, it is possible that the source code of the model is missing, but engineers still have an image of the model. Thus, to ensure the maintenance of multi-domain systems we investigated the suitability of optical character recognition (OCR) as a uniform approach. We observed that even though OCR has shortcomings, it produces satisfactory results, and once the identified shortcomings are addressed, OCR can become a crucial technology to support the evolution of multi-domain systems. To this end we envision the development of an infrastructure where we can use OCR to identify relationships between models from different domains, store them in a structured manner making it easier to maintain the consistency of the entire system.

Software bots, which are widely adopted by Open Source Software (OSS) projects, support developers on several activities, including code review. However, as with any new technology adoption, bots may impact group dynamics. Since understanding and anticipating such effects is important for planning and management, we investigate how several activity indicators change after the adoption of a code review bot. We employed a regression discontinuity design on 1,194 software projects from GitHub. Our results indicate that the adoption of code review bots increases the number of monthly merged pull requests, decreases monthly non-merged pull requests, and decreases communication among developers. Practitioners and maintainers may leverage our results to understand, or even predict, bot effects on their projects' social interactions.

Pull requests (PRs) selection is a challenging task faced by integrators in pull-based development (PbD), with hundreds of PRs submitted on a daily basis to large open-source projects. Managing these PRs manually consumes integrators' time and resources and may lead to delays in the acceptance, response, or rejection of PRs that can propose bug fixes or feature enhancements. On the one hand, well-known platforms for performing PbD, like GitHub, do not provide built-in recommendation mechanisms for facilitating the management of PRs. On the other hand, prior research on PRs recommendation has focused on the likelihood of either a PR being accepted or receive a response by the integrator. In this paper, we consider both those likelihoods, this to help integrators in the PRs selection process by suggesting to them the appropriate actions to undertake on each specific PR. To this aim, we propose an approach, called CARTESIAN (aCceptance And Response classificaTion-based requESt IdentificAtioN) modeling the PRs recommendation according to PR actions. In particular, CARTESIAN is able to recommend three types of PR actions: accept, respond, and reject. We evaluated CARTESIAN on the PRs of 19 popular GitHub projects. The results of our study demonstrate that our approach can identify PR actions with an average precision and recall of about 86%. Moreover, our findings also highlight that CARTESIAN outperforms the results of two baseline approaches in the task of PRs selection.

Models, as the main artifact in model-driven engineering, have been extensively used in the area of embedded systems for code generation and verification. One of the most popular behavioral modeling techniques is state machine. Many state machine modeling guidelines recommend that a state machine should have more than one state in order to be meaningful. However, single-state state machines (SSSMs) violating this recommendation have been used in modeling cases reported in the literature.

Static analysis tools are useful to detect common programming errors. However, they generate a large number of false positives. Postprocessing of these alarms using a model checker has been proposed to automatically eliminate false positives from them. To scale up the automated false positives elimination (AFPE), several techniques, e.g., program slicing, are used. However, these techniques increase the time taken by AFPE, and the increased time is a major concern during application of AFPE to alarms generated on large systems.To reduce the time taken by AFPE, we propose two techniques. The techniques achieve the reduction by identifying and skipping redundant calls to the slicer and model checker. The first technique is based on our observation that, (a) combination of application-level slicing, verification with incremental context, and the context-level slicing helps to eliminate more false positives; (b) however, doing so can result in redundant calls to the slicer. In this technique, we use data dependencies to compute these redundant calls. The second technique is based on our observation that (a) code partitioning is commonly used by static analysis tools to analyze very large systems, and (b) applying AFPE to alarms generated on partitioned-code can result in repeated calls to both the slicer and model checker. We use memoization to identify the repeated calls and skip them.The first technique is currently under evaluation. Our initial evaluation of the second technique indicates that it reduces AFPE time by up to 56%, with median reduction of 12.15%.

The adoption of Continuous Integration (CI) has been shown multiple benefits for software engineering practices related to build, test and dependency management. However, the impact of CI on the social aspects of software development has been overlooked so far. Specifically, we focus on studying the impact of CI on a paradigmatic socio-technical activity within the software engineering domain, namely code reviews. Indeed, one might expect that the introduction of CI allows reviewers to focus on more challenging aspects of software quality that could not be assessed using CI. To assess validity of this expectation we conduct an exploratory study of code reviews in 685 GitHub projects that have adopted Travis-CI, the most popular CI-service on GitHub. We observe that with the introduction of CI, pull requests are being discussed less. On average CI saves up to one review comment per pull request. This decrease in amount of discussion, however, cannot be explained by the decrease in the number of updates of the pull requests. This means that in presence of CI developers perform the same amount of work by communicating less, giving rise to the idea of CI as a silent helper.

Exception handling is a feature provided by most mainstream programming languages, and typically involves constructs to throw and handle error signals. On the one hand, early work has argued extensively about the benefits of exception handling, such as promoting modularity by defining how exception handlers can be implemented and maintained independently of the normal behavior of the system and easing but localization. On the other hand, some studies argue that exception handling can make the programming languages unnecessarily complex and promote the introduction of subtle bugs in programs. In 2015 we published a paper describing a study investigating the prevalence and nature of exception handling bugs in two large, widely adopted Java systems. This study also confronted its findings about real exception handling bugs with the perceptions of developers about those bugs, also accounting for bugs not related to exception handling. The goal of this reflection paper is to investigate the state of the art in exception handling research, with a particular emphasis on exception handling bugs, and how our paper has influenced other studies in the area. We found that our paper was cited by 33 articles, and all themes for future work we raised in our paper have been tackled by other studies in the short span of five years.

Technical Debt (TD) expresses the need for improvements in a software system, e.g., to its source code or architecture. In certain circumstances, developers “self-admit” technical debt (SATD) in their source code comments. Previous studies investigate when SATD is admitted, and what changes developers perform to remove it. Building on these studies, we present a first step towards the automated recommendation of SATD removal strategies. By leveraging a curated dataset of SATD removal patterns, we build a multi-level classifier capable of recommending six SATD removal strategies, e.g., changing API calls, conditionals, method signatures, exception handling, return statements, or telling that a more complex change is needed. SARDELE (SAtd Removal using DEep LEarning) combines a convolutional neural network trained on embeddings extracted from the SATD comments with a recurrent neural network trained on embeddings extracted from the SATD-affected source code. Our evaluation reveals that SARDELE is able to predict the type of change to be applied with an average precision of ~55%, recall of ~ 57%, and AUC of 0.73, reaching up to 73% precision, 63% recall, and 0.74 AUC for certain categories such as changes to method calls. Overall, results suggest that SATD removal follows recurrent patterns and indicate the feasibility of supporting developers in this task with automated recommenders.

Abstract When identifying the origin of software bugs, many studies assume that “a bug was introduced by the lines of code that were modified to fix it”. However, this assumption does not always hold and at least in some cases, these modified lines are not responsible for introducing the bug. For example, when the bug was caused by a change in an external API. The lack of empirical evidence makes it impossible to assess how important these cases are and therefore, to which extent the assumption is valid. To advance in this direction, and better understand how bugs “are born”, we propose a model for defining criteria to identify the first snapshot of an evolving software system that exhibits a bug. This model, based on the perfect test idea, decides whether a bug is observed after a change to the software. Furthermore, we studied the model’s criteria by carefully analyzing how 116 bugs were introduced in two different open source software projects. The manual analysis helped classify the root cause of those bugs and created manually curated datasets with bug-introducing changes and with bugs that were not introduced by any change in the source code. Finally, we used these datasets to evaluate the performance of four existing SZZ-based algorithms for detecting bug-introducing changes. We found that SZZ-based algorithms are not very accurate, especially when multiple commits are found; the F-Score varies from 0.44 to 0.77, while the percentage of true positives does not exceed 63%. Our results show empirical evidence that the prevalent assumption, “a bug was introduced by the lines of code that were modified to fix it”, is just one case of how bugs are introduced in a software system. Finding what introduced a bug is not trivial: bugs can be introduced by the developers and be in the code, or be created irrespective of the code. Thus, further research towards a better understanding of the origin of bugs in software projects could help to improve design integration tests and to design other procedures to make software development more robust.

Background: Evolution of open source projects frequently depends on a small number of core developers. The loss of such core developers might be detrimental for projects and even threaten their entire continuation. However, it is possible that new core developers assume the project maintenance and allow the project to survive. Aims: The objective of this paper is to provide empirical evidence on: 1) the frequency of project abandonment and survival, 2) the differences between abandoned and surviving projects, and 3) the motivation and difficulties faced when assuming an abandoned project. Method: We adopt a mixed-methods approach to investigate project abandonment and survival. We carefully select 1,932 popular GitHub projects and recover the abandoned and surviving projects, and conduct a survey with developers that have been instrumental in the survival of the projects. Results: We found that 315 projects (16%) were abandoned and 128 of these projects (41%) survived because of new core developers who assumed the project development. The survey indicates that (i) in most cases the new maintainers were aware of the project abandonment risks when they started to contribute; (ii) their own usage of the systems is the main motivation to contribute to such projects; (iii) human and social factors played a key role when making these contributions; and (iv) lack of time and the difficulty to obtain push access to the repositories are the main barriers faced by them. Conclusions: Project abandonment is a reality even in large open source projects and our work enables a better understanding of such risks, as well as highlights ways in avoiding them.

Sustained participation by contributors in opensource software is critical to the survival of open-source projects and can provide career advancement benefits to individual contributors. However, not all contributors reap the benefits of open-source participation fully, with prior work showing that women are particularly underrepresented and at higher risk of disengagement. While many barriers to participation in open-source have been documented in the literature, relatively little is known about how the social networks that open-source contributors form impact their chances of long-term engagement. In this paper we report on a mixed-methods empirical study of the role of social capital (i.e., the resources people can gain from their social connections) for sustained participation by women and men in open-source GitHub projects. After combining survival analysis on a large, longitudinal data set with insights derived from a user survey, we confirm that while social capital is beneficial for prolonged engagement for both genders, women are at disadvantage in teams lacking diversity in expertise.

The benefits of modeling the design to improve the quality and maintainability of software systems have long been advocated and recognized. Yet, the empirical evidence on this remains scarce. In this paper, we fill this gap by reporting on an empirical study of the relationship between UML modeling and software defect proneness in a large sample of open-source GitHub projects. Using statistical modeling, and controlling for confounding variables, we show that projects containing traces of UML models in their repositories experience, on average, a statistically minorly different number of software defects (as mined from their issue trackers) than projects without traces of UML models.

Eclipse framework provides two interfaces: stable interfaces (APIs) and unstable interfaces (non-APIs). Despite the non-APIs being discouraged and unsupported, their usage is not uncommon. Previous studies showed that applications using relatively old non-APIs are more likely to be compatible with new releases compared to the ones that used newly introduced non-APIs; that the growth rate of non-APIs is nearly twice as much as that of APIs; and that the promotion of non-API to APIs happens at a slow pace since API providers have no assistance to identify public interface candidates. Motivated by these findings, our main aim was to empirically investigate the entire population (2,380K) of non-APIs to find the non-APIs that remain stable for a long period of time. We employ cross-project clone detection to identify whether nonAPIs introduced in a given Eclipse release remain stable over successive releases. We provide a dataset of 327K stable nonAPI methods that can be used by both Eclipse interface providers as possible candidates of promotion. Instead of promoting nonAPIs which are too fine-grained, we summarized the non-API methods groups in given classes that are stable together and present class-level non-APIs that possible candidates promotion. We have shown that it is possible to predict the stability of a non-API in subsequent Eclipse releases with a precision of ≥ 56%, a recall of ≥ 96% and an AUC of ≥ 92% and an F-measure of ≥ 81%. We have also shown that the metrics of length of a method and number of method parameters in a nonAPI method are very good predictors for the stability of the non-API in successive Eclipse releases. The results provided can help the API providers to estimate a priori how much work could be involved in performing the promotion.

Code review is a software quality assurance practice widely employed in both open source and commercial software projects to detect defects, transfer knowledge and encourage adherence to coding standards. Notwithstanding, code reviews can also delay the incorporation of a code change into a code base, thus slowing down the overall development process. Part of this delay is often a consequence of reviewers not understanding, becoming confused by, or being uncertain about the intention, behavior, or effect of a code change.We investigate the reasons and impacts of confusion in code reviews, as well as the strategies developers adopt to cope with confusion. We employ a concurrent triangulation strategy to combine the analyses of survey responses and of the code review comments, and build a comprehensive confusion framework structured along the dimensions of the review process, the artifact being reviewed, the developers themselves and the relation between the developer and the artifact. The most frequent reasons for confusion are the missing rationale, discussion of non-functional requirements of the solution, and lack of familiarity with existing code. Developers report that confusion delays the merge decision, decreases review quality, and results in additional discussions. To cope with confusion developers request information, improve familiarity with existing code, and discuss off-line. Based on the results, we provide a series of implications for tool builders, as well as insights and suggestions for researchers. The results of our work offer empirical justification for the need to improve code review tools to support developers facing confusion.

Inferring behavioral models (e.g., state machines) of software systems is an important element of re-engineering activities. Model inference techniques can be categorized as active or passive learning, constructing models by (dynamically) interacting with systems or (statically) analyzing traces, respectively. Application of those techniques in the industry is, however, hindered by the trade-off between learning time and completeness achieved (active learning) or by incomplete input logs (passive learning). We investigate the learning time/completeness achieved trade-off of active learning with a pilot study at ASML, provider of lithography systems for the semiconductor industry. To resolve the trade-off we advocate extending active learning with execution logs and passive learning results. We apply the extended approach to eighteen components used in ASML TWINSCAN lithography machines. Compared to traditional active learning, our approach significantly reduces the active learning time. Moreover, it is capable of learning the behavior missed by the traditional active learning approach.

Model-driven engineering (MDE) enables the rise in abstraction during development in software and system design. In particular, meta-models become a central artifact in the process, and are supported by various other artifacts such as editors and transformation. In order to define constraints, invariants, and queries on model-driven artifacts, a generic language has been developed: the Object Constraint Language (OCL). In literature, many studies into OCL have been performed on small collections of data, mostly originating from a single source (e.g., OMG standards). As such, generalization of results beyond the data studied is often mentioned as a threat to validity. Creation of a benchmark dataset has already been identified as a key enabler to address the generalization threat. To facilitate further empirical studies in the field of OCL, we present the first large-scale dataset of 103262 OCL expression, systematically extracted from 671 GitHub repositories. In particular, our dataset has extracted these expressions from various types of files (a.o. metamodels and model-to-text transformations). In this work we showcase a variety of different studies performed using our dataset, and describe several other types that could be performed. We extend previous work with data and experiments regarding OCL in model-to-text (mtl) transformations.

“There can be no vulnerability without risk; there can be no community without vulnerability; there can be no peace, and ultimately no life, without community.” - [M. Scott Peck] The open-source phenomenon has reached the point in which it is virtually impossible to find large applications that do not rely on it. Such grand adoption may turn into a risk if the community regulatory aspects behind open-source work (e.g., contribution guidelines or release schemas) are left implicit and their effect untracked. We advocate the explicit study and automated support of such aspects and propose Yoshi ( Y ielding O pen- S ource H ealth I nformation), a tool able to map open-source communities onto community patterns , sets of known organisational and social structure types and characteristics with measurable core attributes. This mapping is beneficial since it allows, for example, (a) further investigation of community health measuring established characteristics from organisations research, (b) reuse of pattern-specific best-practices from the same literature, and (c) diagnosis of organisational anti-patterns specific to open-source, if any. We evaluate the tool in a quantitative empirical study involving 25 open-source communities from GitHub, finding that the tool offers a valuable basis to monitor key community traits behind open-source development and may form an effective combination with web-portals such as OpenHub or Bitergia. We made the proposed tool open source and publicly available.

Background Agile methods have been shown to have a negative impact on security. Several studies have investigated challenges in aligning security practices with agile methods, however, none of these have examined security challenges in the context of large-scale agile. Large-scale agile can present unique challenges, as large organizations often involve highly interdependent teams that need to align with other (non-agile) departments. Goal Our objective is to identify security challenges encountered in large-scale agile software development from the perspective of agile practitioners. Method Cooperative Method Development is applied to guide a qualitative case study at Rabobank, a Dutch multinational banking organization. A total of ten interviews is conducted with members in different agile roles from five different agile development teams. Data saturation has been obtained. By open card sorting we identify challenges pertaining to security in agile. Results The following challenges appear to be unique to large-scale agile: alignment of security objectives in a distributed setting, developing a common understanding of the roles and responsibilities in security activities, and integration of low-overhead security testing tools. Additional challenges reported appear to be common to security in software development in general or concur with challenges reported for small-scale agile. Conclusions The reported findings suggest the presence of multiple security challenges unique to large-scale agile. Future work should focus on confirming these challenges and investigating possible mitigations.

Background: Many studies in the software research literature on bug fixing are built upon the assumption that "a given bug was introduced by the lines of code that were modified to fix it", or variations of it. Although this assumption seems very reasonable at first glance, there is little empirical evidence supporting it. A careful examination surfaces that there are other possible sources for the introduction of bugs such as modifications to those lines that happened before the last change an changes external to the piece of code being fixed. Goal: We aim at understanding the complex phenomenon of bug introduction and bug fix. Method: We design a preliminary approach distinguishing between bug introducing commits (BIC) and first failing moments (FFM). We apply this approach to Nova and ElasticSearch, two large and well-known open source software projects. Results: In our initial results we obtain that at least 24% bug fixes in Nova and 10% in ElasticSearch have not been caused by a BIC but by co-evolution, compatibility issues or bugs in external API. Merely 26--29% of BICs can be found using the algorithm based on the assumption that "a given bug was introduced by the lines of code that were modified to fix it". Conclusions: The approach allows also for a better framing of the comparison of automatic methods to find bug inducting changes. Our results indicate that more attention should be paid to whether a bug has been introduced and, when it was introduced.

Stack Overflow (SO) is the reference for asking and answering programming-related questions. In early 2014 Stack Overflow em Português (SO-PT) was announced with the goal to reach developers that are not sufficiently proficient in the English language to fully participate in SO. Almost four years later we study how the simultaneous availability of SO and SO-PT impacted Brazilian software developers. A priori, the impact could have been either empowering or impeding. To address this question, we combine interviews, analysis of trace data from SO and SO-PT and a survey of 229 Brazilian software developers. Our results indicate that the developers recognize availability of the information, response speed and accessibility as strong points of SO, and lower barrier to entry and presence of Brazilian-specific information as strong points of SO-PT. In large, SO remains more popular than SO-PT, and SO-PT is not perceived as a viable alternative to SO.

During code review, developers request clarifications, suggest improvements, or ask for explanations about the rationale behind the implementation choices. We envision the emergence of tools to support developers during code review based on the automatic analysis of the argumentation structure and communicative intentions conveyed by developers' comments. As a preliminary step towards this goal, we conducted an exploratory case study by manually classifying 499 questions extracted from 399 Android code reviews to understand the real communicative intentions they convey. We observed that the majority of questions actually serve information seeking goals. Still, they represent less than half of the annotated sample, with other questions being used to serve a wider variety of developers' communication goals, including suggestions, request for action, and criticism. Based on our findings we formulate hypotheses on communicative intentions in code reviews that should be confirmed or rejected by follow-up studies.

The large number of alarms reported by static analysis tools is often recognized as one of the major obstacles to industrial adoption of such tools.

Model-driven engineering is used in the design of systems to (a.o.) enable analysis early in the design process. For instance, by using domain-specific languages, enabling engineers to model systems in terms of their domain, rather then encoding them into general purpose modeling languages. Domain-specific languages, like classical software, evolve over time. When domain languages evolve, they may trigger co-evolution of models, model-to-model transformations, editors (both graphical and textual), and other artifacts that depend on the domain-specific language. This co-evolution can be tedious and very costly. In literature, various approaches are proposed towards automated co-evolution. However, these approaches do not reach full automation. Several other studies have shown that there are theoretical limitations to the level of automation that can be achieved in certain scenarios. For several scenarios full automation can never be achieved. We wish to gain insight to which extent practically occurring scenarios can be automated. To gain this insight, in this paper, we investigate on a large-scale industrial repository, which (co-)evolutionary scenarios occur in practice, and compare them with the various scenarios and their theoretical automatability. We then assess whether practically occurring scenarios can be fully automated.

Swift is a new programming language developed by Apple as a replacement to Objective-C. It features a sophisticated error handling (EH) mechanism that provides the kind of separation of concerns afforded by exception handling mechanisms in other languages, while also including constructs to improve safety and maintainability. However, Swift also inherits a software development culture stemming from Objective-C being the de-facto standard programming language for Apple platforms for the last 15 years. It is, therefore, a priori unclear whether Swift developers embrace the novel EH mechanisms of the programming language or still rely on the old EH culture of Objective-C even working in Swift.

Technical Debt (TD) has been defined as "code being not quite right yet", and its presence is often self-admitted by developers through comments. The purpose of such comments is to keep track of TD and appropriately address it when possible. Building on a previous quantitative investigation by Maldonado et al. on the removal of self-admitted technical debt (SATD), in this paper we perform an in-depth quantitative and qualitative study of how SATD is addressed in five Java open source projects. On the one hand, we look at whether SATD is "accidentally" removed, and the extent to which the SATD removal is being documented. We found that that (i) between 20% and 50% of SATD comments are accidentally removed while entire classes or methods are dropped, (ii) 8% of the SATD removal is acknowledged in commit messages, and (iii) while most of the changes addressing SATD require complex source code changes, very often SATD is addressed by specific changes to method calls or conditionals. Our results can be used to better plan TD management or learn patterns for addressing certain kinds of TD and provide recommendations to developers.

Code smells are symptoms of poor design and implementation choices weighing heavily on the quality of produced source code. During the last decades several code smell detection tools have been proposed. However, the literature shows that the results of these tools can be subjective and are intrinsically tied to the nature and approach of the detection. In a recent work the use of Machine-Learning (ML) techniques for code smell detection has been proposed, possibly solving the issue of tool subjectivity giving to a learner the ability to discern between smelly and non-smelly source code elements. While this work opened a new perspective for code smell detection, it only considered the case where instances affected by a single type smell are contained in each dataset used to train and test the machine learners. In this work we replicate the study with a different dataset configuration containing instances of more than one type of smell. The results reveal that with this configuration the machine learning techniques reveal critical limitations in the state of the art which deserve further research.

The mining of software repositories has provided significant advances in a multitude of software engineering fields, including defect prediction. Several studies show that the performance of a software engineering technology (e.g., prediction model) differs across different project repositories. Thus, it is important that the project selection is replicable. The aim of this paper is to present STRESS, a semi-automated and fully replicable approach that allows researchers to select projects by configuring the desired level of diversity, fit, and quality. STRESS records the rationale behind the researcher decisions and allows different users to re-run or modify such decisions. STRESS is open-source and it can be used used locally or even online (www.falessi.com/STRESS/). We perform a systematic mapping study that considers studies that analyzed projects managed with JIRA and Git to asses the project selection replicability of past studies. We validate the feasible application of STRESS in realistic research scenarios by applying STRESS to select projects among the 211 Apache Software Foundation projects. Our systematic mapping study results show that none of the 68 analyzed studies is completely replicable. Regarding STRESS, it successfully supported the project selection among all 211 ASF projects. It also supported the measurement of 100 projects characteristics, including the 32 criteria of the studies analyzed in our mapping study. The mapping study and STRESS are, to our best knowledge, the first attempt to investigate and support the replicability of project selection. We plan to extend them to other technologies such as GitHub.

Large open source software projects often have a globally distributed development team. Studies have shown developer turnover has a significant impact on the project success. Frequent developer turnover may lead to loss of productivity due to lacking relevant knowledge and spending extra time learning how projects work. Thus, lots of attention has been paid to which factors are related to developer retention, however, few of them focus on the impact of activities of individual developers. In this paper, we study five open source projects from different organizations and examine whether developer turnover is affected by when they start contributing and what types of contributions they are making. Our study reveals that developers have higher chances to survive in software projects when they 1) start contributing to the project earlier, 2) mainly modify instead of creating files, 3) mainly code instead of dealing with documentations. Our results also shed lights on the potential approaches to improving developer retention.

The behavior of software that uses the Java Reflection API is fundamentally hard to predict by analyzing code. Only recent static analysis approaches can resolve reflection under unsound yet pragmatic assumptions. We survey what approaches exist and what their limitations are. We then analyze how real-world Java code uses the Reflection API, and how many Java projects contain code challenging state-of-the-art static analysis. Using a systematic literature review we collected and categorized all known methods of statically approximating reflective Java code. Next to this we constructed a representative corpus of Java systems and collected descriptive statistics of the usage of the Reflection API. We then applied an analysis on the abstract syntax trees of all source code to count code idioms which go beyond the limitation boundaries of static analysis approaches. The resulting data answers the research questions. The corpus, the tool and the results are openly available. We conclude that the need for unsound assumptions to resolve reflection is widely supported. In our corpus, reflection can not be ignored for 78% of the projects. Common challenges for analysis tools such as non-exceptional exceptions, programmatic filtering meta objects, semantics of collections, and dynamic proxies, widely occur in the corpus. For Java software engineers prioritizing on robustness, we list tactics to obtain more easy to analyze reflection code, and for static analysis tool builders we provide a list of opportunities to have significant impact on real Java code.

Code reviews are an important mechanism for assuring quality of source code changes. Reviewers can either add general comments pertaining to the entire change or pinpoint concerns or shortcomings about a specific part of the change using inline comments. Recent studies show that reviewers often do not understand the change being reviewed and its context.Our ultimate goal is to identify the factors that confuse code reviewers and understand how confusion impacts the efficiency and effectiveness of code review(er)s. As the first step towards this goal we focus on the identification of confusion in developers' comments. Based on an existing theoretical framework categorizing expressions of confusion, we manually classify 800 comments from code reviews of the Android project. We observe that confusion can be reasonably well-identified by humans: raters achieve moderate agreement (Fleiss' kappa 0.59 for the general comments and 0.49 for the inline ones). Then, for each kind of comment we build a series of automatic classifiers that, depending on the goals of the further analysis, can be trained to achieve high precision (0.875 for the general comments and 0.615 for the inline ones), high recall (0.944 for the general comments and 0.988 for the inline ones), or substantial precision and recall (0.696 and 0.542 for the general comments and 0.434 and 0.583 for the inline ones, respectively). These results motivate further research on the impact of confusion on the code review process. Moreover, other researchers can employ the proposed classifiers to analyze confusion in other contexts where software development-related discussions occur, such as mailing lists.

Technical debt refers to the phenomena of taking shortcuts to achieve short term gain at the cost of higher maintenance efforts in the future. Recently, approaches were developed to detect technical debt through code comments, referred to as Self-Admitted Technical Debt (SATD). Due to its importance, several studies have focused on the detection of SATD and examined its impact on software quality. However, preliminary findings showed that in some cases SATD may live in a project for a long time, i.e., more than 10 years. These findings clearly show that not all SATD may be regarded as 'bad' and some SATD needs to be removed, while other SATD may be fine to take on.Therefore, in this paper, we study the removal of SATD. In an empirical study on five open source projects, we examine how much SATD is removed and who removes SATD? We also investigate for how long SATD lives in a project and what activities lead to the removal of SATD? Our findings indicate that the majority of SATD is removed and that the majority is self-removed (i.e., removed by the same person that introduced it). Moreover, we find that SATD can last between approx. 18-172 days, on median. Finally, through a developer survey, we find that developers mostly use SATD to track future bugs and areas of the code that need improvements. Also, developers mostly remove SATD when they are fixing bugs or adding new features. Our findings contribute to the body of empirical evidence on SATD, in particular, evidence pertaining to its removal.

Career perspectives are known to motivate software engineers. However, so far, career perspectives have been mostly studied within traditional software development companies. In our work we take a complementary approach and study career paths of open source developers, focusing on their advancement through the code review hierarchy, from developers to reviewers and further to core reviewers.To gain understanding of code review career paths we conduct an exploratory case study of the OpenStack community. Based on the case study we have publicized anonymized research data and formulated four hypotheses pertaining to career paths of contributors in modern multi-company open source projects. We conjecture that(i) developers and reviewers are separate subpopulations with little movement between them, (ii-a) the turnover of the core reviewers is high and rapid, (ii-b) companies are interested in having core reviewers among their staff, and (iii) being a core reviewer is beneficial for career.Validity of those hypotheses in other multi-company open source projects should be investigated in the follow-up studies.

Over the past years, there has been an increase in the application of model driven engineering in industry. Similar to traditional software engineering, understanding how technologies are actually used in practice is essential for developing good tooling, and decision making processes. Unfortunately, obtaining and analyzing empirical data in a model-driven context is still tedious and time consuming, introducing large lead-times. In this paper we present a framework for the automated extraction, analysis, and visualization of data and metrics on model-driven artifacts. We subsequently present various examples of how the framework was successfully applied in a large industrial setting to answer a plethora of different questions with respect to decision making and tool development.

Continuous Integration (CI) has become a disruptive innovation in software development: with proper tool support and adoption, positive effects have been demonstrated for pull request throughput and scaling up of project sizes. As any other innovation, adopting CI implies adapting existing practices in order to take full advantage of its potential, and "best practices" to that end have been proposed. Here we study the adaptation and evolution of code writing and submission, issue and pull request closing, and testing practices as TRAVIS CI is adopted by hundreds of established projects on GITHUB. To help essentialize the quantitative results, we also survey a sample of GITHUB developers about their experiences with adopting TRAVIS CI. Our findings suggest a more nuanced picture of how GITHUB teams are adapting to, and benefiting from, continuous integration technology than suggested by prior work.

In model driven engineering (MDE), meta-models are the central artifacts. As a complement, the Object Constraint Language (OCL) is a language used to express constraints and operations on meta-models. The Eclipse Modeling Framework (EMF) provides an implementation of OCL, enabling OCL-annotated meta-models. Existing empirical studies of the OCL have been conducted on small collections of data. To facilitate empirical research into the OCL on a larger scale, we present the first publicly available data set of OCL expressions. The data set contains 9188 OCL expressions originating from 504 EMF meta-models in 245 systematically selected GitHub repositories. Both the original meta-models and the generated abstract syntax trees are included, allowing for a variety of empirical studies of the OCL. To illustrate the applicability of this data set in practice, we performed three case studies.

Research shows the importance of selecting good names to identifiers in software code: more meaningful names improve readability. In particular, several guidelines encourage long and descriptive variable names. A recent study analyzed the use of variable names in five programming languages, focusing on single-letter variable names, because of the apparent contradiction between their frequent use and the fact that these variables violate the aforementioned guidelines. In this paper, we analyze variables in Scratch, a popular block-based language aimed at children. We start by replicating the above single-letter study for Scratch. We augment this study by analyzing single-letter procedure names, and by investigating the use of Scratch specific naming patterns: spaces in variable names, numerics as variables and textual labels in procedure names. The results of our analysis show that Scratch programmers often prefer longer identifier names than developers in other languages, while Scratch procedure names have even longer names than Scratch variables. For the single-letter variables, the most frequent names are x, y, and i. Single-letter procedures are less popular, but show more tendency to be in upper case. When compared to the other programming languages, the usage of single uppercase letters in Scratch variables seems to be similar to the pattern found in Perl, while for the lowercase letters-to the pattern found in Java. Concerning Scratch specific features, 44% of the unique variable names and 34% of the projects in the dataset include at least one space. The usage of textual labels between parameters in procedure names appears as not common, however textual patterns used imply an influence from textual languages, for example by using brackets. Previous research indicate the identifier names as one significant issue in transitioning from visual block-based to textual programming languages. The naming patterns we found support this claim for Scratch programmers who may incur difficulties when transitioning to the use of mainstream textual programming languages. Those languages restrict the use of spaces in identifiers and more often divert into short and single-letter names- tendencies opposite to the naming preferences in Scratch.

Open source projects rely on collaboration of members from all around the world using web technologies like GitHub and Gerrit. This mixture of people with a wide range of backgrounds including minorities like women, ethnic minorities, and people with disabilities may increase the risk of offensive and destroying behaviours in the community, potentially leading affected project members to leave towards a more welcoming and friendly environment. To counter these effects, open source projects increasingly are turning to codes of conduct, in an attempt to promote their expectations and standards of ethical behaviour. In this first of its kind empirical study of codes of conduct in open source software projects, we investigated the role, scope and influence of codes of conduct through a mixture of quantitative and qualitative analysis, supported by interviews with practitioners. We found that the top codes of conduct are adopted by hundreds to thousands of projects, while all of them share 5 common dimensions.

Recent years have seen an increasing attention to social aspects of software engineering, including studies of emotions and sentiments experienced and expressed by the software developers. Most of these studies reuse existing sentiment analysis tools such as SentiStrength and NLTK. However, these tools have been trained on product reviews and movie reviews and, therefore, their results might not be applicable in the software engineering domain. In this paper we study whether the sentiment analysis tools agree with the sentiment recognized by human evaluators (as reported in an earlier study) as well as with each other. Furthermore, we evaluate the impact of the choice of a sentiment analysis tool on software engineering studies by conducting a simple study of differences in issue resolution times for positive, negative and neutral texts. We repeat the study for seven datasets (issue trackers and Stack Overflow questions) and different sentiment analysis tools and observe that the disagreement between the tools can lead to diverging conclusions. Finally, we perform two replications of previously published studies and observe that the results of those studies cannot be confirmed when a different sentiment analysis tool is used.

INTRODUCTION During the preparation of the corresponding chapter in Davy Landman's PhD thesis, some minor graphical and statistical discrepancies were found in the paper “Empirical analysis of the relationship between CC and SLOC in a large corpus of Java methods and C functions.” To support future reproduction and use of this work, we prepared the current erratum, containing several updated figures, a diagnosis of the cause of the errors, and an explanation of the effect on the original paper. None of the issues reported in this erratum influence the conclusions of the original paper. ISSUES DISCOVERED The hexagonal scatter plots in Figure lack a more prominent line at CC = 0. This was caused by a bug *reported and confirmed: https://github.com/tidyverse/ggplot2/issues/2061in ggplot, which would filter out data around the limits. The R2 values in the Tables B and B of the C corpus were off by a maximum of 0.01 from the actual result. The cause was that this table was not re-calculated after fixing a bug in the “remove out-of-scope code” phase. Note that the impact of this error is scattered throughout the paper, as the correlations of Tables and are often repeated for clarity in the remaining sections (for example, the R2 of the linear model for all the C functions is 0.43 instead of 0.44). Our R code calculating the log-transformed linear fit contained an error. The dashed lines in Figures, and are impacted and the shape of the residual plot in 11. The biggest impact is in Figure, where the original fit seemed to miss the data almost entirely. We misinterpreted this phenomenon in the last sentence of the second paragraph of section 4.4.2; it is not caused by the skewness of the distributions of the two metrics, but rather by the current bug. The custom implementation of the log-scaled y-axis of the residual plots in Figure contained two errors: ∘ The labels on the y-axis were off by a factor 10 ∘For the negative side of the residual plot, we took the absolute, calculated the log10 value, and made it negative again. However, values between 0 and 1 (the values close to the linear fit) turn into a negative value (as log10(1) equals 0). This caused strange outliers in the original plots that were not scrutinized. The fixed residual plots do not have this outliers and look much more like the data in Figure. We republished the data sets related to the current paper on Zenodo to increase their availability: ∘ Landman, Davy. (2015). A Curated Corpus of Java Source Code based on Sourcerer (2015) [Data set]. Zenodo. http://doi.org/10.5281/zenodo.208213 ∘Landman, Davy. (2015). A Large Corpus of C Source Code based on Gentoo packages [Data set]. Zenodo. http://doi.org/10.5281/zenodo.208215 ∘Davy Landman. (2015, February 26). cwi-swat/jsep-sloc-versus-cc. Zenodo. http://doi.org/10.5281/zenodo.293795 NEW IMAGES The remaining part of this erratum contains updated tables and figures as replacements for the original paper. 8 (Figure presented.) Scatter plots of SLOC vs CC zoomed in on the bottom left quadrant. The solid and dashed lines are the linear regression before and after the log transform. The grayscale gradient of the hexagons is logarithmic 4 Correlations for part of the tail of the independent variable SLOC. All correlations have a high significance level p≤1×10−16.(b) C functions (Table presented.) 5 Correlations for part of the tail of the independent variable SLOC removed. All correlations have a high significance level p≤1×10−16.(b) C functions (Table presented.) 9 (Figure presented.) Scatter plots of SLOC vs CC on a log-log scale. The solid and dashed lines are the linear regression before and after the log transform. The grayscale gradient of the hexagons is logarithmic 11 (Figure presented.) Residual plot of the linear regressions after the log transform, both axis are on a log scale. The grayscale gradient of the hexagons is logarithmic 12 (Figure presented.) Scatter plots of SLOC vs CC for Java and C files. The solid and dashed lines are the linear regression before and after the log transform. The grayscale gradient of the hexagons is logarithmic.

Domain specific languages (DSLs) ease the adoption of formal specification in industry. They allow developers to describe their specification models in concepts of their domain. However, DSLs evolve over time, causing specification models to have to co-evolve to reflect the evolution in the DSL. The maintenance overhead introduced by these, often manual, changes to specification models threatens to overshadow the advantages of DSL usage in industry. To this extent, many approaches have been proposed in the literature to facilitate DSL maintenance by automating model co-changes. In this paper, we evaluate the ability of a tool, Edapt, to support the change and co-change in twenty-two industrial DSLs and corresponding specification models over a maintenance period of four years. We observe that the tool is only able to automatically co-change specification models for 72% of the DSL changes. To address the remaining 28% of the changes, we extend Edapt. The resulting extension allows automatically co-changing specification models for 98% of the DSL changes.

Domain-specific languages (DSLs) allow users to model systems using concepts from a specific domain. Evolution of DSLs triggers co-evolution of models developed in these languages. Manual co-evolution of the thousands of models is unfeasible, calling for an automated support. A prerequisite to automating model co-evolution with respect to DSL evolution is the ability to formally specify DSL evolution, e.g., using predefined evolution operators. Success or failure of the practical application of the operator-based approach therefore depends heavily on the operators offered by the operator library at hand. In this paper we evaluate the completeness of the state-of-the-art operator library claimed to be "practically complete" (which we denote as H) by using it to specify evolution of an ecosystem of 22 commercial DSLs over the period of four years. We observe that 11% of the changes cannot be specified. However, there is no guarantee that extending the library with the identified deficiencies will be sufficient to specify evolution of other DSLs. To mitigate this, we design a theoretically complete library of operators, R. We observe that 77% of the operators from R are absent from H. Of the deficiencies in H, 72% could not be revealed by means of studying the extensive industrial ecosystem above. Our study suggests that the existing operator libraries are not extensive enough to specify evolution of large model-driven software ecosystems. Since extending operator libraries on a per-case study basis does not yield satisfactory results so far, we advocate an alternative, i.e. a theoretically complete library of operators R.

Software development remains a predominantly male activity, despite coordinated efforts from research, industry, and policy makers. This gender imbalance is most visible in social programming, on platforms such as Stack Overflow.

Static analysis tools have showcased their importance and usefulness in automated detection of code anomalies and defects. However, the large number of alarms reported and cost incurred in their manual inspections have been the major concerns with the usage of static analysis tools. Existing studies addressing these concerns differ greatly in their approaches to handle the alarms, varying from automatic postprocessing of alarms, supporting the tool-users during manual inspections of the alarms, to designing of light-weight static analysis tools. A comprehensive study of approaches for handling alarms is, however, not found. In this paper, we review 79 alarms handling studies collected through a systematic literature search and classify the approaches proposed into seven categories. The literature search is performed by combining the keywords-based database search and snowballing. Our review is intended to provide an overview of various alarms handling approaches, their merits and shortcomings, and different techniques used in their implementations. Our findings include that the categorized alarms handling approaches are complementary and they can be combined together in different ways. The categorized approaches and techniques employed in them can help the designers and developers of static analysis tools to make informed choices.

Utility functions are general purpose functions, which are useful in many parts of a system. To facilitate reuse, they are usually implemented in specific libraries. However, developers frequently miss opportunities to implement general-purpose functions in utility libraries, which decreases the chances of reuse. In this paper, we describe our ongoing investigation on using Random Forest classifiers to automatically identify utility functions. Using a list of static source code metrics we train a classifier to identify such functions, both in Java (using 84 projects from the Qualitas Corpus) and in JavaScript (using 22 popular projects from GitHub). We achieve the following median results for Java: 0.90 (AUC), 0.83 (precision), 0.88 (recall), and 0.84 (F-measure). For JavaScript, the median results are 0.80 (AUC), 0.75 (precision), 0.89 (recall), and 0.76 (F-measure).

Recently, Apple released Swift, a modern programming language built to be the successor of Objective-C. In less than a year and a half after its first release, Swift became one of the most popular programming languages in the world, considering different popularity measures. A significant part of this success is due to Apple's strict control over its ecosystem, and the clear message that it will replace Objective-C in a near future. According to Apple, "Swift is a powerful and intuitive programming language[...]. Writing Swift code is interactive and fun, the syntax is concise yet expressive." However, little is known about how Swift developers perceive these benefits. In this paper, we conducted two studies aimed at uncovering the questions and strains that arise from this early adoption. First, we perform a thorough analysis on 59,156 questions asked about Swift on StackOverflow. Second, we interviewed 12 Swift developers to cross-validate the initial results. Our study reveals that developers do seem to find the language easy to understand and adopt, although 17.5% of the questions are about basic elements of the language. Still, there are many questions about problems in the toolset (compiler, Xcode, libraries). Some of our interviewees reinforced these problems.

Abstract Measuring the internal quality of source code is one of the traditional goals of making software development into an engineering discipline. Cyclomatic complexity (CC) is an often used source code quality metric, next to source lines of code (SLOC). However, the use of the CC metric is challenged by the repeated claim that CC is redundant with respect to SLOC because of strong linear correlation. We conducted an extensive literature study of the CC/SLOC correlation results. Next, we tested correlation on large Java (17.6 M methods) and C (6.3 M functions) corpora. Our results show that linear correlation between SLOC and CC is only moderate as a result of increasingly high variance. We further observe that aggregating CC and SLOC as well as performing a power transform improves the correlation. Our conclusion is that the observed linear correlation between CC and SLOC of Java methods or C functions is not strong enough to conclude that CC is redundant with SLOC. This conclusion contradicts earlier claims from literature but concurs with the widely accepted practice of measuring of CC next to SLOC. Copyright © 2015 John Wiley & Sons, Ltd.

We investigate quality improvement in QVT operational mappings (QVTo) model transformations, one of the languages defined in the OMG standard on model-to-model transformations. Two research questions are addressed. First, how can we assess quality of QVTo model transformations? Second, how can we develop higher-quality QVTo transformations? To address the first question, we utilize a bottom–up approach, starting with a broad exploratory study including QVTo expert interviews, a review of existing material, and introspection. We then formalize QVTo transformation quality into a QVTo quality model. The quality model is validated through a survey of a broader group of QVTo developers. We find that although many quality properties recognized as important for QVTo do have counterparts in general purpose languages, a number of them are specific to QVTo or model transformation languages. To address the second research question, we leverage the quality model to identify developer support tooling for QVTo. We then implemented and evaluated one of the tools, namely a code test coverage tool. In designing the tool, code coverage criteria for QVTo model transformations are also identified. The primary contributions of this paper are a QVTo quality model relevant to QVTo practitioners and an open-source code coverage tool already usable by QVTo transformation developers. Secondary contributions are a bottom–up approach to building a quality model, a validation approach leveraging developer perceptions to evaluate quality properties, code test coverage criteria for QVTo, and numerous directions for future research and tooling related to QVTo quality.

Software development is usually a collaborative venture. Open Source Software (OSS) projects are no exception; indeed, by design, the OSS approach can accommodate teams that are more open, geographically distributed, and dynamic than commercial teams. This, we find, leads to OSS teams that are quite diverse. Team diversity, predominantly in offline groups, is known to correlate with team output, mostly with positive effects. How about in OSS? Using GitHub, the largest publicly available collection of OSS projects, we studied how gender and tenure diversity relate to team productivity and turnover. Using regression modeling of GitHub data and the results of a survey, we show that both gender and tenure diversity are positive and significant predictors of productivity, together explaining a sizable fraction of the data variability. These results can inform decision making on all levels, leading to better outcomes in recruiting and performance.

Recent years have seen an increasing attention to social aspects of software engineering, including studies of emotions and sentiments experienced and expressed by the software developers. Most of these studies reuse existing sentiment analysis tools such as SentiStrength and NLTK. However, these tools have been trained on product reviews and movie reviews and, therefore, their results might not be applicable in the software engineering domain. In this paper we study whether the sentiment analysis tools agree with the sentiment recognized by human evaluators (as reported in an earlier study) as well as with each other. Furthermore, we evaluate the impact of the choice of a sentiment analysis tool on software engineering studies by conducting a simple study of differences in issue resolution times for positive, negative and neutral texts. We repeat the study for seven datasets (issue trackers and Stack Overflow questions) and different sentiment analysis tools and observe that the disagreement between the tools can lead to contradictory conclusions.

Thresholds are essential for promoting source code metrics as an effective instrument to control the internal quality of software applications. However, little is known about the relation between software quality as identified by metric thresholds and as perceived by real developers. In this paper, we report the first results of a study designed to validate a technique that extracts relative metric thresholds from benchmark data. We use this technique to extract thresholds from a benchmark of 79 Pharo/Smalltalk applications, which are validated with five experts and 25 developers. Our preliminary results indicate that good quality applications - as cited by experts - respect metric thresholds. In contrast, we observed that noncompliant applications are not largely viewed as requiring more effort to maintain than other applications.

Like any other team oriented activity, the software development process is effected by social diversity in the programmer teams. The effect of team diversity can be significant, but also complex, especially in decentralized teams. Discerning the precise contribution of diversity on teams' effectiveness requires quantitative studies of large data sets. Here we present for the first time a large data set of social diversity attributes of programmers in GitHub teams. Using alias resolution, location data, and gender inference techniques, we collected a team social diversity data set of 23,493 GitHub projects. We illustrate how the data set can be used in practice with a series of case studies, and we hope its availability will foster more interest in studying diversity issues in software teams.

Historically, mailing lists have been the preferred means for coordinating development and user support activities. With the emergence and popularity growth of social Q&A sites such as the StackExchange network (e.g., StackOverflow), this is beginning to change. Such sites offer different socio-technical incentives to their participants than mailing lists do, e.g., rich web environments to store and manage content collaboratively, or a place to showcase their knowledge and expertise more vividly to peers or potential recruiters. A key difference between StackExchange and mailing lists is gamification, i.e., StackExchange participants compete to obtain reputation points and badges. In this paper, we use a case study of R (a widely-used tool for data analysis) to investigate how mailing list participation has evolved since the launch of StackExchange. Our main contribution is the assembly of a joint data set from the two sources, in which participants in both the texttt{r-help} mailing list and StackExchange are identifiable. This permits their activities to be linked across the two resources and also over time. With this data set we found that user support activities show a strong shift away from texttt{r-help}. In particular, mailing list experts are migrating to StackExchange, where their behaviour is different. First, participants active both on texttt{r-help} and on StackExchange are more active than those who focus exclusively on only one of the two. Second, they provide faster answers on StackExchange than on texttt{r-help}, suggesting they are motivated by the emph{gamified} environment. To our knowledge, our study is the first to directly chart the changes in behaviour of specific contributors as they migrate into gamified environments, and has important implications for knowledge management in software engineering.

Measuring the internal quality of source code is one of the traditional goals of making software development into an engineering discipline. Cyclomatic Complexity (CC) is an often used source code quality metric, next to Source Lines of Code (SLOC). However, the use of the CC metric is challenged by the repeated claim that CC is redundant with respect to SLOC due to strong linear correlation. We test this claim by studying a corpus of 17.8M methods in 13K open-source Java projects. Our results show that direct linear correlation between SLOC and CC is only moderate, as caused by high variance. We observe that aggregating CC and SLOC over larger units of code improves the correlation, which explains reported results of strong linear correlation in literature. We suggest that the primary cause of correlation is the aggregation. Our conclusion is that there is no strong linear correlation between CC and SLOC of Java methods, so we do not conclude that CC is redundant with SLOC. This conclusion contradicts earlier claims from literature, but concurs with the widely accepted practice of measuring of CC next to SLOC.

Meaningful thresholds are essential for promoting source code metrics as an effective instrument to control the internal quality of software systems. Despite the increasing number of source code measurement tools, no publicly available tools support extraction of metric thresholds. Moreover, earlier studies suggest that in larger systems significant number of classes exceed recommended metric thresholds. Therefore, in our previous study we have introduced the notion of a relative threshold, i.e., a pair including an upper limit and a percentage of classes whose metric values should not exceed this limit. In this paper we propose RTTOOL, an open source tool for extracting relative thresholds from the measurement data of a benchmark of software systems. RTTOOL is publicly available at http://aserg.labsoft.dcc.ufmg.br/rttool.

Continuous integration is a software engineering practice of frequently merging all developer working copies with a shared main branch, e.g., several times a day. With the advent of GitHub, a platform well known for its "social coding" features that aid collaboration and sharing, and currently the largest code host in the open source world, collaborative software development has never been more prominent. In GitHub development one can distinguish between two types of developer contributions to a project: direct ones, coming from a typically small group of developers with write access to the main project repository, and indirect ones, coming from developers who fork the main repository, update their copies locally, and submit pull requests for review and merger. In this paper we explore how GitHub developers use continuous integration as well as whether the contribution type (direct versus indirect) and different project characteristics (e.g., main programming language, or project age) are associated with the success of the automatic builds.

Software engineers share experiences with modern technologies by means of software information sites, such as Stack Overflow. These sites allow developers to label posted content, referred to as software objects, with short descriptions, known as tags. However, tags assigned to objects tend to be noisy and some objects are not well tagged. To improve the quality of tags in software information sites, we propose EnTagRec, an automatic tag recommender based on historical tag assignments to software objects and we evaluate its performance on four software information sites, Stack Overflow, Ask Ubuntu, Ask Different, and Free code. We observe that that EnTagRec achieves Recall@5 scores of 0.805, 0.815, 0.88 and 0.64, and Recall@10 scores of 0.868, 0.876, 0.944 and 0.753, on Stack Overflow, Ask Ubuntu, Ask Different, and Free code, respectively. In terms of Recall@5 and Recall@10, averaging across the 4 datasets, EnTagRec improves Tag Combine, which is the state of the art approach, by 27.3% and 12.9% respectively.

In recent years, GitHub has become the largest code host in the world, with more than 5M developers collaborating across 10M repositories. Numerous popular open source projects (such as Ruby on Rails, Homebrew, Bootstrap, Django or jQuery) have chosen GitHub as their host and have migrated their code base to it. GitHub offers a tremendous research potential. For instance, it is a flagship for current open source development, a place for developers to showcase their expertise to peers or potential recruiters, and the platform where social coding features or pull requests emerged. However, GitHub data is, to date, largely underexplored. To facilitate studies of GitHub, we have created GHTorrent, a scalable, queriable, offline mirror of the data offered through the GitHub REST API. In this paper we present a novel feature of GHTorrent designed to offer customisable data dumps on demand. The new GHTorrent data-on-demand service offers users the possibility to request via a web form up-to-date GHTorrent data dumps for any collection of GitHub repositories. We hope that by offering customisable GHTorrent data dumps we will not only lower the "barrier for entry" even further for researchers interested in mining GitHub data (thus encourage researchers to intensify their mining efforts), but also enhance the replicability of GitHub studies (since a snapshot of the data on which the results were obtained can now easily accompany each study).

Application security is becoming increasingly prevalent during software and especially web application development. Consequently, countermeasures are continuously being discussed and built into applications, with the goal of reducing the risk that unauthorized code will be able to access, steal, modify, or delete sensitive data. In this paper we gauged the presence and atmosphere surrounding security-related discussions on GitHub, as mined from discussions around commits and pull requests. First, we found that security related discussions account for approximately 10% of all discussions on GitHub. Second, we found that more negative emotions are expressed in security-related discussions than in other discussions. These findings confirm the importance of properly training developers to address security concerns in their applications as well as the need to test applications thoroughly for security vulnerabilities in order to reduce frustration and improve overall project atmosphere.

In this data paper we describe a data set obtained by means of performing an on-line survey to over 2,000 Free Libre Open Source Software (FLOSS) contributors. The survey includes questions related to personal characteristics (gender, age, civil status, nationality, etc.), education and level of English, professional status, dedication to FLOSS projects, reasons and motivations, involvement and goals. We describe as well the possibilities and challenges of using private information from the survey when linked with other, publicly available data sources. In this regard, an example of data sharing will be presented and legal, ethical and technical issues will be discussed.

Online communities are flourishing as social meeting web spaces for users and peer community members. Different online communities require different levels of competence for participants to join, and scattered evidence suggests that females and minorities as participants can be under-represented. Additional anecdotal evidence suggests that women withdraw from unfriendly online communities. Owing to the limited amount of empirical evidence on the matter, this paper provides a quantitative study of the phenomenon, in order to assess the representation and social impact of gender in online communities. This study positions itself within recent and focused international initiatives, launched by the European Commission in order to encourage women in the field of science and technology. Focusing on technical support networks around web content management tools (e.g. Drupal and WordPress) and on questions & answers websites (e.g. StackOverflow), this paper unearths a spectrum of online communities, in which women participate to various degrees.

The Mining Software Repositories community typically focuses on data from software configuration management tools, mailing lists, and bug tracking repositories to uncover interesting and actionable information about the evolution of software systems. However, the techniques employed and the challenges faced when mining are not restricted to these types of repositories. In this paper, we present an atypical dataset of software engineering conferences, containing historical data about the accepted papers and the composition of programme committees for eleven well-established conferences. The dataset (published on Github at https://github.com/tue-mdse/conferenceMetrics) can be used, e.g., by conference steering committees or programme committee chairs to assess their selection process and compare against other conferences in the field, or by prospective authors to decide in which conferences to publish.

SUMMARY With the growing need for quality assessment of entire software systems in the industry, new issues are emerging. First, because most software quality metrics are defined at the level of individual software components, there is a need for aggregation methods to summarize the results at the system level. Second, because a software evaluation requires the use of different metrics, with possibly widely varying output ranges, there is a need to combine these results into a unified quality assessment. In this paper we derive, from our experience on real industrial cases and from the scientific literature, requirements for an aggregation method. We then present a solution through the Squale model for metric aggregation, a model specifically designed to address the needs of practitioners. We empirically validate the adequacy of Squale through experiments on Eclipse. Additionally, we compare the Squale model to both traditional aggregation techniques (e.g., the arithmetic mean), and to econometric inequality indices (e.g., the Gini or the Theil indices), recently applied to aggregation of software metrics. Copyright © 2012 John Wiley & Sons, Ltd.

Today numerous software systems are being developed on top of frameworks. In this study, we analyzed the survival of 467 Eclipse third-party plug-ins altogether having 1,447 versions. We classify these plug-ins into two categories: those that depend on only stable and supported Eclipse APIs and those that depend on at least one of the potentially unstable, discouraged and unsupported Eclipse non-APIs. Comparing the two categories of plug-ins, we observed that the plug-ins depending solely on APIs have a very high source compatibility success rate compared to those that depend on at least one of the non-APIs. However, we have also observed that recently released plug-ins that depend on non-APIs also have a very high forward source compatibility success rate. This high source compatibility success rate is due to the dependency structure of these plug-ins: recently released plug-ins that depend on non-APIs predominantly depend on old Eclipse nonAPIs rather than on newly introduced ones. Finally, we showed that the majority of plug-ins hosted on SourceForge do not evolve beyond the first year of release.

Understanding an individual's contribution to an ecosystem often necessitates integrating information from multiple repositories corresponding to different projects within the ecosystem or different kinds of repositories (e.g., mail archives and version control systems). However, recognising that different contributions belong to the same contributor is challenging, since developers may use different aliases. It is known that existing identity merging algorithms are sensitive to large discrepancies between the aliases used by the same individual: the noisier the data, the worse their performance. To assess the scale of the problem for a large software ecosystem, we study all Gnome Git repositories, classify the differences in aliases, and discuss robustness of existing algorithms with respect to these types of differences. We then propose a new identity merging algorithm based on Latent Semantic Analysis (LSA), designed to be robust against more types of differences in aliases, and evaluate it empirically by means of cross-validation on Gnome Git authors. Our results show a clear improvement over existing algorithms in terms of precision and recall on worst-case input data.

The public data available in Open Source Software (OSS) repositories has been used for many practical reasons: detecting community structures; identifying key roles among developers; understanding software quality; predicting the arousal of bugs in large OSS systems, and so on; but also to formulate and validate new metrics and proof-of-concepts on general, non-OSS specific, software engineering aspects. One of the results that has not emerged yet from the analysis of OSS repositories is how to help the “career advancement” of developers: given the available data on products and processes used in OSS development, it should be possible to produce measurements to identify and describe a developer, that could be used externally as a measure of recognition and experience. This paper builds on top of the h-index, used in academic contexts, and which is used to determine the recognition of a researcher among her peers. By creating similar indices for OSS (or any) developers, this work could help defining a baseline for measuring and comparing the contributions of OSS developers in an objective, open and reproducible way.

Incompatibility between applications developed on top of frameworks with new versions of the frameworks is a big nightmare to both developers and users of the applications. Understanding the factors that cause incompatibilities is a step to solving them. One such direction is to analyze and identify parts of the reusable code of the framework that are prone to change. In this study we carried out an empirical investigation on 11 Eclipse SDK releases (1.0 to 3.7) and 288 Eclipse third-party plug-ins (ETPs) with two main goals: First, to determine the relationship between the age of Eclipse non-APIs (internal implementations) used by an ETP and the compatibility of the ETP. We found that third-party plug-in that use only old non-APIs have a high chance of compatibility success in new SDK releases compared to those that use at least one newly introduced non-API. Second, our goal was to build and test a predictive model for the compatibility of an ETP, supported in a given SDK release in a newer SDK release. Our findings produced 23 statistically significant prediction models having high values of the strength of the relationship between the predictors and the prediction (logistic regression R2 of up to 0.810). In addition, the results from model testing indicate high values of up to 100% of precision and recall and up to 98% of accuracy of the predictions. Finally, despite the fact that SDK releases with API breaking changes, i.e., 1.0, 2.0 and 3.0, have got nothing to do with non-APIs, our findings reveal that non-APIs introduced in these releases have a significant impact on the compatibility of the ETPs that use them.

A code generator is a program translating an input model into code. In this paper we focus on template-based code generators in the context of the model view controller architecture (MVC).

A popular approach to assessing software maintainability and predicting its evolution involves collecting and analyzing software metrics. However, metrics are usually defined on a micro-level (method, class, package), and should therefore be aggregated in order to provide insights in the evolution at the macro-level (system). In addition to traditional aggregation techniques such as the mean, median, or sum, recently econometric aggregation techniques, such as the Gini, Theil, Kolm, Atkinson, and Hoover inequality indices have been proposed and applied to software metrics. In this paper we present the results of an extensive correlation study of the most widely-used traditional and econometric aggregation techniques, applied to lifting SLOC values from class to package level in the 106 systems comprising the Qualitas Corpus. Moreover, we investigate the nature of this relation, and study its evolution on a subset of 12 systems from the Qualitas Corpus. Our results indicate high and statistically significant correlation between the Gini, Theil, Atkinson, and Hoover indices, i.e., aggregation values obtained using these techniques convey the same information. However, we discuss some of the rationale behind choosing between one index or another.

An Enterprise Java Beans (EJB) interceptor is a software mechanism that provides for introducing behavior implemented as separate code into the execution of a Java application. In this way EJB interceptors provide a clear separation of the core functionality of the bean and other concerns, such as logging or performance analysis. Despite the beauty of the idea behind the interceptors, developing, testing and managing dependencies introduced by the interceptors are considered to be daunting tasks. For example, the developers can specify interceptors at multiple locations and by multiple means. However, different locations and specification means influence the order of the interceptor invocation, which is governed by more than fifteen different intertwined rules according to the EJB standard. To facilitate development of EJB applications we have designed I2SD, Interceptors to Sequence Diagrams, a tool for reverse engineering EJB applications with interceptors to UML sequence diagrams. I2SD provides the developer with a visual feedback and can be used by quality managers to get a broader understanding of the way interceptors are used in their project.

We propose a new approach to aggregating software metrics from the micro-level of individual artifacts (e.g., methods, classes and packages) to the macro-level of the entire software system. The approach, Theil index, is a well-known econometric measure of inequality. The Theil index allows to study the impact of different categorizations of the artifacts, e.g., based on the development technology or developers' teams, on the inequality of the metrics values measured. We apply the Theil index in a series of experiments. We have observed that the Theil index and the related notions provide valuable insights in organization and evolution of software systems, as well as in sources of inequality.

Abstract Termination is an important and well-studied property for logic programs. However, almost all approaches for automated termination analysis focus on definite logic programs, whereas real-world Prolog programs typically use the cut operator. We introduce a novel pre-processing method which automatically transforms Prolog programs into logic programs without cuts, where termination of the cut-free program implies termination of the original program. Hence after this pre-processing, any technique for proving termination of definite logic programs can be applied. We implemented this pre-processing in our termination prover AProVE and evaluated it successfully with extensive experiments.

Since their introduction in 1994 the Martin's metrics became popular in assessing object-oriented software architectures. While one of the Martin metrics, normalised distance from the main sequence Dn, has been originally designed with assessing individual packages, it has also been applied to assess quality of entire software architectures. The approach itself, however, has never been studied. In this paper we take the first step to formalising the Dn-based architecture assessment of Java open source software. We present two aggregate measures: average normalised distance from the main sequence Dmacrn, and parameter of the fitted statistical model lambda. Applying these measures to a carefully selected collection of benchmarks we obtain a set of reference values that can be used to assess quality of a system architecture. Furthermore, we show that applying the same measures to different versions of the same system provides valuable insights in system architecture evolution.

The research domain of process discovery aims at constructing a process model (e.g. a Petri net) which is an abstract representation of an execution log. Such a model should (1) be able to reproduce the log under consideration and (2) be independent of the number of cases in the log. In this paper, we present a process discovery algorithm where we use concepts taken from the language-based theory of regions, a well-known Petri net research area. We identify a number of shortcomings of this theory from the process discovery perspective, and we provide solutions based on integer linear programming.

There are two kinds of approaches for termination analysis of logic programs: “transformational” and “direct” ones. Direct approaches prove termination directly on the basis of the logic program. Transformational approaches transform a logic program into a Term Rewrite System (TRS) and then analyze termination of the resulting TRS instead. Thus, transformational approaches make all methods previously developed for TRSs available for logic programs as well. However, the applicability of most existing transformations is quite restricted, as they can only be used for certain subclasses of logic programs. (Most of them are restricted to well-moded programs.) In this article we improve these transformations such that they become applicable for any definite logic program. To simulate the behavior of logic programs by TRSs, we slightly modify the notion of rewriting by permitting infinite terms. We show that our transformation results in TRSs which are indeed suitable for automated termination analysis. In contrast to most other methods for termination of logic programs, our technique is also sound for logic programming without occur check , which is typically used in practice. We implemented our approach in the termination prover AProVE and successfully evaluated it on a large collection of examples.

Abstract In this paper we introduce a class of constraint logic programs such that their termination can be proved by using affine level mappings. We show that membership to this class is decidable in polynomial time.

Abstract Refactoring is an established technique from the object-oriented (OO) programming community to restructure code: it aims at improving software readability, maintainability, and extensibility. Although refactoring is not tied to the OO-paradigm in particular, its ideas have not been applied to logic programming until now. This paper applies the ideas of refactoring to Prolog programs. A catalogue is presented listing refactorings classified according to scope. Some of the refactorings have been adapted from the OO-paradigm, while others have been specifically designed for Prolog. The discrepancy between intended and operational semantics in Prolog is also addressed by some of the refactorings. In addition, ViPReSS, a semi-automatic refactoring browser, is discussed and the experience with applying ViPReSS to a large Prolog legacy system is reported. The main conclusion is that refactoring is both a viable technique in Prolog and a rather desirable one.

The term meta-programming refers to the ability of writing programs that have other programs as data and exploit their semantics. The aim of this paper is presenting a methodology allowing us to perform a correct termination analysis for a broad class of practical meta-interpreters, including negation and performing different tasks during the execution. It is based on combining the power of general orderings, used in proving termination of term-rewrite systems and programs, and on the well-known acceptability condition, used in proving termination of logic programs. The methodology establishes a relationship between the ordering needed to prove termination of the interpreted program and the ordering needed to prove termination of the meta-interpreter together with this interpreted program. If such a relationship is established, termination of one of those implies termination of the other one, i.e. the meta-interpreter preserves termination. Among the meta-interpreters that are analysed correctly are a proof trees constructing meta-interpreter, different kinds of tracers and reasoners.

We present a new approach to termination analysis of numerical computations in logic programs. Traditional approaches fail to analyse them due to non well-foundedness of the integers. We present a technique that allows overcoming these difficulties. Our approach is based on transforming a program in a way that allows integrating and extending techniques originally developed for analysis of numerical computations in the framework of query-mapping pairs with the well-known framework of acceptability. Such an integration not only contributes to the understanding of termination behaviour of numerical computations, but also allows us to perform a correct analysis of such computations automatically, by extending previous work on a constraint-based approach to termination. Finally, we discuss possible extensions of the technique, including incorporating general term orderings.

Abstract EquiX is a search language for XML that combines the power of querying with the simplicity of searching. Requirements for such languages are discussed, and it is shown that EquiX meets the necessary criteria. Both a graph‐based abstract syntax and a formal concrete syntax are presented for EquiX queries. In addition, the semantics is defined and an evaluation algorithm is presented. The evaluation algorithm is polynomial under combined complexity. EquiX combines pattern matching, quantification, and logical expressions to query both the data and meta‐data of XML documents. The result of a query in EquiX is a set of XML documents. A DTD describing the result documents is derived automatically from the query.

We investigate the problem of rewriting queries with aggregate\noperators using views that may or may not contain aggregate\noperators. A rewriting of a query is a second query\nthat uses view predicates such that evaluating first the views\nand then the rewriting yields the same result as evaluating\nthe original query. In this sense, the original query and the\nrewriting are equivalent modulo the view definitions. The\nqueries and views we consider correspond to unnested SQL\nqueries, possibly with union, that employ the operators min,\nmax, count, and sum.\nOur approach is based on syntactic characterizations of the equivalence of aggregate queries. One contribution of this paper are characterizations of the equivalence of disjunctive aggregate queries, which generalize our previous results for the conjunctive case.\nFor each operator a, we introduce several types of queries using views as candidates for rewritings. We unfold such a candidate by replacing each occurrence of a view predicate with its definition, thus obtaining a regular aggregate query. The candidates have a different, usually more complex operator than a. We prove that unfolding the candidate, however, results in a regular aggregate query that is equivalent to the candidate modulo the view definitions. This property justifies considering these types of queries as natural candidates for rewritings. In this way, we reduce the problem of whether there exist rewritings of a particular type to a problem involving equivalence.\nWe distinguish between partial rewritings that contain at least one view predicate and complete rewritings that contain only view predicates. In contrast to previous work on this topic, we not only give sufficient, but also necessary conditions for a rewriting to exist. More precisely, we show for each type of candidate that the existence of both, partial and complete rewritings is decidable, and we provide upper and lower complexity bounds.