Agentic AI systems powered by Large Language Models (LLMs) are increasingly used to autonomously contribute code in modern software development. While prior work has shown that such systems can accelerate development tasks, their reliability and maintenance behavior in real-world Continuous Integration (CI) workflows remain poorly understood. In this study, we analyze 11,771 pull requests (PRs) from GitHub, including 7,619 agentic and 4,152 human-authored PRs, to investigate how agentic code behaves during CI workflows. We examine (1) CI failure rates at the pull-request level, (2) responsibility for introducing and fixing CI failures, and (3) time-to-fix at the commit level using fail–fix mappings. Our results show that human-authored CI fixes exhibit a median time to fix of 71.70 minutes, whereas AI agentic-authored CI fixes resolve failures nearly four times faster, with a median of 17.23 minutes. Our results show that agent-authored fixes resolve CI failures nearly four times faster than human fixes (median 17.23 vs. 71.70 minutes). However, agents introduce most CI failures (79.15%) while performing a smaller share of fixes (60.63%), indicating that human developers remain heavily involved in failure resolution despite faster agent responses.

Although coding agents have introduced new coordination dynamics in collaborative software development, detailed interactions in practice remain underexplored, especially for the code review process. In this study, we mine agent-authored PR references from the AIDev dataset [15] and introduce a taxonomy to characterize the intent of these references across Human-to-Agent and Agent-to-Agent interactions in the form of Pull Requests (i.e. PRs). Our analysis shows that while humans initiate most references to agent-authored PRs, a substantial portion of these interactions are AI-assisted, indicating the emergence of meta-collaborative workflows, where humans mostly use references to build new features, whereas agents make them to fix errors.

Agentic AI systems are increasingly integrated into software development workflows, contributing code alongside human developers. However, some AI-authored changes are later reverted, reflecting situations where agent-generated contributions are judged unsuitable after integration. This paper presents a large-scale empirical study of reverted changes introduced by AI coding agents to better understand the causes behind their rejection. We analyze 33,580 agentic pull requests comprising 86,315 commits authored by five major AI coding agents: Claude, Copilot, Cursor, Devin, and OpenAI Codex. Our results show that 2.66% of agentic pull requests contain at least one reverting commit, with substantial variation across agents, ranging from 0.7% for OpenAI Codex to 7.6% for GitHub Copilot indicating notable differences in code generation reliability. Through a manual analysis of 500 reverting commits, we derive a taxonomy comprising eight categories and 25 themes that explain why agent-generated code is reverted. The most common causes are unintended side effects (22.33%), overengineering (22.13%), functional incorrectness (17.71%), and dependency management problems (12.47%). Overall, our findings indicate that AI coding agents struggle primarily with scope management and contextual understanding, rather than purely functional defects. This study provides actionable guidance for practitioners, informs the design of human-AI collaboration workflows, and highlights priority areas for improving agentic code generation systems.

Infrastructure-as-Code (IaC) enables practitioners to configure and manage software infrastructure through machine-readable code files. Various IaC tools facilitate code reuse and modularity via IaC modules that act as dependencies. These modules are maintained by IaC providers to introduce new features, resolve bugs, or address security vulnerabilities. However, there is a limited understanding of how practitioners update their IaC module dependencies in their software projects, including updates frequency, delays, as well as motivations behind such updates. To fill this gap, this paper aims to understand current update practices in IaC module dependencies, focusing on Terraform (TF), being currently one of the most popular IaC tools. In particular, we investigate (i) the frequency in which IaC practitioners update their module dependencies, (ii) the technical lag phenomena, which represents the time that the infrastructure configurations remain outdated relative to their upstream modules, and (iii) the motivations that drive these updates. To achieve these, we conduct an empirical study on 13,490 TF-related commits from 131 open-source projects. Our results reveal that only 1.2% of the analyzed commits involve updating module dependencies. Furthermore, we observe an increasing technical lag from 2021 until 2024, reaching ten months on average by 2024. Then, we conduct a qualitative study using thematic analysis on code changes involving TF module dependencies updates to investigate practitioners’ motivations behind such updates. We identify that TF practitioners revolve around six main motivations, with IaC Ecosystem Compatibility, Security Vulnerabilities Fixes, and IaC Code Quality Improvement being the three most prevalent motivations. Our findings advocate that TF practitioners need customized IaC tool support for safe module dependency updates while addressing compatibility concerns.

GitHub Actions (GHA) has become among the most popular Continuous Integration (CI) platforms in open-source software (OSS) and commercial projects. Collecting such build data remains crucial for practitioners and researchers to allow build performance monitoring, optimization and improvement. However, mining GHA builds to collect build-related data and metrics remains challenging and time-consuming. This paper introduces GHAminer, an open-source tool designed to collect build-related metrics for GitHub Actions. GHAminer covers various aspects of data such as the build-related code changes and tests, the build duration and status (e.g., passed, failed, timeout, etc.), and repository metadata, which would be useful for practitioners and researchers to make data-driven decisions to enhance CI efficiency and quality. The tool has a modular architecture that supports efficient data extraction with minimal API load. Specifically, it consists of a set of modules that are related to repository information collection, build analysis, commit history analysis, and build log parsing. We evaluate the performance of GHAminer on a representative sample of 3,151 OSS projects. Results show that GHAminer is efficient in handling projects of various sizes with relatively stable performance to collect build data for larger projects. GHAminer is publicly available with a demo video at: https:lIgithub.com/stilab-ets/GHAminer

Modern Code Review (MCR) is an essential practice in software engineering, supporting early defect detection, enhancing code quality, and fostering knowledge. To manage code review tasks effectively, developers need to understand the intent behind code changes, such as a bug fix, test, refactoring, or new feature. Traditional methods for categorizing code changes in MCR rely on rule-based heuristics with predefined keywords. However, these methods lack context regarding the code changes, leading to limited generalizability, particularly when dealing with sparsely documented changes. This paper addresses these limitations by investigating the potential of Large Language Models (LLMs) for changes' intent classification. We introduce LLM Change Classifier (LLMCC), an LLM-based approach that classifies code changes based on their underlying intent. We evaluate the effectiveness of LLMCC by conducting an empirical study on three open-source projects: Android, OpenS tack, and Qt. The performance of LLMCC was benchmarked against traditional heuristic methods, conventional machine learning algorithms (including Decision Trees and Random Forests), and state-of-the-art transformer models (including BERT and RoBERTa). Results show that LLMCC significantly enhances code change intent classification accuracy, achieving up to a 33 % improvement in F1 score over heuristic-based methods. Additionally, LLMCC outperformed both traditional machine learning and transformer models, achieving an average 77% improvement in terms of Matthew Correlation Coefficient (MCC). These findings underscore the potential of LLMCC to streamline code change intent classification.

Infrastructure-as-Code (IaC) constitutes a pivotal DevOps methodology, leading edge of software deployment onto cloud platforms. IaC relies on source code files rather than manual configuration to manage the infrastructure of a software system. Terraform, an IaC tool and its declarative configuration language named HCL, has recently garnered considerable attention among IaC practitioners. Like other software artefacts, Terraform files could be affected by misconfigurations, faults, and smells. Therefore, DevOps practitioners might benefit from a quality assurance tool to help them perform quality assurance activities on Terrafrom artefacts. This paper introduces TerraMetrics, an open-source tool designed to characterize the quality of Terraform artefacts by providing a catalogue of 40 quality metrics. TerraMetrics leverages the Terraform Abstract Syntax Tree (AST) to extract the metric list, offering a potentially enduring solution compared to conventional regular expressions. This tool comprises three main components: (i) a parser transforming HCL code into an AST, (ii) visitors that traverse the AST nodes to extract the metrics, and (iii) collectors for storing the collected metrics in JSON format. The TerraMetrics tool is publicly available as an Open Source tool, with a demo video, at: https://github.com/stilab-ets/terametrics.

Infrastructure-as-Code (IaC) is an emerging software engineering practice that leverages source code to facilitate automated configuration of software systems' infrastructure. IaC files are typically complex, containing hundreds of lines of code and dependencies, making them prone to defects, which can result in breaking online services at scale. To help developers early identify and fix IaC defects, research efforts have introduced IaC defect prediction models at the file level. However, the granularity of the proposed approaches remains coarse-grained, requiring developers to inspect hundreds of lines of code in a file, while only a small fragment of code is defective. To alleviate this issue, we introduce a machine-learning-based approach to predict IaC defects at a fine-grained level, focusing on IaC blocks, i.e., small code units that encapsulate specific behaviours within an IaC file. We trained various machine learning algorithms based on a mixture of code, process, and change-level metrics. We evaluated our approach on 19 open-source projects that use Terraform, a widely used IaC tool. The results indicated that there is no single algorithm that consistently outperforms the others in 19 projects. Overall, among the six algorithms, we observed that the LightGBM model achieved a higher average of 0.21 in terms of MCC and 0.71 in terms of AUC. Models analysis reveals that the developer's experience and the relative number of added lines tend to be the most important features. Additionally, we found that blocks belonging to the most frequent types are more prone to defects. Our defect prediction models have also shown sensitivity to concept drift, indicating that IaC practitioners should regularly retrain their models.

Nowadays, Large Language Models (LLMs) play a pivotal role in software engineering. Developers can use LLMs to address software development-related tasks such as documentation, code refactoring, debugging, and testing. ChatGPT, released by OpenAI, has become the most prominent LLM. In particular, ChatGPT is a cutting-edge tool for providing recommendations and solutions for developers in their pull requests (PRs). However, little is known about the characteristics of PRs that incorporate ChatGPT compared to those without it and what developers usually use it for. To this end, we quantitatively analyzed 243 PRs that listed at least one ChatGPT prompt against a representative sample of 384 PRs without any ChatGPT prompts. Our findings show that developers use ChatGPT in larger, time-consuming pull requests that are five times slower to be closed than PRs that do not use ChatGPT. Furthermore, we perform a qualitative analysis to build a taxonomy of the topics developers primarily address in their prompts. Our analysis results in a taxonomy comprising 8 topics and 32 sub-topics. Our findings highlight that ChatGPT is often used in review-intensive pull requests. Moreover, our taxonomy enriches our understanding of the developer's current applications of ChatGPT.

Modern code review (MCR) is recognized as an effective software quality assurance practice that is broadly adopted by open-source and commercial software projects. MCR is most effective when developers follow best practices, as it improves code quality, enhances knowledge transfer, increases team awareness and shares code ownership. However, prior work highlights that poor code review practices are common and often manifest in the form of low review participation and engagement, shallow review, and toxic communications. To address these issues, we introduce GitRev, a novel approach that applies gamification mechanisms to boost developer motivation and engagement. GitRev is built on top of a Large Language Model (LLM), used as a points-based reward system that leverages the code change context, and code review activities. We implement GitRev as a GitHub app with a web browser extension that consists of a client-side web browser extension that gamifies the GitHub user interface, and a server-side composed of a Node.js server for authentication and data management. To evaluate GitRev, we conduct a controlled experiment with 86 graduate and undergraduate students. Results indicate the promising potential of our approach for improving the code review process and developers' engagement. GitRev is publicly available at https://anonymous.40pen.science/r/GitRev-OB74

Modern Code Review (MCR) is an essential process in software development to ensure high-quality code. However, developers often spend considerable time reviewing code changes before being merged into the main code base. Previous studies attempted to predict whether a code change was going to be merged or abandoned soon after it was submitted to improve the code review process. However, these approaches require complex cost-sensitive learning, which makes their adoption challenging since it is difficult for developers to understand the main factors behind the models’ predictions. To address this issue, we introduce in this article, MULTICR , a multi-objective search-based approach that uses Multi-Objective Genetic Programming (MOGP) to learn early code review prediction models as IF-THEN rules. MULTICR evolves predictive models while maximizing the accuracy of both merged and abandoned classes, eliminating the need for misclassification cost estimation. To evaluate MULTICR, we conducted an empirical study on 146,612 code reviews from Eclipse, LibreOffice, and Gerrithub. The obtained results show that MULTICR outperforms existing baselines in terms of Matthew Correlation Coefficient (MCC) and F1 scores while learning less complex models compared to decision trees. Our experiments also showed how MULTICR allows identifying the main factors related to abandoned code reviews as well as their associated thresholds, making it a promising approach for early code review prediction with notable performance and inter-operability. Additionally, we qualitatively evaluate MULTICR by conducting a user study through semi-structured interviews involving 10 practitioners from different organizations. The obtained results indicate that 90% of the participants find that MULTICR is useful and can help them to improve the code review process. Additionally, the learned IF-THEN rules of MULTICR are transparent.

Abstract Open source software development is regarded as a collaborative activity in which developers interact to build a software product. Such a human collaboration is described as an organized effort of the “social” activity of organizations, individuals, and stakeholders, which can affect the development community and the open source project health. Negative effects of the development community manifest typically in the form of community smells, which represent symptoms of organizational and social issues within the open source software development community that often lead to additional project costs and reduced software quality. Recognizing the advantages of the early detection of potential community smells in a software project, we introduce a novel approach that learns from various community organizational, social, and emotional aspects to provide an automated support for detecting community smells. In particular, our approach learns from a set of interleaving organizational–social and emotional symptoms that characterize the existence of community smell instances in a software project. We build a multi‐label learning model to detect 10 common types of community smells. We use the ensemble classifier chain (ECC) model that transforms multi‐label problems into several single‐label problems, which are solved using genetic programming (GP) to find the optimal detection rules for each smell type. To evaluate the performance of our approach, we conducted an empirical study on a benchmark of 143 open source projects. The statistical tests of our results show that our approach can detect community smells with an average F‐measure of 93%, achieving a better performance compared to different state‐of‐the‐art techniques. Furthermore, we investigate the most influential community‐related metrics to identify each community smell type.

Modern code review is a widely used technique employed in both industrial and open-source projects to improve software quality, share knowledge, and ensure adherence to coding standards and guidelines. During code review, developers may discuss refactoring activities before merging code changes in the code base. To date, code review has been extensively studied to explore its general challenges, best practices and outcomes, and socio-technical aspects. However, little is known about how refactoring is being reviewed and what developers care about when they review refactored code. Hence, in this work, we present a quantitative and qualitative study to understand what are the main criteria developers rely on to develop a decision about accepting or rejecting a submitted refactored code, and what makes this process challenging. Through a case study of 11,010 refactoring and non-refactoring reviews spread across OpenStack open-source projects, we find that refactoring-related code reviews take significantly longer to be resolved in terms of code review efforts. Moreover, upon performing a thematic analysis on a significant sample of the refactoring code review discussions, we built a comprehensive taxonomy consisting of 28 refactoring review criteria. We envision our findings reaffirming the necessity of developing accurate and efficient tools and techniques that can assist developers in the review process in the presence of refactorings.

Community smells represent symptoms of sub-optimal organizational and social issues within software development communities that often lead to additional project costs and reduced software quality. Previous research identified a variety of community smells that are connected to sub-optimal patterns under different perspectives of organizational-social structures in the software development community. To detect community smells and understanding the characteristics of such organizational-social structures in a project, we propose csDetector, an open source tool that is able to automatically detect community smells within a project and provide relevant socio-technical metrics. csDetector uses a machine learning based detection approach that learns from various existing bad community development practices to provide automated support in detecting related community smells. We evaluate the effectiveness of csDetector on a benchmark of 143 open source projects from GitHub. Our results show that the csDetector tool can detect ten commonly occurring community smells in open software projects with an average F1 score of 84%. csDetector is publicly available, with a demo video, at: https://github.com/Nuri22/csDetector.

Continuous Integration (CI) aims at supporting developers in inte-grating code changes quickly through automated building. How-ever, there is a consensus that CI build failure is a major barrierthat developers face, which prevents them from proceeding furtherwith development. In this paper, we introduceBF-Detector, anautomated tool to detect CI build failure. Based on the adaptationof Non-dominated Sorting Genetic Algorithm (NSGA-II), our toolaims at finding the best prediction rules based on two conflictingobjective functions to deal with both minority and majority classes.We evaluated the effectiveness of our tool on a benchmark of 56,019CI builds. The results reveal that our technique outperforms state-of-the-art approaches by providing a better balance between bothfailed and passed builds.BF-Detectortool is publicly available,with a demo video, at: https://github.com/stilab-ets/BF-Detector.

Modern code review (MCR) is now broadly adopted as an established and effective software quality assurance practice, with an increasing number of open-source as well as commercial software projects identifying code review as a crucial practice. During the MCR process, developers review, provide constructive feedback, and/or critique each others’ patches before a code change is merged into the codebase. Nevertheless, code review is basically a human task that involves technical, personal and social aspects. Existing literature hint the existence of poor reviewing practices i.e., anti-patterns, that may contribute to a tense reviewing culture, degradation of software quality, slow down integration, and may affect the overall sustainability of the project. To better understand these practices, we present in this paper the concept of Modern Code Review Anti-patterns (MCRA) and take a first step to define a catalog that enumerates common poor code review practices. In detail we explore and characterize MCRA symptoms, causes, and impacts. We also conduct a series of preliminary experiments to investigate the prevalence and co-occurrences of such anti-patterns on a random sample of 100 code reviews from various OpenStack projects.

Community smells are symptoms of organizational and social issues within the software development community that often increase the project costs and impact software quality. Recent studies have identified a variety of community smells and defined them as suboptimal patterns connected to organizational-social structures in the software development community such as the lack of communication, coordination and collaboration. Recognizing the advantages of the early detection of potential community smells in a software project, we introduce a novel approach that learns from various community organizational and social practices to provide an automated support for detecting community smells. In particular, our approach learns from a set of interleaving organizational-social symptoms that characterize the existence of community smell instances in a software project. We build a multi-label learning model to detect 8 common types of community smells. We use the ensemble classifier chain (ECC) model that transforms multi-label problems into several single-label problems which are solved using genetic programming (GP) to find the optimal detection rules for each smell type. To evaluate the performance of our approach, we conducted an empirical study on a benchmark of 103 open source projects and 407 community smell instances. The statistical tests of our results show that our approach can detect the eight considered smell types with an average F-measure of 89% achieving a better performance compared to different state-of-the-art techniques. Furthermore, we found that the most influential factors that best characterize community smells include the social network density and closeness centrality as well as the standard deviation of the number of developers per time zone and per community.