Large Language Models (LLMs) are increasingly used through autonomous agents (e.g., Copilot, Cursor, Devin, Claude) to perform complex software development tasks. However, little is known about how these agents introduce and document technical debt through Self-Admitted Technical Debt (SATD) comments. Understanding SATD in AI-generated code is critical, as such comments explicitly reveal acknowledged limitations and deferred fixes that affect long-term maintenance. In this study, we quantitatively and qualitatively analyze 525 SATD comments authored by AI agents using the AIDev dataset. Our results show that AI-generated SATD is slightly more technically detailed than human-authored SATD, yet both often describe problems without clear guidance on resolution. Through thematic analysis, we identify 34 SATD topics grouped into 10 categories, with AI agents predominantly documenting requirement- and design-related debt. While many SATD topics overlap between AI and humans, our taxonomy reveals new debt categories and emphases specific to AI-authored SATD, particularly related to infrastructure, pipelines, dependency management, and requirement interpretation driven by developer prompts. Overall, our findings suggest that AI- and human-authored SATD share common characteristics but differ in expression and focus, highlighting the need for deeper investigation into how agentic systems communicate and manage technical debt.

Agentic AI systems powered by Large Language Models (LLMs) are increasingly used to autonomously contribute code in modern software development. While prior work has shown that such systems can accelerate development tasks, their reliability and maintenance behavior in real-world Continuous Integration (CI) workflows remain poorly understood. In this study, we analyze 11,771 pull requests (PRs) from GitHub, including 7,619 agentic and 4,152 human-authored PRs, to investigate how agentic code behaves during CI workflows. We examine (1) CI failure rates at the pull-request level, (2) responsibility for introducing and fixing CI failures, and (3) time-to-fix at the commit level using fail–fix mappings. Our results show that human-authored CI fixes exhibit a median time to fix of 71.70 minutes, whereas AI agentic-authored CI fixes resolve failures nearly four times faster, with a median of 17.23 minutes. Our results show that agent-authored fixes resolve CI failures nearly four times faster than human fixes (median 17.23 vs. 71.70 minutes). However, agents introduce most CI failures (79.15%) while performing a smaller share of fixes (60.63%), indicating that human developers remain heavily involved in failure resolution despite faster agent responses.

AI coding agents are increasingly being adopted to autonomously author pull requests (PRs). While these agents can perform a wide range of tasks, little is known about how much human intervention is required to collaborate with them and integrate AI Agents contributions in practice. In this paper, we conduct an empirical study on human intervention in agent-authored PRs (APRs) as a measure of human effort and oversight. Using the AIDev dataset, we first compare how often humans intervene in APRs vs human-authored PRs (HPRs), as well as the resulting outcomes of this intervention. We then conduct a qualitative thematic analysis of human interventions in APRs and derive a taxonomy including 4 high-level categories and 42 intervention actions. Our results show that, human interventions occur less frequently in APRs than in HPRs (52.17% vs. 83.59%), but when it occurs in APRs, it requires higher review effort, including larger code churn and longer durations. Our taxonomy results show that most human effort is spent on guidance-level interventions with 58.02%, focusing on restricting the agent’s actions and enforcing project-conventions, followed by decision-level interventions at 21.16%, direct code changes at 17.05% and operational-level intervention at 3.69%. Which indicates that, collaboration with coding agents, is shifting developer work from implementation to supervision, guidance and quality control.

Agentic AI systems are increasingly integrated into software development workflows, contributing code alongside human developers. However, some AI-authored changes are later reverted, reflecting situations where agent-generated contributions are judged unsuitable after integration. This paper presents a large-scale empirical study of reverted changes introduced by AI coding agents to better understand the causes behind their rejection. We analyze 33,580 agentic pull requests comprising 86,315 commits authored by five major AI coding agents: Claude, Copilot, Cursor, Devin, and OpenAI Codex. Our results show that 2.66% of agentic pull requests contain at least one reverting commit, with substantial variation across agents, ranging from 0.7% for OpenAI Codex to 7.6% for GitHub Copilot indicating notable differences in code generation reliability. Through a manual analysis of 500 reverting commits, we derive a taxonomy comprising eight categories and 25 themes that explain why agent-generated code is reverted. The most common causes are unintended side effects (22.33%), overengineering (22.13%), functional incorrectness (17.71%), and dependency management problems (12.47%). Overall, our findings indicate that AI coding agents struggle primarily with scope management and contextual understanding, rather than purely functional defects. This study provides actionable guidance for practitioners, informs the design of human-AI collaboration workflows, and highlights priority areas for improving agentic code generation systems.

Infrastructure-as-Code (IaC) enables practitioners to configure and manage software infrastructure through machine-readable code files. Various IaC tools facilitate code reuse and modularity via IaC modules that act as dependencies. These modules are maintained by IaC providers to introduce new features, resolve bugs, or address security vulnerabilities. However, there is a limited understanding of how practitioners update their IaC module dependencies in their software projects, including updates frequency, delays, as well as motivations behind such updates. To fill this gap, this paper aims to understand current update practices in IaC module dependencies, focusing on Terraform (TF), being currently one of the most popular IaC tools. In particular, we investigate (i) the frequency in which IaC practitioners update their module dependencies, (ii) the technical lag phenomena, which represents the time that the infrastructure configurations remain outdated relative to their upstream modules, and (iii) the motivations that drive these updates. To achieve these, we conduct an empirical study on 13,490 TF-related commits from 131 open-source projects. Our results reveal that only 1.2% of the analyzed commits involve updating module dependencies. Furthermore, we observe an increasing technical lag from 2021 until 2024, reaching ten months on average by 2024. Then, we conduct a qualitative study using thematic analysis on code changes involving TF module dependencies updates to investigate practitioners’ motivations behind such updates. We identify that TF practitioners revolve around six main motivations, with IaC Ecosystem Compatibility, Security Vulnerabilities Fixes, and IaC Code Quality Improvement being the three most prevalent motivations. Our findings advocate that TF practitioners need customized IaC tool support for safe module dependency updates while addressing compatibility concerns.

Large Language Models (LLMs), like ChatGPT, have gained widespread popularity and usage in various software engineering tasks, including refactoring, testing, code review, and program comprehension. Despite recent studies delving into refactoring documentation in commit messages, issues, and code review, little is known about how developers articulate their refactoring needs when interacting with ChatGPT. In this paper, our goal is to explore conversations between developers and ChatGPT related to refactoring to better understand how developers identify areas for improvement in code and how ChatGPT addresses developers' needs. Our approach relies on text mining refactoring-related conversations from 17,913 ChatGPT prompts and responses, and investigating developers' explicit refactoring intention. Our results reveal that (1) developer-ChatGPT conversations commonly involve generic and specific terms/phrases; (2) developers often make generic refactoring requests, while ChatGPT typically includes the refactoring intention; and (3) various learning settings when prompting ChatGPT in the context of refactoring. We envision that our findings contribute to a broader understanding of the collaboration between developers and AI models.

Infrastructure-as-Code (IaC) is an emerging software engineering practice that leverages source code to facilitate automated configuration of software systems' infrastructure. IaC files are typically complex, containing hundreds of lines of code and dependencies, making them prone to defects, which can result in breaking online services at scale. To help developers early identify and fix IaC defects, research efforts have introduced IaC defect prediction models at the file level. However, the granularity of the proposed approaches remains coarse-grained, requiring developers to inspect hundreds of lines of code in a file, while only a small fragment of code is defective. To alleviate this issue, we introduce a machine-learning-based approach to predict IaC defects at a fine-grained level, focusing on IaC blocks, i.e., small code units that encapsulate specific behaviours within an IaC file. We trained various machine learning algorithms based on a mixture of code, process, and change-level metrics. We evaluated our approach on 19 open-source projects that use Terraform, a widely used IaC tool. The results indicated that there is no single algorithm that consistently outperforms the others in 19 projects. Overall, among the six algorithms, we observed that the LightGBM model achieved a higher average of 0.21 in terms of MCC and 0.71 in terms of AUC. Models analysis reveals that the developer's experience and the relative number of added lines tend to be the most important features. Additionally, we found that blocks belonging to the most frequent types are more prone to defects. Our defect prediction models have also shown sensitivity to concept drift, indicating that IaC practitioners should regularly retrain their models.

Nowadays, Large Language Models (LLMs) play a pivotal role in software engineering. Developers can use LLMs to address software development-related tasks such as documentation, code refactoring, debugging, and testing. ChatGPT, released by OpenAI, has become the most prominent LLM. In particular, ChatGPT is a cutting-edge tool for providing recommendations and solutions for developers in their pull requests (PRs). However, little is known about the characteristics of PRs that incorporate ChatGPT compared to those without it and what developers usually use it for. To this end, we quantitatively analyzed 243 PRs that listed at least one ChatGPT prompt against a representative sample of 384 PRs without any ChatGPT prompts. Our findings show that developers use ChatGPT in larger, time-consuming pull requests that are five times slower to be closed than PRs that do not use ChatGPT. Furthermore, we perform a qualitative analysis to build a taxonomy of the topics developers primarily address in their prompts. Our analysis results in a taxonomy comprising 8 topics and 32 sub-topics. Our findings highlight that ChatGPT is often used in review-intensive pull requests. Moreover, our taxonomy enriches our understanding of the developer's current applications of ChatGPT.

Modern code review is a widely used technique employed in both industrial and open-source projects to improve software quality, share knowledge, and ensure adherence to coding standards and guidelines. During code review, developers may discuss refactoring activities before merging code changes in the code base. To date, code review has been extensively studied to explore its general challenges, best practices and outcomes, and socio-technical aspects. However, little is known about how refactoring is being reviewed and what developers care about when they review refactored code. Hence, in this work, we present a quantitative and qualitative study to understand what are the main criteria developers rely on to develop a decision about accepting or rejecting a submitted refactored code, and what makes this process challenging. Through a case study of 11,010 refactoring and non-refactoring reviews spread across OpenStack open-source projects, we find that refactoring-related code reviews take significantly longer to be resolved in terms of code review efforts. Moreover, upon performing a thematic analysis on a significant sample of the refactoring code review discussions, we built a comprehensive taxonomy consisting of 28 refactoring review criteria. We envision our findings reaffirming the necessity of developing accurate and efficient tools and techniques that can assist developers in the review process in the presence of refactorings.

Understanding the practice of refactoring documentation is of paramount importance in academia and industry. Issue tracking systems are used by most software projects enabling developers, quality assurance, managers, and users to submit feature requests and other tasks such as bug fixing and code review. Although recent studies explored how to document refactoring in commit messages, little is known about how developers describe their refactoring needs in issues. In this study, we aim at exploring developer-reported refactoring changes in issues to better understand what developers consider to be problematic in their code and how they handle it. Our approach relies on text mining 45,477 refactoring-related issues and identifying refactoring patterns from a diverse corpus of 77 Java projects by investigating issues associated with 15,833 refactoring operations and developers' explicit refactoring intention. Our results show that (1) developers mostly use move refactoring related terms/phrases to target refactoring-related issues; and (2) developers tend to explicitly mention the improvement of specific quality attributes and focus on duplicate code removal. We envision our findings enabling tool builders to support developers with automated documentation of refactoring changes in issues.