The integration of AI-powered coding agents into Continuous Integration/Continuous Delivery (CI/CD) pipelines has fundamentally altered how software verification is conducted. While these agents successfully automate the test generation, current evaluation benchmarks (e.g., SWE-bench) largely focus on pass-rates rather than the intrinsic quality of the generated tests. This raises the possibility of "stealth technical debt", in which test suites pass execution but do not offer comprehensive coverage or semantic value. We address this methodological gap through a large-scale, empirical comparison of 204,673 test artifacts which comprises of 24,941 human-authored files and 179,732 agent-generated files; sourced from the AIDev dataset. Using the Abstract Syntax Tree (AST) parsing with Python's naive ast module, we implemented a "white-box" static analysis framework to evaluate three quality dimensions: Assertion Strength (RQ1), Edge-Case Coverage (RQ2), and Flakiness Potential (RQ3). Our results present a nuanced inversion of traditional assumptions. AI agents performed better than humans in Edge-Case Coverage, with almost twice the variety of boundary checks (Variety Score: 0.62 vs 0.32) and a higher frequency of null-safety testing (13.40% vs. 8.3%), even though human developers had a slight advantage in Assertion Strength (88.1% strong assertions vs. 85.37% for agents). But this thoroughness comes at a price: due mostly to their reliance on file I/O and non-deterministic logic, agent-generated tests exhibited a higher risk of flakiness (Candidate Rate: 0.41 vs. 0.30). These findings suggest that while AI agents excel at rigorous boundary testing, they lack the "environmental awareness" needed to write stable, hermetic tests.

Pull requests (PRs) are a central mechanism for reviewing and integrating code changes in modern software repositories. As AI coding agents begin to submit more code changes alongside human developers, maintainers face a new challenge: deciding which PRs are likely to be accepted and which ones may require substantial review effort. This paper studies whether such outcomes can be estimated at the time a PR is opened, before reviewer discussion, CI feedback, or merge decisions are available. Using the AIDev dataset, we construct a leakage-aware prediction pipeline for human- and agent-authored PRs. The feature set is limited to submission-time information, including PR text characteristics, metadata, repository context, temporal signals, and lightweight diff statistics. We evaluate classical machine-learning models, including Logistic Regression, Random Forests, Gradient Boosting, Extra Trees, and MLPs, across pooled, human-only, agent-only, and balanced contributor views. Our results show that acceptance prediction is feasible from early signals: tree-based models achieve F1 scores above 0.95, with textual clarity and metadata among the most influential predictors. Review-effort prediction is more difficult. Comment counts and time-to-merge are only modestly explained by submission-time features, suggesting that reviewer availability, project workflow, and team-specific review practices play a major role. These findings indicate that early PR models can support triage and reviewer prioritization, but should be used as advisory tools rather than automated decision-makers.

Large Language Models (LLMs) have become an increasingly popular tool in all aspects of code development. By now, the capabilities of LLMs in code production have been substantially researched. Despite recent studies investigating the capabilities of LLMs in code production, little is known about the practical use of LLMs by developers in a refactoring context. In an effort to close this gap, we conducted an empirical study based on interactions between developers and Chat Generative Pre-trained Transformer (ChatGPT). Using Developer-Generative Pre-trained Transformer (DevGPT) dataset, our study aimed to further preliminary research on how developers can use ChatGPT to refactor code effectively. To do so, we focus on the extent to which ChatGPT was helpful, the prompt that gives a successful answer in the fewest interactions, and the programming languages in which ChatGPT is most effective. We found that, overall, ChatGPT provided developers with code that they could directly utilize in their projects more often than code that needed modifications. Our prompt taxonomy suggested that developers can often avoid lengthier conversations by using more direct prompts that define ChatGPT’s role or providing code snippets, rather than the approach of requesting code diagnosis, explanations, or code generation. Finally, ChatGPT demonstrates varying proficiency levels in different programming languages, with Cascading Style Sheets (CSS) receiving the most effective support. These insights provide valuable guidance for developers looking to optimize their use of ChatGPT for code refactoring.

Refactoring is a crucial practice in software maintenance that aims at improving design and coding practices while addressing design flaws. The Extract Method refactoring is particularly popular for consolidating duplicate code fragments into a single method. Various studies have explored ways to recommend Extract Method refactoring opportunities using techniques such as program slicing, program dependency graph analysis, change history analysis, structural similarity, and feature extraction. Despite their effectiveness, these approaches often disrupt the developer’s workflow, requiring them to pause their coding and assess the refactoring opportunities suggested throughout the project, without considering the specific development context. To enhance the adoption of Extract Method refactoring, our previous work proposed AntiCopyPaster 2.0 and investigated the effectiveness of detecting and extracting code clones without disrupting the developer’s workflow. To address these limitations, we develop a new approach in this article that supports the detection of Type-1 and Type-2 clones using the Program Structure Interface (PSI) and includes a custom-built Extract Method refactoring tool. We implement our approach using an IntelliJ IDEA extension plugin. Additionally, we integrated name recommendation models, including IntelliJ’s built-in recommender and Code2Vec , to enhance the quality of method names and improve developer productivity. To evaluate the accuracy and usefulness of our approach, we conducted a qualitative study involving 13 developers. The results indicate that (1) developers appreciate the approach and are satisfied with various aspects of the plugin’s functionality, (2) PSI effectively identifies clones by analyzing the structural and semantic aspects of the code, (3) IntelliJ’s naming recommender often provides default generic names, while code2vec produces descriptive and relevant names based on the code context, (4) the performance of AntiCopyPaster remains stable regardless of the file size and the number of clones present, (5) despite different detection and correction mechanisms, JDeodorant and AntiCopyPaster were able to perform method extraction, and AntiCopyPaster features just-in-time detection and correction, and (6) our results show an improvement in code quality after performing Extract Method refactoring with both refactoring tools. We envision that our AntiCopyPaster solution can streamline the Extract Method refactoring process, enhancing both developer efficiency and code quality by seamlessly integrating Type-2 clone detection and name recommendation capabilities in the development workflow.

Background: Mocking is a common technique for isolating test dependencies, yet duplicated mock setup code-what we call “mock clones”-can increase test maintenance overhead and reduce clarity. While code clone detection is a mature research area, the prevalence, characteristics, and refactoring of mock clones remain insufficiently understood.Aims: This study investigates the prevalence, detection challenges, and practical elimination of mock clones in real-world Java test suites, aiming to identify systematic patterns and assess the feasibility and value of mock clone refactoring.Method: We analyzed 698 mock clone instances across six open-source Java projects. We evaluated the effectiveness of existing code clone detection tools in capturing mock clones, manually refactored a large subset of clones to assess behavior preservation, and measured the structural impact of mock clone elimination.Results: Mock clones are prevalent, with frequently mocked classes often exhibiting extensive duplication. Existing clone detection tools failed to reliably detect mock clones due to scattered, reordered, and semantically varied mocking logic. Manual refactoring successfully eliminated $60 \%$ of identified mock clones while preserving test behavior, resulting in up to $64 \%$ reduction in mock objects and up to $61 \%$ reduction in mocking-related LOC. Targeting only the top $\mathbf{5 - 1 0}$ high-impact clones still achieved substantial simplification.Conclusions: Mock clone elimination is both feasible and highly valuable for improving test code maintainability. Systematic refactoring strategies can address common patterns, and prioritizing high-impact clones enables efficient gains with minimal effort. These findings motivate the need for mock-aware detection and automated refactoring tools to unlock broader maintainability improvements in testing practices.

In software development environments, code quality is crucial. This study aims to assist Machine Learning (ML) engineers in enhancing their code by identifying and correcting Data Leakage issues within their models. Data Leakage occurs when information from the test dataset is inadvertently included in the training data when preparing a data science model, resulting in misleading performance evaluations. ML developers must carefully separate their data into training, evaluation, and test sets to avoid introducing Data Leakage into their code. In this paper, we develop a new Visual Studio Code (VS Code) extension, called leakagedetector, that detects Data Leakage - mainly Overlap, Preprocessing and Multi-test leakage - from Jupyter Notebook files. Beyond detection, we included two correction mechanisms: a conventional approach, known as a quick fix, which manually fixes the leakage, and an LLM-driven approach that guides ML developers toward best practices for building ML pipelines. The plugin and its source code are publicly available on GitHub at https://github.com/SE4AIResearch/DataLeakage_JupyterNotebook_Fall2024. The demonstration video can be found on YouTube: https://youtu.be/7YiYVBiID_8. The website can be found at https://leakage-detector.vercel.app/.

The paper presents the Source Code Analysis and Lexical Annotation Runtime (SCALAR), a tool specialized for mapping (annotating) source code identifier names to their corresponding part-of-speech tag sequence (grammar pattern). SCALAR's internal model is trained using scikit-learn's GradientBoostingClassifier in conjunction with a manually-curated oracle of identifier names and their grammar patterns. This specializes the tagger to recognize the unique structure of the natural language used by developers to create all types of identifiers (e.g., function names, variable names etc.). SCALAR's output is compared with a previous version of the tagger, as well as a modern off-the-shelf part-of-speech tagger to show how it improves upon other taggers' output for annotating identifiers. The code is available on Github11https://github.com/SCANL/scanl_tagger

Code quality is of paramount importance in all types of software development settings. Our work seeks to enable Machine Learning (ML) engineers to write better code by helping them find and fix instances of Data Leakage in their models. Data Leakage often results from bad practices in writing ML code. As a result, the model effectively “memorizes” the data on which it trains, leading to an overly optimistic estimate of the model performance and an inability to make generalized predictions. ML developers must carefully separate their data into training, evaluation, and test sets to avoid introducing Data Leakage into their code. Training data should be used to train the model, evaluation data should be used to repeatedly confirm a model's accuracy, and test data should be used only once to determine the accuracy of a production-ready model. In this paper, we develop Leakagedetector, a Python plugin for the PyCharm IDE that identifies instances of Data Leakage in ML code and provides suggestions on how to remove the leakage. The plugin and its source code are publicly available on GitHub at https://github.com/SE4AIResearchlDataLeakage_Fall2023. The demonstration video can be found on YouTube: https://youtu.be/yXj3wihSaIU.

Abstract Identifier names are crucial components of code, serving as primary clues for developers to understand program behavior. This paper investigates the linguistic structure of identifier names by extending the concept of grammar patterns, which represent the part-of-speech (PoS) sequences underlying identifier phrases. The specific focus is on closed syntactic categories (e.g., prepositions, conjunctions, determiners), which are rarely studied in software engineering despite their central role in general natural language. To study these categories, the Closed Category Identifier Dataset (CCID), a new manually annotated dataset of 1,275 identifiers drawn from 30 open-source systems, is constructed and presented. The relationship between closed-category grammar patterns and program behavior is then analyzed using grounded-theory-inspired coding, statistical, and pattern analysis. The results reveal recurring structures that developers use to express concepts such as control flow, data transformation, temporal reasoning, and other behavioral roles through naming. This work contributes an empirical foundation for understanding how linguistic resources encode behavior in identifier names and supports new directions for research in naming, program comprehension, and education.

Large Language Models (LLMs), like ChatGPT, have gained widespread popularity and usage in various software engineering tasks, including programming, testing, code review, and program comprehension. However, their effectiveness in improving software quality in the classroom remains uncertain. In this paper, our aim is to shed light on our experience in teaching the use of Programming Mistake Detector (PMD) to cultivate a bugfix culture and leverage LLMs to improve software quality in educational settings. This paper discusses the results of an experiment involving 102 submissions that carried out a code review activity of 1,230 rules. Our quantitative and qualitative analysis reveals that a set of PMD quality issues influences the acceptance or rejection of the issues, and design-related categories that take longer to resolve. Although students acknowledge the potential of using ChatGPT during code review, some skepticism persists. We envision our findings to enable educators to support students with code review strategies to raise students' awareness about LLMs and promote software quality in education.

Educating students about software testing practices is integral to the curricula of many computer science-related courses and typically involves students writing unit tests. Similar to production/source code, students might inadvertently deviate from established unit testing best practices, and introduce problematic code, referred to as test smells, into their test suites. Given the extensive catalog of test smells, it becomes challenging for students to identify test smells in their code, especially for those who lack experience with testing practices. In this experience report, we aim to increase students' awareness of bad unit testing practices, and detail the outcomes of having 184 students from three higher educational institutes utilize an IDE plugin to automatically detect test smells in their code. Our findings show that while students report on the plugin's usefulness in learning about and detecting test smells, they also identify specific test smells that they consider harmless. We anticipate that our findings will support academia in refining course curricula on unit testing and enabling educators to support students with code review strategies of test code.

Large Language Models (LLMs), like ChatGPT, have gained widespread popularity and usage in various software engineering tasks, including refactoring, testing, code review, and program comprehension. Despite recent studies delving into refactoring documentation in commit messages, issues, and code review, little is known about how developers articulate their refactoring needs when interacting with ChatGPT. In this paper, our goal is to explore conversations between developers and ChatGPT related to refactoring to better understand how developers identify areas for improvement in code and how ChatGPT addresses developers' needs. Our approach relies on text mining refactoring-related conversations from 17,913 ChatGPT prompts and responses, and investigating developers' explicit refactoring intention. Our results reveal that (1) developer-ChatGPT conversations commonly involve generic and specific terms/phrases; (2) developers often make generic refactoring requests, while ChatGPT typically includes the refactoring intention; and (3) various learning settings when prompting ChatGPT in the context of refactoring. We envision that our findings contribute to a broader understanding of the collaboration between developers and AI models.

Infrastructure-as-Code (IaC) is an emerging software engineering practice that leverages source code to facilitate automated configuration of software systems' infrastructure. IaC files are typically complex, containing hundreds of lines of code and dependencies, making them prone to defects, which can result in breaking online services at scale. To help developers early identify and fix IaC defects, research efforts have introduced IaC defect prediction models at the file level. However, the granularity of the proposed approaches remains coarse-grained, requiring developers to inspect hundreds of lines of code in a file, while only a small fragment of code is defective. To alleviate this issue, we introduce a machine-learning-based approach to predict IaC defects at a fine-grained level, focusing on IaC blocks, i.e., small code units that encapsulate specific behaviours within an IaC file. We trained various machine learning algorithms based on a mixture of code, process, and change-level metrics. We evaluated our approach on 19 open-source projects that use Terraform, a widely used IaC tool. The results indicated that there is no single algorithm that consistently outperforms the others in 19 projects. Overall, among the six algorithms, we observed that the LightGBM model achieved a higher average of 0.21 in terms of MCC and 0.71 in terms of AUC. Models analysis reveals that the developer's experience and the relative number of added lines tend to be the most important features. Additionally, we found that blocks belonging to the most frequent types are more prone to defects. Our defect prediction models have also shown sensitivity to concept drift, indicating that IaC practitioners should regularly retrain their models.

Nowadays, Large Language Models (LLMs) play a pivotal role in software engineering. Developers can use LLMs to address software development-related tasks such as documentation, code refactoring, debugging, and testing. ChatGPT, released by OpenAI, has become the most prominent LLM. In particular, ChatGPT is a cutting-edge tool for providing recommendations and solutions for developers in their pull requests (PRs). However, little is known about the characteristics of PRs that incorporate ChatGPT compared to those without it and what developers usually use it for. To this end, we quantitatively analyzed 243 PRs that listed at least one ChatGPT prompt against a representative sample of 384 PRs without any ChatGPT prompts. Our findings show that developers use ChatGPT in larger, time-consuming pull requests that are five times slower to be closed than PRs that do not use ChatGPT. Furthermore, we perform a qualitative analysis to build a taxonomy of the topics developers primarily address in their prompts. Our analysis results in a taxonomy comprising 8 topics and 32 sub-topics. Our findings highlight that ChatGPT is often used in review-intensive pull requests. Moreover, our taxonomy enriches our understanding of the developer's current applications of ChatGPT.

Refactoring is the practice of improving software quality without altering its external behavior. Developers intuitively refactor their code for multiple purposes, such as improving program comprehension, reducing code complexity, dealing with technical debt, and removing code smells. However, no prior studies have exposed the students to an experience of the process of antipatterns detection and refactoring correction, and provided students with toolset to practice it. To understand and increase the awareness of refactoring concepts, in this paper, we aim to reflect on our experience with teaching refactoring and how it helps students become more aware of bad programming practices and the importance of correcting them via refactoring. This paper discusses the results of an experiment in the classroom that involved carrying out various refactoring activities for the purpose of removing antipatterns using JDeodorant, an IDE plugin that supports antipatterns detection and refactoring. The results of the quantitative and qualitative analysis with 171 students show that students tend to appreciate the idea of learning refactoring and are satisfied with various aspects of the JDeodorant plugin's operation. Through this experiment, refactoring can turn into a vital part of the computing educational plan. We envision our findings enabling educators to support students with refactoring tools tuned towards safer and trustworthy refactoring.

Duplicating a code fragment is the act of copying and pasting it with or without minor modifications into another section of the code base. Despite being an intuitive practice of code reuse, duplicate code brings its own challenges to software maintenance and evolution. To combat the propagation of duplicate code, we developed a plugin for IntelliJ IDEA called AntiCopyPaster, which tracks the pasting of code fragments inside the IDE and suggests the appropriate Extract Method refactoring. Unlike the existing approaches, our tool is integrated with the development workflow, and pro-actively recommends refactorings. Since not all code fragments need to be extracted, we develop a classification model to make this decision. When a code fragment is copy-pasted, the plugin searches for duplicates in the currently opened file, waits for a short period of time to allow the developer to edit the code, and finally inferences the refactoring decision based on a number of features. This tool can be used by educators to control the spread of duplicate code in their students code, and raise their awareness of the exitence of automated refactoring tehchniques that exist in the IDE. The plugin and its source code are publicly available on GitHub at https://github.com/refactorings/anti-copy-paster. The demonstration video can be found on YouTube: https://youtu.be/Y1sbfpds2Ms.

Static analysis tools are frequently used to scan the source code and detect deviations from the project coding guidelines. Yet, their adoption is challenged by their high false positive rate, which makes them not suitable for students and novice developers. However, Large Language Models (LLMs), such as ChatGPT, have gained widespread popularity and usage in various software engineering tasks, including testing, code review, and program comprehension. Such models represent an opportunity to reduce the ambiguity of static analysis tools and support their adoption. Yet, the effectiveness of using static analysis (i.e., PMD) to detect coding issues, and relying on LLMs (i.e., ChatGPT) to explain and recommend fix, has not yet been explored. In this talk, we aim to shed light on our experience in teaching the use of ChatGPT to cultivate a bugfix culture and leverage LLMs to improve software quality in educational settings. We share our findings to support educators in teaching students better code review strategies, and to increase students' awareness about LLM and promote software quality in education.

Abstract Refactoring is widely recognized as one of the efficient techniques to manage technical debt and maintain a healthy software project through enforcing best design practices, or coping with design defects. Previous refactoring surveys have shown that code refactoring activities are mainly executed by developers who have sufficient knowledge of the system's design, and disposing of leadership roles in their development teams. However, these surveys were mainly limited to specific projects and companies. In this paper, we explore the generalizability of the previous results by analyzing 800 open‐source projects. We mine their refactoring activities, and we identify their corresponding contributors. Then, we associate an experience score to each contributor in order to test various hypotheses related to whether developers with higher scores tend to (1) perform a higher number of refactoring operations, (2) exhibit different motivations behind their refactoring, and (3) better document their refactoring activity. We found that (1) although refactoring is not restricted to a subset of developers, those with higher contribution scores tend to perform more refactorings than others; (2) although there is no correlation between experience and motivation behind refactoring, top contributed developers are found to perform a wider variety of refactoring operations, regardless of their complexity; and (3) top contributed developer tend to document less their refactoring activity. Our qualitative analysis of three randomly sampled projects shows that the developers who are responsible for the majority of refactoring activities are typically in advanced positions in their development teams, demonstrating their extensive knowledge of the design of the systems they contribute to.

Background:Code refactoring is widely recognized as an essential software engineering practice to improve the understandability and maintainability of the source code. TheExtract Methodrefactoring is considered as “Swiss army knife” of refactorings, as developers often apply it to improve their code quality,e.g., decompose long code fragments, reduce code complexity, eliminate duplicated code, etc. In recent years, several studies attempted to recommendExtract Methodrefactorings allowing the collection, analysis, and revelation of actionable data-driven insights about refactoring practices within software projects.Aim:In this paper, we aim at reviewing the current body of knowledge on existing Extract Method refactoring research and explore their limitations and potential improvement opportunities for future research efforts. That is,Extract Methodis considered one of the most widely-used refactorings, but difficult to apply in practice as it involves low-level code changes such as statements, variables, parameters, return types, etc. Hence, researchers and practitioners begin to be aware of the state-of-the-art and identify new research opportunities in this context.Method:We review the body of knowledge related toExtract Methodrefactoring in the form of a systematic literature review (SLR). After compiling an initial pool of 1,367 papers, we conducted a systematic selection and our final pool included 83 primary studies. We define three sets of research questions and systematically develop and refine a classification schema based on several criteria including their methodology, applicability, and degree of automation.Results:The results construct a catalog of 83Extract Methodapproaches indicating that several techniques have been proposed in the literature. Our results show that: (i) 38.6% ofExtract Methodrefactoring studies primarily focus on addressing code clones; (ii) Several of theExtract Methodtools incorporate the developer’s involvement in the decision-making process when applying the method extraction, and (iii) the existing benchmarks are heterogeneous and do not contain the same type of information, making standardizing them for the purpose of benchmarking difficult.Conclusions:Our study serves as an “index” to the body of knowledge in this area for researchers and practitioners in determining theExtract Methodrefactoring approach that is most appropriate for their needs. Our findings also empower the community with information to guide the future development of refactoring tools.

During the last few years, Continuous Integration (CI) has become a common practice in open-source and industrial environments to reduce the scope for errors and increase the speed to market through the automated build and test processes. However, despite this wide adoption throughout the years, little is known about the challenges developers discuss. Analyzing the discussions of developers is required to understand what researchers, educators and practitioners should focus on, and how discussion communities can be helpful to shed the light on CI challenges. In this study, we examine Stack Overflow (SO), the most popular crowd-sourced forum, to understand the challenges developers face in the CI context. We collect a corpus of 27,728 CI-related developers posts from SO and analyze those posts through a mixed method with quantitative and qualitative analyzes. To study the trends of CI discussions, we investigated the metadata of CI questions, users and tags. Then, we extract the CI main topics using Latent Dirichlet Allocation (LDA) tuned with Genetic Algorithm (GA). Finally, we investigate the most popular and difficult topics faced by developers based on unanswered questions to get further insights into CI challenges. The LDA clustering reveals that developers face challenges with six main topics namely Build, Testing, Version Control, Configuration, Deployment, and CI Culture. Particularly, we found that the build topic is the most popular among the studied topics and that version control and testing topics are the most difficult for the SO community. Our study uncovers insights about CI challenges and adds evidence to existing knowledge about CI issues related especially to software build.

Background. Infrastructure-as-Code (IaC) is an emerging practice to manage cloud infrastructure resources for software systems. Modern software development has evolved to embrace IaC as a best practice for consistently provisioning and managing infrastructure using various tools such as Terraform and Ansible. However, recent studies highlighted that developers still encounter various challenges with IaC tools. Aims. We aim in this paper to understand the different challenges that developers encounter with IaC and analyze the trend of seeking assistance on Q&A platforms in the context of IaC. To this end, we conduct a large-scale empirical study investigating developers' discussions in Stack Overflow. Method. We first collect IaC-relevant tags on Stack Overflow, constituting a dataset that comprises 52,692 questions and 64,078 answers. Then, we group questions into specific topics using the Latent Dirichlet Allocation (LDA) method, which we optimize using a Genetic Algorithm (GA) for parameter's fine-tuning. Finally, to gain better insights, we analyze the identified topics based on different criteria such as popularity and difficulty. Results. Our findings reveal an average yearly increase of 150% in terms of IaC-related questions and 135% in terms of users between 2011 and 2022. Furthermore, we observe that IaC questions revolve around seven main topics: server configuration, policy configuration, networking, deployment pipelines, variable management, templating, and file management. Notably, we found that server configuration and file management are the most popular topics, i.e., the most discussed among IaC developers, while the deployment pipelines and templating topics are the most difficult. Conclusions. Our results shed light on IaC challenges that are often encountered by developers on popular Q&A platforms. These findings reveal important implications for practitioners seeking better support for IaC tools in real-world settings and for researchers to better understand the IaC community needs and further investigate IaC in different aspects.

We aim to explore how developers document their refactoring activities during the software life cycle. We call such activity Self-Affirmed Refactoring (SAR), which indicates developers’ documentation of their refactoring activities. SAR is crucial in understanding various aspects of refactoring, including the motivation, procedure, and consequences of the performed code change. After that, we propose an approach to identify whether a commit describes developer-related refactoring events to classify them according to the refactoring common quality improvement categories. To complement this goal, we aim to reveal insights into how reviewers decide to accept or reject a submitted refactoring request and what makes such a review challenging.Our SAR taxonomy and model can work with refactoring detectors to report any early inconsistency between refactoring types and their documentation. They can serve as a solid background for various empirical investigations. Our survey with code reviewers has revealed several difficulties related to understanding the refactoring intent and implications on the functional and non-functional aspects of the software. In light of our findings from the industrial case study, we recommended a procedure to properly document refactoring activities, as part of our survey feedback.

We developed a plugin for IntelliJ IDEA called AntiCopyPaster, which tracks the pasting of code fragments inside the IDE and suggests the appropriate Extract Method refactoring to combat the propagation of duplicates. Unlike the existing approaches, our tool is integrated with the developer’s workflow, and pro-actively recommends refactorings. Since not all code fragments need to be extracted, we develop a classification model to make this decision. When a developer copies and pastes a code fragment, the plugin searches for duplicates in the currently opened file, waits for a short period of time to allow the developer to edit the code, and finally inferences the refactoring decision based on a number of features.

In the process of developing software, code review is crucial. By identifying problems before they arise in production, it enhances the quality of the code. Finding the best reviewer for a code change, however, is extremely challenging especially in large scale, especially open source software stacks with cross functioning designs and collaborations among multiple developers and teams. Additionally, a review by someone who lacks knowledge and understanding of the code can result in high resource consumption and technical errors. The reviewers who have the specialty in both functioning (domain knowledge) and non-functioning areas of a commit are considered as the most qualified reviewer to look over any changes to the code. Quality attributes serve as the connection among the user requirements, delivered function description, software architecture and implementation through put the entire software stack cycle. In this study, we target on auto reviewer assignment in large scale software stacks and aim to build a self-learning, and self-correct platform for intelligently matching between a commit based on its quality attributes and the skills sets of reviewers. To achieve this, quality attributes are classified and abstracted from the commit messages and based on which, the commits are assigned to the reviewers with the capability in reviewing the target commits. We first designed machine learning schemes for abstracting quality attributes based on historical data from the OpenStack repository. Two models are built and trained for automating the classification of the commits based on their quality attributes using the manual labeling of commits and multi-class classifiers. We then positioned the reviewers based on their historical data and the quality attributes characteristics. Finally we selected the recommended reviewer based on the distance between a commit and candidate reviewers. In this paper, we demonstrate how the models can choose the best quality attributes and assign the code review to the most qualified reviewers. With a comparatively small training dataset, the models are able to achieve F-1 scores of 77% and 85.31%, respectively.

Modern code review is a widely used technique employed in both industrial and open-source projects to improve software quality, share knowledge, and ensure adherence to coding standards and guidelines. During code review, developers may discuss refactoring activities before merging code changes in the code base. To date, code review has been extensively studied to explore its general challenges, best practices and outcomes, and socio-technical aspects. However, little is known about how refactoring is being reviewed and what developers care about when they review refactored code. Hence, in this work, we present a quantitative and qualitative study to understand what are the main criteria developers rely on to develop a decision about accepting or rejecting a submitted refactored code, and what makes this process challenging. Through a case study of 11,010 refactoring and non-refactoring reviews spread across OpenStack open-source projects, we find that refactoring-related code reviews take significantly longer to be resolved in terms of code review efforts. Moreover, upon performing a thematic analysis on a significant sample of the refactoring code review discussions, we built a comprehensive taxonomy consisting of 28 refactoring review criteria. We envision our findings reaffirming the necessity of developing accurate and efficient tools and techniques that can assist developers in the review process in the presence of refactorings.

Understanding the practice of refactoring documentation is of paramount importance in academia and industry. Issue tracking systems are used by most software projects enabling developers, quality assurance, managers, and users to submit feature requests and other tasks such as bug fixing and code review. Although recent studies explored how to document refactoring in commit messages, little is known about how developers describe their refactoring needs in issues. In this study, we aim at exploring developer-reported refactoring changes in issues to better understand what developers consider to be problematic in their code and how they handle it. Our approach relies on text mining 45,477 refactoring-related issues and identifying refactoring patterns from a diverse corpus of 77 Java projects by investigating issues associated with 15,833 refactoring operations and developers' explicit refactoring intention. Our results show that (1) developers mostly use move refactoring related terms/phrases to target refactoring-related issues; and (2) developers tend to explicitly mention the improvement of specific quality attributes and focus on duplicate code removal. We envision our findings enabling tool builders to support developers with automated documentation of refactoring changes in issues.

In recent years, mobile accessibility has become an important trend with the goal of allowing all users the possibility of using any app without many limitations. User reviews include insights that are useful for app evolution. However, with the increase in the amount of received reviews, manually analyzing them is tedious and time-consuming, especially when searching for accessibility reviews. The goal of this paper is to support the automated identification of accessibility in user reviews, to help technology professionals in prioritizing their handling, and thus, creating more inclusive apps. Particularly, we design a model that takes as input accessibility user reviews, learns their keyword-based features, in order to make a binary decision, for a given review, on whether it is about accessibility or not. The model is evaluated using a total of 5,326 mobile app reviews. The findings show that (1) our model can accurately identify accessibility reviews, outperforming two baselines, namely keyword-based detector and a random classifier; (2) our model achieves an accuracy of 85% with relatively small training dataset; however, the accuracy improves as we increase the size of the training dataset.

It is good practice to name test methods such that they are comprehensible to developers; they must be written in such a way that their purpose and functionality are clear to those who will maintain them. Unfortunately, there is little automated support for writing or maintaining the names of test methods. This can lead to inconsistent and low-quality test names and increase the maintenance cost of supporting these methods. Due to this risk, it is essential to help developers in maintaining their test method names over time. In this paper, we use grammar patterns, and how they relate to test method behavior, to understand test naming practices. This data will be used to support an automated tool for maintaining test names.

Despite the availability of refactoring as a feature in popular IDEs, recent studies revealed that developers are reluctant to use them, and still prefer the manual refactoring of their code. At JetBrains, our goal is to fully support refactoring features in IntelliJ-based IDEs and improve their adoption in practice. Therefore, we start by raising the following main questions. How exactly do people refactor code? What refactorings are the most popular? Why do some developers tend not to use convenient IDE refactoring tools?

The concept of Self-Affirmed Refactoring (SAR) was introduced to explore how developers document their refactoring activities in commit messages, i.e., developers explicit documentation of refactoring operations intentionally introduced during a code change. In our previous study, we have manually identified refactoring patterns and defined three main common quality improvement categories including internal quality attributes, external quality attributes, and code smells, by only considering refactoring-related commits. However, this approach heavily depends on the manual inspection of commit messages. In this paper, we propose a two-step approach to first identify whether a commit describes developer-related refactoring events, then to classify it according to the refactoring common quality improvement categories. Specifically, we combine the N-Gram TF–IDF feature selection with binary and multiclass classifiers to build a new model to automate the classification of refactorings based on their quality improvement categories. We challenge our model using a total of 2,867 commit messages extracted from well engineered open-source Java projects. Our findings show that (1) our model is able to accurately classify SAR commits, outperforming the pattern-based and random classifier approaches, and allowing the discovery of 40 more relevant SAR patterns, and (2) our model reaches an F-measure of up to 90% even with a relatively small training dataset.

Background. Refactoring is a critical task in software maintenance and is generally performed to enforce the best design and implementation practices or to cope with design defects. Several studies attempted to detect refactoring activities through mining software repositories allowing to collect, analyze and get actionable data-driven insights about refactoring practices within software projects. Aim. We aim at identifying, among the various quality models presented in the literature, the ones that are more in-line with the developer's vision of quality optimization, when they explicitly mention that they are refactoring to improve them. Method. We extract a large corpus of design-related refactoring activities that are applied and documented by developers during their daily changes from 3,795 curated open source Java projects. In particular, we extract a large-scale corpus of structural metrics and anti-pattern enhancement changes, from which we identify 1,245 quality improvement commits with their corresponding refactoring operations, as perceived by software engineers. Thereafter, we empirically analyze the impact of these refactoring operations on a set of common state-of-the-art design quality metrics. Results. The statistical analysis of the obtained results shows that (i) a few state-of-the-art metrics are more popular than others; and (ii) some metrics are being more emphasized than others. Conclusions. We verify that there are a variety of structural metrics that can represent the internal quality attributes with different degrees of improvement and degradation of software quality. Most of the metrics that are mapped to the main quality attributes do capture developer intentions of quality improvement reported in the commit messages.

Refactoring is a critical task in software maintenance. It is typically performed to enforce best design practices or to cope with design defects. Research in refactoring has been driven by the need to improve system structures. However, recent studies have shown that developers may incorporate refactoring strategies in other development-related activities that go beyond improving the design. Unfortunately, these studies are limited to developer interviews and a reduced set of projects. In this context, we aim at exploring how developers document their refactoring activities during the software life cycle, we call such activity Self-Affirmed Refactoring (SAR), by understanding developers perception of refactorings so that we can bridge the gap between refactoring and automation in general. We aim in more accurately mimicking the human decision making when recommending better software refactoring and remodularization.