Software development agents such as Claude Code, GitHub Copilot, Cursor Agent, Devin, and OpenAI Codex are being increasingly integrated into developer workflows. While prior work has evaluated agent capabilities for code completion and task automation, there is little work investigating how these agents perform Java refactoring in practice, the types of changes they make, and their impact on code quality. In this study, we present the first analysis of agentic refactoring pull requests in Java, comparing them to developer refactorings across 86 projects per group. Using RefactoringMiner and DesigniteJava 3.0, we identify refactoring types and detect code smells before and after refactoring commits. Our results show that agent refactorings are dominated by annotation changes (the 5 most common refactoring types done by agents are annotation related), in contrast to the diverse structural improvements typical of developers. Despite these differences in refactoring types, we find Cursor to be the only model to show a statistically significant increase in refactoring smells.

In Unreal Engine, a popular game engine for AAA (high budget, high profile) title video games, Blueprint Visual Scripting is a widely used tool for developing gameplay elements using visual node and edge-based source code. Despite its widespread adoption, there is limited research on the intersection of software engineering and Blueprint-based visual programming. This dataset aims to address this gap by providing parsed Blueprint graphs extracted from Unreal Engine’s binary UAsset files. We developed extractors and a custom parser to mine Blueprint graphs from 335,753 Blueprint UAsset files across $\mathbf{2 4, 0 0 9}$ GitHub projects. By providing this dataset, we hope to encourage future research on the structure and usage of Unreal Engine Blueprints, and promote the development of tools—such as code smell detectors and language models for code completion-that can optimize visual programming practices within Unreal Engine.

Pure Data (PD) is a visual programming language for computer music that allows users to create applications through a graph-based, drag-and-drop interface, using objects and connections to manage program flow. There is a lack of tool support for computer musicians using PD, particularly for code completion. In this paper, we introduce TriGraph, a graph-based probabilistic model specifically designed for code completion in PD. TriGraph uses statistical analysis of 2-node and 3-node subgraph frequencies to predict nodes and connections in PD graphs. Using a dataset of parsed PD files, we train and evaluate 5 TriGraph models, assessing their performance in predicting nodes and edges in PD graphs. Our evaluations indicate that the models achieve an average Mean Reciprocal Rank (MRR) score of 0.39 for node prediction, placing the correct answer within the top 3 suggestions, and outperforming the n-grambased KenLM model on similar tasks. For edge prediction, the models achieve an average MRR score of 0.57, with results showing that incorporating both 2 -node and 3-node subgraphs yields better results than using only 3 -node subgraphs. These findings suggest that TriGraph could enhance the productivity of PD programmers by providing code completion support that may speed up development, reduce errors, and assist in discovering available options. These potential benefits highlight its promise as a valuable support tool for end-user programmers in graphical environments.

Pure Data (PD), a data-flow based visual programming language utilized for music and sound synthesis, remains underexplored in software engineering research. Existing literature fails to address the nuanced programming practices within PD, prompting the need to investigate how end-users manipulate nodes and edges in this visual language. This paper systematically extracts and analyzes 6,534 publicly available PD projects from GitHub. Employing source code parsing, pattern matching, and statistical analysis, we unveil usage patterns of PD by the end-user programmers. We found that most revisions of the PD files are small and simple, with fewer than 64 nodes, 51 connections, and 3 revisions. Most PD projects have less than 17 PD files, 31 commits, and only 1 author working on the PD files. The median differences in the number of nodes and edges between each commit and its parents, modifying the same file, are 3 and 0, respectively, implying small changes across various revisions of a PD file. Our findings contribute a valuable dataset for future studies, addressing the dearth of research in PD. By unraveling usage patterns, we provide insights that empower scholars and practitioners to optimize the programming experience for end-users in the realm of visual programming languages.

Smartphone application (app) developers measure the energy consumption of their apps to ensure that they do not consume excessive energy. However, existing techniques require developers to generate and execute test cases on expensive, sophisticated hardware. To address these challenges, we propose a static-analysis approach that estimates the energy consumption of API usage in an app, eliminating the need for test case execution. To instantiate our approach, we have profiled the energy consumption of the Swift SQLite API operations. Given a Swift app, we first scan it for uses of SQLite. We then combine that information with the measured energy profile to compute E-factor, an estimate of the energy consumption of the API usage in an app. To evaluate the usability of E-factor, we have calculated the E-factor of 56 real-world iOS apps. We have also compared the E-factor of 16 versions and 11 methods from 3 of those apps to their hardware-based energy measurements. Our findings show that E-factor positively correlates with the hardware-based energy measurements, indicating that E-factor is a practical estimate to compare the energy consumption difference in API usage across different versions of an app. Developers may also use E-factor to identify excessive energy-consuming methods in their apps and focus on optimizing them. Our approach is most useful in an Integrated Development Environment (IDE) or Continuous Integration (CI) pipeline, where developers receive energy consumption insights within milliseconds of making a code modification.

Software testing helps developers minimize bugs and errors in their code, improving the overall software quality. In 2013, Kochhar et al. analyzed 20,817 software projects in order to study how prevalent the practice of software testing is in open-source projects. They found that projects with more lines of code (LOC) and projects with more developers tend to have more test cases. Additionally, they found a weak positive correlation between the number of test cases and the number of bugs. Since the conclusions of a study might become irrelevant over time because of the latest practices in the relevant fields, in this paper, we investigate if these conclusions remain valid if we re-evaluate Kochhar et al.’s findings on the Java projects that were developed from 2012 to 2021. For evaluation, we use a random sample of 20,000 open-source Java projects each year. Our results show that Kochhar et al.’s conclusions regarding the projects with test cases having more LOC, the weak positive correlation between the number of test cases and authors, and the weak positive correlation between the number of test cases and bugs remain stable until 2021. Our study corroborates Kochhar et al.’s conclusions and helps developers refocus in light of the latest findings regarding the practice of software testing.

The value of teamwork is being recognized by project owners, resulting in an increased acknowledgement of collaboration among developers in software engineering. A good understanding of how developers work together could positively impact software development practices. In this paper, we investigate the collaboration habits of developers in project files by leveraging the World of Code (WoC) dataset and GitHub API. We first identify the collaboration level of developers within the project files, such as the source, test, documentation, and build files, using the Author Cross Entropy (ACE). From the results we find out that test files report the highest degree of collaboration among the developers, perhaps because collaboration is critical to ensure convergence of functionality tests. Furthermore, the source code files show the least degree of collaboration, perhaps because of code ownership and the complexity and difficulty in code modification. Secondly, given the widespread usage of the Python programming language, we investigate the Python code tokens that are more prone to change and collaboration. Our findings offer insights into the specific project files and Python code tokens that developers typically collaborate on in the open-source community. This information can be used by researchers and developers to enhance existing collaboration platforms and tools.

Docker is becoming ubiquitous with containerization for developing and deploying applications. Previous studies have analyzed Dockerfiles that are used to create container images in order to better understand how to improve Docker tooling. These studies obtain Dockerfiles using either Docker Hub or Github. In this paper, we revisit the findings of previous studies using the largest set of Dockerfiles known to date with over 9.4 million unique Dockerfiles found in the World of Code infrastructure spanning from 2013-2020. We contribute a historical view of the Dockerfile format by analyzing the Docker engine changelogs and use the history to enhance our analysis of Dockerfiles. We also reconfirm previous findings of a downward trend in using OS images and an upward trend of using language images. As well, we reconfirm that Dockerfile smell counts are slightly decreasing meaning that Dockerfile authors are likely getting better at following best practices. Based on these findings, it indicates that previous analyses from prior works have been correct in many of their findings and their suggestions to build better tools for Docker image creation are further substantiated.

Single-statement bugs (SStuBs) can have a severe impact on developer productivity. Despite usually being simple and not offering much of a challenge to fix, these bugs may still disturb a developer's workflow and waste precious development time. However, few studies have paid attention to these simple bugs, focusing instead on bugs of any size and complexity. In this study, we explore the occurrence of SStuBs in some of the most popular open-source Python projects on GitHub, while also characterizing their patterns and distribution. We further compare these bugs to SStuBs found in a previous study on Java Maven projects. We find that these Python projects have different SStuB patterns than the ones in Java Maven projects and identify 7 new SStuB patterns. Our results may help uncover the importance of understanding these bugs for the Python programming language, and how developers can handle them more effectively.

Machine learning, a branch of Artificial Intelligence, is now popular in software engineering community and is successfully used for problems like bug prediction, and software development effort estimation. Developers' understanding of machine learning, however, is not clear, and we require investigation to understand what educators should focus on, and how different online programming discussion communities can be more helpful. We conduct a study on Stack Overflow (SO) machine learning related posts using the SOTorrent dataset. We found that some machine learning topics are significantly more discussed than others, and others need more attention. We also found that topic generation with Latent Dirichlet Allocation (LDA) can suggest more appropriate tags that can make a machine learning post more visible and thus can help in receiving immediate feedback from sites like SO.

Organizations like Mozilla, Microsoft, and Apple are flooded with thousands of automated crash reports per day. Although crash reports contain valuable information for debugging, there are often too many for developers to examine individually. Therefore, in industry, crash reports are often automatically grouped together in buckets. Ubuntu's repository contains crashes from hundreds of software systems available with Ubuntu. A variety of crash report bucketing methods are evaluated using data collected by Ubuntu's Apport automated crash reporting system. The trade-off between precision and recall of numerous scalable crash deduplication techniques is explored. A set of criteria that a crash deduplication method must meet is presented and several methods that meet these criteria are evaluated on a new dataset. The evaluations presented in this paper show that using off-the-shelf information retrieval techniques, that were not designed to be used with crash reports, outperform other techniques which are specifically designed for the task of crash bucketing at realistic industrial scales. This research indicates that automated crash bucketing still has a lot of room for improvement, especially in terms of identifier tokenization.

Software energy consumption is a relatively new concern for mobile application developers. Poor energy performance can harm adoption and sales of applications. Unfortunately for the developers, the measurement of software energy consumption is expensive in terms of hardware and difficult in terms of expertise. Many prior models of software energy consumption assume that developers can use hardware instrumentation and thus cannot evaluate software running within emulators or virtual machines. Some prior models require actual energy measurements from the previous versions of applications in order to model the energy consumption of later versions of the same application.

The improvement in battery technology for battery-driven devices is insignificant compared to their computing ability. In spite of the overwhelming advances in processing ability, adoption of sophisticated applications is hindered by the fear of shorter battery life. This is one of the several reasons software developers are becoming conscious of writing energy efficient code.

Developers summarize their changes to code in commit messages. When a message seems "unusual", however, this puts doubt into the quality of the code contained in the commit. We trained n-gram language models and used cross-entropy as an indicator of commit message "unusualness" of over 120,000 commits from open source projects. Build statuses collected from Travis-CI were used as a proxy for code quality. We then compared the distributions of failed and successful commits with regards to the "unusualness" of their commit message. Our analysis yielded significant results when correlating cross-entropy with build status.

Computer musicians are a community of end-user programmers who often use visual programming languages such as Max/MSP or Pure Data to realize their musical compositions. This research study conducts a multifaceted analysis of the software development practices of computer musicians when programming in these visual music-oriented languages. A statistical analysis of project metadata harvested from software repositories hosted on GitHub reveals that in comparison to the general population of software developers, computer musicians' repositories have less commits, less frequent commits, more commits on weekends, yet similar numbers of bug reports and similar numbers of contributing authors. Analysis of source code in these repositories reveals that the vast majority of code can be reconstructed from duplicate fragments. Finally, these results are corroborated by a survey of computer musicians and interviews with individuals in this end-user community. Based on this analysis and feedback from computer musicians we find that there are many avenues where software engineering can be applied to help aid this community of end-user programmers.

Internet Relay Chat (IRC) is a commonly used tool by Open Source developers. Developers use IRC channels to discuss programming related problems, but much of the discussion is irrelevant and off-topic. Essentially if we treat IRC discussions like email messages, and apply spam filtering, we can try to filter out the spam (the off-topic discussions) from the ham (the programming discussions). Yet we need labelled data that unfortunately takes time to curate. To avoid costly cur ration in order to filter out off-topic discussions, we need positive and negative data-sources. On-line discussion forums, such as Stack Overflow, are very effective for solving programming problems. By engaging in open-data, Stack Overflow data becomes a powerful source of labelled text regarding programming. This work shows that we can train classifiers using Stack Overflow posts as positive examples of on-topic programming discussion. You Tube video comments, notorious for their lack of quality, serve as training set of off-topic discussion. By exploiting these datasets, accurate classifiers can be built, tested and evaluated that require very little effort for end-users to deploy and exploit.

Github is a very popular collaborative software-development platform that provides typical source-code management and issue tracking features augmented by strong social-networking features such as following developers and watching projects. These features help ``spread the word'' about individuals and projects, building the reputation of the former and increasing the popularity of the latter. In this paper, we investigate the relation between project popularity and regular, consistent documentation updates. We found strong indicators that consistently popular projects exhibited consistent documentation effort and that this effort tended to attract more documentation collaborators. We also found that frameworks required more documentation effort than libraries to achieve similar adoption success, especially in the initial phase.

A frustrating aspect of software development is that compiler error messages often fail to locate the actual cause of a syntax error. An errant semicolon or brace can result in many errors reported throughout the file. We seek to find the actual source of these syntax errors by relying on the consistency of software: valid source code is usually repetitive and unsurprising. We exploit this consistency by constructing a simple N-gram language model of lexed source code tokens. We implemented an automatic Java syntax-error locator using the corpus of the project itself and evaluated its performance on mutated source code from several projects. Our tool, trained on the past versions of a project, can effectively augment the syntax error locations produced by the native compiler. Thus we provide a methodology and tool that exploits the naturalness of software source code to detect syntax errors alongside the parser.

Green Mining is a field of MSR that studies software energy consumption and relies on software performance data. Unfortunately there is a severe lack of publicly available software power use performance data. This means that green mining researchers must generate this data themselves by writing tests, building multiple revisions of a product, and then running these tests multiple times (10+) for each software revision while measuring power use. Then, they must aggregate these measurements to estimate the energy consumed by the tests for each software revision. This is time consuming and is made more difficult by the constraints of mobile devices and their OSes. In this paper we propose, implement, and demonstrate Green Miner: the first dedicated hardware mining software repositories testbed. The Green Miner physically measures the energy consumption of mobile devices (Android phones) and automates the testing of applications, and the reporting of measurements back to developers and researchers. The Green Miner has already produced valuable results for commercial Android application developers, and has been shown to replicate other power studies' results.

With the advent of mobile computing, the responsibility of software developers to update and ship energy efficient applications has never been more pronounced. Green mining attempts to address this responsibility by examining the impact of software change on energy consumption. One problem with green mining is that power performance data is not readily available, unlike many other forms of MSR research. Green miners have to create tests and run them across numerous versions of a software project because power performance data was either missing or never existed for that particular project. In this paper we describe multiple open green mining datasets used in prior green mining work. The dataset includes numerous power traces and parallel system call and CPU/IO/Memory traces of multiple versions of multiple products. These datasets enable those more interested in data-mining and modeling to work on green mining problems as well.

Bug-tracking and issue-tracking systems tend to be populated with bugs, issues, or tickets written by a wide variety of bug reporters, with different levels of training and knowledge about the system being discussed. Many bug reporters lack the skills, vocabulary, knowledge, or time to efficiently search the issue tracker for similar issues. As a result, issue trackers are often full of duplicate issues and bugs, and bug triaging is time consuming and error prone. Many researchers have approached the bug-deduplication problem using off-the-shelf information-retrieval tools, such as BM25F used by Sun et al. In our work, we extend the state of the art by investigating how contextual information, relying on our prior knowledge of software quality, software architecture, and system-development (LDA) topics, can be exploited to improve bug-deduplication. We demonstrate the effectiveness of our contextual bug-deduplication method on the bug repository of the Android ecosystem. Based on this experience, we conclude that researchers should not ignore the context of software engineering when using IR tools for deduplication.

A project's documentation is the primary source of information for developers using that project. With hundreds of thousands of programming-related questions posted on programming Q&A websites, such as Stack Overflow, we question whether the developer-written documentation provides enough guidance for programmers. In this study, we wanted to know if there are any topics which are inadequately covered by the project documentation. We combined questions from Stack Overflow and documentation from the PHP and Python projects. Then, we applied topic analysis to this data using latent Dirichlet allocation (LDA), and found topics in Stack Overflow that did not overlap the project documentation. We successfully located topics that had deficient project documentation. We also found topics in need of tutorial documentation that were outside of the scope of the PHP or Python projects, such as MySQL and HTML.

In this paper we mine the Android bug tracker repository and study the characteristics of the architectural layers of the Android system. We have identified the locality of the Android bugs in the architectural layers of the its infrastructure, and analysed the bug lifetime patterns in each one of them. Additionally, we mined the bug tracker reporters and classified them according to its social centrality in the Android bug tracker community. We report three interesting findings, firstly while some architectural layers have a diverse interaction of people, attracting not only non-central reporters but highly important ones, other layers are mostly captivating for peripheral actors. Second, we exposed that even the bug lifetime is similar across the architectural layers, some of them have higher bug density and differential percentages of unsolved bugs. Finally, comparing the popularity distribution between layers, we have identified one particular layer that is more important to developers and users alike.

Power consumption is becoming more and more important with the increased popularity of smart-phones, tablets and laptops. The threat of reducing a customer's battery-life now hangs over the software developer who asks, “will this next change be the one that causes my software to drain a customer's battery?” One solution is to detect power consumption regressions by measuring the power usage of tests, but this is time-consuming and often noisy. An alternative is to rely on software metrics that allow us to estimate the impact that a change might have on power consumption thus relieving the developer from expensive testing. This paper presents a general methodology for investigating the impact of software change on power consumption, we relate power consumption to software changes, and then investigate the impact of static OO software metrics on power consumption. We demonstrated that software change can effect power consumption using the Firefox web-browser and the Azureus/Vuze BitTorrent client. We found evidence of a potential relationship between some software metrics and power consumption. In conclusion, we explored the effect of software change on power consumption on two projects; and we provide an initial investigation on the impact of software metrics on power consumption.

Android is an operating system designed specifically for mobile devices. It has a layered architecture. In this paper, we extract Android's concrete layered architecture by analyzing the build dependency relation between Android sub-projects and use it to validate the proposed conceptual architecture. Our experiment shows that Android's concrete architecture conforms to the conceptual architecture. Apart from that, we also show the extracted architecture can help developers and users better understand the Android system and further demonstrate its potential benefits in studying the impact of changes.

Deployed software systems are typically composed of many pieces, not all of which may have been created by the main development team. Often, the provenance of included components -- such as external libraries or cloned source code -- is not clearly stated, and this uncertainty can introduce technical and ethical concerns that make it difficult for system owners and other stakeholders to manage their software assets. In this work, we motivate the need for the recovery of the provenance of software entities by a broad set of techniques that could include signature matching, source code fact extraction, software clone detection, call flow graph matching, string matching, historical analyses, and other techniques. We liken our provenance goals to that of Bertillonage, a simple and approximate forensic analysis technique based on bio-metrics that was developed in 19th century France before the advent of fingerprints. As an example, we have developed a fast, simple, and approximate technique called anchored signature matching for identifying library version information within a given Java application. This technique involves a type of structured signature matching performed against a database of candidates drawn from the Maven2 repository, a 150GB collection of open source Java libraries. An exploratory case study using a proprietary e-commerce Java application illustrates that the approach is both feasible and effective.

Researchers have employed a variety of techniques to extract underlying topics that relate to software development artifacts. Typically, these techniques use semi-unsupervised machine-learning algorithms to suggest candidate word-lists. However, word-lists are difficult to interpret in the absence of meaningful summary labels. Current topic modeling techniques assume manual labelling and do not use domainspecific knowledge to improve, contextualize, or describe results for the developers. We propose a solution: automated labelled topic extraction. Topics are extracted using Latent Dirichlet Allocation (LDA) from commit-log comments recovered from source control systems such as CVS and Bit-Keeper. These topics are given labels from a generalizable cross-project taxonomy, consisting of non-functional requirements. Our approach was evaluated with experiments and case studies on two large-scale RDBMS projects: MySQL and MaxDB. The case studies show that labelled topic extraction can produce appropriate, context-sensitive labels relevant to these projects, which provides fresh insight into their evolving software development activities.

Software readability is a property that influences how easily a given piece of code can be read and understood. Since readability can affect maintainability, quality, etc., programmers are very concerned about the readability of code. If automatic readability checkers could be built, they could be integrated into development tool-chains, and thus continually inform developers about the readability level of the code. Unfortunately, readability is a subjective code property, and not amenable to direct automated measurement. In a recently published study, Buse et al. asked 100 participants to rate code snippets by readability, yielding arguably reliable mean readability scores of each snippet; they then built a fairly complex predictive model for these mean scores using a large, diverse set of directly measurable source code properties. We build on this work: we present a simple, intuitive theory of readability, based on size and code entropy, and show how this theory leads to a much sparser, yet statistically significant, model of the mean readability scores produced in Buse's studies. Our model uses well-known size metrics and Halstead metrics, which are easily extracted using a variety of tools. We argue that this approach provides a more theoretically well-founded, practically usable, approach to readability measurement.

In a young field, such as Mining Software Repositories (MSR), there is always a call for benchmarks so that researchers can compare their results against others. Thus in order to explore and discover the breadth of MSR research, the MSR community has banded together behind the MSR Mining Challenge. The mining challenge allows researchers to demonstrate current working techniques against a common set of repositories or datasets with the express purpose of mining interesting facts from these datasets and then comparing these results against the results from other researchers. This year, 2010, the MSR Mining Challenge has expanded the size of its underlying dataset to include the version control, bug tracker, and mailinglists of the following software distributions and projects: FreeBSD, GNOME Desktop and Debian/Ubuntu. Researchers are asked to look beyond the boundaries of a project and investigate the relationship between the evolution of various programs contained within these software ecosystems. 9 general challenge submissions were submitted, 6 were accepted with a 66% acceptance rate.

Research in the mining of software repositories has frequently ignored commits that include a large number of files (we call these large commits). The main goal of this paper is to understand the rationale behind large commits, and if there is anything we can learn from them. To address this goal we performed a case study that included the manual classification of large commits of nine open source projects. The contributions include a taxonomy of large commits, which are grouped according to their intention. We contrast large commits against small commits and show that large commits are more perfective while small commits are more corrective. These large commits provide us with a window on the development practices of maintenance teams.