Modern compilers provide improved diagnostics, performance, and security. The industry lacks the data and tools to estimate the cost to upgrade a compiler toolchain for complex projects. A knowledge base mined from defect databases, mailing lists, experience reports, commits, and grey literature will improve the planning process.

In 2024, the Linux kernel became its own Common Vulnerabilities and Exposures (CVE) Numbering Authority (CNA), formalizing how kernel vulnerabilities are identified and tracked. We analyze the anatomy and dynamics of kernel CVEs using metadata, associated commits, and patch latency to understand what drives patching. Results show that severity and Common Vulnerability Scoring System (CVSS) metrics have a negligible association with patch latency, whereas kernel recency is a reasonable predictor in survival models. Kernel developers fix newer kernels sooner, while older ones retain unresolved CVEs. Commits introducing vulnerabilities are typically broader and more complex than their fixes, though often only approximate reconstructions of development history. The Linux kernel remains a unique open-source project—its CVE process is no exception.

In February 2024, the Linux kernel became a CNA (CVE numbering authority). The number of CVEs issued for the kernel increased by an order of magnitude. This increase places additional patching demands on kernel vendors and software companies maintaining custom Linux kernels. The industry needs the software analytics research community’s help to understand the patch velocity, develop the prediction models, and estimate the effort required to patch the kernel.

Abstract Increasing code velocity is a common goal for a variety of software projects. The efficiency of the code review process significantly impacts how fast the code gets merged into the final product and reaches the customers. We conducted a qualitative survey to study the code velocity-related beliefs and practices in place. We analyzed 75 completed surveys from SurIndustryDevs participants from the industry and 36 from the open-source community. Our critical findings are (a) the industry and open-source community hold a similar set of beliefs, (b) quick reaction time is of utmost importance and applies to the tooling infrastructure and the behavior of other engineers, (c) time-to-merge is the essential code review metric to improve, (d) engineers are divided about the benefits of increased code velocity for their career growth, (e) the controlled application of the commit-then-review model can increase code velocity. Our study supports the continued need to invest in and improve code velocity regardless of the underlying organizational ecosystem.

Build logs from a variety of Continuous Integration (CI) systems contain temporal data about the presence and distribution of compiler warnings. Results from the analysis and mining of that data will indicate what warnings engineers find useful and fix, or continuously ignore. The findings will include resolution times and resolution types for different warning categories. That data will help compiler developers adjust the warning levels according to the ground truth, clarify the diagnostic messages, and improve the non-actionable warnings. The empirical findings will also help engineers to decide what warnings are worth fixing and which ones are not.

This paper investigates how the duration of various code review periods changes over a projects’ lifetime. We study four open-source software (OSS) projects: Blender, FreeBSD, LLVM, and Mozilla. We mine and analyze the characteristics of 283,235 code reviews that cover, on average, seven years’ worth of development. Our main conclusion is that neither the passage of time or the project’s size impact code velocity. We find that (a) the duration of various code review periods (time-to-first-response, time-to-accept, and time-to-merge) for FreeBSD, LLVM, and Mozilla either becomes shorter or stays the same; no directional trend is present for Blender, (b) an increase in the size of the code bases (annually 3–17%) does not accompany a decrease in code velocity, and (c) for FreeBSD, LLVM, and Mozilla, the 30-day moving median stays in a fixed range for time-to-merge. These findings do not change with variabilities in code churn metrics, such as the number of commits or distinct authors of code changes.

Satisfactory software performance is essential for the adoption and the success of a product. In organizations that follow traditional software development models (e.g., waterfall), Software Performance Engineering (SPE) involves time-consuming experimental modeling and performance testing outside the actual production environment. Such existing SPE methods, however, are not optimized for environments utilizing Continuous Integration (CI) and Continuous Delivery (CD) that result in high frequency and high volume of code changes. We present a summary of lessons learned and propose improvements to the SPE process in the context of CI/CD. Our findings are based on SPE work on two products conducted over 5 years at a major online services company. We find that

Engineers consider the presence of dead code as an undesirable attribute of the code base. The industry lacks methods to quantify the benefits of deleting dead code efficiently. The current approach utilizes a simplistic metric that uses the lines of code (LOC) deleted as a proxy to estimate the benefit gained. However, not all LOC are equal. The research community can support the industry and propose methods and metrics that can help to (a) determine the priority order for dead code removal, and (b) quantify the benefits of dead code removal. Improved metrics can result in a more objective ranking of dead code deletion efforts when compared to other competing tasks.

Using custom memory allocators is an efficient performance optimization technique. However, dependency on a custom allocator can introduce several maintenance-related issues. We present lessons learned from the industry and provide critical guidance for using custom memory allocators and enumerate various challenges associated with integrating them. These recommendations are based on years of experience incorporating custom allocators into different industrial software projects.

Studies on software evolution explore code churn and code velocity at the abstraction level of a company or an entire project. We argue that this approach misses the differences among abstractions layers and subsystems of large projects. We conduct a case study on four BSD family operating systems: DragonFlyBSD, FreeBSD, NetBSD, and OpenBSD, to investigate the evolution of code churn and code velocity across kernel and non-kernel code. We mine commits for characteristics such as annual growth rate, commit types, change type ratio, and size taxonomy, indicating code churn. Likewise, we investigate code velocity in terms of code review periods, i.e., time-to-first-response, time-to-accept, and time-to-merge.Our study provides evidence that software evolves differently at abstraction layers: kernel and non-kernel. The study finds similarities in the code base growth rate and distribution of commit types (neutral, additive, and subtractive) across BSD subsystems, however, (a) most commits contain either kernel or non-kernel code, (b) kernel commits are larger than non-kernel commits, and (c) code reviews for kernel code take longer than non-kernel code.

Increasing code velocity (or the speed with which code changes are reviewed and merged) is integral to speeding up development and contributes to the work satisfaction of engineers. While factors affecting code change acceptance have been investigated in the past, solutions to decrease the code review lifetime are less understood. This study investigates the code review process to quantify delays and investigate opportunities to potentially increase code velocity. We study the temporal characteristics of half a million code reviews hosted on Gerrit and Phabricator, starting from the first response, to a decision to accept or reject the changes, and until the changes are merged into a target branch. We identified two types of time delays: (a) the wait time from the proposal of code changes until first response, and (b) the wait time between acceptance and merging. Our study indicates that reducing the time between acceptance and merging has the potential to speed up Phabricator code reviews by 29--63%. Small code changes and changes made by authors with a large number of previously accepted code reviews have a higher chance of being immediately accepted, without code review iterations. Our analysis suggests that switching from manual to automatic merges can help increase code velocity.

Phabricator is a modern code collaboration tool used by popular projects like FreeBSD and Mozilla. However, unlike the other well-known code review environments, such as Gerrit or GitHub, there is no readily accessible public code review dataset for Phabricator. This paper describes our experience mining code reviews from five different projects that use Phabricator (Blender, FreeBSD, KDE, LLVM, and Mozilla). We discuss the challenges associated with the data retrieval process and our solutions, resulting in a dataset with details regarding 317,476 Phabricator code reviews. Our dataset1 is available in both JSON and MySQL database dump formats. The dataset enables analyses of the history of code reviews at a more granular level than other platforms. In addition, given that the projects we mined are publicly accessible via the Conduit API [18], our dataset can be used as a foundation to fetch additional details and insights.

Code velocity, or the speed with which code changes are integrated into a production environment, plays a crucial role in Continuous Integration and Continuous Deployment. Many studies report factors influencing code velocity. However, solutions to increase code velocity are unclear. Meanwhile, the industry continues to issue guidelines on "ideal" code change size, believing it increases code velocity despite lacking evidence validating the practice. Surprisingly, this fundamental question has not been studied to date. This study investigates the practicality of improving code velocity by optimizing pull request size and composition (ratio of insertions, deletions, and modifications).

Instituting an automated unit testing practice across a large software development team can be technically challenging and time consuming. As a result, teams may question the economic value of instituting such a practice. One large Microsoft team consisting of 32 developers transitioned from ad hoc and individualized unit testing practices to the utilization of the NUnit automated unit testing framework by all members of the team. These automated unit tests were typically written by developers after they completed coding functionality, approximately every two to three days. After a period of one year of utilizing this automated unit testing practice on Version 2 of a product, the team realized a 20.9% decrease in test defects at a cost of approximately 30% more development time relative to Version 1 of the product. The product also had a relative decrease in defects found by customers during the first two years of field use. Comparatively, other industrial teams have experienced larger decreases in defects when automated unit tests are written iteratively, as is done with the test driven development practice, for a similar time increase. These results indicate automated unit testing is beneficial but additional quality improvements may be realized if the tests are written iteratively.

Code churn, the amount of code change taking place within a software unit over time, has been correlated with fault-proneness in software systems. We investigate the use of code churn and static metrics collected at regular time intervals during the development cycle to predict faults in an iterative, in-process manner. We collected 159 churn and structure metrics from six, four-month snapshots of a 1 million LOC Microsoft product. The number of software faults fixed during each period is recorded per binary module. Using stepwise logistic regression, we create a prediction model to identify fault-prone binaries using three parameters: code churn (the number of new and changed blocks); class Fan In and class Fan Out (normalized by lines of code). The iteratively-built model is 80.0% accurate at predicting fault-prone and non-fault-prone binaries. These fault-prediction models have the advantage of allowing the engineers to observe how their fault-prediction profile evolves over time.

The use of assertions in software development is thought to help produce quality software. Unfortunately, there is scant empirical evidence in commercial software systems for this argument to date. This paper presents an empirical case study of two commercial software components at Microsoft Corporation. The developers of these components systematically employed assertions, which allowed us to investigate the relationship between software assertions and code quality. We also compare the efficacy of assertions against that of popular bug finding techniques like source code static analysis tools. We observe from our case study that with an increase in the assertion density in a file there is a statistically significant decrease in fault density. Further, the usage of software assertions in these components found a large percentage of the faults in the bug database