Code evolution is prevalent in software ecosystems, which can provide many benefits, such as new features, bug fixes, security patches, while still introducing breaking changes that make downstream projects fail to work. Breaking changes cause a lot of effort to both downstream and upstream developers: downstream developers need to adapt to breaking changes and upstream developers are responsible for identifying and documenting them. In the NPM ecosystem, characterized by frequent code changes and a high tolerance for making breaking changes, the effort is larger. For better comprehension of breaking changes in the NPM ecosystem and to enhance breaking change detection tools, we conduct a large-scale empirical study to investigate breaking changes in the NPM ecosystem. We construct a dataset of explicitly documented breaking changes from 381 popular NPM projects. We find that 95.4% of the detected breaking changes can be covered by developers’ documentation, and 19% of the breaking changes cannot be detected by regression testing. Then in the process of investigating source code of our collected breaking changes, we yield a taxonomy of JavaScript- and TypeScript-specific syntactic breaking changes and a taxonomy of major types of behavioral breaking changes. Additionally, we investigate the reasons why developers make breaking changes in NPM and find three major reasons, i.e., to reduce code redundancy, to improve identifier names, and to improve API design, and each category contains several sub-items. We provide actionable implications for future research, e.g., automatic naming and renaming techniques should be applied in JavaScript projects to improve identifier names, future research can try to detect more types of behavioral breaking changes. By presenting the implications, we also discuss the weakness of automatic renaming and breaking change detection approaches, such as the lack of support for public identifiers and various types of breaking changes.

Automated program repair (APR) has been gaining ground with substantial effort devoted to the area, opening up many challenges and opportunities. One such challenge is that the state-of-the-art repair techniques often resort to incomplete specifications, e.g., test cases that witness buggy behavior, to generate repairs. In practice, bug-exposing test cases are often available when: (1) developers, at the same time of (or after) submitting bug fixes, create the tests to assure the correctness of the fixes, or (2) regression errors occur. The former case – a scenario commonly used for creating popular bug datasets – however, may not be suitable to assess how APR performs in the wild. Since developers already know where and how to fix the bugs, tests created in this case may encapsulate knowledge gained only after bugs are fixed. Thus, more effort is needed to create datasets for more realistically evaluating APR.We address this challenge by creating a dataset focusing on bugs identified via continuous integration (CI) failures – a special case of regression errors – wherein bugs happen when the program after being changed is re-executed on the existing test suite. We argue that CI failures, wherein bug-exposing tests are created before bug fixes and thus assume no prior knowledge of developers on the bugs to be involved, are more realistic for evaluating APR. Toward this end, we curated 102 CI failures from 40 popular real-world software on GitHub. We demonstrate various features and the usefulness of the dataset via an evaluation of five well-known APR techniques, namely GenProg, Kali, Cardumen, RsRepair and Arja. We subsequently discuss several findings and implications for future APR studies. Overall, experiment results show that our dataset is complementary to existing datasets such as Defect4J in realistic evaluations of APR.

Code review is an important activity in software development, which offers benefits such as improving code quality, reducing defects and distributing knowledge. Tencent, as a giant company, hosts a great number of proprietary software projects that are only open to specific internal developers. Since these proprietary projects receive up to 100,000 of newly submitted code changes per month, it is extremely needed to automatically recommend code reviewers. To this end, we first conduct an empirical study on a large scale of proprietary projects from Tencent, to understand their characteristics and how code reviewer recommendation approaches work on them. Based on the derived findings and implications, we propose a new approach named Camp that recommends reviewers by considering their collaboration and expertise in multiple projects, to fit the context of proprietary software development. The evaluation results show that Camp can achieve higher scores on proprietary projects across most metrics than other state-of-the-art approaches, i.e., Revfinder, CHREV, Tie and Comment Network and produce acceptable performance scores for more projects. In addition, we discuss the possible directions of code reviewer recommendation.