Pull request (PR) descriptions generated by AI coding agents are the primary channel for communicating code changes to human reviewers. However, the alignment between these messages and the actual changes remains unexplored, raising concerns about the trustworthiness of AI agents. To fill this gap, we analyzed 23,247 agentic PRs across five agents using PR message-code inconsistency (PR-MCI). We contributed 974 manually annotated PRs, found 406 PRs (1.7%) exhibited high PR-MCI, and identified eight PR-MCI types, revealing that descriptions claim unimplemented changes was the most common issue (45.4%). Statistical tests confirmed that high-MCI PRs had 51.7% lower acceptance rates (28.3% vs. 80.0%) and took 3.5 × longer to merge (55.8 vs. 16.0 hours). Our findings suggest that unreliable PR descriptions undermine trust in AI agents, highlighting the need for PR-MCI verification mechanisms and improved PR generation to enable trustworthy human-AI collaboration.

Abstract The ever‐growing open‐source software tools in different domains increase the difficulty of software selection from the end‐users perspective. The process of evaluating, comparing, and selecting open‐source solutions is far from trivial. Especially, when additional requirements need to be considered, the existing methodologies will fail to adapt to the new tasks. The objective of this study is to present a solution for dealing with this issue. A novel approach, CTHP, is presented for the evaluation and selection of open‐source software in the Bioinformatics domain. First, the ISO 25010 quality model is chosen as the basis. This model is customised according to the special characteristics of the Bioinformatics applications. The customisation is done by extracting the quality factors from the Bioinformatics applications, weighting these factors from the viewpoints of both developers and end‐users, and adding them to the model. After that, Three‐way Decision and Delphi Hierarchy Process are integrated to assist in the selection for adoption. Finally, as a case study, the proposed approach is applied to assist the decision‐making process of two popular natural language processing frameworks in the Bioinformatics area. Our study is a valuable contribution since it provides a systematic way to document the decision‐making process and help the researchers and practitioners of Bioinformatics to make better decisions among the alternatives.

Abstract Recent studies conducted on the single releases of multiple software products showed that dependencies concentrate on smaller modules, that is, smaller modules have more dependencies per source line of code. This phenomenon, called the Theory of Relative Dependency, explains why some earlier studies reported that smaller modules were proportionally more defect prone. It is important to test the Theory of Relative Dependency from multiple perspectives so that it can be used as an explanatory argument when garnering organizational support to give a higher quality assurance (QA) priority to smaller modules. In this study, we test the validity of this theory from an evolutionary perspective by examining the consecutive releases of a number of software products. Dependencies do concentrate over smaller modules regardless of the product age. Furthermore, continuous refactoring efforts are associated with increasing concentration of dependencies on smaller modules over product lifetime. Based on the consistent results, software managers and developers should consider giving a higher QA priority to smaller modules over the lifetime of a software product. In the projects where refactoring is adopted continuously, the QA priority on smaller modules should be further increased as the software product ages. Copyright © 2016 John Wiley & Sons, Ltd.