Demo Data from SIGIR 2026 / reviewers in the wild / expert
Evgeny Pasynkov
dblp:270/6393
· DBLP profile ↗
1ranked-venue papers
0as first author
0since 2021 · last 2020
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Software engineering, systems software and programming languages · 1
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Software engineering, system software, and programming languages
1 paper |
Software maintenance and evolution · 67% Empirical software engineering · 33% | |
| Databases, data mining, and information retrieval
1 paper |
Recommender systems · 50% Information retrieval · 50% |
Topics — the 5 heaviest of 5, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Software maintenance and evolution
code review |
0.4 | 1 | 2020 | Does Reviewer Recommendation Help Developers? · IEEE Trans. Software Eng. 2020 |
Empirical software engineering
developer studies |
0.4 | 1 | 2020 | Does Reviewer Recommendation Help Developers? · IEEE Trans. Software Eng. 2020 |
Software maintenance and evolution › code review
reviewer assignment |
0.4 | 1 | 2020 | Does Reviewer Recommendation Help Developers? · IEEE Trans. Software Eng. 2020 |
Recommender systems
recommender system evaluation |
0.1 | 1 | 2020 | Does Reviewer Recommendation Help Developers? · IEEE Trans. Software Eng. 2020 |
Information retrieval › evaluation
user-oriented evaluation |
0.1 | 1 | 2020 | Does Reviewer Recommendation Help Developers? · IEEE Trans. Software Eng. 2020 |
Methods — techniques the papers use, named apart from their topics
survey · 0.9interviews · 0.9in vivo evaluation · 0.9
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2020 | Does Reviewer Recommendation Help Developers?abstractSelecting reviewers for code changes is a critical step for an efficient code review process. Recent studies propose automated reviewer recommendation algorithms to support developers in this task. However, the evaluation of recommendation algorithms, when done apart from their target systems and users (i.e., code review tools and change authors), leaves out important aspects: perception of recommendations, influence of recommendations on human choices, and their effect on user experience. This study is the first to evaluate a reviewer recommender in vivo. We compare historical reviewers and recommendations for over 21,000 code reviews performed with a deployed recommender in a company environment and set out to measure the influence of recommendations on users' choices, along with other performance metrics. Having found no evidence of influence, we turn to the users of the recommender. Through interviews and a survey we find that, though perceived as relevant, reviewer recommendations rarely provide additional value for the respondents. We confirm this finding with a larger study at another company. The confirmation of this finding brings up a case for more user-centric approaches to designing and evaluating the recommenders. Finally, we investigate information needs of developers during reviewer selection and discuss promising directions for the next generation of reviewer recommendation tools. Preprint: https://doi.org/10.5281/zenodo.1404814. Vladimir Kovalenko, Nava Tintarev, Evgeny Pasynkov, Christian Bird, Alberto Bacchelli |
IEEE Trans. Software Eng. | 3 |