Demo Data from VLDB 2026 / reviewers in the wild / expert
Matteo Corsi
dblp:301/3714
· DBLP profile ↗
1ranked-venue papers
1as first author
1since 2021 · last 2024
0009-0001-2862-2616ORCID · reported
Domains — the database's venue-derived domains; a paper can count in several
Databases, data management, data science and information retrieval · 1 · 1 first-author · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Databases, data mining, and information retrieval
1 paper |
Information retrieval · 100% |
Topics — the 3 heaviest of 3, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Information retrieval › evaluation
rank correlation |
0.8 | 1 | 2024 | The Treatment of Ties in Rank-Biased Overlap · SIGIR 2024 |
Information retrieval › similarity measure
rank similarity measure |
0.8 | 1 | 2024 | The Treatment of Ties in Rank-Biased Overlap · SIGIR 2024 |
Information retrieval
retrieval evaluation |
0.8 | 1 | 2024 | The Treatment of Ties in Rank-Biased Overlap · SIGIR 2024 |
Methods — techniques the papers use, named apart from their topics
spearman's correlation · 0.8rank-biased overlap · 0.8kendall's tau · 0.8
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2024 | The Treatment of Ties in Rank-Biased OverlapabstractRank-Biased Overlap (RBO) is a similarity measure for indefinite rankings: it is top-weighted, and can be computed when only a prefix of the rankings is known or when they have only some items in common. It is widely used for instance to analyze differences between search engines by comparing the rankings of documents they retrieve for the same queries. In these situations, though, it is very frequent to find tied documents that have the same score. Unfortunately, the treatment of ties in RBO remains superficial and incomplete, in the sense that it is not clear how to calculate it from the ranking prefixes only. In addition, the existing way of dealing with ties is very different from the one traditionally followed in the field of Statistics, most notably found in rank correlation coefficients such as Kendall's and Spearman's. In this paper we propose a generalized formulation for RBO to handle ties, thanks to which we complete the original definitions by showing how to perform prefix evaluation. We also use it to fully develop two variants that align with the ones found in the Statistics literature: one when there is a reference ranking to compare to, and one when there is not. Overall, these three variants provide researchers with flexibility when comparing rankings with RBO, by clearly determining what ties mean, and how they should be treated. Finally, using both synthetic and TREC data, we demonstrate the use of these new tie-aware RBO measures. We show that the scores may differ substantially from the original tie-unaware RBO measure, where ties had to be broken at random or by arbitrary criteria such as by document ID. Overall, these results evidence the need for a proper account of ties in rank similarity measures such as RBO. Matteo Corsi, Julián Urbano |
SIGIR | 1 |