Demo Data from SIGIR 2026 / reviewers in the wild / expert
Théo Sourget
dblp:362/6588
· DBLP profile ↗
1ranked-venue papers
0as first author
1since 2021 · last 2024
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Interdisciplinary, comprehensive, and emerging computing
1 paper |
Medical and health informatics · 100% | |
| Databases, data mining, and information retrieval
1 paper |
Machine learning and data management · 100% |
Topics — the 2 heaviest of 3, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Medical and health informatics › medical imaging › medical image data management
medical imaging dataset curation |
0.8 | 1 | 2024 | Copycats: the many lives of a publicly available medical imaging dataset · NeurIPS 2024 |
Machine learning and data management
data management for machine learning |
0.2 | 1 | 2024 | Copycats: the many lives of a publicly available medical imaging dataset · NeurIPS 2024 |
Methods — techniques the papers use, named apart from their topics
metadata analysis · 1.5dataset audit · 1.5
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2024 | Copycats: the many lives of a publicly available medical imaging datasetabstractMedical Imaging (MI) datasets are fundamental to artificial intelligence in healthcare. The accuracy, robustness, and fairness of diagnostic algorithms depend on the data (and its quality) used to train and evaluate the models. MI datasets used to be proprietary, but have become increasingly available to the public, including on community-contributed platforms (CCPs) like Kaggle or HuggingFace. While open data is important to enhance the redistribution of data's public value, we find that the current CCP governance model fails to uphold the quality needed and recommended practices for sharing, documenting, and evaluating datasets. In this paper, we conduct an analysis of publicly available machine learning datasets on CCPs, discussing datasets' context, and identifying limitations and gaps in the current CCP landscape. We highlight differences between MI and computer vision datasets, particularly in the potentially harmful downstream effects from poor adoption of recommended dataset management practices. We compare the analyzed datasets across several dimensions, including data sharing, data documentation, and maintenance. We find vague licenses, lack of persistent identifiers and storage, duplicates, and missing metadata, with differences between the platforms. Our research contributes to efforts in responsible data curation and AI algorithms for healthcare. Amelia Jiménez-Sánchez, Natalia Rozalia Avlona, Dovile Juodelyte, Théo Sourget, Caroline Vang-Larsen, Anna Rogers, Hubert Dariusz Zajac, Veronika Cheplygina |
NeurIPS | 4 |