Demo Data from EDBT 2026 / reviewers in the wild / expert
Jeff Omhover
dblp:240/9448
· DBLP profile ↗
2ranked-venue papers
0as first author
1since 2021 · last 2023
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 2 · 1 since 2021Databases, data management, data science and information retrieval · 1Graphics, computer vision, multimedia, augmented reality and games · 1 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Databases, data mining, and information retrieval
1 paper |
Data mining · 50% Information retrieval · 50% |
Topics — the 2 heaviest of 2, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Information retrieval › evaluation › online evaluation
a/b testing |
0.4 | 1 | 2019 | Diagnosing Sample Ratio Mismatch in Online Controlled Experiments: A Taxonomy and Rules of Thumb for Practitioners · KDD 2019 |
Data mining › causal inference
online controlled experiments |
0.4 | 1 | 2019 | Diagnosing Sample Ratio Mismatch in Online Controlled Experiments: A Taxonomy and Rules of Thumb for Practitioners · KDD 2019 |
Methods — techniques the papers use, named apart from their topics
statistical testing · 0.4
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2023 | Federated Learning Toolkit with Voice-based User Verification Demo
Prathamesh Mandke, Rachel Oberst, Matthias Reisser, Avijit Chakraborty, Christos Louizos, Joseph B. Soriaga, Daniel Madrigal 0001, Andre Manoel, Nalin Singal, Jeff Omhover, Robert Sim |
INTERSPEECH | 10 |
| 2019 | Diagnosing Sample Ratio Mismatch in Online Controlled Experiments: A Taxonomy and Rules of Thumb for PractitionersabstractAccurately learning what delivers value to customers is difficult. Online Controlled Experiments (OCEs), aka A/B tests, are becoming a standard operating procedure in software companies to address this challenge as they can detect small causal changes in user behavior due to product modifications (e.g. new features). However, like any data analysis method, OCEs are sensitive to trustworthiness and data quality issues which, if go unaddressed or unnoticed, may result in making wrong decisions. One of the most useful indicators of a variety of data quality issues is a Sample Ratio Mismatch (SRM) ? the situation when the observed sample ratio in the experiment is different from the expected. Just like fever is a symptom for multiple types of illness, an SRM is a symptom for a variety of data quality issues. While a simple statistical check is used to detect an SRM, correctly identifying the root cause and preventing it from happening in the future is often extremely challenging and time consuming. Ignoring the SRM without knowing the root cause may result in a bad product modification appearing to be good and getting shipped to users, or vice versa. The goal of this paper is to make diagnosing, fixing, and preventing SRMs easier. Based on our experience of running OCEs in four different software companies in over 25 different products used by hundreds of millions of users worldwide, we have derived a taxonomy for different types of SRMs. We share examples, detection guidelines, and best practices for preventing SRMs of each type. We hope that the lessons and practical tips we describe in this paper will speed up SRM investigations and prevent some of them. Ultimately, this should lead to improved decision making based on trustworthy experiment analysis. Aleksander Fabijan, Jayant Gupchup, Somit Gupta, Jeff Omhover, Lukas Vermeer, Pavel A. Dmitriev |
KDD | 4 |