Demo Data from VLDB 2026 / reviewers in the wild / expert
Matthew Dalton
dblp:433/9568
· DBLP profile ↗
1ranked-venue papers
0as first author
1since 2021 · last 2026
0009-0004-6347-3267ORCID · reported
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021Databases, data management, data science and information retrieval · 1 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Interdisciplinary, comprehensive, and emerging computing
1 paper |
Computational social science and digital humanities · 100% | |
| Artificial intelligence
1 paper |
Probabilistic and Bayesian machine learning · 100% |
Topics — the 3 heaviest of 3, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Computational social science and digital humanities › online controlled experiments
a/b testing |
1.0 | 1 | 2026 | Case Study: Learning Robust, Long-run Surrogate Metrics with Modeling and Instrumental Variables · KDD (1) 2026 |
Machine learning › Probabilistic and Bayesian machine learning
causal inference |
0.3 | 1 | 2026 | Case Study: Learning Robust, Long-run Surrogate Metrics with Modeling and Instrumental Variables · KDD (1) 2026 |
Machine learning › Probabilistic and Bayesian machine learning › causal inference
instrumental variable |
0.3 | 1 | 2026 | Case Study: Learning Robust, Long-run Surrogate Metrics with Modeling and Instrumental Variables · KDD (1) 2026 |
Methods — techniques the papers use, named apart from their topics
non-parametric machine learning · 2.0causal inference · 2.0instrumental variables · 1.0instrumental variable · 1.0
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | Case Study: Learning Robust, Long-run Surrogate Metrics with Modeling and Instrumental VariablesabstractWhen developing product changes to improve long-run user outcomes, surrogate metrics are a powerful tool to help accelerate decision making. Randomized, online A/B testing has become the gold standard for measuring long-run outcomes, but waiting for long-run outcomes to materialize in a test greatly reduces iteration speed. To accelerate decision making, we instead can use surrogate metrics that predict the long-run outcome more quickly. However, finding accurate, robust surrogate metrics is challenging in practice. Existing methods propose finding candidate surrogate metrics in various ways, such as re-analyzing past A/B experiments where candidates strongly correlate with the long-run outcome. However, these methods face practical limitations such as being affected by hidden confounders or needing to rely on outdated, historical data. We developed a new approach where we learned user-level surrogate metrics with non-parametric machine learning and instrumental variable (IV) analysis. We trained surrogate metrics for two key, long-run business metrics, BV1 and BV2, in classification and regression settings, respectively. We deployed each surrogate model such that it would be available as a metric to use in all A/B tests in the target product surface. Our surrogate models predicted each user's long-run outcomes more accurately than baseline heuristics (e.g., our BV1 surrogate model's AUC was 0.93 vs. 0.79 for the baseline heuristic). At the A/B test level, our surrogate metrics showed greater than 0.9 Pearson correlation with the long-run outcome across A/B tests and achieved at least a 50% improvement in accuracy over other baselines. Our surrogate metrics were >30x more predictive of whether an A/B test's had consistent post-launch effects. Finally, we provide evidence of surrogacy for our metrics where we show in multiple independent A/B tests that directly optimizing for our surrogate metrics drove changes in the long-run target outcome. Our results demonstrate that combining machine learning and causal inference leads not only to accurate surrogate metrics but also evidence they can act like true surrogates in practice. Daniel Day, Matthew Dalton, Samuel Daulton, Benjamin Letham, Aojia Zhao, Eytan Bakshy |
KDD (1) | 2 |