Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Daniel Day

dblp:16/9633 · DBLP profile ↗
← Back
1ranked-venue papers
1as first author
1since 2021 · last 2026
0009-0006-8875-6487ORCID · reported

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 1 · 1 first-author · 1 since 2021Databases, data management, data science and information retrieval · 1 · 1 first-author · 1 since 2021

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Interdisciplinary, comprehensive, and emerging computing
1 paper
Computational social science and digital humanities · 100%
Artificial intelligence
1 paper
Probabilistic and Bayesian machine learning · 100%

Topics — the 3 heaviest of 3, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Computational social science and digital humanities › online controlled experiments
a/b testing
1.012026
Case Study: Learning Robust, Long-run Surrogate Metrics with Modeling and Instrumental Variables · KDD (1) 2026
Machine learning › Probabilistic and Bayesian machine learning
causal inference
0.312026
Case Study: Learning Robust, Long-run Surrogate Metrics with Modeling and Instrumental Variables · KDD (1) 2026
Machine learning › Probabilistic and Bayesian machine learning › causal inference
instrumental variable
0.312026
Case Study: Learning Robust, Long-run Surrogate Metrics with Modeling and Instrumental Variables · KDD (1) 2026

Methods — techniques the papers use, named apart from their topics

non-parametric machine learning · 2.0causal inference · 2.0instrumental variables · 1.0instrumental variable · 1.0
YearPublicationVenuePosition
2026 Case Study: Learning Robust, Long-run Surrogate Metrics with Modeling and Instrumental Variables
abstract
When developing product changes to improve long-run user outcomes, surrogate metrics are a powerful tool to help accelerate decision making. Randomized, online A/B testing has become the gold standard for measuring long-run outcomes, but waiting for long-run outcomes to materialize in a test greatly reduces iteration speed. To accelerate decision making, we instead can use surrogate metrics that predict the long-run outcome more quickly. However, finding accurate, robust surrogate metrics is challenging in practice. Existing methods propose finding candidate surrogate metrics in various ways, such as re-analyzing past A/B experiments where candidates strongly correlate with the long-run outcome. However, these methods face practical limitations such as being affected by hidden confounders or needing to rely on outdated, historical data. We developed a new approach where we learned user-level surrogate metrics with non-parametric machine learning and instrumental variable (IV) analysis. We trained surrogate metrics for two key, long-run business metrics, BV1 and BV2, in classification and regression settings, respectively. We deployed each surrogate model such that it would be available as a metric to use in all A/B tests in the target product surface. Our surrogate models predicted each user's long-run outcomes more accurately than baseline heuristics (e.g., our BV1 surrogate model's AUC was 0.93 vs. 0.79 for the baseline heuristic). At the A/B test level, our surrogate metrics showed greater than 0.9 Pearson correlation with the long-run outcome across A/B tests and achieved at least a 50% improvement in accuracy over other baselines. Our surrogate metrics were >30x more predictive of whether an A/B test's had consistent post-launch effects. Finally, we provide evidence of surrogacy for our metrics where we show in multiple independent A/B tests that directly optimizing for our surrogate metrics drove changes in the long-run target outcome. Our results demonstrate that combining machine learning and causal inference leads not only to accurate surrogate metrics but also evidence they can act like true surrogates in practice.
Daniel Day, Matthew Dalton, Samuel Daulton, Benjamin Letham, Aojia Zhao, Eytan Bakshy
KDD (1)1