Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Huizhi Xie

dblp:183/9920 · DBLP profile ↗
← Back
3ranked-venue papers
1as first author
1since 2021 · last 2021
—ORCID · unresolved

Domains — the database's venue-derived domains; a paper can count in several

Databases, data management, data science and information retrieval · 3 · 1 first-author · 1 since 2021Artificial intelligence and machine learning · 2 · 1 first-author

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Software engineering, system software, and programming languages
1 paper
Empirical software engineering · 100%

Topics — the 1 heaviest of 1, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Empirical software engineering › controlled experiment
online controlled experiments
0.212016
Improving the Sensitivity of Online Controlled Experiments: Case Studies at Netflix · KDD 2016

Methods — techniques the papers use, named apart from their topics

stratified sampling · 0.2post stratification · 0.2CUPED · 0.2
YearPublicationVenuePosition
2021 LinkLouvain: Link-Aware A/B Testing and Its Application on Online Marketing Campaign
Tianchi Cai, Daxi Cheng, Lihong Gu, Huizhi Xie, Zhiqiang Zhang 0012, Xiaodong Zeng, Jinjie Gu
DASFAA (3)6
2020 Auxiliary Decision-making for Controlled Experiments based on Mid-term Treatment Effect Prediction: Applications in Ant Financial's Offline-payment Business
Huizhi Xie
DATA2
2016 Improving the Sensitivity of Online Controlled Experiments: Case Studies at Netflix
abstract
Controlled experiments are widely regarded as the most scientific way to establish a true causal relationship between product changes and their impact on business metrics. Many technology companies rely on such experiments as their main data-driven decision-making tool. The sensitivity of a controlled experiment refers to its ability to detect differences in business metrics due to product changes. At Netflix, with tens of millions of users, increasing the sensitivity of controlled experiments is critical as failure to detect a small effect, either positive or negative, can have a substantial revenue impact. This paper focuses on methods to increase sensitivity by reducing the sampling variance of business metrics. We define Netflix business metrics and share context around the critical need for improved sensitivity. We review popular variance reduction techniques that are broadly applicable to any type of controlled experiment and metric. We describe an innovative implementation of stratified sampling at Netflix where users are assigned to experiments in real time and discuss some surprising challenges with the implementation. We conduct case studies to compare these variance reduction techniques on a few Netflix datasets. Based on the empirical results, we recommend to use post-assignment variance reduction techniques such as post stratification and CUPED instead of at-assignment variance reduction techniques such as stratified sampling in large-scale controlled experiments.
Huizhi Xie, Juliette Aurisset
KDD1