A/B testing is a powerful tool for a company to make informed decisions about their services and products. A limitation of A/B tests is that they do not easily extend to measure post-experiment (long-term) differences. In this talk, we study a different approach inspired by recent advances in off-policy evaluation in reinforcement learning (RL). The basic RL approach assumes customer behavior follows a stationary Markovian process, and estimates the average engagement metric when the process reaches the steady state. However, in realistic scenarios, the stationary assumption is often violated due to weekly variations and seasonality effects. To tackle this challenge, we propose a variation by relaxing the stationary assumption. We empirically tested both stationary and nonstationary approaches in a synthetic dataset and an online store dataset.

Twenty years ago, Carol Dweck published a seminal article showing that students with a fixed mindset underachieve compared to students with a growth mindset. Many existing approaches promote a growth mindset by teaching about neuroplasticity, but include little interactivity and dynamism -- two important aspects in neuroscience education. We present Rewire, a web-based game that introduces students to neuroplasticity and growth mindset through constructionist approaches. Learners build neural networks using principles from neuroscience, such as Hebbian learning and myelination. In the process, they build an intuition for how the brain changes and grows through targeted effort and practice. In this paper, we reflect on the design mechanics that allow students to discover neuroscience principles, and the game's implications for future work in teaching students about growth mindset.

People often complete tasks and to-dos not only for themselves but also for others in their household. In this work, we examine how household members share and accomplish errands both individually and together. We conducted a three-week diary study with eight households to understand the types of errands that family members and roommates share with each other. We explore their motivations for offering and requesting help to complete their errands and the variety of methods for doing so. Our findings reveal when participants sometimes face challenges completing their errands, and how household members request and receive help. We learned that the cooperative performance of errands is typically dependent on household members' location, availability, and capability. Using these findings, we discuss design opportunities for cooperative errands sharing systems that can assist households.