An increasingly important aspect of designing recommender systems involves considering how recommendations will influence consumer choices. This paper addresses this issue by introducing a method for collecting user beliefs about un-experienced goods – a critical predictor of choice behavior. We implemented this method on the MovieLens platform, resulting in a rich dataset that combines user ratings, beliefs, and observed recommendations. We document challenges to such data collection, including selection bias in response and limited coverage of the product space. This unique resource empowers researchers to delve deeper into user behavior and analyze user choices absent recommendations, measure the effectiveness of recommendations, and prototype algorithms that leverage user belief data, ultimately leading to more impactful recommender systems. The dataset can be found at https://grouplens.org/datasets/movielens/ml_belief_2024/.

In this paper I report the results of an experiment where I continuously monitor how participants spend time on digital services and shut off their access to Instagram or YouTube on their phones for 1 or 2 weeks. I use the resulting data on how participants substitute their time during and after the restrictions in order to uncover a rich picture of the demand for social media and entertainment applications. I illustrate how the estimated substitution patterns can be used to guide questions of market definition that have troubled regulators.

We conduct a 6 month field experiment on a movie-recommendation platform to identify if and how recommendation systems affect consumption. We use within-consumer randomization at the good level and elicit beliefs about unconsumed goods to disentangle exposure from informational effects. We have three experimental groups: (a) control, (b) exposed, and (c) recommended + exposed goods where only goods in (c) are recommended and we elicit beliefs about goods in (b) and (c). Comparing across these treatment arms we find recommendations increase consumption beyond its role in exposing goods to consumers. We provide support for an informational mechanism: recommendations affect consumers' beliefs, which in turn explain consumption. Recommendations reduce uncertainty about goods consumers are most uncertain about and induce information acquisition. Finally, we find evidence for spatial correlation in beliefs.

This paper studies the effects of the EU's General Data Protection Regulation (GDPR) on the ability of firms to collect consumer data, identify consumers over time, accrue revenue via online advertising, and predict their behavior. We utilize a novel dataset that spans many different firms in the online travel industry that allows us to observe consumer visits, purchases, advertisements, and the output of a commercially used algorithm that predicts consumer behavior. We make use of a difference-in-differences analysis that exploits the geographic reach of GDPR. We find a 12.5% drop in observed consumers as a result of GDPR, but at the same time that remaining set of consumers is more persistently identifiable. We provide suggestive evidence that this is driven by a selected set of consumers who substitute from pre-existing privacy means towards those offered as part of GDPR. This substitution affects the ability to predict consumer behavior and preferences as well as improves the ability of advertisers to measure the effectiveness of advertising. In sum, this differential use of privacy tools leads to an increase in the average value of remaining consumers to advertisers, offsetting some of the losses from consumers that opt-out. Our results highlight the externalities that consumer privacy decisions have both on other consumers and for firms.

We study a model of user decision-making in the context of recommender systems via numerical simulation. Our model provides an explanation for the findings of Nguyen, et. al (2014), where, in environments where recommender systems are typically deployed, users consume increasingly similar items over time even without recommendation. We find that recommendation alleviates these natural filter-bubble effects, but that it also leads to an increase in homogeneity across users, resulting in a trade-off between homogenizing across-user consumption and diversifying within-user consumption. Finally, we discuss how our model highlights the importance of collecting data on user beliefs and their evolution over time both to design better recommendations and to further understand their impact.