This study identifies the differential impact of algorithmic and friend-based recommendations---the two predominant mechanisms of online content recommendation---on users' engagement with novel information, characterized as diverse and non-redundant. Our analysis focuses on the influence of different content recommended by algorithms versus friends and the role of social influence, specifically the impact of social cues inherent in friend-based recommendations. We designed and conducted a large-scale field experiment on WeChat, involving 2.1 million users. Participants were randomly assigned to one of three groups: a control group that received content recommended by algorithms, a treatment group that viewed content shared by friends with visible social cues (e.g., friends' "likes"), and another treatment group that was exposed to friend-shared content with the social cues hidden. The findings reveal a general preference for less novel content across all groups. However, the presence of social cues significantly mitigated this trend, indicating that social influence can encourage engagement with more novel information. Despite algorithms tending to recommend content of lower novelty, users engage more with novel content when recommended by algorithms than by friends with and without social cues. The study also discovered significant variations in engagement with novel content among users of different genders, ages, and city tiers. These results carry important implications for the design of content recommendation systems and inform policymaking regarding the dissemination of information online.

Online controlled experiments, often referred to as A/B tests, are extensively conducted by major technology companies to evaluate the effectiveness of product strategies and inform product decision-making. The sampling process in A/B tests is not instantaneous; subjects, such as users of online platforms, arrive at the platform over time and are recruited continuously throughout the experiment. This ongoing nature of sampling can lead to shifts in sample characteristics over the experimental duration, raising issues of external validity. In other words, the causal findings derived from an experiment of a particular duration may not be applicable to the target population, potentially biasing decision-making.

Randomized controlled trials (RCTs), also known as A/B tests, have become the gold standard for evaluating the effectiveness of product changes on digital platforms. Accurately estimating the effects of long-term treatments still remains a challenge. Product updates such as new user interfaces or recommendation algorithms are intended to persist in the system for an extended period. However, A/B testing is typically conducted for short durations, often less than two weeks, to facilitate rapid product iterations. Conducting lengthy experiments to capture the long-term impact of product changes becomes impractical due to potential negative impacts on user experiences, high opportunity costs associated with user traffic, and delays in decision-making processes.