Exposure bias and its induced feedback loop effect are well-known problems in recommender systems. Exploration is believed to be the key to break such feedback loops. While classical contextual bandit algorithms such as Upper-Confidence-Bound and Thompson Sampling have been successful in addressing the exploration-exploitation trade-off in the single-task settings with one clear reward signal, modern recommender systems often leverage multiple rich sources of feedback such as clicks, likes, dislikes, shares, satisfaction survey responses, and employ multi-task learning in practice. It is unclear how one can incorporate exploration in the multi-task setup with different objectives. In this paper, we study an efficient bandit algorithm tailored to multi-task recommender systems, named Multi-task Neural Linear Bandit (mtNLB). In particular, we investigate efficient feature embeddings in the multi-task setups that could be used as contextual features in the Neural Linear Bandit, a contextual bandit algorithm that nicely combines the representation power from DNN and simplicity in uncertainty calculation from linear models. We further study cost-effective approximations of the uncertainty estimate and principled ways to incorporate uncertainty into the multi-task scoring of items. To showcase the efficacy of our proposed method, we conduct live experiments on a large-scale commercial recommendation platform that serves billions of users. We evaluate the quality of the uncertainty estimate and demonstrate its ability to improve exploration across the different dimensions of the reward signals in comparison to baseline approaches.

Sequential models are invaluable for powering personalized recommendation systems. In the context of short-form video (SFV) feeds, where user behavior history is typically longer, systems must be able to understand users’ long-term interests. However, deploying large sequence models to extensive web-scale applications faces challenges due to high serving cost. To address this, we propose an industrial framework designed for efficiently serving large user sequence models. Specifically, the proposed infrastructure decouples serving of the user sequence model and the main recommendation model, with the user sequence model being served offline (asynchronously) with periodical refresh. The proposed infrastructure is also model-agnostic; thus, it can be used to support any type of user sequence models (even LLMs) with controllable costs. Empirical results show that large user models deployed with our framework significantly and consistently enhance the quality of the main recommendation model with minimal serving costs increase.

Traditional recommendation systems are subject to a strong feedback loop by learning from and reinforcing past user-item interactions, which in turn limits the discovery of novel user interests. To address this, we introduce a hybrid hierarchical framework combining Large Language Models (LLMs) and classic recommendation models for user interest exploration. The framework controls the interfacing between the LLMs and the classic recommendation models through “interest clusters”, the granularity of which can be explicitly determined by algorithm designers. It recommends the next novel interests by first representing “interest clusters” using language, and employs a fine-tuned LLM to generate novel interest descriptions that are strictly within these predefined clusters. At the low level, it grounds these generated interests to an item-level policy by restricting classic recommendation models, in this case a transformer-based sequence recommender to return items that fall within the novel clusters generated at the high level. We showcase the efficacy of this approach on an industrial-scale commercial platform serving billions of users. Live experiments show a significant increase in both exploration of novel interests and overall user enjoyment of the platform.

Multi-task prediction models and value models are the de-facto standard ranking components in modern large-scale content recommendation systems. However, they are typically optimized to model users’ passive consumption behaviors, and rank content in a way to grow only consumption-centric values. In this talk, we discuss the key insight that it is possible to model sparse participatory content-generation actions as well and grow ecosystem value through a new ranking system. We made the following key technical contributions in this system: (1) introducing ranking for content generation based on a categorization of user participation actions of different sparsity, including proxy intent action or access point clicks. (2) improving sparse task prediction quality and stability by causal task relationship modeling, conditional loss modeling and ResNet based shared bottom network. (3) personalizing the value model to minimize conflicts between different values, through e.g. ranking inspiring content higher for users who actively generate content. (4) conducting systematic evaluation of proposed approach in a large short-form video UGC (User-Generated Content) platform.

Effective exploration is believed to positively influence the long-term user experience on recommendation platforms. Determining its exact benefits, however, has been challenging. Regular A/B tests on exploration often measure neutral or even negative engagement metrics while failing to capture its long-term benefits. We here introduce new experiment designs to formally quantify the long-term value of exploration by examining its effects on content corpus, and connecting content corpus growth to the long-term user experience from real-world experiments. Once established the values of exploration, we investigate the Neural Linear Bandit algorithm as a general framework to introduce exploration into any deep learning based ranking systems. We conduct live experiments on one of the largest short-form video recommendation platforms that serves billions of users to validate the new experiment designs, quantify the long-term values of exploration, and to verify the effectiveness of the adopted neural linear bandit algorithm for exploration.

In recent years, social media users spend significant amount of time on Short-Form Video (SFV) platforms. Its success in creating an immersive viewership experience is not only from the content, but also due to its unique UI innovation: instead of providing choices for users to click, SFV platforms actively recommend content to users to watch one at a time. In this paper, we highlight unique challenges rooted from such UI changes for SFV recommendation system design. Firstly, there is yet much unexplored for sources of system biases under the new UI, as there are no clicks nor the common click-based position biases. Additionally, when training multiple types of user activities, positive labels for activities like sharing and commenting can be much sparser and more skewed than traditional click-based recommendation systems, as the latter can filter non-click impressions when generating "post-click" activities.

Sequential recommenders have been widely used in industry due to their strength in modeling user preferences. While these models excel at learning a user’s positive interests, less attention has been paid to learning from negative user feedback. Negative user feedback is an important lever of user control, and comes with an expectation that recommenders should respond quickly and reduce similar recommendations to the user. However, negative feedback signals are often ignored in the training objective of sequential retrieval models, which primarily aim at predicting positive user interactions. In this work, we incorporate explicit and implicit negative user feedback into the training objective of sequential recommenders in the retrieval stage using a "not-to-recommend" loss function that optimizes for the log-likelihood of not recommending items with negative feedback. We demonstrate the effectiveness of this approach using live experiments on a large-scale industrial recommender system. Furthermore, we address a challenge in measuring recommender responsiveness to negative feedback by developing a counterfactual simulation framework to compare recommender responses between different user actions, showing improved responsiveness from the modeling change.