Short-video recommender systems typically optimize ranking models using dense user behavioral signals, such as clicks and watch time. However, these signals are only indirect proxies of user satisfaction and often suffer from noise and bias. Recently, explicit satisfaction feedback collected through questionnaires has emerged as a high-quality direct alignment supervision, but is extremely sparse and easily overwhelmed by abundant behavioral data, making it difficult to incorporate into online recommendation models. To address these challenges, we propose a novel framework which is towards End-to-End Alignment of user Satisfaction via Questionnaire, named EASQ, to enable real-time alignment of ranking models with true user satisfaction. Specifically, we first construct an independent parameter pathway for sparse questionnaire signals by combining a multi-task architecture and a lightweight LoRA module. The multi-task design separates sparse satisfaction supervision from dense behavioral signals, preventing the former from being overwhelmed. The LoRA module pre-inject these preferences in a parameter-isolated manner, ensuring stability in the backbone while optimizing user satisfaction. Furthermore, we employ a DPO-based optimization objective tailored for online learning, which aligns the main model outputs with sparse satisfaction signals in real time. This design enables end-to-end online learning, allowing the model to continuously adapt to new questionnaire feedback while maintaining the stability and effectiveness of the backbone. Extensive offline experiments and large-scale online A/B tests demonstrate that EASQ consistently improves user satisfaction metrics across multiple scenarios. EASQ has been successfully deployed in a production short-video recommendation system, delivering significant and stable business gains.

Live streaming platforms have become a dominant form of online content consumption, offering dynamically evolving content, real-time interactions, and highly engaging user experiences. These unique characteristics introduce new challenges that differentiate live streaming recommendation from traditional recommendation settings and have garnered increasing attention from industry in recent years. However, research progress in academia has been hindered by the lack of publicly available datasets that accurately reflect the dynamic nature of live streaming environments. To address this gap, we introduce KuaiLive, the first real-time, interactive dataset collected from Kuaishou, a leading live streaming platform in China with over 400 million daily active users. The dataset records the interaction logs of 23,772 users and 452,621 streamers over a 21-day period. Compared to existing datasets, KuaiLive offers several advantages: it includes precise live room start and end timestamps, multiple types of real-time user interactions (click, comment, like, gift), and rich side information features for both users and streamers. These features enable more realistic simulation of dynamic candidate items and better modeling of user and streamer behaviors. We conduct a thorough analysis of KuaiLive from multiple perspectives and evaluate several representative recommendation methods on it, establishing a strong benchmark for future research. KuaiLive can support a wide range of tasks in the live streaming domain, such as top-K recommendation, click-through rate prediction, watch time prediction, and gift price prediction. Moreover, its fine-grained behavioral data also enables research on multi-behavior modeling, multi-task learning, and fairness-aware recommendation. We believe that KuaiLive will serve as a valuable resource to advance the development of intelligent live streaming services. The dataset and related resources are publicly available at: https://imgkkk574.github.io/KuaiLive.

The popularity of short video applications has brought new opportunities and challenges to video recommendation.In addition to the traditional ranking-based pipeline, industrial solutions usually introduce additional distribution management components to guarantee a diverse and content-rich user experience.However, existing solutions are either non-personalized or fail to generalize well to the ever-changing user preferences.Inspired by the success of uplift modeling in online marketing, we attempt to implement uplift modeling in the video recommendation scenario to mitigate the problems.However, we face two main challenges when migrating the technique: 1) the complex-response causal relation in distribution management problem, and 2) the modeling of long-term and real-time user preferences.To address these challenges, we correspond each treatment to a specific adjustment of the distribution over video types, then propose a Coarse-to-fine Dynamic Uplift Modeling (CDUM) framework for real-time video recommendation scenarios.Specifically, CDUM consists of two modules, a coarse-grained module that utilizes the offline features of users to model their long-term preferences, and a fine-grained * Both authors contributed equally to this research.

Generative recommender systems have gained increasing attention as an innovative approach that directly generates item identifiers for recommendation tasks. Despite their potential, a major challenge is the effective construction of item identifiers that align well with recommender systems. Current approaches often treat item tokenization and generative recommendation training as separate processes, which can lead to suboptimal performance. To overcome this issue, we introduce ETEGRec, a novel End-To-End Generative Recommender that unifies item tokenization and generative recommendation into a cohesive framework. Built on a dual encoder-decoder architecture, ETEGRec consists of an item tokenizer and a generative recommender. To enable synergistic interaction between these components, we propose a recommendation-oriented alignment strategy, which includes two key optimization objectives: sequence-item alignment and preference-semantic alignment. These objectives tightly couple the learning processes of the item tokenizer and the generative recommender, fostering mutual enhancement. Additionally, we develop an alternating optimization technique to ensure stable and efficient end-to-end training of the entire framework. Extensive experiments demonstrate the superior performance of our approach compared to traditional sequential recommendation models and existing generative recommendation baselines. Our code is available at https://github.com/RUCAIBox/ETEGRec.

Reranking models solve the final recommendation lists that best fulfill users' demands.While existing solutions focus on finding parametric models that approximate optimal policies, recent approaches find that it is better to generate multiple lists to compete for a "pass" ticket from an evaluator, where the evaluator serves as the supervisor who accurately estimates the performance of the candidate lists.In this work, we show that we can achieve a more efficient and effective list proposal with a multi-generator framework and provide empirical evidence on two public datasets and online A/B tests.More importantly, we verify that the effectiveness of a generator is closely related to how much it complements the views of other generators with sufficiently different rerankings, which derives the metric of list comprehensiveness.With this intuition, we design an automatic complementary generator-finding framework that learns a policy that simultaneously aligns the users' preferences and maximizes the list comprehensiveness metric.The experimental results indicate that the proposed framework can further improve the multi-generator reranking performance.

Diffusion models (DMs) have emerged as promising approaches for sequential recommendation due to their strong ability to model data distributions and generate high-quality items.Existing work typically adds noise to the next item and progressively denoises it guided by the user's interaction sequence, generating items that closely align with user interests.However, we identify two key issues in this paradigm.First, the sequences are often heterogeneous in length and content, exhibiting noise due to stochastic user behaviors.Using such sequences as guidance may hinder DMs from accurately understanding user interests.Second, DMs are prone to data bias and tend to generate only the popular items that dominate the training dataset, thus failing to meet the personalized needs of different users.To address these issues, we propose Distinguished Quantized Guidance for Diffusion-based Sequence Recommendation (DiQDiff), which aims to extract robust guidance to understand user interests and generate distinguished items for personalized user interests within DMs.To extract robust guidance, DiQDiff introduces Semantic Vector Quantization (SVQ) to quantize sequences into semantic vectors (e.g., collaborative signals and category interests) using a codebook, which can enrich the guidance to better understand user interests.To generate distinguished items, DiQDiff personalizes the generation through Contrastive Discrepancy Maximization (CDM), which maximizes the distance between denoising trajectories using contrastive loss to prevent biased generation for different users.Extensive experiments are conducted to compare DiQDiff with multiple baseline models across four widely-used datasets.The superior recommendation performance of DiQDiff against leading approaches demonstrates its effectiveness in sequential recommendation tasks.

Recent advances in recommender systems have shown that user-system interaction essentially formulates long-term optimization problems, and online reinforcement learning can be adopted to improve recommendation performance. The general solution framework incorporates a value function that estimates the user's expected cumulative rewards in the future and guides the training of the recommendation policy. To avoid local maxima, the policy may explore potential high-quality actions during inference to increase the chance of finding better future rewards. To accommodate the stepwise recommendation process, one widely adopted approach to learning the value function is learning from the difference between the values of two consecutive states of a user. However, we argue that this paradigm involves a challenge of Mixing Random Factors: there exist two random factors from the stochastic policy and the uncertain user environment, but they are not separately modeled in the standard temporal difference (TD) learning, which may result in a suboptimal estimation of the long-term rewards and less effective action exploration. As a solution, we show that these two factors can be separately approximated by decomposing the original temporal difference loss. The disentangled learning framework can achieve a more accurate estimation with faster learning and improved robustness against action exploration. As an empirical verification of our proposed method, we conduct offline experiments with simulated online environments built on the basis of public datasets.

The field of Reinforcement Learning (RL) has garnered increasing attention for its ability of optimizing user retention in recommender systems. A primary obstacle in this optimization process is the environment non-stationarity stemming from the continual and complex evolution of user behavior patterns over time, such as variations in interaction rates and retention propensities. These changes pose significant challenges to existing RL algorithms for recommendations, leading to issues with dynamics and reward distribution shifts. This paper introduces a novel approach called Adaptive User Retention Optimization (AURO) to address this challenge. To navigate the recommendation policy in non-stationary environments, AURO introduces an state abstraction module in the policy network. The module is trained with a new value-based loss function, aligning its output with the estimated performance of the current policy. As the policy performance of RL is sensitive to environment drifts, the loss function enables the state abstraction to be reflective of environment changes and notify the recommendation policy to adapt accordingly. Additionally, the non-stationarity of the environment introduces the problem of implicit cold start, where the recommendation policy continuously interacts with users displaying novel behavior patterns. AURO encourages exploration guarded by performance-based rejection sampling to maintain a stable recommendation quality in the cost-sensitive online environment. Extensive empirical analysis are conducted in a user retention simulator, the MovieLens dataset, and a live short-video recommendation platform, demonstrating AURO's superior performance against all evaluated baseline algorithms. Code is available at https://github.com/AIDefender/AURO

Achieving fairness among different individuals or groups is an essential task for industrial recommender systems. Due to the group's personalized selection tendencies and the non-uniform population distributions, existing industrial recommenders tend to make unfair predictions towards the preferences of minority groups. To alleviate this unfairness, we propose a model-agnostic self-adaptive fairness constraint framework (SaFair) based on the posterior preferences of different groups. We construct group-level and individual-level fairness constraints. The former measures consistency between group-level posterior preferences and predicted interests, and the latter relies on the degree of consistency in interests between a user and their associated group to perform self-adaptive constraints. In particular, to balance effectiveness and fairness, we utilize uncertainty estimation to adjust the intensity of constraints according to the model's learning status called self-adaptive constraints. Extensive offline experiments and online A/B Testing are conducted and the results validate the superiority of our proposed method over the baselines. SaFair has been successfully deployed in Kuaishou, one of China's most popular short-video streaming platforms with hundreds of millions of active users.

Recommender systems aim to fulfill the user's daily demands. While most existing research focuses on maximizing the user's engagement with the system, it has recently been pointed out that how frequently the users come back for the service also reflects the quality and stability of recommendations. However, optimizing this user retention behavior is non-trivial and poses several challenges including the intractable leave-and-return user activities, the sparse and delayed signal, and the uncertain relations between users' retention and their immediate feedback towards each item in the recommendation list. In this work, we regard the retention signal as an overall estimation of the user's end-of-session satisfaction and propose to estimate this signal through a probabilistic flow. This flow-based modeling technique can back-propagate the retention reward towards each recommended item in the user session, and we show that the flow combined with traditional learning-to-rank objectives eventually optimizes a non-discounted cumulative reward for both immediate user feedback and user retention. We verify the effectiveness of our method through both offline empirical studies on two public datasets and online A/B tests in an industrial platform.

In recent years, graph contrastive learning (GCL) has received increasing attention in recommender systems due to its effectiveness in reducing bias caused by data sparsity. However, most existing GCL models rely on heuristic approaches and usually assume entity independence when constructing contrastive views. We argue that these methods struggle to strike a balance between semantic invariance and view hardness across the dynamic training process, both of which are critical factors in graph contrastive learning. To address the above issues, we propose a novel GCL-based recommendation framework RGCL, which effectively maintains the semantic invariance of contrastive pairs and dynamically adapts as the model capability evolves through the training process. Specifically, RGCL first introduces decision boundary-aware adversarial perturbations to constrain the exploration space of contrastive augmented views, avoiding the decrease of task-specific information. Furthermore, to incorporate global user-user and item-item collaboration relationships for guiding on the generation of hard contrastive views, we propose an adversarial-contrastive learning objective to construct a relation-aware view-generator. Besides, considering that unsupervised GCL could potentially narrower margins between data points and the decision boundary, resulting in decreased model robustness, we introduce the adversarial examples based on maximum perturbations to achieve margin maximization. We also provide theoretical analyses on the effectiveness of our designs. Through extensive experiments on five public datasets, we demonstrate the superiority of RGCL compared against twelve baseline models.

In recommender systems, reinforcement learning solutions have shown promising results in optimizing the interaction sequence between users and the system over the long-term performance. For practical reasons, the policy's actions are typically designed as recommending a list of items to handle users' frequent and continuous browsing requests more efficiently. In this list-wise recommendation scenario, the user state is updated upon every request in the corresponding MDP formulation. However, this request-level formulation is essentially inconsistent with the user's item-level behavior. In this study, we demonstrate that an item-level optimization approach can better utilize item characteristics and optimize the policy's performance even under the request-level MDP. We support this claim by comparing the performance of standard request-level methods with the proposed item-level actor-critic framework in both simulation and online experiments. Furthermore, we show that a reward-based future decomposition strategy can better express the item-wise future impact and improve the recommendation accuracy in the long term. To achieve a more thorough understanding of the decomposition strategy, we propose a model-based re-weighting framework with adversarial learning that further boost the performance and investigate its correlation with the reward-based strategy.

When visiting an online platform, a user generates various actions, such as clicks, long views, likes, comments, etc.To capture user preferences in these aspects, we learn these objectives and return multiple rankings of candidate items for each user.We need to aggregate them into one to truncate the candidate set, and ranking ensemble model is proposed for this task.However, there is a critical issue: though we input abundant information, what model learns depends on the supervision.Unfortunately, the existing supervision is poorly designed, leading to serious information loss issue.To address this issue, we designed an unsupervised loss to compel the ranking ensemble model to learn all information of input rankings, including sequential and numerical information.(1) For sequential information, we design a distance measure between two rankings, and train the ensemble ranking to have similar order with all input rankings by minimizing the distance.(2) For numerical information, we design a decoder to reconstruct values of original rankings from the hidden layer of the model, to guarantee that the model captures as much input information as possible.Our unsupervised loss is compatible with all ranking ensemble models.We optimize several widely-used structures to propose unsupervised ranking ensemble models.We devise comprehensive experiments on two real-world datasets to demonstrate the effectiveness of the proposed models.We also apply our model in a short video platform with billions of users, and achieve significant improvement.

Natural language explainable recommendation has become a promising direction to facilitate more efficient and informed user decisions. Previous models mostly focus on how to enhance the explanation accuracy. However, the robustness problem has been largely ignored, which requires the explanations generated for similar user-item pairs should not be too much different. Different from traditional classification problems, improving the robustness of natural languages has two unique characteristics: (1) Different token importances, that is, different tokens play various roles in representing the complete sentence, and the robustness requirements for predicting them should also be different. (2) Continuous token semantics, that is, the similarity of the output should be judged based on semantics, and the sequences without any token-level overlap may also be highly similar. Based on these characteristics, we formulate and solve a novel problem in the recommendation domain, that is, robust natural language explainable recommendation. To the best of our knowledge, it is the first time in this field. Specifically, we base our modeling on adversarial robust optimization and design four types of heuristic methods to modify the adversarial outputs with weighted token probabilities and synonym replacements. Furthermore, to consider the mutual influence between the above characteristics, we regard language generation as a decision-making problem and design a dual-policy reinforcement learning framework to improve the robustness of the generated languages. We conduct extensive experiments to demonstrate the effectiveness of our framework.

In scenarios involving sequence recommendations on large screen devices, such as tablets or TVs, the equipment is often shared among multiple users. This sharing leads to a mixture of behaviors from different users, posing significant challenges to recommendation systems, especially when clear supervisory signals for distinguishing among users are absent. Current solutions tend to either operate in an unsupervised manner or rely on constructed supervisory signals that are not entirely reliable. Moreover, the peculiarities of short video recommendations in this context have not been thoroughly explored in existing research. In response to these challenges, this paper introduces Multi-User Contrastive Decoupling Model (MODEM), a novel short video recommendation model specifically designed for large screen devices. MODEM leverages an attention mechanism, grounded in session segmentation, to disentangle the intertwined user behavior histories. It also discriminates between the impacts of long and short viewing behaviors on short video recommendations by cross-analyzing sequences of both. Furthermore, we have developed a contrastive learning method to oversee the decoupling of user behaviors effectively. Our evaluations demonstrate noticeable improvements through both offline assessments within public datasets and online A/B testing within Kuaishou’s short video recommendation environment on large screen devices. Specifically, our online A/B tests resulted in a 0.55% increase in watch time. These results underscore MODEM’s efficacy in enhancing recommendation quality in shared account contexts.

Recommender systems filter out information that meets user interests. However, users may be tired of the recommendations that are too similar to the content they have been exposed to in a short historical period, which is the so-called user fatigue. Despite the significance for a better user experience, user fatigue is seldom explored by existing recommenders. In fact, there are three main challenges to be addressed for modeling user fatigue, including what features support it, how it influences user interests, and how its explicit signals are obtained. In this paper, we propose to model user Fatigue in interest learning for sequential Recommendations (FRec). To address the first challenge, based on a multi-interest framework, we connect the target item with historical items and construct an interest-aware similarity matrix as features to support fatigue modeling. Regarding the second challenge, built upon feature cross, we propose a fatigue-enhanced multi-interest fusion to capture long-term interest. In addition, we develop a fatigue-gated recurrent unit for short-term interest learning, with temporal fatigue representations as important inputs for constructing update and reset gates. For the last challenge, we propose a novel sequence augmentation to obtain explicit fatigue signals for contrastive learning. We conduct extensive experiments on real-world datasets, including two public datasets and one large-scale industrial dataset. Experimental results show that FRec can improve AUC and GAUC up to 0.026 and 0.019 compared with state-of-the-art models, respectively. Moreover, large-scale online experiments demonstrate the effectiveness of FRec for fatigue reduction. Our codes are released at https://github.com/tsinghua-fib-lab/SIGIR24-FRec.

In Recommender System (RS) applications, reinforcement learning (RL) has recently emerged as a powerful tool, primarily due to its proficiency in optimizing long-term rewards. Nevertheless, it suffers from instability in the learning process, stemming from the intricate interactions among bootstrapping, off-policy training, and function approximation. Moreover, in multi-reward recommendation scenarios, designing a proper reward setting that reconciles the inner dynamics of various tasks is quite intricate. To this end, we propose a novel decision transformer-based recommendation model, DT4IER, to not only elevate the effectiveness of recommendations but also to achieve a harmonious balance between immediate user engagement and long-term retention. The DT4IER applies an innovative multi-reward design that adeptly balances short and long-term rewards with user-specific attributes, which serve to enhance the contextual richness of the reward sequence, ensuring a more informed and personalized recommendation process. To enhance its predictive capabilities, DT4IER incorporates a high-dimensional encoder to identify and leverage the intricate interrelations across diverse tasks. Furthermore, we integrate a contrastive learning approach within the action embedding predictions, significantly boosting the model's overall performance. Experiments on three real-world datasets demonstrate the effectiveness of DT4IER against state-of-the-art baselines in terms of both immediate user engagement and long-term retention. The source code is accessible online to facilitate replication.

Multi-domain recommendation and multi-task recommendation have demonstrated their effectiveness in leveraging common information from different domains and objectives for comprehensive user modeling. Nonetheless, the practical recommendation usually faces multiple domains and tasks simultaneously, which cannot be well-addressed by current methods. To this end, we introduce M3oE, an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework. M3oE integrates multi-domain information, maps knowledge across domains and tasks, and optimizes multiple objectives. We leverage three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences respectively to address the complex dependencies among multiple domains and tasks in a disentangled manner. Additionally, we design a two-level fusion mechanism for precise control over feature extraction and fusion across diverse domains and tasks. The framework's adaptability is further enhanced by applying AutoML technique, which allows dynamic structure optimization. To the best of the authors' knowledge, our M3oE is the first effort to solve multi-domain multi-task recommendation self-adaptively. Extensive experiments on two benchmark datasets against diverse baselines demonstrate M3oE's superior performance. The implementation code is available to ensure reproducibility.

Contrastive learning has recently emerged as an effective strategy for improving the performance of sequential recommendation. However, traditional models commonly construct the contrastive loss by directly optimizing human-designed positive and negative samples, resulting in a model that is overly sensitive to heuristic rules. To address this limitation, we propose a novel soft contrastive framework for sequential recommendation in this article. Our main idea is to extend the point-wise contrast to a region-level comparison, where we aim to identify instances near the initially selected positive/negative samples that exhibit similar contrastive properties. This extension improves the model’s robustness to human heuristics. To achieve this objective, we introduce an adversarial contrastive loss that allows us to explore the sample regions more effectively. Specifically, we begin by considering the user behavior sequence as a holistic entity. We construct adversarial samples by introducing a continuous perturbation vector to the sequence representation. This perturbation vector adds variability to the sequence, enabling more flexible exploration of the sample regions. Moreover, we extend the aforementioned strategy by applying perturbations directly to the items within the sequence. This accounts for the sequential nature of the items. To capture these sequential relationships, we utilize a recurrent neural network to associate the perturbations, which introduces an inductive bias for more efficient exploration of adversarial samples. To demonstrate the effectiveness of our model, we conduct extensive experiments on five real-world datasets.