Generative List Recommendation (GLR) models have shown superior performance in E-commerce by directly generating high-quality recommendation lists through sequential item selection. While Online Learning (OL) has proven valuable for streaming point-wise recommendation models in adapting to dynamic user preferences, its application to GLR remains largely unexplored, due in large part to the inefficiency and instability of conventional on-policy reinforcement learning algorithms used in existing GLR approaches. Existing approaches typically rely on surrogate losses, which provide indirect and biased gradient estimates, making them ill-suited for the rapid, subtle distribution shifts common in real-world E-commerce environments. In this paper, we propose Environment Policy Optimization (EPO), a novel GLR model that fundamentally reshapes policy learning by exploiting the differentiability of the environment within the Generator-Evaluator framework. EPO recognizes that the evaluator is a neural network, capable of providing gradient signals. By directly utilizing these gradients, EPO enables end-to-end optimization of the total list-wise reward—the true objective. To ensure differentiable list generation, EPO introduces a new indexing and exploration strategy based on NeuralSort and Gumbel noise, which relaxes discrete item selection into a continuous, gradient-friendly operation. EPO not only demonstrates strong performance in offline evaluations but also unlocks the potential of online learning for GLR at an industrial scale, yielding a 1.18% relative improvement in user clicks in online A/B tests. The results underscore the critical role of EPO in providing the sensitivity, stability, and gradient fidelity necessary for effective real-time adaptation in streaming and dynamic recommendation environments. Moreover, EPO reached the baseline performance with a training time deduction of 76% under identical hardware conditions.

Question and answer (Q&A) platforms usually recommend question-answer pairs to meet users' knowledge acquisition needs, unlike traditional recommendations that recommend only one item. This makes user behaviors more complex, and presents two challenges for Q&A recommendation, including: the collaborative information entanglement, which means user feedback is influenced by either the question or the answer; and the semantic information entanglement, where questions are correlated with their corresponding answers, and correlations also exist among different question-answer pairs. Traditional recommendation methods treat the question-answer pair as a whole or only consider the answer as a single item, which overlooks the two challenges and cannot effectively model user interests. To address these challenges, we introduce a graph neural network model named Question & Answer Graph Collaborative Filtering (QAGCF). QAGCF creates graphs separately from collaborative and semantic views to disentangle the collaborative and semantic information of question-answer pairs. The collaborative view disentangles questions and answers to individually model collaborative information, while the semantic view captures the semantic information both within and between question-answer pairs. These views are further merged into a global graph to integrate the collaborative and semantic information. Polynomial-based graph filters are used to address the high heterophily issues of the global graph. Additionally, contrastive learning is utilized to obtain robust embeddings during training. Extensive experiments on industrial and public datasets demonstrate that QAGCF consistently outperforms baselines and achieves state-of-the-art results.

Live streaming has emerged as a dynamic content format that delivers real-time and interactive experiences to users. Distinguished by the short lifespan and immersive nature of live rooms, live streaming poses two key challenges for recommendation: (1) Timeliness: the model must rapidly identify and promote relevant live rooms to target users within a limited window; and (2) Accurate staytime prediction: since extended watching often reflects content quality and user satisfaction, precisely predicting staytime serves as a critical indicator of recommendation relevance and user engagement. Existing approaches often improve timeliness by repeatedly sending staytime signals to accelerate model learning. However, this introduces label truncation bias, distorting the unbiased estimation of high staytime samples. To reconcile these competing demands, we propose MS3M (Multi-Stream Segmented Staytime Modeling), a novel framework that leverages multiple data streams for faster learning while employing segmented staytime modeling-converting staytime regression into a series of time-segmented classification tasks to ensure unbiased training. Furthermore, to address the sparsity of high staytime samples, MS3M's task-dependent architecture allows high staytime parameters to leverage prior knowledge from low staytime data, significantly improving generalization for long-duration watching behaviors. Extensive offline experiments and online A/B tests on TikTok confirm that MS3M effectively balances timeliness and unbiased learning, leading to substantial gains in recommendation accuracy. The proposed approach currently serves TikTok's live streaming recommendation system, contributing to continuous improvement in user watching experience.

Recommender systems have been widely used in e-commerce, and re-ranking models are playing an increasingly significant role in the domain, which leverages the inter-item influence and determines the final recommendation lists. Online learning methods keep updating a deployed model with the latest available samples to capture the shifting of the underlying data distribution in e-commerce. However, they depend on the availability of real user feedback, which may be delayed by hours or even days, such as item purchases, leading to a lag in model enhancement. In this paper, we propose a novel extension of online learning methods for re-ranking modeling, which we term LAST, an acronym for Learning At Serving Time. It circumvents the requirement of user feedback by using a surrogate model to provide the instructional signal needed to steer model improvement. Upon receiving an online request, LAST finds and applies a model modification on the fly before generating a recommendation result for the request. The modification is request-specific and transient. It means the modification is tailored to and only to the current request to capture the specific context of the request. After a request, the modification is discarded, which helps to prevent error propagation and stabilizes the online learning procedure since the predictions of the surrogate model may be inaccurate. Most importantly, as a complement to feedback-based online learning methods, LAST can be seamlessly integrated into existing online learning systems to create a more adaptive and responsive recommendation experience. Comprehensive experiments, both offline and online, affirm that LAST outperforms state-of-the-art re-ranking models.

Reinforcement learning (RL) has gained popularity in recommender systems for improving long-term performance by effectively exploring users' interests. However, modern recommender systems face the challenge of different user behavioral patterns among millions of items, making exploration more difficult. For example, users with varying activity levels require different exploration intensities. Unfortunately, previous studies often overlook this aspect and apply a uniform exploration strategy to all users, which ultimately hampers long-term user experiences. To tackle these challenges, we propose User-Oriented Exploration Policy (UOEP), a novel approach that enables fine-grained exploration among user groups. We first construct a distributional critic that allows policy optimization based on varying quantile levels of cumulative reward feedback from users, representing user groups with different activity levels. Using this critic as a guide, we design a population of distinct actors dedicated to effective and fine-grained exploration within their respective user groups. To simultaneously enhance diversity and stability during the exploration process, we also introduce a population-level diversity regularization term and a supervision module. Experimental results on public recommendation datasets validate the effectiveness of our approach, as it outperforms all other baselines in terms of long-term performance. Moreover, further analyses reveal the benefits of our approach, including improved performance for low-activity users and increased fairness among users.

Multi-stage ranking pipelines have become widely used strategies in modern recommender systems, where the final stage aims to return a ranked list of items that balances a number of requirements such as user preference, diversity, novelty etc. Linear scalarization is arguably the most widely used technique to merge multiple requirements into one optimization objective, by summing up the requirements with certain preference weights. Existing final-stage ranking methods often adopt a static model where the preference weights are determined during offline training and kept unchanged during online serving. Whenever a modification of the preference weights is needed, the model has to be re-trained, which is time and resources inefficient. Meanwhile, the most appropriate weights may vary greatly for different groups of targeting users or at different time periods (e.g., during holiday promotions). In this paper, we propose a framework called controllable multi-objective re-ranking (CMR) which incorporates a hypernetwork to generate parameters for a re-ranking model according to different preference weights. In this way, CMR is enabled to adapt the preference weights according to the environment changes in an online manner, without retraining the models. Moreover, we classify practical business-oriented tasks into four main categories and seamlessly incorporate them in a new proposed re-ranking model based on an Actor-Evaluator framework, which serves as a reliable real-world testbed for CMR. Offline experiments based on the dataset collected from Taobao App showed that CMR improved several popular re-ranking models by using them as underlying models. Online A/B tests also demonstrated the effectiveness and trustworthiness of CMR.