Towards End-to-End Alignment of User Satisfaction via Questionnaire in Video Recommendation
Abstract
Short-video recommender systems typically optimize ranking models using dense user behavioral signals, such as clicks and watch time. However, these signals are only indirect proxies of user satisfaction and often suffer from noise and bias. Recently, explicit satisfaction feedback collected through questionnaires has emerged as a high-quality direct alignment supervision, but is extremely sparse and easily overwhelmed by abundant behavioral data, making it difficult to incorporate into online recommendation models. To address these challenges, we propose a novel framework which is towards End-to-End Alignment of user Satisfaction via Questionnaire, named EASQ, to enable real-time alignment of ranking models with true user satisfaction. Specifically, we first construct an independent parameter pathway for sparse questionnaire signals by combining a multi-task architecture and a lightweight LoRA module. The multi-task design separates sparse satisfaction supervision from dense behavioral signals, preventing the former from being overwhelmed. The LoRA module pre-inject these preferences in a parameter-isolated manner, ensuring stability in the backbone while optimizing user satisfaction. Furthermore, we employ a DPO-based optimization objective tailored for online learning, which aligns the main model outputs with sparse satisfaction signals in real time. This design enables end-to-end online learning, allowing the model to continuously adapt to new questionnaire feedback while maintaining the stability and effectiveness of the backbone. Extensive offline experiments and large-scale online A/B tests demonstrate that EASQ consistently improves user satisfaction metrics across multiple scenarios. EASQ has been successfully deployed in a production short-video recommendation system, delivering significant and stable business gains.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Yimeng Bai | 59 | 52 Leveraging Watch-time Feedback for Short-Video Recommendations: A Causal Labeling Framework (2023) Recommendation Systems and Evaluation 52 LabelCraft: Empowering Short Video Recommendations with Automated Label Crafting (2024) Recommendation Systems and Evaluation 45 GradCraft: Elevating Multi-task Recommendations through Holistic Gradient Crafting (2024) Multi-Task and Multi-Objective Optimization |
|
| 2 | Romain Deffayet | 56 | 61 An Offline Metric for the Debiasedness of Click Models (2023) User Behavior and Search Interaction 61 Evaluating the Robustness of Click Models to Policy Distributional Shift (2023) User Behavior and Search Interaction 52 SARDINE: Simulator for Automated Recommendation in Dynamic and Interactive Environments (2024) Recommendation Systems and Evaluation |
|
| 3 | Xuanhui Wang | 48 | 70 Revisiting Two-tower Models for Unbiased Learning to Rank (2022) User Behavior and Search Interaction 70 Learning with Sparse and Biased Feedback for Personal Search (2018) User Behavior and Search Interaction 61 Cross-Positional Attention for Debiasing Clicks (2021) User Behavior and Search Interaction |
Area chair
| Area chair | Match | |
|---|---|---|
| Ziyi Ye | 55 |
Area-chair candidates (5)
| Ziyi Ye assigned | 55 | 14 papers | |
| Yanan Niu COI: co-author | 47 | 15 papers | |
| Zhijing Wu 0001 | 44 | 9 papers | |
| Xiaoxue Zang COI: co-author | 43 | 4 papers | |
| Pengyue Jia | 42 | 10 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Vuong Thanh Tung | 68 | 3 | 74 EntityBot: Supporting Everyday Digital Tasks with Entity Recommendations (2021) User Behavior and Search Interaction 74 Entity Recommendation for Everyday Digital Tasks (2021) User Behavior and Search Interaction |
|
| 2 | Sara Allawati | 68 | 3 | 69 A Comparative Analysis of Linguistic and Retrieval Diversity in LLM-Generated Search Queries (2025) User Behavior and Search Interaction 61 What do Users Really Ask Large Language Models? An Initial Log Analysis of Google Bard Interactions in the Wild (2024) User Behavior and Search Interaction |
|
| 3 | Jiqun Liu | 65 | 3 | 61 State-Aware Meta-Evaluation of Evaluation Metrics in Interactive Information Retrieval (2021) User Behavior and Search Interaction 61 A Reference-Dependent Model for Web Search Evaluation: Understanding and Measuring the Experience of Boundedly Rational Users (2023) User Behavior and Search Interaction |
|
| 4 | Fan Zhang 0053 | 64 | 3 | 70 Incorporating Query Reformulating Behavior into Web Search Evaluation (2021) User Behavior and Search Interaction 69 Evaluating Mobile Search with Height-Biased Gain (2017) User Behavior and Search Interaction |
|
| 5 | Yimeng Bai assigned | 59 | 3 | 52 Leveraging Watch-time Feedback for Short-Video Recommendations: A Causal Labeling Framework (2023) Recommendation Systems and Evaluation 52 LabelCraft: Empowering Short Video Recommendations with Automated Label Crafting (2024) Recommendation Systems and Evaluation |
|
| 6 | Romain Deffayet assigned | 56 | 3 | 61 An Offline Metric for the Debiasedness of Click Models (2023) User Behavior and Search Interaction 61 Evaluating the Robustness of Click Models to Policy Distributional Shift (2023) User Behavior and Search Interaction |
|
| 8 | Jaime Arguello | 54 | 3 | 71 The Effects of Vertical Rank and Border on Aggregated Search Coherence and Search Behavior (2014) User Behavior and Search Interaction 71 The effect of aggregated search coherence on search behavior (2012) User Behavior and Search Interaction |
|
| 9 | Xuanhui Wang assigned | 48 | 3 | 70 Revisiting Two-tower Models for Unbiased Learning to Rank (2022) User Behavior and Search Interaction 70 Learning with Sparse and Biased Feedback for Personal Search (2018) User Behavior and Search Interaction |
|
| 10 | Mark D. Smucker | 48 | 3 | 69 IR Evaluation: Modeling User Behavior for Measuring Effectiveness (2015) User Behavior and Search Interaction 69 Evaluating Streams of Evolving News Events (2015) User Behavior and Search Interaction |
|
| 12 | Zhen Qin 0001 | 47 | 3 | 70 Revisiting Two-tower Models for Unbiased Learning to Rank (2022) User Behavior and Search Interaction 70 Towards Disentangling Relevance and Bias in Unbiased Learning to Rank (2023) User Behavior and Search Interaction |
|
| 13 | Wei Guo 0006 | 46 | 3 | 52 User Behavior Modeling with Deep Learning for Recommendation: Recent Advances (2023) Recommendation Systems and Evaluation 52 P-Law: Predicting Quantitative Scaling Law with Entropy Guidance in Large Recommendation Models (2025) Recommendation Systems and Evaluation |
|
| 14 | Falk Scholer | 46 | 3 | 70 Can Users Predict Relative Query Effectiveness? (2022) User Behavior and Search Interaction 69 Classifying Term Variants in Query Formulation (2025) User Behavior and Search Interaction |
|
| 15 | Huifeng Guo | 46 | 3 | 52 AutoML for Deep Recommender Systems: Fundamentals and Advances (2023) Recommendation Systems and Evaluation 52 A Comprehensive Survey on Automated Machine Learning for Recommendations (2024) Recommendation Systems and Evaluation |
|
| 16 | Junwei Pan COI: co-author | 45 | 3 | 52 Ads Recommendation in a Collapsed and Entangled World (2024) CTR and Conversion Rate Prediction 45 STEM: Unleashing the Power of Embeddings for Multi-Task Recommendation (2024) Multi-Task and Multi-Objective Optimization |
|
| 17 | Yang Song 0008 COI: co-author | 45 | 3 | 78 Query-Less: Predicting Task Repetition for NextGen Proactive Search and Recommendation Engines (2016) User Behavior and Search Interaction 52 LabelCraft: Empowering Short Video Recommendations with Automated Label Crafting (2024) Recommendation Systems and Evaluation |
|
| 18 | Lichan Hong | 45 | 3 | 61 Improving Training Stability for Multitask Ranking Models in Recommender Systems (2023) Recommendation Systems and Evaluation 45 Co-optimize Content Generation and Consumption in a Large Scale Video Recommendation System (2024) Multi-Task and Multi-Objective Optimization |
|
| 20 | Shuchang Liu 0001 COI: co-author | 44 | 3 | 54 LLM-Powered User Simulator for Recommender System (2025) Recommendation Systems and Evaluation 54 KuaiSim: A Comprehensive Simulator for Recommender Systems (2023) Recommendation Systems and Evaluation |
|
| 21 | Yong He 0009 | 44 | 3 | 60 MMLRec: A Unified Multi-Task and Multi-Scenario Learning Benchmark for Recommendation (2024) Recommendation Systems and Evaluation 45 Mixture of Graph Enhanced Expert Networks for Multi-task Recommendation (2022) Multi-Task and Multi-Objective Optimization |
|
| 22 | Ke Fei | 43 | 3 | 46 Stable and Adaptive Fusion for Multi-domain Multi-task Recommendation (2026) Multi-Scenario and Multi-Domain CTR Prediction 35 Entire-Space Variational Information Exploitation for Post-Click Conversion Rate Prediction (2025) CTR and Conversion Rate Prediction |
|
| 23 | Gustavo Penha | 43 | 3 | 65 As It Was: Aligning LLM Search Evaluation with Historical User Preferences (2026) User Behavior and Search Interaction 52 Contextualizing Spotify's Audiobook List Recommendations with Descriptive Shelves (2025) Recommendation Systems and Evaluation |
|
| 25 | Chongming Gao | 43 | 3 | 52 KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems (2022) Recommendation Systems and Evaluation 48 On the Theories Behind Hard Negative Sampling for Recommendation (2023) Recommendation Systems and Evaluation |
|
| 26 | Hyunsik Yoo | 42 | 3 | 52 Continual Recommender Systems (2025) Recommendation Systems and Evaluation 52 Embracing Plasticity: Balancing Stability and Plasticity in Continual Recommender Systems (2025) Recommendation Systems and Evaluation |
|
| 27 | Saber Zerhoudi | 42 | 3 | 68 In-Browser Agents for Search Assistance (2026) User Behavior and Search Interaction 67 Beyond the Click: A Framework for Inferring Cognitive Traces in Search (2026) User Behavior and Search Interaction |
|
| 29 | Yifan Wang 0017 | 41 | 3 | 65 User Perception of Recommendation Explanation: Are Your Explanations What Users Need? (2023) User Behavior and Search Interaction 36 Improving Long-tail User CTR Prediction via Hierarchical Distribution Alignment (2025) Bias Mitigation Methods |
|
| 30 | Paul Thomas 0001 | 41 | 3 | 69 User Models, Metrics and Measures of Search: A Tutorial on the C/W/L Evaluation Framework (2021) User Behavior and Search Interaction 65 Large Language Models can Accurately Predict Searcher Preferences (2024) User Behavior and Search Interaction |
|
| 31 | Jin Chen 0008 | 41 | 3 | 52 RecStudio: Towards a Highly-Modularized Recommender System (2023) Recommendation Systems and Evaluation 48 Learning-Efficient Yet Generalizable Collaborative Filtering for Item Recommendation (2024) Recommendation Systems and Evaluation |
|
| 32 | Ed H. Chi | 41 | 3 | 45 Co-optimize Content Generation and Consumption in a Large Scale Video Recommendation System (2024) Multi-Task and Multi-Objective Optimization 45 DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning (2021) Multi-Task and Multi-Objective Optimization |
|
| 33 | Xiao Zhang 0034 COI: co-author | 41 | 3 | 42 UniSAR: Modeling User Transition Behaviors between Search and Recommendation (2024) Recommendation with Search and Ranking 40 GenSAR: Unifying Balanced Search and Recommendation with Generative Retrieval (2025) Recommendation with Search and Ranking |
|
| 34 | Shuaiqiang Wang | 41 | 3 | 73 M2oERank: Multi-Objective Mixture-of-Experts Enhanced Ranking for Satisfaction-Oriented Web Search (2025) User Behavior and Search Interaction 72 RankExpert: A Mixture of Textual-and-Behavioral Experts for Multi-Objective Learning-to-Rank in Web Search (2025) User Behavior and Search Interaction |
|
| 35 | Bo Peng 0009 | 41 | 3 | 70 SWaT: Statistical Modeling of Video Watch Time through User Behavior Analysis (2025) User Behavior and Search Interaction 36 Hybrid collaborative filtering methods for recommending search terms to clinicians (2021) Recommendation with Search and Ranking |