Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Advantage-Conditioned Flow Policy for Offline Reinforcement Learning in Recommendation

sigir26-102 · Full research paper · Xiaocong Chen, Siyu Wang, Lina Yao
Diffusion and Flow Recommenders · 50% Offline RL for Recommendation · 50%
Abstract

Offline reinforcement learning (RL) is a useful approach for recommender systems because it can optimize long-term user feedback from logged interaction data without online exploration. A key challenge is the multi-modal nature of user preferences: a user may like several unrelated item types, so a unimodal policy (for example, a Gaussian) tends to average across modes and generate actions that do not match any interest. Recent diffusion-based policies can model complex preference distributions, but they often require many denoising steps. We propose PerfRec (Preference-aware Flow for Recommendation), a flow-matching offline RL framework that learns an expressive behavioral policy and distills it into an efficient one-step policy. PerfRec (i) trains a conditional flow model to clone the logged action distribution, (ii) trains twin Q-networks using next actions sampled from the learned flow policy, and (iii) trains an advantage-conditioned one-step policy with Q-guidance for improvement and a distillation loss that keeps the policy close to the flow policy. We use binary advantage conditioning to separate high-advantage and low-advantage regions of the flow-induced action distribution, so that at inference we can sample from the high-advantage mode with a single forward pass. Experiments on five benchmark datasets and one online simulation platform show that PerfRec improves recommendation performance over strong offline RL baselines.

Assigned reviewers

#ReviewerMatchWhy (closest evidence publications in the taxonomy)
1 Shuchang Liu 0001
58
88 Modeling User Retention through Generative Flow Networks (2024) Diffusion and Flow Recommenders
74 Exploration and Regularization of the Latent Action Space in Recommendation (2023) Offline RL for Recommendation
69 Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation (2025) Diffusion and Flow Recommenders
2 Yuta Saito
47
51 Off-Policy Evaluation of Ranking Policies under Diverse User Behavior (2023) Recommendation with Search and Ranking
49 Evaluating the Robustness of Off-Policy Evaluation (2021) Recommendation Systems and Evaluation
49 Asymmetric Tri-training for Debiasing Missing-Not-At-Random Explicit Feedback (2020) Popularity Bias and Debiasing in Recommendation
3 Ziru Liu
47
69 Diff-MSR: A Diffusion Model Enhanced Paradigm for Cold-Start Multi-Scenario Recommendation (2024) Diffusion and Flow Recommenders
60 Modeling User Retention through Generative Flow Networks (2024) Multi-Behavior Recommendation
60 Multi-Task Recommendations with Reinforcement Learning (2023) Offline RL for Recommendation

Area chair

Area chairMatch
Leyu Lin 48
Area-chair candidates (6)
Leyu Lin assigned 4815 papers
Xin Xin 0003 4710 papers
Yanan Niu 4515 papers
Kun Gai 445 papers
Maolin Wang 0001 4315 papers
Pengyue Jia 4210 papers

Candidates from the panel ranked by match score

#ReviewerMatchLoadWhy
1 Zhengyi Yang 0007 59 3
74 Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion (2023) Diffusion and Flow Recommenders
74 On Efficiency-Effectiveness Trade-off of Diffusion-based Recommenders (2025) Diffusion and Flow Recommenders
2 Shuchang Liu 0001 assigned 58 3
88 Modeling User Retention through Generative Flow Networks (2024) Diffusion and Flow Recommenders
74 Exploration and Regularization of the Latent Action Space in Recommendation (2023) Offline RL for Recommendation
3 Lantao Hu 54 3
74 Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation (2025) Diffusion and Flow Recommenders
70 Distinguished Quantized Guidance for Diffusion-based Sequence Recommendation (2025) Diffusion and Flow Recommenders
4 Xuanhua Yang 49 3
69 Towards Personalized Bundle Creative Generation with Contrastive Non-Autoregressive Decoding (2022) Diffusion and Flow Recommenders
56 GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation (2026) Generative Retrieval and Ranking
7 Chongming Gao 47 3
60 Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation (2023) Popularity Bias and Debiasing in Recommendation
52 Reformulating Conversational Recommender Systems as Tri-Phase Offline Policy Learning (2024) Conversational Recommendation Systems
8 Fei Sun 0001 47 3
58 Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning (2021) Conversational Recommendation Systems
54 Exact-K Recommendation via Maximal Clique Optimization (2019) Graph-based User and Item Representation Learning
9 Yuta Saito assigned 47 3
51 Off-Policy Evaluation of Ranking Policies under Diverse User Behavior (2023) Recommendation with Search and Ranking
49 Evaluating the Robustness of Off-Policy Evaluation (2021) Recommendation Systems and Evaluation
10 Ziru Liu assigned 47 3
69 Diff-MSR: A Diffusion Model Enhanced Paradigm for Cold-Start Multi-Scenario Recommendation (2024) Diffusion and Flow Recommenders
60 Modeling User Retention through Generative Flow Networks (2024) Multi-Behavior Recommendation
11 Jianling Wang 46 3
53 A Hierarchical Self-Attentive Model for Recommending User-Generated Item Lists (2019) Efficient and Long-Sequence Sequential Recommendation
49 Learning to Augment for Casual User Recommendation (2022) Contrastive and Self-Supervised Sequential Recommendation
12 Seongmin Park 0002 46 3
52 Temporal Linear Item-Item Model for Sequential Recommendation (2025) Efficient and Long-Sequence Sequential Recommendation
48 MUFFIN: Mixture of User-Adaptive Frequency Filtering for Sequential Recommendation (2025) Sequential Recommendation with User Modeling and Profiling
13 Yongjing Hao 46 3
63 Meta-Optimized Joint Generative and Contrastive Learning for Sequential Recommendation (2024) Diffusion and Flow Recommenders
60 Click is not equal to purchase: multi-task reinforcement learning for multi-behavior recommendation (2023) Multi-Behavior Recommendation
14 Ching Law 45 3
74 Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluation (2025) Diffusion and Flow Recommenders
74 MoFu: Scale-Aware Modulation and Fourier Fusion for Multi-Subject Video Generation (2026) Diffusion and Flow Recommenders
16 Han Li 0005 45 3
74 DiffusionGS: Generative Search with Query Conditioned Diffusion in Kuaishou (2026) Diffusion and Flow Recommenders
49 Stratified Expert Cloning for Retention-Aware Recommendation at Scale (2025) Long-Term Engagement and Retention Optimization
17 Jiancan Wu 45 3
69 Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion (2023) Diffusion and Flow Recommenders
63 Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation (2025) Diffusion and Flow Recommenders
18 Bo Peng 0009 44 3
53 M2: Mixed Models With Preferences, Popularities and Transitions for Next-Basket Recommendation (2023) Efficient and Long-Sequence Sequential Recommendation
48 eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data (2024) Efficiency and Deployment
20 Ed H. Chi 44 3
48 PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations (2026) Item Tokenization and Semantic IDs
46 Top-K Off-Policy Correction for a REINFORCE Recommender System (2019) Long-Term Engagement and Retention Optimization
21 Mengyuan Jing 44 3
67 Generating Difficulty-aware Negative Samples via Conditional Diffusion for Multi-modal Recommendation (2025) Diffusion and Flow Recommenders
66 Interest-Aware Graph Contrastive Learning for Recommendation With Diffusion-Based Augmentation (2026) Graph Contrastive and Self-Supervised Learning
22 Xiao Zhang 0034 44 3
50 When Search Meets Recommendation: Learning Disentangled Search Representation for Recommendation (2023) Recommendation with Search and Ranking
48 Test-Time Alignment with State Space Model for Tracking User Interest Shifts in Sequential Recommendation (2025) Sequential Recommendation with User Modeling and Profiling
23 Hao Liu 0078 44 3
49 WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Merging (2026) Efficiency and Deployment
48 Mitigating Distribution Shifts in Sequential Recommendation: An Invariance Perspective (2025) Contrastive and Self-Supervised Sequential Recommendation
24 Jieming Zhu 43 3
48 EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration (2024) Item Tokenization and Semantic IDs
47 Directional Adversarial Training for Recommender Systems (2020) Robustness to Noise and Distribution Shift
25 Jin Chen 0008 43 3
48 Cooperative Retriever and Ranker in Deep Recommenders (2023) Recommendation with Search and Ranking
47 Dual Test-Time Training for Out-of-Distribution Recommender System (2025) Robustness to Noise and Distribution Shift
26 Bowen Zheng 0005 43 3
48 Enhancing Sequential Recommender with Large Language Models for Joint Video and Comment Recommendation (2025) Sequential and Session-based Recommendation
48 Reciprocal Sequential Recommendation (2023) Sequential Recommendation with User Modeling and Profiling
27 Qiang Luo 0004 43 3
56 OneLoc: Geo-Aware Generative Recommender Systems for Local Life Service (2026) Item Tokenization and Semantic IDs
56 DiffGRM: Diffusion-based Generative Recommendation Model (2026) Diffusion and Flow Recommenders
28 Bo Chen 0023 43 3
49 ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation (2024) Sequential and Session-based Recommendation
48 Full-Stack Optimized Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation (2026) Sequential and Session-based Recommendation
30 Jizhi Zhang 43 3
48 Navigating Large Language Models for Recommendation: From Architecture to Learning Paradigms and Deployment (2025) Efficiency and Deployment
48 BSL: Understanding and Improving Softmax Loss for Recommendation (2024) Popularity Bias and Debiasing in Recommendation
31 Yongxin Ni 43 3
46 Bridging NIP and MLM: A Unified Meta-Learning Framework for Sequential Recommendation (2026) Meta-Learning for Cold-Start Recommendation
45 Collaborative filtering with implicit feedback via learning pairwise preferences over user-groups and item-sets (2022) Negative Sampling and Loss Functions
32 Jiangxia Cao 43 3
72 From Agnostic to Specific: Latent Preference Diffusion for Multi-Behavior Sequential Recommendation (2026) Diffusion and Flow Recommenders
46 CDSRNP: Cross-Domain Sequential Recommendation via Neural Process (2025) Cross-Domain Sequential Recommendation
33 Riwei Lai 43 3
47 DAR: Dimension-Adaptive Recommendation with Multi-Granular Noise Control (2025) Robustness to Noise and Distribution Shift
47 Proxy-Aware Cross-Domain Sequential Recommendation (2023) Cross-Domain Sequential Recommendation
34 Lichan Hong 43 3
51 Off-policy Learning in Two-stage Recommender Systems (2020) Recommendation with Search and Ranking
47 PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations (2026) Generative Recommendation and Item Tokenization
35 Noveen Sachdeva 43 3
48 Explicit Modelling of the Implicit Short Term User Preferences for Music Recommendation (2018) Sequential Recommendation with User Modeling and Profiling
47 ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation (2025) Item Tokenization and Semantic IDs