Task-Aware Automated User Profile Generation for Recommendation Simulation Using Large Language Models
Abstract
Large Language Model (LLM)-based agent simulation has emerged as a promising approach to meet the increasing demand for real-time and rigorous evaluation in modern recommender systems. A typical LLM-driven simulation framework comprises three essential components: the profile module, memory module, and action module. However, existing studies have primarily concentrated on enhancing the memory and action modules, with limited attention to profile generation, which plays a pivotal role in ensuring realistic agent behaviours and aligning simulated interactions with real user dynamics. Moreover, the scarcity of datasets specifically designed for recommendation simulations has led to heavy reliance on manually crafted profiles, significantly limiting the scalability and generalisability of simulation frameworks across different datasets. To address these challenges, this work proposes an Automated Profile Generation Framework for Recommendation Simulation, APG4RecSim, that constructs realistic, coherent, and robust user profiles with minimal supervision. Extensive experiments on three benchmark datasets demonstrate that APG4RecSim achieves the best overall performance on discrimination, ranking, and rating tasks, improving ranking quality by up to 7% in nDCG@10 and reducing rating distribution divergence by 8% in JSD compared to existing profile-generation baselines. Beyond overall performance gains, our results show that APG4RecSim produces profiles that are resilient to popularity- and position-induced biases and maintain stable performance across datasets and different LLMs.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Saber Zerhoudi | 59 | 71 Theory and Toolkits for User Simulation in the Era of Generative AI: User Modeling, Synthetic Data Generation, and System Evaluation (2025) User Simulation for Interactive Systems 71 UXSim: Towards a Hybrid User Search Simulation (2025) User Simulation for Interactive Systems 71 Simulation of Interactive Information Retrieval: A Guided Tour (2026) User Simulation for Interactive Systems |
|
| 2 | Shijie Wang 0002 | 41 | 71 Towards Next-Generation Recommender Systems: A Benchmark for Personalized Recommendation Assistant with LLMs (2026) Efficiency and Deployment 44 CheatAgent: Attacking LLM-Empowered Recommender Systems via LLM Agent (2024) Preference Alignment and Optimization 43 Continuous-time Discrete-space Diffusion Model for Recommendation (2026) Diffusion and Flow Recommenders |
Area chair
| Area chair | Match | |
|---|---|---|
| Pengyue Jia | 48 |
Area-chair candidates (6)
| Pengyue Jia assigned | 48 | 10 papers | |
| Ziyi Ye | 48 | 14 papers | |
| Jun Xu 0001 | 42 | 2 papers | |
| Maolin Wang 0001 | 41 | 15 papers | |
| Kun Gai | 40 | 5 papers | |
| Yanan Niu | 40 | 15 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Saber Zerhoudi assigned | 59 | 3 | 71 Theory and Toolkits for User Simulation in the Era of Generative AI: User Modeling, Synthetic Data Generation, and System Evaluation (2025) User Simulation for Interactive Systems 71 UXSim: Towards a Hybrid User Search Simulation (2025) User Simulation for Interactive Systems |
|
| 2 | Yang Zhang 0072 | 52 | 3 | 71 AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment (2026) Efficiency and Deployment 57 Preliminary Study on Incremental Learning for Large Language Model-based Recommender Systems (2024) Efficiency and Deployment |
|
| 3 | Chongming Gao | 51 | 3 | 73 Agentic Feedback Loop Modeling Improves Recommendation and User Simulation (2025) User Profile and Persona Modeling 53 Don't Start Over: A Cost-Effective Framework for Migrating Personalized Prompts Between LLMs (2026) Efficiency and Deployment |
|
| 4 | Jizhi Zhang | 49 | 3 | 73 Agentic Feedback Loop Modeling Improves Recommendation and User Simulation (2025) User Profile and Persona Modeling 57 Navigating Large Language Models for Recommendation: From Architecture to Learning Paradigms and Deployment (2025) Efficiency and Deployment |
|
| 5 | Bo Chen 0023 | 49 | 3 | 56 Towards efficient and effective unlearning of large language models for recommendation (2025) Efficiency and Deployment 54 Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models (2024) Knowledge Infusion and Open-World Augmentation |
|
| 8 | Bo Peng 0009 | 47 | 3 | 57 eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data (2024) Efficiency and Deployment 44 M2: Mixed Models With Preferences, Popularities and Transitions for Next-Basket Recommendation (2023) Efficient and Long-Sequence Sequential Recommendation |
|
| 9 | Junjie Zhang 0009 | 47 | 3 | 57 Tapping the Potential of Large Language Models as Recommender Systems: A Comprehensive Framework and Empirical Analysis (2025) Efficiency and Deployment 57 Large Language Models are Zero-Shot Rankers for Recommender Systems (2024) Efficiency and Deployment |
|
| 10 | Huifeng Guo | 47 | 3 | 68 Prompt Tuning as User Inherent Profile Inference Machine (2025) Efficiency and Deployment 56 LLMTreeRec: Unleashing the Power of Large Language Models for Cold-Start Recommendations (2025) Generative Recommendation and Item Tokenization |
|
| 11 | Zhengyi Yang 0007 | 45 | 3 | 45 On Softmax Direct Preference Optimization for Recommendation (2024) Preference Alignment and Optimization 45 AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization (2025) Preference Alignment and Optimization |
|
| 12 | Jianling Wang | 45 | 3 | 56 LLMs for User Interest Exploration in Large-scale Recommendation Systems (2024) Sequential and Session-based Recommendation 53 Fresh Content Recommendation at Scale: A Multi-funnel Solution and the Potential of LLMs (2024) Efficiency and Deployment |
|
| 13 | Jaime Arguello | 44 | 3 | 64 Tip of the Tongue Query Elicitation for Simulated Evaluation (2025) User Simulation for Interactive Systems 64 Multilingual and Domain-Agnostic Tip-of-the-Tongue Query Generation for Simulated Evaluation (2026) User Simulation for Interactive Systems |
|
| 14 | Han Li 0005 | 44 | 3 | 73 Lasso: Large Language Model-based User Simulator for Cross-Domain Recommendation (2025) User Simulation for Interactive Systems 51 LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application (2025) Efficiency and Deployment |
|
| 15 | Shuchang Liu 0001 | 44 | 3 | 59 LLM-Powered User Simulator for Recommender System (2025) Recommendation Systems and Evaluation 59 KuaiSim: A Comprehensive Simulator for Recommender Systems (2023) Recommendation Systems and Evaluation |
|
| 16 | Xiao Zhang 0034 | 43 | 3 | 60 Similarity = Value? Consultation Value-Assessment and Alignment for Personalized Search (2025) Personalized and Context-Aware Retrieval 46 Large Language Models Enhanced Collaborative Filtering (2024) Preference Alignment and Optimization |
|
| 17 | Jin Chen 0008 | 43 | 3 | 58 When large language models meet personalization: perspectives of challenges and opportunities (2024) User Profile and Persona Modeling 47 RecStudio: Towards a Highly-Modularized Recommender System (2023) Recommendation Systems and Evaluation |
|
| 18 | Yongqi Li 0001 | 43 | 3 | 60 Where and What: Reasoning Dynamic and Implicit Preferences in Situated Conversational Recommendation (2026) Conversational Recommendation Systems 53 Efficient Inference for Large Language Model-based Generative Recommendation (2025) Generative Retrieval and Ranking |
|
| 19 | Wenlin Zhang 0001 | 43 | 3 | 74 Exploring Recommender System Evaluation: A Multi-Modal LLM Agent Framework for A/B Testing (2026) Recommendation Systems and Evaluation 56 LLMTreeRec: Unleashing the Power of Large Language Models for Cold-Start Recommendations (2025) Generative Recommendation and Item Tokenization |
|
| 20 | Fei Sun 0001 | 42 | 3 | 47 Neural Re-ranking in Multi-stage Recommender Systems: A Review (2022) Recommendation Systems and Evaluation 47 Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning (2021) Conversational Recommendation Systems |
|
| 21 | Wei Guo 0006 | 42 | 3 | 57 Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation (2025) Efficiency and Deployment 47 Dataset Regeneration for Sequential Recommendation (2024) Recommendation Systems and Evaluation |
|
| 23 | Yang Song 0008 | 42 | 3 | 47 LabelCraft: Empowering Short Video Recommendations with Automated Label Crafting (2024) Recommendation Systems and Evaluation 45 Query-Less: Predicting Task Repetition for NextGen Proactive Search and Recommendation Engines (2016) User Behavior and Search Interaction |
|
| 24 | Shijie Wang 0002 assigned | 41 | 3 | 71 Towards Next-Generation Recommender Systems: A Benchmark for Personalized Recommendation Assistant with LLMs (2026) Efficiency and Deployment 44 CheatAgent: Attacking LLM-Empowered Recommender Systems via LLM Agent (2024) Preference Alignment and Optimization |
|
| 25 | Keqin Bao | 41 | 3 | 73 Agentic Feedback Loop Modeling Improves Recommendation and User Simulation (2025) User Profile and Persona Modeling 55 Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation (2023) Preference Alignment and Optimization |
|
| 26 | Jiancan Wu | 41 | 3 | 49 Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learning (2024) Reasoning Evaluation and Benchmarks 45 Larger or Smaller Reward Margins to Select Preferences for LLM Alignment? (2025) Preference Alignment and Optimization |
|
| 27 | Seongmin Park 0002 | 41 | 3 | 46 Empowering Retrieval-based Conversational Recommendation with Contrasting User Preferences (2025) Conversational and Interactive Recommendation 45 Temporal Linear Item-Item Model for Sequential Recommendation (2025) Efficient and Long-Sequence Sequential Recommendation |
|
| 28 | Gustavo Penha | 41 | 3 | 70 Evaluating Podcast Recommendations with Profile-Aware LLM-as-a-Judge (2025) Recommendation Systems and Evaluation 63 What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation (2020) Conversational Recommendation Systems |
|
| 29 | Bowen Zheng 0005 | 41 | 3 | 47 Towards a More User-Friendly and Easy-to-Use Benchmark Library for Recommender Systems (2023) Recommendation Systems and Evaluation 44 Reciprocal Sequential Recommendation (2023) Sequential Recommendation with User Modeling and Profiling |
|
| 30 | Min Hou 0004 | 41 | 3 | 56 RecCocktail: A Generalizable and Efficient Framework for LLM-Based Recommendation (2026) Preference Alignment and Optimization 54 WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Merging (2026) Efficiency and Deployment |
|
| 31 | Yimeng Bai | 41 | 3 | 47 LabelCraft: Empowering Short Video Recommendations with Automated Label Crafting (2024) Recommendation Systems and Evaluation 47 Leveraging Watch-time Feedback for Short-Video Recommendations: A Causal Labeling Framework (2023) Recommendation Systems and Evaluation |
|
| 33 | Jieming Zhu | 40 | 3 | 57 Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation (2025) Efficiency and Deployment 54 Efficient and Deployable Knowledge Infusion for Open-World Recommendations via Large Language Models (2026) Knowledge Infusion and Open-World Augmentation |
|
| 35 | Ed H. Chi | 40 | 3 | 60 LLMs for User Interest Exploration in Large-scale Recommendation Systems (2024) Generative Recommendation and Item Tokenization 52 PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations (2026) Item Tokenization and Semantic IDs |