Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

IPQA: A Benchmark for Core Intent Identification in Personalized Question Answering

sigir26-150 · Full research paper · Jieyong Kim, Maryam Amirizaniani, Soojin Yoon, Dongha Lee
Retrieval-Augmented and Open-Domain · 50% Efficiency and Deployment · 25% Personalized Question Answering and · 25%
Abstract

Intent identification serves as the foundation for generating appropriate responses in personalized question answering (PQA). However, existing benchmarks evaluate only response quality or retrieval performance without directly measuring intent identification capabilities. This gap is critical because without understanding which intents users prioritize, systems cannot generate responses satisfying individual information needs. To address this, we introduce the concept of core intents: intents users prioritize when selecting answers to satisfy their information needs. To evaluate these core intents, we propose IPQA, a benchmark for core Intent identification in Personalized Question Answering. Since users do not explicitly state their prioritized intents, we derive core intents from observable behavior patterns in answer selection, grounded in satisficing theory where users choose answers meeting their acceptance thresholds. We construct a dataset with various domains through systematic filtering, LLM-based annotation, and rigorous quality control combining automated verification with human validation. Experimental evaluations across state-of-the-art language models reveal that current systems struggle with core intent identification in personalized contexts. Models fail to identify core intents from user histories, with performance degrading as question complexity increases.

Assigned reviewers

#ReviewerMatchWhy (closest evidence publications in the taxonomy)
1 Idan Szpektor
57
72 On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method (2023) Conversational Search and Retrieval
71 Will My Question Be Answered? Predicting "Question Answerability" in Community Question-Answering Sites (2013) Retrieval-Augmented and Open-Domain QA
71 Learning from the past: answering new questions with past answers (2012) Retrieval-Augmented and Open-Domain QA
2 Jinheon Baek
45
72 Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (2023) Conversational Search and Retrieval
70 Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks (2023) Retrieval-Augmented and Open-Domain QA
58 Knowledge-Augmented Large Language Models for Personalized Contextual Query Suggestion (2024) Conversational Search and Retrieval

Area chair

Area chairMatch
Pengyue Jia 46
Area-chair candidates (6)
Pengyue Jia assigned 4610 papers
Jun Xu 0001 452 papers
Xiaoxue Zang 444 papers
Kam-Fai Wong 426 papers
Xiaoxi Li 0005 4215 papers
Zhijing Wu 0001 419 papers

Candidates from the panel ranked by match score

#ReviewerMatchLoadWhy
1 Idan Szpektor assigned 57 3
72 On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method (2023) Conversational Search and Retrieval
71 Will My Question Be Answered? Predicting "Question Answerability" in Community Question-Answering Sites (2013) Retrieval-Augmented and Open-Domain QA
2 Jing Liu 0022 50 3
71 A Regularized Competition Model for Question Difficulty Estimation in Community Question Answering Services (2014) Retrieval-Augmented and Open-Domain QA
67 Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation (2025) Knowledge Conflict and Faithfulness
3 Farzad Shami 49 3
71 From Single to Multi-agent Reasoning: Advancing GeneGPT for Genomics QA (2026) Retrieval-Augmented and Open-Domain QA
37 Benchmarking Large Language Models for Knowledge Graph Validation (2026) KG-Augmented LLM Reasoning
4 Yang Zhang 0072 48 3
53 Don't Start Over: A Cost-Effective Framework for Migrating Personalized Prompts Between LLMs (2026) Efficiency and Deployment
51 Preliminary Study on Incremental Learning for Large Language Model-based Recommender Systems (2024) Efficiency and Deployment
5 Chen Zhao 0013 47 3
71 A dataset and baselines for sequential open-domain question answering (2018) Retrieval-Augmented and Open-Domain QA
59 Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation (2021) Dense Bi-Encoder Training and Representation Learning
6 Adam Jatowt 47 3
69 ArchivalQA: A Large-scale Benchmark Dataset for Open-Domain Question Answering over Historical News Collections (2022) Multi-hop and Complex Reasoning QA
46 Inferential Question Answering (2026) Multi-hop and Complex Reasoning QA
7 Bo Chen 0023 46 3
52 How Can Recommender Systems Benefit from Large Language Models: A Survey (2025) Efficiency and Deployment
52 Large language models make sample-efficient recommender systems (2025) Efficiency and Deployment
9 Yongqi Li 0001 45 3
54 EARN: Efficient Inference Acceleration for LLM-based Generative Recommendation by Register Tokens (2025) Generative Retrieval and Ranking
54 Efficient Inference for Large Language Model-based Generative Recommendation (2025) Generative Retrieval and Ranking
11 Mariano Barone 45 3
67 A Multilingual Multimodal Medical Examination Dataset for Visual Question Answering in Healthcare (2025) Retrieval-Augmented and Open-Domain QA
66 SOLVE-Med: Specialized Orchestration for Leading Vertical Experts Across Medical Specialties (2025) Retrieval-Augmented and Open-Domain QA
12 Wenlin Zhang 0001 45 3
57 Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery (2026) Deep Research and Report Generation
55 Bridging Personalization and AI: From RAG to Agent (2026) Personalized and Conversational RAG
13 Jinheon Baek assigned 45 3
72 Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (2023) Conversational Search and Retrieval
70 Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks (2023) Retrieval-Augmented and Open-Domain QA
14 Qinggang Zhang 45 3
50 Self-Monitoring Large Language Models for Click-Through Rate Prediction (2026) Efficiency and Deployment
49 LLM Collaborative Filtering: User-Item Graph as New Language (2026) Efficiency and Deployment
15 Falk Scholer 45 3
72 Document Summarization for Answering Non-Factoid Queries (2018) Retrieval-Augmented and Open-Domain QA
72 Ranking Documents by Answer-Passage Quality (2018) Retrieval-Augmented and Open-Domain QA
16 Junjie Zhang 0009 44 3
51 Tapping the Potential of Large Language Models as Recommender Systems: A Comprehensive Framework and Empirical Analysis (2025) Efficiency and Deployment
51 Large Language Models are Zero-Shot Rankers for Recommender Systems (2024) Efficiency and Deployment
17 Kaixin Ma 44 3
71 Chain-of-Skills: A Configurable Model for Open-Domain Question Answering (2023) Retrieval-Augmented and Open-Domain QA
70 Open Domain Question Answering with A Unified Knowledge Interface (2022) Multimodal and Graph RAG
18 Jialong Wu 0007 44 3
53 PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation (2025) Efficiency and Deployment
43 AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation (2025) Query Reformulation and Expansion
19 Jiapu Wang 44 3
64 Large-Small Model Synergy with Multimodal Fine-Grained Heuristics for Knowledge-Based Visual Question Answering (2025) Retrieval-Augmented and Open-Domain QA
58 Decomposing and Revising What Language Models Generate (2025) Multi-Hop and Iterative Retrieval Reasoning
20 Liwei Wang 0009 44 3
71 MVP-Tuning: Multi-View Knowledge Retrieval with Prompt Tuning for Commonsense Reasoning (2023) Retrieval-Augmented and Open-Domain QA
46 Making Long-Context Language Models Better Multi-Hop Reasoners (2024) Multi-hop and Complex Reasoning QA
22 Wenyu Huang 44 3
56 Prompting large language models with knowledge graphs for question answering involving long-tail facts (2025) KG-Augmented LLM Reasoning
46 Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation (2025) Multi-hop and Complex Reasoning QA
23 Yujia Zhou 0002 43 3
68 RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation (2025) Evidence Selection and Context Refinement
46 Enhancing Generative Retrieval with Reinforcement Learning from Relevance Feedback (2023) Recommendation with Search and Ranking
24 Weiwei Sun 0001 43 3
45 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking
44 TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy (2025) Reasoning-Intensive Retrieval and Ranking
25 Huifeng Guo 43 3
55 A Survey of Personalization: From RAG to Agent (2026) Agentic and Deep Search RAG
55 Personalize Before Retrieve: LLM-based Personalized Query Expansion for User-Centric Retrieval (2026) Personalized and Conversational RAG
26 Han Li 0005 43 3
56 Retrieval Augmented Generation with Collaborative Filtering for Personalized Text Generation (2025) Personalized and Conversational RAG
51 LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application (2025) Efficiency and Deployment
27 Jizhi Zhang 42 3
51 Navigating Large Language Models for Recommendation: From Architecture to Learning Paradigms and Deployment (2025) Efficiency and Deployment
51 Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation (2023) Preference Alignment and Optimization
28 Gustavo Penha 42 3
49 Evaluating Podcast Recommendations with Profile-Aware LLM-as-a-Judge (2025) Recommendation Systems and Evaluation
44 Curriculum Learning Strategies for IR (2020) Ranking Distillation and Adaptation
30 Yingyi Zhang 0001 42 3
57 Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery (2026) Deep Research and Report Generation
55 Bridging Personalization and AI: From RAG to Agent (2026) Personalized and Conversational RAG
32 Shu Zhou 0002 42 3
72 LOSDF: A logical optimization and semantic decoupling framework for question answering in multi-party conversations (2025) Conversational Search and Retrieval
60 Enhancing Extractive Question Answering in Multiparty Dialogues with Logical Inference Memory Network (2025) Conversational Search and Retrieval
33 Yuchen Li 0006 42 3
45 S2phere: Semi-Supervised Pre-training for Web Search over Heterogeneous Learning to Rank Data (2023) Web-Scale Learning to Rank
43 ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026) LLM-Based Reranking
34 Xinyu Ma 0001 41 3
46 Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation (2025) LLM-Derived Knowledge Infusion
45 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking
35 Hao Liu 0078 41 3
51 WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Merging (2026) Efficiency and Deployment
38 Mitigating Fine-tuning Bias: A Parameter-Efficient Debiasing Framework for Large Language Models (2026) Bias Mitigation Methods