Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Towards Knowledgeable Deep Research: Framework and Benchmark

sigir26-093 · Full research paper · Wenxuan Liu, Zixuan Li, Long Bai, Chunmao Zhang, Fenghui Zhang, Zhuo Chen, Wei Li, Yuxin Zuo, Fei Wang, Bingbing Xu, Xuhui Jiang, Jin Zhang, Xiaolong Jin, Jiafeng Guo, Tat‐Seng Chua, Xueqi Cheng
Deep Research and Report Generation · 50% Evidence Selection and Context Refin · 25% Agentic and Deep Search RAG · 25%
Abstract

Deep Research (DR) requires LLM agents to autonomously perform multi-step information seeking, processing, and reasoning to generate comprehensive reports. In contrast to existing studies that mainly focus on unstructured web content, a more challenging DR task should additionally utilize structured knowledge to provide a solid data foundation, facilitate quantitative computation, and lead to in-depth analyses. In this paper, we refer to this novel task as Knowledgeable Deep Research (KDR), which requires DR agents to generate reports with both structured and unstructured knowledge. Furthermore, we propose the Hybrid Knowledge Analysis framework (HKA), a multi-agent architecture that reasons over both kinds of knowledge and integrates the texts, figures, and tables into coherent multimodal reports. The key design is the Structured Knowledge Analyzer, which utilizes both coding and vision-language models to produce figures, tables, and corresponding insights. To support systematic evaluation, we construct KDR-Bench, which covers 9 domains, includes 41 expert-level questions, and incorporates a large number of structured knowledge resources (e.g., 1,252 tables). We further annotate the main conclusions and key points for each question and propose three categories of evaluation metrics including general-purpose, knowledge-centric, and vision-enhanced ones. Experimental results demonstrate that HKA consistently outperforms most existing DR agents on general-purpose and knowledge-centric metrics, and even surpasses the Gemini DR agent on vision-enhanced metrics, highlighting its effectiveness in deep, structure-aware knowledge analysis. Finally, we hope this work can serve as a new foundation for structured knowledge analysis in DR agents and facilitate future multimodal DR studies.

Assigned reviewers

#ReviewerMatchWhy (closest evidence publications in the taxonomy)
1 Jialong Wu 0007
57
73 WebWalker: Benchmarking LLMs in Web Traversal (2025) Agentic and Deep Search RAG
72 OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking (2025) Deep Research and Report Generation
71 WebDancer: Towards Autonomous Information Seeking Agency (2025) Deep Research and Report Generation
2 Wenlin Zhang 0001
49
87 MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search (2026) Agentic and Deep Search RAG
73 To Search or Not to Search: Aligning the Decision Boundary of Deep Search Agents via Causal Intervention (2026) Agentic and Deep Search RAG
69 Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery (2026) Deep Research and Report Generation
3 Yingyi Zhang 0001
47
87 MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search (2026) Agentic and Deep Search RAG
71 Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery (2026) Deep Research and Report Generation
60 To Search or Not to Search: Aligning the Decision Boundary of Deep Search Agents via Causal Intervention (2026) Agentic and Deep Search RAG

Area chair

Area chairMatch
Xiaoxi Li 0005 59
Area-chair candidates (6)
Xiaoxi Li 0005 assigned 5915 papers
Hongjin Qian 4912 papers
Jun Xu 0001 COI: co-author 422 papers
Xiaoxue Zang 404 papers
Fernando Diaz 0001 3912 papers
Pengyue Jia COI: co-author 3810 papers

Candidates from the panel ranked by match score

#ReviewerMatchLoadWhy
2 Jialong Wu 0007 assigned 57 3
73 WebWalker: Benchmarking LLMs in Web Traversal (2025) Agentic and Deep Search RAG
72 OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking (2025) Deep Research and Report Generation
3 Wenlin Zhang 0001 assigned 49 3
87 MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search (2026) Agentic and Deep Search RAG
73 To Search or Not to Search: Aligning the Decision Boundary of Deep Search Agents via Causal Intervention (2026) Agentic and Deep Search RAG
5 Kaixin Ma 47 3
60 Retrieval-augmented GUI Agents with Generative Guidelines (2025) Agentic and Deep Search RAG
55 Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models (2024) Evidence Selection and Context Refinement
6 Yingyi Zhang 0001 assigned 47 3
87 MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search (2026) Agentic and Deep Search RAG
71 Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery (2026) Deep Research and Report Generation
7 Weiwei Sun 0001 45 3
60 Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning (2025) Agentic and Deep Search RAG
56 Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models (2026) Evidence Selection and Context Refinement
8 Yujia Zhou 0002 43 3
57 RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation (2025) Evidence Selection and Context Refinement
56 CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks (2024) Generative Retrieval
9 Navve Wasserman 43 3
48 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval
48 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval
10 Jing Liu 0022 43 3
56 REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering (2024) Evidence Selection and Context Refinement
55 Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generation (2026) Evidence Selection and Context Refinement
11 Yuchen Li 0006 43 3
61 Retain to Refine: Adaptive Online Question Answering via Query Routing and Long-Short Memory (2026) Agentic and Deep Search RAG
56 Enhancing Retrieval-Augmented Generation via Evidence Tree Search (2025) Evidence Selection and Context Refinement
12 Farzad Shami 42 3
45 GROKE: Vision-Free Navigation Instruction Evaluation via Graph Reasoning on OpenStreetMap (2026) Reasoning Evaluation and Benchmarks
41 From Single to Multi-agent Reasoning: Advancing GeneGPT for Genomics QA (2026) Retrieval-Augmented and Open-Domain QA
13 Wonduk Seo 42 3
45 Automated Visualization Code Synthesis via Multi-path Reasoning and Feedback-Driven Optimizations (2026) Reasoning Evaluation and Benchmarks
44 SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science (2026) Multi-Agent and Tool-Use Orchestration
14 Yu Gu 0016 42 3
72 Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge (2025) Deep Research and Report Generation
45 AgentBench: Evaluating LLMs as Agents (2024) Reasoning Evaluation and Benchmarks
15 Shuaiqiang Wang COI: co-author 42 3
56 Enhancing Retrieval-Augmented Generation via Evidence Tree Search (2025) Evidence Selection and Context Refinement
47 Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning (2025) Multi-Hop and Iterative Retrieval Reasoning
16 Xinyu Ma 0001 COI: co-author 42 3
60 Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning (2025) Agentic and Deep Search RAG
56 Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models (2026) Evidence Selection and Context Refinement
18 Chen Zhao 0013 42 3
72 Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revision (2026) Deep Research and Report Generation
57 SportReason: Evaluating Retrieval-Augmented Reasoning across Tables and Text for Sports Question Answering (2025) Multi-Hop and Iterative Retrieval Reasoning
19 Weipeng Chen 41 3
61 ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning (2025) RL Training of Search Agents
57 RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation (2025) Evidence Selection and Context Refinement
20 Daiting Shi COI: co-author 41 3
56 Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation (2025) Evidence Selection and Context Refinement
44 Reinforced Efficient Reasoning via Semantically Diverse Exploration (2026) RL Training of Search Agents
21 Xingyu Tan 0001 40 3
42 Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning (2025) KG-Augmented LLM Reasoning
42 HydraRAG: Structured Cross-Source Enhanced Large Language Model Reasoning (2025) KG-Augmented LLM Reasoning
22 Tingyu Song 40 3
56 LimRank: Less is More for Reasoning-Intensive Information Reranking (2025) Evidence Selection and Context Refinement
40 Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems (2026) Reasoning-Intensive Retrieval and Ranking
23 Xi Ye 0003 40 3
72 Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revision (2026) Deep Research and Report Generation
45 MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning (2024) Reasoning Evaluation and Benchmarks
24 Haritz Puerto 40 3
45 MetaQA: Combining Expert Agents for Multi-Skill Question Answering (2023) Retrieval-Augmented and Open-Domain QA
41 Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs (2024) Reward Modeling and Process Supervision
25 Ori Yoran 40 3
57 Making Retrieval-Augmented Language Models Robust to Irrelevant Context (2024) Evidence Selection and Context Refinement
46 Answering Questions by Meta-Reasoning over Multiple Chains of Thought (2023) Reasoning-Intensive Retrieval and Ranking
27 Qinggang Zhang 40 3
44 You Don't Need Pre-Built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures (2026) Multi-Hop and Iterative Retrieval Reasoning
42 FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation (2025) Knowledge Conflict and Faithfulness
29 Han Li 0005 39 3
56 QE-RAG: A Robust Retrieval-Augmented Generation Benchmark for Query Entry Errors (2026) Evidence Selection and Context Refinement
44 Towards Context-aware Reasoning-enhanced Generative Searching in E-commerce (2026) Recommendation with Search and Ranking
30 Jinheon Baek 39 3
57 Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks (2023) Retrieval-Augmented and Open-Domain QA
47 Efficient Long Context Language Model Retrieval with Compression (2025) Learned Sparse Retrieval Efficiency
31 Peitian Zhang 39 3
87 Search-o1: Agentic Search-Enhanced Large Reasoning Models (2025) Agentic and Deep Search RAG
50 A Multi-Task Embedder For Retrieval Augmented LLMs (2024) Evidence Selection and Context Refinement
32 Mariano Barone 39 3
45 SOLVE-Med: Specialized Orchestration for Leading Vertical Experts Across Medical Specialties (2025) Retrieval-Augmented and Open-Domain QA
41 Combining Evidence and Reasoning for Biomedical Fact-Checking (2025) Automated Fact Checking and Claim Verification
33 Jiapu Wang 39 3
57 Atomic Fact Decomposition Helps Attributed Question Answering (2025) Multi-Hop and Iterative Retrieval Reasoning
57 Decomposing and Revising What Language Models Generate (2025) Multi-Hop and Iterative Retrieval Reasoning
34 Venktesh V 38 3
55 The Surprising Effectiveness of Rankers trained on Expanded Queries (2024) Evidence Selection and Context Refinement
52 Reproducing Adaptive Reranking for Reasoning-Intensive IR (2026) Evidence Selection and Context Refinement
35 Wenyu Huang 38 3
42 Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation (2025) Multi-hop and Complex Reasoning QA
41 Prompting large language models with knowledge graphs for question answering involving long-tail facts (2025) KG-Augmented LLM Reasoning