Chain of Evidence: Pixel-Level Visual Attribution for Iterative Retrieval-Augmented Generation
sigir26-016 · Full research paper · Peiyang Liu, Ziqiang Cui, Xi Wang, Di Liang, Wei Ye
Universal Multimodal Embedding and M · 50% Multi-Hop and Iterative Retrieval Re · 25% Multimodal and Graph RAG · 25%
Abstract
Iterative Retrieval-Augmented Generation (iRAG) has emerged as a powerful paradigm for answering complex multi-hop questions by progressively retrieving and reasoning over external documents. However, current systems predominantly operate on parsed text, which creates two critical bottlenecks: (1) Coarse-grained attribution, where users are burdened with manually locating evidence within lengthy documents based on vague text-level citations; and (2) Visual semantic loss, where the conversion of visually rich documents (e.g., slides, PDFs with charts) into text discards spatial logic and layout cues essential for reasoning. To bridge this gap, we present Chain of Evidence (CoE), a retriever-agnostic visual attribution framework that leverages Vision-Language Models to reason directly over screenshots of retrieved document candidates. CoE eliminates format-specific parsing and outputs precise bounding boxes, visualizing the complete reasoning chain within the retrieved candidate set. We evaluate CoE on two distinct benchmarks: Wiki-CoE, a large-scale dataset of structured web pages derived from 2WikiMultiHopQA, and SlideVQA, a challenging dataset of presentation slides featuring complex diagrams and free-form layouts. Experiments demonstrate that fine-tuned Qwen3-VL-8B-Instruct achieves robust performance, significantly outperforming text-based baselines in scenarios requiring visual layout understanding, while establishing a retriever-agnostic solution for pixel-level interpretable iRAG. Our code is available at https://github.com/PeiYangLiu/CoE.git.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Navve Wasserman | 66 | 87 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval 87 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval 20 Paint by Inpaint: Learning to Add Image Objects by Removing Them First (2025) Generative Models for Recommendation and Content › Others |
|
| 2 | Liwei Wang 0009 | 63 | 68 Improving One-Stage Visual Grounding by Recursive Sub-query Construction (2020) Universal Multimodal Embedding and MLLM Retrieval 68 A Fast and Accurate One-Stage Approach to Visual Grounding (2019) Universal Multimodal Embedding and MLLM Retrieval 68 Learning Deep Structure-Preserving Image-Text Embeddings (2016) Universal Multimodal Embedding and MLLM Retrieval |
|
| 3 | Junjie Zhou 0001 | 51 | 68 Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 68 VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval 68 MegaPairs: Massive Data Synthesis for Universal Multimodal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
Area chair
| Area chair | Match | |
|---|---|---|
| Hongjin Qian | 42 |
Area-chair candidates (5)
| Xiaoye Qu | 45 | 15 papers | |
| Hongjin Qian assigned | 42 | 12 papers | |
| Zhijing Wu 0001 | 41 | 9 papers | |
| Hao Fei 0001 | 40 | 4 papers | |
| Xiaoxi Li 0005 | 40 | 15 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Haokun Wen | 70 | 3 | 68 FiRE: Enhancing MLLMs with Fine-Grained Context Learning for Complex Image Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 68 Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 2 | Navve Wasserman assigned | 66 | 3 | 87 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval 87 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 3 | Liwei Wang 0009 assigned | 63 | 3 | 68 Improving One-Stage Visual Grounding by Recursive Sub-query Construction (2020) Universal Multimodal Embedding and MLLM Retrieval 68 A Fast and Accurate One-Stage Approach to Visual Grounding (2019) Universal Multimodal Embedding and MLLM Retrieval |
|
| 4 | Jian Lang | 58 | 3 | 68 Generative Thinking, Corrective Action: User-Friendly Composed Image Retrieval via Automatic Multi-Agent Collaboration (2025) Universal Multimodal Embedding and MLLM Retrieval 68 Echoes in the Feed: Evolution-aware Prompt-augmented Micro-video Popularity Prediction (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 5 | Xinying Qian | 57 | 3 | 68 Hyperbolic Multimodal Generative Representation Learning for Generalized Zero-Shot Multimodal Information Extraction (2026) Universal Multimodal Embedding and MLLM Retrieval 68 SMIR: Span-based multi-grained information refinement for joint multimodal entity-relation extraction (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 6 | Minghui Fang 0002 | 56 | 3 | 68 Multimodal Conditional Retrieval with High Controllability (2025) Universal Multimodal Embedding and MLLM Retrieval 68 CART: A Generative Cross-Modal Retrieval Framework With Coarse-To-Fine Semantic Modeling (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 7 | Xuhui Sui | 54 | 3 | 68 SMIR: Span-based multi-grained information refinement for joint multimodal entity-relation extraction (2026) Universal Multimodal Embedding and MLLM Retrieval 68 Coarse-to-fine multimodal prototype network for few-shot multimodal relation extraction (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 8 | Xingyu Tan 0001 | 52 | 3 | 59 HydraRAG: Structured Cross-Source Enhanced Large Language Model Reasoning (2025) KG-Augmented LLM Reasoning 59 Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning (2025) KG-Augmented LLM Reasoning |
|
| 9 | Yanzhao Zhang | 52 | 3 | 68 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval 68 Towards Text-Image Interleaved Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 10 | Junjie Zhou 0001 assigned | 51 | 3 | 68 Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 68 VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 11 | Jinpeng Wang 0002 | 51 | 3 | 68 Modeling Uncertainty in Composed Image Retrieval via Probabilistic Embeddings (2025) Universal Multimodal Embedding and MLLM Retrieval 68 Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learning (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 12 | Di Wu 0054 | 47 | 3 | 87 VisRet: Visualization Improves Knowledge-Intensive Text-to-Image Retrieval (2026) Multimodal and Graph RAG 42 LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (2025) Conversational Search and Retrieval |
|
| 13 | Linchao Zhu | 47 | 3 | 68 Attention as Selector: Unlocking VLM Attention for Long Document Page Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval 68 Knowledge-Enhanced Dual-Stream Zero-Shot Composed Image Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 14 | Jinheon Baek | 46 | 3 | 87 UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities (2026) Multimodal and Graph RAG 50 Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity (2024) Multi-Hop and Iterative Retrieval Reasoning |
|
| 15 | Meiyu Liang | 46 | 3 | 68 Knowledge Graph Enhanced Multimodal Transformer for Image-Text Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval 68 Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 17 | Jianfeng Dong | 45 | 3 | 66 Feature Re-Learning with Data Augmentation for Content-based Video Recommendation (2018) Universal Multimodal Embedding and MLLM Retrieval 65 Exploring Content-based Video Relevance for Video Click-Through Rate Prediction (2019) Universal Multimodal Embedding and MLLM Retrieval |
|
| 18 | Sheng-Chieh Lin | 44 | 3 | 68 Unifying Multimodal Retrieval via Document Screenshot Embedding (2024) Universal Multimodal Embedding and MLLM Retrieval 67 Mm-Embed: Universal Multimodal Retrieval with Multimodal LLMS (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 19 | Shu Zhou 0002 | 44 | 3 | 86 When MLLMs Meet ICH: A visual retrieval-augmented generation-based method for intangible cultural heritage image recognition-take Shadow Puppetry as a case (2026) Universal Multimodal Embedding and MLLM Retrieval 60 DynKGRAG: A framework for dynamic knowledge graph construction and multi-hop querying over numerical and textual data (2026) Multi-Hop and Iterative Retrieval Reasoning |
|
| 20 | Tingyu Song | 44 | 3 | 68 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval 42 LimRank: Less is More for Reasoning-Intensive Information Reranking (2025) Evidence Selection and Context Refinement |
|
| 21 | Jiuqiang Li 0002 | 42 | 3 | 66 Multi-view Semantic Contrastive Alignment for Multimodal Recommendation (2026) Universal Multimodal Embedding and MLLM Retrieval 25 Disentangled Contrastive Bundle Recommendation with Conditional Diffusion (2025) Diffusion and Flow Recommenders |
|
| 23 | Ruqing Zhang 0001 | 42 | 3 | 42 Thinking Forward and Backward: Multi-Objective Reinforcement Learning for Retrieval-Augmented Reasoning (2026) Multi-Hop and Iterative Retrieval Reasoning 42 Boosting Retrieval-Augmented Generation with Generation-Augmented Retrieval: A Co-Training Approach (2025) Generative Retrieval |
|
| 24 | Haiyang Xu 0001 COI: co-author | 42 | 3 | 43 Efficient and Effective In-context Demonstration Selection with Coreset (2026) Multimodal Retrieval and Embedding 43 Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation (2023) Multimodal Retrieval and Embedding |
|
| 25 | Qinggang Zhang | 42 | 3 | 68 You Don't Need Pre-Built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures (2026) Multi-Hop and Iterative Retrieval Reasoning 50 KnowGPT: Knowledge Graph based Prompting for Large Language Models (2024) Multimodal and Graph RAG |
|
| 26 | Zihao Yue | 41 | 3 | 41 Movie101v2: Improved Movie Narration Benchmark (2025) Video and Temporal Understanding 41 OpenMMEgo: Enhancing Egocentric Understanding for LMMs with Open Weights and Data (2025) Video and Temporal Understanding |
|
| 27 | Xinyu Ma 0001 | 41 | 3 | 43 Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense Retrieval (2023) Sparse and Dense Retrieval Encoders 42 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking |
|
| 29 | Weipeng Chen | 41 | 3 | 45 ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning (2025) RL Training of Search Agents 42 RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation (2025) Evidence Selection and Context Refinement |
|
| 30 | Huiqiang Jiang | 41 | 3 | 68 Attentive Mask CLIP (2023) Universal Multimodal Embedding and MLLM Retrieval 40 SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents (2025) Conversational Search and Retrieval |
|
| 31 | Venktesh V | 40 | 3 | 52 SUNAR: Semantic Uncertainty based Neighborhood Aware Retrieval for Complex QA (2025) Multi-Hop and Iterative Retrieval Reasoning 41 The Surprising Effectiveness of Rankers trained on Expanded Queries (2024) Evidence Selection and Context Refinement |
|
| 32 | Mariano Barone | 40 | 3 | 45 A Multilingual Multimodal Medical Examination Dataset for Visual Question Answering in Healthcare (2025) Retrieval-Augmented and Open-Domain QA 39 AMICO: A Semantic and Multimodal Framework for AI-Assisted Clinical Reporting (2025) Multimodal Evaluation and Benchmarks |
|
| 34 | Qingyi Si | 40 | 3 | 73 Multimodal Hypothetical Summary for Retrieval-based Multi-image Question Answering (2025) Universal Multimodal Embedding and MLLM Retrieval 39 Combo of Thinking and Observing for Outside-Knowledge VQA (2023) Retrieval-Augmented and Open-Domain QA |