RegionSLM: Region-aware Question Answering on Document Screenshots
Abstract
Real-world document question-answering that relies on screenshots, such as bills and forms, requires evidence that is often spatially localised and visually cluttered. However, most Screenshot Language Models (SLMs) encode the entire page holistically and rely on implicit attention to ''find'' relevant content, which limits both accuracy and efficiency. We present RegionSLM, a region-aware SLM designed to explicitly connect the question to its supporting regions. RegionSLM has two key components: (1) a patch-relevance router that learns a query–region relevance distribution, enabling the model to produce a box-free relevance prior at inference; and (2) Relevance-Guided Region Pooling (RGRP), a query-conditional attention–pooling module that aggregates dense features into a small set of region tokens, which preserves grounding signals while reducing computational overhead. To support training and evaluation, we further curate ReDoc, a region-supervised corpus with 105k documents and 350k question-answer pairs, obtained via a question-guided two-step filtering procedure. Extensive experiments on 12 datasets demonstrate that explicitly learning query–region relevance and pooling it into compact region tokens is an effective and practical recipe for document retrieval and understanding.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Liwei Wang 0009 | 64 | 68 Improving One-Stage Visual Grounding by Recursive Sub-query Construction (2020) Universal Multimodal Embedding and MLLM Retrieval 68 A Fast and Accurate One-Stage Approach to Visual Grounding (2019) Universal Multimodal Embedding and MLLM Retrieval 68 Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models (2017) Universal Multimodal Embedding and MLLM Retrieval |
|
| 2 | Junjie Zhou 0001 | 47 | 68 VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval 68 MegaPairs: Massive Data Synthesis for Universal Multimodal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 68 Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Haokun Wen | 67 | 3 | 68 FiRE: Enhancing MLLMs with Fine-Grained Context Learning for Complex Image Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 68 A Comprehensive Survey on Composed Image Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 2 | Liwei Wang 0009 assigned | 64 | 3 | 68 Improving One-Stage Visual Grounding by Recursive Sub-query Construction (2020) Universal Multimodal Embedding and MLLM Retrieval 68 A Fast and Accurate One-Stage Approach to Visual Grounding (2019) Universal Multimodal Embedding and MLLM Retrieval |
|
| 3 | Navve Wasserman | 56 | 3 | 73 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval 73 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 4 | Minghui Fang 0002 | 55 | 3 | 69 Vela: Scalable Embeddings with Voice Large Language Models for Multimodal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 68 Multimodal Conditional Retrieval with High Controllability (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 5 | Jian Lang | 55 | 3 | 69 REDEEMing Modality Information Loss: Retrieval-Guided Conditional Generation for Severely Modality Missing Learning (2025) Universal Multimodal Embedding and MLLM Retrieval 68 Generative Thinking, Corrective Action: User-Friendly Composed Image Retrieval via Automatic Multi-Agent Collaboration (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 6 | Xinying Qian | 53 | 3 | 68 Hyperbolic Multimodal Generative Representation Learning for Generalized Zero-Shot Multimodal Information Extraction (2026) Universal Multimodal Embedding and MLLM Retrieval 68 SMIR: Span-based multi-grained information refinement for joint multimodal entity-relation extraction (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 7 | Idan Szpektor | 50 | 3 | 69 On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method (2023) Conversational Search and Retrieval 68 Will My Question Be Answered? Predicting "Question Answerability" in Community Question-Answering Sites (2013) Retrieval-Augmented and Open-Domain QA |
|
| 8 | Yanzhao Zhang | 50 | 3 | 69 Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs (2025) Universal Multimodal Embedding and MLLM Retrieval 69 Bridging Modalities: Improving Universal Multimodal Retrieval by Multimodal Large Language Models (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 9 | Xuhui Sui | 49 | 3 | 68 Coarse-to-fine multimodal prototype network for few-shot multimodal relation extraction (2025) Universal Multimodal Embedding and MLLM Retrieval 68 SMIR: Span-based multi-grained information refinement for joint multimodal entity-relation extraction (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 10 | Junjie Zhou 0001 assigned | 47 | 3 | 68 VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval 68 MegaPairs: Massive Data Synthesis for Universal Multimodal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 11 | Jinpeng Wang 0002 | 47 | 3 | 69 Imagine with Layout and Sketch: Enhancing Vision-Language Retrieval with Dual-Stream Multi-Modal Query Refinement (2026) Universal Multimodal Embedding and MLLM Retrieval 69 Suit the Remedy to the Retriever: Interpretable Query Optimization with Retriever Preference Alignment for Vision-Language Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 12 | Mariano Barone | 46 | 3 | 72 A Multilingual Multimodal Medical Examination Dataset for Visual Question Answering in Healthcare (2025) Retrieval-Augmented and Open-Domain QA 55 SOLVE-Med: Specialized Orchestration for Leading Vertical Experts Across Medical Specialties (2025) Retrieval-Augmented and Open-Domain QA |
|
| 13 | Jing Liu 0022 | 44 | 3 | 68 A Regularized Competition Model for Question Difficulty Estimation in Community Question Answering Services (2014) Retrieval-Augmented and Open-Domain QA 62 Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation (2025) Knowledge Conflict and Faithfulness |
|
| 14 | Chen Zhao 0013 | 44 | 3 | 68 A dataset and baselines for sequential open-domain question answering (2018) Retrieval-Augmented and Open-Domain QA 58 Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation (2021) Dense Bi-Encoder Training and Representation Learning |
|
| 15 | Linchao Zhu COI: co-author | 43 | 3 | 68 Attention as Selector: Unlocking VLM Attention for Long Document Page Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval 68 Knowledge-Enhanced Dual-Stream Zero-Shot Composed Image Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 16 | Meiyu Liang | 43 | 3 | 68 Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 68 Knowledge Graph Enhanced Multimodal Transformer for Image-Text Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 17 | Farzad Shami | 43 | 3 | 68 From Single to Multi-agent Reasoning: Advancing GeneGPT for Genomics QA (2026) Retrieval-Augmented and Open-Domain QA 36 Doctron: A Web-based Collaborative Annotation Tool for Ground Truth Creation in IR (2025) Retrieval Toolkits and Reproducibility |
|
| 19 | Qingyi Si | 42 | 3 | 100 Multimodal Hypothetical Summary for Retrieval-based Multi-image Question Answering (2025) Universal Multimodal Embedding and MLLM Retrieval 66 Combo of Thinking and Observing for Outside-Knowledge VQA (2023) Retrieval-Augmented and Open-Domain QA |
|
| 20 | Jinheon Baek | 42 | 3 | 74 UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities (2026) Multimodal and Graph RAG 69 Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (2023) Conversational Search and Retrieval |
|
| 21 | Di Wu 0054 | 42 | 3 | 74 VisRet: Visualization Improves Knowledge-Intensive Text-to-Image Retrieval (2026) Multimodal and Graph RAG 40 LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (2025) Conversational Search and Retrieval |
|
| 22 | Sheng-Chieh Lin | 42 | 3 | 69 Mm-Embed: Universal Multimodal Retrieval with Multimodal LLMS (2025) Universal Multimodal Embedding and MLLM Retrieval 68 Unifying Multimodal Retrieval via Document Screenshot Embedding (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 23 | Archiki Prasad | 40 | 3 | 68 MeetingQA: Extractive Question-Answering on Meeting Transcripts (2023) Retrieval-Augmented and Open-Domain QA 53 PRInTS: Reward Modeling for Long-Horizon Information Seeking (2026) Reward Modeling and Process Supervision |
|
| 24 | Jianfeng Dong | 40 | 3 | 64 Feature Re-Learning with Data Augmentation for Content-based Video Recommendation (2018) Universal Multimodal Embedding and MLLM Retrieval 63 Exploring Content-based Video Relevance for Video Click-Through Rate Prediction (2019) Universal Multimodal Embedding and MLLM Retrieval |
|
| 25 | Haiyang Xu 0001 | 40 | 3 | 45 Efficient and Effective In-context Demonstration Selection with Coreset (2026) Multimodal Retrieval and Embedding 45 Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation (2023) Multimodal Retrieval and Embedding |
|
| 26 | Shu Zhou 0002 | 40 | 3 | 74 When MLLMs Meet ICH: A visual retrieval-augmented generation-based method for intangible cultural heritage image recognition-take Shadow Puppetry as a case (2026) Universal Multimodal Embedding and MLLM Retrieval 69 LOSDF: A logical optimization and semantic decoupling framework for question answering in multi-party conversations (2025) Conversational Search and Retrieval |
|
| 27 | Tingyu Song | 40 | 3 | 68 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval 40 LimRank: Less is More for Reasoning-Intensive Information Reranking (2025) Evidence Selection and Context Refinement |
|
| 28 | Xi Ye 0003 | 39 | 3 | 58 Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking (2025) Multi-hop and Complex Reasoning QA 40 Connecting Attributions and QA Model Behavior on Realistic Counterfactuals (2021) Multi-hop and Complex Reasoning QA |
|
| 29 | Kaixin Ma | 39 | 3 | 72 Open Domain Question Answering with A Unified Knowledge Interface (2022) Multimodal and Graph RAG 71 Chain-of-Skills: A Configurable Model for Open-Domain Question Answering (2023) Retrieval-Augmented and Open-Domain QA |
|
| 30 | Xinyu Ma 0001 | 39 | 3 | 40 Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense Retrieval (2023) Sparse and Dense Retrieval Encoders 40 B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 31 | Adam Jatowt | 39 | 3 | 63 ArchivalQA: A Large-scale Benchmark Dataset for Open-Domain Question Answering over Historical News Collections (2022) Multi-hop and Complex Reasoning QA 40 BracketRank: Large Language Model Document Ranking via Reasoning-based Competitive Elimination (2026) Reasoning-Intensive Retrieval and Ranking |