ProEchoMem: Enhancing Long Video Understanding via Multi-Trace Probe-Echo Memory
sigir26-075 · Full research paper · Derong Xu, Y S Chen, Wanyu Wang, Pengyue Jia, Chao Zhang, Maolin Wang, Yiqi Wang, Jipeng Qiang, Xuetao Wei, Hongzhi Yin, Tong Xu, Xiangyu Zhao
Multimodal and Graph RAG · 50% Multimodal Retrieval and Embedding · 25% Video and Temporal Understanding · 25%
Abstract
Large vision-language models (LVLMs) have shown significant progress in video understanding, but they struggle to scale to long videos due to limited context windows. Existing methods reduce input dimensionality via frame sampling and feature compression, yet discard details and incur high computational cost for post-training. In contrast, retrieval-augmented generation (RAG) that indexes long videos for query retrieval and memory-based methods that maintain evolving long-term stores, offer a lighter and deployment-friendly solution. Nevertheless, they rely on shallow retrieval that selects only top-ranked segments and fails to integrate information across multiple relevant video episodes. Inspired by Multiple-Trace Theory in cognitive psychology, we revisit long video understanding from a probe-echo perspective, in which human episodic memories are activated and integrated in parallel. Building on this insight, we propose ProEchoMem, a cognitive-inspired framework that simulates the probe-echo mechanism: (1) Incremental Episodic Memory Construction builds structured knowledge graphs from video streams; (2) Probe-Driven Memory Activation generates probe signals from user queries to activate all stored traces simultaneously; (3) Memory Echo Synthesis integrates activated traces into a coherent and structured memory echo. Experiments on LongerVideos, LVBench, and cross-domain settings demonstrate the effectiveness of ProEchoMem, with multi-trace probing achieving up to 14.2% higher relevance and ablation studies validating the contribution of each module. The code is available at https://github.com/Applied-Machine-Learning-Lab/SIGIR26_ProEchoMem
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Minghui Fang 0002 | 56 | 58 SyncTalklip: Highly Synchronized Lip-Readable Speaker Generation with Multi-Task Learning (2024) Multimodal Retrieval and Embedding 58 Mitigating Hallucinations in LM-Based TTS Models via Distribution Alignment Using GFlowNets (2025) Multimodal Retrieval and Embedding 58 ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control (2025) Multimodal Retrieval and Embedding |
|
| 2 | Tingyu Song | 45 | 58 VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos (2025) Video and Temporal Understanding 56 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval 40 LimRank: Less is More for Reasoning-Intensive Information Reranking (2025) Evidence Selection and Context Refinement |
Area chair
| Area chair | Match | |
|---|---|---|
| Xiaoye Qu | 51 |
Area-chair candidates (4)
| Xiaoye Qu assigned | 51 | 15 papers | |
| Hongjin Qian | 37 | 12 papers | |
| Xiaoxi Li 0005 | 37 | 15 papers | |
| Chenyan Xiong COI: co-author | 36 | 15 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Minghui Fang 0002 assigned | 56 | 3 | 58 SyncTalklip: Highly Synchronized Lip-Readable Speaker Generation with Multi-Task Learning (2024) Multimodal Retrieval and Embedding 58 Mitigating Hallucinations in LM-Based TTS Models via Distribution Alignment Using GFlowNets (2025) Multimodal Retrieval and Embedding |
|
| 2 | Linchao Zhu | 55 | 3 | 56 Attention as Selector: Unlocking VLM Attention for Long Document Page Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval 55 From Trial to Triumph: Advancing Long Video Understanding via Visual Context Sample Scaling and Self-Reward Alignment (2025) Video and Temporal Understanding |
|
| 3 | Zihao Yue | 54 | 3 | 58 Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding (2025) Multimodal Reasoning and Grounding 58 VideoOrion: Tokenizing Object Dynamics in Videos (2025) Multimodal Reasoning and Grounding |
|
| 4 | Haiyang Xu 0001 | 54 | 3 | 58 Efficient and Effective In-context Demonstration Selection with Coreset (2026) Multimodal Retrieval and Embedding 58 Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation (2023) Multimodal Retrieval and Embedding |
|
| 6 | Navve Wasserman | 48 | 3 | 57 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval 57 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 7 | Xingyu Tan 0001 | 47 | 3 | 62 PRoH: Dynamic Planning and Reasoning over Knowledge Hypergraphs for Retrieval-Augmented Generation (2026) Multimodal and Graph RAG 49 MemoTime: Memory-Augmented Temporal Knowledge Graph Enhanced Large Language Model Reasoning (2026) KG-Augmented LLM Reasoning |
|
| 8 | Tingyu Song assigned | 45 | 3 | 58 VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos (2025) Video and Temporal Understanding 56 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 9 | Haokun Wen COI: co-author | 44 | 3 | 84 Differential-Perceptive and Retrieval-Augmented MLLM for Change Captioning (2024) Multimodal and Graph RAG 42 FiRE: Enhancing MLLMs with Fine-Grained Context Learning for Complex Image Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 10 | Liwei Wang 0009 | 44 | 3 | 55 AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning (2025) Video and Temporal Understanding 42 Improving One-Stage Visual Grounding by Recursive Sub-query Construction (2020) Universal Multimodal Embedding and MLLM Retrieval |
|
| 11 | Junjie Zhou 0001 | 43 | 3 | 58 MLVU: Benchmarking Multi-task Long Video Understanding (2025) Video and Temporal Understanding 55 Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding (2025) Video and Temporal Understanding |
|
| 12 | Xinying Qian | 42 | 3 | 50 TimeR⁴ : Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering (2024) Multi-Hop and Iterative Retrieval Reasoning 42 Hyperbolic Multimodal Generative Representation Learning for Generalized Zero-Shot Multimodal Information Extraction (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 13 | Archiki Prasad | 42 | 3 | 58 How Accents Confound: Probing for Accent Information in End-to-End Speech Recognition Systems (2020) Multimodal Retrieval and Embedding 58 An Investigation of End-to-End Models for Robust Speech Recognition (2021) Multimodal Retrieval and Embedding |
|
| 14 | Sheng-Chieh Lin | 41 | 3 | 57 Mm-Embed: Universal Multimodal Retrieval with Multimodal LLMS (2025) Universal Multimodal Embedding and MLLM Retrieval 56 Unifying Multimodal Retrieval via Document Screenshot Embedding (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 15 | Yanzhao Zhang | 41 | 3 | 57 Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs (2025) Universal Multimodal Embedding and MLLM Retrieval 57 Bridging Modalities: Improving Universal Multimodal Retrieval by Multimodal Large Language Models (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 16 | Qinggang Zhang | 41 | 3 | 63 You Don't Need Pre-Built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures (2026) Multi-Hop and Iterative Retrieval Reasoning 62 KnowGPT: Knowledge Graph based Prompting for Large Language Models (2024) Multimodal and Graph RAG |
|
| 17 | Di Wu 0054 | 41 | 3 | 70 VisRet: Visualization Improves Knowledge-Intensive Text-to-Image Retrieval (2026) Multimodal and Graph RAG 42 LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (2025) Conversational Search and Retrieval |
|
| 18 | Jinheon Baek | 40 | 3 | 70 UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities (2026) Multimodal and Graph RAG 44 Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (2023) Conversational Search and Retrieval |
|
| 19 | Mariano Barone | 40 | 3 | 45 A Multilingual Multimodal Medical Examination Dataset for Visual Question Answering in Healthcare (2025) Retrieval-Augmented and Open-Domain QA 41 AMICO: A Semantic and Multimodal Framework for AI-Assisted Clinical Reporting (2025) Multimodal Evaluation and Benchmarks |
|
| 20 | Jinpeng Wang 0002 | 40 | 3 | 57 Imagine with Layout and Sketch: Enhancing Vision-Language Retrieval with Dual-Stream Multi-Modal Query Refinement (2026) Universal Multimodal Embedding and MLLM Retrieval 57 Suit the Remedy to the Retriever: Interpretable Query Optimization with Retriever Preference Alignment for Vision-Language Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 21 | Jian Lang | 39 | 3 | 57 REDEEMing Modality Information Loss: Retrieval-Guided Conditional Generation for Severely Modality Missing Learning (2025) Universal Multimodal Embedding and MLLM Retrieval 42 Echoes in the Feed: Evolution-aware Prompt-augmented Micro-video Popularity Prediction (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 22 | Xuhui Sui | 39 | 3 | 43 TimeR⁴ : Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering (2024) RL Training of Search Agents 42 Coarse-to-fine multimodal prototype network for few-shot multimodal relation extraction (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 23 | Qingyi Si | 39 | 3 | 47 Combo of Thinking and Observing for Outside-Knowledge VQA (2023) Retrieval-Augmented and Open-Domain QA 45 Object Attribute Matters in Visual Question Answering (2024) Multimodal Reasoning and Grounding |
|
| 24 | Xubin Ren | 39 | 3 | 70 VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos (2026) Multimodal and Graph RAG 58 MiniRAG: A Lightweight RAG system with Small Language Models (2026) Multimodal and Graph RAG |
|
| 25 | Peitian Zhang | 38 | 3 | 55 Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding (2025) Video and Temporal Understanding 41 A Multi-Task Embedder For Retrieval Augmented LLMs (2024) Evidence Selection and Context Refinement |
|
| 26 | Jianfeng Dong | 38 | 3 | 42 IVCR-200K: A Large-Scale Multi-turn Dialogue Benchmark for Interactive Video Corpus Retrieval (2025) Conversational Search and Retrieval 41 Rethinking Video Sentence Grounding From a Tracking Perspective With Memory Network and Masked Attention (2024) Video and Moment Retrieval |
|
| 27 | Shu Zhou 0002 | 38 | 3 | 56 When MLLMs Meet ICH: A visual retrieval-augmented generation-based method for intangible cultural heritage image recognition-take Shadow Puppetry as a case (2026) Universal Multimodal Embedding and MLLM Retrieval 50 DynKGRAG: A framework for dynamic knowledge graph construction and multi-hop querying over numerical and textual data (2026) Multi-Hop and Iterative Retrieval Reasoning |
|
| 28 | Ruqing Zhang 0001 | 38 | 3 | 40 Boosting Retrieval-Augmented Generation with Generation-Augmented Retrieval: A Co-Training Approach (2025) Generative Retrieval 40 CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Tasks (2026) Generative Retrieval |
|
| 29 | Weipeng Chen | 38 | 3 | 58 Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs (2025) Video and Temporal Understanding 41 AITQE: An Adaptive Image-Text Quality Enhancer for Scalable MLLM Pretraining (2026) Multimodal Evaluation and Benchmarks |
|
| 30 | Xinyu Ma 0001 COI: co-author | 38 | 3 | 42 Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation (2025) LLM-Derived Knowledge Infusion 39 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking |
|
| 31 | Weiwei Sun 0001 | 38 | 3 | 41 DukeNet: A Dual Knowledge Interaction Network for Knowledge-Grounded Conversation (2020) Conversational Search and Retrieval 39 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking |