Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Agentic Spatio-Temporal Grounding via Collaborative Reasoning

sigir26-004 · Full research paper · Heng Zhao, Yew-Soon Ong, Joey Tianyi Zhou
Video and Moment Retrieval · 50% Video and Temporal Understanding · 25% Multimodal Reasoning and Grounding · 25%
Abstract

Spatio-Temporal Video Grounding (STVG) aims to retrieve the spatio-temporal tube of a target object or person in a video given a text query. Most existing approaches perform frame-wise spatial localization within a predicted temporal span, resulting in redundant computation, heavy supervision requirements, and limited generalization. Weakly-supervised variants mitigate annotation costs but remain constrained by the dataset-level train-and-fit paradigm with an inferior performance. To address these challenges, we propose the Agentic Spatio-Temporal Grounder (ASTG) framework for the task of STVG in an open-world and zero-shot setting. Specifically, two specialized agents SRA (Spatial Reasoning Agent) and TRA (Temporal Reasoning Agent) constructed leveraging modern Multi-modal Large Language Models (MLLMs) work collaboratively to retrieve the target tube in an autonomous and self-guided manner. Following a propose-and-evaluation paradigm, ASTG duly decouples spatio-temporal reasoning and automates the tube extraction, verification and temporal localization processes. With a dedicated visual memory and dialogue context, ASTG achieves architectural efficiency by minimizing the number of reasoning calls compared to exhaustive per-frame reasoning, eliminating the logical redundancy inherent in joint-reasoning systems. Experiments on popular benchmarks demonstrate the superiority of the proposed approach where it outperforms existing weakly-supervised and zero-shot approaches by a margin and is comparable to some of the fully-supervised methods.

Assigned reviewers

#ReviewerMatchWhy (closest evidence publications in the taxonomy)
1 Linchao Zhu
80
69 ActBERT: Learning Global-Local Video-Text Representations (2020) Video and Moment Retrieval
69 Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding (2023) Video and Moment Retrieval
69 Stitching Segments and Sentences towards Generalization in Video-Text Pre-training (2024) Video and Moment Retrieval
2 Haiyang Xu 0001
63
69 Learning Semantics-Grounded Vocabulary Representation for Video-Text Retrieval (2023) Video and Moment Retrieval
69 Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval (2024) Video and Moment Retrieval
59 mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models (2025) Video and Temporal Understanding

Area chair

Area chairMatch
Xiaoye Qu 57
Area-chair candidates (8)
Xiaoye Qu assigned 5715 papers
Xun Yang 0001 514 papers
Hao Fei 0001 364 papers
Hongjin Qian 3512 papers
Kam-Fai Wong 346 papers
Chenyan Xiong 3315 papers
Xiaoxi Li 0005 3315 papers
Yi R. Fung 0001 3210 papers

Candidates from the panel ranked by match score

#ReviewerMatchLoadWhy
1 Linchao Zhu assigned 80 3
69 ActBERT: Learning Global-Local Video-Text Representations (2020) Video and Moment Retrieval
69 Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding (2023) Video and Moment Retrieval
2 Jianfeng Dong 63 3
69 IVCR-200K: A Large-Scale Multi-turn Dialogue Benchmark for Interactive Video Corpus Retrieval (2025) Conversational Search and Retrieval
69 Temporal Sentence Grounding with Relevance Feedback in Videos (2024) Video and Moment Retrieval
3 Haiyang Xu 0001 assigned 63 3
69 Learning Semantics-Grounded Vocabulary Representation for Video-Text Retrieval (2023) Video and Moment Retrieval
69 Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval (2024) Video and Moment Retrieval
4 Zihao Yue 58 3
69 Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding (2025) Multimodal Reasoning and Grounding
69 VideoOrion: Tokenizing Object Dynamics in Videos (2025) Multimodal Reasoning and Grounding
7 Huaiwen Zhang 46 3
69 Sequence-Event Semantic Consistent Learning for Text-to-Motion Retrieval (2025) Video and Moment Retrieval
69 Robust Video-Text Retrieval Via Noisy Pair Calibration (2023) Video and Moment Retrieval
8 Zhe Xu 0009 45 3
69 Point-Supervised Video Temporal Grounding (2023) Video and Moment Retrieval
69 HiSA: Hierarchically Semantic Associating for Video Temporal Grounding (2022) Video and Moment Retrieval
9 Junjie Zhou 0001 45 3
69 MomentSeeker: A Task-Oriented Benchmark For Long-Video Moment Retrieval (2025) Video and Moment Retrieval
59 Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding (2025) Video and Temporal Understanding
10 Qingyi Si 43 3
52 Combo of Thinking and Observing for Outside-Knowledge VQA (2023) Retrieval-Augmented and Open-Domain QA
51 Object Attribute Matters in Visual Question Answering (2024) Multimodal Reasoning and Grounding
11 Liwei Wang 0009 41 3
59 AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning (2025) Video and Temporal Understanding
42 RAST: Domain-Robust Dialogue Rewriting as Sequence Tagging (2021) Conversational Search and Retrieval
12 Minghui Fang 0002 40 3
44 Vela: Scalable Embeddings with Voice Large Language Models for Multimodal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval
38 VoxpopuliTTS: a large-scale multilingual TTS corpus for zero-shot speech generation (2025) Multimodal Retrieval and Embedding
13 Haokun Wen 38 3
41 Differential-Perceptive and Retrieval-Augmented MLLM for Change Captioning (2024) Multimodal and Graph RAG
38 HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval
14 Tingyu Song 38 3
55 VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos (2025) Video and Temporal Understanding
43 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval
15 Jinpeng Wang 0002 37 3
69 Enhancing Partially Relevant Video Retrieval with Hyperbolic Learning (2025) Video and Moment Retrieval
44 Imagine with Layout and Sketch: Enhancing Vision-Language Retrieval with Dual-Stream Multi-Modal Query Refinement (2026) Universal Multimodal Embedding and MLLM Retrieval
17 Navve Wasserman 35 3
42 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval
42 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval
18 Jian Lang 35 3
44 REDEEMing Modality Information Loss: Retrieval-Guided Conditional Generation for Severely Modality Missing Learning (2025) Universal Multimodal Embedding and MLLM Retrieval
38 Generative Thinking, Corrective Action: User-Friendly Composed Image Retrieval via Automatic Multi-Agent Collaboration (2025) Universal Multimodal Embedding and MLLM Retrieval
19 Sheng-Chieh Lin 35 3
44 Mm-Embed: Universal Multimodal Retrieval with Multimodal LLMS (2025) Universal Multimodal Embedding and MLLM Retrieval
43 Unifying Multimodal Retrieval via Document Screenshot Embedding (2024) Universal Multimodal Embedding and MLLM Retrieval
20 Yanzhao Zhang 35 3
44 Bridging Modalities: Improving Universal Multimodal Retrieval by Multimodal Large Language Models (2025) Universal Multimodal Embedding and MLLM Retrieval
44 Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs (2025) Universal Multimodal Embedding and MLLM Retrieval
21 Xinying Qian 35 3
38 Hyperbolic Multimodal Generative Representation Learning for Generalized Zero-Shot Multimodal Information Extraction (2026) Universal Multimodal Embedding and MLLM Retrieval
38 SMIR: Span-based multi-grained information refinement for joint multimodal entity-relation extraction (2026) Universal Multimodal Embedding and MLLM Retrieval
24 Archiki Prasad 34 3
51 Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models (2024) Multimodal Reasoning and Grounding
48 GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs (2026) Language Model Pre-training and Adaptation
25 Peitian Zhang 34 3
59 Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding (2025) Video and Temporal Understanding
34 INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning (2024) LLM-Based Reranking
26 Xuhui Sui 33 3
38 Coarse-to-fine multimodal prototype network for few-shot multimodal relation extraction (2025) Universal Multimodal Embedding and MLLM Retrieval
38 SMIR: Span-based multi-grained information refinement for joint multimodal entity-relation extraction (2026) Universal Multimodal Embedding and MLLM Retrieval
27 Huiqiang Jiang 33 3
42 SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents (2025) Conversational Search and Retrieval
38 Attentive Mask CLIP (2023) Universal Multimodal Embedding and MLLM Retrieval
30 Xingyu Tan 0001 32 3
37 PRoH: Dynamic Planning and Reasoning over Knowledge Hypergraphs for Retrieval-Augmented Generation (2026) Multimodal and Graph RAG
34 MemoTime: Memory-Augmented Temporal Knowledge Graph Enhanced Large Language Model Reasoning (2026) KG-Augmented LLM Reasoning
32 Weipeng Chen 32 3
55 Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs (2025) Video and Temporal Understanding
36 AITQE: An Adaptive Image-Text Quality Enhancer for Scalable MLLM Pretraining (2026) Multimodal Evaluation and Benchmarks
33 Weiwei Sun 0001 32 3
42 DukeNet: A Dual Knowledge Interaction Network for Knowledge-Grounded Conversation (2020) Conversational Search and Retrieval
32 ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026) LLM-Based Reranking
34 Jinheon Baek 32 3
44 UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities (2026) Multimodal and Graph RAG
39 Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (2023) Conversational Search and Retrieval
35 Qinggang Zhang 32 3
38 Facilitating Generative Retrieval with Logical Denoising for Interpretable Conversational Search (2026) Personalized and Conversational RAG
37 You Don't Need Pre-Built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures (2026) Multi-Hop and Iterative Retrieval Reasoning
36 Shengyao Zhuang 32 3
40 Embark on DenseQuest: A System for Selecting the Best Dense Retriever for a Custom Collection (2024) Zero-Shot and Domain Generalization for Retrieval
34 A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models (2024) LLM-Based Reranking
37 Xinyu Ma 0001 32 3
33 Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense Retrieval (2023) Sparse and Dense Retrieval Encoders
32 A Contrastive Pre-training Approach to Discriminative Autoencoder for Dense Retrieval (2022) Sparse and Dense Retrieval Encoders
38 Yujia Zhou 0002 32 3
34 Cognitive Personalized Search Integrating Large Language Models with an Efficient Memory Mechanism (2024) Conversational Search and Retrieval
34 Grounding Language Model with Chunking-Free In-Context Retrieval (2024) Generative Retrieval