Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Think with Grounding: Curriculum Reinforced Reasoning with Video Grounding for Long Video Understanding

sigir26-232 · Full research paper · Houlun Chen, Xin Wang, Guangyao Li, Yuwei Zhou, Yihan Chen, Jia Jia, Wenwu Zhu
Multimodal Reasoning and Grounding · 50% Video and Temporal Understanding · 50%
Abstract

Long video understanding (LVU) is challenging due to rich and complicated multimodal clues in long temporal range. Current methods adopt reasoning to improve the model's ability to analyze complex video clues in long videos via text-form reasoning. However, the existing literature suffers from the fact that the text-only reasoning under fixed video context may exacerbate hallucinations since detailed crucial clues are often ignored under limited video context length due to the temporal redundancy of long videos. To address this gap, we propose Video-TwG, a curriculum reinforced framework that employs a novel Think-with-Grounding paradigm, enabling video LLMs to actively decide when to perform on-demand grounding during interleaved text–video reasoning, selectively zooming into question-relevant clips only when necessary. Video-TwG can be trained end-to-end in a straightforward manner, without relying on complex auxiliary modules or heavily annotated reasoning traces. In detail, we design a Two-stage Reinforced Curriculum Strategy, where the model first learns think-with-grounding behavior on a small short-video GQA dataset with grounding labels, and then scales to diverse general QA data with videos of diverse domains to encourage generalization. Further, to handle complex think-with-grounding reasoning for various kinds of data, we propose the TwG-GRPO algorithm, which features the fine-grained grounding reward, self-confirmed pseudo reward, and accuracy-gated mechanism. Finally, we propose to construct a new TwG-51K dataset that facilitates training. Experiments on Video-MME, LongVideoBench, and MLVU show that Video-TwG consistently outperforms strong LVU baselines. Further ablation validates the necessity of our Two-stage Reinforced Curriculum Strategy and shows our TwG-GRPO better leverages diverse unlabeled data to improve grounding quality and reduce redundant groundings without sacrificing QA performance. https://github.com/hlchen23/Video-TwG

Assigned reviewers

#ReviewerMatchWhy (closest evidence publications in the taxonomy)
1 Zihao Yue
61
100 Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding (2025) Multimodal Reasoning and Grounding
100 VideoOrion: Tokenizing Object Dynamics in Videos (2025) Multimodal Reasoning and Grounding
73 Unified Multimodal Understanding via Byte-Pair Visual Encoding (2025) Multimodal Reasoning and Grounding
2 Qingyi Si
45
73 Object Attribute Matters in Visual Question Answering (2024) Multimodal Reasoning and Grounding
73 Check It Again: Progressive Visual Question Answering via Visual Entailment (2021) Multimodal Reasoning and Grounding
66 Cross-modality Multiple Relations Learning for Knowledge-based Visual Question Answering (2024) Knowledge Graph Question Answering

Area chair

Area chairMatch
Xiaoye Qu 43
Area-chair candidates (7)
Xiaoye Qu assigned 4315 papers
Hao Fei 0001 354 papers
Yi R. Fung 0001 3310 papers
Dacheng Tao 335 papers
Kam-Fai Wong 326 papers
Xiaoxi Li 0005 3215 papers
Hongjin Qian 3212 papers

Candidates from the panel ranked by match score

#ReviewerMatchLoadWhy
1 Zihao Yue assigned 61 3
100 Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding (2025) Multimodal Reasoning and Grounding
100 VideoOrion: Tokenizing Object Dynamics in Videos (2025) Multimodal Reasoning and Grounding
2 Haiyang Xu 0001 59 3
73 mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models (2025) Video and Temporal Understanding
73 SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization (2025) Multimodal Reasoning and Grounding
3 Linchao Zhu COI: co-author 53 3
73 From Trial to Triumph: Advancing Long Video Understanding via Visual Context Sample Scaling and Self-Reward Alignment (2025) Video and Temporal Understanding
73 MIST : Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering (2023) Video and Temporal Understanding
4 Qingyi Si assigned 45 3
73 Object Attribute Matters in Visual Question Answering (2024) Multimodal Reasoning and Grounding
73 Check It Again: Progressive Visual Question Answering via Visual Entailment (2021) Multimodal Reasoning and Grounding
6 Minghui Fang 0002 38 3
42 Mitigating Hallucinations in LM-Based TTS Models via Distribution Alignment Using GFlowNets (2025) Multimodal Retrieval and Embedding
42 SyncTalklip: Highly Synchronized Lip-Readable Speaker Generation with Multi-Task Learning (2024) Multimodal Retrieval and Embedding
7 Archiki Prasad 37 3
73 Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models (2024) Multimodal Reasoning and Grounding
63 GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs (2026) Language Model Pre-training and Adaptation
8 Junjie Zhou 0001 37 3
73 Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding (2025) Video and Temporal Understanding
71 MLVU: Benchmarking Multi-task Long Video Understanding (2025) Video and Temporal Understanding
9 Tingyu Song 37 3
71 VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos (2025) Video and Temporal Understanding
36 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval
10 Liwei Wang 0009 36 3
73 AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning (2025) Video and Temporal Understanding
38 RAST: Domain-Robust Dialogue Rewriting as Sequence Tagging (2021) Conversational Search and Retrieval
12 Xingyu Tan 0001 34 3
38 PRoH: Dynamic Planning and Reasoning over Knowledge Hypergraphs for Retrieval-Augmented Generation (2026) Multimodal and Graph RAG
36 MemoTime: Memory-Augmented Temporal Knowledge Graph Enhanced Large Language Model Reasoning (2026) KG-Augmented LLM Reasoning
13 Jiapu Wang 34 3
65 Large-Small Model Synergy with Multimodal Fine-Grained Heuristics for Knowledge-Based Visual Question Answering (2025) Retrieval-Augmented and Open-Domain QA
33 Decomposing and Revising What Language Models Generate (2025) Multi-Hop and Iterative Retrieval Reasoning
14 Jianfeng Dong 34 3
40 IVCR-200K: A Large-Scale Multi-turn Dialogue Benchmark for Interactive Video Corpus Retrieval (2025) Conversational Search and Retrieval
37 Temporal Sentence Grounding with Relevance Feedback in Videos (2024) Video and Moment Retrieval
16 Weipeng Chen 33 3
71 Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs (2025) Video and Temporal Understanding
38 AITQE: An Adaptive Image-Text Quality Enhancer for Scalable MLLM Pretraining (2026) Multimodal Evaluation and Benchmarks
18 Mariano Barone 33 3
38 AMICO: A Semantic and Multimodal Framework for AI-Assisted Clinical Reporting (2025) Multimodal Evaluation and Benchmarks
36 A Multilingual Multimodal Medical Examination Dataset for Visual Question Answering in Healthcare (2025) Retrieval-Augmented and Open-Domain QA
19 Qinggang Zhang 33 3
38 KnowGPT: Knowledge Graph based Prompting for Large Language Models (2024) Multimodal and Graph RAG
38 Augmenting Intra-Modal Understanding in MLLMs for Robust Multimodal Keyphrase Generation (2026) Multimodal Evaluation and Benchmarks
20 Xinying Qian 33 3
36 TimeR⁴ : Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering (2024) Multi-Hop and Iterative Retrieval Reasoning
34 Beyond Timestamps: Bridging Forward and Backward Reasoning in Temporal Numerical and Relational Understanding (2026) Multi-hop and Complex Reasoning QA
21 Kaixin Ma 32 3
67 COLUMBUS: Evaluating COgnitive Lateral Understanding Through Multiple-Choice reBUSes (2025) Multimodal Reasoning and Grounding
38 Open Domain Question Answering with A Unified Knowledge Interface (2022) Multimodal and Graph RAG
22 Nancy F. Chen 32 3
42 Multimodal Dialogue State Tracking (2022) Conversational Search and Retrieval
42 Rescoring Hypothesized Detections of Out-of-Vocabulary Keywords Using Subword Samples (2016) Multimodal Retrieval and Embedding
26 Di Wu 0054 31 3
37 VisRet: Visualization Improves Knowledge-Intensive Text-to-Image Retrieval (2026) Multimodal and Graph RAG
36 LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (2025) Conversational Search and Retrieval
27 Jian Lang 31 3
37 REDEEMing Modality Information Loss: Retrieval-Guided Conditional Generation for Severely Modality Missing Learning (2025) Universal Multimodal Embedding and MLLM Retrieval
33 TAMEing Long Contexts in Personalization: Towards Training-Free and State-Aware MLLM Personalized Assistant (2026) Personalized Question Answering and RAG
28 Navve Wasserman 31 3
35 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval
35 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval
29 Yu Gu 0016 31 3
36 HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (2024) KG-Augmented LLM Reasoning
34 Few-shot In-context Learning on Knowledge Base Question Answering (2023) Multi-hop and Complex Reasoning QA
30 Armin Toroghi 31 3
34 CommonWhy: A Dataset for Evaluating Entity-Based Causal Commonsense Reasoning in Large Language Models (2026) Knowledge Graph Question Answering
32 Bayesian Inference with Complex Knowledge Graph Evidence (2024) Knowledge Graph Question Answering
31 Peitian Zhang 31 3
73 Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding (2025) Video and Temporal Understanding
33 INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning (2024) LLM-Based Reranking
32 Yanzhao Zhang 31 3
38 An End-to-End Model for Photo-Sharing Multi-Modal Dialogue Generation (2025) Multimodal Evaluation and Benchmarks
37 Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs (2025) Universal Multimodal Embedding and MLLM Retrieval
33 Jinheon Baek 31 3
39 Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (2023) Conversational Search and Retrieval
37 UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities (2026) Multimodal and Graph RAG
34 Idan Szpektor 31 3
39 On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method (2023) Conversational Search and Retrieval
38 Bridging the Visual Gap: Fine-Tuning Multimodal Models with Knowledge-Adapted Captions (2025) Multimodal Evaluation and Benchmarks
35 Weiwei Sun 0001 31 3
38 DukeNet: A Dual Knowledge Interaction Network for Knowledge-Grounded Conversation (2020) Conversational Search and Retrieval
33 Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models (2026) Evidence Selection and Context Refinement
36 Ben Liu 0002 31 3
31 Plan Then Retrieve: Reinforcement Learning-Guided Complex Reasoning over Knowledge Graphs (2026) RL Training of Search Agents
30 Filter-then-Generate: Large Language Models with Structure-Text Adapter for Knowledge Graph Completion (2025) Knowledge Graph Embedding and Completion
37 Chen Zhao 0013 30 3
38 Are Multimodal LLMs Robust Against Adversarial Perturbations? RoMMath: A Systematic Evaluation on Multimodal Math Reasoning (2025) Multimodal Evaluation and Benchmarks
34 TaPERA: Enhancing Faithfulness and Interpretability in Long-Form Table QA by Content Planning and Execution-based Reasoning (2024) Multi-hop and Complex Reasoning QA