Joey Tianyi Zhou

dblp:123/5110 · also Joey Zhou, Tianyi Zhou 0007 · DBLP profile ↗
← Back
7ranked-venue papers in the field
0as first author
6since 2021 · last 2026
0000-0002-4675-7055ORCID · conflict

Domains — venue-derived; a paper can count in several

Knowledge Engineering, Semantic Web & Information Systems · 4Information Retrieval & Web Search · 3
YearPublicationVenuePosition
2026 Agentic Spatio-Temporal Grounding via Collaborative Reasoning
abstract
Spatio-Temporal Video Grounding (STVG) aims to retrieve the spatio-temporal tube of a target object or person in a video given a text query. Most existing approaches perform frame-wise spatial localization within a predicted temporal span, resulting in redundant computation, heavy supervision requirements, and limited generalization. Weakly-supervised variants mitigate annotation costs but remain constrained by the dataset-level train-and-fit paradigm with an inferior performance. To address these challenges, we propose the Agentic Spatio-Temporal Grounder (ASTG) framework for the task of STVG in an open-world and zero-shot setting. Specifically, two specialized agents SRA (Spatial Reasoning Agent) and TRA (Temporal Reasoning Agent) constructed leveraging modern Multi-modal Large Language Models (MLLMs) work collaboratively to retrieve the target tube in an autonomous and self-guided manner. Following a propose-and-evaluation paradigm, ASTG duly decouples spatio-temporal reasoning and automates the tube extraction, verification and temporal localization processes. With a dedicated visual memory and dialogue context, ASTG achieves architectural efficiency by minimizing the number of reasoning calls compared to exhaustive per-frame reasoning, eliminating the logical redundancy inherent in joint-reasoning systems. Experiments on popular benchmarks demonstrate the superiority of the proposed approach where it outperforms existing weakly-supervised and zero-shot approaches by a margin and is comparable to some of the fully-supervised methods.
Heng Zhao 0004, Yew-Soon Ong, Joey Tianyi Zhou
SIGIR3
2026 LaV-CoT: Language-Aware Visual CoT with Multi-Aspect Reward Optimization for Multilingual Text-Centric VQA
abstract
Multilingual Text-Centric Visual Question Answering (TEC-VQA) has become crucial for real-world applications, as it requires fine-grained understanding and reasoning over multilingual scene text. Recent advances in vision-language models (VLMs) have demonstrated strong potential in tackling multimodal tasks. However, most existing approaches rely primarily on textual Chain-of-Thought (CoT) and provide limited support for multilingual multimodal reasoning. To address this gap, we introduce LaV-CoT, the first Language-aware Visual CoT framework with Multi-Aspect Reward Optimization. LaV-CoT incorporates an interpretable multi-stage reasoning pipeline consisting of text summary with bounding box, language identification, spatial object-level captioning, and step-by-step logical reasoning. To improve reasoning accuracy and cross-lingual generalization, we propose a novel verifiable Multi-Aspect Reward Optimization in addition to supervised fine-tuning that incorporates rewards for linguistic consistency, structural fidelity, and response accuracy. Extensive evaluations on public datasets, including MMMB, Multilingual MMBench, and MTVQA, show that LaV-CoT outperforms open-source models of similar size by up to ~9.5% accuracy, even surpassing open-source models more than twice its size, and further exceeding several state-of-the-art proprietary models. Moreover, LaV-CoT has been integrated into our online Intelligent Document Processing platform. A further online A/B test demonstrates an \(\sim\)8.7% improvement in acceptance rate, validating its effectiveness in industrial deployment and commercial applications. Our code is available at this https://github.com/HJNVR/LaV-CoT repository.
Zhiya Tan, Shutao Gong, Fanwei Zeng, Joey Tianyi Zhou, Changtao Miao, Huazhe Tan, Weibin Yao, Jianshu Li
WWW5
2024 A principled framework for explainable multimodal disentanglement
Zongbo Han, Tao Luo 0014, Huazhu Fu, Qinghua Hu, Joey Tianyi Zhou, Changqing Zhang 0002
Inf. Sci.5
2023 Multi-spectral template matching based object detection in a few-shot learning manner
Chen Feng 0002, Zhiguo Cao 0001, Yang Xiao 0007, Zhiwen Fang, Joey Tianyi Zhou
Inf. Sci.5
2021 Video Corpus Moment Retrieval with Contrastive Learning
abstract
Given a collection of untrimmed and unsegmented videos, video corpus moment retrieval (VCMR) is to retrieve a temporal moment (i.e., a fraction of a video) that semantically corresponds to a given text query. As video and text are from two distinct feature spaces, there are two general approaches to address VCMR: (i) to separately encode each modality representations, then align the two modality representations for query processing, and (ii) to adopt fine-grained cross-modal interaction to learn multi-modal representations for query processing. While the second approach often leads to better retrieval accuracy, the first approach is far more efficient. In this paper, we propose a Retrieval and Localization Network with Contrastive Learning (ReLoCLNet) for VCMR. We adopt the first approach and introduce two contrastive learning objectives to refine video encoder and text encoder to learn video and text representations separately but with better alignment for VCMR. The video contrastive learning (VideoCL) is to maximize mutual information between query and candidate video at video-level. The frame contrastive learning (FrameCL) aims to highlight the moment region corresponds to the query at frame-level, within a video. Experimental results show that, although ReLoCLNet encodes text and video separately for efficiency, its retrieval accuracy is comparable with baselines adopting cross-modal interaction learning.
Hao Zhang 0048, Aixin Sun, Guoshun Nan, Liangli Zhen, Joey Tianyi Zhou, Rick Siow Mong Goh
SIGIR6
2021 LPQ++: A discriminative blur-insensitive textural descriptor with spatial-channel interaction
Yang Xiao 0007, Zhiguo Cao 0001, Zhiwen Fang, Joey Tianyi Zhou
Inf. Sci.6
2019 Action recognition for depth video using multi-view dynamic images
Yang Xiao 0007, Jun Chen 0001, Yancheng Wang 0002, Zhiguo Cao 0001, Joey Tianyi Zhou, Xiang Bai
Inf. Sci.5