Open-vocabulary multiple object tracking (MOT) aims to track arbitrary objects in the real world. Although significant progress has been achieved in object classification by leveraging the knowledge from large vision-language models, advances in data association for open-vocabulary MOT remain limited. Existing methods primarily rely on appearance cues to establish associations. However, these cues are often unreliable in the face of occlusions and ambiguous object appearances, resulting in suboptimal tracking performance in complex scenarios. In this paper, we propose a novel open-vocabulary MOT method, Spatial-temporal Scene Graph Tracker (SSGTrack), which introduces a fundamentally different approach to data association by building a Spatial-temporal Scene Graph (SSG) that captures rich semantic and spatial relationships between objects across adjacent frames. Specifically, SSGTrack constructs proposal-level relationships by extracting diverse contextual information from the multi-head self-attention layers of the Transformer decoder. These relationships, derived from the keyframe and reference frame, are compressed into the compact SSG, where nodes represent detected objects, and edge weights denote frame-level connectivity. Furthermore, to address the challenge of differentiating visually similar objects and background distractors, we propose a Context-aware Contrastive Learning (CCL) strategy. By identifying background features that significantly differ from positive samples and incorporating them as negative samples, CCL enhances the ability of the model to learn discriminative representations, thus improving tracking robustness. Extensive experiments conducted on several challenging MOT benchmarks demonstrate the effectiveness of our method, which achieves superior tracking performance.

Referring multi-object tracking is a new rising research topic that aims at detecting and tracking the referred objects in a video sequence based on a natural language expression. Compared with traditional multi-object tracking, this setting guides object tracking with high-level semantic information, which may bring more flexible and robust tracking performance in practical scenarios. However, existing methods perform cross-modal fusion in only one phase. The limitation of visual-linguistic representation is prone to causing visionlanguage mismatching and producing poor tracking results. To effectively fuse vision and language modalities, we propose DeepRMOT with deep cross-modality fusion, including an enhanced early-fusion module, a bidirectional crossmodality encoder, and a cross-modality decoder. Therefore, DeepRMOT can boost object detection and data association by the enhanced visual-linguistic representation. Extensive experiments on the Refer-KITTI demonstrate the effectiveness of our method.

Multi-object tracking (MOT) has gained remarkable progress in recent years, while due to the complexity of real-world environments, there are still many challenges that remain unsolved, such as object occlusion and deformation. To effectively alleviate this problem, we propose a simple yet effective Transformer-based tracker, named CLNet, consisting of an Instance-Aware Localization (IAL) module and a Temporal Context Aggregation (TCA) module. Specifically, the former learns the correlation of object positions for potential location estimation, and the latter learns the correlation of background contexts to obtain robust re-ID features for data association. Experimental results show that CLNet outperforms the baseline method by +2.2 MOTA and +2.3 IDF1 on MOT17 and +6.3 MOTA and +3.6 IDF1 on MOT20 respectively, which demonstrate the effectiveness of the proposed method.

Open-vocabulary multi-object tracking (MOT) aims to track arbitrary objects encountered in the real world beyond the training set. However, recent methods rely solely on instance-level detection and association of novel objects, which may not consider the valuable fine-grained semantic representations of the targets within key and reference frames. In this paper, we propose a Global and Local Awareness open-vocabulary MOT method (GLATrack), which learns to tackle the task of real-world MOT from both global and instance-level perspectives. Specifically, we introduce a region-aware feature enhancement module to refine global knowledge for complementing local target information, which enhances semantic representation and bridges the distribution gap between the image feature map and the pooled regional features. We propose a bidirectional semantic complementarity strategy to mitigate semantic misalignment arising from missing target information in key frames, which dynamically selects valuable information within reference frames to enrich object representation during the knowledge distillation process. Furthermore, we introduce an appearance richness measurement module to provide appropriate representations for targets with different appearances. The proposed method gains an improvement of 6.9% in TETA and 5.6% in mAP on the large-scale TAO benchmark.