Single object tracking aims to localize target object with specific reference modalities (bounding box, natural language or both) in a sequence of specific video modalities (RGB, RGB+Depth, RGB+Thermal or RGB+Event.). Different reference modalities enable various human-machine interactions, and different video modalities are demanded in complex scenarios to enhance tracking robustness. Existing trackers are designed for single or several video modalities with single or several reference modalities, which leads to separate model designs and limits practical applications. Practically, a unified tracker is needed to handle various requirements. To the best of our knowledge, there is still no tracker that can perform tracking with these above reference modalities across these video modalities simultaneously. Thus, in this paper, we present a unified tracker, UniSOT, for different combinations of three reference modalities and four video modalities with uniform parameters. Extensive experimental results on 18 visual tracking, vision-language tracking and RGB+X tracking benchmarks demonstrate that UniSOT shows superior performance against modality-specific counterparts. Notably, UniSOT outperforms previous counterparts by over 3.0% AUC on TNL2K across all three reference modalities and outperforms Un-Track by over 2.0% main metric across all three RGB+X video modalities.

Natural language tracking aims to locate the target of a video based on a language description. The rich contextual information inside the language and video sequence is essential to describe the target movements and appearance variations. However, existing natural language trackers design a fixed-length memory to store historical target information, which merely uses limited context information and necessitates manually designed modules, resulting in sub-optimal localization performance and numerous computational costs. Inspired by the success of the state space model, we propose a novel Context-adaptive Mamba Tracker (CMTrack). It enjoys several merits. First, we propose a novel context-aware state space model that enables language features to serve as hidden states to interact with relevant image features adaptively. Second, CMTrack transfers the hidden states frame-by-frame to continuously incorporate contextual target information into language features, enabling context-adaptive language cues. Third, the proposed context-adaptive language cues can effectively capture the long-range behavior of the target and guide the tracker in locating the target accurately without any extra design. Finally, CMTrack provides a neat pipeline for training and tracking with linear complexity. Experimental results demonstrate that CMTrack achieves new state-of-the-art performance.

3D single object tracking (3D SOT) remains a challenging task due to the sparsity of point clouds, appearance variations caused by occlusions, and the difficulty of modeling long-term temporal context. Although recent Transformer-based approaches leverage memory mechanisms to propagate temporal information, their quadratic complexity and reliance on discrete historical snapshots limit both efficiency and temporal coherence. To address these limitations, we propose SSMTrack, a novel 3D SOT framework built upon state space models (SSMs), which efficiently models long-term temporal dependencies through a continuously evolving hidden state with linear complexity. Specifically, we introduce a serialization and bidirectional scanning (SBS) strategy to enhance intra-frame feature interactions and design a Target-Aware Encoder (TAE) to extract target cues while maintaining stable temporal representations. Furthermore, we propose a Temporal Causal Shape Learning (TCSL) mechanism that preserves critical historical information while adaptively integrating current inputs, progressively enriching target feature representations over time. Extensive experiments on three benchmark datasets demonstrate that SSMTrack achieves state-of-the-art performance with strong temporal coherence and high efficiency. The code will be released upon publication.

Natural language tracking aims to locate the position of a target specified by a natural language description. Existing methods are trained on vision-language datasets with a small number of language descriptions, which may lead to limited semantic generalization. Moreover, they extract visual and language features separately, which limits visual-semantic capabilities. To overcome these limitations, we propose a novel semantic-aware tracking framework, SATrack, which integrates a semantic-aware attention module and a cross-modal aggregation module. The proposed SATrack enjoys several merits. First, the semantic-aware attention module utilizes language semantics as a bridge to build associations between visual features, enabling stronger visual-semantic capabilities. Second, the cross-modal aggregation module transfers the semantic knowledge of CLIP into the tracking framework for semantic generalization. Extensive experimental results demonstrate that SATrack outperforms previous state-of-the-art trackers on four natural language tracking benchmarks.

Single object tracking aims to locate the target object in a video sequence according to the state specified by different modal references, including the initial bounding box (BBOX), natural language (NL), or both (NL+BBOX). Due to the gap between different modalities, most existing trackers are designed for single or partial of these reference settings and overspecialize on the specific modality. Differently, we present a unified tracker called UVLTrack, which can simultaneously handle all three reference settings (BBOX, NL, NL+BBOX) with the same parameters. The proposed UVLTrack enjoys several merits. First, we design a modality-unified feature extractor for joint visual and language feature learning and propose a multi-modal contrastive loss to align the visual and language features into a unified semantic space. Second, a modality-adaptive box head is proposed, which makes full use of the target reference to mine ever-changing scenario features dynamically from video contexts and distinguish the target in a contrastive way, enabling robust performance in different reference settings. Extensive experimental results demonstrate that UVLTrack achieves promising performance on seven visual tracking datasets, three vision-language tracking datasets, and three visual grounding datasets. Codes and models will be open-sourced at https://github.com/OpenSpaceAI/UVLTrack.