Despite significant progress in Transformer-based visual tracking, standard self-attention mechanisms inherently lack the discriminative power required for accurate template search and matching. Due to the global normalization of softmax, the model inevitably assigns probabilistic quality to background noise terms that are semantically similar to the target. This "attention noise" not only reduces tracking accuracy but also forces the network to require substantial training data and computational costs to learn robust feature suppression. To address this, we propose DINOTrack, a noise-aware tracking framework designed to systematically eliminate background clutter. First, we construct a discriminative representation foundation using a frozen DINOv3 backbone with hierarchical multi-level feature fusion, ensuring the model captures robust semantic features essential for distinguishing the target from distractors. Second, to mitigate noise before feature interaction, we design a Class-Guided Heatmap Modulation (CGHM) module. By utilizing the template’s class label as a semantic prior, this module explicitly suppresses background interference and enhances potential target responses. Crucially, we introduce a Differential Denoising Interaction (DDI) mechanism for active noise cancellation. By performing a difference operation on dual query-key pairs, the DDI module effectively eliminates shared attention noise caused by semantic aliasing, enabling precise target matching. Extensive experiments demonstrate that DINOTrack achieves significant performance improvements on datasets such as LaSOT, GOT-10k, TNL2K, LaSOText, and TrackingNet.

Current mainstream methods for multimodal image fusion rely on CNN and Transformers, which often suffer from severe information loss or high computational complexity. However, state-space models offer a promising alternative with their linear complexity and reduced historical information loss. This paper proposes a novel adaptive Mamba network, named MGMFuse, for infrared and visible image fusion, aiming to seamlessly integrate thermal targets with high-resolution texture details. Specifically, feature extraction is performed using a dual-branch architecture. We design the MoEGMamba module that uses the Mixture of Experts'prompt generation mechanism to dynamically guide Mamba for adaptive state space modeling. Furthermore, a specialized mechanism is proposed to execute modality-specific channel-wise rectification and cross-modal interaction to align deep semantic features. Experimental results on multiple datasets demonstrate that the proposed algorithm outperforms existing methods. Furthermore, it significantly enhances the performance of downstream visual tasks.

Existing infrared and visible image fusion methods commonly use two structurally identical networks to extract deep features from source images, followed by a handcrafted or learnable feature fusion strategy. These methods overlook the modality-specific characteristics of the two image types, impairing the model’s ability to fully exploit their complementary information. Additionally, their fusion results often exhibit issues such as texture detail loss or unclear thermal targets. This is because the fusion rules they used are either too simple or too redundant. To address these challenges, we start from the infrared physics priors that are naturally complementary to visible images and incorporate the thermal diffusion equation and Stefan-Boltzmann Law into the image fusion architecture. Based on these two physical priors, we design a Thermal Diffusion Convolution (TDC) and a Stefan Thermal Attention (STA) to better extract infrared-specific features. Specifically, the TDC module leverages the anisotropic and isotropic characteristics of thermal diffusion adaptively to sharpen the edges of thermal targets and remove infrared noise, minimizing artifacts in the fused results. By decomposing the Stefan-Boltzmann Law, STA pays more attention on thermal features while suppressing redundant information, enabling more effective aggregation of complementary modality-specific details. To make full use of layer-wise complementary features, we propose an Interactive Injection Fusion framework(IIF) that hierarchically integrates these features, enhancing the richness of fused image content. Furthermore, an energy conservation constraint is designed to ensure the fused images adhere to physical principles. Extensive experimental results on five datasets demonstrate that our method sets a new state-of-the-art. Code is available at https://github.com/QiaoLiuHit/PPIFuse.

Existing deep Thermal InfraRed (TIR) trackers often use RGB datasets for training due to the lack of large-scale labeled TIR datasets. However, the performance of these methods on TIR image sequences is significantly degraded, because of the domain shift problem between the RGB and TIR datasets. To solve this problem, in this paper, we propose an unsupervised Dual-level Domain Adaptation TIR Tracking framework (DDAT), which can benefit from training on large-scale labeled RGB datasets and unlabeled TIR datasets. Specifically, to transfer the useful knowledge learned from RGB dataset to TIR tracking, we first propose an adversarial-based adaptation module on both the semantic-level and the feature-level. While the semantic-level adaptation can reduce the semantic gap between the TIR and RGB tracking tasks, the feature-level adaptation can learn domain-invariant features for more robust tracking. Second, we propose a partial domain adaptation module to alleviate the negative transfer problem because the RGB and TIR tracking domains have a non-identical class and feature spaces. Instead of aligning the entire feature space, this module adaptively selects partial similarity samples and features for alignment, thus getting more fine-grained aligned results. Third, we collect a currently largest-scale unlabeled TIR dataset to train the proposed framework. Extensive experiments on five TIR tracking benchmarks demonstrate the proposed method is effective and sets a new state-of-the-art.

The current two-stage tracking method locates the target using the position with the highest confidence score, and updates the template using a carefully designed template update strategy. However, we identify two key issues with these trackers: (1) the update strategy lacks continuous, cost-free template adaptation, leading to suboptimal tracking under appearance changes and (2) the location with the highest confidence score does not always yield accurate bounding boxes, potentially resulting in incomplete target coverage. In this article, we propose a novel tracker that incorporates two key innovations. First, the tracker employs a dual update strategy that performs online template updates at both the image and feature levels. This strategy enables continuous adaptation to target appearance changes without introducing additional computational overhead. Second, we enhance the existing loss function by introducing a Classification–Regression Interaction (CRI) loss, which guides the training process to produce confidence scores that more accurately reflect the quality of the predicted bounding boxes. Extensive experiments are conducted to evaluate the performance of our tracker and the effectiveness of the proposed methods. The experimental results show that our method has achieved a comprehensive improvement over the baseline on five datasets, and achieves competitive performance compared to state-of-the-art trackers.

Constrained by the scarcity of labeled Thermal InfraRed (TIR) training data, current TIR trackers commonly rely on pre-trained RGB trackers. However, the domain discrepancy between TIR and RGB images limits effective utilization of RGB features, significantly degrades TIR tracking performance. To solve this challenge, we propose a hierarchical domain adaptation model to transfer useful pre-trained RGB features into TIR tracking more effective and efficient. Specifically, we first design a reflectance consistency network to learn style-invariant representations. Second, we present a target-aware adversarial network to align the target semantic features of the two domains. These two modules respectively narrow the distribution gap at the stylistic and semantic levels in a hierarchical manner. Third, to solve the inefficiency problem of domain adaptive training, we also propose a Bi-rank adapter side network to accelerate this process. While significantly reducing training time by 90%, our method achieves a new state-of-the-art on four TIR tracking benchmarks.

Hyperspectral videos contain rich spectral and spatial information, which has enormous potential for object tracking compared to traditional RGB videos. However, the limited hyperspectral data, spectral differences across different bands, and high computational costs result in existing trackers being unable to effectively build connections between spectral and spatial information, leading to suboptimal tracking performance. To address these issues, this paper proposes a multi-stream Visual-Spectral-Spatial adaptive hyperspectral object tracking model(VSS). First, we design a multi-stream hyperspectral Transformer module to receive spectral data from different bands and extract Visual-Spectral-Spatial features across those bands. Then, to address the band differences between different modalities, we introduce the Bidirectional Visual-Spectral-Spatial Adapter module, which adaptively fuses visual, spectral, and spatial information from different bands. Finally, experiments on the HOTC dataset demonstrate the excellent performance of the VSS model.

Unlike traditional single-modal visual tracking, vision-language tracking leverages textual descriptions to provide semantic understanding, aiding in handling challenges such as appearance variations, occlusions, and motion blur. However, the semantic gap between textual and visual information complicates cross-modal alignment and fusion. Moreover, fixed textual descriptions may fail to reflect real-time variations in dynamic video frames, thereby affecting tracking accuracy. To address these challenges, we propose MACT, the Multi-scale Adaptive Cascaded Tracker, which enhances vision-language integration through a series of key mechanisms. First, Multi-Scale Feature Alignment (MSFA) encodes image and text features at multiple scales and dynamically aligns them using cross-modal attention, strengthening information flow and interaction between modalities. Second, Adaptive Cascaded Fusion (ACF) adaptively learns the correspondence between text descriptions and video frames, extracting appropriate hierarchical interaction features to ensure that textual information effectively aids visual tracking. Finally, Scale Recovery(SR) integrates multi-scale deep feature representations back to the original scale before passing them to the tracking head for final predictions. Extensive experiments on TNL2K, LaSOT, and OTB99 demonstrate that our proposed MACT achieves state-of-the-art tracking performance.

Thermal infrared (TIR) object tracking is a significant subject within the field of computer vision. Currently, TIR object tracking faces challenges such as insufficient representation of object texture information and underutilization of temporal information, which severely affects the tracking accuracy of TIR tracking methods. To address these issues, we propose a TIR object tracking method (called: FFTR) based on fine-grained feature and template reconstruction. Specifically, aiming at the fine-grained information of the TIR object, we employ a frequency channel attention mechanism that transforms TIR images into the frequency domain using discrete cosine transform features. By capturing the fine-grained feature of TIR images from the frequency domain, we enhance the model’s ability to comprehend these images. To better leverage temporal information, we utilize a template region reconstruction method. This method reconstructs the template from the previous frame based on the search area of the current frame, which is then incorporated into the attention computation for the subsequent frame, thereby improving the tracking capability of TIR objects. Extensive quantitative and qualitative experiments show that our method achieves competitive tracking performance on the TIR benchmarks.

Most extant underwater object tracking (UOT) utilize generic tracking algorithms, which lack applicability to underwater tracking scenarios. Moreover, these algorithms primarily emphasize minimizing interference from various challenging tasks to prevent target drift, but pay less attention to the strategies for mitigating target drift once it occurs. To alleviate the above problems, we propose a simple, effective, and UOT-focused adaptive trajectory correction framework, named ATCTrack. From the perspective of tracking failure, this methodology aims to promptly identify and rectify unreasonable target drift through accurate trajectory coordinate correction and trajectory template updates. Additionally, to mitigate the adverse effects of potential erroneous corrections, we implement an adaptive strategy that corrects only significant target drift, allowing for self-correction within a certain margin. Finally, we introduce an adaptive underwater image enhancement technique to improve the underwater image quality and maintain the trajectory’s stability and clarity. Our tracker achieves state-of-the-art performance on the currently prevalent UOT tracking benchmarks compared to other trackers.

Unlike visual object tracking, thermal infrared (TIR) object tracking methods can track the target of interest in poor visibility such as rain, snow, and fog, or even in total darkness. This feature brings a wide range of application prospects for TIR object-tracking methods. However, this field lacks a unified and large-scale training and evaluation benchmark, which has severely hindered its development. To this end, we present a large-scale and high-diversity unified TIR single object tracking benchmark, called LSOTB-TIR, which consists of a tracking evaluation dataset and a general training dataset with a total of 1416 TIR sequences and more than 643 K frames. We annotate the bounding box of objects in every frame of all sequences and generate over 770 K bounding boxes in total. To the best of our knowledge, LSOTB-TIR is the largest and most diverse TIR object tracking benchmark to date. We spilt the evaluation dataset into a short-term tracking subset and a long-term tracking subset to evaluate trackers using different paradigms. What's more, to evaluate a tracker on different attributes, we also define four scenario attributes and 12 challenge attributes in the short-term tracking evaluation subset. By releasing LSOTB-TIR, we encourage the community to develop deep learning-based TIR trackers and evaluate them fairly and comprehensively. We evaluate and analyze 40 trackers on LSOTB-TIR to provide a series of baselines and give some insights and future research directions in TIR object tracking. Furthermore, we retrain several representative deep trackers on LSOTB-TIR, and their results demonstrate that the proposed training dataset significantly improves the performance of deep TIR trackers. Codes and dataset are available at https://github.com/QiaoLiuHit/LSOTB-TIR.

Convolutional neural networks (CNNs) have been successfully applied to the single target tracking task in recent years. Generally, training a deep CNN model requires numerous labeled training samples, and the number and quality of these samples directly affect the representational capability of the trained model. However, this approach is restrictive in practice, because manually labeling such a large number of training samples is time-consuming and prohibitively expensive. In this article, we propose an active learning method for deep visual tracking, which selects and annotates the unlabeled samples to train the deep CNN model. Under the guidance of active learning, the tracker based on the trained deep CNN model can achieve competitive tracking performance while reducing the labeling cost. More specifically, to ensure the diversity of selected samples, we propose an active learning method based on multiframe collaboration to select those training samples that should be and need to be annotated. Meanwhile, considering the representativeness of these selected samples, we adopt a nearest-neighbor discrimination method based on the average nearest-neighbor distance to screen isolated samples and low-quality samples. Therefore, the training samples' subset selected based on our method requires only a given budget to maintain the diversity and representativeness of the entire sample set. Furthermore, we adopt a Tversky loss to improve the bounding box estimation of our tracker, which can ensure that the tracker achieves more accurate target states. Extensive experimental results confirm that our active-learning-based tracker (ALT) achieves competitive tracking accuracy and speed compared with state-of-the-art trackers on the seven most challenging evaluation benchmarks. Project website: https://sites.google.com/view/altrack/.

Existing methods for video-based person re- identification (ReID) mainly learn the appearance feature of a given pedestrian via a feature extractor and a feature aggregator. However, the appearance models would fail to learn a large inter-class variance when different pedestrians have similar appearances. Considering that different pedestrians have different walking postures and body proportions, we propose to learn the discriminative pose feature beyond the appearance feature for video retrieval. Specifically, we implement a two-branch architecture to separately learn the appearance feature and pose feature, and then concatenate them together for inference. To learn the pose feature, we first detect the pedestrian pose in each frame through an off-the-shelf pose detector, and construct a temporal graph using the pose sequence. We then exploit a recurrent graph convolutional network (RGCN) to learn the node embeddings of the temporal pose graph, which devises a global information propagation mechanism to simultaneously achieve the neighborhood aggregation of intra-frame nodes and message passing among inter-frame graphs. Finally, we propose a dual-attention method (DAM) consisting of node-attention and time-attention to obtain the temporal graph representation from the node embeddings, where the self-attention mechanism is employed to learn the importance of each node and each frame. We verify the proposed method on three video-based ReID datasets, i.e., Mars, DukeMTMC and iLIDS-VID, whose experimental results demonstrate that the learned pose feature can effectively improve the performance of existing appearance models.

The feature models used by existing Thermal InfraRed (TIR) tracking methods are usually learned from RGB images due to the lack of a large-scale TIR image training dataset. However, these feature models are less effective in representing TIR objects and they are difficult to effectively distinguish distractors because they do not contain fine-grained discriminative information. To this end, we propose a dual-level feature model containing the TIR-specific discriminative feature and fine-grained correlation feature for robust TIR object tracking. Specifically, to distinguish inter-class TIR objects, we first design an auxiliary multi-classification network to learn the TIR-specific discriminative feature. Then, to recognize intra-class TIR objects, we propose a fine-grained aware module to learn the fine-grained correlation feature. These two kinds of features complement each other and represent TIR objects in the levels of inter-class and intra-class respectively. These two feature models are constructed using a multi-task matching framework and are jointly optimized on the TIR object tracking task. In addition, we develop a large-scale TIR image dataset to train the network for learning TIR-specific feature patterns. To the best of our knowledge, this is the largest TIR tracking training dataset with the richest object class and scenario. To verify the effectiveness of the proposed dual-level feature model, we propose an offline TIR tracker (MMNet) and an online TIR tracker (ECO-MM) based on the feature model and evaluate them on three TIR tracking benchmarks. Extensive experimental results on these benchmarks demonstrate that the proposed algorithms perform favorably against the state-of-the-art methods.

In visual tracking, it is challenging to distinguish the target from similar objects called noises in the background. As deep trackers use convolutional neural networks for image classification as feature extractors, the extracted features are insensitive to different instances in the same class, which is prone to make prediction models confuse the target and the similar noises in the background. To this end, we propose a noise-suppressing algorithm to learn the discriminative representation for distinguishing the target from the noises in the background. First, we learn polynomial kernels for a search patch under the semantic guidance to increase the difference between representations of the target and the noises in the background. Second, we formulate the online foreground-background functions for the target and the noises in the background to learn an adaptive kernel, which suppresses the features positive for the noises and promotes the features positive for the target. We evaluate the proposed method on seven public datasets including OTB-2013, OTB-2015, VOT-2018, LaSOT, TrackingNet, GOT10k, and NFS. The comprehensive experimental results show that the proposed algorithm performs favorably against state-of-the-art methods, while running at real-time speed.

The training of a feature extraction network typically requires abundant manually annotated training samples, making this a time-consuming and costly process. Accordingly, we propose an effective self-supervised learning-based tracker in a deep correlation framework (named: self-SDCT). Motivated by the forward-backward tracking consistency of a robust tracker, we propose a multi-cycle consistency loss as self-supervised information for learning feature extraction network from adjacent video frames. At the training stage, we generate pseudo-labels of consecutive video frames by forward-backward prediction under a Siamese correlation tracking framework and utilize the proposed multi-cycle consistency loss to learn a feature extraction network. Furthermore, we propose a similarity dropout strategy to enable some low-quality training sample pairs to be dropped and also adopt a cycle trajectory consistency loss in each sample pair to improve the training loss function. At the tracking stage, we employ the pre-trained feature extraction network to extract features and utilize a Siamese correlation tracking framework to locate the target using forward tracking alone. Extensive experimental results indicate that the proposed self-supervised deep correlation tracker (self-SDCT) achieves competitive tracking performance contrasted to state-of-the-art supervised and unsupervised tracking methods on standard evaluation benchmarks.

Existing deep Thermal InfraRed (TIR) trackers only use semantic features to represent the TIR object, which lack the sufficient discriminative capacity for handling distractors. This becomes worse when the feature extraction network is only trained on RGB images. To address this issue, we propose a multi-level similarity model under a Siamese framework for robust TIR object tracking. Specifically, we compute different pattern similarities using the proposed multi-level similarity network. One of them focuses on the global semantic similarity and the other computes the local structural similarity of the TIR object. These two similarities complement each other and hence enhance the discriminative capacity of the network for handling distractors. In addition, we design a simple while effective relative entropy based ensemble subnetwork to integrate the semantic and structural similarities. This subnetwork can adaptive learn the weights of the semantic and structural similarities at the training stage. To further enhance the discriminative capacity of the tracker, we propose a large-scale TIR video sequence dataset for training the proposed model. To the best of our knowledge, this is the first and the largest TIR object tracking training dataset to date. The proposed TIR dataset not only benefits the training for TIR object tracking but also can be applied to numerous TIR visual tasks. Extensive experimental results on three benchmarks demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods.

Existing deep Thermal InfraRed (TIR) trackers usually use the feature models of RGB trackers for representation. However, these feature models learned on RGB images are neither effective in representing TIR objects nor taking fine-grained TIR information into consideration. To this end, we develop a multi-task framework to learn the TIR-specific discriminative features and fine-grained correlation features for TIR tracking. Specifically, we first use an auxiliary classification network to guide the generation of TIR-specific discriminative features for distinguishing the TIR objects belonging to different classes. Second, we design a fine-grained aware module to capture more subtle information for distinguishing the TIR objects belonging to the same class. These two kinds of features complement each other and recognize TIR objects in the levels of inter-class and intra-class respectively. These two feature models are learned using a multi-task matching framework and are jointly optimized on the TIR tracking task. In addition, we develop a large-scale TIR training dataset to train the network for adapting the model to the TIR domain. Extensive experimental results on three benchmarks show that the proposed algorithm achieves a relative gain of 10% over the baseline and performs favorably against the state-of-the-art methods. Codes and the proposed TIR dataset are available at https://github.com/QiaoLiuHit/MMNet.

In this paper, we present a Large-Scale and high-diversity general Thermal InfraRed (TIR) Object Tracking Benchmark, called LSOTB-TIR, which consists of an evaluation dataset and a training dataset with a total of 1,400 TIR sequences and more than 600K frames. We annotate the bounding box of objects in every frame of all sequences and generate over 730K bounding boxes in total. To the best of our knowledge, LSOTB-TIR is the largest and most diverse TIR object tracking benchmark to date. To evaluate a tracker on different attributes, we define 4 scenario attributes and 12 challenge attributes in the evaluation dataset. By releasing LSOTB-TIR, we encourage the community to develop deep learning based TIR trackers and evaluate them fairly and comprehensively. We evaluate and analyze more than 30 trackers on LSOTB-TIR to provide a series of baselines, and the results show that deep trackers achieve promising performance. Furthermore, we re-train several representative deep trackers on LSOTB-TIR, and their results demonstrate that the proposed training dataset significantly improves the performance of deep TIR trackers. Codes and dataset are available at https://github.com/QiaoLiuHit/LSOTB-TIR.

Thermal infrared (TIR) pedestrian tracking is one of the important components among numerous applications of computer vision, which has a major advantage: it can track pedestrians in total darkness. The ability to evaluate the TIR pedestrian tracker fairly, on a benchmark dataset, is significant for the development of this field. However, there is not a benchmark dataset. In this paper, we develop a TIR pedestrian tracking dataset for the TIR pedestrian tracker evaluation. The dataset includes 60 thermal sequences with manual annotations. Each sequence has nine attribute labels for the attribute based evaluation. In addition to the dataset, we carry out the large-scale evaluation experiments on our benchmark dataset using nine publicly available trackers. The experimental results help us understand the strengths and weaknesses of these trackers. In addition, in order to gain more insight into the TIR pedestrian tracker, we divide its functions into three components: feature extractor, motion model, and observation model. Then, we conduct three comparison experiments on our benchmark dataset to validate how each component affects the tracker's performance. The findings of these experiments provide some guidelines for future research.