Abstract Transformer-based visual trackers have demonstrated significant advancements due to their powerful modeling capabilities. However, their practicality is limited on resource-constrained devices because of their slow processing speeds. To address this challenge, we present HiT, a novel family of efficient tracking models that achieve high performance while maintaining fast operation across various devices. The core innovation of HiT lies in its Bridge Module, which connects lightweight transformers to the tracking framework, enhancing feature representation quality. Additionally, we introduce a dual-image position encoding approach to effectively encode spatial information. HiT achieves an impressive speed of 61 frames per second (fps) on the NVIDIA Jetson AGX platform, alongside a competitive AUC of 64.6% on the LaSOT benchmark, outperforming all previous efficient trackers. Building on HiT, we propose DyHiT, an efficient dynamic tracker that flexibly adapts to scene complexity by selecting routes with varying computational requirements. DyHiT uses search area features extracted by the backbone network and inputs them into an efficient dynamic router to classify tracking scenarios. Based on the classification, DyHiT applies a divide-and-conquer strategy, selecting appropriate routes to achieve a superior trade-off between accuracy and speed. The fastest version of DyHiT achieves 111 fps on NVIDIA Jetson AGX while maintaining an AUC of 62.4% on LaSOT. Furthermore, we introduce a training-free acceleration method based on the dynamic routing architecture of DyHiT. This method significantly improves the execution speed of various high-performance trackers without sacrificing accuracy. For instance, our acceleration method enables the state-of-the-art tracker SeqTrack-B256 to achieve a $$2.68\times $$ 2.68 × speedup on an NVIDIA GeForce RTX 2080 Ti GPU while maintaining the same AUC of 69.9% on the LaSOT. Codes, models, and results are available at https://github.com/kangben258/HiT .

Due to the substantial gap between vision and language modalities, along with the mismatch problem between fixed language descriptions and dynamic visual information, existing vision-language tracking methods exhibit performance on par with or slightly worse than vision-only tracking. Effectively exploiting the rich semantics of language to enhance tracking robustness remains an open challenge. To address these issues, we propose a self-adaptive vision-language tracking framework that leverages the pre-trained multi-modal CLIP model to obtain well-aligned visual-language representations. A novel context-aware prompting mechanism is introduced to dynamically adapt linguistic cues based on the evolving visual context during tracking. Specifically, our context prompter extracts dynamic visual features from the current search image and integrates them into the text encoding process, enabling self-updating language embeddings. Furthermore, our framework employs a unified one-stream Transformer architecture, supporting joint training for both vision-only and vision-language tracking scenarios. Our method not only bridges the modality gap but also enhances robustness by allowing language features to evolve with visual context. Extensive experiments on four vision-language tracking benchmarks demonstrate that our method effectively leverages the advantages of language to enhance visual tracking. Our large model can obtain 55.0% AUC on $\text {LaSOT}_{\text {EXT}}$ and 69.0% AUC on TNL2K. Additionally, our language-only tracking model achieves performance comparable to that of state-of-the-art vision-only tracking methods on TNL2K. Code is available at https://github.com/zj5559/SAVLT.

Re-detection is a necessary capability for long-term tracking. Target candidate proposals in the whole image can provide a chance of tracking reset when tracking fails due to tracking drift or target invisibility. In this paper, we propose a unified local-global tracker based on the same transformer architecture sharing weights, which can not only search in a continuous local region but also provide target candidates of the global image in every frame. The requirements of both long-term and short-term scenarios can be addressed using a unified model. A simple proposal selection scheme is adopted to properly select the candidate proposals of re-detection, to assist tracking and obtain better performance. The scheme performs reevaluation of all high-quality proposals based on a transformer-based embedding network, once the predicted state of the local tracking is not sufficient to be accurate. To capture appearance variations brought by online updates in minimum risks, a long-term-friendly dynamic template update scheme is also designed. Extensive experiments are conducted to demonstrate the effectiveness of our proposed tracker, including three short-term tracking benchmarks and six long-term benchmarks. Our tracker can achieve results comparable to that of the state-of-the-art. The proposed tracker can also work well in balancing the performance and speed, achieving an average speed of approximately 25 fps tested on LaSOT testing set.

Vehicle logo detection (VLD) is a special and significant topic in object detection for vehicle identification system applications. Nevertheless, the range of the research and analysis for VLD are seriously narrow in the real complex scenes, although it’s a critical role in the object detection of small sizes. In this paper, we make further analysis work toward vehicle logo recognition and detection in real-world situations. To begin with, we propose a new multi-class VLD dataset, called VLD-45 (Vehicle Logo Dataset), which contains 45000 images and 50359 objects from 45 categories respectively. Our new dataset provides several research challenges involve in small sizes object, shape deformation, low contrast and so on. Meanwhile, we use 6 existing classifiers and 6 detectors to evaluate our dataset and show the baseline performance. According to the result, our dataset has very significant research value for the task of small-scale object detection. The dataset source:https://github.com/YangShuoys/VLD-45-B-DATASET-Detection

In this study, we propose a novel RGB-T tracking framework by jointly modeling both appearance and motion cues. First, to obtain a robust appearance model, we develop a novel late fusion method to infer the fusion weight maps of both RGB and thermal (T) modalities. The fusion weights are determined by using offline-trained global and local multimodal fusion networks, and then adopted to linearly combine the response maps of RGB and T modalities. Second, when the appearance cue is unreliable, we comprehensively take motion cues, i.e., target and camera motions, into account to make the tracker robust. We further propose a tracker switcher to switch the appearance and motion trackers flexibly. Numerous results on three recent RGB-T tracking datasets show that the proposed tracker performs significantly better than other state-of-the-art algorithms.

In this work, we develop an effective person search algorithm with natural language descriptions. The contributions of this work mainly include two aspects. First, we design a baseline language person search framework including three basic components: a deep CNN model to extract visual features, a bi-directional LSTM to encode language descriptions and the triplet loss to conduct cross-modal feature embedding. Second, we propose a novel mutually connected classification loss to fully exploit the identity-level information, which not only introduces the identification information into both image and language descriptions but also encourages the cross-modal classification probabilities of the same identity to be more similar. The experimental results on the CUHK-PEDES dataset demonstrate that our method achieves significantly better performance than other state-of-the-art algorithms.

This paper presents a novel tracking algorithm based on the convex hull representation model with sparse representation. The tracked object is assumed to be within the object convex hull and the candidate convex hull in the meanwhile. The object convex hull consists of a principle component analysis (PCA) subspace, and the candidate convex hull is constructed by all candidate samples with the sparsity constraint. Then we propose the objective function for our convex hull representation model, and design an iterative algorithm to solve it effectively. Finally, we present a tracking framework based on the proposed convex hull model and a simple online update scheme. Both qualitative and quantitative evaluations on some challenging video clips show that our tracker achieves better performance than other state-of-theart methods.

In this letter, we propose a novel hyperspectral image (HSI) classification method based on the joint collaborative representation (JCR) and support vector machine (SVM) models with decision fusion. First, motivated by the joint model, we adopt a JCR model to deal with HSI classification and develop an effective method to learn contextual basis vectors for the JCR model. Second, the mid-features are first extracted based on representation coefficients obtained by the JCR method and then used to train a multiclass SVM classifier. After that, we exploit a multiplicative fusion rule to combine the JCR and SVM models. We conduct numerous experiments to evaluate our method in comparison with other algorithms. The experimental results on three standard data sets demonstrate that our method achieves better performance than other competing ones.

In this paper, we propose a novel weighted local cosine similarity (WLCS) and apply it to visual tracking. First, we present the local cosine similarity to measure the similarities between the target template and candidates, and provide some theoretical insights on it. Second, we develop an objective function to model the discriminative ability of local components, and use a quadratic programming method to solve the objective function and to obtain the discriminative weights. Finally, we design an effective and efficient tracker based on the WLCS method and a simple update manner within the particle filter framework. Experimental results on several challenging image sequences show that the proposed tracker achieves better performance than other competing methods.

This paper presents a novel visual tracking method based on linear representation. First, we present a novel probability continuous outlier model (PCOM) to depict the continuous outliers within the linear representation model. In the proposed model, the element of the noisy observation sample can be either represented by a principle component analysis subspace with small Guassian noise or treated as an arbitrary value with a uniform prior, in which a simple Markov random field model is adopted to exploit the spatial consistency information among outliers (or inliners). Then, we derive the objective function of the PCOM method from the perspective of probability theory. The objective function can be solved iteratively by using the outlier-free least squares and standard max-flow/min-cut steps. Finally, for visual tracking, we develop an effective observation likelihood function based on the proposed PCOM method and background information, and design a simple update scheme. Both qualitative and quantitative evaluations demonstrate that our tracker achieves considerable performance in terms of both accuracy and speed.

In this letter, we present a novel online tracking method based on sparse representation. In contrast to existing “sparse representation”-based tracking algorithms, this work adopts the sparse representation method to construct both object and state models. The tracked object can be sparsely represented by a series of object templates, and also can be sparsely represented by candidate samples in the current frame. Furthermore, we propose a unified objective function to integrate object and state models, and cast the tracking problem as an optimization problem that can be solved in an iteration manner. Finally, we compare the proposed tracker with nine state-of-the-art tracking methods by using some challenging image sequences. Both qualitative and quantitative evaluations demonstrate that our tracker achieves favorable performance in terms of both accuracy and speed.