Most existing visual tracking methods typically employ image patches as target references and endeavor to enhance tracking performance by maximizing the utilization of visual information through various deep networks. However, due to the intrinsic limitations of visual information, the performance of the trackers significantly deteriorates when confronted with drastic target variations or complex background environments. To address these issues, we propose a vision-language multimodal fusion tracker for object tracking. Firstly, we use semantic information from language descriptions to compensate for the instability of visual information, and establish multimodal cross-relations through the fusion of visual and language features. Secondly, we propose an attention-based token screening mechanism that utilizes semantic-guided attention and masking operations to eliminate irrelevant search tokens devoid of target information, thereby enhancing both accuracy and efficiency. Furthermore, we optimize the localization head by introducing channel attention, which effectively improves the accuracy of target positioning. Extensive experiments conducted on the OTB99, LaSOT, and TNL2K datasets demonstrate the effectiveness of our proposed tracking method, achieving success rates of 71.2%, 69.5%, and 58.9%, respectively. • We propose ABOTrack, a novel vision-language joint object tracker. • We propose an attention-based token screening mechanism to eliminate irrelevant target information. • The localization head is optimized by employing channel attention to adaptively fuse multi-scale features. • ABOTrack achieves 71.2% AUC on OTB99 and 69.5% AUC on LaSOT, surpassing existing methods.

Temporal action localization in long-term untrimmed videos remains a critical yet challenging task in video understanding, with existing methods often relying on anchor-based or fully-supervised frameworks that incur heavy computation and require labor-intensive frame-level annotations. This paper presents a novel weakly-supervised approach, CL-WTAL, which leverages multi-scale contrast learning and graph convolution for accurate action localization and recognition. The method comprises three key components: (i) A multi-scale sliding window mechanism (long/normal/short sequences) to segment sub-actions from complex videos, adapting to diverse action durations; (ii) A spatio-temporal graph convolution network (ST-RGCN) to extract skeletal feature vectors, integrating human motion dynamics and environmental context; (iii) A contrastive learning-based similarity evaluation framework that combines cosine similarity and Dynamic Time Warping (DTW) distance to measure feature vector relationships, enabling precise action boundary detection without extensive fine-tuning. Experiments on daily-life video datasets demonstrate that CL-WTAL effectively localizes action intervals and classifies actions with high accuracy, outperforming state-of-the-art weakly-supervised methods.

Anchor-free based trackers introduce an extra branch in addition to classification and regression branches in the network to achieve comparable performance with anchor-based trackers. This extra branch is usually trained independently in the training phase and is used in combination with other branches in the inference phase. However, this can increase the inconsistency between the inference phase and the training phase, potentially degrading the tracking performance. To address this problem, we propose a new Siamese network-based object tracking framework that eliminates this inconsistency by unifying classification and additional branch tasks to achieve learning location quality estimation. Furthermore, regression tasks for bounding boxes are widely formulated based on Dirac$\delta $distribution. Though this assumption works well for many scenarios, it restricts the prediction of regression branches. To overcome this restriction, we propose discretizing the continuous offset of the regression branch into multiple offset predictions, which enables the network to learn more flexible distributions automatically. Meanwhile, the discrete distribution prediction of regression branches is utilized to further guide the classification of the trackers. Extensive experiments on the widely accepted benchmarks demonstrate the effectiveness and efficiency of the proposed model.

Visual contents, such as illustrations and images, play a big role in product manual understanding. Existing Product Manual Question Answering (PMQA) datasets tend to ignore visual contents and only retain textual parts. In this work, to emphasize the importance of multimodal contents, we propose a Multimodal Product Manual Question Answering (MPMQA) task. For each question, MPMQA requires the model not only to process multimodal contents but also to provide multimodal answers. To support MPMQA, a large-scale dataset PM209 is constructed with human annotations, which contains 209 product manuals from 27 well-known consumer electronic brands. Human annotations include 6 types of semantic regions for manual contents and 22,021 pairs of question and answer. Especially, each answer consists of a textual sentence and related visual regions from manuals. Taking into account the length of product manuals and the fact that a question is always related to a small number of pages, MPMQA can be naturally split into two subtasks: retrieving most related pages and then generating multimodal answers. We further propose a unified model that can perform these two subtasks all together and achieve comparable performance with multiple task-specific models. The PM209 dataset is available at https://github.com/AIM3-RUC/MPMQA.

Mare Nubium is one of the most ancient circular impact basins on the Moon with the lava flow units ranging from Imbrian to Eratosthenian. In this paper, the China Chang'E-2 lunar microwave radiometer data are employed to evaluate the basaltic units of the Mare Nubium. The results indicate that the relationship between the brightness temperature difference (dTB) performances and the basaltic units is rather weak. Moreover, the low dTB anomaly in the middle-western part is interpreted as the floor deposits produced during the formation of Mare Nubium.

The exiting covariance matching method is not suited for real-time applications due to its demand for exhaustive search. Aiming at this problem, we developed a novel approach based on fuzzy genetic algorithm (GA) to boost the computing efficiency of covariance matching. The approach employs GA in searching for optimal solution in a large image region. To avoid premature convergence or local optimum which often occur in traditional GAs, we use a fuzzy inference system to adaptively estimate the crossover and mutation probabilities to gain convergence in a much higher speed than using a conventional GA. Experimental results show that the proposed approach can significantly improve the processing speed of covariance matching, while keeping the matching results almost unchanged. The runtime performance of the proposed approach is faster than its counterparts using exhaustive search with eight times and more.