Current multi-modal object re-identification approaches based on large-scale pre-trained backbones (i.e., ViT) have displayed remarkable progress and achieved excellent performance. However, these methods usually adopt the standard full fine-tuning paradigm, which requires the optimization of considerable backbone parameters, causing extensive computational and storage requirements. In this work, we propose an efficient prompt-tuning framework tailored for multi-modal object re-identification; dubbed DMPT, which freezes the main backbone and only optimizes several newly added decoupled modality-aware parameters. Specifically, we explicitly decouple the visual prompts into modality-specific prompts which leverage prior modality knowledge from a powerful text encoder and modality-independent semantic prompts which extract semantic information from multi-modal inputs, such as visible, near-infrared, and thermal-infrared. Built upon the extracted features, we further design a Prompt Inverse Bind (PromptIBind) strategy that employs bind prompts as a medium to connect the semantic prompt tokens of different modalities and facilitates the exchange of complementary multi-modal information, boosting final re-identification results. Experimental results on multiple common benchmarks demonstrate that our DMPT can achieve competitive results to existing state-of-the-art methods while requiring only 6.5% fine-tuning of the backbone parameters.

Current few-shot action recognition methods reach impressive performance by learning discriminative features for each video via episodic training and designing various temporal alignment strategies. Nevertheless, they are limited in that (a) learning individual features without considering the entire task may lose the most relevant information in the current episode, and (b) these alignment strategies may fail in misaligned instances. To overcome the two limitations, we propose a novel Hybrid Relation guided Set Matching (HyRSM) approach that incorporates two key components: hybrid relation module and set matching metric. The purpose of the hybrid relation module is to learn task-specific embeddings by fully exploiting associated relations within and cross videos in an episode. Built upon the task-specific features, we reformulate distance measure between query and support videos as a set matching problem and further design a bidirectional Mean Hausdorff Metric to improve the resilience to misaligned instances. By this means, the proposed HyRSM can be highly informative and flexible to predict query categories under the few-shot settings. We evaluate HyRSM on six challenging benchmarks, and the experimental results show its superiority over the state-of-the-art methods by a convincing margin. Project page: https://hyrsm-cvpr2022.github.io/.

Most recent approaches for online action detection tend to apply Recurrent Neural Network (RNN) to capture long-range temporal structure. However, RNN suffers from non-parallelism and gradient vanishing, hence it is hard to be optimized. In this paper, we propose a new encoder-decoder framework based on Transformers, named OadTR, to tackle these problems. The encoder attached with a task token aims to capture the relationships and global inter-actions between historical observations. The decoder extracts auxiliary information by aggregating anticipated future clip representations. Therefore, OadTR can recognize current actions by encoding historical information and predicting future context simultaneously. We extensively evaluate the proposed OadTR on three challenging datasets: HDD, TVSeries, and THUMOS14. The experimental results show that OadTR achieves higher training and inference speeds than current RNN based approaches, and significantly outperforms the state-of-the-art methods in terms of both mAP and mcAP. Code is available at https://github.com/wangxiang1230/OadTR.

In this paper a dim point target detection algorithm based on Markov Random Field (MRF) is proposed. Firstly, the Min-Difference Filter (MDF) is applied to suppress the background in each single frame. Then an adaptive segmentation process based on 3D MRF is used to detect targets in image sequence. Unlike the traditional 3D MRF, the spatial MRF is modified and a Point-to-Region mapping according to the specific situation is proposed. Experiment results show that the proposed algorithm allows to detect IR point targets in complex backgrounds.

This paper discusses the research in small target detection in infrared images with heavy clutter background. For most infrared images, ship objects are rather dim in the relative dark sea surface background. The existence of scan line disturbance and noise also increases the difficulty in proper detection. Dim objects must be distinguished from a dark background. On the other hand, the small targets must also be distinguished from clutters. Through analysis of the targets and background, we build characteristic models of small ship objects, noise and sea backgrounds respectively, and indicate their differences in spatial and frequency domains among them. Based on the principles of signal processing, pattern recognition and artificial intelligence, we propose a combined algorithm for detecting sea surface small targets. In this algorithm, components of background and noise are first suppressed by a multilevel filter designed accordingly, meanwhile enhancing the target ones of interest. The pixels of the candidate targets are then discriminated by minimum risk Bayes test. Finally, according to a priori knowledge about the targets such as the ranges of their sizes, the targets of interest can be detected. In particular, the related probability distributions used by statistic decision are obtained by offline learning of typical training samples. Experiments show that the algorithm is excellent for such kinds of target detection and is robust to noise.