UAV surveillance offers a unique aerial perspective, enabling the monitoring of large areas and capturing targets from angles that fixed ground cameras cannot achieve. UAV-based object re-identification (ReID) differs from the extensively studied city camera scenarios, as it involves identifying specific objects in aerial images captured from a dynamic bird’s-eye view. The challenge lies in the significant variation in object perspectives and the often uncertain rotational changes captured by UAVs. Existing ReID methods designed for city cameras struggle to adapt to these rotational variations. To address these challenges, we propose a Transformer-based learnable rotation generalization enhancement method specifically for UAV-based ReID. To improve the model’s adaptability to uncertain rotational changes, we introduce a learnable feature-level rotation simulation technique that generates multiple rotated features. Building on this, we design a rotation diversification loss to decorrelate different rotated features, ensuring a rich feature representation. Additionally, to mitigate the negative effects of image-level rotation augmentation, we propose instance-level and distribution-level rotation invariance regularization. This approach establishes explicit associations between images and their rotated counterparts, facilitating the learning of visually consistent rotation-invariant features. Instance-level constraints ensure that detailed features remain consistent during rotation, while distribution-level constraints maintain the model’s semantic understanding. Notably, our method demonstrates strong versatility, covering a wide range of objects, including persons, vehicles, and various animals. Evaluations on multiple UAV-collected person and vehicle ReID datasets, as well as several animal datasets, consistently show outstanding performance, underscoring its robustness and adaptability to the unique challenges posed by UAV-based ReID.

Unsupervised object re-identification (Re-ID) aims to learn discriminative features without identity annotations. Existing mainstream methods are usually developed based on convolutional neural networks for feature extraction and pseudo-label estimation. However, convolutional neural networks suffer from limitations in capturing dispersed long-range dependencies and integrating global information. In comparison, vision transformers demonstrate superior robustness in complex environments, leveraging their versatile modeling capabilities to process diverse data structures with greater precision. In this paper, we delve into the potential of vision transformers in unsupervised Re-ID, proposing a Transformer-based perception-assisted framework (PAT). Considering Re-ID is a typical fine-grained task, existing unsupervised Re-ID methods relying on pseudo-labels generated by clustering algorithms provide only category-level discriminative supervision, with limited attention to local details. Therefore, we propose a novel target-aware mask alignment (TMA) strategy that provides additional supervision signals by leveraging low-level visual cues. Specifically, we employ pseudo-labels to guide the fine-grained alignment of features with local pixel information from critical discriminative regions. This method establishes a mutual learning mechanism via a shared Transformer, effectively balancing discriminative learning and detailed understanding. Furthermore, we propose a perceptual fusion feature augmentation (PFA) method to optimize instance-level discriminative learning. The proposed method is evaluated on multiple Re-ID datasets, demonstrating superior performance and robustness in comparison to state-of-the-art techniques. Notably, without annotations, our method achieves better results than many supervised counterparts. The code will be released.

Recognizing a target of interest from the UAVs is much more challenging than the existing object re-identification tasks across multiple city cameras. The images taken by the UAVs usually suffer from significant size difference when generating the object bounding boxes and uncertain rotation variations. Existing methods are usually designed for city cameras, incapable of handing the rotation issue in UAV scenarios. A straightforward solution is to perform the image-level rotation augmentation, but it would cause loss of useful information when inputting the powerful vision transformer as patches. This motivates us to simulate the rotation operation at the patch feature level, proposing a novel rotation invariant vision transformer (RotTrans). This strategy builds on high-level features with the help of the specificity of the vision transformer structure, which enhances the robustness against large rotation differences. In addition, we design invariance constraint to establish the relationship between the original feature and the rotated features, achieving stronger rotation invariance. Our proposed transformer tested on the latest UAV datasets greatly outperforms the current state-of-the-arts, which is 5.9% and 4.8% higher than the highest mAP and Rank1. Notably, our model also performs competitively for the person re-identification task on traditional city cameras. In particular, our solution wins the first place in the UAV-based person re-recognition track in the Multi-Modal Video Reasoning and Analyzing Competition held in ICCV 2021. Code is available at https://github.com/whucsy/RotTrans.