Transformer architecture has demonstrated significant potential in hyperspectral object tracking by leveraging global correlation learning to accurately represent the data distribution. However, existing hyperspectral object trackers based on transformer models typically rely on costly pre-trained models, making them prone to crashing due to overfitting when tuned on small-scale hyperspectral videos, greatly limiting their performance. To address this challenge, in this paper, a Hybrid Transformer with Adaptive Content and Position Embedding (HTACPE) tracker is proposed to improve the learning efficiency of the tracking model, and fully explore the spectral-spatial information. Specifically, an Adaptive Content and Position Embedding Module (ACPEM) is designed to dynamically learn the balance between focusing on positional and content-based information, which allows the model to effectively handle datasets of various sizes. To enhance the spectral-spatial information, a Spectral Grouping Module (SGM) is designed to learn the highfrequency information in complex scenarios, thereby enhancing diversified features. It operates in parallel with the ACPEM feature learning module. Furthermore, a Dynamic Reliability Refinement Module (DRRM) is incorporated to address challenges related to accurate object position perception, iteratively refining prediction parameters to enhance the reliability of the model. Extensive experiments demonstrate that the proposed HTACPE achieves satisfactory tracking performance both qualitatively and quantitatively, especially with insufficient training data.

Deep Neural Networks (DNNs) have demonstrated remarkable effectiveness in remote sensing (RS) image processing. However, they remain vulnerable to adversarial examples, which are generated by adding tiny but purposeful perturbations to clean examples. Such vulnerabilities in critical applications like environmental monitoring and urban planning can lead to significant negative consequences. To mitigate the interference of adversarial examples on DNNs, in this paper, a feature decoupling based adversarial examples detection (FD-AED) method for RS images is proposed, where non-robust features are employed in the detection process. Specifically, a loss function is designed for the de-coupler to disentangle the features into robust and non-robust features. Non-robust features are particularly useful because they often contain subtle clues that distinguish between clean and adversarial examples. By focusing on these non-robust features, the adversarial example detector can more effectively capture the differences between clean and adversarial examples. Experimental results indicate that the proposed FD-AED method effectively decouples robust and non-robust features, achieving more precise and reliable detection of adversarial examples.

Image-to-image (I2I) translation methods based on Generative Adversarial Networks (GANs) have shown general solutions for aerial visible-to-infrared image translation (AVIT) task. Though existing approaches have made impressive results, they still struggle to produce diverse or high-realism translated aerial infrared images (AIIs). In this paper, a novel model is proposed to achieve both diverse and realistic AVIT, named DR-AVIT. Specifically, we introduce disentangled representation learning to disentangle the image representation of aerial visible images (AVIs) and AIIs into a domain-invariant semantic structure space and two domain-specific imaging style spaces. By leveraging this disentanglement, our model can perform the translation process conditioned on semantic structure information derived from the input AVI and randomly sampled imaging style features from the AII domain to obtain diverse outputs. Furthermore, a new constraint is present to encourage GANs to learn efficient mappings between AVI and AII domains by integrating geometry-consistency constraint and a dual learning framework, named dual geometry-consistency constraint. Coping with these two designs, our method exhibits superiority in both realism and diversity of the translation results over several state-of-the-art I2I translation methods on AVIID dataset and two new benchmark datasets for AVIT, which are obtained by extracting data from publicly available datasets. Code of DR-AVIT and proposed benchmark datasets are available at https://github.com/silver-hzh/DR-AVIT.

Deep Neural Networks (DNNs) have been extensively utilized in aerial detection. However, DNNs are susceptible and vulnerable to adversarial examples Recently, physical attacks have gradually garnered attention due to their effectiveness and practicality, which pose great threats to some security-critical applications. In this paper, we take the first attempt to perform physical attacks in contextual form against aerial detection in the physical world. We propose an innovative contextual attack method against aerial detection in real scenarios, which achieves powerful attack performance and transfers well between various aerial object detectors without smearing or blocking the interested objects. Based on the findings that the targets’ contextual information plays an important role in aerial detection by observing the detectors’ attention maps, we fully use the contextual feature of the interested targets to elaborate background perturbations for the uncovered attacks in physical scenarios. Experiments with proportional scaling are conducted to evaluate the effectiveness of the proposed method, demonstrating its superiority in terms of both attack efficacy and physical practicality.

Patch-based physical attacks have increasingly aroused concerns. However, most existing methods focus on obscuring targets captured on the ground, and some of these methods are simply extended to deceive aerial detectors. They smear the targeted objects in the physical world with the elaborated adversarial patches, which can only slightly sway the aerial detectors’ prediction and with weak attack transferability. To address the above issues, a novel Contextual Background Attack (CBA) framework is proposed to fool aerial detectors in the physical world, which can achieve strong attack efficacy and transferability in real-world scenarios even without smudging the interested objects at all. Specifically, the targets of interest, i.e. the aircraft in aerial images, are adopted to mask adversarial patches. The pixels outside the mask area are optimized to make the generated adversarial patches closely cover the critical contextual background area for detection, which contributes to gifting adversarial patches with more robust and transferable attack potency in the real world. To further strengthen the attack performance, the adversarial patches are forced to be outside targets during training, by which the detected objects of interest, both on and outside patches, benefit the accumulation of attack efficacy. Consequently, the sophisticatedly designed patches are gifted with solid fooling efficacy against objects both on and outside the adversarial patches simultaneously. Extensive proportionally scaled experiments are performed in physical scenarios, demonstrating the superiority and potential of the proposed framework for physical attacks. We expect that the proposed physical attack method will serve as a benchmark for assessing the adversarial robustness of diverse aerial detectors and defense methods. The code has been released at https://github.com/JiaweiLian/CBA.

Despite deep neural networks (DNNs) have been widely applied in remote sensing (RS) scene classification and achieved satisfying performance, the vulnerability of DNNs towards adversarial examples significantly degrades their performance. Moreover, the relatively limited labeled samples of RS scene classification make DNNs more likely to overfit, leading to weak generalizability and noise sensitivity. This may result in DNNs being more vulnerable to adversarial examples. Consequently, the defense of adversarial examples is of crucial importance to improve both the generalizability and robustness of DNNs in the RS scene classification task. However, few studies have been conducted on defense for RS scene classification, especially ignoring the intrinsic characteristics of RS images. In this paper, an effective defense framework for RS scene classification, named reconstruction-assisted and distance-optimized adversarial training (RDAT), is proposed to defend adversarial examples. In order to solve the problems caused by high interclass similarity, a distance-optimized (DO) strategy is designed for adversarial training to strengthen the learning of underfitting content, increase the interclass distance, and improve the robustness of the networks. Furthermore, in order to generate high quality samples for adversarial training, a reconstruction-assisted (RA) block is proposed to eliminate adversarial perturbations in adversarial examples. Specifically, in this block, by swin transformer (SwinT) block and multi-scale convolution (MSC) block, SwinT-MSC-UNet (SMUNet) is constructed to fully extract global and multi-scale local features to adapt to the characteristics of RS images with large variance of ground object scales. Extensive experiments on the benchmark datasets, i.e., UC Merced (UCM) and Aerial Image Dataset (AID), have demonstrate that the proposed RDAT can effectively resist multiple adversarial attacks and yield superior results than other defense methods for RS scene classification.

Hyperspectral videos, which provide extra spectral characteristics besides spatial and temporal information, can improve the performance of object tracking using spectral signatures. However, there is a lack of labeled hyperspectral videos to support deep learning based model design. On the contrary, object tracking in the color space has been well developed in the past decade with many benchmark tracking models, e.g., SiamBAN. Therefore, how to transfer models designed in the color space to the hyperspectral space is of great importance. In this paper, hyperspectral videos are reduced into 3 bands using a band reduction algorithm, by which the existing well-trained trackers can be directly used. Specifically, Gaussian Information Entropy (GIE) is used to transform a hyperspectral video into a 3-band pseudo-color video, by which hyperspectral object tracking is conducted in an unsupervised mode. Experimental results demonstrate that object trackers designed in the color space can be transferred to hyperspectral videos using band reduction algorithms and the GIE based reduction is more effective than several well-known band reduction algorithms when using SiamBAN.