Remote sensing images usually reveal various objects with complex structures and different locations within vast ground area backgrounds. That leads to a major challenge for conventional generative models in handling remote sensing objects with correct shapes and clear textures. Integrating additional object-level controls can be a potential solution to improve generation quality, yet previous approaches inject the object-related conditions by specifying their locations, causing a limitation in object layout in generated results. To enable high object fidelity, high layout diversity and object customizable generation for remote sensing images, we propose a remote sensing image generation via object text decoupling, namely OTD-GAN. OTD-GAN takes advantage of the inherent text-to-image generation procedure and adaptively integrates the decoupled textual representations of visual objects into the global captions, thus achieving object-level controls without layout restrictions. Specifically, we design an object text decoupling module to predict a semantically consistent textual representation for each object. By decoupling the textual representation into a class invariant part and an object specific part, the converted representation is able to catch general semantic for similar objects as well as differentiated details for individual objects. After that, we use an object text semantic enhancement module to fuse the obtained object text representations with the global captions to enrich the object-related semantic within the textual modality. As a result, the generator will benefit from the object conditions and reinforce the generation quality while remaining flexibility to create diverse layouts. Extensive experiments on remote sensing image-caption datasets including NWPU-Captions and RSICD demonstrate that our method achieves leading performance compared to existing state-of-the-art approaches.

Inadequate remote sensing image training data usually makes remote sensing image classification models achieve low accuracy. Thus, we propose a diverse text-prompt generation learning (DPL) method. The context optimization (CoOp) model transfers the feature extraction capabilities of the CLIP model to downstream tasks with learnable text prompts. However, the small number of samples in remote sensing images can easily lead to noise. In contrast, DPL introduces a diverse text-prompt generation structure. Due to the diversity of multiple prompts, noise generated by inadequate samples is suppressed. Moreover, in order to keep the prompts diverse, we propose a prompt diversity loss. This loss pulls the prompts away from each other, which suppresses the noise generated by the limited training samples. Extensive experiments show that our method achieves superior performance than the existing methods on DOTA, HRRSD, and NWPU VHR-10 datasets. The model and code are available athttps://github.com/LvXiangzhu/DPL.

Low-resolution characteristics of multispectral images restrict their usability. Various approaches (e.g., single-image super-resolution reconstruction (SISR) and pansharpening method) have been proposed to enrich the spatial details of low-resolution multispectral images (LRMSs) to obtain high-resolution ones. While the pansharpening method inevitably depends on panchromatic (PAN) images, which limits its application scenarios, SISR does not need auxiliary images, yet the blurred edge details within the reconstructed super-resolution multispectral (SRMS) images still remain a big challenge. In this work, we propose a novel high-frequency feature transfer (HFFT-PAN) method for multispectral image super-resolution to tackle the above drawbacks. Specifically, we first exploit the inherent low-frequency features among LRMS images to facilitate the extraction of high-frequency features from PAN images. After that, the high-frequency features from PAN images are transferred to the reconstruction procedure of multispectral images so that the SRMS images can not only benefit from the edge detail information from PAN images but also avoid the utilization of any PAN images during inference. Moreover, we employ an additional contrastive loss during training to ensure the fidelity of the generated SRMS images. Qualitative and quantitative evaluations exhibit that the proposed method performs favorably against state-of-the-art methods. The model and code are available athttps://github.com/wx0110wx/HFFT.

General deep learning-based methods for infrared and visible image fusion rely on the unsupervised mechanism for vital information retention by utilizing elaborately designed loss functions. However, the unsupervised mechanism depends on a well-designed loss function, which cannot guarantee that all vital information of source images is sufficiently extracted. In this work, we propose a novel interactive feature embedding in a self-supervised learning framework for infrared and visible image fusion, attempting to overcome the issue of vital information degradation. With the help of a self-supervised learning framework, hierarchical representations of source images can be efficiently extracted. In particular, interactive feature embedding models are tactfully designed to build a bridge between self-supervised learning and infrared and visible image fusion learning, achieving vital information retention. Qualitative and quantitative evaluations exhibit that the proposed method performs favorably against state-of-the-art methods.

Fusing infrared and visible images can provide more texture details for subsequent object detection task. Conversely, detection task furnishes object semantic information to improve the infrared and visible image fusion. Thus, a joint fusion and detection learning to use their mutual promotion is attracting more attention. However, the feature gap between these two different-level tasks hinders the progress. Addressing this issue, this paper proposes an infrared and visible image fusion via meta-feature embedding from object detection. The core idea is that meta-feature embedding model is designed to generate object semantic features according to fusion network ability, and thus the semantic features are naturally compatible with fusion features. It is optimized by simulating a meta learning. Moreover, we further implement a mutual promotion learning between fusion and detection tasks to improve their performances. Comprehensive experiments on three public datasets demonstrate the effectiveness of our method. Code and model are available at: https://github.com/wdzhao123/MetaFusion.

Both salient object detection (SOD) and camouflaged object detection (COD) are typical object segmentation tasks. They are intuitively contradictory, but are intrinsically related. In this paper, we explore the relationship between SOD and COD, and then borrow successful SOD models to detect camouflaged objects to save the design cost of COD models. The core insight is that both SOD and COD leverage two aspects of information: object semantic representations for distinguishing object and background, and context attributes that decide object category. Specifically, we start by decoupling context attributes and object semantic representations from both SOD and COD datasets through designing a novel decoupling framework with triple measure constraints. Then, we transfer saliency context attributes to the camouflaged images through introducing an attribute transfer network. The generated weakly camouflaged images can bridge the context attribute gap between SOD and COD, thereby improving the SOD models' performances on COD datasets. Comprehensive experiments on three widely-used COD datasets verify the ability of the proposed method. Code and model are available at: https://github.com/wdzhao123/SAT.

Homogeneous regions, which are smooth areas that lack blur clues to discriminate if they are focused or non-focused. Therefore, they bring a great challenge to achieve high accurate multi-focus image fusion (MFIF). Fortunately, we observe that depth maps are highly related to focus and defocus, containing a preponderance of discriminative power to locate homogeneous regions. This offers the potential to provide additional depth cues to assist MFIF task. Taking depth cues into consideration, in this paper, we propose a new depth-distilled multi-focus image fusion framework, namely D2MFIF. In D2MFIF, depth-distilled model (DDM) is designed for adaptively transferring the depth knowledge into MFIF task, gradually improving MFIF performance. Moreover, multi-level fusion mechanism is designed to integrate multi-level decision maps from intermediate outputs for improving the final prediction. Visually and quantitatively experimental results demonstrate the superiority of our method over several state-of-the-art methods.

Remote sensing data distribution generally exposes the long-tail characteristic. This will limit the object recognition performance of existing deep models when they are trained with such unbalanced data. In this paper, we propose a novel hierarchical distillation framework to address the long-tailed object recognition in aerial images. Firstly, we notice that not only student model should learn feature representations from teachers, but also teacher models should learn feature representations from each other. Therefore, we build hierarchical teacher-wise distillation to improve the feature representations of the teacher models trained with middle and tail data, which is achieved by distilling the feature representations of the teacher model trained with head data. Secondly, we notice that the feature representations of the middle and tail classes can not be effectively distilled from the teacher to the student, since too little middle and tail data can be used to learn. Thus, we propose self-calibrated sampling learning that enforces the student to strengthen the learning of the middle and tail data, thereby improving the student’ feature learning ability. Extensive experiments on two widely-used DOTA and FGSC-23 datasets demonstrate superior performance of the proposed method compared with state-of-the-art methods. Model and code are publicly available at: https://github.com/wdzhao123/T2FTS.

Annotating remote sensing object recognition needs high professionalism, and thus limited labeled samples are available. Suffering from this, general remote sensing object recognition methods are facing low recognition accuracy. Addressing this issue, this paper proposes a diversity consistency learning for remote sensing object recognition with limited labels. Specifically, diversity generation model is designed as a teacher model to generate diverse results, which is trained with labeled samples. Then, round consistency distillation model is introduced to distill the knowledge of diverse pseudo labels to a student network, which is trained with unlabeled samples. Especially, diverse pseudo labels are generated by the well-trained diversity generation model, which can improve recognition accuracy since diverse pseudo label errors can cancel each other out. Extensive experiments on two widely-used datasets of FS23 and HRSC2016 demonstrate the superior performance of our method compared with the state of the arts.