Synthetic aperture radar (SAR)-optical data fusion cloud removal is a highly promising cloud removal technology that has attracted considerable attention. In this field, primary researches are based on deep learning, which can be divided into two categories: convolutional neural network (CNN)-based and Transformer-based. In the cases of extensive cloud coverage, CNN-based methods, with their local spatial awareness, effectively capture local structural information in SAR data, preserving clear contours of cloud-removed land covers. However, these methods struggle to capture global land cover information in optical images, often resulting in notable color discrepancies between recovered and cloud-free regions. Conversely, Transformer-based methods, with their global modeling capability and inherent low-pass filtering properties, excel at capturing long-range spatial correlations in optical images, thereby maintaining color consistency across cloud-removal outputs. However, these methods are less effective at capturing the fine structural details in SAR data, which can lead to blurred local contours in the final cloud-removed images. In this context, a novel framework called heterogeneous parallel network for cloud removal (HPN-CR) is proposed to achieve high-quality cloud removal under extensive cloud coverage conditions. HPN-CR employs the proposed heterogeneous encoder with its SAR-optical input approach to effectively extract and fuse the local structural information in cloudy areas from SAR images, with the spectral information about land covers in cloud-free areas from the whole optical images. In particular, it uses a ResNet network with local spatial awareness to extract SAR features. It also uses the proposed Decloudformer, which globally models multiscale spatial correlations, to extract optical features. The output features are fused by the heterogeneous encoder and then reconstructed to cloud-removal images through a pixelshuffle-based decoder. Comprehensive experiments were conducted and the experimental results demonstrated the effectiveness and superiority of the proposed method. The code is available athttps://github.com/G-pz/HPN-CR.

Semantic change detection (SCD) involves detecting changed regions and classifying their corresponding semantic change categories in remote sensing images. However, in most SCD scenarios, only the land-cover category information of the changed regions is provided. The mainstream multi-task Siamese network optimization process fails to fully leverage the information from unchanged land-cover types, which limits its ability to recognize land-cover change types and ultimately limits SCD performance. To address the issue of suppressed change type recognition caused by sparse labels in the SCD task, this paper proposes a confusion-aware contrastive-based semi-supervised semantic change detection method through space collaboration (SC2A-SCD). The proposed SC2A-SCD framework first integrates bi-temporal land-cover classification (LCC)-predicted logits from both the classification and representation spaces using joint classification and representation space modeling (JSM), providing high-quality pseudo-labels for unchanged land-cover types and enhancing the model’s ability to recognize bi-temporal land-cover types. Meanwhile, confusion-aware contrastive learning (CACL) is conducted in the representation space, utilizing confusion sampling strategies to sample more confusable anchors and negatives that are prone to misclassification, thereby effectively improving the representation capability of bi-temporal land-cover types and enhancing the quality of the pseudo-labels obtained via JSM, ultimately boosting the ability of SC2A-SCD to recognize semantic changes. Compared to the state-of-the-art SCD methods, the comprehensive experimental results confirm that the proposed SC2A-SCD framework can effectively improve the recognition ability for change types, demonstrating its effectiveness in enhancing SCD performance.

At present, most remote sensing image change detection methods are applicable to equal-resolution scenarios, that is, the bi-temporal images are assumed to have the same spatial resolution. Real-world tasks such as disaster emergency response have put forward the need for change detection of bi-temporal images with different spatial resolutions, that is, cross-resolution change detection. However, due to the significant differences in spatial details between high-resolution images and low-resolution images, it is difficult to extract the spatial features of changed landcovers in complex scenarios and distinguish between changed landcovers and unchanged landcovers. To overcome the above issues, we propose a Resolution-Difference Embedding Network (RDENet). RDENet combines two innovative methods: pseudo-continuous resolution sequence representation (PRSR) and bi-temporal resolution-difference modulation (BRDM). The PRSR method effectively extracts features of landcovers in complex scenarios by constructing pseudo image sequence with smooth transition of resolution and developing a resolution-guided feature fusion module. The BRDM method significantly enhances the model’s ability to distinguish between changed and unchanged landcovers in complex scenarios through the design of a resolution-aware self-modulation module and resolution-aware mutual-modulation module, which utilizes the resolution-difference factor as prior knowledge to dynamically enhance key features of changed landcovers in cross-resolution image pairs. Extensive experiments conducted on three publicly available change detection datasets demonstrate that the proposed RDENet achieves superior detection performance in cross-resolution scenarios.

Multimodal remote sensing (RS) image land use/land cover (LULC) classification using optical and synthetic aperture radar (SAR) images has raised attention for recent studies. Current methods primarily employ multimodal fusion operations to directly explore relationships between multimodal features and obtain fused features, leading to the loss of beneficial modality-specific information problem. To solve this problem, this study introduces a multimodal feature decomposition and fusion (MDF) approach combined with a visual state space (VSS) block, namely MDF-VSS block. The MDF-VSS block emphasizes beneficial modality-specific information and perceives shared land cover information through modality-difference and modality-share features, which are then adaptively integrated to obtain discriminative fused features. Based on the MDF-VSS block, an MDF decoder is designed to retain beneficial multi-scale modality-specific information. Then, a multimodal specific information enhancement (MSIE) decoder is designed to perform modality-difference feature guided auxiliary classification tasks, further enhancing modality-specific information that is expert in classification. Combining the MDF and MSIE decoders, a novel retain-enhance fusion network (REF-Net) is proposed to retain and enhance modality-specific information that benefits classification, thus improving the performance of multimodal RS image LULC classification. Extensive experimental results obtained on three public datasets demonstrate the effectiveness of the proposed REF-Net. The source code will be available at https://github.com/TINYWAI/REF_Net.

Semantic Change Detection (SCD) is recognized as both a crucial and challenging task in the field of image analysis. Traditional methods for SCD have predominantly relied on the comparison of image pairs. However, this approach is significantly hindered by substantial imaging differences, which arise due to variations in shooting times, atmospheric conditions, and angles. Such discrepancies lead to two primary issues: the under-detection of minor yet significant changes, and the generation of false alarms due to temporal variances. These factors often result in unchanged objects appearing markedly different in multi-temporal images. In response to these challenges, the MapChange framework has been developed. This framework introduces a novel paradigm that synergizes temporal-invariant historical map data with contemporary high-resolution images. By employing this combination, the temporal variance inherent in conventional image pair comparisons is effectively mitigated. The efficacy of the MapChange framework has been empirically validated through comprehensive testing on two public datasets. These tests have demonstrated the framework's marked superiority over existing state-of-the-art SCD methods.

At present, deep learning (DL) methods for remote sensing (RS) change detection (CD) are developing rapidly. However, there are still many challenges in complex RS scenarios. Factors, such as season and illumination, contribute to minimal differences in radiation characteristics between the changed area and the background, making them difficult to distinguish. This letter proposes an axial-shift feature interaction and prototype-guided penalty constraint network (ASPGNet) to address this problem. ASPGNet integrates axial-shift feature interaction (ASFI) module and prototype-guided penalty constraint (PGPC) loss. The ASFI module facilitates interaction among adjacent features through axial-shift operations in the width/height directions, aiming to obtain discriminative feature representations of the changed area. The PGPC loss utilizes prototypes to adaptively identify and weigh confusing pixel features, ensuring distinguishability between change and nonchange features and, thereby, generating accurate CD results. We evaluate the proposed method on the WHU-CD and LEVIR-CD datasets, achieving the$F1$scores of 93.22% and 91.49%, respectively. These results demonstrate the effectiveness of the proposed method.

Scene change detection (SCD) involves recognizing the high-level semantic change types for a bitemporal remote sensing image scene pair. Previous methods have implicitly considered joint bitemporal features as a change representation to perform SCD. However, they have not effectively enhanced the discriminative power of the change representation. Consequently, these methods struggle to recognize scene changes with both intraclass variation and interclass similarity. In this paper, scene change contrastive (SCC) learning based on contrastive learning is proposed to ensure that bitemporal features are discriminative change representations for SCD recognition. Contrastive learning can learn specific discriminative features by gathering predefined specific positives and separating negatives in the projection space. In the SCC learning, the change representations, which are represented by the joint bitemporal features, are mapped to real and pseudo multi-view change projections by the proposed multi-view change (MVC) projector and the pseudo change augmentation (PeCA) strategy. The change projections are then guided to be discriminative, exploiting both local spatial and global information. By doing so, the joint bitemporal scene features become more discriminative change representations, which enable accurate recognition of scene changes. The extensive experimental results obtained on two public datasets consistently demonstrate the effectiveness of the proposed method. The code is available at https://github.com/wdczs/SCC.

Semantic change detection (SCD) for high-resolution remote sensing imagery involves simultaneously locating the changed regions and identifying the semantic change categories. Recently, a series of multitask Siamese networks have been proposed to model the SCD task by merging the binary change detection (BCD) results and the bitemporal land-cover classification (LCC) results. However, due to the large reflectance variability of the land cover in bitemporal high-resolution images, the land-cover feature clusters extracted by these methods are still considerably mixed, leading to the misidentification of change and their semantic changes types. In this article, to handle this problem, the SiamContrast method is proposed to learn temporally invariant discriminative land-cover features for SCD. As part of SiamContrast, a novel SCD contrastive loss (SCD-CL) is proposed to enhance the temporally invariant feature discrimination across bitemporal images. SCD-CL utilizes supervised contrastive learning and consists of two complementary components: mono-temporal contrastive loss (MCL) and cross-temporal contrastive loss (CCL). In particular, MCL contrasts the land-cover features within each temporal image, to enhance the mono-temporal feature discrimination. Meanwhile, CCL with a change-aware hard anchor sampling (CHAS) strategy contrasts the land-cover features across bitemporal images, to align the land cover features of the same category. To validate the effectiveness of SCD-CL, the SiamContrast method incorporates a difference feature pyramid (DFP) decoder, which leverages feature distance to model changes, allowing it to directly benefit from the discriminative features learned by SCD-CL. The comprehensive experimental results consistently confirm the effectiveness of the proposed SiamContrast in improving SCD performance, compared with the existing SCD methods. The code is available athttps://github.com/wdczs/SiamContrast.

Change detection (CD) is a challenging task on high-resolution bitemporal remote sensing images. Many recent studies of CD have focused on designing fully convolutional Siamese network architectures. However, most of these methods initialize their encoders by random values or an ImageNet pretrained model, without any prior for the CD task, thus limiting the performance of the CD model. In this article, the novel supervised contrastive pretraining and fine-tuning CD (SCPFCD) framework, which is made up of two cascaded stages, is presented to train a CD network based on a pretrained encoder. In the first supervised contrastive pretraining stage, the encoder of the Siamese network is asked to solve a joint pretext task introduced by the proposed CDContrast pretraining method on labeled CD data. The proposed CDContrast pretraining method includes land contrastive learning (LCL), which is based on supervised contrastive learning, and proxy CD learning. The LCL focuses on learning the spatial relationships among the land cover from bitemporal images by solving a land contrast task, while the proxy CD learning performs a proxy CD task on the top of the upsampling projector to avoid local optima for the LCL and learn features for the CD. Then, in the second fine-tuning stage, the whole Siamese network initialized with the pretrained encoder is fine-tuned to perform the CD task in an end-to-end manner. The proposed SCPFCD framework was verified with three CD datasets of high-resolution remote sensing images. The extensive experimental results consistently show that the proposed framework can effectively improve the ability to extract change information for Siamese networks.

In this paper, a new semantic change detection (CD) dataset based on Chinese Gaofen-2 (GF-2) satellite images with high spatial resolution (HSR) namely Wuhan Urban Semantic Understanding (WUSU) dataset is built up and a CD framework combining binary and semantic CD tasks based on deep learning and a conditional random field model (SDCRF) is proposed. Existing CD datasets mostly focus on “change/no change”. Traditional CD methods pay attention only on either of the binary CD task or the semantic CD task. Although there are methods to handle both tasks simultaneously but they ignore the inconsistency between the two tasks. In the SDCRF framework, any state-of-the-art feature extraction model can be used to extract the class and change probabilities as the unary potential of a fully connected conditional random field (FC-CRF) model which is adopted as a post-processing to enhance the location information of deep networks and reduce outlier noise.

The power of features considerably influences the classification performance of remote sensing scene classification (RSSC). Recently, deep convolutional neural networks (DCNNs) have been used to extract powerful scene features. Nevertheless, confusion and overlap still occur in the feature space, leading to inaccurate RSSC. To alleviate this problem, we propose a novel deep metric learning loss function incorporated into a sphere loss to enhance the discrimination of feature representations. Inspired by two representative loss functions (i.e., angular loss and center loss), the proposed sphere loss learns a unique cluster center for each class in a remote sensing scene. Because the cluster centers and features are restricted by an introduced geometrical constraint, the intraclass distance of features decreases, while the interclass distance increases. Moreover, we introduce a spatial constraint, i.e., a uniformity coefficient on different cluster centers, which causes the centers to form a uniform distribution that maximizes the interclass distances between features. Extensive analysis and experiments on three commonly used RSSC data sets consistently show that, compared with state-of-the-art methods, the proposed sphere loss can effectively learn discriminative feature representations and significantly improve RSSC.

Recently, remote sensing image scene classification technology has been widely applied in many applicable industries. As a result, several remote sensing image scene classification frameworks have been proposed; in particular, those based on deep convolutional neural networks have received considerable attention. However, most of these methods have performance limitations when analyzing images with large intraclass variations. To overcome this limitation, this letter presents the marginal center loss with an adaptive margin. The marginal center loss separates hard samples and enhances the contributions of hard samples to minimize the variations in features of the same class. Experimental results on public remote sensing image scene data sets demonstrate the effectiveness of our method. After the model is trained using the marginal center loss, the variations in the features of the same class are reduced. Furthermore, a comparison with state-of-the-art methods proves that our model has competitive performance in the field of remote sensing image scene classification.

Automatical inshore ship change detection in optical remote sensing images is meaningful for a wide range of applications, but still a challenging task. In this paper, a visual search inspired model is proposed for inshore ship change detection. The proposed model achieves inshore ship change information extraction based on spatial-temporal saliency detection which mimics visual search mechanism. Experimental results performed on a Google Earth optical remote sensing image data set demonstrate the effectiveness of the proposed model in terms of visual and objective evaluations.

Object detection in complex optical remote sensing images is a challenging problem due to the wide variety of scales, densities, and shapes of object instances on the earth surface. In this letter, we focus on the wide-scale variation problem of multiclass object detection and propose an effective object detection framework in remote sensing images based on YOLOv2. To make the model adaptable to multiscale object detection, we design a network that concatenates feature maps from layers of different depths and adopt a feature introducing strategy based on oriented response dilated convolution. Through this strategy, the performance for small-scale object detection is improved without losing the performance for large-scale object detection. Compared to YOLOv2, the performance of the proposed framework tested in the DOTA (a large-scale data set for object detection in aerial images) data set improves by 4.4% mean average precision without adding extra parameters. The proposed framework achieves real-time detection for$1024\times 1024$image using Titan Xp GPU acceleration.11https://github.com/WenchaoliuMUC/Detection-of-Multiclass-Objects-in-Optical-Remote-Sensing-Images

In recent times, many efforts have been made to improve remote sensing image scene classification, especially using popular deep convolutional neural networks. However, most of these methods do not consider the specific scene orientation of the remote sensing images. In this letter, we propose the improved oriented response network (IORN), which is based on the ORN, to handle the orientation problem in remote sensing image scene classification. We propose average active rotating filters (A-ARFs) in the IORN. While IORNs are being trained, A-ARFs are updated by a method that is different from the ARFs of the ORN, without additional computations. This change helps IORN improve its ability to encode orientation information and speeds up optimization during training. We also propose Squeeze-ORAlign (S-ORAlign) by adding a squeeze layer to ORAlign of ORN. With the squeeze layer, S-ORAlign can address large-scale images, unlike ORAlign. An ablation study and comparison experiments are designed on a public remote sensing image scene classification data set. The experimental results demonstrate the effectiveness and better performance of the proposed model over that of other state-of-the-art models.