Deep learning networks, such as convolutional neural networks (CNNs), are increasingly applied to synthetic aperture radar (SAR) feature representation and image classification. However, the performance of most deep learning methods relies on sufficient labeled data, which is difficult to collect for SAR images. As a result, self-supervised contrastive learning (CL) methods have attracted enthusiasm in recent studies to learn SAR representations with unlabeled data. Most existing CL-based methods for SAR representation learning simply use the amplitude information as the network input but neglect the particular features of complex-valued SAR images such as spectral information, leading to insufficient feature extraction. To address this issue, we propose a self-supervised sublook CL (SCL) method to learn spectral representations from unlabeled data. First, we extract sublooks from SAR images to establish a spectral representation branch (SRB) to discover the spectral information. Second, a novel SCL method with a sublook-matching task is proposed based on a contrastive network to learn spectral representations. The branch is applied to SAR image classification tasks by integrating it with a typical amplitude network through feature concatenation. Experimental results have validated the effectiveness and generalization ability of the proposed method with a different experimental protocol that distinguishes the pretrain and downstream datasets.

Radio frequency interference (RFI) is an important factor affecting microwave remote sensing observations, causing random degradation of synthetic aperture radar (SAR) images. Due to the huge amount of raw echo and single-look complex (SLC) data, there are some SAR interpretation scenarios when only amplitude images can be obtained, and traditional signal transformation and matrix operations can hardly meet the suppression requirements in the absence of phase information at this time. Although deep learning algorithms have made some progresses on this issue, they still suffer from the following limitations: (i) There are few models dedicated to SAR RFI in the image domain; (ii) The network structure is relatively complex due to not fully exploit the physical characteristics of SAR and RFI. To this end, we propose an end-to-end suppression network (PMNet), which includes a novel explainable feature decomposition module (FDM) based on the idea of non-negative matrix factorization and a composite loss function to achieve dynamic separation of foreground and background features of supervised RFI-contaminated images. The ablation experiment proves that compared with the baseline algorithm, the visual similarity of the proposed PMNet on the test set can be improved by up to 14.15%. The suppression result on Sentinel-1 and Gaofen-3 real data also verifies the effectiveness of the PMNet in different SAR platforms.

Most existing deep learning models for remote sensing scene classification (RSSC) adopt offline learning paradigm, which are trained on closed datasets and fail to dynamically update with new class data. Currently, class-incremental learning (CIL) allows models to learn new classes while retaining discrimination of old ones. However, most CIL approaches aim to overcome catastrophic forgetting by employing techniques such as exemplarmemory and knowledge distillation, while ignoring the prediction bias caused by imbalanced datasets, where old classes retain fewer samples than new ones. Moreover, they do not adequately account for the multilevel semantic structure and multiscale feature information inherent in remote sensing images (RSIs). To address these issues, we propose an effective CIL framework for RSSC, named class bias correction network (CBCNet). Specifically, a cross-dimensional and interaction-aware attention mechanism (CIAM) is designed to incorporate channel, position, and direction-aware information in feature maps, enabling the model to highlight informative regions within RSIs. Next, a contextual information fusion module (CIFM) is proposed to explore the correlations among multilevel features and enhance representation quality through their fusion. In addition, the designed taskwise classifier head decoupling mechanism (TCDM) imposes a constraint to mitigate the prediction bias toward new classes, and enhances model’s discrimination among all seen classes. Finally, a multilevel integrated knowledge distillation module (MKDM) is developed to ensure comprehensive knowledge transfer, empowering the model to maintain critical representations in feature space and make well-informed decisions in output probability space. Experiments on five open datasets demonstrate the outperformance and robustness of our method.

Change Detection (CD) plays a crucial role in remote sensing analysis. Semantic Change Detection (SCD) further expands CD by incorporating Land Use and Land Cover (LULC) segmentation before and after the changes, identifying specific change categories alongside the change area detection. While recent studies combining change area detection and bitemporal LULC segmentation within a multi-task framework demonstrate promising performance, they often only utilize pixels in changed areas with change classification labels in LULC segmentation training, overlooking substantial unlabeled LULC data in unchanged areas, which restricts the model’s effectiveness. Accordingly, we propose an enhanced multi-task SCD method with semi-supervised learning in LULC segmentation, effectively leveraging the extensive unlabeled LULC data and improving the overall performance of the multi-task framework. Besides, we introduce a novel loss tailored for this semi-supervised method based on the unique relationship between bi-temporal pixel labels in change areas and change classification. Optimization of the semi-supervised loss weighting further refines the training. Experiments on the public dataset validate the effectiveness of these improvements, especially in enhancing change classification performance. Applying our method to the naive model yields improvements in SeK and Fscd, with increases of up to 2%. The code will be available after the acceptance at https://github.com/ijnokml/scd-enhanced.

In recent years, significant progress has been made in extracting buildings from high spatial resolution (HSR) remote sensing images due to the rapid development of deep learning (DL). However, existing methods still have some limitations in maintaining the detail integrity of building footprint. Firstly, skip connections typically involve the direct concatenation of feature maps from adjacent levels, which inevitably leads to misalignment due to semantic differences. Second, the integration of building-related details remains a challenging task in the context of cross-modal remote sensing image. Third, the oversimplified upsampling structure used in previous methods may lead to loss of spatial details. In this paper, we propose a novel building extraction method ACMFNet based on cross-modal HSR remote sensing images using an encoder-decoder structure. First, we propose a global and local feature refinement module (GL-FRM) to refine features and establish contextual dependencies at multiple scales and levels, mitigating the spatial discrepancy among multi-level features. Meanwhile, a cross-modal fusion module (CFM) is utilized to integrate complementary features extracted from multispectral (MS) data and normalized digital surface model (nDSM) data. Additionally, we employed a lightweight residual upsampling module (RUM) for feature resolution recovery. We conducted complete experiments on two benchmark datasets, and the results indicate that our proposed ACMFNet achieves state-of-the-art (SOTA) performance without bells and whistles.

Several studies of weakly supervised learning have been applied to object detection in remote sensing images (RSIs), while critical challenges like part domination and class confusion remain, which lead to poor accuracy compared with fully supervised object detection tasks (e.g. FRCNN, YOLOv4) and natural images set tasks (e.g. PASCAL VOC 2007). The model is prone to focus on the most discriminate part of the object due to the fact that image-level annotations are lack of instances’ box information. Moreover, class confusion arises when the model is attempted to recognize instances of different categories that consistently coexist within a single training sample. To address the problems, we developed the low-shot instance transforming net (ITNet). ITNet is able to transform part domination boxes and misidentified confusion classes to be more accurate, which is trained with a combination of a small number of strong annotations and weak annotations. First of all, elastic cluster selection (ECS) is proposed to mine high quality weak pseudo annotations from the output of only weakly supervised object detection models (e.g. online instance classifier refinement). Label re-assignment (LRA) allows the correction of weak pseudo annotations with category noise by the recognition knowledge learned in strong annotations to alleviate 2-class confusion. Then semi-supervised elastic match (SSEM) is employed to update the annotations to make full use of strong annotations. Comprehensive experiments are carried out on NWPU VHR-10.v2 and DIOR, proving that the proposed ITNet outperforms the previous state-of-the-art significantly.

With the abomination of time-consuming or even impractical large-scale labeling, few-shot object detection (FSOD) based on natural scenes has attracted extensive attention. However, directly migrating FSOD methods designed for natural images to large-size remote sensing images (RSIs) still remains challenges. 1) Labels of novel instances within the base dataset are inconsistently assigned between the base training and the few-shot fine-tuning stage, which confuses the detector and leads to significant performance degradation over novel classes. 2) The region proposal network (RPN) of detectors cannot provide sufficient high-quality proposals for remote sensing objects with various aspect ratios and irregular shapes, leading to decreased detection performance. To tackle these issues, we specify a novel few-shot object detector for RSIs, to avoid the significant performance degradation caused by inconsistent labeling assignments, as well as efficiently leveraging the novel instances that existed in the base dataset. Furthermore, the proposed detector utilizes a coarse-to-fine regression method with an enhanced feature extractor called Gradual RPN to improve the recall of RPN. Experiments on a newly constructed few-shot detection benchmark show that our approach improves the mAP of novel classes by up to 8.4% and the average recall of RPN by up to 12.3%. The source code is available at here.

The traditional CNN-based methods usually employ the spatial information in the amplitude of complex Synthetic Aperture Radar (SAR) images. Several studies have started to concentrate on merging the unique physical properties of SAR images, such as DSN-v1, extracting the backscattering characteristic from the frequency domain. Although DSN-v1 has obtained impressive classification ability, there is some room for improvement. In this letter, DSN-v2 is proposed to boost the classification ability of man-made and natural objects in SAR images. The improvement is reflected in two aspects. First, a multi-scale sub-band feature extraction (MSFE) component is designed for natural objects. Since we observe their multi-scale sub-band spectrum is significantly different, multiple encoders are used to extract effective features. Second, the additive angular margin (AAM) loss is introduced to distinguish man-made objects more clearly by manually adding a margin to the decision boundary. The experimental results on the Sentinel-1 (S1) dataset show DSN-v2 achieves superior classification performance and model training speed compared with DSN-v1.

In the practical application of remote sensing scene classification (RSSC), domain adaptation is introduced to handle the situation where the distribution of training data (source) and test data (target) is different. Compared to general domain adaptation, open set domain adaptation (OSDA) is suitable for more realistic situations where there are additional unknown classes in the target domain. The key to solving this problem is to separate unknown samples from target data to avoid negative transfer caused by mismatching unknown/known samples. In this letter, we propose a novel and effective method, named Instance Affinity metric-based Fine-grained Adaptation Network (IAFAN), for OSDA in RSSC. Concretely, an Unknown Sample Separation (USS) mechanism based on the instance affinity-aware matrix is pioneeringly proposed to endow the model with the ability to distinguish unknown samples. In addition, considering the high inter-class similarity and rich intra-class diversity of remote sensing images, we introduce the Sample Discriminability Enhancement (SDE) loss to further increase the inter-class distance and narrow the intra-class difference, thereby alleviating the negative transfer caused by sample misclassification and mismatching. In the feature confusion stage, we specially design Mask-mmd for OSDA as an adaptation metric to conduct semantic fine-grained cross-domain alignment of known samples while keeping unknown samples out of alignment, which avoids negative transfer during the adaptation process. Finally, we evaluate IAFAN on transfer tasks between different public remote sensing datasets, and the results verify that our method significantly outperforms previous methods in RSSC.

Change detection plays an important role in Earth surface analysis. Current change detection methods have achieved good performance in large flat areas, but change detection of detailed parts is still a great challenge, and the loss of detail causes many faults around the change boundaries and on small objects. By analyzing the feature map of the widely used U-Net architecture in existing methods, we ascribe the detail loss to the depletion of detailed features during the top-to-down delivery in the U-Net architecture. The Feature Refine Change Detection(FRCD) model is proposed in which the detection results are predicted directly from the multiscale features instead of the U-Net architecture. By direct prediction, the representation ability of details is enhanced, and thus the detection accuracy of boundaries and small objects improves. Moreover, the normal upsampling in direct prediction is replaced with the deformable upsampling, which delivers detailed information from the low-level to the high-level via the deformable convolution, allowing the results to further fit boundaries in the FRCD model. Experimental results on two datasets confirm the effectiveness of FRCD compared to state-of-the-art methods, and the change detection results of boundaries and small objects are improved significantly by the proposed method. Code will be available after the acceptance of the paper in https://github.com/ijnokml/cdfr.

Single-frame infrared small target (SIRST) detection has been applied in many civilian and military applications and has greatly developed since deep-learning methods grow in recent years. However, the contradiction between semantic information and spatial details limits detection performance, especially for dim and small targets with complex interference. To overcome the restrictions, we propose an asymmetric patch attention fusion network (APAFNet) in this letter to merge high-level semantics and low-level spatial details, which consists of an APAF module based on a patch channel attention branch and a dilation context block, guiding the network to collect local semantics and spatial details. The experimental results on the NUAA-SIRST dataset and IRSTD-1k dataset show that the proposed APAFNet can achieve excellent performance under complex backgrounds.

Tracking moving targets in satellite videos has attracted wide attention recently. However, the development of target tracking in satellite videos is much slower than that in general videos for the following key reasons. First, typical moving objects in satellite videos consist of few pixels and lose most of their appearance features, making it difficult for the tracker to distinguish the target from the background. Second, the appearance of satellite video objects often changes due to occlusion, illumination variation, or other factors. Classic Siamese tracking networks only use the first frame as the target template, leading to poor tracking results. Third, when the target is fully occluded, it is difficult for the tracker to recapture the target. To address the above problems, we propose a Siamese tracking network based on multiple(M) response map fusion and spatiotemporal constraints in this paper. By generating response maps at different layers of the tracking network and fusing them adaptively, small objects in the satellite videos can be tracked more accurately. Furthermore, a dynamic(D) template update strategy is proposed to cope with possible changes in the appearance of objects in satellite videos, preventing the high dependence on the initial frame. To recapture the target, a score-guided target motion(M) trajectory prediction model is proposed. We call the proposed Siamese tracking network SiamMDM for short. We conducted complete experiments on SatSOT and SV248S, two large satellite video target tracking datasets. The results show that our method achieves state-of-the-art tracking performance while running at over 110 FPS.

Ship detection from synthetic aperture radar (SAR) remote sensing images is essential for monitoring water traffic and marine safety. Numerous methods for ship detection have been developed; however, the detection of small ships presents unique challenges. SAR image characteristics, such as the sidelobe effect and blurred outline induced by the special imaging mechanism, as well as the small ship size, are the primary factors that lower the detection accuracy. This paper provides a sidelobe-aware small ship detection network for synthetic aperture radar imagery. First, considering the sidelobe effect and blurred outline, dual-pooling, i.e., average pooling and max pooling, was utilized to build a feature extraction module that lowered the effects of strong scattering points outside of the ship body and enhanced the ship body information. Second, as the bipartition process of the average pooling and maximum pooling caused some loss of original data information, different feature maps in the network were concatenated to construct a new network structure to compensate for the information lost and enrich the small ship features. Third, because the traditional loss function based on centroid distance and aspect ratio may result in the same loss function value for different prediction box sizes, a novel loss function based on the dual Euclidean distances of the corner point coordinates between the prediction box and the real box was proposed, which could accurately describe various overlapping box situations. Experiments using the Large-Scale SAR Ship Detection Dataset (LS-SSDD), SAR Ship Detection Dataset (SSDD), and AIR-SARShip dataset validated the efficacy and state-of-the-art performance.

The existence of Radio Frequency (RF) Interference will cause an adverse effect on the interpretation of Synthetic Aperture Radar (SAR) images. There are various types of interference, and their pattens in images vary in different situations. Previous algorithms have disadvantages of low precision and large amount of computation. In this paper, we propose a prior-induced deep neural network. Based on the sparse and low-rank properties of interference signals in the time-frequency domain, an interference suppression network is designed to reconstruct useful signals. At the same time, a new loss function is designed, which integrates the sparse and low-rank properties with the training of network. The network combines the idea of semi-parametric interference suppression and the deep learning method, which can make good use of the characteristics of SAR echoes, making it more suitable for the field of signal processing and having a better effect. The proposed algorithm is applied to real SAR data with interference to validate its effect and efficiency.

Synthetic aperture radar automatic target recognition (SAR ATR) has been suffering from the insufficient labeled samples as the annotation of SAR data is time-consuming. Thus, adding unlabeled samples into training has attracted the attention of researchers. In this letter, a semi-supervised method based on consistency criterion, domain adaptation and Top-k loss is proposed to alleviate the need for labeled samples. According to consistency criterion that samples generated by the weak and strong augmentations from the same sample belong to the same category, we use the weak and strong augmented unlabeled samples to predict pseudo labels and train the model respectively. Then, to overcome the issue caused by the domain discrepancy between labeled and unlabeled samples especially when labeled samples concentrate on a narrow azimuth range, a domain adaptation component is designed to reduce their discrepancy. Besides, considering the incorrect pseudo labels will hamper the model training, the Top-k loss is adopted for unlabeled samples to mitigate the negative effects. The experimental results on Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset demonstrate the superiority of our method in semi-supervised SAR ATR. Specifically, we achieve about a 14.29% improvement in recognition accuracy compared to the state-of-the-art when the labeled samples concentrate on a narrow azimuth range.

Radio frequency electromagnetic interference is a relatively common phenomenon, especially for synthetic aperture radar (SAR) systems working in P- or L-band. Compared with the suppression of narrowband interference, that of wideband interference, particularly of those whose signal parameters have frequently changing property, is still a sophisticated problem. In this letter, a suppression algorithm for interference with wideband and complicated parameters is proposed, based on low-rank and sparse matrix decomposition (LRSMD) in time–frequency domain (TFD) of the signal. The proposed algorithm begins with transforming the SAR signal into TFD. After that, LRSMD based on bilateral random projection (BRP) is applied to decompose the time–frequency spectrum matrix into three parts, a low-rank matrix standing for interference, a sparse matrix standing for SAR signal, and a noise matrix. Finally, inversely transform the sparse matrix into the time domain to obtain SAR signal without interference. The proposed algorithm is applied to a single look complex (SLC) SAR data of Sentinel-1 to validate its effect and efficiency.

Remote sensing image-based ship detection is an important tool for ocean surveillance. Geostationary orbit (GEO) satellite is characteristic with a high temporal resolution, which makes continuous monitoring possible. However, the ships in such images are generally quite tiny. Traditional methods usually detect tiny ships with the character of shape, lightness and contrast, which are not resistant to false alarms from fractus. In this letter, a network-based detection method is adopted to extract spatio-temporal jointly features to distinguish the real ships from other distractions, especially in fractus concentered scenes. Based on detection results, an intersection over union (IoU) based target matching method is proposed to form the trails, suppressing the false alarms at the same time. To reduce the false alarm further, a structure similarity (SSIM) based appearance consistency measurement is utilized to remove objects whose appearance change over time. The experiment results show that the proposed method detects and tracks ships with high recall and low false alarm, compared with the traditional tracking methods. It could be applicated in GEO remote sensing images based ocean surveillance in various kinds of scenes.

Cloud-contaminated images seriously disturb the effective information in ground observations and reduce the availability of remote sensing images. This letter presents a thin cloud removal method for cloud-contaminated images based on a residual channel attention network. Compared with the present thin cloud removal methods not showing different attention to cloud components and ground details, the proposed method introduces channel attention mechanism into the residual learning (RL) path, which achieves suppressing thin clouds and enhancing the details of ground scenes simultaneously. Based on this, both residual channel attention module (RCAM) and residual group (RG) are constructed, playing the role of basic modules to stack an encoder–decoder-based residual channel attention network, which effectively avoids the loss of ground information in the deep layers. Compared with recent state-of-the-art (SOTA) methods, experimental results on both actual and synthesized cloudy images show the proposed method’s superiority in reconstructing rich details of ground scenes.

Deep learning methods have attracted much attention in the field of polarimetric synthetic aperture radar (PolSAR) image classification over the past few years. However, for supervised learning based methods, it is quite difficult to get large amounts of high-quality and labeled PolSAR data in real applications. In addition, there is a problem of poor generalization for the method of specific supervision labels. To solve the above issue, we explore how to learn representations from unlabeled data from a new perspective. In this letter, a self-supervised PolSAR representation learning (SSPRL) method is proposed. Different from supervised learning based methods, SSPRL aims to learn PolSAR image representations via unsupervised learning approach. Specifically, a self-supervised learning (SSL) method without negative samples is explored and a positive sample generation approach and a novel encoder architecture designed for PolSAR images are proposed. Moreover, mixup is implemented as a regularization strategy. Further the convolutional encoder is utilized to transfer the feature representation from the unlabeled PolSAR data to the downstream task, that is, to achieve the few-shot PolSAR classification. Comparative experimental results on two widely-used PolSAR benchmark datasets verify the effectiveness of proposed method and demonstrate that SSPRL produces impressive performance on few-shot classification task compared with state-of-the-art algorithms.

In recent years, convolutional neural networks (CNNs) have made significant progress in remote sensing scene classification (RSSC) tasks. Because obtaining a large number of labeled images is time-consuming and expensive and the generalization ability of supervised models is limited, domain adaptation is widely introduced into RSSC. However, existing adaptation approaches mainly aim to align the distribution of features in a single representation space, which results in losing information and limiting the spatial range for extracting domain-invariant features. In addition, some of the methods simultaneously align pixel-level (local) and image-level (global) features for better results but suffer from searching for the best weight of the two parts manually, which is time-consuming and computing-expensive. To overcome the above issues, a novel feature fusion-and-alignment approach named Multi-Representation Dynamic Adaption Network (MRDAN) is proposed for cross-domain RSSC. Concretely, a Feature-Fusion Adaptation (FFA) module is embedded into the network, which maps samples to multiple representations and fuses them to obtain a broader domain-invariant feature space. Based on this hybrid space, we introduce a cross-domain Dynamic Feature-Alignment Mechanism (DFAM) to quantitatively evaluate and adjust the relative importance of the local and global adaptation losses during domain adaptation. The experimental results on the 12 transfer tasks between the UC Merced land-use, WHU-RS19, AID, and RSSCN7 data sets demonstrate the effectiveness of the proposed MRDAN over the state-of-the-art domain adaptation methods in RSSC.

In the complex electromagnetic environment, radio frequency interference (RFI) from other radiation sources often conflicts with synthetic aperture radar (SAR) systems, which overlaps and destroys the useful data in the same frequency band, causing adverse impact to the quality of SAR imaging. When faced with wideband or mixed complicated RFI, traditional methods inevitablely damage the original signal, and cannot effectively protect and reconstruct the useful information. Besides, the current semi-parametric algorithms have large computations and limited generalization ability. To address these issues, this paper proposes a prior-induced-learning framework (PISNet) to achieve RFI suppression and useful signal recovery in time-frequency domain. Both narrowband and wideband interference are uniformly modeled as a sparse distribution in time-frequency domain, and the stationarity of SAR echoes determines its low-rank characteristic. These properties of RFI and SAR data are treated as prior knowledge to inject into our PISNet. An iterative reconstruction module is raised to achieve low-rank reorganization of the fused residual features. Meanwhile, a novel loss function is put forward to induce the network training to ensure that each component conform to the prior. The proposed approach innovatively integrates deep learning with semi-parametric methods for RFI suppression, which achieves superior performance on simulated and real data. Compared to existing learning-based methods, the image quality of the restored Sentinel-1 data is improved by 9.37% AG. The code and dataset will be available online (https://github.com/JyuanShen/PISNet).

Land use and land cover (LULC) classification plays a significant role in Earth observation tasks. Nowadays, we can observe the same scene with multiple heterogeneous sensors. Combining diverse information therein for multisource joint classification has become a promising research topic in the remote sensing community. For example, the fusion of hyperspectral image (HSI) and lidar detection and ranging (LiDAR) data has been under active research. The current methodology for HSI and LiDAR joint classification tends to ignore the topological relationship between pixels, limiting the effectiveness of feature extraction and fusion. Another obstacle to satisfactory performance is the scarcity of annotated data. To overcome the above challenges, this article proposes a multisource attention network called MAGE to improve the collective classification. We use a semi-supervised graph transductive module to underline the relevance among pixels by explicitly constructing a multimodal adjacency matrix. Specifically, MAGE designs a self-supervised feature extraction module for pre-training, mitigating the dependence on annotated samples and alleviating the common overfitting and over-smoothing problems encountered by the deep graph neural network (GNN). The experimental results of three standard datasets, i.e., MUUFL, Trento, and Houston, demonstrate the effectiveness of the proposed approach. In particular, MAGE achieves an overall accuracy of 95.26% and an average accuracy of 96.27% on the challenging MUUFL dataset, surpassing the state-of-the-art methods. The code and models are publicly available at https://github.com/d1x1u/MAGE.

The classification of large-scale high-resolution synthetic aperture radar (SAR) land cover images acquired by satellites is a challenging task, facing several difficulties such as semantic annotation with expertise, changing data characteristics due to varying imaging parameters or regional target area differences, and complex scattering mechanisms being different from optical imaging. Given a large-scale SAR land cover data set collected from TerraSAR-X images with a hierarchical three-level annotation of 150 categories and comprising more than 100 000 patches, three main challenges in automatically interpreting SAR images of highly imbalanced classes, geographic diversity, and label noise are addressed. In this letter, a deep transfer learning method is proposed based on a similarly annotated optical land cover data set (NWPU-RESISC45). Besides, a top-2 smooth loss function with cost-sensitive parameters was introduced to tackle the label noise and imbalanced classes' problems. The proposed method shows high efficiency in transferring information from a similarly annotated remote sensing data set, a robust performance on highly imbalanced classes, and is alleviating the overfitting problem caused by label noise. What is more, the learned deep model has a good generalization for other SAR-specific tasks, such as MSTAR target recognition with a state-of-the-art classification accuracy of 99.46%.

Understanding the physical properties and scattering mechanisms contributes to synthetic aperture radar (SAR) image interpretation. For single-polarized SAR data, however, it is difficult to extract the physical scattering mechanisms due to lack of polarimetric information. Time-frequency analysis (TFA) on complex-valued SAR image provides extra information in frequency perspective beyond the “image” domain. Based on TFA theory, we propose to generate the subband scattering pattern for every object in complex-valued SAR image as the physical property representation, which reveals backscattering variations along slant-range and azimuth directions. In order to discover the inherent patterns and generate a scattering classification map from single-polarized SAR image, an unsupervised hierarchical deep embedding clustering (HDEC) algorithm based on TFA (HDEC-TFA) is proposed to learn the embedded features and cluster centers simultaneously and hierarchically. The polarimetric analysis result for quad-pol SAR images is applied as reference data of physical scattering mechanisms. In order to compare the scattering classification map obtained from single-polarized SAR data with the physical scattering mechanism result from full-polarized SAR, and to explore the relationship and similarity between them in a quantitative way, an information theory based evaluation method is proposed. We take Gaofen-3 quad-polarized SAR data for experiments, and the results and discussions demonstrate that the proposed method is able to learn valuable scattering properties from single-polarization complex-valued SAR data, and to extract some specific targets as well as polarimetric analysis. At last, we give a promising prospect to future applications.

Deep learning based patch-wise Synthetic Aperture Radar (SAR) image classification usually requires a large number of labeled data for training. Aiming at understanding SAR images with very limited annotation and taking full advantage of complex-valued SAR data, this paper proposes a general and practical framework for quad-, dual-, and single-polarized SAR data. In this framework, two important elements are taken into consideration: image representation and physical scattering properties. Firstly, a convolutional neural network is applied for SAR image representation. Based on time-frequency analysis and polarimetric decomposition, the scattering labels are extracted from complex SAR data with unsupervised deep learning. Then, a bag of scattering topics for a patch is obtained via topic modeling. By assuming that the generated scattering topics can be regarded as the abstract attributes of SAR images, we propose a soft constraint between scattering topics and image representations to refine the network. Finally, a classifier for land cover and land use semantic labels can be learned with only a few annotated samples. The framework is hybrid for the combination of deep neural network and explainable approaches. Experiments are conducted on Gaofen-3 complex SAR data and the results demonstrate the effectiveness of our proposed framework.

Deep convolutional neural networks (DCNNs) have attracted much attention in remote sensing recently. Compared with the large-scale annotated data set in natural images, the lack of labeled data in remote sensing becomes an obstacle to train a deep network very well, especially in synthetic aperture radar (SAR) image interpretation. Transfer learning provides an effective way to solve this problem by borrowing knowledge from the source task to the target task. In optical remote sensing application, a prevalent mechanism is to fine-tune on an existing model pretrained with a large-scale natural image data set, such as ImageNet. However, this scheme does not achieve satisfactory performance for SAR applications because of the prominent discrepancy between SAR and optical images. In this article, we attempt to discuss three issues that are seldom studied before in detail: 1) what network and source tasks are better to transfer to SAR targets; 2) in which layer are transferred features more generic to SAR targets; and 3) how to transfer effectively to SAR targets recognition. Based on the analysis, a transitive transfer method via multisource data with domain adaptation is proposed in this article to decrease the discrepancy between the source data and SAR targets. Several experiments are conducted on OpenSARShip. The results indicate that the universal conclusions about transfer learning in natural images cannot be completely applied to SAR targets, and the analysis of what and where to transfer in SAR target recognition is helpful to decide how to transfer more effectively.

SAR image simulation plays an important role in the process of SAR target interpretation and recognition, especially when the number of SAR images is limited. Due to the restriction of acquisition process, the numbers of SAR target images are always insufficient. The traditional SAR image simulation, which is based on calculation of electromagnetic theory, is easily to be affected by parameter distortion due to the lack of joint optimization. Consequently, it makes a big effect on the quality of the simulated images. This paper presents a novel approach, end-to-end models, to simulate the desired images from the SAR image database. A series of generative adversarial networks include DCGAN, weight clipping WGAN and WGAN with gradient penalty are optimized and applied to generate typical SAR target images. Three kinds of network structures are used, include structure of DCGAN, newly proposed structure of four residual blocks networks and Resnet. Experimental results show that the proposed novel method is not only efficient for SAR image simulation, but also can generate excellent SAR images. Furthermore, we analysis the results and the characteristics of different networks, which pave a good way for SAR image simulation based on artificial intelligence method.

Deep learning algorithms are widely used in remote sensing image scene understanding. Generally, a large-scale annotated dataset is essential to train a deep neural network for classification. In practical terms, however, a large amount of unknown remote sensing images obtained from different sensors need to be understood which may vary from resolution, geolocation and imaging conditions compared with annotated datasets. In this paper, an unsupervised domain adaptation framework based on ResNet-18 is presented to transfer the knowledge of an existing annotated land cover dataset to other remote sensing data, decreasing the discrepancy among images across sensors. The results show a significant improvement in scene understanding of new remote sensing images.

Objects in remote sensing images are difficult to detect due to arbitrarily rotated angles and the wide variance of scales. As bounding box plays an important role in object detection, we proposes a new bounding box type named rotated bounding box (rBox). With the application of rBox, we have proposed a series of detection techniques (DrBox, DrBoxLight, DrBoxSemi, DrBoxPro) to effectively handle the situation where the orientation angles of the objects are arbitrary. This article is a brief overview of these techniques. The original DrBox detector applies VGG-net as its main network framework, with image pyramid input to address multi-scale problem. DrBoxLight is a mini version of DrBox, which applies MobileNet and knowledge distillation to be deployed on embedded devices. DrBoxSemi is a semi-supervised version of DrBox, so annotation of all training samples is no longer necessary. DrBoxPro is the most important update for DrBox with professional designing of abundant prior-rBoxes on feature pyramid networks. In our experiments, we demonstrated how rBox helps to improve the performance of object detection compared with traditional bounding boxes. Besides, we evaluated the performance of our DrBox family on a series of object detection tasks.

A Siamese network based metric learning method is proposed for SAR target classification with few training samples. The network consists of two identical CNNs sharing the weights. Different from classification networks that predict the category of one sample, the Siamese network implements a metric learning to measure the similarity between two samples. Since the input is the sample pair, the amount of training data dramatically increases which contributes to training a better network. When generating the pairs, a hard negative mining scheme is proposed for improving the performance. To avoid computing the similarity between the test sample and each training sample at the test stage, which is time consuming, a two stages scheme is employed with an additional classification network taking the output of the single branch of Siamese network as the input and predicting the category. Experiments on the MSTAR dataset validate the effectiveness of the proposed method.

Convolutional neural network (CNN)-based methods have been successfully applied to SAR target detection. Different from prevalently used detection approaches with rectangle bounding box, rotatable bounding box (RBox)-based methods, such as DRBox-v1, can effectively reduce the interference of background pixels and locate the targets more finely for geospatial object detection. Although DRBox-v1 has achieved impressive detected performance, there still exist some remaining problems and room for improvement. In this paper, an improved RBox-based target detection framework is proposed to boost precision and recall rates of detection, and we refer to the method as DRBox-v2 and apply it to target detection in SAR images. The main improvements of DRBox-v2 as well as the contributions of this paper are fourfold. First, a multi-layer prior box generation strategy is designed for detecting small-scale targets. Since shallow layers lack strong sematic information, the feature pyramid network (FPN) module is applied. Second, a modified encoding scheme for RBox is proposed for more precisely estimating the position of RBox and orientation of targets. Third, a focal loss (FL) combined with hard negative mining (HNM) technique is proposed to mitigate the issue of the imbalance between positive and negative samples, which produces better results than solely employing either one. Fourth, comprehensive ablation studies are conducted to reveal the effect of each improvement on detected results. The results of the target detection on three data sets are illustrated and our method obtains 0.135, 0.081, 0.115 gains in average precision compared with three state-of-the-art methods, respectively.

Deep learning (DL) has been successfully applied to single image super-resolution (SISR), which aims at reconstructing a high-resolution (HR) image from its low-resolution (LR) counterpart. Different from most current DL-based methods, which perform reconstruction in the spatial domain, we use a scheme based in the frequency domain to reconstruct the HR image at various frequency bands. Further, we propose a method that incorporates the wavelet transform (WT) and the recursive Res-Net. The WT is applied to the LR image to divide it into various frequency components. Then, an elaborately designed network with recursive residual blocks is used to predict high-frequency components. Finally, the reconstructed image is obtained via the inverse WT. This paper has three main contributions: 1) an SISR scheme based on the frequency domain is proposed under a DL framework to fully exploit the potential to depict images at different frequency bands; 2) recursive block and residual learning in global and local manners are adopted to ease the training of the deep network, and the batch normalization layer is removed to increase the flexibility of the network, save memory, and promote speed; and 3) the low-frequency wavelet component is replaced by an LR image with more details to further improve performance. To validate the effectiveness of the proposed method, extensive experiments are performed using the NWPU-RESISC45 data set, and the results demonstrate that the proposed method outperforms several state-of-the-art methods in terms of both objective evaluation and subjective perspective.

High-resolution (HR) images are always preferred for many remote sensing applications, which can be obtained from their low-resolution (LR) counterparts via a technique referred to as super-resolution (SR). Among SR approaches, single image SR (SISR) methods aim at reconstructing the HR image from only one LR image. In this paper, a residual dense backprojection network (RDBPN)-based SISR method is proposed to promote the resolution of RGB remote sensing images with median- and large-scale factors. The proposed network consists of several residual dense backprojection blocks that contain two kinds of modules, named the upprojection module and the downprojection module, and these modules are densely connected in one block. Different from the chain-connected backprojection structure, the proposed method applies a residual backprojection block structure, which can utilize residual learning in both global and local manners. We further simplify the network by replacing the downprojection unit with the downscaling unit to accelerate the speed of reconstruction, and this implementation is called fast RDBPN (FRDBPN). Several experiments under the UC Merced data set are conducted to validate the effectiveness of the proposed method, and the results indicate that: 1) the proposed residual block structure is superior to the chain-connected structure; 2) FRDBPN achieves a speedup of about 1.3 times with similar and even better-reconstructed performance in comparison with RDBPN; and 3) RDBPN and FRDBPN outperform several state-of-the-art methods in terms of both quantitative evaluation and visual quality.

Object detection is a challenging task in computer vision. Now many detection networks can get a good detection result when applying large training dataset. However, annotating sufficient amount of data for training is often time-consuming. To address this problem, a semi-supervised learning based method is proposed in this paper. Semi-supervised learning trains detection networks with few annotated data and massive amount of unannotated data. In the proposed method, Generative Adversarial Network is applied to extract data distribution from unannotated data. The extracted information is then applied to improve the performance of detection network. Experiment shows that the method in this paper greatly improves the detection performance compared with supervised learning using only few annotated data. The results prove that it is possible to achieve acceptable detection result when only few target object is annotated in the training dataset.

Inshore ship detection in SAR image faces difficulties on correctly identifying near-shore ships and onshore objects. This article proposes a multi-scale full convolutional network (MS-FCN) based sea-land segmentation method and applies a rotatable bounding box based object detection method (DR-Box) to solve the inshore ship detection problem. The sea region and land region are separated by MS-FCN then DR-Box is applied on sea region. The proposed method combines global information and local information of SAR image to achieve high accuracy. The networks are trained with Chinese Gaofen-3 satellite images. Experiments on the testing image show most inshore ships are successfully located by the proposed method.

Target classification is an important part in automatic target recognition (ATR) systems. Deep learning methods get state of the art performance in SAR target classification. Simulation is a useful data augmentation method when the numbers of real samples for training is not sufficient. This article discusses how to release the full potential of simulated samples which is used to improve performance of SAR target classifier. The proposed method is based on cycle adversarial network (CycleGAN), which can transfer simulated samples to be more similar with real samples in image domain. Experiments show that adding simulated samples straightforward into training dataset is not helpful to improve the performance. However, adding the transferred simulated samples for training results in about 10% increase in accuracy in the designed SAR airplane classification experiment, compared with training without data augmentation.

Single image super-resolution (SR) has been widely studied in recent years as a crucial technique for remote sensing applications. This paper proposes a SR method for remote sensing images based on a transferred generative adversarial network (TGAN). Different from the previous GAN-based SR approaches, the novelty of our method mainly reflects from two aspects. First, the batch normalization layers are removed to reduce the memory consumption and the computational burden, as well as raising the accuracy. Second, our model is trained in a transfer-learning fashion to cope with the insufficiency of training data, which is the crux of applying deep learning methods to remote sensing applications. The model is firstly trained on an external dataset DIV2K and further fine-tuned with the remote sensing dataset. Our experimental results demonstrate that the proposed method is superior to SRCNN and SRGAN in terms of both the objective evaluation and the subjective perspective.

A salient seed extraction based target detection method is proposed in this paper, aiming to distinguish target points from background points in SAR images. Different from recent superpixel based method which generates superpixels firstly, and for each superpixel decides whether it belongs to part of a target. The proposed method employs a salient point to region scheme. At first, salient seeds are extracted by mean-shift and region feature based approach. Then, pixels are assigned to the most similar seed and those assigned to the salient seeds are extracted to form the foreground region. Finally, constant false alarm rate (CFAR) operation is employed to detect the target points from the foreground region. The effectiveness of the proposed method is validated by comparing with five state-of-the-art methods on TerraSAR-X images.

An identity regularized sparse representation (IRSR) based SAR target recognition method is proposed in this paper. The method aims to find a transformation that can map the data to a transformed space, in which targets from the same class are close with each other, no matter the distance of them in the original space. This identity constraint can be formulated as a ℓ1-norm minimization problem. By decoupling the problem into the sparse coding problem and the dictionary learning problem, the solution can be obtained iteratively. The solution is simply the weighted average of the sparse coding of all training data. Experimental results demonstrate that the proposed method is superior to several related methods.

In this paper, a relaxation labelling based land masking method is proposed for separating sea and land in SAR images. Land masking, also known as sea-land segmentation, is a part of ship detection system for SAR images to avoid detecting false alarms in the land. Relaxation labeling is an iterative method, which can separate foreground pixels from background ones using the neighborhood information of pixels in the image. When relaxation labelling converges, the segmented result is often unsatisfactory, since it tends to label more foreground pixels. To overcome this issue, a loss composed of the background probability distribution diversity and the gradient magnitude of the result is introduced to indicate when to stop the iteration. Experimental results on several Gaofen-3 SAR images demonstrate the effectiveness of the proposed method.

The edges of shadows are blurred in Synthetic Aperture Radar (SAR) images due to the moving of the radar when data are collected. This phenomenon becomes obvious in High Resolution (HR) SAR images. In this work, an adaptive approach for shadow enhancements is proposed. The performance of the shadow enhancement has some relationship with the precision of the estimation of the height and this rule is used in this work. The Height-Variant Phase Compensation (HVPC) and golden section algorithm are employed. The adaptive method is built by iterative progress of calculating the quantities of pixels corresponding to the shadow region. This method provides an automatic way for shadow enhancements and it is suitable for objects mainly composed of flat-like structures. The experiments based on the Mini-SAR of a helicopter are implemented to test the validity of the approach. The work in this paper provides a way for shadow enhancement for HR SAR images and would be useful for SAR Automatic Target Recognition.

Target recognition in synthetic aperture radar (SAR) images has become a hotspot in recent years. The backscattering characteristic of target is a significant issue taken into consideration in SAR applications. Almost all of the previous work focus on the scatter point extraction to depict the backscattering characteristic of the target; however, a point-target corresponds to a region rather than a single point due to the convolution during the imaging. Based on this fact, we first analyze the extent to how a point-target spreads, then propose a novel scatter cluster extraction (SCE) method, and utilize the scatter cluster as the feature to solve the airplane recognition problem in SAR images. In practice, there often exist interfering objects near the target to be classified. To overcome this issue, we design a reweighted sparse representation (RSR)-based automatic purifying method by assigning a weight to each element of the feature iteratively according to the representation error. Since the element with large representation error always corresponds to the interfering objects, we give it a small weight, consequently suppressing the influence of the interference. Experimental results demonstrate that the proposed SCE method outperforms the traditional scatter point extraction-based method as well as some state-of-the-art methods. The comparison result also validates the effectiveness of the proposed RSR method.

Ship target recognition has always been a hot issue in the field of ocean surveillance. Due to the serious shortage of samples in ship target recognition for synthetic aperture radar (SAR) images, the template-based method is still one of the most effective ways to solve the problem. In this letter, we put forward a novel ship recognition method based on the projection shape template (PST), aiming at increasing both the accuracy and the robustness of the recognition. The PST of each category is calculated by projecting the 3-D model obtained from the two-view images of the target to the 2-D slant-plane image according to the SAR imaging model. Then, we propose a contour extraction method to detect the profile of ships, which served as the feature. Finally, the identity of the query ship is obtained through contour matching. Experimental results indicate that the proposed method is effective even when the number of samples is extremely small, consequently providing a promising way for the automatic interpretation of ship targets in the SAR images.

A super-resolution (SR) method based on compressive sensing (CS), structural self-similarity (SSSIM), and dictionary learning is proposed for reconstructing remote sensing images. This method aims to identify a dictionary that represents high resolution (HR) image patches in a sparse manner. Extra information from similar structures which often exist in remote sensing images can be introduced into the dictionary, thereby enabling an HR image to be reconstructed using the dictionary in the CS framework. We use the K-Singular Value Decomposition method to obtain the dictionary and the orthogonal matching pursuit method to derive sparse representation coefficients. To evaluate the effectiveness of the proposed method, we also define a new SSSIM index, which reflects the extent of SSSIM in an image. The most significant difference between the proposed method and traditional sample-based SR methods is that the proposed method uses only a low-resolution image and its own interpolated image instead of other HR images in a database. We simulate the degradation mechanism of a uniform 2 × 2 blur kernel plus a downsampling by a factor of 2 in our experiments. Comparative experimental results with several image-quality-assessment indexes show that the proposed method performs better in terms of the SR effectivity and time efficiency. In addition, the SSSIM index is strongly positively correlated with the SR quality.