The disperse structure distributions (discreteness) and variant scattering characteristics (variability) of SAR airplane targets lead to special challenges of object detection and recognition. The current deep learning-based detectors encounter challenges in distinguishing fine-grained SAR airplanes against complex backgrounds. To address it, we propose a novel physics-guided detector (PGD) learning paradigm for SAR airplanes that comprehensively investigate their discreteness and variability to improve the detection performance. It is a general learning paradigm that can be extended to different existing deep learning-based detectors with ”backbone-neck-head” architectures. The main contributions of PGD include the physics-guided self-supervised learning, feature enhancement, and instance perception, denoted as PGSSL, PGFE, and PGIP, respectively. PGSSL aims to construct a self-supervised learning task based on a wide range of SAR airplane targets that encodes the prior knowledge of various discrete structure distributions into the embedded space. Then, PGFE enhances the multi-scale feature representation of a detector, guided by the physics-aware information learned from PGSSL. PGIP is constructed at the detection head to learn the refined and dominant scattering point of each SAR airplane instance, thus alleviating the interference from the complex background. We propose two implementations, denoted as PGD and PGD-Lite, and apply them to various existing detectors with different backbones and detection heads. The experiments demonstrate the flexibility and effectiveness of the proposed PGD, which can improve existing detectors on SAR airplane detection with fine-grained classification task (an improvement of 3.1% mAP most), and achieve the state-of-the-art performance (90.7% mAP) on SAR-AIRcraft-1.0 dataset. The project is open-source at https://github.com/XAI4SAR/PGD.

Effective noise suppression is crucial for the subsequent interpretation tasks of SAR imagery. Traditional SAR image processing techniques often overlook the coherent nature of noise, leading to a loss of vital detail during filtering. With advancements in deep-learning, significant strides have been made in image processing. However, existing deep-learning methods do not fully leverage the imaging mechanisms of SAR, resulting in a lack of specificity and interpretability in the filtering process. To balance noise reduction with detail preservation and to address the “black box” issue in filtering, we propose an interpretable filtering method that employs a correlation-based upward search for density peaks. Initially, we develop an MeanShift-Markov Random Fields filter (MS-MRF) that integrates MeanShift with Markov Random Fields (MRF) in the joint spatial-spectral domain, ensuring both correlation and detail preservation; the derivation of the MS-MRF filter is rigorously grounded in mathematical theory. Subsequently, we integrate MS-MRF with convolutional operations in deep-learning to create a novel convolutional filter, Interpretable MS-MRF Convolution (IMMC), which enhances the model’s interpretability, noise reduction capabilities, and detail retention. Extensive experiments demonstrate that our method outperforms State of the art(SOTA) SAR denoising techniques, achieving an average SSIM of over 85.00% and an average PSNR exceeding 35.00dB across synthetic datasets with varying noise levels, showing significant improvements in noise suppression, detail preservation, and interpretability.

Few-shot object detection (FSOD) in remote sensing images is a marginally explored but highly challenging task that focuses on identifying unseen classes of objects with a limited number of annotations. Current FSOD approaches often fail to accurately localize the foreground and misalign targets with various orientations, resulting in poor detection performance. For this purpose, we develop a fresh and powerful meta-learning framework based on the idea of imprinting, which leverages tailored support information to model the regional correlation between query and support objects in different stages. Specifically, a global-integrated scheme is first proposed to guide the generation of high-quality proposals by increasing the activation of foreground features and integrating global support information. Considering the orientation discrepancy of objects in query and support sets, we introduce a drift-rectified technique to achieve adaptive alignment by implicitly capturing the positional correspondence between the instances in two sets. In stark contrast to conventional FSOD approaches, our method can extract key clues and establish directional relationships between objects from different training sets, leading to better generalization capability. Extensive experiments on two standard benchmarks (DIOR and NWPU VHR-10.V2) manifest the effectiveness, and our proposed method exhibits superior performance to other competitors with similar motivation. The source code is available athttps://github.com/Ybowei/GIDR

Synthetic aperture radar (SAR) imaging provides a distinct advantage in scene understanding due to its capability for all-weather data acquisition. However, in comparison to easily annotated optical remote sensing images, the lower imaging quality of SAR images presents significant challenges in obtaining manually annotated training data, which poses substantial issues for SAR image analysis. In this paper, we employ the domain adaptation (DA) that leverages labeled optical images to better understand unlabeled SAR images. Global feature alignment as a method for DA has demonstrated effectiveness in transferring knowledge, yet it faces challenges in cross-modality adaptation from optical remote sensing to SAR images due to their differing imaging mechanisms. With distinct visual features between optical and SAR images, the semantic dependency is difficult to construct, which results in low-quality pseudo-label assignment for SAR images. To address the above issue, we propose a semantic graph learning framework to comprehensively align the global features of optical remote sensing and SAR images by modeling the cross-modality semantics and generating high-quality pseudo-labels. It can be applied for SAR scene classification and object detection when only optical remote sensing images are labeled. Specifically, a cross-modality semantic graph alignment (CSGA) module is constructed to model and align the second-order semantic dependencies by aggregating cross-modality visual semantic information. Then, an uncertainty-based robust pseudo-label generation (URPG) module is designed to generate pseudo-labels for effective semantic alignment and self-training by modeling the uncertainty of pseudo-labels for each SAR image. Comprehensive experiments show that our proposed method outperforms the state-of-the-art methods on scene classification (NWPU-RESISC45→WHU-SAR6, MLRSNet→NWPU-SAR6, MLRSNet→NWPU-SAR6, and NWPU-RESISC45→NWPU-SAR6) and object detection (MASATI-ship→SSDD, MVSRD→SARDet-vehicle, and DIOR-airplane→SAR-airplane) tasks. The code and datasets are publicly accessible at https://github.com/XZhang878/SGLF.

Synthetic aperture radar (SAR) image simulation has attracted much attention due to its great potential to supplement the scarce training data for deep learning algorithms. Consequently, evaluating the quality of the simulated SAR image is crucial for practical applications. The current literature primarily uses image quality assessment (IQA) techniques for evaluation that rely on human observers' perceptions. However, because of the unique imaging mechanism of SAR, these techniques may produce evaluation results that are not entirely valid. The distribution inconsistency between real and simulated data is the main obstacle that influences the utility of simulated SAR images. To this end, we propose a novel trustworthy utility evaluation framework with a counterfactual explanation for simulated SAR images for the first time, denoted as X-Fake. It unifies a probabilistic evaluator and a causal explainer to achieve a trustworthy utility assessment. We construct the evaluator using a probabilistic Bayesian deep model to learn the posterior distribution, conditioned on real data. Quantitatively, the predicted uncertainty of simulated data can reflect the distribution discrepancy. We build the causal explainer with an introspective variational auto-encoder (IntroVAE) to generate high-resolution counterfactuals. The latent code of IntroVAE is finally optimized with evaluation indicators and prior information to generate the counterfactual explanation, thus revealing the inauthentic details of simulated data explicitly. The proposed framework is validated on four simulated SAR image datasets obtained from electromagnetic models and generative artificial intelligence approaches. The results demonstrate the proposed X-Fake framework outperforms other IQA methods in terms of utility. Furthermore, the results illustrate that the generated counterfactual explanations are trustworthy, and can further improve the data utility in applications.

The current SAR ship detection datasets follow the traditional paradigm in computer vision field, that is, the image data and the corresponding annotations are provided. However, due to the special imaging mechanism of SAR, the amplitude or intensity data is limited and cannot offer sufficient electromagnetic information of target. Although the single-look complex SAR data is informative, it requires a large number of storage space and is difficult to interpret visually. To this end, a new benchmark for SAR ship detection is proposed in this paper, namely PhySAR-Det. It is constructed based on different levels of SAR products of Gaofen-3 satellite, including L1A complex-valued image and the geo-coded L2 product. We propose another microwave visual characteristic (MVC) product derived from L1A data and projected to L2 coordinates to characterize the scattering mechanisms. The L2 images with high visual quality attached with the corresponding MVC products form the proposed PhySAR-Det dataset. The experiments show that the MVC product improves the performance compared with only image data available. It will be available at https://github.com/XAI4SAR/PhySARDet.

Most existing sparse representation based approaches for attributed scattering center (ASC) extraction adopt traditional iterative optimization algorithms, which suffer from lengthy computation time and limited precision. This paper presents a solution by introducing an interpretable network that can effectively and rapidly extract ASC via deep unfolding. Initially, we create a dictionary containing reliable prior knowledge and apply it to iterative shrinkage-thresholding algorithm (ISTA). Then, we unfold ISTA to a neural network, employing it to autonomously and precisely optimize the hyperparameters. The interpretability in physics is retained by applying a dictionary with physical meaning. The experiments are conducted on multiple test sets with diverse data distribution and demonstrate the superior performance and generalizability of our method.

In this paper, we briefly introduce the concept of physics guided learning for Synthetic Aperture Radar (SAR) and the potential advantages. Specifically, we propose a physics guided learning method for SAR airplane target feature representation, where the airplane scattering characteristics are extracted to guide the model training. To this end, the feature representation of physics guided learning is constrained to be aware of target scattering characteristics. The experimental results tentatively illustrate the effectiveness.

Deep learning-based synthetic aperture radar (SAR) target detection has been developed for years, with many advanced methods proposed to achieve higher indicators of accuracy and speed. In spite of this, the current deep detectors cannot express the reliability and interpretation in trusting the predictions, which are crucial especially for those ordinary users without much expertise in understanding SAR images. To achieve explainable SAR target detection, it is necessary to answer the following questions: how much should we trust and why cannot we trust the results. With this purpose, we explore the uncertainty for SAR target detection in this article by quantifying the model uncertainty and explaining the ignorance of the detector. First, the Bayesian deep detectors (BDDs) are constructed for uncertainty quantification, answering how much to trust the classification and localization result. Second, an occlusion-based explanation method (U-RISE) for BDD is proposed to account for the SAR scattering features that cause uncertainty or promote trustworthiness. We introduce the probability-based detection quality (PDQ) and multielement decision space for evaluation besides the traditional metrics. The experimental results show that the proposed BDD outperforms the counterpart frequentist object detector, and the output probabilistic results successfully convey the model uncertainty and contribute to more comprehensive decision-making. Furthermore, the proposed U-RISE generates an attribution map with intuitive explanations to reveal the complex scattering phenomena about which BDD is uncertain. We deem our work will facilitate explainable and trustworthy modeling in the field of SAR image understanding and increase user comprehension of model decisions.

Fine-grained sea-ice classification in SAR images is challenging due to the scarce labeled data and the imperfect annotation. Pre-training strategies are commonly carried out to prevent severe overfitting with limited labeled data. In spite of this, the observation noise still exists in the transferred features, which can be captured by aleatoric uncertainty. In this paper, we propose an aleatoric uncertainty embedded sea-ice classification method together with transfer learning of two different pre-training strategies. Instead of representing the transferred feature as a deterministic embedding, the proposed method concerns the feature uncertainty and models the embedding as a Gaussian distribution with variance. The experiments demonstrate that the proposed aleatoric uncertainty estimation is beneficial to improving the classification result of transfer learning. Based on the measured feature uncertainty, we analyze the potential of integrating two different pre-trained models to further enhance the performance.

Synthetic aperture radar automatic target recognition (SAR ATR) has been suffering from the insufficient labeled samples as the annotation of SAR data is time-consuming. Thus, adding unlabeled samples into training has attracted the attention of researchers. In this letter, a semi-supervised method based on consistency criterion, domain adaptation and Top-k loss is proposed to alleviate the need for labeled samples. According to consistency criterion that samples generated by the weak and strong augmentations from the same sample belong to the same category, we use the weak and strong augmented unlabeled samples to predict pseudo labels and train the model respectively. Then, to overcome the issue caused by the domain discrepancy between labeled and unlabeled samples especially when labeled samples concentrate on a narrow azimuth range, a domain adaptation component is designed to reduce their discrepancy. Besides, considering the incorrect pseudo labels will hamper the model training, the Top-k loss is adopted for unlabeled samples to mitigate the negative effects. The experimental results on Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset demonstrate the superiority of our method in semi-supervised SAR ATR. Specifically, we achieve about a 14.29% improvement in recognition accuracy compared to the state-of-the-art when the labeled samples concentrate on a narrow azimuth range.

This paper proposes a novel unsupervised learning method to learn discriminative physics-aware features of Synthetic Aperture Radar images with deep neural networks. We conduct a case study of sea-ice classification using Sentinel-1 Dual-polarized SAR data and the corresponding scattering mechanisms derived from H/α Wishart classification. The scattering mechanisms are encoded as a combination of topics for each SAR image as physics attributes, which guide the deep convolutional neural network to learn physics-aware features automatically. A novel objective function is designed to demonstrate how to conduct the physics-guided learning processing. The experiments show the proposed method can learn discriminative features from SAR images without labeled data, which can achieve a comparable classification result with supervised CNN learning.

The classification of large-scale high-resolution synthetic aperture radar (SAR) land cover images acquired by satellites is a challenging task, facing several difficulties such as semantic annotation with expertise, changing data characteristics due to varying imaging parameters or regional target area differences, and complex scattering mechanisms being different from optical imaging. Given a large-scale SAR land cover data set collected from TerraSAR-X images with a hierarchical three-level annotation of 150 categories and comprising more than 100 000 patches, three main challenges in automatically interpreting SAR images of highly imbalanced classes, geographic diversity, and label noise are addressed. In this letter, a deep transfer learning method is proposed based on a similarly annotated optical land cover data set (NWPU-RESISC45). Besides, a top-2 smooth loss function with cost-sensitive parameters was introduced to tackle the label noise and imbalanced classes' problems. The proposed method shows high efficiency in transferring information from a similarly annotated remote sensing data set, a robust performance on highly imbalanced classes, and is alleviating the overfitting problem caused by label noise. What is more, the learned deep model has a good generalization for other SAR-specific tasks, such as MSTAR target recognition with a state-of-the-art classification accuracy of 99.46%.

Understanding the physical properties and scattering mechanisms contributes to synthetic aperture radar (SAR) image interpretation. For single-polarized SAR data, however, it is difficult to extract the physical scattering mechanisms due to lack of polarimetric information. Time-frequency analysis (TFA) on complex-valued SAR image provides extra information in frequency perspective beyond the “image” domain. Based on TFA theory, we propose to generate the subband scattering pattern for every object in complex-valued SAR image as the physical property representation, which reveals backscattering variations along slant-range and azimuth directions. In order to discover the inherent patterns and generate a scattering classification map from single-polarized SAR image, an unsupervised hierarchical deep embedding clustering (HDEC) algorithm based on TFA (HDEC-TFA) is proposed to learn the embedded features and cluster centers simultaneously and hierarchically. The polarimetric analysis result for quad-pol SAR images is applied as reference data of physical scattering mechanisms. In order to compare the scattering classification map obtained from single-polarized SAR data with the physical scattering mechanism result from full-polarized SAR, and to explore the relationship and similarity between them in a quantitative way, an information theory based evaluation method is proposed. We take Gaofen-3 quad-polarized SAR data for experiments, and the results and discussions demonstrate that the proposed method is able to learn valuable scattering properties from single-polarization complex-valued SAR data, and to extract some specific targets as well as polarimetric analysis. At last, we give a promising prospect to future applications.

Deep learning based patch-wise Synthetic Aperture Radar (SAR) image classification usually requires a large number of labeled data for training. Aiming at understanding SAR images with very limited annotation and taking full advantage of complex-valued SAR data, this paper proposes a general and practical framework for quad-, dual-, and single-polarized SAR data. In this framework, two important elements are taken into consideration: image representation and physical scattering properties. Firstly, a convolutional neural network is applied for SAR image representation. Based on time-frequency analysis and polarimetric decomposition, the scattering labels are extracted from complex SAR data with unsupervised deep learning. Then, a bag of scattering topics for a patch is obtained via topic modeling. By assuming that the generated scattering topics can be regarded as the abstract attributes of SAR images, we propose a soft constraint between scattering topics and image representations to refine the network. Finally, a classifier for land cover and land use semantic labels can be learned with only a few annotated samples. The framework is hybrid for the combination of deep neural network and explainable approaches. Experiments are conducted on Gaofen-3 complex SAR data and the results demonstrate the effectiveness of our proposed framework.

Deep convolutional neural networks (DCNNs) have attracted much attention in remote sensing recently. Compared with the large-scale annotated data set in natural images, the lack of labeled data in remote sensing becomes an obstacle to train a deep network very well, especially in synthetic aperture radar (SAR) image interpretation. Transfer learning provides an effective way to solve this problem by borrowing knowledge from the source task to the target task. In optical remote sensing application, a prevalent mechanism is to fine-tune on an existing model pretrained with a large-scale natural image data set, such as ImageNet. However, this scheme does not achieve satisfactory performance for SAR applications because of the prominent discrepancy between SAR and optical images. In this article, we attempt to discuss three issues that are seldom studied before in detail: 1) what network and source tasks are better to transfer to SAR targets; 2) in which layer are transferred features more generic to SAR targets; and 3) how to transfer effectively to SAR targets recognition. Based on the analysis, a transitive transfer method via multisource data with domain adaptation is proposed in this article to decrease the discrepancy between the source data and SAR targets. Several experiments are conducted on OpenSARShip. The results indicate that the universal conclusions about transfer learning in natural images cannot be completely applied to SAR targets, and the analysis of what and where to transfer in SAR target recognition is helpful to decide how to transfer more effectively.

Deep learning algorithms are widely used in remote sensing image scene understanding. Generally, a large-scale annotated dataset is essential to train a deep neural network for classification. In practical terms, however, a large amount of unknown remote sensing images obtained from different sensors need to be understood which may vary from resolution, geolocation and imaging conditions compared with annotated datasets. In this paper, an unsupervised domain adaptation framework based on ResNet-18 is presented to transfer the knowledge of an existing annotated land cover dataset to other remote sensing data, decreasing the discrepancy among images across sensors. The results show a significant improvement in scene understanding of new remote sensing images.

Target recognition in synthetic aperture radar (SAR) images has become a hotspot in recent years. The backscattering characteristic of target is a significant issue taken into consideration in SAR applications. Almost all of the previous work focus on the scatter point extraction to depict the backscattering characteristic of the target; however, a point-target corresponds to a region rather than a single point due to the convolution during the imaging. Based on this fact, we first analyze the extent to how a point-target spreads, then propose a novel scatter cluster extraction (SCE) method, and utilize the scatter cluster as the feature to solve the airplane recognition problem in SAR images. In practice, there often exist interfering objects near the target to be classified. To overcome this issue, we design a reweighted sparse representation (RSR)-based automatic purifying method by assigning a weight to each element of the feature iteratively according to the representation error. Since the element with large representation error always corresponds to the interfering objects, we give it a small weight, consequently suppressing the influence of the interference. Experimental results demonstrate that the proposed SCE method outperforms the traditional scatter point extraction-based method as well as some state-of-the-art methods. The comparison result also validates the effectiveness of the proposed RSR method.