In the field of synthetic aperture radar automatic target recognition (SAR ATR), inherent distributional discrepancies between electromagnetic synthetic and measured SAR images pose significant challenges to the potential applications of the former. To bridge the gap, a novel unsupervised domain adaptation framework based on Multi-modal Feature fusion and global-local Joint Alignment (MFJA) is proposed in this article. The multi-modal feature fusion focuses on describing each target more comprehensively by leveraging both visual and scattering topological information. In the visual branch, the full-aperture image is decomposed into multiple sub-aperture images to explore the scattering variations of the target at different azimuths, facilitating a richer visual description. Meanwhile, both local scattering and spatial position information of keypoints are simultaneously integrated into the feature extraction in the scattering topological branch, promoting a more comprehensive scattering topological representation. Subsequently, a gated feature fusion module is developed to effectively fuse features derived from different modalities. The global-local joint alignment aims to align different domains with greater precision. Specifically, a power normalized weighted gradient reversal layer is proposed to guide the network to focus more on hard-to-align samples during global domain alignment, thus mitigating their interference with local domain alignment. While MFJA achieves satisfactory cross-domain recognition performance, its inference efficiency is somewhat constrained. Therefore, a domain-invariant cross-modal knowledge distillation (DCKD) algorithm with a tri-path collaborative alignment strategy is further developed to distill discriminative and domain-invariant knowledge from the multi-modal model into a compact visual model based on full-aperture images, thereby accelerating inference. Experiments conducted in three scenarios on the public Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset validate the effectiveness of both MFJA and DCKD.

Recently, metric-based meta-learning has been widely adopted to solve few-shot synthetic aperture radar (SAR) target classification, where the global features are employed to measure the similarity of each sample and the class prototypes. However, due to the high inter-class similarity of SAR images, the global features may suppress the detailed information in local features beneficial for SAR target classification. Besides, the prototypes obtained from a few SAR support samples at sparse azimuth angles tend to be biased. To tackle the above problems, we first integrate a local feature classification module into meta-learning and propose a multi-scale local classification network (MLC-Net) to enhance the target’s critical local detail features. Then, the feature generation and calibration network (FGC-Net) is proposed to generate SAR support features at the full range of azimuth angles to compensate for the real support features extracted by MLC-Net. Specifically, FGC-Net consists of a feature generative adversarial network (FGAN) and an adaptive feature calibration module (AFCM). FGAN is designed to generate support features for each class under the full range of azimuth angles. AFCM is proposed to calibrate the generated support features by adaptively re-weighting the generated and real support features of the same class. Finally, we devise the prototype mean square error (PMSE) loss and the central constraint (CC) loss to further narrow the distribution margin between the generated and real support features. FGC-Net and MLC-Net are unified for end-to-end training, resulting in consistent performance gains in feature generation and few-shot classification. Extensive experiments on the moving and stationary target acquisition and recognition (MSTAR) benchmark dataset under different few-shot settings demonstrate the effectiveness of our proposed method.

Image registration is the basis for joint utilization of multisource scene information. However, accurate automatic registration of multisource remote sensing images remains a challenging task, especially for optical and synthetic aperture radar (SAR) images. Due to the large geometric and intensity differences between images, many algorithms often fail to accurately register or even mismatch. In this article, we propose a novel coarse-to-fine method with stable high accuracy, which mainly consists of three steps. First, extract consistent features for robust coarse registration. Considering the differences in local gradient magnitudes between optical and SAR images, image intensity preprocessing is performed. Meanwhile, instead of the simple calculations via horizontal and vertical gradient operators, multidirectional gradient operators are defined in two scale spaces to obtain consistent local gradient information. Via multidirectional consistent gradients, highly repeatable keypoints are detected. On the multidirectional gradient maps, support regions with multiple scales are utilized to construct multiscale and multidirectional consistent cross-modal feature descriptors. Second, match the extracted features. Aiming to obtain a reliable alignment in coarse registration step, novel strategies are implemented for the first matching of a cascaded feature matching method. Sufficiently reliable initial matches are established via a new two-way matching strategy, and obvious outliers are removed by exploring the consistencies of both spatial scale and local neighborhood elements of the correct matches. Third, form distinctive pixelwise feature representations for accurate fine registration. In order to increase the distinctiveness of features to distinguish adjacent pixels, a new filter bank based on small receptive fields is defined. Fine features are constructed, appearing as thinner structures at the edges. Meanwhile, through multiscale and multidirectional convolutions, sufficient neighborhood information is mined. Therefore, more precise correspondences can be found to fine-tune the roughly corrected image pair. Overall, a combination method is proposed with a feature-based method and an area-based method for coarse registration and fine registration, respectively. On simulated and real image pairs, the above three steps and the two-stage framework are verified. Experimental results show that the proposed optical-to-SAR image registration method based on the designed multiscale, multidirectional consistent feature and multiscale, multidirectional fine feature (M2F2M) is superior to the current representative feature-based and area-based methods in robustness and accuracy.

In recent years, how to accurately identify targets from measured synthetic aperture radar (SAR) images according to electromagnetic synthetic SAR images is attracting more and more research interest. Most existing algorithms only focus on decreasing the domain differences in visual representations, while the special characteristics of SAR images are not explored enough. Besides, these algorithms tend to align only the overall distribution of synthetic and measured images, while domain shifts between subclasses are ignored. To solve these problems, a novel unsupervised domain adaptation framework named visual and scattering topological feature fusion and alignment network (VSFA) is proposed in this article. First, considering that visual features are crucial for recognition, image reconstruction is introduced to enhance the generalization of visual features. Second, by analyzing the imaging mechanism of SAR, we explore the differences in scattering topologies between synthetic and measured images for the first time. To measure the differences quantitatively, we model the non-Euclidean scattering topology of the target as graph data and introduce graph neural networks (GNNs) to extract scattering topological features. Moreover, in order to describe the scattering topology of the target more comprehensively, we introduce two different but complementary scattering topological point extraction algorithms and achieve their fusion at the feature level by GNN for the first time. Finally, a simple but effective two-stage domain adaptation loss is proposed to constrain the network to align the distribution of synthetic and measured images class by class. Benefiting from the simultaneous reduction of distribution differences in visual space and scattering topological space, the proposed method achieves 99.15% and 98.18% recognition accuracies in two typical experiment scenarios of the Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset, demonstrating its effectiveness.

Due to the difficulties of obtaining sufficient real synthetic aperture radar (SAR) images, introducing simulated images can effectively enrich the training dataset in SAR target recognition. This letter explores how to accurately identify the targets in real SAR images by only using simulated training images. The key challenge is that there are distribution differences between the simulated (training) and real (test) data, which limits the performance of recognition methods. To solve this problem, a hierarchical recognition method is proposed. A well-trained convolutional neural network (CNN) is first utilized to pre-classify all the test images. Then, the focus of our proposed method is to find the hard test samples that are easy to be misclassified according to the CNN classification confidence and re-classify them. In fact, the distribution of these samples is relatively inconsistent with the distribution of the training data. Thus, we propose a multi-similarity fusion (MSF) classifier to re-classify them by comprehensively measuring the correlation between the hard samples and the training images through five similarity measures. During the fusion process, the bagging ensemble technique is used and the similarity measures are sampled to generate different subsets to enhance the diversity of sub-classifiers, thus the performance is improved. A large number of experiments finally verify the robustness and accuracy of the proposed method.