Deep learning-based change detection (CD) using remote sensing images has received increasing attention in recent years. However, how to effectively extract and fuse the deep features of bi-temporal images for improving the accuracy of CD is still a challenge. To address that, a novel adjacent-level feature fusion network with 3D convolution (named AFCF3D-Net) is proposed in this article. First, through the inner fusion property of 3D convolution, we design a new feature fusion way that can simultaneously extract and fuse the feature information from bi-temporal images. Then, to alleviate the semantic gap between low-level features and high-level features, we propose an adjacent-level feature cross-fusion (AFCF) module to aggregate complementary feature information between the adjacent levels. Furthermore, the full-scale skip connection strategy is introduced to improve the capability of pixel-wise prediction and the compactness of changed objects in the results. Finally, the proposed AFCF3D-Net has been validated on the three challenging remote sensing CD datasets: the Wuhan building dataset (WHU-CD), the LEVIR building dataset (LEVIR-CD), and the Sun Yat-Sen University dataset (SYSU-CD). The results of quantitative analysis and qualitative comparison demonstrate that the proposed AFCF3D-Net achieves better performance compared to other state-of-the-art methods. The code for this work is available at https://github.com/wm-Githuber/AFCF3D-Net.

Identifying feature correspondences between multimodal images is facing enormous challenges because of the significant differences both in radiation and geometry. To address these problems, we propose a novel feature matching method (named R2FD2) that is robust to radiation and rotation differences, which consists of a repeatable feature detector and a rotation-invariant feature descriptor. In the first stage, a repeatable feature detector called the Multi-channel Auto-correlation of the Log-Gabor (MALG) is presented for feature detection, which combines the multi-channel auto-correlation strategy with the Log-Gabor wavelets to detect interest points (IPs) with high repeatability and uniform distribution. In the second stage, a rotation-invariant feature descriptor is constructed, named the Rotation-invariant Maximum index map of the Log-Gabor (RMLG), which includes fast assignment of dominant orientation and construction of feature representation. In the process of fast assignment of dominant orientation, a Rotation-invariant Maximum Index Map (RMIM) is built to address rotation deformations. Then, the proposed RMLG incorporates the rotation-invariant RMIM with the spatial configuration of DAISY to improve RMLG’s resistance to radiation and rotation variances. Finally, we conduct experiments to validate the matching performance of our R2FD2utilizing different types of multimodal image datasets. Experimental results show that the proposed R2FD2outperforms five state-of-the-art feature matching methods. Moreover, our R2FD2achieves the accuracy of matching within two pixels and has a great advantage in matching efficiency over contrastive methods.

Due to the rapid development of deep learning techniques and the collection of large-scale remote sensing datasets, convolutional neural networks (CNNs) have made significant progress in remote sensing object detection. However, due to the diversity of objects in remote sensing images, multiscale object detection is still a challenging task. In this letter, a novel object detection framework based on feature pyramid network (FPN) is proposed to improve the detection performance of multiscale objects. First, a receptive field expansion block (RFEB) is designed and added on the top of the backbone to expand the receptive field of FPN adaptively. In this way, the context information around each object is well captured. Then, the features obtained via RFEB are delivered to feature maps at all pyramid levels, remedying the drawback of FPN that semantic information captured by deep layers is gradually diluted when transmitted to lower layers. Third, since the classic backbone of FPN, which produces large receptive fields based on large downsampling factors, may limit the effectiveness of RFEB, the backbone of the original FPN is modified using dilated convolution to ease the resolution drop of feature maps while maintaining a large receptive field. As a feature extractor, the proposed framework can be easily deployed in other FPN-based methods. The experiments on the benchmark for object DetectIon in Optical Remote sensing images (DIOR) dataset demonstrate the proposed method’s superiority over considered state-of-the-art baseline methods in terms of detection accuracy.

Recently, deep learning algorithms, especially feature pyramid network (FPN), have achieved significant progress in object detection of natural scene images. However, due to the complex scenes of remote sensing images and the diversity of remote sensing objects, FPN still faces the following drawback when applied to remote sensing object detection. Specifically, in the original FPN, the features of each proposal are extracted by RoIAlign. However, these features have limited effective receptive fields, making FPN lack of crucial contextual information to accurately classify and locate objects, as well as filter some background noises that possess similar appearance with objects. To alleviate the above problem, in this letter, we propose a gated context aware module (G-CAM), and replace the original RoIAlign in FPN with the proposed G-CAM to adaptively incorporate the useful local context surrounding each proposal and the global context of the whole image into FPN, enabling FPN to effectively detect objects in remote sensing images Extensive experiments have been conducted on the DIOR and RSOD datasets, which validates that the proposed method achieves superior performance to the considered state-of-the-art methods in terms of detection accuracy.

In this letter, a novel object detection method based on feature pyramid network (FPN) is proposed to improve the detection performance of remote sensing objects. First, since the information in the background regions may interfere with object detection, a novel multi-scale deformable attention module (MSDAM) is designed and added on the top of the backbone of FPN to make the network suppress the background features while highlight the target features. The proposed MSDAM generates attention maps from feature maps with multi-scale deformable receptive fields, thus can fit remote sensing objects of various shapes and sizes better and predict more precise attention maps for remote sensing images. Second, in the original FPN, each proposal is predicted based on feature grids pooled from only one feature level. This process is suboptimal as the information discarded in other feature levels and the global contextual information are also meaningful to object detection. Thus, a multi-level features aggregation module (MLFAM) is proposed to aggregate the multi-level outputs of FPN and the global context of the whole image, generating more powerful pyramidal representations for the subsequent object detection. The experiments conducted on the DIOR and RSOD datasets demonstrate the superiority of the proposed method over the considered state-of-the-art baseline methods in terms of detection accuracy.

Automatic and precise matching between optical and synthetic aperture radar (SAR) images is still a challenging task because of significant radiation and texture differences between such images. Recently, structure feature-based methods are popular for the matching of SAR and optical images. However, current structure descriptors include many noninformative features, which degrade their matching performance. To address that, we present a robust matching method by a multiscale masked structure feature representation. We first extract pixelwise gradient structure features on multiple scales of images. Then, a mask is constructed according to large contours of an image, which is used to increase the contribution of the main structure region and alleviate the influence of noninformative regions. Finally, a fast template scheme based on fast Fourier transform (FFT) is employed to obtain correspondences. The proposed method is tested using the optical and SAR images from the Sentinel and GaoFen sensors. Experiment results show that the proposed method significantly improves the matching performance compared with the state-of-the-art methods, especially for the images with poor structure features.

Image matching is a key preprocessing step for the integrated application of synthetic aperture radar (SAR) and optical images. Due to significant nonlinear intensity differences between such images, automatic matching for them is still quite challenging. Recently, structure features have been effectively applied to SAR-to-optical image matching because of their robustness to nonlinear intensity differences. However, structure features designed by handcraft are limited to achieve further improvement. Accordingly, this letter employs the deep learning technique to refine structure features for improving image matching. First, we extract multiorientated gradient features to depict the structure properties of images. Then, a shallow pseudo-Siamese network is built to convolve the gradient feature maps in a multiscale manner, which produces the multiscale convolutional gradient features (MCGFs). Finally, MCGF is used to achieve image matching by a fast template scheme. MCGF can capture finer common features between SAR and optical images than traditional handcrafted structure features. Moreover, it also can overcome some limitations of current matching methods based on deep learning, which requires solving a huge number of model parameters by a large number of training samples. Two sets of SAR and optical images with different resolutions are used to evaluate the matching performance of MCGF. The experimental results show its advantage over other state-of-the-art methods.

Hyperspectral anomaly detection (HAD) aims to distinguish anomalies from background-by-background modeling. Deep learning has been applied to HAD and achieves promising detection results. However, there exist several issues that need to be addressed: 1) unrealistic Gaussian assumption on the latent representations may limit its application; 2) deep features are not well-suited to anomaly detection due to the separation between feature learning and anomaly detection; 3) lack of adequate exploitation of spectral-spatial features; 4) negative effect caused by spectral band redundancy. In this article, we propose an end-to-end trainable deep one-class classification network for HAD. Specifically, a minimal enclosing hypersphere is trained to involve the deep features of background samples. These background samples are selected by a density clustering-based method. In this way, feature learning and anomaly detection are incorporated into a unified framework. Meanwhile, there is no explicit Gaussian assumption on the background features. Moreover, due to the complementarity of spectral and spatial features, a novel feature fusion strategy is proposed to fuse spectral and spatial features extracted by a two-stream deep convolutional autoencoder network. Finally, a band attention module is used to automatically learn small weights for redundant bands and thus reduce the negative effect caused by redundant bands. Experimental results on five public datasets demonstrate the superiority of the proposed method compared to several state-of-the-art HAD methods in the detection performance.

Due to the orientation variation and structural complexity, designing generalized and unified features to highlight manmade target scattering from polarimetric synthetic aperture radar (PolSAR) data is challenging. Inspired by the thought of mathematical programming (MP) and coupled with the model-based decomposition, this article proposes two polarimetric features: scattering contribution combiner (SCC) and scattering contribution angle (SCA) for unified scattering characterization of manmade targets. To this end, a rotated dihedral scattering model is first constructed concerning the analogous difference reciprocal and sigmoid function transformations, which adequately reflects the transition of co- and cross-pol responses caused by the orientation variation. Along with the dipole-like compound scattering models and through designing a discriminant-based model solution method, a fine eight-component decomposition using full polarimetric information is proposed. Through skillfully employing the MP strategies, the proposed decomposition achieves the physical optimization of scattering modeling and reasonable inversion of model parameters. Thus, it can accurately describe the local structure scattering and remarkably improve the overestimation of volume scattering. Subsequently, by analyzing the significance distribution on targets of different scattering mechanisms, the SCC is constructed via the linear/nonlinear combination of scattering contributions on the one hand. On the other hand, by further mining the information implied in the scattering contributions, the SCA is proposed with the strategy of trigonometric function transformation. Experimental results conducted on real PolSAR data not only demonstrate the effectiveness and superiority of the constructed features but also exhibit a clear advantage of fine polarimetric decomposition in scattering understanding, which encourages the use of them for further applications.

Recently, deep learning techniques have been introduced to address hyperspectral image (HSI) classification problems and have achieved the state-of-the-art performances. In this article, we propose a novel clustering algorithm for HSI based on learning embedding using the set-to-set and sample-to-sample distances (LSSDs). This technique consists of four main components: 1) oversegmentation; 2) generation of set-to-set and sample-to-sample distances; 3) learning embedding by training a siamese network; and 4) density-based spectral clustering. First, the HSI is oversegmented into superpixels by using the entropy rate superpixel (ERS) algorithm. Second, the set-to-set distances are obtained by representing the segmented sets of samples as affine hull (AH) models, whereas the sample-to-sample distances are computed by employing the local covariance matrix representation (LCMR) method. Third, sample pairs with the smallest and largest similarities are extracted according to the two distances. Then, these pairs are fed into the siamese multilayer perceptron (MLP) network and discriminative embeddings are learned by training the network with contrastive loss. Finally, density-based spectral clustering is applied to the deep embedding to obtain clustering results. Experimental results on three real HSIs demonstrate that the proposed method can achieve better performance than the considered baseline methods.

Efficient detection of targets immersed in a complex background with a low signal-to-clutter ratio (SCR) is very important in infrared search and tracking (IRST) applications. In this paper, we address the target detection problem in terms of local image segmentation and propose a novel small target detection algorithm derived from facet kernel and random walker (RW) algorithm which includes four main stages. First, since the RW algorithm is suitable for images with less noises, local order-statistic and mean filtering are applied to remove the pixel-sized noises with high brightness (PNHB) and smooth the infrared images. Second, the infrared image is filtered by the facet kernel to enhance the target pixels and candidate target pixels are extracted by an adaptive threshold operation. Third, inspired by the properties of infrared targets, a novel local contrast descriptor (NLCD) based on the RW algorithm is proposed to achieve clutter suppression and target enhancement. Then, the candidate target pixels are selected as central pixels to construct the local regions and the NLCD map of all local regions is computed. The obtained NLCD map is weighted by the filtered map of facet kernel to further enhance target. Finally, the target is detected by a thresholding operation on the weighted map. Experimental results on three data sets show that the proposed method outperforms conventional baseline methods in terms of target detection accuracy.

This paper presents a tensor alignment (TA) based domain adaptation (DA) method for hyperspectral image (HSI) classification. To be specific, HSIs in both domains are first segmented into superpixels, and tensors of both domains are constructed to include neighboring samples from a single superpixel. Then the subspace alignment (SA) between the two domains is achieved through alignment matrices, and the original tensors are projected as core tensors with lower dimensions into the invariant tensor subspace by applying projection matrices. To preserve the geometric information of original tensors, we employ a manifold regularization term for core tensors into the optimization process. The alignment matrices, projection matrices, and core tensors are solved in the framework of Tucker decomposition with an alternating optimization strategy. In addition, a postprocessing strategy is defined via pure samples extraction for each superpixel to further improve classification performance. Experimental results on four real HSIs demonstrate that the proposed method can achieve better performance compared with the state-of-the-art subspace learning methods when a limited amount of source labeled samples are available.

This paper introduces a novel heterogeneous domain adaptation (HDA) method for hyperspectral image (HSI) classification with a limited amount of labeled samples in both domains. The method is achieved in the way of cross-domain collaborative learning (CDCL), which is addressed via cluster canonical correlation analysis (C-CCA) and random walker (RW) algorithms. To be specific, the proposed CDCL method is an iterative process of three main components, i.e., RW-based pseudolabeling, cross-domain learning via C-CCA, and final classification based on extended RW (ERW) algorithm. First, given the initially labeled target samples as the training set (TS), the RW-based pseudolabeling is employed to update TS and extract target clusters (TCs) by fusing the segmentation results obtained by RW and ERW classifiers. Second, cross-domain learning via C-CCA is applied using labeled source samples and TCs. The unlabeled target samples are then classified with the estimated probability maps using the model trained in the projected correlation subspace. The newly estimated probability map and TS are used for updating TS again via RW-based pseudolabeling. Finally, when the iterative process converges, the result obtained by the ERW classifier using the final TS and estimated probability maps is regarded as the final classification map. Experimental results on four real HSIs demonstrate that the proposed method can achieve better performance compared with the state-of-the-art HDA and ERW methods.