Multimodal change detection (MCD) is a topic of increasing interest in remote sensing. Due to different imaging mechanisms, the multimodal images cannot be directly compared to detect the changes. In this article, we explore the topological structure of multimodal images and construct the links between class relationships (same/different) and change labels (changed/unchanged) of pairwise superpixels, which are imaging modality-invariant. With these links, we formulate the MCD problem within a mathematical framework termed the locality-preserving energy model (LPEM), which is used to maintain the local consistency constraints embedded in the links: the structure consistency based on feature similarity and the label consistency based on spatial continuity. Because the foundation of LPEM, i.e., the links, is intuitively explainable and universal, the proposed method is very robust across different MCD situations. Noteworthy, LPEM is built directly on the label of each superpixel, so it is a paradigm that outputs the change map (CM) directly without the need to generate intermediate difference image (DI) as most previous algorithms have done. Experiments on different real datasets demonstrate the effectiveness of the proposed method. Source code of the proposed method is made available at https://github.com/yulisun/LPEM.

Although remote sensing image registration has been developing at a high speed for decades, polarimetric synthetic aperture radar (PolSAR) image registration is still a challenging task because of the presence of polarimetric scattering differences, geometric distortions, and speckle noise. Due to the lack of PolSAR image training data, the generalization performance of deep learning-based registration methods is poor and cannot fundamentally solve the problem of PolSAR image registration. In this article, we propose a novel PolSAR image registration framework that integrates feature selection, feature descriptor extraction, and template matching. First, we use structural similarity (SSIM) to select polarimetric features that are similar in structural information, thus using structural information to overcome polarimetric scattering differences. On this basis, a feature descriptor named oriented gradient of polarimetric feature (OGPF) is proposed to overcome the polarimetric scattering information difference by extracting geometric structure information using oriented gradient channels (OGCs) and 3-D Gaussian convolution. Finally, we propose to use polarimetric whitening filter (PWF) and nonmaximum suppression (NMS) to extract keypoints with significant structural information to reduce the interference of speckle noise on keypoint selection, and further propose a template downsampling strategy to reduce the complexity of template matching. The proposed method is evaluated using six pairs of PolSAR images with different scenes, and the results show that its registration performance outperforms the state-of-the-art methods.

Change detection (CD) between heterogeneous images is an increasingly interesting topic in remote sensing. The different imaging mechanisms lead to the failure of homogeneous CD methods on heterogeneous images. To address this challenge, we propose a structure cycle consistency-based image regression method, which consists of two components: the exploration of structure representation and the structure-based regression. We first construct a similarity relationship-based graph to capture the structure information of image; here, a k -selection strategy and an adaptive-weighted distance metric are employed to connect each node with its truly similar neighbors. Then, we conduct the structure-based regression with this adaptively learned graph. More specifically, we transform one image to the domain of the other image via the structure cycle consistency, which yields three types of constraints: forward transformation term, cycle transformation term, and sparse regularization term. Noteworthy, it is not a traditional pixel value-based image regression, but an image structure regression, i.e., it requires the transformed image to have the same structure as the original image. Finally, change extraction can be achieved accurately by directly comparing the transformed and original images. Experiments conducted on different real datasets show the excellent performance of the proposed method. The source code of the proposed method will be made available at https://github.com/yulisun/AGSCC.

Change detection using multimodal remote sensing images is a very important and challenging topic. Due to the different imaging conditions, multimodal images cannot be directly compared to obtain changes. To address this challenge, in this letter we propose a change smoothness based signal decomposition (CSSD) model to decompose the post-event image into a regression image of pre-event image and a changed image. By establishing the pairwise relationship between the superpixels within the image, we construct a graph for the changed image and use the edge weights to measure the state consistency of connected vertexes, i.e., the probability of being both changed or unchanged. We show that the changed image is smooth on the constructed graph. In contrast to the previous methods that only use the prior change sparsity, we also exploit the change smoothness in the signal decomposition model, which makes the CSSD more robust and accurate, and outputs a better changed image, thus improving the change detection performance. Experimental results and comparisons with seven state-of-the-art methods on three datasets demonstrate the effectiveness of the proposed method.

This paper provides a new strategy for the heterogeneous change detection (HCD) problem: solving HCD from the perspective of graph signal processing (GSP). We construct a graph to represent the structure of each image, and treat each image as a graph signal defined on the graph. In this way, we convert the HCD into a GSP problem: a comparison of the responses of signals on systems defined on the graphs, which attempts to find structural differences and signal differences due to the changes between heterogeneous images. Firstly, we analyze the GSP for HCD from the vertex domain. We show that once a region has changed, the local structure of image changes,i.e. the connectivity of the vertex containing this region changes. Therefore, we can compare the output signals of the same input graph signal passing through filters defined on the two graphs to detect changes. We analyze the negative effects of changing regions on the change detection results from the viewpoint of signal propagation, and we also design different filters from the vertex domain to explore the high-order neighborhood information hidden in original graphs. Secondly, we analyze the GSP for HCD from the spectral domain. We explore the spectral properties of different images on the same graph, and show that their spectra exhibit commonalities and dissimilarities. Specifically, it is the change that leads to the dissimilarities of their spectra. With the help of graph spectral analysis, we propose a regression model for the HCD, which decomposes the source signal into the regressed signal and changed signal, and constrains the spectral property of the regressed signal. Experiments conducted on seven real data sets show the effectiveness of the vertex domain filtering based and spectral domain analysis based HCD methods. Source code will be made available at https://github.com/yulisun/HCD-GSP.

Change detection of heterogeneous multitemporal satellite images is an important and challenging topic in remote sensing. Since the imaging mechanisms of heterogeneous sensors are different, it is not possible to directly compare heterogeneous images to detect changes as in the homogeneous images. To address this challenge, we propose an unsupervised image regression-based change detection method based on the structure consistency. The proposed method first adaptively constructs a similarity graph to represent the structure of a pre-event image, then uses the graph to translate the pre-event image to the domain of the post-event image, and then computes the difference image. Finally, a superpixel-based Markovian segmentation model is designed to segment the difference image into changed and unchanged classes. The proposed adaptive structure consistency-based image regression model can not only alleviate the impact of noise and changed pixels on the regression process by using the structure-based transformation, but also easily distinguish between changed and unchanged classes in the difference image by using the prior sparse knowledge of changes. Experimental results on six different datasets demonstrate the effectiveness of the proposed method by comparing with some state-of-the-art methods.

Speckle noise inherent in synthetic aperture radar (SAR) images seriously affects the visual effect and brings great difficulties to the postprocessing of the SAR image. Due to the edge-preserving feature, total variation (TV) regularization-based techniques have been extensively utilized to reduce the speckle. However, the strong scatters in SAR image with radiometry several orders of magnitude larger than their surrounding regions limit the effectiveness of TV regularization. Meanwhile, the ℓ1-norm first-order TV regularization sometimes causes staircase artifacts as it favors solutions that are piecewise constant, and it usually underestimates high-amplitude components of image gradient as the ℓ1-norm uniformly penalizes the amplitude. To overcome these shortcomings, a new hybrid variation model, called Fisher-Tippett (FT) distribution-ℓp-norm first-and second-order hybrid TVs (HTpVs), is proposed to reduce the speckle after removing the strong scatters. Especially, the FT-HTpV inherits the advantages of the distribution based data fidelity term, the nonconvex regularization, and the higher order TV regularization. Therefore, it can effectively remove the speckle while preserving point scatters and edges and reducing staircase artifacts well. To efficiently solve the nonconvex minimization problem, an iterative framework with a nonmonotone-accelerated proximal gradient (nmAPG) method and a matrix-vector acceleration strategy are used. Extensive experiments on both the simulated and real SAR images demonstrate the effectiveness of the proposed method.

In this article, we propose a fast superpixel region merging algorithm for synthetic aperture radar (SAR) image segmentation. With our previously proposed adaptive superpixel generation approach (ALFCE), an initial over-segmentation superpixel map for SAR imagery can be obtained. A sketch edge map is used here to eliminate the mixed superpixels to refine the over-segmentation. Then, we focus on rapid superpixel merging for efficient and accurate SAR image segmentation by using the statistical region merging (SRM) framework. This article proposes a new merging order with the consideration of statistical dissimilarity measure and common boundary length penalty, as well as the homogeneity constraint for each superpixel pair. For the merging predicate, we define an adaptive merging threshold according to the image complexity, making the proposed superpixel merging no need to set any merging parameters in advance. Disjoint set is utilized in this article to map the superpixel pairs to pixel pairs for the sake of fast region merging, which has a low computation cost even with the increasing of superpixels. Experimental results on synthetic and real SAR images demonstrate that the segmentation precision of our proposed method can reach more than 85% and also superior to other state-of-the-art methods in terms of computational efficiency.

This work presents a robust graph mapping approach for the unsupervised heterogeneous change detection problem in remote sensing imagery. To address the challenge that heterogeneous images cannot be directly compared due to different imaging mechanisms, we take advantage of the fact that the heterogeneous images share the same structure information for the same ground object, which is imaging modality-invariant. The proposed method first constructs a robust K -nearest neighbor graph to represent the structure of each image, and then compares the graphs within the same image domain by means of graph mapping to calculate the forward and backward difference images, which can avoid the confusion of heterogeneous data. Finally, it detects the changes through a Markovian co-segmentation model that can fuse the forward and backward difference images in the segmentation process, which can be solved by the co-graph cut. Once the changed areas are detected by the Markovian co-segmentation, they will be propagated back into the graph construction process to reduce the influence of changed neighbors. This iterative framework makes the graph more robust and thus improves the final detection performance. Experimental results on different data sets confirm the effectiveness of the proposed method. Source code of the proposed method is made available at https://github.com/yulisun/IRG-McS.

Speckle noise is an inherent nature of synthetic aperture radar (SAR) images, which degrades the quality of the images and makes the interpretation of SAR images difficult. In this letter, we propose a despeckling method by simultaneously exploring low-rank prior and multi-scale prior of SAR images. Especially, we propose a low-rank minimization model by considering a data fidelity term derived from the Fisher-Tippett distribution and a weighted nuclear norm regularization term. Furthermore, we explore the multi-scale prior by selecting similar patches from different scales of the SAR image. The resulting optimization problem is solved by the alternating direction method of multipliers (ADMM). Experiments conducted on both simulated and real SAR images demonstrate that the proposed method can provide promising despeckling results in terms of speckle reduction and texture and edge details preservation.

This article proposes an efficient and adaptive statistical superpixel merging approach with edge penalty for polarimetric synthetic aperture radar (PolSAR) image segmentation. Based on the initial superpixel over-segmentation result obtained by our previously proposed adaptive polarimetric superpixel generation algorithm (Pol-ASLIC), this work achieves efficient and accurate PolSAR image segmentation by merging superpixels using the statistical region merging (SRM) framework. This article proposes to define a new dissimilarity measure between superpixels, which takes the edge penalty into consideration, leading to a reasonable and accurate merging order for superpixel pairs. With regard to the merging predicate of superpixels, a polarimetric homogeneity measurement (HoM) is used to define the merging threshold, making the merging predicate and merging threshold adaptive to the PolSAR image content. Experimental results on three airborne and one spaceborne PolSAR data sets demonstrate that the proposed approach can effectively improve the computation efficiency and segmentation accuracy in comparison with state-of-the-art merging-based methods for PolSAR data. More importantly, the proposed approach is free of parameters and easy to use.

Motivated by the idea of low-rank prior, we propose a despeckling method based on multi-scale nonlocal low-rank model. Specially, the proposed low-rank model consists of a data fidelity term derived from the Fisher-Tippett logarithmic-space speckle distribution and a weighted nuclear norm regularization term. Furthermore, we exploit a multi-scale prior by selecting similar patches from different scales of the SAR image. The resulting optimization problem is solved by the alternating direction method of multipliers (ADMM). Experiments conducted on one real SAR image demonstrate that the proposed method can achieve comparable and even better despeckling results than state-of-the-art SAR despeckling methods, both visually and quantitatively.

In this paper, we propose a new synthetic aperture radar (SAR) image despeckling method based on the nonlocal low-rank minimization model. First, some similar image patches are selected for each pixel to construct the patch group matrix (PGM). Then, a new low-rank minimization model, called Fisher-Tippett distribution (FT)-weighted nuclear norm minimization (WNNM), is proposed to recover the underlying low-rank component from the PGM. Specifically, the FT-WNNM is developed by reformulating the despeckling problem as the maximizing a posterior probability problem. The new model consists of a data fidelity term and a regularization term (also called prior term). The data fidelity term is derived from the statistical distribution of SAR images in the logarithm domain, which is known as the Fisher-Tippett distribution, and the regularization term is the recent weighted nuclear norm. Then, the alternating direction method of multipliers (ADMM) is introduced to solve the corresponding optimization problem. Under ADMM framework, the resulting subproblems can be solved efficiently and the convergence can be guaranteed. Extensive experiments on both simulated and real SAR images demonstrate that the proposed method can achieve comparable or even better despeckling performance than some state-of-the-art despeckling algorithms.

Due to the speckle noise and complex geometric distortions within SAR images, it is still a challenge to develop a stable method that can produce superpixels with both high boundary adherence and visual compactness with low computational costs at the same time. In this paper, we propose an adaptive superpixel generation approach with linear feature clustering and edge constraint for synthetic aperture radar (SAR) images, which consists of three stages. First, the local gradient ratio pattern of each pixel in SAR imagery is extracted as features, which was previously proposed by us for SAR target recognition and has been proven to be insensitive to speckle noise. Second, we propose to use the feature-ratio-based edge detector with Gauss-shaped window instead of the traditional rectangle-shaped window to obtain the edge strength map and final edges for SAR images. Finally, a modified normalized cut (Ncut)-based superpixel generation strategy is adopted using a distance metric that simultaneously measures both the feature similarity and space proximity. In this strategy, we approximate the similarity measure through a positive semidefinite kernel function rather than directly using the traditional eigen-based algorithm. Therefore, the objective functions of weighted local K-means and Ncuts can achieve the same optimum point by appropriately weighting each point in this feature space, which greatly reduces the computation cost. During the linear feature clustering, the coefficient of variation is used to automatically determine the tradeoff factor between the feature similarity and space proximity, which helps change the superpixel shape and size adaptively according to the image homogeneity. Furthermore, the edge information is also introduced to constrain the clustering for the sake of high boundary adherence. By bridging the local K-means clustering and Ncuts, as well as the benefits of edge constraint, our method not only produces superpixels with good boundary adherence but also captures the global image structure information. Experimental results with simulated and real SAR images demonstrate the effectiveness of our proposed method, which performs better than other state-of-the-art algorithms.

For synthetic aperture radar (SAR) image land cover classification, traditional feature-based methods are not always effective because of the heavy multiplicative noise. To solve this problem, we herein propose a new classification method for SAR images considering adaptive spatial contextual information. In contrast to preceding studies, the spatial contextual information of the SAR images is exploited via composite kernels (CKs). Additionally, an image superpixel strategy is employed to design an adaptive neighborhood, which enables the extraction of more accurate spatial information than a fixed-size neighborhood. Specifically, a modified superpixel map is first generated to produce the neighborhood. With this neighborhood, a context kernel is then defined by means of the Gaussian radial basis function. The resulting context kernel is combined with the conventional feature kernel via the designed CKs scheme. The relative proportion of these two kernels is controlled by a weight parameter. The label of each pixel is predicted by feeding the final CKs into a support vector machine classifier. Experiments on two real SAR images demonstrate that the proposed method can greatly improve the classification performance, both visually and quantitatively, in comparison to other traditional feature-based methods.