Sentiment is complex feedback of human perception of the outside world, which contains important potential information. Traditional sentiment analysis methods often use unimodal processing, which cannot accurately recognize complex emotional expressions. The existing multimodal sentiment analysis (MSA) methods based on feature fusion and post-fusion paradigms do not pay attention to the accuracy of the semantic expression of unimodal features and do not mine the correlation relationship between heterogeneous features, which leads to serious deviation of the fused multimodal sentiment. In this paper, a novel MSA method based on feature alignment and reconstruction constraints is proposed. The multimodal feature alignment module utilizes the cross-attention mechanism to establish the intrinsic connection between multimodal features and reduce the differences between multimodal heterogeneous features with the same sentiment. The multimodal feature reconstruction module is used to retain the unique semantics of unimodal features and reduce the loss of key information during heterogeneous feature alignment. Experimental results on the CH-SIMS v2.0 dataset show that the proposed method can significantly improve the model’s ability to cope with the recognition of complex sentiments.

Although remote sensing image registration has been developing at a high speed for decades, polarimetric synthetic aperture radar (PolSAR) image registration is still a challenging task because of the presence of polarimetric scattering differences, geometric distortions, and speckle noise. Due to the lack of PolSAR image training data, the generalization performance of deep learning-based registration methods is poor and cannot fundamentally solve the problem of PolSAR image registration. In this article, we propose a novel PolSAR image registration framework that integrates feature selection, feature descriptor extraction, and template matching. First, we use structural similarity (SSIM) to select polarimetric features that are similar in structural information, thus using structural information to overcome polarimetric scattering differences. On this basis, a feature descriptor named oriented gradient of polarimetric feature (OGPF) is proposed to overcome the polarimetric scattering information difference by extracting geometric structure information using oriented gradient channels (OGCs) and 3-D Gaussian convolution. Finally, we propose to use polarimetric whitening filter (PWF) and nonmaximum suppression (NMS) to extract keypoints with significant structural information to reduce the interference of speckle noise on keypoint selection, and further propose a template downsampling strategy to reduce the complexity of template matching. The proposed method is evaluated using six pairs of PolSAR images with different scenes, and the results show that its registration performance outperforms the state-of-the-art methods.

Precision agriculture management relies on the delineation of crop field edges. Multi-polarization SAR technology has the ability to penetrate clouds and capture morphological structures or moistures, suited for extracting crop field edges. Due to the time-dependent characteristics and phenological evolutions of crops, the methods with single-date data are difficult to detect complete edges. Moreover, the existing methods fail to extract the dynamic time-varying patterns, limiting the improvement of edge detection accuracy. Based on this, this paper proposes a novel crop field edge detection method based on the time-varying polarimetric characteristics. First, a spatial-temporal homogeneity measure is proposed to pre-identify the edge and homogenous area, for guiding the adaptive calculation of edge strength. Based on the time-series polarimetric stationarity and the trace moment estimation theory, the proposed measure enlarges the separating degree of various crop parcels. Second, a joint edge strength is proposed to enlarge strength contrast between edge and homogenous area. With the spatial-temporal homogeneity measure, it combines the similarity with the root mean square and the similarity with time-series average covariance matrix. Based on the advantages of two kinds of similarities, it highlights the field edges and reduces the impact of speckle noises. Evaluated by 8 quad-polarization and 14 dual-polarization SAR images, the proposed edge detection method achieves better visual presentations and detection accuracies than traditional methods. With the statistics of the signal-noise ratio (SNR), the joint edge strength also has higher strength contrast than conventional strengths. The relevant codes can be found in https://github.com/DawnHanGeo/TSPolEdge.git.

Polarimetric synthetic aperture radar (PolSAR) is an active microwave imaging system. Due to the coherence characteristic of PolSAR imaging, inherent coherent speckle noise exists in PolSAR images. The registration of PolSAR images is severely affected by speckle noise. Therefore, we first propose a joint filter that combines Refined-Lee filtering and polarimetric whitening filtering (PWF). The filter first applies Refined-Lee filtering to PolSAR images, which greatly reduces the speckle noise while maintaining high-resolution detailed information of the texture, and then uses PWF to normalize and whiten the polarimetric matrix to further limit the interference of speckle noise. After that, the binary robust invariant scalable keypoints (BRISK) algorithm is used to extract high-quality keypoints from the denoised PolSAR image. Then a novel Siamese Multiscale Attention Network (SMAN) is designed, which uses attention modules to construct feature descriptors with different scales. To fully utilize polarimetric information, we adopt the polarimetric covariance matrix and three polarimetric features as inputs to the network and the Second Order Similarity (SOS) as the loss function to train the network. In the keypoint matching stage, we present to use the symmetric displacement distance to further constrain the keypoint pairs obtained by the initial matching, which improves the accuracy of matching keypoint pairs. Experimental results show that our proposed method can effectively reduce the interference of speckle noise and overcome non-linear differences, geometric distortions, and differences in polarimetric scattering information to achieve accurate PolSAR image registration.

Sidelobe suppression is a crucial preprocessing technique for Synthetic Aperture Radar (SAR) image analysis. The presence of strong scattering targets generates sidelobes that can interfere the targets with relatively weak scattering. The overlapping of multiple strong cross-shaped sidelobes may even generate fake targets, significantly influencing the accuracy of SAR target detection and recognition. Among existing sidelobe suppression methods, the Spectral Reshaping Sidelobe Reduction (SRSR) method has shown promising results. It separates the mainlobe and sidelobes through altering the sidelobe direction while preserving the SAR image resolution. However, this method exhibits limitations in effectively suppressing strong cross-shaped sidelobes. It also introduces additional sidelobes, blurring the surroundings of the scattering points. This paper proposes an improved SRSR method to resolve this disadvantage. It constructs a feature image representing the superposition of sidelobes. This is achieved by analyzing the statistical differences of complex data between orthogonal and non-orthogonal sidelobe regions before and after spectral reshaping. Further modulus selection ensures that the feature image only contains the sidelobe information that needs to be eliminated. The proposed method successfully resolves the drawback of introducing new sidelobes in the original SRSR while achieving better suppression of strong cross-shaped sidelobes. Experimental results on airborne and spaceborne SAR images demonstrate that the proposed method outperforms other state-of-the-art techniques. Improved peak sidelobe ratio (PSLR) and integrated sidelobe ratio (ISLR) in both range and azimuth directions and smaller image entropy can be achieved by our method. Due to its superior sidelobe suppression capability, the SAR images processed by our method exhibit significantly improved accuracy in target detection.

When the geometric distortion of the SAR images to be registered is large, the spatial correspondence between the feature points of the two images will change significantly. Hence, the registration of SAR images with large geometric distortion is challenging. To solve this problem, a two-stage registration method of SAR images with large distortion based on superpixel segmentation is proposed in this paper. Firstly, the two SAR images are coarsely registered by geographic coordinate referencing. After coarse registration, superpixel segmentation is performed on the two SAR images respectively. Next, in the superpixel neighborhood of the reference image, we slide the corresponding superpixel template of the sensed image, finding its position with the highest similarity in the reference image. Compared with the traditional fixed-size template, the superpixel template can segment the distorted region more effectively. Meanwhile, with the help of the adaptive threshold detector proposed in this paper, the regions with varying degrees of distortion can be distinguished based on the similarity. Further, the geometry mapping relationship is calculated for the regions with different distortion degrees in the images respectively, and the corresponding feature points in different images are accurately matched to complete the fine registration. Finally, the registration results of regions with different degrees of distortion are fused to obtain the final SAR image registration results. Experimental results based on Sentinel-1 data show that the registration accuracy of the proposed method can reach within 1 pixel.

Ship detection in synthetic aperture radar (SAR) images is critical to ocean surveillance and rescue. Although many deep learning SAR ship detection methods have been proposed, the performance of these methods depends on the size and quality of the training samples. To resolve these issues, this letter presents an unsupervised ship detection method in SAR images using superpixel segmentation and cross stage partial network (CSPNet). First, the SAR image is over-segmented into superpixels based on our previously proposed superpixel generation algorithm. Then, the complex signal kurtosis (CSK) and a local superpixel contrast are integrated as a statistical indicator for the automatic identification of ship superpixels and background superpixels, thus leading to generation of training samples. Finally, the segmented superpixels are input to the CSPNet, which can learn a representative feature set with high discrimination ability between ships and backgrounds. Our method can achieve pixel-level detection map rather than the bounding box result. Experiments based on the Gaofen-3 and TerraSAR SAR data demonstrate that our method can achieve above 90% actual detection rate.

In this letter, we propose a fast superpixel-based clustering algorithm (FSC) for synthetic aperture radar (SAR) image segmentation. First, the SAR image is over-segmented into superpixels by our previously proposed edge-aware superpixel generation method with one iteration merging (ESOM). Second, based on the obtained superpixels, the number of clusters is automatically selected by the density peak (DP) algorithm and knee point method instead of manual specification. Finally, the modified$k$-means clustering with the generalized-likelihood ratio (GLR) dissimilarity is performed on the superpixels to generate the final segmentation result. Experimental results on two real SAR images show that the proposed method outperforms other state-of-the-art methods in terms of both segmentation accuracy and computational efficiency. Moreover, our method is free of clustering parameters and achieves automatic SAR image segmentation.

As flood occurs unpredictably, there is not enough time to label the data in practice. The use of clustering inside flood detection deep networks can reduce their demand for labeled data. However, existing clustering algorithms aim at assigning a unique cluster for each pixel. This leads to the fact that clustering process is non-differentiable to the inputs, hindering their incorporation into deep networks. In this study, we introduce a new assignment strategy for single-polarization SAR images to make the clustering differentiable, named “soft association.” Here, each pixel is assigned to various clusters with different probabilities. The greater the probability value, the more likely the pixel will be finally assigned to the cluster. Based on this, an end-to-end trainable semi-supervised clustering network for SAR flood detection is established. Compared with the existing state-of-the-art semi-supervised methods, it can achieve similar performance with fewer labeled samples.

Small target detection is a very challenging problem since a small target contains only a few pixels in size. At present, many deep learning-based detection algorithms for small targets have achieved remarkable results, mainly including improvements in data augmentation, multiscale images, multiscale features, training strategies, and so on. However, these deep learning-based methods cannot select the positive and negative samples for the large-difference-scale targets well in the label assignment operation. The reason is that intersection over union (IoU), which is widely used in the most target detection networks, has great limitations for small target detection. However, in practical applications, there are often some large-difference-scale targets in synthetic aperture radar (SAR) images, especially existing some tiny targets due to the limitation of resolution. To fundamentally break the limitations of IoU, we propose to use the Bhattacharyya distance (BD) instead of the IoU metric to improve the performance of small target detection. We further revise the Bhattacharyya distance (RBD) to better measure the deviation of bounding boxes for targets with large differences in size. RBD can embed anchor-based detectors to replace the IoU metric in label assignment and nonmaximum suppression (NMS). The proposed method is evaluated on the LS-SSDD-v1.0 dataset and the experimental results show that the proposed method outperforms the state-of-the-art methods.

Image matching is a primary technology to fuse the complementary information from optical and SAR images. Due to the high nonlinear radiometric and geometric relationship, the optical and SAR image matching task remains a widely unsolved challenge. In this study, we propose to use a Siamese convolutional neural network (CNN) architecture to learn pixelwise deep dense features. The proposed network is able to balance the learning of high-level semantic information and low-level fine-grained information, which is nonnegligible for feature matching task. Under the local searching framework, the loss function is defined based on the score map produced by the sum of squared differences (SSDs) between the learned pixelwise dense features of local optical and the SAR image patches, with a fast implementation in the frequency domain. The hardest negative mining strategy is adopted to increase the discrimination of the network. Extensive experiments are conducted on optical and SAR image pairs of different spatial resolution and different landcover types, verifying the superiority and robustness of the proposed method in terms of matching accuracy and matching precision.

Superpixel segmentation is essential to the rapid information extraction from synthetic aperture radar (SAR) imagery. In this letter, we propose a fast multiscale superpixel segmentation method based on the minimum spanning tree (MST), which can generate all scales of superpixels accurately in real time. Therefore, our method has the ability to segment SAR imagery with different scales efficiently and is meaningful for applications that require different levels of SAR image details. Experimental results on two real SAR images demonstrate that our proposed superpixel segmentation method can capture the image information of different levels, resulting in better hierarchical segmentation performance in comparison with other state-of-the-art methods.

Time-series polarimetric synthetic aperture radar (PolSAR) has been proven to be an effective technique for crop classification and agricultural activity monitoring. However, the characterization and utilization of time-series PolSAR data by existing methods are still inadequate. They are unable to extract and utilize time-varying features, which can describe the dynamic changes of crop polarimetric information. In this paper, we propose a tensor form to comprehensively describe the information of time-series PolSAR data, including spatial context information, polarimetric scattering information, and temporal context information. And we define a novel similarity value for the tensors (TSV), which can simultaneously consider distance and shape similarity of tensors. Then, we construct a tensor-based graph representation to capture the global similarity information of time-series PolSAR data. Finally, we propose a tensor-based graph convolutional network (Tensor-GCN) to extract deep features of graph node tensors for crop classification. Experimental results and analysis on two time-series PolSAR data firmly demonstrate the superiority of the proposed Tensor-GCN to other state-of-the-art methods.

Convolutional neural networks (CNNs) have demonstrated impressive ability to achieve promising results in PolSAR image classification. However, the traditional CNN performs convolution on local square regions with fixed sizes. The selection of these local square regions (patches) cannot fully take advantage of the boundary information of land covers and cannot search optimal neighborhoods in the whole image. To overcome these shortcomings, we propose a superpixel-based graph convolutional network (SP-GCN) for PolSAR image classification. SP-GCN utilizes superpixels as graph nodes, which makes full use of boundary information of superpixels and significantly reduces the computational cost of GCN, making it possible to apply GCN to large-scale PolSAR image classification. To reduce the impact of superpixel scale on classification results, we further propose a multiscale superpixel-based graph convolutional network (MSSP-GCN) based on the SP-GCN. Experimental results on three PolSAR datasets firmly demonstrate the superiority of the proposed SP-GCN and MSSP-GCN to other state-of-the-art methods.

The superpixel generation is a key step for object-based classification and change detection. For the time-series polarimetric synthetic aperture radar (PolSAR) superpixel generation, the traditional polarimetric similarity measure based on the joint covariance matrix has limitations in discriminating different time-series similarity sequences with different fluctuations. Besides, in the traditional time-series PolSAR superpixel generation methods, it is difficult to determine the tradeoff factor between polarimetric and spatial similarity. In this article, an adaptive time-series PolSAR superpixel generation method based on the simple local iterative clustering (SLIC) is proposed, named time-series polarimetric SAR (TSPol)-adaptive simple local iterative clustering (ASLIC). There are three main improvements. First, a novel time-series polarimetric similarity measure based on the root mean square (rms) is proposed. Multitemporal polarimetric statistical information is combined to describe the polarimetric proximity between pixels, referring to the rms of the multitemporal proximities. Second, an edge detection method based on the stacked 2-D Gaussian-shaped (s2-D GS) window is proposed to initialize the central seeds for superpixel generation. Third, an improved SLIC clustering similarity combined with the time-series polarimetric, time-series power, and spatial similarities is proposed. Meanwhile, a homogeneity factor is applied to adaptively balance the relative weights of various similarities. We use eight Radarsat-2 quad-polarization synthetic aperture radar (SAR) images and 14 Sentinel-1 dual-polarization SAR images to evaluate the effectiveness. The results show our similarity measure and superpixel generation results are superior to those of the traditional methods. For example, as for the Radarsat-2 data, the improvement of the boundary recall by the proposed similarity measure and homogeneity factor is about 4% and 10%, respectively.

In this article, we present a content-sensitive superpixel generation method with edge penalty and the contraction–expansion search strategy (EPCES) for synthetic aperture radar (SAR) images. Specifically, the edge information can be obtained by our previously proposed ratio-based edge detector with recurrent guidance filter, which has been proven to be robust to speckle noise and capable of detecting weak edges in low-contrast areas. The content-sensitive superpixel seeds’ initialization method is proposed with respect to the heterogeneous state of the SAR imagery, benefiting from which EPCES can generate an exact number of superpixels set by the user and the fine details can be preserved well. In EPCES, a new dissimilarity with edge penalty is defined to generate the superpixels with better edge adherence. Rather than adopting the conventional clustering method based on local$k$-means, we propose the contraction–expansion search strategy (CES), which explicitly utilizes the continuity information contained in neighboring pixels and enforces the connectivity of the superpixel without any postprocessing step. With the aid of the CES, our proposed method can attain superpixels with low computational cost and high edge adherence. Experimental results on both synthetic and real-world SAR images verify that the proposed method consistently performs favorably against several state-of-the-art methods in terms of both quality and efficiency.

In existing superpixel-wise segmentation algorithms, superpixel generation most often is an isolated preprocessing step. The segmentation performance is determined to a certain extent by the accuracy of superpixels. However, it is still a challenge to develop a stable superpixel generation method. In this article, we attempt to incorporate the superpixel generation and merging steps into an end-to-end trainable deep network. First, we employ a recently proposed differentiable superpixel generation method to over-segment the single-polarization synthetic aperture radar (SAR) image. It outputs the statistical likelihood that each pixel belongs to different superpixels. In superpixel merging part, as one of our main contributions, we propose a superpixel-wise statistical dissimilarity measure method for converting the soft superpixels set into a self-connected weighted graph. More importantly, inspired by the concept of the number of walks in graph theory, we define the$k$-order connectivity of each vertex. This definition can intelligently indicate the potential soft cluster centers and class assignments in graph. This merging method is differentiable, computationally simple, and free of empirical parameters. The superpixel generation and merging phases can be implemented under a unified deep network. The benefit is that our method can iteratively adjust the shapes of the superpixels according to the boundaries and segmentation results during training, until the satisfactory segmentation results are captured. Experimental results on real SAR images demonstrate that the segmentation precision of our proposed method is superior to other state-of-the-art methods in terms of precision and computational efficiency.

Since the number of superpixels is lower than that of pixels, superpixels can substantially speed up subsequent processing steps and have been widely used in synthetic aperture radar (SAR) image segmentation. However, in most of the existing superpixel-wise segmentation algorithms, superpixel prediction is an isolated preprocessing step and is independent of the segmentation task. The performance of the segmentation results is determined by the accuracy of superpixels. Once superpixels are generated, their shape cannot be changed in the following segmentation stage, even if the same superpixels contain pixels of different landcovers. To address this, we propose an end-to-end trainable superpixel-wise segmentation method for single-polarization SAR images. First, we design a differentiable boundary-ware clustering method for estimating task-specific superpixels. Instead of the hard association between pixels and superpixels in the existing superpixel algorithms, this method introduces the soft association map to make the clustering differentiable. Hence, it can be implemented using a simple deep fully convolutional network. In the segmentation part, we propose a novel soft graph convolution network (Soft-GCN), which takes the association map as input and performs superpixel-wise segmentation. The advantage of our method is that superpixel generation and graph convolution parts can be trained under a unified framework, until two parts obtain the optimum parameters. In the training process, it can adaptively adjust the shape of the superpixels according to the segmentation results, ensuring the superpixels correctly adhere the boundaries. Experimental results with simulated and real SAR images demonstrate that our method outperforms other state-of-the-art segmentation algorithms, while also being faster.

Due to the orientation variation and structural complexity, designing generalized and unified features to highlight manmade target scattering from polarimetric synthetic aperture radar (PolSAR) data is challenging. Inspired by the thought of mathematical programming (MP) and coupled with the model-based decomposition, this article proposes two polarimetric features: scattering contribution combiner (SCC) and scattering contribution angle (SCA) for unified scattering characterization of manmade targets. To this end, a rotated dihedral scattering model is first constructed concerning the analogous difference reciprocal and sigmoid function transformations, which adequately reflects the transition of co- and cross-pol responses caused by the orientation variation. Along with the dipole-like compound scattering models and through designing a discriminant-based model solution method, a fine eight-component decomposition using full polarimetric information is proposed. Through skillfully employing the MP strategies, the proposed decomposition achieves the physical optimization of scattering modeling and reasonable inversion of model parameters. Thus, it can accurately describe the local structure scattering and remarkably improve the overestimation of volume scattering. Subsequently, by analyzing the significance distribution on targets of different scattering mechanisms, the SCC is constructed via the linear/nonlinear combination of scattering contributions on the one hand. On the other hand, by further mining the information implied in the scattering contributions, the SCA is proposed with the strategy of trigonometric function transformation. Experimental results conducted on real PolSAR data not only demonstrate the effectiveness and superiority of the constructed features but also exhibit a clear advantage of fine polarimetric decomposition in scattering understanding, which encourages the use of them for further applications.

Automatic registration of optical and synthetic aperture radar (SAR) images is one of the most challenging tasks due to the influence of speckle noise and nonlinear radiation differences. In this article, we propose a compound registration method for optical and SAR images based on feature decoupling network (FDNet), which consists of a residual denoising network (RDNet) and a pseudo-Siamese fully convolutional network (PSFCN). First, we propose a fast compound matching algorithm, which can overcome the respective weaknesses of the registration accuracy and computational complexity of the feature-based and area-based methods. Specifically, FAST keypoint detection is used to generate the center of the initial template. The extraction of local feature descriptors and the matching of the initial templates are implemented by PSFCN. Second, we design an RDNet to learn the statistical model of speckle noise in SAR images and define a new loss function based on mean-square error (mse) and total variation (TV) to achieve the propagation of speckle noise. PSFCN and RDNet are used to learn deep representations of semantic and noise information, respectively. Then, the semantic and noise features are decoupled on spaced convolutional layers. Finally, the optimal matching templates are searched in a small search window around the initial matching templates. In addition, we propose a strategy for adaptively selecting the template size based on 2-D entropy, which can select the appropriate template size according to the content richness of SAR images. Registration results on a public registration dataset show that our proposed method achieves better performance than other state-of-the-art methods.

To fully explore the complementary information from optical and synthetic aperture radar (SAR) imageries, they need first to be coregistered with high accuracy. Due to the vast radiometric and geometric disparity, the problem to match high-resolution optical and SAR images is quite challenging. The present deep learning-based methods have shown advantages over the traditional approaches, but the performance increment is not significant. In this article, we explore a better network framework for high-resolution optical and SAR image matching from three aspects. First, we propose an effective multilevel feature fusion method, which helps to take advantage of both the low-level fine-grained features for precious feature location and the high-level semantic features for better discriminative ability. Second, a feature channel excitation procedure is conducted using a novel multifrequency channel attention module, which is able to make image features of different types and multiple levels effectively collaborate with each other and produce image matching features with high diversity. Third, the self-adaptive weighting loss is introduced, with which, each sample is assigned with an adaptive weighting factor, and therefore, information buried in all nearby samples can be better exploited. Under a pseudo-Siamese architecture, the proposed optical and SAR image matching network (OSMNet) is trained and tested on a large and diverse high-resolution optical and SAR dataset. Extensive experiments demonstrate that each component of the proposed deep framework helps to improve the matching accuracy. Also, the OSMNet shows overwhelming superior to the state-of-the-art handcrafted approaches on imageries of different land-cover types.

While traditional edge detectors concentrate on modifying the shape of the window function, we consider the edge detection problem from a new perspective, and an effective recurrent guidance filter is proposed in this letter. The proposed filter is elaborately designed for edge detection tasks and aims to remove the nonedge information including speckle noise and detailed texture and preserve edge information simultaneously. We first filter the image by the proposed filter and a filtered image is obtained. Then, by using the edge detector with the Gaussian-shaped window, which was previously proposed by us and performing the postprocessing method, the edge response is extracted from the filtered image. Both objective and subjective experimental results on simulated and real synthetic aperture radar (SAR) images demonstrate that the edge detector based on the recurrent guidance filter yields better performance than the state-of-the-art edge detectors.

Most of the existing superpixel generation methods are based on local iterative clustering. However, such methods have the following shortcomings: 1) these methods require several iterations and the number of iterations is difficult to determine and 2) the generated superpixel lacks explicit connectivity without a postprocessing step. Aiming to overcome the limitations, we propose an edge-aware superpixel generation with one iteration merging (ESOM) for synthetic aperture radar (SAR) imagery. In specific, we introduce a ratio-based edge detector with a Gaussian-shaped window to extract the edge information and an edge-aware dissimilarity is defined. Then, a new merging method termed as one iteration merging is proposed, which leverages the continuity of the adjacent pixels and ensures the connectivity of superpixel. Furthermore, instead of iterative clustering, the one iteration merging is achieved in only one iteration without determining the number of iterations and hence efficient in computation. Experiments on two real SAR images demonstrate that the proposed method yields substantially better performance than some state-of-the-art methods.

Remote sensing data can be sequentially acquired from different sources or feature spaces, which are regarded as multiple views. For the classification task where the training data arrive in a sequence, online learning (OL) methods are effective by learning new knowledge from incoming samples incrementally. However, it is known that shallow OL models usually have limited performance. In this letter, an online multiview deep forest (OMDF) architecture is proposed, which consists of multiple layers and employs a cascade structure. Each layer is an ensemble of multiple random forests, which process data from different views, respectively. For each view, the outputs of one layer concatenated with the original feature are fed into the next layer. The proposed method learns a deep forest model in an online manner from a stream of multiview data. The structure of every random forest and the weights adjusting the importance among different views will be updated dynamically. Experimental results on multifeature or multifrequency PolSAR data and the fusion of PolSAR and optical data demonstrate that the proposed method can achieve higher test accuracy and significantly improve the performance, especially on small-scale training data, compared with the other methods.

In this article, we propose a fast superpixel region merging algorithm for synthetic aperture radar (SAR) image segmentation. With our previously proposed adaptive superpixel generation approach (ALFCE), an initial over-segmentation superpixel map for SAR imagery can be obtained. A sketch edge map is used here to eliminate the mixed superpixels to refine the over-segmentation. Then, we focus on rapid superpixel merging for efficient and accurate SAR image segmentation by using the statistical region merging (SRM) framework. This article proposes a new merging order with the consideration of statistical dissimilarity measure and common boundary length penalty, as well as the homogeneity constraint for each superpixel pair. For the merging predicate, we define an adaptive merging threshold according to the image complexity, making the proposed superpixel merging no need to set any merging parameters in advance. Disjoint set is utilized in this article to map the superpixel pairs to pixel pairs for the sake of fast region merging, which has a low computation cost even with the increasing of superpixels. Experimental results on synthetic and real SAR images demonstrate that the segmentation precision of our proposed method can reach more than 85% and also superior to other state-of-the-art methods in terms of computational efficiency.

Speckle noise is an inherent nature of synthetic aperture radar (SAR) images, which degrades the quality of the images and makes the interpretation of SAR images difficult. In this letter, we propose a despeckling method by simultaneously exploring low-rank prior and multi-scale prior of SAR images. Especially, we propose a low-rank minimization model by considering a data fidelity term derived from the Fisher-Tippett distribution and a weighted nuclear norm regularization term. Furthermore, we explore the multi-scale prior by selecting similar patches from different scales of the SAR image. The resulting optimization problem is solved by the alternating direction method of multipliers (ADMM). Experiments conducted on both simulated and real SAR images demonstrate that the proposed method can provide promising despeckling results in terms of speckle reduction and texture and edge details preservation.

Subject to the statistical distribution assumption and the fixed window shape, the classical edge detectors for polarimetric synthetic aperture radar (PolSAR) images generally generate inaccurate results in heterogeneous scenes. In this letter, a PolSAR image edge detector integrating the scattering characteristics and optimal contrast (OC) is proposed. Hierarchical model-based decomposition is first implemented for the scattering mechanism characterization. On this basis, a scattering mechanism-driven adaptive window is then designed, which contains pixels with uniform polarimetric scattering. Finally, to avoid making the assumption, the OC measurement is adopted for the edge strength calculation. Experimental results conducted on different PolSAR data confirm the effectiveness of the proposed method and its superiority over the classical edge detectors, especially in heterogeneous areas.

This article proposes an efficient and adaptive statistical superpixel merging approach with edge penalty for polarimetric synthetic aperture radar (PolSAR) image segmentation. Based on the initial superpixel over-segmentation result obtained by our previously proposed adaptive polarimetric superpixel generation algorithm (Pol-ASLIC), this work achieves efficient and accurate PolSAR image segmentation by merging superpixels using the statistical region merging (SRM) framework. This article proposes to define a new dissimilarity measure between superpixels, which takes the edge penalty into consideration, leading to a reasonable and accurate merging order for superpixel pairs. With regard to the merging predicate of superpixels, a polarimetric homogeneity measurement (HoM) is used to define the merging threshold, making the merging predicate and merging threshold adaptive to the PolSAR image content. Experimental results on three airborne and one spaceborne PolSAR data sets demonstrate that the proposed approach can effectively improve the computation efficiency and segmentation accuracy in comparison with state-of-the-art merging-based methods for PolSAR data. More importantly, the proposed approach is free of parameters and easy to use.

In this article, we investigate the scattering components of ships and find that the surface scattering may be the primary scattering for some ships, especially small ships. Meanwhile, the drawbacks of the complete polarimetric covariance difference matrix [CP] are also pointed out in theory. Based on these analyses, two new methods are then constructed to detect the ships. More specifically, the first one RsP is constructed by directly combining the similarity parameter of surface scattering Rs and the power-maximization synthesis (PMS) detector. The second one RsDVH is designed by taking advantage of four different features (i.e., Rs, double-bounce scattering, volume scattering, and helix scattering), which are all derived from the joint polarimetric information that is developed by combing the information of the polarimetric covariance matrix [C] and [CP]. Subsequently, the generalized Gamma distribution (GΓD) is found suitable for characterizing the RsDVH values of the sea clutter. At last, an adaptive constant false-alarm-rate (CFAR) detector developed from RsDVH is proposed for ship detection. To verify the effectiveness of RsP and RsDVH, four polarization synthetic aperture radar (PolSAR) imageries are tested, including one L-band UAVSAR imagery with 19 ships, two L-band AIRSAR imageries with 22 and 53 ships, respectively, and one C-band GF-3 imagery with ten ships. The experimental results show that: 1) the surface scattering is beneficial to detecting ships, especially the ships with prominent surface scatterings; 2) compared with other state-of-the-art methods, RsDVH can more effectively enhance the target-to-clutter ratio (TCR) values of small ships in the case of rough sea surface; and 3) the joint polarimetric information that is put forward and exploited for the first time in this article has a greater potential to help ship detectors improve their detection performances than the traditional polarimetric information included in [C].

Motivated by the idea of low-rank prior, we propose a despeckling method based on multi-scale nonlocal low-rank model. Specially, the proposed low-rank model consists of a data fidelity term derived from the Fisher-Tippett logarithmic-space speckle distribution and a weighted nuclear norm regularization term. Furthermore, we exploit a multi-scale prior by selecting similar patches from different scales of the SAR image. The resulting optimization problem is solved by the alternating direction method of multipliers (ADMM). Experiments conducted on one real SAR image demonstrate that the proposed method can achieve comparable and even better despeckling results than state-of-the-art SAR despeckling methods, both visually and quantitatively.

This paper presents a new approach for the study of ship discrimination in complex sea ice ocean environment using single polarimetric synthetic aperture radar data and sub-spectra strategy. A statistic descriptor related to the signal coherence in the Time-Frequency domain, is proposed to enhance the ship/background contrast and improve discrimination capabilities. Using RADARSAT-2 single polarization data over complex sea ice scenes in Arctic ocean, experimental results demonstrate the efficiency of this method in terms of ship location retrieval and response characterization.

Ground-penetrating radar (GPR) has been widely used for antipersonnel mine (APM) detection. However, its efficiency is often impaired by high false alarm rate (FAR) caused by the ground clutters. In this letter, a novel robust principal component analysis (RPCA)-based method is proposed for fast prescreening of APM in GPR image. Taking advantage of low rank and sparse structure of GPR image, the proposed method first adopts an efficient RPCA technique—Go Decomposition (GoDec)—to extract the target image. Then, thresholds are applied to the extracted image to detect the target and reject false alarms. The proposed method enjoys two advantages over traditional methods: 1) the ability of reducing FAR while maintaining high probability of detection (PD) in strong noise and clutter environment and 2) the fast detection guaranteed by the modified GoDec that yields results within several iterations. Extensive simulations and laboratory experiments are conducted to validate the proposed method, and the results are satisfactory (high PDs up to 99% and low FARs).

In this paper, we propose a new synthetic aperture radar (SAR) image despeckling method based on the nonlocal low-rank minimization model. First, some similar image patches are selected for each pixel to construct the patch group matrix (PGM). Then, a new low-rank minimization model, called Fisher-Tippett distribution (FT)-weighted nuclear norm minimization (WNNM), is proposed to recover the underlying low-rank component from the PGM. Specifically, the FT-WNNM is developed by reformulating the despeckling problem as the maximizing a posterior probability problem. The new model consists of a data fidelity term and a regularization term (also called prior term). The data fidelity term is derived from the statistical distribution of SAR images in the logarithm domain, which is known as the Fisher-Tippett distribution, and the regularization term is the recent weighted nuclear norm. Then, the alternating direction method of multipliers (ADMM) is introduced to solve the corresponding optimization problem. Under ADMM framework, the resulting subproblems can be solved efficiently and the convergence can be guaranteed. Extensive experiments on both simulated and real SAR images demonstrate that the proposed method can achieve comparable or even better despeckling performance than some state-of-the-art despeckling algorithms.

Due to the speckle noise and complex geometric distortions within SAR images, it is still a challenge to develop a stable method that can produce superpixels with both high boundary adherence and visual compactness with low computational costs at the same time. In this paper, we propose an adaptive superpixel generation approach with linear feature clustering and edge constraint for synthetic aperture radar (SAR) images, which consists of three stages. First, the local gradient ratio pattern of each pixel in SAR imagery is extracted as features, which was previously proposed by us for SAR target recognition and has been proven to be insensitive to speckle noise. Second, we propose to use the feature-ratio-based edge detector with Gauss-shaped window instead of the traditional rectangle-shaped window to obtain the edge strength map and final edges for SAR images. Finally, a modified normalized cut (Ncut)-based superpixel generation strategy is adopted using a distance metric that simultaneously measures both the feature similarity and space proximity. In this strategy, we approximate the similarity measure through a positive semidefinite kernel function rather than directly using the traditional eigen-based algorithm. Therefore, the objective functions of weighted local K-means and Ncuts can achieve the same optimum point by appropriately weighting each point in this feature space, which greatly reduces the computation cost. During the linear feature clustering, the coefficient of variation is used to automatically determine the tradeoff factor between the feature similarity and space proximity, which helps change the superpixel shape and size adaptively according to the image homogeneity. Furthermore, the edge information is also introduced to constrain the clustering for the sake of high boundary adherence. By bridging the local K-means clustering and Ncuts, as well as the benefits of edge constraint, our method not only produces superpixels with good boundary adherence but also captures the global image structure information. Experimental results with simulated and real SAR images demonstrate the effectiveness of our proposed method, which performs better than other state-of-the-art algorithms.

For synthetic aperture radar (SAR) image land cover classification, traditional feature-based methods are not always effective because of the heavy multiplicative noise. To solve this problem, we herein propose a new classification method for SAR images considering adaptive spatial contextual information. In contrast to preceding studies, the spatial contextual information of the SAR images is exploited via composite kernels (CKs). Additionally, an image superpixel strategy is employed to design an adaptive neighborhood, which enables the extraction of more accurate spatial information than a fixed-size neighborhood. Specifically, a modified superpixel map is first generated to produce the neighborhood. With this neighborhood, a context kernel is then defined by means of the Gaussian radial basis function. The resulting context kernel is combined with the conventional feature kernel via the designed CKs scheme. The relative proportion of these two kernels is controlled by a weight parameter. The label of each pixel is predicted by feeding the final CKs into a support vector machine classifier. Experiments on two real SAR images demonstrate that the proposed method can greatly improve the classification performance, both visually and quantitatively, in comparison to other traditional feature-based methods.

This paper concerns two polarization orientation parameters of built-up areas derived from the polarimetric synthetic aperture radar (PolSAR) data considering two modeling orthogonal dihedral structures. The first orientation parameter is derived from the well-known circular polarization algorithm with the enrichment of arc distance median filtering using an adaptive neighborhood. The derivation of the second orientation parameter is realized by combining the slope-induced changes in polarimetric orientation angle with the shape-from-shading technique. The combination provides the possibility to measure the incidence angle and the azimuth component of the terrain slopes from the cross-pol SAR intensity image. With reference to the cross scattering model, a doubled cross scattering model (DCSM) is introduced by incorporating the orientation parameters, thus serving to guide the model-based decomposition. Using the DCSM refines the estimation of the cross-pol component by enabling us to further reveal the scattering characteristics of built-up areas. Following a novel criterion, the decomposition is implemented at two layers: one for urban areas and one for nonurban areas. The performance of parameter derivation is demonstrated and evaluated with airborne synthetic aperture radar, uninhabited aerial vehicle synthetic aperture radar, and GF-3 fully PolSAR data over different test sites. The decomposed results are consistent with the reference information provided by the National Land Cover Database 2011 about the land cover classification of test sites and encourage the use of the proposed decomposition scheme for different applications.

Detecting shallow buried antipersonnel mines (APMs) with a ground-penetrating radar (GPR) is a challenging task because of clutter contamination, which often obscures the APM response. In this letter, a novel method combining migration imaging with the low-rank and sparse representation method to suppress clutter and extract target image is presented. The proposed method first focuses and strengthens the target response with migration imaging. Then, since the focused target response and clutter, respectively, constitute the sparse component and the low-rank component of the recorded data, the recently proposed robust principal component analysis (RPCA) can be applied to the recorded data to separate the target response (sparse component) from the clutter (low-rank component). Numerical simulation and experiments with real GPR systems are conducted. Results demonstrate the effectiveness of the proposed method in improving signal-to-clutter ratio and retrieving geometrical information of the target, which permits a better APM identification in heavy clutter environment.

Simple linear iterative clustering (SLIC) algorithm was proposed for superpixel generation on optical images and showed promising performance. Several studies have been proposed to modify SLIC to make it applicable for polarimetric synthetic aperture radar (PolSAR) images, where the Wishart distance is adopted as the similarity measure. However, the superpixel segmentation results of these methods were not satisfactory in heterogeneous urban areas. Further, it is difficult to determine the tradeoff factor which controls the relative weight between polarimetric similarity and spatial proximity. In this research, an adaptive polarimetric SLIC (Pol-ASLIC) superpixel generation method is proposed to overcome these limitations. First, the spherically invariant random vector (SIRV) product model is adopted to estimate the normalized covariance matrix and texture for each pixel. A new edge detector is then utilized to extract PolSAR image edges for the initialization of central seeds. In the local iterative clustering, multiple cues including polarimetric, texture, and spatial information are considered to define the similarity measure. Moreover, a polarimetric homogeneity measurement is used to automatically determine the tradeoff factor, which can vary from homogeneous areas to heterogeneous areas. Finally, the SLIC superpixel generation scheme is applied to the airborne Experimental SAR and PiSAR L-band PolSAR data to demonstrate the effectiveness of this proposed superpixel generation approach. This proposed algorithm produces compact superpixels which can well adhere to image boundaries in both natural and urban areas. The detail information in heterogeneous areas can be well preserved.

The use of contextual information is beneficial to improve both the accuracy and reliability of image classification. Based on the robust fuzzy${c}$-means (RFCM) clustering method and an adaptive Markov random field model, this letter proposes a contextual${H}/{\bar {\alpha }}$classifier for polarimetric synthetic aperture radar images. At each iterative step of RFCM clustering, the prior probability extracted from the local neighborhood is combined with the fuzzy membership derived from inherent polarimetric characteristics, thus the enhanced fuzzy membership is more reliable. In addition, an adaptive smoothing factor is proposed for use during contextual information retrieval, which can prevent oversmoothing and preserve the local spatial details. The experimental results implemented using AIRSAR and ESAR L-band data validate the efficacy of the proposed method. Compared with the iterated Wishart classifier and fuzzy${H}/{\bar {\alpha }}$classifier, the proposed method significantly improves the classification accuracy, with less noise and increased preservation of details.

The classic constant false alarm rate edge detector with a rectangle-shaped filter has been proven to be effective and widely used in polarimetric synthetic aperture radar (PolSAR) images. However, in practical use, the assumption of complex Wishart distribution is often not respected, particularly in heterogeneous urban areas. In addition, as a simple smoothing filter, the rectangle-shaped window is often shown to be easy to incur false edge pixels near true edges. Therefore, its performance is limited. To overcome this restriction, we propose a new edge detector for PolSAR images, which utilizes the spherically invariant random vector product model to estimate the normalized covariance matrix for each pixel, and then replace the rectangle-shaped filter with a Gauss-shaped filter. The performance of our proposed methodology is presented and analyzed on two real PolSAR data sets, and the results show that the new edge detector attains better performance than the classic one, particularly for urban areas.

Cross-polarized scattering (HV) is not only caused by vegetation but also by rotated dihedrals. In this letter, we use rotated dihedral corner reflectors to form a cross scattering matrix and propose an extended model-based decomposition method for polarimetric synthetic aperture radar (PolSAR) data over urban areas. Unlike other urban decomposition techniques which need to discriminate between urban and natural areas before decomposition, this proposed method is applied directly on the PolSAR image. The building orientation angle is considered in this scattering matrix, making it flexible and adaptive in the decomposition process. This enables the separation of the cross scattering of urban areas from the overall HV component. The cross and helix scattering components are also compared in this study. RADARSAT-2 quad-pol C band and AIRSAR L band data are used to validate the performance of the proposed method. The cross scattering power of oriented buildings is generated, leading to a better decomposition result for urban areas with respect to other urban decomposition techniques.

The presence of multiplicative noise in synthetic aperture radar (SAR) images makes segmentation and classification difficult to handle. Although a fuzzy C-means (FCM) algorithm and its variants (e.g., the FCM_S, the fast generalized FCM, the fuzzy local information C-means, etc.) can achieve satisfactory segmentation results and are robust to Gaussian noise, uniform noise, and salt and pepper noise, they are not adaptable to SAR image speckle. This letter presents a kernel FCM algorithm with pixel intensity and location information for SAR image segmentation. We incorporate a weighted fuzzy factor into the objective function, which considers the spatial and intensity distances of all neighboring pixels simultaneously. In addition, the energy measures of SAR image wavelet decomposition are used to represent the texture information, and a kernel metric is adopted to measure the feature similarity. The weighted fuzzy factor and the kernel distance measure are both robust to speckle. Experimental results on synthetic and real SAR images demonstrate that the proposed algorithm is effective for SAR image segmentation.

Since superpixel takes spatial relationship between pixels into account, which makes the image classification process more understandable and the results more satisfactory, superpixel-based classification methods have been widely studied in recent years. However, due to speckle noise, traditional superpixel generating algorithms still have some drawbacks for synthetic aperture radar (SAR) image. In this letter, we propose a novel superpixel generating algorithm based on pixel intensity and location similarity (PILS) for SAR image. In addition, for the sake of image classification, features of Gabor filters and gray level co-occurrence matrix (GLCM) are extracted from each superpixel. The proposed superpixel generating method has the following three characteristics: (1) the terrain boundaries of SAR image are preserved well; (2) the method has more robustness against speckle noise; and (3) it has high computational efficiency. Experiments on synthetic and real SAR images demonstrate that our method significantly outperforms several state-of-the-art superpixel methods and PILS superpixel-based classification obtains better results than other pixel-based methods.