The remote sensing scene classification (RSSC) based on convolutional neural networks (CNNs) are generally limited by the complex background interference and the difficulty of identifying key targets in image. Thus, this letter proposes a semantic-aware fusion network for RSSC, abbreviated as SAF-Net, to better construct discriminative features and effectively fuse features for classification. The proposed SAF-Net, which employs the ResNet50 pretrained on the ImageNet dataset as the backbone network, mainly contains the semantic-aware module and the multilayer feature fusion module (MFFM). The semantic-aware module utilizes a spatial enhanced module (SEM) and a covariance channel attention module (CCAM) to accurately capture the discriminative semantic features. It can precisely identify and extract the essential semantic elements in image, such as distinct object types and their spatial distributions. Then, the MFFM uses the features learned by the semantic-aware module to guide other layers for effective feature fusion through a self-attention mechanism. It can not only enriches the feature representation of SAF-Net but also ensure the effective fusion of the semantic information. Extensive comparisons and ablation experiments on remote sensing datasets demonstrate the effectiveness of the proposed SAF-Net, and verify that it can greatly improve the classification performance.

At present, the research on the method of improving the obstacle avoidance ability of UAV mainly focuses on digital simulation and sensor-dependent technology. However, in complex and unpredictable environments, sensors may be affected by strong light, bad weather and other factors, resulting in ineffective obstacle avoidance and so on. Therefore, this paper proposes an obstacle avoidance method based on virtual-real combination technology, and constructs a virtual-real combination obstacle avoidance test platform, in the case of sensor failure, the test platform will control the UAV for obstacle avoidance operation. Therefore, the obstacle avoidance performance of UAV in complex environment can be greatly improved, and the purpose of safe and efficient flight can be realized.

Due to the significant differences in geometric and nonlinear intensity, multimodal image matching is still a challenging problem. To address this issue, this paper proposes a novel matching method using phase congruency (PC)-based self-similarity structural features for multimodal images. Firstly, the feature points are extracted from the PC maps of the original images by the Harris detector. Then, combined with the theory of the self-similarity, a PC-based self-similarity structural (PCSS) descriptor is designed for multimodal images. Finally, the Euclidean distance is used as the matching measure for the corresponding point recognition. Experimental results conducted on various real multimodal image pairs demonstrate that the proposed method can achieve better matching performance in terms of the number of correct matches and the registration precision in comparison with the traditional methods.

Deep learning algorithms have been used on a large scale in high-resolution remote sensing scene classification. However, traditional deep learning models usually suffer from incomplete consideration of spatial features, inadequate extraction of detail and texture features and difficulty in decoding deep features. In order to improve the extraction and generalization ability of convolutional neural networks for detail and texture features, a wavelet attention ResNeXt (WAResNeXt) is designed in this paper. The proposed WAResNeXt firstly extracts the multi-scale detail and texture information of the input feature map by wavelet transform, and then enhances the useful information and suppresses the redundant information by the attention mechanism. Finally, it reconstructs the feature map by the inverse wavelet transform. Experiments on the NWPU-RESISC45 dataset show that the WAResNeXt can effectively extract the spatial features and the texture features of high-resolution remote sensing images, and can greatly improve the scene classification accuracy.

High-dimensional features extraction and selection is of great significance for synthetic aperture radar (SAR) image change detection. In this paper, a feature selection based on sparse coefficient correlation, abbreviated as SR-PCC, is proposed to realize the local reconstruction of known samples, so as to improve the accuracy of change detection. Firstly, high-dimensional texture features are extracted from real SAR images and then fused by stacking. Secondly, for the known samples, the sparse representation is performed and then the sparse coefficients are obtained. Then, the Pearson correlation coefficient method is used to select sparse coefficients related to the image itself, thus realizing local optimal reconstruction. Finally, the selected features are inputted into the support vector machine (SVM) to realize change detection. Experiments on real SAR images demonstrate the effectiveness of the proposed SR-PCC in high-dimensional feature selection and illustrate that it can provide better change detection maps.

For better exploiting discriminative texture features and encoding high-level structures, this letter presents a high- order hybrid discriminative random field improved by two-layer random forest, abbreviated as RF-HoDRF, for synthetic aperture radar (SAR) image change detection. First, RF-HoDRF constructs a two-layer random forest (TL-RF) model to realize the selection of high-dimensional texture features, and then provides the class probabilities for constructing the unary potential in RF-HoDRF. Second, it defines a high-order potential on high-order cliques generated by superpixels to encode the high-level structures and maintain the region consistency. Finally, considering the pairwise potential by improved generalized Ising model and the statistics by generalized Gamma distribution (GΓD), the RF-HoDRF model is derived under the discriminative model framework. Then, by iteratively maximizing the local posterior probabilities, the class labels and the parameters are optimally estimated until they converge. Extensive comparisons and ablation experiments on measured SAR images verify the effectiveness of our method, and demonstrate that discriminative features selection and high-order structures maintenance have great contributions to improving change detection performances.

Deep learning has powerful feature extraction abilities and has achieved promising results in polarimetric synthetic aperture radar (PolSAR) image classification. However, the labeled samples of PolSAR images are generally limited, which could lead to the overfitting of deep networks and the inefficiency of deep features. To overcome this problem, in this article, we propose a complex-valued enforcing population and lifetime sparsity (CV-EPLS) model to extract nonredundant sparse features from PolSAR images. CV-EPLS achieves unsupervised learning of sparse polarimetric features with limited and unlabeled samples, including amplitude and phase information in multiple polarimetric channels. Concretely, CV-EPLS defines an activation metric function to achieve strong population sparsity. Additionally, a grid search strategy is designed to ensure that activation items are evenly distributed among the sparse targets, thus forming strong lifetime sparsity. In this way, CV-EPLS constructs the complex sparse matrices and extracts discriminative sparse features in an unsupervised way, with the dependence of features being effectively reduced. Experimental results on PolSAR images demonstrate the effectiveness of CV-EPLS in the extraction of features and its application to image classification.

To effectively fuse the high-dimensional features, in this letter, we propose the composite kernel and hybrid discriminative random field model, abbreviated as CK-hybrid discriminative random field (HDRF), for polarimetric synthetic aperture radar (PolSAR) image classification. In the CK-HDRF model, given high-dimensional features with different characteristics, the unary potential is constructed by relating multiple kernel k-means (MKKM) clustering to the traditional HDRF model. In this way, the high-dimensional decomposition and texture features can be well fused, thus making their deserved contributions to the inference of the attributive class and further increasing the discrimination capacity of CK-HDRF. The pairwise potential is constructed by the generalized Ising model with an additional edge penalty function, and thus, it can well capture the underlying spatial relationship and maintain the edge locations in classification. Moreover, the statistics of PolSAR data are modeled by the Wishart-generalized gamma (WG Γ) distribution. Experiments on real PolSAR images demonstrate the effectiveness of CK-HDRF in classification.

Aiming at exploiting the discriminative deep features and encoding the high-level structures, this letter presents a deep-features-based high-order triple discriminative random field model, abbreviated as DF-HoTDF, for nonstationary polarimetric synthetic aperture radar (PolSAR) image classification. First, the DF-HoTDF model extracts the discriminative deep features by a graph-based complex-valued 3-D convolutional neural network (CV-3-D-CNN) and then constructs the unary potential by a negative log function. Second, it introduces an auxiliary field u to explicitly regulate the nonstationary label patterns of the PolSAR image and then constructs a pairwise potential guided by u to capture greater pairwise label interactions. Third, it defines a high-order potential on high-order cliques to encode high-level structures. Finally, under the discriminative model framework, the DF-HoTDF model has a weighted fusion of the unary potential, the pairwise potential, and the high-order potential. Then, with the DF-HoTDF model, we iteratively optimize the class label and the stationary maps until they converge. The experimental results demonstrate that the proposed DF-HoTDF model is of superior performances in nonstationary PolSAR image classification and that it can provide better label consistency in homogeneous region and better target structures and edge locations in heterogeneous region.

Recently, convolutional neural network (CNN) has proved itself as a successful deep model and has been successfully utilized in polarimetric synthetic aperture radar (PolSAR) image classification. Most CNN-based models, however, concentrate on the correlation between the pixels and the labels in images and have fewer constraints on interclass or intraclass features. For fully utilizing the polarimetric data, we utilized complex-valued (CV) distance to learn the PolSAR features and proposed a PolSAR classification method by CV distance comparisons. First, we proposed a triplet CV network (TCVN) to learn the CV representations from PolSAR data by maximizing the interclass distance and minimizing intraclass distance. It uses the CV convolution and the CV Euclidean to maintain the phase components and applies the CV-dropout and CV$L_{2}$parameter regularization to reduce the overfitting and further improve the network performance. Subsequently, CV K nearest neighbor (CV-KNN) computes the distance of the CV representations and groups similar pixels. CV-KNN is well coupled with the TCVN because both of them are based on the Euclidean distance in the complex domain. Compared with the CNN-based methods, the proposed deep metric learning model can simultaneously extract the hierarchical features by comparing the polarimetric resolution cells in the complex domain and maintain the phase component by performing CV convolutions. The effectiveness and the superiorities of CV Euclidean distance in TCVN are demonstrated. Experiments on real PolSAR images illustrate that TCVN can deal with PolSAR data more effectively and achieve comparable performance in the PolSAR image classification even with a smaller data set.

Conditional random fields (CRFs) model is suitable for image segmentation because it can capture the dependencies of observed data and incorporate the spatial correlations into the segmentation process. In this article, to deal with the segmentation of nonstationary synthetic aperture radar (SAR) image, we combine the modeling power of the CRF model with the representation-learning ability of principal component analysis network (PCANet), and thus propose a high-order triplet CRF model based on PCANet (HOTCRF-PCANet). HOTCRF-PCANet introduces an auxiliary field to explicitly regulate nonstationary label structure patterns. Under the guidance of this auxiliary field, HOTCRF-PCANet defines a discrete quadrilateral nonstationary Markov fields model, and thus considers both the nonstationary property of image and high-order label interactions. In addition, guided by the auxiliary field, HOTCRF-PCANet proposes to use a product-of-expert (POE) potential to enforce the regions’ labeling consistency for pixels within the weak-structured region. To automatically learn rich feature representations, HOTCRF-PCANet modifies PCANet into an unsupervised mode, i.e., unsupervised PCANet (UPCANet), and constructs an UPCANet-based unary potential to effectively predict the local class probability. The effectiveness of HOTCRF-PCANet is demonstrated by the application to the unsupervised segmentation of the simulated images and real SAR images.

In this paper, we combine the modeling power of conditional random fields (CRF) model with the representation-learning ability of principal component analysis network (PCANet), and propose a high-order triplet CRF model, named as HOTCRF-PCANet, for unsupervised synthetic aperture radar (SAR) image segmentation. HOTCRF-PCANet introduces an auxiliary field to explicitly regulate label interactions of complex SAR image. In the label and auxiliary fields, HOTCRF-PCANet defines a discrete quadrilateral pairwise Markov fields (DQPMF) model, and thus constructs a high-order DQPMF potential to model the high-order label interactions in an unsupervised way. Additionally, HOTCRF-PCANet uses a product-of-expert (POE) potential to enforce the regions' labeling consistency for pixels within the weak-structured region. Moreover, HOTCRF-PCANet modifies PCANet into an unsupervised mode, i.e. UPCANet, automatically learns rich features of SAR image and constructs an UPCANet-based unary potential to predict the local class probability. The effectiveness of HOTCRF-PCANet is demonstrated by the application to the unsupervised segmentation of simulated and real SAR images.

Recently, convolutional neural network (CNN) has been successfully utilized in the terrain classification of polarimetric synthetic aperture radar (PolSAR) images. However, most CNN-based models are currently limited to handle 2-D real-valued inputs, and therefore, the physical scattering mechanism contained in the complex-valued (CV) covariance/coherency matrix cannot be extracted effectively. For this reason, CV 3-D CNN (CV-3D-CNN) is proposed for PolSAR image classification. Compared with CNN, CV-3D-CNN simultaneously extracts hierarchical features in both the spatial and the scattering dimensions by performing 3-D CV convolutions, thereby capturing the physical property from polarimetric adjacent resolution cells. Experiments on real PolSAR images classification demonstrate the effectiveness and the superiorities of CV-3D-CNN and illustrate that CV-3D-CNN can deal with scattering characteristic in a more complete manner and achieve better performance in PolSAR image classification.

The selection of training samples is important for the accuracy and efficiency of the synthetic aperture radar (SAR) image change detection task. However, training samples are traditionally extracted from the whole image, which leads to longer training time and an unbalanced number of pixels in the changed and unchanged classes. To overcome this problem, we propose a novel change detection method combining saliency detection with a principal component analysis network, named SDPCANet. To enhance the reliability of the training samples and reduce the amount of training samples, the SDPCANet uses context-aware saliency detection to obtain the salient region, from which the training samples are extracted. In addition, to alleviate the gap between the numbers of training samples in two classes, we regulate the candidate samples using the uniform-selecting strategy to enhance the reliability of the training samples for the SDPCANet. Then, the SDPCANet is trained with the extracted training samples and the remaining pixels are classified in the salient region to obtain the final change map. The experimental results on four sets of multitemporal SAR images demonstrate that the SDPCANet outperforms the reference methods proposed recently.

Dominant scattering mechanism (DSM) obtained by Freeman decomposition is significant for polarimetric synthetic aperture radar (PolSAR) image classification. To preserve the purity of scattering characteristics, it restricts pixels in a scattering category to be classified with other pixels in the same scattering category. However, due to the speckle and the limited image resolution, it is difficult to obtain the DSMs of some pixels, which are defined as the fuzziness of polarized scattering mechanisms. Therefore, we first consider a particular-and pertinent-auxiliary field, and then propose the fuzzy triplet discriminative random fields (FTDF) model to describe the fuzziness of polarized scattering mechanisms, thus categorizing the scattering mechanisms into four classes: surface scattering, double-bounce scattering, volume scattering, and mixed scattering. The pixels in the first three categories are with specific DSMs, and the FTDF model introduces an exponential kernel distance to combine the multiple features of PolSAR data into classification. For the pixels in the mixed scattering, FTDF introduces a fuzzy clustering algorithm regularized by Kullback-Leibler information to consider the fuzzy DSMs, thus enhancing the classification. Then the fuzziness modeling of polarized scattering mechanisms can guide the classification of PolSAR images. The experimental results on real PolSAR images demonstrate the effectiveness of the FTDF model, and illustrate that it can improve the classification accuracy, and simultaneously preserve the purity of scattering mechanisms.

The WGΓ model has been validated as an effective model for the characteristic of polarimetric synthetic aperture radar (PolSAR) data statistics. However, due to the complexity of natural scene and the influence of coherent wave, the WGΓ model still needs to be improved to fully consider the polarimetric information. Then, we propose the WGΓ mixture model (WGΓMM) for PolSAR data to maintain the correlations among statistics in PolSAR data. To further consider the spatial-contextual information in PolSAR image classification, we propose a novel mixture model, named mixture WGΓ-Markov random field (MWGΓMRF) model, by introducing the MRF to improve the WGΓMM model for classification. In each law of the MWGΓ-MRF model, the interaction term based on the edge penalty function is constructed by the edge-based multilevel-logistic model, while the likelihood term being constructed by the WGΓ model, so that each law of the MWGΓ-MRF model can achieve an energy function and has its contribution to the inference of attributive class. Then, the mixture energy function of the MWGΓ-MRF model has the fusion of the weighted component, given the energy functions of every law. The mixture coefficient and the corresponding mean covariance matrix of the MWGΓ-MRF model are estimated by the expectation-maximization algorithm, while the parameters of the WGΓ model being estimated by the method of matrix log-cumulants. Experiments on simulated data and real PolSAR images demonstrate the effectiveness of the MWGΓ-MRF model and illustrate that it can provide strong noise immunity, get smoother homogeneous areas, and obtain more accurate edge locations.

How to produce the difference data of the two temporal images is a crucial factor in image change detection. In this letter, we propose multicontextual mutual information data (MMID) based on the bivariate Gaussian distribution (BGD) for synthetic aperture radar (SAR) image change detection and illustrate their superiorities over the classical difference data. MMID, which are an improved form of image spatial mutual information, are constructed based on the quadrilateral Markov random field (QMRF) and can be factored into the linear combination of the entropies. Then to adapt MMID to the change detection, we construct the 2-D entropies based on the BGD. In this way, MMID are able to capture the intertemporal statistical dependence of the two temporal images and thus can be taken as the feature-level difference data rather than the pixel-level data. The maximum-likelihood method, the automatic threshold method, and the Markov random field method are performed on the MMID of the real two temporal SAR images for the change detection. Experimental results demonstrate the superiorities of MMID over the traditional difference data.

In this letter, we propose a hybrid conditional random field (HCRF) model for synthetic aperture radar (SAR) image change detection. The HCRF model is constructed by incorporating the statistics of the log-ratio image derived from the two-temporal SAR images into conditional random field model. In this way, it is able to integrate the SAR images information, including the texture features of the two-temporal SAR images, the statistics, and the spatial interactions of the log-ratio image, into the change detection. Moreover, to achieve the integration of the information, the HCRF model consists of three parts, namely, the unary potential, the pairwise potential, and the data term modeled by the statistics of the log-ratio image. The unary potential is modeled by a support vector machine using the texture features extracted from the two-temporal SAR images, and the pairwise potential is constructed by the multilevel logistical model to capture the spatial interactions of the log-ratio image. Generalized Gamma distribution (GΓD) is utilized to model the statistics of the intensity data in the log-ratio image. Finally, experimental results on three sets of two-temporal SAR images validate the effectiveness of the proposed HCRF model.

Statistical modeling for the statistics of polarimetric synthetic aperture radar (SAR) data is a critical factor in polarimetric SAR data processing. In this letter, we utilize the complex Wishart-generalized Gamma (WGΓ) distribution to model multilook polarimetric SAR data, in which the complex Wishart distribution and generalized Gamma distribution model the speckle and texture components, respectively. Moreover, we derive a closed-form expression for the WGΓ distribution based on the product model and propose a parameter estimation technique of the WGΓ distribution in this letter. We perform the experiments on the polarimetric SAR data acquired by the AIRSAR and ESAR to verify the superiority and effectiveness of the WGΓ distribution over the K and KummerU distributions in the goodness of fit of polarimetric SAR data histograms and the polarimetric SAR image classification. The experimental results demonstrate that the WGΓ distribution has a greater flexibility than the K and KummerU distributions in the statistical modeling of multilook polarimetric SAR data.