In hyperspectral anomaly detection (HAD), tensor low-rankness is essential for effectively separating background and anomaly. However, most of the current low-rank-based methods do not use the spatial-spectral low-rankness and the nonlocal self-similarity simultaneously. To address this issue, we propose a tensor double nuclear norm-based tensor approximation (TDNN-TA) model with all the priors in a unified convex framework, which can be efficiently handled through a well-organized alternating direction method of multipliers. Especially, to thoroughly model the background by tensor approximation, we propose a tensor double nuclear norm (TDNN), which achieves a more precise and flexible exploration of low-rankness and nonlocal self-similarity by applying different low-rank constraints to the global tensor and the group tensor. Moreover, to explore the intrinsic characteristics of different priors by various tensor ranks, we employ the Fourier transform-based three-directional tensor nuclear norm to approximate the nonlocal group tensor rank, and the framelet-based three-modal tensor nuclear norm to approximate the global tensor rank. Experimental results validated on several real hyperspectral datasets demonstrate that TDNN-TA is effective in detecting different sizes of anomalous targets and achieves competitive results for various scenes.

With the advancement of spectral imaging technology, hyperspectral image (HSI) resources have been rapidly expanding, making cross-dataset HSI classification a critical technique and an inevitable trend for large-scale Earth observation applications. However, most existing approaches transfer from one HSI to another in a supervised way, which limits their ability to leverage multi-source HSI information. Therefore, this paper proposes an unsupervised cross-dataset HSI classification method based on parallel adversarial domain adaptation (PADA), which learns and integrates task-relevant and domain-invariant knowledge from multi-source HSIs to classify a target HSI. Specifically, a source-and-target shared information mining module is designed to mine transferable knowledge from each source HSI. This module employs parallel adversarial learning between a spectral-spatial feature extractor and a task-relevant controller with a domain-invariant discriminator to learn task-relevant and domain-invariant features, thereby mitigating domain shift. Moreover, a source-to-target transferability learning module is proposed to evaluate the cross-dataset transferability of knowledge, which computes domain correlation score to evaluate inter-domain correlation and guide adaptive knowledge transfer, effectively suppressing redundant information and reducing negative transfer caused by low-correlated sources. Finally, a multi-source collaborative classification module is developed to accomplish cross-dataset knowledge transfer, which designs correlation-aware fusion strategy to produce the final classification result by integrating information from each source, ensuring balanced and robust decision-making. Extensive experiments conducted under challenging multi-source cross-dataset settings validate the classification performance and the domain extensibility of the proposed method, demonstrating its potential for large-scale Earth observation applications.

Hyperspectral target detection (HTD) identifies objects of interest from complex backgrounds at the pixel level, playing a vital role in Earth observation. However, the limited target priors constrain the ability to obtain sufficient features or patterns for background-target discrimination, and spectral variation further exacerbates the difficulty of achieving reliable and robust performance. To address these challenges, this article proposes an efficient self-supervised HTD method with a pyramid state space model (SSM), named HTD-Mamba, which employs spectrally contrastive learning to distinguish between target and background based on the similarity measurement of intrinsic features. Specifically, to obtain sufficient training samples and leverage spatial contextual information, we propose a spatial-encoded spectral augmentation (SESA) technique that encodes all surrounding pixels within a patch into a transformed view of the center pixel. In addition, to explore global band correlations, we divide pixels into continuous group-wise spectral embeddings and introduce Mamba to HTD for the first time to model long-range dependencies of the spectral sequence with linear complexity. Furthermore, to alleviate spectral variation and enhance robust representation, we propose a pyramid SSM as a backbone to capture and fuse multiresolution spectral-wise intrinsic features. Extensive experiments conducted on four public datasets demonstrate that the proposed method outperforms state-of-the-art methods in both quantitative and qualitative evaluations. The code is available athttps://github.com/shendb2022/HTD-Mamba.

As Whole Slide Images(WSI) are characterized by their extremely high resolution and lack of pixel-level annotation, Multiple Instance Learning(MIL) has become an effective tool for addressing WSI classification challenge. However, current MIL methods typically rely on the independent and identically distributed assumption, neglecting correlation among different instances and only analyze WSI at a single scale, failing to effectively utilize the complementarity between multi-scale features. To enhance classification performance and obtain more accurate diagnostic results, we propose a MIL model based on multi-scale Graph-Transformer framework. It utilizes graph convolutional network to capture spatial, semantic, and scale correlations among patches, and performs graph pooling to select discriminative patches in order to reduce computational costs. Additionally, we introduce a multi-scale feature fusion module aimed at integrating features across different scales. The experimental results on the TCGA-NSCLC and TCGA-RCC datasets show that our method achieves consistent high performance for different cancer types, obtaining accuracy of 0.9281 and 0.9508 on the above two datasets, respectively. Compared with other comparison methods, our method has surperior classification performance.

Human gesture recognition, due to its indispensable role in a myriad of emerging applications, has attracted close attention in the visual and wireless sensing community. The intrinsic characteristics of visual and wireless modalities are complementary to each other, e.g., wireless signals are robust to illumination changes and occluded conditions but suffer from low-spatial resolution, while visual signals have a high-spatial resolution but are vulnerable to scenario variations. Intuitively, integrating them has the potential chance to improve the overall discriminative ability. However, due to their different physical patterns and semantics, how to explore the relationship between the two modalities and leverage their complementary information to improve recognition performance still remains unsolved. In this article, we propose a complete representation projection method, which projects the signals from the heterogeneous modalities into a complete representation feature space by performing a bidirectional projection constraint. Furthermore, to relieve the low-projection efficiency problem caused by the heterogeneity of the two modality features, we propose an attention-based cross-modality interaction (ACMI) mechanism to perform implicit semantic feature alignment, thereby better capturing the complex dependencies between the two modalities and improving the feature projection efficiency. To evaluate the proposed method, we build a visual-radar cross-modality gesture recognition system and conduct extensive experiments. Experimental results demonstrate that the proposed approach not only performs favorably against vision-only and wireless-only solutions by a large margin, but also shows superiority over traditional fusion solutions.

Human gesture recognition provides great potentials in Human Computer Interaction (HCI), and the wireless or visual signals based technologies have been explored in their respective fields. The intrinsic characteristics of both modalities are complementary to each other, e.g. the wireless signal is robust to illumination changes and occluded conditions but suffers from low space resolution, while the visual signal has high space resolution but vulnerable to scenario variations. Intuitively, integrating the two modalities has potential chance to improve the overall discriminative power. However, existing multi-modal fusion methods could not fully exploit their complementarity to achieve accurate estimation, and also lack physical interpretability. In order to solve this issue, we introduce WiVi-GR: a Wireless-Visual joint representation based accurate Gesture Recognition system, which constructs a complete velocity representation to guarantee robust and accurate gesture recognition. Specifically, we analyze the complementarity of the two modalities in data dimension and spatial-temporal feature resolution, and propose an Interpretable Orthogonal Representation (IOR), which applies multi-channel coding to get image plane velocity, utilizes frequency domain analysis to get radial velocity, and aggregates both to achieve the complete representation of the dynamic pattern. Based on the IOR, we perform a data-level fusion with channel superposition convolutions to accomplish the accurate gesture recognition task. Experimental results show that the proposed WiVi-GR outperforms traditional multi-modal approaches by large margins, especially in small training sample set condition.

As one of the most important application for remote sensing monitoring, land cover mapping has witnessed notable advancements in data acquisition, algorithmic diversity, and classification accuracy. Despite the instrumental role data-driven algorithms have played in the development of global land cover products, their inherent limitations as “black box” methods often fall short of meeting end-users’ specific requirements. In this study, built upon the foundation of the earlier land cover monitoring platform [FROM-GLC plus(FGP)], a data and knowledge dual-driven framework (FGP 2.0) was developed as a user-adaptive framework for intelligent remote sensing land cover mapping. By incorporating ontology-based semantic descriptions with advanced data-driven algorithms, FGP 2.0 provides the capacity for both traditional annual mapping and emerging dynamic mapping. Our results illustrate that FGP 2.0 significantly improves the overall accuracy of annual maps by ~5%, and dynamic maps by ~20% compared to FGP. Moreover, an operational dynamic mapping tool has been developed on the Google Earth engine (GEE), enabling the generation of near-real-time land cover maps for any given place. With an extensible and flexible mapping framework, FGP 2.0 demonstrates the potential of customized land cover monitoring results to suit different application scenarios. This innovative approach not only meets the current demand for reliable annual and dynamic land cover maps but also sets a new benchmark for the integration of geoscientific expertise with machine learning techniques in remote sensing monitoring.

Hyperspectral target detection (HTD) is an important issue in earth observation, with applications in both military and civilian domains. However, conventional representation-based detectors are hindered by the reliance on the unknown background dictionary, the limited ability to capture nonlinear representations using the linear mixing model (LMM), and the insufficient background-target recognition based on handcrafted priors. To address these problems, this paper proposes an interpretable representation network that intuitively realizes LMM for HTD, making nonlinear feature expression and physical interpretability compatible. Specifically, a subspace representation network is designed to separate the background and target components, where the background subspace can be adaptively learned. In addition, to further enhance the nonlinear representation and more accurately learn the coefficients, a lightweight multi-scale Transformer is proposed by modeling long-distance feature dependencies between channels. Furthermore, to supplement the depiction for target-background discrimination, a constrained energy minimization (CEM) loss is tailored by minimizing the output background energy and maximizing the target response. The effectiveness of the proposed method is demonstrated on four benchmark datasets, showing its superiority over state-of-the-art methods. The code for this work is available at https://github.com/shendb2022/HTD-IRN for reproducibility purposes.

Device-free wireless sensing (DFWS) has drawn lots of attention due to its potential application in the fields of human–computer interaction and smart home. Deep networks-based DFWS technique has achieved excellent sensing performance. However, network complexity limits its deployment on resource-limited sensing devices. A feasible way is to implement a simple network to accomplish the DFWS task. However, the sensing performance will drop dramatically due to its limited learning ability. In this article, to realize lightweight DFWS with acceptable performance, we propose an information-for-complexity strategy to promote the learning ability of the simple network. We leverage knowledge distillation framework to explore external information to augment the extrinsic learning ability, and utilize multiscale receptive fields to explore the internal information to augment the intrinsic learning ability. Extensive experiments on a 77 GHz mmWave testbed show that the performance degradation of the lightweight DFWS system is within 3%, while the complexity decreases remarkably.

The problem of target detection in hyperspectral images is an unsupervised binary classification problem with extremely uneven samples. To highlight the target and suppress the background as much as possible, this paper proposes a target detection algorithm based on dual sparse constraints. Specifically, the original image can be decomposed into a background image and a target image. Combined with sparse representation, the target detection problem can be transformed into a problem of optimizing the target and background coefficient matrices. This problem can be solved by the alternating direction method of multipliers. Considering that both the target dictionary and background dictionary are unknown, this paper also proposes a dictionary construction algorithm based on spectral similarity and clustering to obtain relatively complete and pure target and background dictionaries. The experimental results demonstrate that the proposed method outperforms other competing methods in both quantitative performance and visual effect. Code and datasets are available at https://github.com/shendb2022/DSC.

Change detection, which gives a quantitative analysis of the change information for the target area, is an important technology for lots of remote sensing tasks. Supervised change detection methods can achieve satisfactory performance when there are enough labeled samples, which is a harsh requirement for change detection tasks using remote sensing images. To solve this problem, we propose a change detection method based on semi-supervised segmentation and adversarial learning. The proposed method can learn knowledge from both the limited labeled samples and the abundant unlabeled samples to improve the generalization performance of the model. Firstly, the segmentation maps of both labeled samples and unlabeled samples are obtained through a segmentation network. Then, the labeled samples is used to train the discriminator network, which is responsible for distinguishing the segmentation prediction from the ground truth. Finally, the discriminator output is used as a measurement for self-training to minimize the feature difference between the segmentation prediction and the ground truth. Experimental results on the Sun Yat-Sen University (SYSU) dataset show that the proposed method can use unlabeled samples to improve the quality of prediction, which guarantees the proposed method can achieve good performance with few labeled samples.

Hyperspectral image has been widely used in the field of remote sensing due to the rich spectral and spatial information. During land-cover investigation, new types of ground objects appear constantly, which need the classification model to make quick judgments. However, a trained hyperspectral classification model can only make predictions on pre-defined classes, which limits its application. In order to deal with the aforementioned issue, we propose an incremental learning method based on constantly updated classifier, which is able to recognize new classes by few-shot samples and classify old classes without storing any old samples. Specifically, we propose a decoupled structure of feature representation module and classifier module, the feature representation is trained in the initial stage and then frozen, while the classifier module is updated in the next incremental stages, which alleviates knowledge forgetting as well as over-fitting. Besides, we adopt attention mechanism with a kernel of cosine distance to measure the similarity between prototypes and test samples of each class, which is more robust with few-shot samples. Extensive experiments and analyses based on typical hyperspectral images verified the effectiveness of the proposed method.

Ship target detection in synthetic aperture radar (SAR) images is essential for many applications in marine monitoring and port security. Though considerable developments have been achieved, there still exist some issues toward multiscale and dense ship targets in complex inshore scenes. Under such common but challenging situations, it is difficult to extract effective target information, which drives the missing alarm rate rising dramatically. In complex scenes, it is hard to disentangle background noise from ship target information, which causes false alarm frequently. In this article, an anchor-free SAR ship detection method based on key-point estimation and attention mechanism is proposed to address the aforementioned issues. Specially, an anchor-free framework with skip connections and aggregation nodes is designed to fuse multiresolution features and detect multiscale ship targets. Moreover, a key-point estimation module is proposed to eliminate the undetected ship targets caused by dense target distribution. Furthermore, a channel attention module is explored to enhance network attention on ship targets and suppress background noise. Sufficient experimental results on the open SAR ship detection dataset demonstrate that compared with some state-of-the-art methods, the proposed method is able to achieve higher detection accuracy with a lower false alarm rate.

The visual loop-closure detection for autonomous underwater vehicles (AUVs) is a key component to reduce the drift error accumulated in simultaneous localization and mapping tasks. However, due to viewpoint changes, textureless images, and fast-moving objects, the loop closure detection in dramatically changing underwater environments remains a challenging problem to traditional geometric methods. Inspired by strong feature learning ability of deep neural networks, we propose an underwater loop-closure detection method based on a variational autoencoder network in this article. Our proposed method can learn effective image representations to deal with the challenges caused by dynamic underwater environments. Specifically, the proposed network is an unsupervised method, which avoids the difficulty and cost of labeling a great quantity of underwater data. Also included is a semantic object segmentation module, which is utilized to segment the underwater environments and assign weights to objects in order to alleviate the impact of fast-moving objects. Furthermore, an underwater image description scheme is used to enable efficient access to geometric and object-level semantic information, which helps to build a robust and real-time system in dramatically changing underwater scenarios. Finally, we test the proposed system under complex underwater environments and get a recall rate of 92.31% in the tested environments.

Hyperspectral image classification, which is a crucial task for various remote sensing applications, can achieve qualified performance under a conventionalized assumption, i.e., there are sufficient samples in every concerned class. However, in field investigation, collecting enough samples for every class is extremely difficult, which results in defective training sets with insufficient or imbalanced samples and deteriorates the classification performance dramatically. In order to address this issue, we propose a hyperspectral image classification method based on a task-specific learning network. The proposed network works under an episode-based framework, which learns general knowledge from the tasks with sufficient samples by metalearning and relation learning and then inference specific knowledge for the tasks with few samples by parameters adjustment and representation comparison. In particular, a task-specific feature learner is designed to learn unbiased features, and a comparison-based classifier is utilized to adapt the minority classes. As a result, the proposed method can obtain a qualified overall accuracy over all samples with a comparative averaged accuracy over all classes. Extensive experiments on three real hyperspectral images show that our method can achieve state-of-the-art performance under both the few-shot and imbalanced training set settings.

The cross-data set knowledge is vital for hyperspectral image classification, which can reduce the dependence on the sample quantity by transferring knowledge from other data sets and improve the training efficiency by sharing knowledge between different data sets. However, due to the capturing environment change and imaging equipment difference, domain shift troubles the exploitation of the cross-data set knowledge. To address the aforementioned issue, this article proposes an unsupervised cross-data set hyperspectral image classification method based on adversarial domain adaptation. The proposed method, which employs multiple classifiers to build a discriminator and uses variational autoencoders to constitute a generator, works in an adversarial manner to drive the target samples under the support of the source domain. In particular, the classification error and the classification disagreement are considered in the objective function, which helps to align different domains while keeping the boundaries of different classes. Experimental results of the multidomain data set demonstrate that the proposed method can transfer and share cross-data set knowledge and achieve state-of-the-art performance without using the labeled information of the target data set.

Ship detection from SAR images is an important remote sensing application. However, in complex scenes, i.e., the shore or harbor area, traditional ship detection methods cannot disentangle background information from the target ship, and the detection performance drops dramatically. Moreover, the severe coherent speckle noise also challenges ship detection from SAR images. In order to address the aforementioned issues, this paper proposes a SAR image ship detection method based on scene interpretation to improve the performance of ship detection in complex scenes. Firstly, segmentation algorithm based on Mask R-CNN is utilized to interpret the scene into two catalogs, i.e., the sea and the land. Then, ship detection algorithm based on Faster R-CNN is performed on the sea area and the land area respectively. Finally, non-maximum suppression is used to integrate detection results. Experimental results on SAR ship detection dataset illustrate that the proposed method produces high detection accuracy and low false alarm rate in complex scenes.

Recent advances in device-free wireless sensing have created the emerging technique of device-free human gesture recognition (DFHGR), which could recognize human gestures by analyzing their shadowing effect on surrounding wireless signals. DFHGR has many potential applications in the fields of human-machine interaction, smart home, intelligent space, etc. State-of-the-art work has achieved satisfactory recognition accuracy when there are a sufficient number of training samples. However, it is time consuming and labor intensive to collect samples, thus how to realize DFHGR under a small training sample set becomes an urgent problem to solve. Motivated by the excellent ability of the generative adversarial network in synthesizing samples, in this article, we explore and exploit the idea of leveraging it to realize virtual samples augmentation. Specifically, we first design a single scenario network with new architecture and better-designed loss function to generate virtual samples using a few number of real samples. Then, we further develop a scenario transferring network to generate virtual samples by utilizing the real samples not only from the current scenario but also from another available scenario as well, which could improve the quality of synthesized samples with the extra knowledge learned from another scenario. We design an mmWave-based DFHGR testbed to test the proposed networks, extensive experimental results demonstrate that the augmented virtual samples are of high quality and facilitate DFHGR systems to achieve better accuracy.

The problem of different characters of heterogeneous synthetic aperture radar (SAR) images leads to poor performances for transfer learning of SAR image classification. To address this issue, a semisupervised model named as deep joint distribution adaptation networks (DJDANs) is proposed for transfer learning from a source SAR image to a different but similar target SAR image, which aims to match the joint probability distributions between the source domain and target domain. In the proposed DJDAN, a marginal distribution adaptation network is developed to map features across the domains into an augmented common feature subspace, which aims to match the marginal probability distributions and unify the dimensions. Then, a conditional distribution adaptation network is proposed to transfer knowledge across the domains, which aims to reduce the discrepancies of the conditional probability distributions and enhance the effectiveness of feature representation. Moreover, one-versus-rest classification is utilized in the proposed framework, which aims to improve the discrimination between the inside and outside class. Experimental results demonstrate the effectiveness of the proposed deep networks.

Device-free gesture recognition (DFGR) is a promising sensing technique, which can recognize a gesture by analyzing its influence on surrounding wireless signals. Most of the DFGR systems are designed based on machine learning. However, the recognition performance will drop dramatically when the testing condition is different with the training one. Inspired by the transferrable knowledge learning ability of humans, this paper develops a practical DFGR system based on metalearning to solve the aforementioned problem. Specifically, we design a deep network which could not only learn discriminative deep features, but also learn a transferrable similarity evaluation ability from the training set and apply the learned knowledge to the new testing conditions. Extensive experiments conducted by four users in two scenarios demonstrate that the proposed system could recognize new types of gestures, or gestures performed in new conditions, with an accuracy of more than 90%, using very few number of new samples.

Cross-scene classification of hyperspectral image (HSI) has been increasingly researched due to its crucial utilization in practical applications. However, cross-scene data generally perform distribution discrepancy, which hampers the transfer learning performance. To address this issue, deep conditional distribution adaptation networks (DCDAN) are proposed for HSI cross-scene classification, which aim to reduce the distribution shift between a source domain and a target domain. The proposed deep network adopts a conditional constraint to match the class conditional distributions across domains, where a great number of training samples from the source domain and a small number of training samples from the target domain are utilized to train the deep model. Cross-scene classification results on two HSIs demonstrate that the proposed network is able to yield superior performance compared with some related methods.

Hyperspectral image, which contains high-resolution spectral information as well as large-scale spatial information, has been widely used in various classification applications of remote sensing area. However, due to the insufficient of the labeled samples in the training set and the unbalance of sample quantity between different classes, traditional supervised classification methods are difficult to achieve satisfying performance. In order to address above issues, this paper studies on how to predict classification parameters more effectively, and finds out that the parameters of the fully-connected layer in the classifier are closely related to the output of the feature mapping layer. Based on above fact, this paper proposes a hyperspectral image classification method base on parameter prediction network, which adapts a pre-trained neural network to novel categories by directly predicting the parameters of classifier from the feature data of the hyperspectral image. Experimental results and analysis demonstrate the competitive performance of the proposed method over other state-of-the-art classification methods based on neural network when the number of labeled samples is very small.

Due to the rapid development of learning-based methods, hyperspectral image classification has achieved remarkable progress. However, since the semantic discrepancy between different land-cover types, the feature distributions of different classes are so nonuniform that the classifier can not measure them with a single rule. In order to consider semantic knowlage in classification, this paper propose a knowledge guided classification method based on hierarchical class tree and deep learning. The proposed method fuses similar classes into several super classes by the knowledge of the confusion matrix, and classify multi-level super classes with different deep networks. Extensive experiments on two hyperspectral images demonstrate that the proposed method can utilize semantic information of different land-cover types, and give better performance than traditional methods.

Change detection is an important task to identify land-cover changes between the acquisitions at different times. For synthetic aperture radar (SAR) images, inherent speckle noise of the images can lead to false changed points, which affects the change detection performance. Besides, the supervised classifier in change detection framework requires numerous training samples, which are generally obtained by manual labeling. In this paper, a novel unsupervised method named saliency-guided deep neural networks (SGDNNs) is proposed for SAR image change detection. In the proposed method, to weaken the influence of speckle noise, a salient region that probably belongs to the changed object is extracted from the difference image. To obtain pseudotraining samples automatically, hierarchical fuzzy C-means (HFCM) clustering is developed to select samples with higher probabilities to be changed and unchanged. Moreover, to enhance the discrimination of sample features, DNNs based on the nonnegative- and Fisher-constrained autoencoder are applied for final detection. Experimental results on five real SAR data sets demonstrate the effectiveness of the proposed approach.

Deep learning-based classification methods are competent to achieve an excellent performance under one necessary condition, i.e., there are sufficient labeled samples in each class, which is extremely impractical in most of the remote sensing tasks. To improve the performance with small training sets, we resort to other hyperspectral images and design a two-phase relation learning network that can be transferred between different images for general information sharing and fine-trained on a specific hyperspectral image for individual information learning. Specifically, we use a relation learning method to compare samples and deal with the task inconsistency between different data sets, and we adopt an episode-based training strategy to mimic the testing setup and learn the transferable comparison ability. Benefited from these two strategies, the proposed network takes the advantage of extra knowledge for information supplement and learns to compare rather than to classify for information exploration, which guarantees a reasonable performance even with small training sets. Extensive experiments and analysis on three benchmarks demonstrate that the proposed method can provide an effective solution for hyperspectral image classification with small training sets, which makes it possible to work on large-scale applications of earth observation with less effort on field investigation.

For hyperspectral image classification, there is a large gap between the theoretical method and the practical application. Hyperspectral image classification in theoretical research trains a new classifier for each data set, which is ineffective and even infeasible in large-scale applications. In this paper, we make a preliminary attempt to recycle the classification model to new data sets in an unsupervised way. Specially, we propose a cross-data set hyperspectral image classification method based on deep domain adaptation. The proposed method contains three modules: domain alignment module that learns to minimize the domain discrepancy with the guide of an irrelevant task, task allocation module that learns to classify on the source domain with the regulation of domain alignment, and domain adaptation module that transfers both the alignment ability and classification ability to the target domain by an adaptation strategy. As a result, with the information of an irrelevant task on dual-domain data sets, we can minimize the domain discrepancy and transfer the task-relevant knowledge from the source domain to the target domain in an unsupervised way. Extensive experiments on three hyperspectral images demonstrate the effectiveness of our method compared with other related methods when dealing with new data sets.

Convolutional neural networks (CNN), which are able to extract spatial semantic features, have achieved outstanding performance in many computer vision tasks. In this paper, hybrid neural networks (HNN) are proposed to extract both spatial and spectral features in the same deep networks. The proposed networks consist of different types of hidden layers, including spatial structure layer, spatial contextual layer, and spectral layer. All those layers work as organic networks to explore as much valuable information as possible from hyperspectral data for classification. Experimental results demonstrate competitive performance of the proposed approach over other state-of-the-art neural networks methods. Moreover, the proposed method is a new way to deal with multidimensional data with deep networks.

The classification of polarimetric synthetic aperture radar (PolSAR) image is of crucial significance for SAR applications. In this letter, a superpixel restrained deep neural network with multiple decisions (SRDNN-MDs) is proposed for PolSAR image classification, which not only extracts effective superpixel spatial features and degrades the influence of speckle noises but also deals with the limited training samples. First, the polarimetric features of coherency matrix and Yamaguchi decomposition are extracted as initial features, and superpixel segmentation is conducted on the Pauli color-coded image to acquire the superpixel averaged features. Then, an SRDNN based on sparse autoencoders is proposed to capture superpixel correlative features and reduce speckle noises. After that, MDs, including nonlocal decision and local decision, are developed to select credible testing samples. Finally, our deep network is updated by the extended training set to yield the final classification map. Experimental results demonstrate that the proposed SRDNN-MD yields higher accuracies compared with other related approaches, which indicate that the proposed method is able to capture superpixel correlative information and adds the information of unlabeled samples to improve the classification performance.

Synthetic aperture radar (SAR) image classification is a fundamental process for SAR image understanding and interpretation. With the advancement of imaging techniques, it permits to produce higher resolution SAR data and extend data amount. Therefore, intelligent algorithms for high-resolution SAR image classification are demanded. Inspired by deep learning technology, an end-to-end classification model from the original SAR image to final classification map is developed to automatically extract features and conduct classification, which is named deep recurrent encoding neural networks (DRENNs). In our proposed framework, a spatial feature learning network based on long-short-term memory (LSTM) is developed to extract contextual dependencies of SAR images, where 2-D image patches are transformed into 1-D sequences and imported into LSTM to learn the latent spatial correlations. After LSTM, nonnegative and Fisher constrained autoencoders (NFCAEs) are proposed to improve the discrimination of features and conduct final classification, where nonnegative constraint and Fisher constraint are developed in each autoencoder to restrict the training of the network. The whole DRENN not only combines the spatial feature learning power of LSTM but also utilizes the discriminative representation ability of our NFCAE to improve the classification performance. The experimental results tested on three SAR images demonstrate that the proposed DRENN is able to learn effective feature representations from SAR images and produce competitive classification accuracies to other related approaches.

Convolution neural network (CNN) utilizes alternating convolutional and pooling layers to learn representative spatial information when the training samples are sufficient. However, for pixelwise classification of hyperspectral image, some important information is neglected by CNN, such as the erased information by the pooling operation and the appearance information from lower layers. Moreover, the lack of training samples is a common situation in remote sensing area, which afflicts CNN with overfitting problem. To address the aforementioned issues, this paper designs an end-to-end deconvolution network with skip architecture to learn the spectral-spatial features. The proposed network starts with two branches, i.e., the spatial branch and spectral branch. In the spatial branch, a band selection layer is designed to reduce parameters and remit the overfitting problem, unpooling and deconvolution operations are utilized to recover the erased information of the pooling layers and learn pixelwise spatial representation hierarchically, and the skip architecture is constructed for merging the deep semantic information with the shallow appearance information. In the spectral branch, a contextual deep network is employed for learning deep spectral features. Experimental results on three benchmark data sets reveal the competitive performance of the proposed approach over several related methods.

Marine reclamation plays an increasingly important role in expanding living space, which should be monitored to ensure legitimate development. In this paper, a patch-based recurrent neural network is developed for change detection of marine reclamation. To capture spatial difference of image patches in two images, a patch-based recurrent neural network is proposed to extract features, where patches from two multispectral images are stacked as a sequence for inputting. After training the deep network, Softmax classifier is applied to detect the changed region. It is illustrated that our network can obtain the difference of two images to improve detection accuracies. Experiments on the study area of the Jinzhou Bay demonstrate that the proposed method outperforms other approaches.

Classification of multisensor data provides potential advantages over a single sensor in accuracy. In this paper, deep bimodal autoencoders are proposed for classification of fusing synthetic aperture radar (SAR) and multispectral images. The proposed deep network based on autoencoders is trained to discover both independencies of each modality and correlations across the modalities. Specifically, the sparse encoding layers in the front are applied to learn features of each modality, then shared representation layers in the middle are developed to learn fused features of two modalities, finally softmax classifier in the top is adopted for classification. Experimental results demonstrate that the proposed network is able to yield superior classification performance compared with some related networks.

The classification of a synthetic aperture radar (SAR) image is a significant yet challenging task, due to the presence of speckle noises and the absence of effective feature representation. Inspired by deep learning technology, a novel deep supervised and contractive neural network (DSCNN) for SAR image classification is proposed to overcome these problems. In order to extract spatial features, a multiscale patch-based feature extraction model that consists of gray level-gradient co-occurrence matrix, Gabor, and histogram of oriented gradient descriptors is developed to obtain primitive features from the SAR image. Then, to get discriminative representation of initial features, the DSCNN network that comprises four layers of supervised and contractive autoencoders is proposed to optimize features for classification. The supervised penalty of the DSCNN can capture the relevant information between features and labels, and the contractive restriction aims to enhance the locally invariant and robustness of the encoding representation. Consequently, the DSCNN is able to produce effective representation of sample features and provide superb predictions of the class labels. Moreover, to restrain the influence of speckle noises, a graph-cut-based spatial regularization is adopted after classification to suppress misclassified pixels and smooth the results. Experiments on three SAR data sets demonstrate that the proposed method is able to yield superior classification performance compared with some related approaches.

Speckle noises are inherent issues in synthetic aperture radar (SAR) images, which hampers the analysis and interpretation of SAR images. In this paper, we propose an iterative low-rank representation algorithm for SAR image despeckling. The original SAR image is first transformed to the logarithmic image, which is then filtered iteratively by the proposed low-rank representation model. Specifically, in each iteration, similar patches measured by the Mahalanobis distance are collected into a group, and then filtered by the nuclear regularized low-rank representation. Finally, all of the filtered patches are aggregated to form the denoised image. Experimental results demonstrate that the proposed algorithm is able to yield state-of-the-art SAR image despeckling performance.

This paper presents a hyperspectral image classification method based on deep network, which has shown great potential in various machine learning tasks. Since the quantity of training samples is the primary restriction of the performance of classification methods, we impose a new prior on the deep network to deal with the instability of parameter estimation under this circumstances. On the one hand, the proposed method adjusts parameters of the whole network to minimize the classification error as all supervised deep learning algorithm, on the other hand, unlike others, it also minimize the discrepancy within each class and maximize the difference between different classes. The experimental results showed that the proposed method is able to achieve great performance under small training set.

Synthetic aperture radar (SAR) image classification is a hot topic in the interpretation of SAR images. However, the absence of effective feature representation and the presence of speckle noise in SAR images make classification difficult to handle. In order to overcome these problems, a deep convolutional autoencoder (DCAE) is proposed to extract features and conduct classification automatically. The deep network is composed of eight layers: a convolutional layer to extract texture features, a scale transformation layer to aggregate neighbor information, four layers based on sparse autoencoders to optimize features and classify, and last two layers for postprocessing. Compared with hand-crafted features, the DCAE network provides an automatic method to learn discriminative features from the image. A series of filters is designed as convolutional units to comprise the gray-level cooccurrence matrix and Gabor features together. Scale transformation is conducted to reduce the influence of the noise, which integrates the correlated neighbor pixels. Sparse autoencoders seek better representation of features to match the classifier, since training labels are added to fine-tune the parameters of the networks. Morphological smoothing removes the isolated points of the classification map. The whole network is designed ingeniously, and each part has a contribution to the classification accuracy. The experiments of TerraSAR-X image demonstrate that the DCAE network can extract efficient features and perform better classification result compared with some related algorithms.