Abstract. Fusing hyperspectral images (HSIs) with multispectral images (MSIs) has become a mainstream approach to enhance the spatial resolution of HSIs. Recently, many HSI-MSI fusion techniques have been developed and have achieved notable performance. Nevertheless, certain challenges still persist, including (a) Most existing fusion methods assume precise registration between HSIs and MSIs, which can be challenging to achieve in practice. (b) The obtained HSI-MSI pairs may not be fully utilized. To address these issues, we propose a coarse-to-fine hybrid registration and fusion approach. In the coarse stage, a joint registration and fusion (JRF) model is developed by integrating batch image alignment into the fusion process, which can achieve HSI-MSI registration and fusion simultaneously. To further enhance fusion performance, we propose a nonconvex low-rank and group-sparse (NLG) model in the fine stage. This model exploits the low-rank property to reconstruct the global image structure (i.e., the clean image), while simultaneously leveraging group sparsity to retrieve the textural details. This combination yields the proposed JRF-NLG method. Then, the JRF-NLG model is solved by the generalized Gauss–Newton (GGN) algorithm and the proximal alternating optimization (PAO) algorithm, respectively. Theoretically, we establish an error bound for the NLG model and prove the quadratic convergence rate of the GGN algorithm. Finally, extensive numerical experiments on simulated and real datasets are performed to verify the effectiveness of our method in registration and fusion. We also validate its effectiveness in enhancing classification performance.

Visible and infrared light images reflect object characteristics in different aspects, which has attracted much attention for object detection in recent years. Nevertheless, the existing multimodal detection networks may fail in the absence of modality. In order to address this problem, a new cross-modal knowledge distillation network (CMKD-net) is proposed for oriented object detection in visible and infrared images. In brief, a teacher-student (T-S) learning network is constructed, where the T-network aims to learn a discriminative feature representation from multimodal images and then guides the S-network training with incomplete modality. Here, multi-dimensional feature distillation (MDFD) and inter-instance relation distillation (IIRD) are designed for cross-modal knowledge propagation. Specifically, the MDFD considers pushing the T-S networks to learn a similar data distribution and feature representation through channel-spatial dimensional feature consistency constraints. The IIRD contributes to retaining the relation structure between individual targets in multimodal images via inter-instance relation modeling and similarity distance measurement. Moreover, to avoid the bias of feature extraction caused by discrete quantization in traditional pooling operations, a rotation-adaptive RoI Pooling (RA-RoI Pooling) is introduced by calculating the continuous double integral within each bin of oriented objects. Ablation experiments and comparison experiments on the VEDAI and DroneVehicle datasets can demonstrate the effectiveness of the proposed CMKD-net.

Recently, unsupervised domain adaptation (UDA) techniques have been introduced for cross-scene hyperspectral image (HSI) classification tasks. These techniques aim to transfer knowledge from labeled source scenes to unlabeled target scenes, addressing the issue of limited supervisory information. However, most UDA methods fail to analyze the variability of domain shifts from different source samples to target ones, thus limiting the domain adaptation effect. To this end, this paper develops a consistency-aware customized learning (CACL) approach for cross-scene HSI classification. Overall, domain-level and class-level distribution alignment are designed separately. The former is implemented by adversarial training between the feature extractor and the domain discriminator. For the latter, the spectral-spatial prototypes of the source and target domains are first dynamically extracted, respectively. Then the prototype-based labels are assigned to the target domain samples, according to the cosine similarity-based cross-domain category prototype matching strategy. Considering that the consistency of the prototype-based labels with the predicted pseudo-labels reflects the degree of domain shifts of the target samples, a customized learning strategy is developed via inter-/intra-domain contrastive learning. With the joint domain-level and fine-grained class-level distribution alignment, the supervised information from the source domain is better migrated to the target domain, improving classification performance. Comprehensive experiments on two single-modal and one multi-modal cross-scene datasets demonstrate the effectiveness of the proposed algorithm.

RGB-infrared (RGB-IR) object detection has attracted significant attention in drone-based applications due to its robustness under all-weather conditions. How to effectively fuse the complementary information in both modalities is one key for accurate object detection. However, the performance is limited by the inherent differences between modalities and the varying illumination conditions across different weather scenarios. Focused on this issue, we propose an illumination-aware multimodal hierarchical fusion network (IMHFNet) for RGB-IR object detection. First, an illumination aware module (IAM) is designed to extract local illumination features from RGB image, which is used to guide the subsequent multimodal feature fusion process. Then, considering the differences in semantic expression and detail representation of different feature layers of multimodal data, we separately design shallow and deep feature fusion strategies. In specific, the shallow feature fusion module is constructed based on convolutional operators and illumination-guided adaptive weight fusion, focusing on capturing and enhancing local detail information. For the deep feature fusion, illumination feature is incorporated as an auxiliary information, to guide the global semantic information integration across different modalities via adopting a transformer structure. In this work, we also construct a new drone-based RGB-IR dataset, named by DroneShip. It contains 4,306 images annotated with 17,054 oriented ship object instances, which covers a wide range of natural illumination conditions from daytime to nighttime. Finally, to validate the effectiveness of the proposed method, we evaluate the IMHFNet on the constructed DroneShip and two publicly available RGB-IR datasets (KAIST and DroneVehicle), which respectively focus on ship, pedestrian and vehicle targets. Experimental results on all three datasets consistently demonstrate the effectiveness and robustness of IMHFNet across diverse scenarios and illumination conditions. The source code of the proposed method will be made publicly available at https://github.com/luting-hnu/DroneShip.

Over the past few decades, a large collection of feature learning and segmentation methods have been developed for building change detection (BCD) in multi-temporal remote sensing images. Focused on this, various datasets were built up for analyzing changes in different countries, lacking attention to different regions in China. In this paper, a new Chinese Region BCD (CRBCD) dataset was established, which collects image pairs between 2017-2022 years in 31 cities of China from Google Earth. In specific, the dataset covers changing buildings in multiple types of regions, including urban, sub-urban, rural and scenic areas. Besides, the differences between constructed dataset and some existing typical BCD datasets are comprehensively compared and analyzed. In addition, considering the local spatial consistency distribution characters, a structure prior guided convolutional neural network (SPG-CNN) framework is introduced here. Both the multi-scale superpixel segmentation and high-frequency derived edge information are inserted to the CNN, improving the difference feature learning capability. The quantitative and visual comparison results on both the public dataset and the proposed dataset demonstrate the effectiveness of the proposed SPG-CNN. The CRBCD dataset can be found at https://github.com/cpfhnu/CRBCD.

Facing the pervasive problem of missing supervised information, cross-scene hyperspectral image (HSI) classification tasks based on unsupervised domain adaptation (UDA) techniques have emerged. However, lacking an integral view of feature-level alignment and decision-level analysis, most UDA methods treat target data with differential domain shift equally. Focused on this problem, a novel consistency-aware customized learning (CACL) approach is proposed, in this paper. Specifically, we develop a convolution-based domain-invariant feature learning network. First, the feature extractor is employed to extract spectral-spatial category prototypes. At the same time, domain-level distribution alignment is performed with the domain discriminator. Then, a customized learning strategy, i.e., inter/intra-domain contrast learning, is designed based on whether the pseudo-labels are consistent with the spectral-spatial prototype matchability labels. In addition, focal loss is introduced for information mining of hard samples. The experimental results demonstrate the state-of-the-art of the method.

Semi-supervised classification of remote sensing hyperspectral image (HSI) aims at exploiting both labeled and unlabeled samples for accurate land cover recognition. However, imbalanced data distribution and different classification difficulties negatively affect classification performance. Focused on this, a novel dual-stream class-adaptive network (DSCA-Net) is proposed for semi-supervised HSI classification, in this paper. First, a superpixel-guided label propagation module is introduced to alleviate the negative effect of imbalanced data distribution. Specifically, approximate estimation of labels for unlabeled samples is achieved via superpixel-wise similarity measure and label propagation, so that equal sampling is applied to each class. Then, a consistency regularization-based dual-stream network is constructed, which shares the same encoder for feature representation of either labeled or unlabeled samples. Based on this, two distinct classifiers are designed to force similar predictions can be achieved for various perturbed versions of the same unlabeled sample, thereby allowing unlabeled samples to train the model in a supervised manner. Finally, since different classes always have various degrees of learning difficulty, equal treatment may lead to overfitting of “easy” classes and biased prediction of “hard” classes. Unlike the traditional selection of unlabeled samples with a fixed threshold, dynamic class-adaptive thresholds are calculated according to the learning status of the model. In this manner, a higher threshold is assigned to “easy” classes to reduce sample redundancy, and a lower threshold is set for “hard” classes to select more samples. Experiment results demonstrate the effectiveness and superiority of the proposed method. Codes are available at https://github.com/luting-hnu/DSCA-Net.

Recently, deep learning presents a promising performance in the joint classification of multimodal remote sensing (RS) data. However, most of the approaches adopt a supervised learning manner, where the discrimination capability is limited by the paucity of labeled samples. Though some attempts have been made to develop semi-supervised methods, they prefer to select the highly confident predictions as pseudo ground truth and discard those unreliable ones. Actually, unreliable samples can also provide useful information, e.g., indicating the categories to which samples may belong and definitely not belong. Focused on this, a novel uncertainty-aware contrastive learning (UACL) method is proposed. Here, label uncertainty analysis based on multi-level probability estimation is first conducted to separate reliable and unreliable samples, which are then processed with a designed hybrid (“hard” or “soft”) contrastive learning (CL) strategy. For reliable samples, the “hard” CL pushes the network to learn features that will minimize the intra-class distance while maximizing the inter-class distance, according to the pseudo-labels. For unreliable samples, the “soft” CL aims to learn the similarity and difference among samples, where the predicted class probabilities are queried to estimate a soft mask for an adaptive feature similarity measurement. Moreover, a multimodal spectral-spatial joint feature representation pipeline of triple branches, i.e., one spectral branch for hyperspectral images (HSIs) and two spatial branches for multimodal data, is also introduced. By jointly learning from both labeled and unlabeled samples, more discriminative spectral-spatial feature representation will lead to a further boost in classification performance. Extensive experiments on four well-known multimodal datasets prove the effectiveness of the proposed semi-supervised classification method. Codes are available at https://github.com/Ding-Kexin/UACL.

Nowadays, high-resolution remote sensing images provide rich data sources and deep learning models show powerful feature representation capability for remote sensing object detection. However, due to the complex object structure as well as the changeable rotation angle, how to efficiently estimate the oriented bounding box regarding the accurate location of objects is still an open issue. Focused on this, a new one-stage structure-adaptive oriented object detection (SOOD) network is proposed, in this article. First, we designed a new rotation angle encoder (RAE), where an angle coordinate system is adopted and periodic angle correction is conducted. Different from the traditional longe-edge definition for angle estimation, the RAE can mitigate boundary discontinuity and square-like problems. Then, structure-adaptive label assignment (SALA) and confidence estimation (SACE) are introduced, to locate the position of objects more accurately. On the one hand, the anchor box determines the label assignment according to the affiliation relationship between the center point and the object’s inner ellipse boundary. By constraining the ellipse boundary and employing non-parametric label assignment, high-quality anchor boxes are initially selected, and low-quality anchor boxes are suppressed. On the other hand, the integration of intersection over union (IoU) prediction and uncertainty prediction constructs a quality evaluation function to guide. In this manner, this function dynamically evaluates the localization and classification ability of each prediction box. Extensive experiments on publicly available datasets such as DOTA1.0, DOTA1.5, DIOR, and MAR20 demonstrate the effectiveness of the proposed model. The source code will be available athttps://github.com/fan609/SOOD.

Deep learning has been a powerful tool for hyperspectral image (HSI) classification. However, it is still an open issue to effectively learn highly discriminative features from the HSI, due to the high-dimensionality and complex spectral-spatial characteristics. To settle this issue, we propose a new band-grouping guided multi-attention module for the performance promotion of spectral-spatial feature learning. First, based on the fact of high relevance between adjacent spectral bands and low dependencies across long-range ones, all the spectral bands are adaptively divided into multiple non-overlapping groups where relevant bands are included. The advantage is to reduce the spectral dimension and data complexity when processing and analyzing each group. Then, a multi-attention mechanism, which not only explore the intra-group salient information but also propagate the inter-group difference information, is embedded into the convolutional neural networks to learn group-specific spectral-spatial features. By emphasizing useful spectral/spatial information and squeezing useless information with attention mechanism, the severability of learned features is enhanced. Based on this module, a spectral-spatial classification network is built, named by grouped multi-attention network (GMA-Net). The GMA-Net contains a two-branch architecture, i.e., pixel-wise spectral feature learning and patch-wise spectral-spatial feature learning. Via fusing the features from two branches, the complementary and discriminative features provided by pixel-wise and patch-wise learning manner can be integrated to further boost classification performance. Experimental results demonstrate that the proposed method is superior than several state-of-the-art approaches. Codes are available at: https://github.com/luting-hnu.

Deep learning based methods are very popular for hyperspectral image classification. However, those methods usually ignore the fact that discriminative information lies on specific spatial positions and spectral bands. To solve this problem, we introduce the attention mechanism, and propose a cross-layer multi-attention guided classification network (CLMA-Net) for HSIs. First, a backbone network, which is a two-branch convolutional neural network, is developed to extract spectral and spatial features. Then, cross-layer multi-attention modules, which integrate attention information of multiple convolutional layers, are embedded into two branches. As a result, spectral and spatial features are optimized by making the network attend to interested parts. Finally, spectral and spatial features are concatenated and used to predict class label by a fully connected layer. Experimental results demonstrate the effectiveness of the proposed method. The code will be available at https://github.com/mengkai-liu/CLMA-Net.

Building change detection is a very important application in the field of remote sensing. Recently, deep learning (DL) has been introduced to solve the change detection task and achieved good performance, mainly due to the capability of automatically learning deep features. However, the lack of using prior knowledge (e.g., edge structure information) leads to inaccurate detection results, especially in the areas of building boundaries. To solve this problem, an end-to-end DL method for building change detection, named by edge-guided recurrent convolutional neural network (EGRCNN), is proposed in this article. The main idea is to incorporate both discriminative information and edge structure prior in one framework to improve change detection results, especially to generate more accurate building boundaries. First, a siamese convolutional neural network is trained to simultaneously extract primary multilevel features from multitemporal images. Then, a difference analysis module (DAM) is introduced to further produce discriminative features, which is constructed based on the basic long short-term memory module. Finally, both the discriminative features and the estimated edge structure information are jointly exploited to predict building change map. On one hand, the proposed DAM helps to enhance the discrimination between the changed and unchanged regions. On the other hand, the prior edge information is used to push the predicted changed buildings to preserve the original structure, which can further improve the accuracy of building change detection. Experimental results demonstrate that the performance of the proposed method outperforms several state-of-the-art approaches, in terms of objective metrics and visual comparison results.

Hyperspectral images (HSI) contain rich spatial and spectral detail information, while light detection and ranging (LiDAR) data can provide the elevation information. Thus, the fusion of HSI and LiDAR data can help for more accurate image classification, which becomes a hot research topic. However, it is difficult to capture complex local and global spatial-spectral associations, meanwhile, how to build an effective interaction between multi-modal data is another important issue. To this end, a novel global-local transformer network (GLT-Net) is proposed for the joint classification of HSI and LiDAR data, in this paper. The main idea is to fully exploit the advantage of the convolution operator in characterizing locally correlated features and the promising capability of transformer architecture in learning long-range dependencies. Moreover, multi-scale feature fusion and probabilistic decision fusion strategies are also designed in one framework, in order to further improve classification performance. Here, the proposed GLT-Net mainly consists of multi-scale local spatial feature learning, global spectral feature learning, and global-local feature fusion classification. In specific, multi-modal image cubes of different sizes are firstly extracted and sent into convolutional neural networks (CNNs) to learn local spatial features, which is followed by multi-modal information propagation and spatial-attention guided multi-scale feature fusion. Afterwards, by considering spectral feature channels from a sequential perspective, vision transformers are introduced to model the global spectral dependencies. Finally, multiple class estimations based on local and global features are integrated via a probabilistic decision fusion strategy. In this way, complementary information of multi-modal data as well as local/global spectral-spatial information can be fully mined and jointly utilized. Extensive experiments on three popular HSI and LiDAR datasets demonstrate that the proposed method performs superiority over state-of-the-art methods. The source code of the proposed method will be made publicly available at https://github.com/Ding-Kexin/GLT-Net.

In recent years, deep learning presents a promising performance in hyperspectral image (HSI) classification, due to the powerful capability of automatically learning deep semantic characteristics of images. However, it is still difficult to learn highly discriminative features when limited samples are available for training a deep network. Focused on this issue, a novel end-to-end supervised contrastive learning network (SCL-Net) for spectral-spatial classification is proposed, in this paper. Instead of learning features of the individual sample, the supervised contrastive learning is introduced to capture the similarity and dissimilarity distribution properties of sample pairs in feature representation space. In this way, the need for plenty of training samples will be alleviated while an effective network training mechanism is provided for learning highly separative features. Here, the SCL-Net mainly consists of one pair-wise contrastive learning (PCL) sub-network and one multi-level spectral-spatial information fusion (MLSIF) sub-network. For the PCL sub-network, spectral vectors are projected into deep spectral features based on convolutional operators, which are then followed by distance evaluation between “positive” pairs of similar samples and “negative” pairs of dissimilar ones. Then, a spectral distance matrix is constructed to push the network to gradually learn better features of higher intra-class compactness and inter-class dispersion. For the MLSIF sub-network, a hybrid feature-decision fusion strategy is designed, where spatial and spectral features are jointly exploited to further boost classification performance. In specific, the feature fusion is conducted by connecting low/mid/high-level spectral and spatial features via weighting, while multiple class estimations based on multi-level fusion features are adaptively integrated via probabilistic decision fusion. Overall, these two sub-networks are collaboratively trained in one framework, by optimizing a defined joint loss function consisting of a contrastive loss and a cross-entropy loss. Compared with several state-of-the-art methods, the proposed method yields a superior classification performance in terms of both objective metrics and visual performance.

Exploring effective spectral–spatial feature extraction methods is one of the most focused problems in current hyperspectral image (HSI) classification research. However, complex spectral–spatial structure characteristics in HSIs, e.g., shape-variable spatial structure and nonlinear spectral structure, are difficult to be effectively extracted and jointly represented. To overcome this issue, a novel superpixel-based Brownian descriptor (SBD) method for HSI classification is proposed in this article. In specific, superpixel segmentation is first used to extract shape-adaptive spatial structure information from dimension-reduced HSI, leading to generate nonoverlapping homogeneous 3-D image blocks. Then, similar pixels within the 3-D image block are jointly represented by a new local spectral–spatial feature based on the Brownian descriptor (BD). This is the first time that the BD is introduced to measure both linear and nonlinear correlations among different spectral bands in HSI. On one hand, the integration of superpixel and BD helps to provide much richer and more valuable information for better discrimination between different categories. On the other hand, the SBD can effectively represent the internal structure characters within each 3-D image block of different spatial shapes by a symmetric positive definite matrix of a united form. Finally, considering that the SBD lies on the Riemannian manifold space, a log-Euclidean kernel sparse representation (LKSR) classifier is introduced to obtain the classification results. Experimental results on three widely used real hyperspectral datasets indicate the performance superiority of the proposed SBD method over several state-of-the-art techniques.

For hyperspectral image (HSI) classification, it is an challenging problem to learn highly discriminative features, since the complex local/non-local spatial-spectral association is difficult to be accurately characterized. Focused on this issue, a novel superpixel-level hybrid discriminant analysis (SHDA) method is proposed, in this paper. Here, the SHDA method takes advantage of superpixel’s merit in characterizing spatial-spectral shape-adaptive structure and the powerful capability of discriminant analysis in enhancing class-separability to learn the feature representation. Moreover, the local/non-local spatial-spectral correlation information among/between superpixels is effectively excavated and fused in one framework to further improve the classification performance of features. This is achieved by first designing two specific discriminant analysis modules, i.e., superpixel-level local discriminant analysis (SLDA) and superpixel-level non-local discriminant analysis (SNDA). In the SLDA, adaptively weighted scatter matrices are defined to characterize the local spectral similarity within each superpixel and the discrepancy among adjacent superpixels. In the SNDA, superpixel-level graphs are built for capturing the non-local contextual information, where the weights of graphs are estimated based on the most similar and dissimilar superpixels. Then, the SLDA and the SNDA are effectively fused to construct the total intra/inter-superpixel scatter matrices. Finally, a joint projection transformation is obtained via solving a simple generalized eigenvalue problem. By this way, the HSI data can be projected from a high-dimensional data space into a low-dimensional feature space, where different classes of land-covers can be more accurately distinguished. Experimental results on three real hyperspectral data sets indicate that the proposed SHDA method outperforms several state-of-the-art techniques.

Remote sensing scene classification aims to assign automatically each aerial image a specific sematic label. In this letter, we propose a new method, called self-attention-based deep feature fusion (SAFF), to aggregate deep layer features and emphasize the weights of the complex objects of remote sensing scene images for remote sensing scene classification. First, the pretrained convolutional neural network (CNN) model is applied to extract the abstract multilayer feature maps from the original aerial imagery. Then, a nonparametric self-attention layer is proposed for spatial-wise and channel-wise weightings, which enhances the effects of the spatial responses of the representative objects and uses the infrequently occurring features more sufficiently. Thus, it can extract more discriminative features. Finally, the aggregated features are fed into a support vector machine (SVM) for classification. The proposed method is experimented on several data sets, and the results prove the effectiveness and efficiency of the scheme for remote sensing scene classification.

Combining a high-spatial-resolution multispectral image (HR-MSI) with a low-spatial-resolution hyperspectral image (LR-HSI) has become a common way to enhance the spatial resolution of the HSI. The existing state-of-the-art LR-HSI and HR-MSI fusion methods are mostly based on the matrix factorization, where the matrix data representation may be hard to fully make use of the inherent structures of 3-D HSI. We propose a nonlocal sparse tensor factorization approach, called the NLSTF_SMBF, for the semiblind fusion of HSI and MSI. The proposed method decomposes the HSI into smaller full-band patches (FBPs), which, in turn, are factored as dictionaries of the three HSI modes and a sparse core tensor. This decomposition allows to solve the fusion problem as estimating a sparse core tensor and three dictionaries for each FBP. Similar FBPs are clustered together, and they are assumed to share the same dictionaries to make use of the nonlocal self-similarities of the HSI. For each group, we learn the dictionaries from the observed HR-MSI and LR-HSI. The corresponding sparse core tensor of each FBP is computed via tensor sparse coding. Two distinctive features of NLSTF_SMBF are that: 1) it is blind with respect to the point spread function (PSF) of the hyperspectral sensor and 2) it copes with spatially variant PSFs. The experimental results provide the evidence of the advantages of the NLSTF_SMBF method over the existing state-of-the-art methods, namely, in semiblind scenarios.

Traditional hyperspectral imaging technique obtains numerous hyperspectral images (HSIs) with hundreds of spectral bands, leading to high cost in data acquisition, transmission, and storage. Compressed sensing (CS) theory provides a new imaging mechanism, which relies on the assumption that signals can be sparsely represented over a dictionary. By the CS imaging technique, original HSIs can be approximately reconstructed from only a few sampled measurements. In this article, a novel context-aware CS (CACS) method for HSIs is proposed by incorporating contextual prior to the dictionary learning and the sparse reconstruction. First, a patch-based online dictionary learning (ODL) algorithm is developed by introducing a joint sparse constraint. On the one hand, the online dictionary learning mechanism enables a more adaptive representation of HSIs with different scenes than using fixed-basis-based dictionaries, e.g., the discrete cosine transform (DCT) and the discrete wavelet transform dictionaries. On the other hand, the introduced joint sparse constraint promotes the learned dictionary to more sparsely and structurally represent spectral pixels. Then, with the well-learned dictionary, a weighted smoothing regularization is introduced to develop a new sparse reconstruction model. Considering the high spectral-spatial similarity of pixels in a neighborhood, the new sparse reconstruction model will encourage a locally smoothing reconstruction result. In this way, the spectral-spatial structures of the HSI can be well preserved, while possible artifacts can be effectively reduced. Experimental results demonstrate the superiority of the proposed method over some state-of-the-art hyperspectral compressive imaging methods.

Active learning (AL) attempts to actively select the most representative or useful training samples in an iterative manner. The aim is to simultaneously improve the classification performance and reduce the manual labeling effort. In this article, a novel subpixel-pixel-superpixel-based multiview AL (MAL) (SPS-MAL) method is proposed for hyperspectral image (HSI) classification. Here, the multiple views are generated via extracting the subpixel-level, pixel-level, and superpixel-level information. The multiple views can reflect various characteristics of HSI, i.e., spectral mixture, spectral discrimination, and spectral-spatial structure. Therefore, the joint use of diverse and complementary information in multiple views will contribute to a better identification ability of different classes. In addition, a coarse-to-fine MAL algorithm is introduced to effectively select the most representative samples with the most uncertainty. Specifically, a disagreement analysis on multiple views and joint posterior probability estimation is used to query unlabeled samples. Along with the expansion of training samples, view-specific confidence scores are estimated to adaptively integrate the classification results of multiple views, according to their discrimination performance. In this way, the classification accuracy will be further boosted while the number of necessary training samples can be significantly reduced. The experimental classification results on three well-known HSIs demonstrate the effectiveness of the proposed SPS-MAL method.

For hyperspectral images (HSIs), it is a challenging task to select discriminative bands due to the lack of labeled samples and complex noise. In this article, we present a novel local-view-assisted discriminative band selection method with hypergraph autolearning (LvaHAl) to solve these problems from both local and global perspectives. Specifically, the whole band space is first randomly divided into several subspaces (LVs) of different dimensions, where each LV denotes a set of low-dimensional representations of training samples consisting of bands associated with it. Then, for different LVs, a robust hinge loss function for isolated pixels regularized by the row-sparsity is adopted to measure the importance of the corresponding bands. In order to simultaneously reduce the bias of LVs and encode the complementary information between them, samples from all LVs are further projected into the label space. Subsequently, a hypergraph model that automatically learns the hyperedge weights is presented. In this way, the local manifold structure of these projections can be preserved, ensuring that samples of the same class have a small distance. Finally, a consensus matrix is used to integrate the importance of bands corresponding to different LVs, resulting in the optimal selection of expected bands from a global perspective. The classification experiments on three HSI data sets show that our method is competitive with other comparison methods.

In this paper, a novel supervised band selection (BS) method based on pair-wise constraint and band-wise correlation information is proposed for the dimension reduction of hyperspectral images. On the one hand, the band-wise correlation information, is used for selecting band-subset with lower redundancy and higher representation. This process is achieved by first partitioning all spectral bands into continuous groups and then calculate a band-wise correlation matrix within each group, which is used later for selecting bands of more representation and lower redundancy. On the other hand, pair-wise supervised information (i.e., whether a pair of labeled samples are from the same class) is exploited for selecting band-subsets to better discriminate different classes. That is, a few bands are adaptively chosen for each pair of labeled samples according to spectral-similarity, to ensure that the distance between samples from different classes is far and keep sample-pair from same class close. By the joint use of both pair-wise constraint information and band-wise correlation information, the proposed BS method can lead to select optimal band-subsets with low-redundancy, high-representation and high-discrimination. Experimental results demonstrate the effectiveness of the proposed BS method.

Recently, deep learning has been introduced to classify hyperspectral images (HSIs) and achieved good performance. In general, deep models adopt a large number of hierarchical layers to extract features. However, excessively increasing network depth will result in some negative effects (e.g., overfitting, gradient vanishing, and accuracy degrading) for conventional convolutional neural networks. In addition, the previous networks used in HSI classification do not consider the strong complementary yet correlated information among different hierarchical layers. To address the above two issues, a deep feature fusion network (DFFN) is proposed for HSI classification. On the one hand, the residual learning is introduced to optimize several convolutional layers as the identity mapping, which can ease the training of deep network and benefit from increasing depth. As a result, we can build a very deep network to extract more discriminative features of HSIs. On the other hand, the proposed DFFN model fuses the outputs of different hierarchical layers, which can further improve the classification accuracy. Experimental results on three real HSIs demonstrate that the proposed method outperforms other competitive classifiers.

In this paper, a novel spectral-spatial very high resolution images shadow detection algorithm based on random walker is proposed. First, a set of training samples is obtained by an improved Otsu based thresholding method automatically. Then, a widely used pixel-wise classifier, i.e., the Support Vector Machine (SVM), is applied to obtain an initial binary classification map. Finally, the initial classification map is refined with the extended random walker model, which can jointly integrating both the spectral characteristics and spatial-correlation among adjacent pixels to further improve shadow detection accuracy. Experimental results performed on real data sets demonstrate the superiority of the proposed method over several state-of-the-art methods.

In this paper, a novel model of Gabor Filtering based Deep Network (GFDN) for hyperspectral image classification is proposed. First, spatial features are extracted via Gabor filtering from the three principal components. Gabor filter can capture physical structures of hyperspectral images, such as specific orientation information. Then, the Gabor features and spectral features are simply staked to form combined features. Finally, high-level features are learnt by a stacked sparse auto-encoder deep network. Since the limited training samples negatively affect the classification performance in deep learning, here, an effective way is designed to simulate more training samples. By using both the real and virtual samples, the parameters of deep network can be better learnt and updated, leading to more robust and accurate classification results. Experiments on the real hyperspectral data set reveal the superior performance of the proposed method over some well-known classification methods.

In this paper, a novel iterative clustering based active learning (ICAL) method for hyperspectral image classification is proposed. On the one hand, the extreme learning machine is combined with the Markov random field (ELM-MRF) for label assignment, to exploit both spectral and spatial information to boost classification result. On the other hand, an iterative clustering based sample selection strategy is introduced to optimally choose the most informative training sample set. This strategy first selects a candidate set of samples, according to the differential map that is obtained by comparing the ELM-MRF based classification results in adjacent iterations. Then, all the pixels in the candidate set are clustered according to spectral characteristics. Finally, from each cluster, the one sample with the highest uncertainty is added to the new training sample set. By this sample selection strategy, the diversity and uncertainty of training samples can be maximized, which can further contribute to the improvement of classification performance. Experimental results show that the proposed ICAL method can achieve competitive classification results even with a limited number of labeled samples.

Supervised classification of hyperspectral images (HSI) is a very challenging task due to the existence of noisy and mixed spectral characteristics. Recently, the widely developed spectral unmixing techniques offer the possibility to extract spectral mixture information at a subpixel level, which can contribute to the categorization of seriously mixed spectral pixels. Besides, it has been demonstrated that the discrimination between different materials will be improved by integrating the geometry and structure information, which can be derived from the variance between neighboring pixels. Furthermore, by incorporating the spatial context, the superpixel-based spectral-spatial similarity information can be used to smooth classification results in homogeneous regions. Therefore, a novel fusion framework for HSI classification that combines subpixel, pixel, and superpixel-based complementary information is proposed in this paper. Here, both feature fusion and decision fusion schemes are introduced. For the feature fusion scheme, the first step is to extract subpixel-level, pixel-level, and superpixel-level features from HSI, respectively. Then, the multiple feature-induced kernels are fused to form one composite kernel, which is incorporated with a support vector machine (SVM) classifier for label assignment. For the decision fusion scheme, class probabilities based on three different features are estimated by the probabilistic SVM classifier first. Then, the class probabilities are adaptively fused to form a probabilistic decision rule for classification. Experimental results tested on different real HSI images can demonstrate the effectiveness of the proposed fusion schemes in improving discrimination capability, when compared with the classification results relied on each individual feature.

In this paper, a decision fusion of pixel-level and superpixel-level classifiers (DFPSC) for the HSI is proposed. First, the support vector machine based classification probability combined with the local spatial information is introduced to classify the HSI in a pixel-by-pixel manner. Then, the HSI is over-segmented into non-overlapping superpixels. Each superpixel contains spatially-connected and spectrally-similar pixels, which are assigned to the same label via joint sparse regularization. Finally, a guided map is generated based on the edge map and superpixel map, which is used to guide the fusion of both pixel-level and superpixel-level classification results. With the proposed decision fusion scheme, the classification results in homogeneous and structural areas can be better balanced, leading to the improvement of the overall classification accuracy. The experimental results demonstrate the superiority of the proposed method over some well-known classification methods.

A novel hyperspectral image (HSI) classification method by the probabilistic fusion of pixel-level and superpixel-level classifiers is proposed. Generally, pixel-level classifiers based on spectral information only may generate “salt and pepper” result in the classification map since spatial correlation is not considered. By incorporating spatial information in homogeneous regions, the superpixel-level classifiers can effectively eliminate the noisy appearance. However, the classification accuracy will be deteriorated if undersegmentation cannot be fully avoided in superpixel-based approaches. Therefore, it is proposed to adaptively combine both the pixel-level and superpixel-level classifiers, to improve the classification performance in both homogenous and structural areas. In the proposed method, a support vector machine classifier is first applied to estimate the pixel-level class probabilities. Then, superpixel-level class probabilities are estimated based on a joint sparse representation. Finally, the two levels of class probabilities are adaptively combined in a maximum a posteriori estimation model, and the classification map is obtained by solving the maximum optimization problem. Experimental results on real HSI images demonstrate the superiority of the proposed method over several well-known classification approaches in terms of classification accuracy.

A novel set-to-set distance-based spectral-spatial classification method for hyperspectral images (HSIs) is proposed. In HSIs, the spatially connected and spectrally similar pixels within each homogeneous region can be considered as one set of test samples, i.e., a test set, which should belong to the same class. In addition, each class of labeled pixels can be regarded as one set of training samples, i.e., a training set. Therefore, it is a natural consideration in the proposed method to measure the similarity between test and training sets via specific set-based distance criteria and then decide the classification label for each test set, accordingly. Specifically, the superpixel algorithm-based oversegmentation technique jointly exploits both the spatial similarity and structural information to first divide the HSI into multiple perceptually uniform regions. As a result, each segmented region corresponds to one test set. Then, each test/training set is represented with an affine hull (AH) model, which utilizes both the similarity and variance of pixels within each set to adaptively characterize the set. Finally, the class label for each test set is determined based on the closest geometry distance between test and training AHs. Experimental results on real HSI data sets demonstrate the superiority of the proposed algorithm over several well-known classification approaches, in terms of classification accuracy and computational speed.

In this paper, a novel spectral–spatial adaptive sparse representation (SSASR) method is proposed for hyperspectral image (HSI) denoising. The proposed SSASR method aims at improving noise-free estimation for noisy HSI by making full use of highly correlated spectral information and highly similar spatial information via sparse representation, which consists of the following three steps. First, according to spectral correlation across bands, the HSI is partitioned into several nonoverlapping band subsets. Each band subset contains multiple continuous bands with highly similar spectral characteristics. Then, within each band subset, shape-adaptive local regions consisting of spatially similar pixels are searched in spatial domain. This way, spectral–spatial similar pixels can be grouped. Finally, the highly correlated and similar spectral–spatial information in each group is effectively used via the joint sparse coding, in order to generate better noise-free estimation. The proposed SSASR method is evaluated by different objective metrics in both real and simulated experiments. The numerical and visual comparison results demonstrate the effectiveness and superiority of the proposed method.

In this paper, a gradient-guided sparse representation method (GGSR) for the hyperspectral image denoising is proposed. In the context of the hyperspectral image, neighbourhood spectral bands always have highly similar spatial and structural characteristics, which can be jointly used to improve the image quality. On the one hand, the sparse representation, as one powerful image processing tool, is introduced to jointly sparsely code similar image patches from different spectral bands. By this way, the redundant spatial similarity can be effectively exploited. On the other hand, the reference gradient is incorporated with the sparse representation model, in order to exploit the redundant structural information to better preserve the structure/texture characteristics. Practically, the gradient reference can be estimated from the neighbouring structural similar spectral bands. Experimental results demonstrate the effectiveness of the proposed method in removing noise as well as preserving structures.