Challenges in remote sensing object detection (RSOD), such as high inter-class similarity, imbalanced foreground-background distribution, and the small size of objects in remote sensing images significantly hinder detection accuracy. Moreover, the trade-off between model accuracy and computational complexity poses additional constraints on the application of RSOD algorithms. To address these issues, this study proposes an efficient and lightweight RSOD algorithm integrating multi-scale receptive fields and foreground focus mechanism, named Robust Foreground Weighted Network (RFWNet). Specifically, we proposed a lightweight backbone network Receptive Field Adaptive Selection Network (RFASNet), leveraging the rich context information of remote sensing images to enhance class separability. Additionally, we developed a Foreground Background Separation Module (FBSM) consisting of a Background Redundant Information Filtering Module (BRIFM) and a Foreground Information Enhancement Module (FIEM) to emphasize critical regions within images while filtering redundant background information. Finally, we designed a loss function, the Weighted CIoU-Wasserstein loss (LWCW), which weights the IoU-based loss by using the Normalized Wasserstein Distance to mitigate model sensitivity to small object position deviations. The comprehensive experimental results demonstrate that RFWNet achieved 95.3% and 73.2% mAP with 6.0M parameters on the DOTA V1.0 and NWPU VHR-10 datasets, respectively, with an inference speed of 52 FPS.

Hyperspectral images (HSIs) and multispectral images (MSIs) possess complementary advantages, with HSI providing rich spectral information and MSI offering fine spatial details. Therefore, fusing HSI and MSI to obtain high-resolution hyperspectral images (HR-HSIs) with both high spatial detail and rich spectral information has drawn increased attention. However, convolutional neural network (CNN)-based methods are limited by their local receptive fields, while traditional Transformer architectures suffer from extremely high computational complexity when applied to HSI with numerous spectral bands. To address these issues, we propose a novel multiscale Spatial-Spectral Dilated Transformer (SSDT) network based on the Transformer framework. The proposed method adopts a dual-branch architecture, consisting of the Spatial Dilated MultiScale Transformer (Spa-DMST) and the Spectral Dilated Positional Embedded Transformer (Spe-DPET) modules. Specifically, Spa-MSDT introduces a multi-level dilated window mechanism to enable cross-scale spatial feature extraction, while Spe-PEDT employs position encoding-enhanced dilated attention to reduce computational complexity. To further optimize the feature fusion process, we design a Gating Fusion Reconstruction (GFR) module that integrates depthwise separable convolutions with a gating mechanism, effectively suppressing redundant information and enhancing detail reconstruction capabilities. Extensive experiments and visualized results on three simulated datasets and one real dataset demonstrate that the proposed method outperforms other state-of-the-art fusion methods. The code of this paper will be available at https://github.com/FxyPd/SSDT.

Hyperspectral anomaly detection (HAD) aims to identify anomalous objects from hyperspectral images (HSIs) whose spectral features significantly deviate from their surroundings. Existing HAD methods still reconstruct some anomalies during the background reconstruction process, which can seriously affect the detection accuracy. Consequently, inspired by Mamba’s ability to effectively model long sequences, we propose a multiscale Mamba reconstruction network for HAD (MMR-HAD) by enhancing the representation of the background and inhibit anomalies from being reconstructed. MMR-HAD first removes most of the anomalous pixels in HSIs using the random mask (RM) strategy, which reduces the interference of anomalous pixels on the background reconstruction and makes the background features more prominent. To further filter out the small amount of residual anomalous pixels, we propose the multiscale dilated attention background enhancement (MDABE) mechanism, which enhances the background representation. Finally, we apply the multiscale dynamic feature fusion (MDFF) strategy to reconstruct the background image, further extracting and strengthening the background information, thus obtaining a pure background image. MMR-HAD, which focuses on generating a pure background, has been experimentally validated on seven real hyperspectral datasets. The results demonstrate that it excels in background enhancement and anomaly suppression, significantly improving detection accuracy. Introducing Mamba offers a promising solution for HAD, with substantial potential for practical application.

Hyperspectral image (HSI) and multispectral image (MSI) fusion aims to generate high-resolution HSI by leveraging the high spectral fidelity of HSI and the fine spatial details of MSI. However, most existing methods rely on static fusion strategies that assume global consistency in modality contributions, ignoring the inherent regional variability in real-world remote sensing scenes. To address this limitation, we propose an adaptive expert learning framework (AELF) that dynamically models the modal dominance of different regions and adaptively adjusts fusion strategies accordingly. A core component of AELF is the modality-guided complementary module (MGCM), which establishes bidirectional cross-attention pathways between HSI and MSI. It enables each modality to adaptively discover complementary cues across multiple scales while suppressing irrelevant information, providing enhanced feature representation for subsequent fine-grained fusion. Building upon this, we designed the attribute-aware mixture of fusion experts (AMoFE) module, which decomposes the fused features into spectral, spatial, and edge subspaces. Each component is modeled by a specialized expert network, with a soft routing mechanism dynamically adjusting expert contributions based on contextual cues. Extensive experiments on benchmark datasets and a real-world dataset demonstrate that AELF achieves state-of-the-art performance in terms of spectral fidelity and spatial sharpness. Furthermore, our results confirm that the improved data quality brought by the proposed method effectively enhances the overall performance of downstream tasks. The code will be available at https://github.com/Hewq77/AELF.

Different tree species have different carbon storage and growth rates. Therefore, accurate segmentation and identification of individual trees can provide more detailed carbon storage data, which is the basis for accurately estimating forest carbon storage. However, individual tree segmentation and recognition in dense forest areas face challenges such as crown overlap, complex terrain, and species diversity. To address these challenges and improve recognition accuracy, this paper proposes a fuzzy boundary-aware network (FBAN) for hyperspectral individual tree segmentation and recognition in dense forests. The proposed FBAN inclues a boundary-aware module (BAM) that explores channel boundaries between trees and non-trees, spatial boundaries of trees, and spectral boundaries between different trees by intergrating channel attention, spaital attention, and spectral attention. This enhances the separability of individual trees, especially those of the same species that are contiguous in dense forest areas. Additionally, an adaptive crown-aware module (ACAM) is constructed to adapt diverse-size crown features by coupling Transformer layers with dialted convolution layers. Experimental results on different hyperspectral datasets show that the proposed FBAN network outperforms existing methods in dense forest areas and different tree canopy areas, e.g., the AP on the SZU-South dataset is 4.5 points higher than that of Mask2former. It not only improves the accuracy of individual tree segmentation and recognition but also exhibits high generalization and robustness.

Hyperspectral image super-resolution aims to reconstruct high-quality spatial-spectral cubes from RGB images. However, the limited spectral coverage of RGB inputs hinders the simultaneous modeling of spatial structures and spectral relationships. To address this limitation, we propose a differential low-rank adaptive spatial-spectral transformer (DiLAST). Initially, a differential operator is employed to enhance RGB features in a bottom-up manner, explicitly amplifying subtle inter-channel differences. The enhanced features are then fed into a U-shaped backbone, which integrates three complementary modules for joint spatial-spectral modeling. Specifically, a center spatial-spectral attention (CSSA) module employs cross-attention mechanisms to capture local-to-global dependencies across both spatial and spectral domains; an adaptive cross-scale fusion (ACF) module utilizes learnable gating weights to establish dynamic interaction pathways between shallow high-frequency details and deep semantic representations; and a low-rank spectral calibration (LRSC) module exploits low-rank matrix priors to reveal low-dimensional manifold structures among spectral bands, thereby enhancing spectral consistency. By leveraging the synergistic effects of spatial non-locality, global spectral correlation, and low-rank properties, the proposed DiLAST achieves PSNR improvements of 33.82 dB, 36.03 dB, and 37.01 dB on benchmark datasets. Moreover, the accuracy and practical applicability of the reconstructed spectra have been effectively validated in remote sensing scenarios and object tracking tasks. The code is accessible at https://github.com/renqi1998/DiLAST.

Multispectral and hyperspectral image fusion (MHIF) seeks to combine high-resolution multispectral images (HR-MSIs) with low-resolution hyperspectral images (LR-HSIs) to create high-resolution hyperspectral images (HR-HSIs). Transformer-based architectures have recently become prominent in MHIF tasks due to their effective global self-attention mechanisms. However, the quadratic computational complexity of the global self-attention in Transformers presents significant challenges for practical applications. In this paper, we propose an enhanced spatial-frequency synergistic (ESFS) approach that leverages both spatial and frequency domain features to enhance fusion quality. Our ESFS framework introduces the condensed spatial augmentation module (CSAM), which condenses window features and employs cross-attention to balance extensive contextual understanding and detailed local feature extraction while reducing computational overhead. Additionally, we develop the selective frequency decomposition module (SFDM), which utilizes global filters composed of phase and amplitude information in the frequency domain to retain features, effectively capturing deep frequency domain characteristics and their interdependencies. Comprehensive experiments on three benchmark MHIF datasets demonstrate that our method achieves superior performance, establishing a new state-of-the-art (SOTA) in both quantitative metrics and visual quality assessments. The code is available at http://szu-hsilab.com/.

Hyperspectral images (HSIs) are vital for scene analysis, as they capture detailed spatial and spectral information to characterize surface materials. However, accurate HSI classification is challenged by significant intra-class spectral variability and spatial complexity. To address this, we leverage the fact that pixels of the same class typically form irregular local regions. We propose a structure-adaptive graph transformer (SAGT) that dynamically captures irregular spatial topologies and homogeneous spectral information to achieve adaptive HSI representation and precise classification. Specifically, a structure-aware self-attention (SASA) module is developed to embed graph structures into the self-attention mechanism as a robust positional indicator, which can be extended easily and effectively. SASA comprehensively accounts for the spatial structures and spectral autocorrelation of ground objects, facilitating the aggregation of homogeneous spectral information for noise-robust spectral representations. Additionally, a structure-adaptive pooling (SAP) module is designed to dynamically adjust graph structures by discarding irrelevant edges, thus better indicating spatial relationships. By coupling the SASA and SAP modules, our proposed SAGT model significantly alleviates spectral variability and tolerates prior noise. Furthermore, data augmentation techniques of random discard and random offset are built, which randomly drop and shift graph nodes to generate more diverse samples during preprocessing. In postprocessing, multiview decision-making integrates results from multiple contextual views to provide more robust predictions. Experimental results on three benchmark datasets consistently demonstrate that SAGT is more effective and reliable than other state-of-the-art methods. To facilitate reproduction, we will release the source code for SAGT at https://github.com/ShuGuoJ/SAGT.git.

As hyperspectral images (HSIs) continue to increase in data resolution and information richness, current deep learning models need to enhance their feature extraction and understanding capabilities for classification tasks. The complementarity between convolution and attention mechanisms in deep learning enables the capture of both local and global information. However, it faces the problems of intrinsic coupling of different operators and precise fusion of different features. In this study, a novel global-local residual fusion network (GLRFNet) is proposed to improve the HSI classification. Firstly, a feature projection with multiple kernels is designed to generate the feature pool before deep extraction and enhance the information connection between operators. Then, a global-local residual (GLR) feature extraction network is built to capture both fine-grained details and large-scale dependencies, improving the feature perception in various classification scenes. It consists of local convolution, global attention, and residual construction branches in a separate and coupled manner. Finally, an optimized inverted bottleneck fusion (IBF) module is built to perform nonlinear and comprehensive feature fusion for mining the high-level semantics and understanding the category relationships. The experiments on four HSI datasets demonstrate the superiority of GLRFNet compared to other state-of-the-art methods, especially its classification performance under small sample conditions, with higher recognition accuracy and better class boundaries. In addition, parameter analysis and ablation experiments are also conducted to determine the optimal parameters and verify the module effectiveness.

Hyperspectral videos (HSVs), with their inherent spatial-spectral-temporal structure, offer distinct advantages in challenging tracking scenarios such as cluttered backgrounds and small objects. However, existing methods primarily focus on spatial interactions between the template and search regions, often overlooking spectral interactions, leading to suboptimal performance. To address this issue, this paper investigates spectral interactions from both the architectural and training perspectives. At the architectural level, we first establish band-wise long-range spatial relationships between the template and search regions using Transformers. We then model spectral interactions using the inclusion-exclusion principle from set theory, treating them as the union of spatial interactions across all bands. This enables the effective integration of both shared and band-specific spatial cues. At the training level, we introduce a spectral loss to enforce material distribution alignment between the template and predicted regions, enhancing robustness to shape deformation and appearance variations. Extensive experiments demonstrate that our tracker achieves state-of-the-art tracking performance. The source code, trained models and results will be publicly available via https://github.com/bearshng/suit to support reproducibility.

The joint classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) data is gaining attention for its improved classification accuracy. However, effectively integrating the rich spectral information of HSI and the elevation features of LiDAR has remained a challenge in multimodal fusion. This article proposes a novel approach called progressive semantic enhancement network (PSENet) for hyperspectral and LiDAR classification based on a progressive joint spatial-spectral attention mechanism. PSENet mainly comprises two modules: the spatial grouping constraint (SAGC) module and the spectral weighting constraint (SEWC) module. The SAGC module extracts multiscale features in the spatial domain, while the SEWC module focuses on enhancing semantic features in spectral dimension. By gradually utilizing spatial and spectral constraint modules to progressively enhance feature extraction, PSENet integrates affluent information for a more refined classification of ground objects. Based on experimental results, it has been demonstrated that PSENet outperforms several most advanced methods on three datasets. The SAGC and SEWC modules proposed in PSENet enable the effective integration of the spatial, spectral, and elevation information from HSI and LiDAR, providing a promising way to perform classification more accurately. The source codes of this work will be publicly available at http://szu-hsilab.com/.

With its strong capability in modeling long-range dependencies, the Transformer achieves competitive performance in hyperspectral image (HSI) and multispectral image (MSI) fusion. However, existing Transformer-based methods face the trade-off between receptive field size and computational efficiency when dealing with spatially non-local features. Furthermore, the Transformer captures deep spectral relationships by modeling pairwise channel interactions. This global interaction may overlook features that contribute little to the overall context but are critical locally, thus affecting the accurate understanding of HSI content. To overcome these challenges, we propose a novel local-global collaborative network with Transformers (LGCT) specifically designed to achieve high-quality HSI reconstruction. The proposed LGCT includes two inverse feature streams to establish multiscale deep representations of the HSI and MSI features. The feature streams comprise collaborative Transformer blocks (CTBs) explicitly designed for the spectral and spatial domains. By combining global and local processing mechanisms, the proposed CTBs can efficiently emphasize potential crucial features that Transformer ignores when capturing deep spectral and spatial relationships, thus enabling efficient modeling of the spectral and spatial domains from details to the whole. Furthermore, to enhance the reusability of multiscale enhanced features from the spectral and spatial domains, a hierarchical and symmetric strategy is adopted to progressively fuse them to generate high-quality images. The results on both simulated and real datasets demonstrate the superior performance of the proposed method in terms of quantitative metrics and visual quality. The code will be released athttps://github.com/Hewq77/LGCT.

Convolutional neural networks (CNNs) are widely used in hyperspectral image (HSI) classification. However, the fixed receptive field of CNN-based methods limits their capability to extract global features. In recent years, transformer has been introduced into networks to tackle this limitation, but it brings other challenges, including a significant increase in model size, the number of labeled training samples required, and the limited effectiveness of sample encoding-reconstruction pretraining methods for HSI classification. To address these issues, a center-masked transformer (CMT) approach is proposed to improve the HSI classification accuracy from two perspectives. On one hand, a local-to-global token embedding (L2GTE) framework coupled with a multiscale convolutional token embedding (MCTE) module is used, which is well-designed to obtain local and global embedding tokens. This effectively reduces the number of model parameters. On the other hand, a regularized center-masked pretraining (RCPT) task is proposed and first introduced into the transformer-based network, which enables the network to learn the dependencies between central ground objects and neighboring objects without labels during the pretraining process. The experimental results conducted on five public HSI datasets demonstrate that our CMT approach outperforms other state-of-the-art methods for HSI classification when training samples are insufficient.

Super-resolution is vital for the quality improvement of hyperspectral images (HSIs) under the spatial and spectral resolution trade-off. However, deep learning HSI super-resolution approaches typically adopt the “one model and one scale” scheme that is inefficient in training and storing. This is difficult in maximizing orbit equipment performance and aligning multiple spatial resolution data in remote sensing. Therefore, this article intends to address HSI arbitrary-scale super-resolution, enabling the scaling of HSIs to arbitrary sizes using a single model. To do this end, we treat HSI arbitrary-scale super-resolution as a retrieval problem. It conceptualizes the HSI as a dictionary of pixelwise tokens with spatial-spectral features, position information, and scale information. Its objective is to employ a set of initialized tokens related to the high-resolution (HR) HSI as queries to retrieve matched spectral features from low-resolution (LR) one, which is so-called token-based query-to-spectrum. Since these query tokens can be constructed flexibly (e.g., through random initialization), we can generate a desired number of them to reconstruct our HR HSI, thus achieving arbitrary-scale super-resolution. This process considers not only position information but also spectral features so that it can decrease spectral distortion. With the above idea, we developed an HSI arbitrary-scale super-resolution method, dubbed as spectral-query transformer (SQformer). Specifically, it begins by converting the LR HSI into a dictionary of LR tokens and then constructs a desired number of HR tokens. To enable flexible token construction, we design an implicit spectral token (particularly a learnable vector) and replicate it$\alpha H \times \alpha W$times to form the HR tokens. Next, the HR and LR tokens are passed into a transformer decoder to find the most matched spectral response for the former by soft-weighting the LR tokens. Finally, the HR tokens are spatially rearranged in order, forming an HR HSI. Extensive experiments have demonstrated its effectiveness on remote sensing data. The code will be released at:https://github.com/ShuGuoJ/SQformer.git.

Forests play an irreplaceable role in carbon sinks. However, there are obvious differences in the carbon sink capacity of different tree species, so the scientific and accurate identification of surface forest vegetation is the key to achieving the double carbon goal. Due to the disordered distribution of trees, varied crown geometry, and high difficulty in labeling tree species, traditional methods have a poor ability to represent complex spatial–spectral structures. Therefore, how to quickly and accurately obtain key and subtle features of tree species to finely identify tree species is an urgent problem to be solved in current research. To address these issues, a texture-aware self-attention model (TASAM) is proposed to improve spatial contrast and overcome spectral variance, achieving accurate classification of tree species hyperspectral images (HSIs). In our model, a nested spatial pyramid module is first constructed to accurately extract the multiview and multiscale features that highlight the distinction between tree species and surrounding backgrounds. In addition, a cross-spectral–spatial attention module is designed, which can capture spatial–spectral joint features over the entire image domain. The Gabor feature is introduced as an auxiliary function to guide self-attention to autonomously focus on latent space texture features, further extract more appropriate and accurate information, and enhance the distinction between the target and the background. Verification experiments on three tree species hyperspectral datasets prove that the proposed method can obtain finer and more accurate tree species classification under the condition of limited labeled samples. This method can effectively solve the problem of tree species classification in complex forest structures and can meet the application requirements of tree species diversity monitoring, forestry resource investigation, and forestry carbon sink analysis based on HSIs.

Unmanned aerial vehicle (UAV) hyperspectral imaging offers an efficient and cost-effective way to map tree species at the individual tree levels. Conventional methods mostly rely on large samples of natural RGB images of tree crowns, lacking the ability to distinguish species, particularly for trees with overlapping crowns. This study proposed a novel scale pyramid graph network (SPGN) for instance segmentation that can simultaneously apply pixel-level (node) classification for discriminating species and edge prediction for delineating individual trees. Based on a graph-in-graph (GiG) convolution, we built a scale pyramid module (SPM) that extracts multiscale features at pixels, superpixels, and subgraph levels to aggregate the over-segmented superpixels into the same species and the same tree. We also proposed an innovative concept of subgraph positional encoding (SPE) to represent the natural spatial relationship of graph-structured data. The SPGN method was evaluated in a case study involving eleven subtropical broadleaf species under an urban environment in south China. The accuracy of species classification achieved 93%, and the area under the curve (AUC) of individual tree segmentation reached 0.96. Compared with state-of-the-art methods such as DeepForest, Detectree2, and segment anything model (SAM), SPGN presented fewer errors in tree detection and outperformed in instances of crown overlaps. Ablation studies proved the effectiveness of SPM and SPE modules, which improved segmentation by 10% and classification by 7% in accuracy, respectively. The findings confirm the benefits of incorporating spatial context, such as crown textures and tree positional relationships, for species differentiation; in return, accurate species identification combined with spectral information assists the individual tree segmentation. This effective strategy can be potentially extended to a broader range of regions and forest types.

The international array for real-time geostrophic oceanography (Argo) project is committed to rapidly and precisely acquiring comprehensive 3-D data on ocean temperature and salinity, which is crucial for monitoring ocean climate change and natural phenomena. During the buoy observation, environmental factors, human mistakes, and equipment malfunctions can cause abnormalities such as density inversion and spike, and thus detecting the errors in Argo data is significant to ensure its reliability and applicability. Traditional methods mainly rely on the knowledge and judgment of marine experts, ensuring high accuracy but requiring large amounts of effort. Machine-learning methods are used for automatic Argo data error detection, while they still struggle with extracting deep and discriminative features from profiles. Recently, deep-learning methods have received increasing attention in this field, yet their effectiveness have not been widely explored, faced with challenges of imbalanced samples, joint detection, and complicated patterns. In this article, a novel vertical attention-based siamese ConvLSTM (VAS-CLSTM) network is proposed for the accurate error detection of Argo data. First, an oversampling approach with optimized deep clustering based on inheritance theory and Mahalanobis distance is designed to effectively augment the error samples. Second, a siamese convolutional long–short-term memory (ConvLSTM) network with contextual connection and spatial–temporal adjacent profile search is built to learn interactively from temperature and salinity profiles. Third, a depth-based vertical attention mechanism with grouped weights and vertical trends is proposed for adaptive modeling and flexible learning. Experimental results of North and South Atlantic datasets show that the proposed VAS-CLSTM method effectively improves the accuracy and reliability of error detection in Argo observation data.

The reconstruction of subsurface ocean temperature using sea surface observations and in situ Argo measurements is an important yet challenging task. The availability of long-term and high-resolution sea surface remote sensing, combined with advancements in deep learning technology, has opened new opportunities for studying subsurface temperature (ST) reconstruction. In this study, a novel spatial–temporal Siamese convolutional neural network (SSCNN) is proposed to improve the accuracy of ST reconstruction in the Indian Ocean. First, considering the distinctions of temperature characteristics among different sea areas, a multiscale division scheme based on the correlation coefficient of integral ST is designed for refined reconstruction modeling. Second, since ocean heat is significantly affected by solar radiation, asymmetric convolutional operation with rectangular patches and kernels is designed to capture the information characteristics in longitude and latitude directions, respectively. Third, given the temporal changes and correlations of ocean temperature, an SSCNN with shared parameters is proposed for multiview feature mining and accurate temperature structure reconstruction. The reconstructed results provide a precise depiction of the subsurface Indian Ocean dipole (sub-IOD)’s evolution, including the spatial distribution of positive and negative anomaly signals and its temporal changes. It demonstrates that the subsurface dipole index series obtained from SSCNN reconstruction is consistent with that from International Pacific Research Center (IPRC) observation, remaining within a reasonable error range. Comparative experiments indicate that the SSCNN model surpasses other existing methods in terms of higher accuracy and smaller error. Overall, this study provides a promising approach for effectively reconstructing the ST using deep learning methods and offers valuable insights for analyzing the evolution of subsurface positive dipole in Indian Ocean.

With the development of hyperspectral sensors, accessible hyperspectral images (HSIs) are increasing, and pixel-oriented classification has attracted much attention. Recently, graph convolutional networks (GCNs) have been proposed to process graph-structured data in non-Euclidean domains and have been employed in HSI classification. But most methods based on GCN are hard to sufficiently exploit information of ground objects due to feature aggregation. To solve this issue, in this article, we proposed a graph-in-graph (GiG) model and a related GiG convolutional network (GiGCN) for HSI classification from a superpixel viewpoint. The GiG representation covers information inside and outside superpixels, respectively, corresponding to the local and global characteristics of ground objects. Concretely, after segmenting HSI into disjoint superpixels, each one is converted to an internal graph. Meanwhile, an external graph is constructed according to the spatial adjacent relationships among superpixels. Significantly, each node in the external graph embeds a corresponding internal graph, forming the so-called GiG structure. Then, GiGCN composed of internal and External graph convolution (EGC) is designed to extract hierarchical features and integrate them into multiple scales, improving the discriminability of GiGCN. Ensemble learning is incorporated to further boost the robustness of GiGCN. It is worth noting that we are the first to propose the GiG framework from the superpixel point and the GiGCN scheme for HSI classification. Experiment results on four benchmark datasets demonstrate that our proposed method is effective and feasible for HSI classification with limited labeled samples. For study replication, the code developed for this study is available at https://github.com/ShuGuoJ/GiGCN.git.

Hyperspectral cameras capture electromagnetic information within hundreds of narrow spectral bands, producing hyperspectral images (HSIs) with the capability to accurately characterize the attribute information of objects. However, mixed noise induced by instrument and atmospheric effects hinders the interpretations and applications of the HSIs. In this paper, we propose a novel subspace representation based mixed noise removal method for hyperspectral images via Robust Subspace Estimation and weighted Group Sparsity constraint (RoSEGS). An outlier detection method is proposed to effectively detect sparse noise and replace the sparse noise with new estimates. A subspace estimation strategy, which is robust to mixed noise, is proposed. The subspace is first estimated after sparse noise detection and then optimized iteratively. In addition to the introduction of a state-of-the-art denoiser based on the plug-and-play technique to exploit self-similarity characteristics of the eigen-images, we impose a weighted group sparse regularization on the eigen-images to better promote the group sparsity of the spatial differences between the eigen-images, which further improves the denoising performance. We performed extensive experiments on two simulated and two real HSIs to fully demonstrate the effectiveness of the proposed method in comparison with seven state-of-the-art competitors.

Hyperspectral images (HSIs) possess the capability to accurately characterize the attribute information of objects. However, they are usually obtained at a high spectral resolution with a compromise of its spatial resolution. In addition, they are easily contaminated by mixed noise induced by instrument and atmospheric effects. These disadvantages, to a certain degree, hinder the interpretations and applications of the HSIs. To overcome these limitations, in this paper, we propose a novel Mixed noise-oriented hyperspectral and multispectral image Fusion method, termed (MixFus). First, a sparse noise detection method is proposed by first leveraging a subset of specifically chosen hyperspectral bands to estimate noise in HSI and then employing Gaussian mixture models to detect sparse noise from the estimated noise. Then, a robust subspace estimation method is introduced by replacing the detected sparse noise with new estimates using median values within a sliding window for a better estimation of the subspace, which offers improved accuracy and robustness of subspace estimation. Finally, in addition to the introduction of a state-of-the-art image prior based on the plug-and-play technique to exploit self-similarity characteristics in the eigen-images, we also impose a weighted group sparse regularization on the eigen-images to better promote the group sparsity of the spatial differences between the eigen-images, which further improve the denoising performance. We evaluate the proposed method by performing extensive experiments on three reduced-resolution HSIs and a full-resolution HSI in comparison with seven state-of-the-art competitors. Experimental results demonstrate the superiority of the proposed method over the competitors in the fusion of hyperspectral and multispectral images against mixed noise.

The necessary prerequisite for effective data fusion is the strict registration of low-resolution hyperspectral images (LR-HSI) and high-resolution multispectral images (HR-MSI). However, registration requires a complex process that takes into account the effects of light, imaging angle, and geometric distortion of the image during acquisition. Therefore, to avoid complex registration, we focused on developing an unregistered HSI and MSI fusion method for pixel shifting, obtaining fused images with high resolution, high signal-to-noise ratio, and feature identifiability. We identified that the unregistered LR-HSI and HR-MSI in the case of pixel shift are very similar to the disparity maps in stereo vision. Inspired by this, we simulate the structure of stereo cameras to propose a stereo cross-attention network (SCANet) to achieve an accurate fusion of unregistered LR-HSI and HR-MSI. Considering the model complexity and computing efficiency, we design a simple and stackable stereo cross-fusion block (SCFBlock) based on a Transformer to simulate the process of light entering the left and right cameras by extracting the abstract features of the images. Moreover, the purpose of cross-convergence fusion self-attention (CCFSA) is to learn cross-complementary attention and collect contextual information in horizontal and vertical directions to fuse unregistered images using multi-directional cross-view information. We have conducted extensive experiments on Pavia University (PaviaU), Chikusei, and PYLake datasets. The results show that SCANet achieves superior or competitive performance in fusing unregistered LR-HSI and HR-MSI in comparison with the other competitors.

Hyperspectral image (HSI) classification based on deep learning is a hot research topic. The convolutional model employs a single rectangular window to interpret the sample neighborhood features, whereas effective characterization of the complex spatial structure of HSI is still an unsolved problem. In this article, we propose a structure-adaptive convolutional neural network (SACNN) for HSI classification, which efficiently exploits the intrinsic spatial geometry information. Four novel strategies are designed to construct the proposed SACNN network. First, superpixel homogeneous region (SHR) sample generation is introduced to achieve neighborhood features within the intercepted rectangular window of the superpixel. Second, online batch-wise standardization uses zero padding to unify the size of inputs in the same batch, thereby realizing parallel processing of irregular inputs. Third, structure-adaptive convolution (SConv) and structure-adaptive average pooling (SAP) are correspondingly constructed to extract deep spectral, spatial, and geometric features from the effective mapping area of superpixels, and further aggregate the information within irregular boundaries. Finally, a sample-adaptive loss weight (SLW) scheme is designed to adjust the influence of different labels on the same input. Experimental results show that the overall classification accuracy of SACNN reaches 93.11%, 90.96%, and 85.04% for 15 randomly selected training samples per class on three HSI datasets, respectively, obtaining an improvement of 0.97%–2.97% with respect to the best-compared method.

Using single-source remote sensing (RS) data for classification of ground objects has certain limitations, however, multi-modal RS data contain different types of features, such as spectral features and spatial features of hyperspectral image (HSI) and elevation information of light detection and ranging (LiDAR) data, which can be used to extract and fuse high-quality features to improve the classification accuracy. Nevertheless, the existing fusion techniques are mostly limited by the number of labeled samples due to the difficulty of label collection in the multi-modal RS data. In this article, a fusion method of collaborative contrastive learning (CCL) is proposed to tackle the abovementioned issues for HSI and LiDAR data classification. The proposed CCL approach includes two stages of pre-training (CCL-PT) and fine-tuning (CCL-FT). In the CCL-PT stage, a collaborative strategy is introduced into contrastive learning (CL), which can extract features from HSI and LiDAR data separately, and achieve the coordinated feature representation and matching between the two-modal RS data without labeled samples. In the CCL-FT stage, a multi-level fusion network is designed to optimize and fuse the unsupervised collaborative features which are extracted in the CCL-PT stage for the classification tasks. Experimental results on three real-world data sets show that the developed CCL approach can perform excellently on the small sample classification tasks and CL is feasible for the fusion of multi-modal RS data.

With the rapid development of deep convolutional neural networks (CNNs), super-resolution (SR) in hyperspectral image (HSI) has achieved good results. Current methods generally use 2-D convolution for feature extraction, but they cannot effectively extract spectral information. Although 3-D convolution can better characterize feature structure of HSI, it will lead to parameter redundancy, model complexity, and severe memory shortage. To address the above problems, we propose a new HSI SR method, named diffused CNN (DCNN). Specifically, spectral convolutions have been added into the enhanced convolutional neural (ECN) block, and a series of spectral convolutions are introduced in the residual network to learn features in the channel direction of different depths. Furthermore, histogram of oriented gradient (HOG) and local binary pattern (LBP) are used to retain the shape and texture information of the image, respectively, which can well represent the spatial structure of the object. To effectively make use of the extracted shallow and deep features, a feature fusion strategy is used to reinforce the reconstruction efficiency. Besides, an image enhancement module has been developed to diffuse the SR image into the image space. Extensive evaluations and comparisons show that our DCNN approach can not only recover the HSI data with richer details but also achieve superiority over several state-of-the-art methods.

Spatial resolution is a crucial indicator for measuring the quality of hyperspectral imaging (HSI) and obtaining high-resolution (HR) hyperspectral images without any auxiliary information has become increasingly challenging. One promising approach is to use deep-learning (DL) techniques to reconstruct HR hyperspectral images from low-resolution (LR) images, namely super-resolution (SR). While convolutional neural networks are commonly used for hyperspectral image SR (HSI-SR), they often lead to unavoidable performance degradation due to the lack of long-range dependence learning ability. In this article, we propose a dual self-attention Swin transformer SR (DSSTSR) network that utilizes the ability of the shifted windows (Swin) transformer in the spatial representation of both global and local features and learns spectral sequence information from adjacent bands of HSI. Additionally, DSSTSR incorporates an image denoising module using the wavelet transformation method to mitigate the impact of stripe noise on HSI-SR. Our extensive experiments using publicly close-range datasets demonstrate that DSSTSR outperforms other state-of-art HSI-SR methods in terms of three image quality metrics. Furthermore, we applied DSSTSR to the SR of satellite hyperspectral images and achieved improved classification results. Compared to its competitors, DSSTSR exhibits superior performance in enhancing spatial resolution while preserving spectral information. These results suggest that the DSSTSR network has great potential for standardization in remote-sensing image processing and practical applications.

Haze in hyperspectral images (HSIs) can lead to crosstalk between multiple bands, resulting in errors that can be amplified and transmitted during data processing. As a consequence, this may cause a reduction in the accuracy and precision of remote sensing data. The purpose of haze removal is to restore high-quality HSIs from degraded ones. The high spectral resolution and typically dozens to hundreds of spectral bands in HSIs pose significant challenges for haze removal. Thus, many methods designed for natural and multispectral images are not effective in removing haze in HSIs. To address this challenge, we develop a model called asymmetric attention convolution network (AACNet) designed for haze removal in HSIs. Specifically, the basic architecture of AACNet is mainly composed of several residual asymmetric attention groups (RAAGs), where the core components are residual asymmetric attention blocks (RAABs). This design enables the full utilization of deep spatial-spectral features while skipping low-frequency regions and focusing more on the haze-affected areas. To more accurately restore the spectral information in areas polluted by haze, a pooling channel self-attention (PCSA) module has been proposed. This module can effectively reconstruct the spectral response curve that is affected by the haze. Our experiments on both simulated and real datasets demonstrate that the proposed AACNet outperforms several leading haze removal methods in both precision and visual quality. The source code and data of this article will be made publicly available at https://github.com/SZU710/AACNet for reproducible research.

Argo observations are spatially sparse and temporally uneven, whereas satellites can provide high-resolution and continuous observations at the sea surface. The reconstruction of subsurface thermohaline structure using multi-source remote sensing data is thus of great significance for investigating the ocean interior dynamics. Aiming at the existing problems of temporal feature extraction and nonlinear relationship fitting, this paper proposes a multivariate temporal self-attention network (MTSAN) to effectively reconstruct the subsurface temperature anomaly (STA) and subsurface salinity anomaly (SSA) in the Pacific Ocean. The model integrates multi-source remote sensing data, including sea surface temperature and salinity, wind speed, absolute dynamic topography, and significant wave height. In order to better extract the complex small- and medium-scale signals, a two-branch asymmetric residual module based on dilation causal convolution is designed to enhance the representation ability. Moreover, zonal weighted loss function with comprehensive indicators is proposed, in order to minimize the real error of grids and raise the accuracy of self-attention network. MTSAN reconstructs the STA and SSA during the El Niño event, and the results show that it has good performance for spatial distribution, vertical variation, and temporal extension. The overallR2and RMSE of STA are 0.536 and 0.241° C, respectively, and the overallR2and RMSE of SSA are 0.645 and 0.037psu, respectively. In addition, the results of comparison experiments illustrate the superiority of MTSAN over other machine learning and deep learning based methods. Overall, we provide a new temporal self-attention approach to accurately reconstruct the three-dimensional thermohaline structure using high-resolution quasi-real-time satellite observations.

Fusing a Hyperspectral image (HSI) and a multispectral image (MSI) from different sensors is an economic and effective approach to get an image with both high spatial and spectral resolution, but localized changes between the multiplatform images can have negative impacts on the fusion. In this letter, we propose a novel sparsity constrained fusion method (SCFus) to fuse multiplatform HSIs and MSIs based on matrix factorization. Specifically, we imposed$\ell _{1}$norm on the residual term of the MSI to account for the localized changes between the hyperspectral and MSIs. Furthermore, we plugged a state-of-the-art denoiser, namely block-matching and 3-D filtering (BM3D), as the prior of the subspace coefficients by exploiting the plug-and-play framework. We refer to the proposed method as SCFus for hyperspectral and MSIs. Experimental results suggest that the proposed fusion method is more effective in fusing hyperspectral and MSIs than the competitors.

The high spectral resolution of hyperspectral images (HSIs) generally comes at the expense of low spatial resolution, which hinders the application of HSIs. Fusing an HSI and a multispectral image (MSI) from different sensors to get an image with the high spatial and spectral resolution is an economic and effective approach, but localized spatial and spectral changes between images acquired at different time instants can have negative impacts on the fusion results, which has rarely been considered in many fusion methods. In this article, we propose a novel group sparsity constrained fusion (GSFus) method to fuse hyperspectral and MSIs based on matrix factorization. Specifically, we imposed$\ell _{2,1}$norm on the residual term of the MSI to account for the localized interimage changes occurring during the acquisition of the hyperspectral and MSIs. Furthermore, by exploiting the plug-and-play framework, we plugged a state-of-the-art denoiser, namely block-matching and 3-D filtering (BM3D), as the prior of the subspace coefficients. We refer to the proposed fusion method as GSFus method. We performed fusion experiments on two kinds of datasets, i.e., with and without obvious localized changes between the HSIs and MSIs, and a full resolution dataset. Extensive experiments in comparison with seven state-of-the-art fusion methods suggest that the proposed fusion method is more effective on fusing hyperspectral and MSIs than the competitors.

With the development of hyperspectral imaging technology, hyperspectral images (HSIs) have become important when analyzing the class of ground objects. In recent years, benefiting from the massive labeled data, deep learning has achieved a series of breakthroughs in many fields of research. However, labeling HSIs requires sufficient domain knowledge and is time-consuming and laborious. Thus, how to apply deep learning effectively to small labeled samples is an important topic of research in HSI classification. To solve this problem, we propose a semisupervised Siamese network that embeds Siamese network into a semisupervised learning scheme. It integrates an autoencoder module and a Siamese network to, respectively, investigate information in a large amount of unlabeled data and rectify it with a limited labeled sample set, which is called 3DAES. First, the autoencoder method is trained on the massive unlabeled data to learn the refinement representation, creating an unsupervised feature. Second, based on this unsupervised feature, limited labeled samples are used to train a Siamese network to rectify the unsupervised feature to improve feature separability among various classes. Furthermore, by training the Siamese network, a random sampling scheme is used to accelerate training and avoid imbalance among various sample classes. Experiments on three benchmark HSI datasets consistently demonstrate the effectiveness and robustness of the proposed 3DAES approach with limited labeled samples. For study replication, the code developed for this study is available athttps://github.com/ShuGuoJ/3DAES.git.

Due to the detailed spectral information through hundreds of narrow spectral bands provided by hyperspectral image (HSI) data, it can be employed to accurately classify diverse materials of interest, which is one of the core applications of hyperspectral remote sensing technology. In recent years, with the rapid development of deep learning, convolutional neural networks (CNNs) have been successfully applied in many fields, including HSI classification. However, the random gradient descent-based parameter updating scheme is too general and leading to the inefficiency of CNN models. Moreover, the high dimensionality and limited training samples of HSI data also exacerbate the overfitting problem. To tackle these issues, in this article, a novel deep network with multilayer and multibranch architecture, named 3-D Gabor CNN (3DG-CNN), is proposed for HSI classification. More precisely, since the predefined 3-D Gabor filters in multiple scales and orientations could well characterize the internal spatial–spectral structure of HSI data from various perspectives, the 3-D Gabor-modulated kernels (3-D GMKs) are employed to replace the random initialization kernels. Moreover, the specially designed multibranch architecture enables the network to better integrating the scalable property of 3-D Gabor filters; thus, the representative ability and robustness of the extracted features can be greatly improved. Alternatively, the number of network parameters is substantially reduced due to the incorporation of 3-D Gabor modulation, relieving the training complexity and also alleviating the training process from overfitting. Experimental results on four real HSI datasets (including two newly released ones in the literature) have demonstrated that the proposed 3DG-CNN model can achieve better performance than several widely used machine-learning-based and deep-learning-based approaches. For the sake of reproducibility, the codes of the proposed 3DG-CNN model are available athttp://jiasen.tech/papers/.

Domain adaptation, which cleverly applies the classifier learned from the source domain with sufficient labeled samples to the target domain with limited labeled samples, provides a feasible alternative to handle the small training sample problem of hyperspectral image (HSI) classification and has attracted much attention in the research field recently. Apparently, feature discriminative ability is vital for domain adaptation, which plays a crucial role during the migration process of transfer learning. In this article, a gradient feature-oriented 3-D domain adaptation (GF-3DDA) approach is proposed for HSI classification. First, 3-D Gabor is employed to remove noise from the original data, and two 2-D gradient-based features, 2-D Sobel gradient (SG) and 2-D derivative-of-Gaussian (DtG), are extended to the 3-D domain to coincide with the integrated spatial–spectral organization of HSI. Thus, the 3-D Sobel–Gabor gradient (3DSGG) and 3-D derivative-of-Gaussian-Gabor (3DDGG) features are achieved. Second, a 3-D domain adaptation method is implemented to jointly exploit the second- and fourth-order statistical descriptors in the spatial–spectral dimensions, which could effectively reduce domain shifts and thus achieve improved domain adaptation. Third, all the extracted domain-adapted feature modules are collaboratively classified by extreme learning machine (ELM), and the probability-like outputs of every ELM classifier are combined together to accomplish the classification task. Four hyperspectral data sets that each contains two scenes, i.e., Pavia, Shanghai–Hangzhou, Indiana, and Houston, are tested in the experiments. When only ten labeled samples per class are used in the target domain, the classification accuracies on four hyperspectral data sets achieved by our GF-3DDA approach are 93.31%, 84.35%, 69.32%, and 80.06%, respectively.

Image super-resolution (SR) methods can generate remote sensing images with high spatial resolution without increasing the cost of acquisition equipment, thereby providing a feasible way to improve the quality of remote sensing images. Clearly, image SR is a severe ill-posed problem. With the development of deep learning, the powerful fitting ability of deep neural networks has solved this problem to some extent. Since the texture information of various remote sensing images are totally different from each other, in this paper, we proposed a network based on generative adversarial network (GAN) to achieve high resolution remote sensing images, named multi-attention generative adversarial network (MA-GAN). The main body of the generator in MA-GAN contains three blocks: pyramid-convolutional residualdense (PCRD) block, attention-based upsampling (AUP) block and attention-based fusion (AF) block. Specifically, the developed attention-pyramid convolutional (AttPConv) operator in PCRD block combines multi-scale convolution and channel attention (CA) to automatically learn and adjust the scale of residuals for better representation. The established AUP block utilizes pixel attention (PA) to perform arbitrary scales of upsampling. And the AF block employs branch attention (BA) to integrate upsampled low-resolution images with high-level features. Besides, the loss function takes both adversarial loss and feature loss into consideration to guide the learning procedure of generator. We have compared our MA-GAN approach with several state-of-the-art methods on a number of remote sensing scenes, and experimental results consistently demonstrate the effectiveness of the proposed MA-GAN. For study replication, the source code will be released at: https://github.com/ZhihaoWang1997/MA-GAN.

Hyperspectral imagery (HSI) contains hundreds of bands, which provide a wealth of spectral information and enable better characterization of features. However, the excessive dimensionality also poses a dimensional disaster for subsequent processing. Fortunately, band selection (BS) gives a straightforward and effective way to pick out a subset of bands with rich information and low correlation. Although many hyperspectral BS methods, especially clustering-based ones, have been proposed by researchers in recent years, the contextual information of adjacent bands and the spatial structural information of materials are not well investigated. Therefore, in this article, a multiscale superpixel-level group-clustering framework (MSGCF) has been proposed for hyperspectral BS. Different from previous, a new superpixel-level distance measure is elaborately utilized to group and cluster the spectral bands, which jointly considers the spectral context and spatial structure information. Concretely, to preserve the spatial structural information of HSI, multiple superpixel segmentation is first performed to generate superpixel maps in multiscales, which enables complementarity of multiple superpixel segmentation algorithms and adaptation to diverse scales of land cover types. Second, the grouping and clustering paradigm is introduced to conduct the contextual information among bands. Here the maximum points of superpixel-level KL-$\ell _{1}$distance of adjacent bands are adopted as partition points to separate bands into groups, which encourages adjacent bands with strong correlation to be divided into the same group. Third, a superpixel-level fast density-based clustering method (SuFDPC) with superpixel-level$\ell _{2, 1}$distance is developed to select representative bands in every group. Finally, BS results are achieved with a ranking-based voting strategy by concerning information entropy and frequency of occurrence in a unified scheme. A series of ablation analyses and experimental comparisons on four real HSI datasets have been conducted, as well as similarity comparisons for the selected bands. The experimental results consistently demonstrated the effectiveness of our MSGCF approach. The codes of this work will be available athttp://jiasen.tech/papers/for the sake of reproducibility.

Supervised classification algorithms on the intricate ground object information of hyperspectral images (HSIs) require a large number of training samples that are annotated manually for model learning. To reduce the labeling cost and improve training sample effectiveness, a multiview spatial–spectral active learning (MVSS-AL) model is proposed in this study. First, a committee model composed of collaborative representation classification is introduced to form a leave-one-class-out (LOCO) multiview strategy, which explores more effective information in the limited training data. Second, the sample query strategy is designed from the perspective of classification confidence (CC) and training contribution (TC). The most inconsistent high-quality samples are screened by making full use of iterative prediction information and spatial–spectral features contained in hyperspectral imagery. Finally, the spatial–spectral LOCO active learning (AL) model obtains target samples through two-layer screening in each iteration and utilizes a support vector machine to obtain the final classification results. The proposed method is tested on three real-world hyperspectral datasets, and the comparison with several novel methods shows that the proposed method is better in the classification performance of restricted sample training.

3-D Gabor, as a typical filter, plays a critical role in extracting discriminative spectral–spatial features from hyperspectral images (HSIs). However, the performance of traditional 3-D Gabor is limited by the uniform response to each direction, which is inconsistent with the complexity of land cover distribution. It has been a continuing concern for researchers to investigate the anisotropic 3-D Gabor filters. In addition, the 3-D Gabor wavelets do not make full use of spatial distribution information, thus reducing the accuracy. This article proposes a superpixel-guided variable 3-D Gabor phase coding fusion (SuVGF) framework for HSI classification with limited training samples. First, the variable 3-D Gabor filters are created based on various asymmetric sinusoidal waves and spatial kernel sizes to achieve multidirectional features. Second, the local Gabor phase ternary pattern is adopted to encode the Gabor phases and improve the feature discrimination. Meanwhile, a scale map is produced by the majority voting of multiscale simple noniterative clustering (SNIC) and entropy rate superpixel (ERS) segmentation, which contains sufficient and complementary spatial distribution information. Then, geometric optimization is employed on the scale map to reduce noise disturbances. Finally, all Gabor features are modified by the filter with the guidance of a scale map and fused together as a confidence cube, and the random forest algorithm is exploited for classification. TheSuVGF is applied to three real hyperspectral datasets to demonstrate the superiority of higher accuracy, stronger robustness, and less computational complexity in comparison with several state-of-the-art ones.

The rapid development of remote sensing sensors makes the acquisition, analysis, and application of hyperspectral images (HSIs) more and more extensive. However, the limited sample sets, high-dimensional features, highly correlated bands, and mixing spectral information make the classification of HSIs a great challenge. In this article, an unsupervised multiscale and diverse feature learning (UMsDFL) method is proposed for HSI classification, which deeply considers the spatial–spectral features via convolutional neural networks (CNNs). Specifically, after employing the simple noniterative clustering (SNIC) algorithm with the heuristic calculation of superpixel size, the HSIs are segmented into superpixels for feature learning. The unsupervised network is designed with the convolutional encoder and decoder, the additional clustering branch, and the multilayer feature fusion to enhance the distinguishability of feature learning and the reusability of feature maps. Then, the spatial relationships and object attributes in large- and small-scale contexts are learned collaboratively through the unsupervised network to utilize the complementary multiscale characteristics. Moreover, the diverse features of hyperspectral information and nonsubsampled contourlet transform (NSCT) textures are learned simultaneously via the unsupervised network to alleviate the insufficiency of geometric representation. Finally, the random forest (RF) is adopted as the comprehensive classifier for land cover mapping based on the UMsDFL, and superpixel regularization is adopted to optimize the classification results. A series of experiments are performed on three real-world HSI datasets to demonstrate the effectiveness of our UMsDFL approach. The experimental results show that the proposed UMsDFL can achieve the overall accuracy of 79.23%, 96.49%, and 77.26% for Houston, Pavia, and Dioni datasets, respectively, when there are only five samples per class for training.

The task of cross-view image geo-localization aims to determine the geo-location (Global Positioning System (GPS) coordinates) of a query ground-view image by matching the image with GPS-tagged aerial (or satellite) images in the reference dataset. Due to the dramatic domain gap between the ground and aerial images, the problem is challenging. The existing approaches mainly adopt convolutional neural networks (CNNs) to learn discriminative features. However, these CNN-based methods mainly leverage appearance and semantic information but fail to jointly model the appearance, positional, and orientation properties of scene objects, which belong to the spatial hierarchy. Since spatial hierarchy information is crucial for cross-view feature correspondence, in this article, we propose an end-to-end network architecture, dubbed GeoNet. GeoNet consists of a ResNetX module and a GeoCaps module. On the one hand, the ResNetX module is developed to learn powerful intermediate feature maps and allows the stable propagation of gradients in deep CNNs. On the other hand, the GeoCaps module utilizes the capsule network to encapsulate the intermediate feature maps into several capsules, whose length and orientation represent the existence probability and spatial hierarchy information of scene objects, respectively. Moreover, by using a dynamic routing-by-agreement mechanism, the GeoCaps module is capable of modeling parts-to-whole relationships between scene objects, which is viewpoint invariant and capable of bridging the cross-view domain gap. In addition to GeoNet, we introduce a simple yet effective metric learning method, based on which two weighted soft margin loss functions with online batch hard sample mining are devised. These functions not only speed up convergence but also improve the generalization ability of the network. Extensive experiments on three well-known datasets demonstrate that our GeoNet not only achieves state-of-the-art results for the ground-to-aerial and aerial-to-ground geo-localization tasks but also outperforms competing approaches for the few-shot geo-localization task.

With the rapid development of deep learning technology and improvement in computing capability, deep learning has been widely used in the field of hyperspectral image (HSI) classification. In general, deep learning models often contain many trainable parameters and require a massive number of labeled samples to achieve optimal performance. However, in regard to HSI classification, a large number of labeled samples is generally difficult to acquire due to the difficulty and time-consuming nature of manual labeling. Therefore, many research works focus on building a deep learning model for HSI classification with few labeled samples. In this article, we concentrate on this topic and provide a systematic review of the relevant literature. Specifically, the contributions of this paper are twofold. First, the research progress of related methods is categorized according to the learning paradigm, including transfer learning, active learning and few-shot learning. Second, a number of experiments with various state-of-the-art approaches has been carried out, and the results are summarized to reveal the potential research directions. More importantly, it is notable that although there is a vast gap between deep learning models (that usually need sufficient labeled samples) and the HSI scenario with few labeled samples, the issues of small-sample sets can be well characterized by fusion of deep learning methods and related techniques, such as transfer learning and a lightweight model. For reproducibility, the source codes of the methods assessed in the paper can be found at https://github.com/ShuGuoJ/HSI-Classification.git.

Due to the importance in many military and civilian applications, hyperspectral anomaly detection has attracted remarkable interest. Low-rank representation (LRR)-based anomaly detectors use the low-rank property to represent background pixels, and pixels that cannot be well represented are detected as anomalies. The ability of an LRR-based detector to separate background pixels and anomalous pixels depends on the dictionary representation ability, which usually can be enhanced by designing a proper prior for dictionary representation coefficients and constructing a better dictionary. However, it is not easy to handcraft effective and meaningful regularizers for dictionary coefficients. In this article, we propose a novel anomaly detection algorithm that uses a plug-and-play prior for representation coefficients and constructs a new dictionary based on clustering. Instead of cumbersomely handcrafting a regularizer for representation coefficients, we propose solving the anomaly detection problem using the plug-and-play framework, which enables us to plug state-of-the-art priors for representation coefficients. An effective convolutional neural network (CNN) denoiser is plugged into our framework to fully exploit the spatial correlation of representation coefficients. We also propose a modified background dictionary construction method, which carefully includes background pixels and excludes anomalous pixels from clustering results. We refer to the proposed anomaly detection method as plug-and-play denoising CNN regularized anomaly detection (DeCNN-AD) method. Extensive experiments were performed on five data sets in a comparison with eight state-of-the-art anomaly detection methods. The experimental results suggest that the proposed method is effective in anomaly detection and can produce better anomaly detection results than that of the comparison methods. The codes of this work will be available athttps://github.com/FxyPdfor the sake of reproducibility.

In the hyperspectral image, each pixel corresponds to a small area on the Earth's surface and represents the intrinsic characteristic of objects, which can be applied for recognition of land covers. Nevertheless, hyperspectral image processing should face some critical issues, and a small sample set problem may be the most challenging one in the research. Deep learning (DL), which has successfully been applied in many fields, has also been introduced for hyperspectral image classification. However, the large gap between the massive parameters to be tuned and limited labeled samples can lead to overfitting scenario, inevitably deteriorating the generalization ability of the DL model. In this article, a lightweight convolutional neural network (LWCNN) is proposed for hyperspectral image classification to mainly tackle the small sample set problem. Especially, spatial-spectral Schroedinger eigenmaps (SSSE) feature extraction is first adopted to obtain the joint spatial-spectral information, and the compressed dimensionality could significantly reduce the number of parameters in the following DL model. Second, a dual-scale convolution (DSC) module is carefully designed to address the SSSE features from a 1-D vector viewpoint (the number of parameters is further decreased), and the DSC procedure is successively employed to obtain the hierarchical structure description that could represent data distribution from different aspects. Subsequently, the feature vectors from all DSC layers are separately filtered by a new bichannel fusion (BCF) module, which could well encode both the intrinsic and contextual information inside DSC features. Finally, the filtered features are concatenated together and imported into a global average pooling classifier to achieve the predicted probability of each category. Experimental results on three famous hyperspectral image data sets illustrate that the developed LWCNN approach is advantageous in both the efficiency and robustness sides for hyperspectral image classification tasks and outperforms other state-of-the-art methods (both traditional-based and DL-based) with very limited labeled samples.

Hyperspectral images encompass abundant information and provide unique characteristics for material classification. However, the labeling of training samples can be challenging in hyperspectral image classification. To address this problem, this study proposes a framework named flexible Gabor-based superpixel-level unsupervised linear discriminant analysis (FG-SuULDA) to extract the most informative and discriminating features for classification. First, a number of 3-D flexible Gabor filters are rigorously designed using an asymmetric sinusoidal wave to sufficiently characterize the spatial–spectral structure in hyperspectral images. Then, an unsupervised linear discriminant analysis strategy guided by the entropy rate superpixel (ERS) segmentation algorithm, calledSuULDA, is skillfully introduced to reduce the extracted large amount of FG features. TheSuULDA method not only boosts the classification capability but also increases the peculiarity of features, with the aid of superpixel information. Finally, the achieved features are imported to the popular support vector machine classifier. The proposed FG-SuULDA framework is applied to four real hyperspectral image data sets, and the experiments constantly prove that our FG-SuULDA is superior to several state-of-the-art methods in both classification performance and computational efficiency, especially with scarce training samples. The codes of this work are available athttp://jiasen.tech/papers/for the sake of reproducibility.

The rapid increase in the number of remote sensing sensors makes it possible to develop multisource feature extraction and fusion techniques to improve the classification accuracy of surface materials. It has been reported that light detection and ranging (LiDAR) data can contribute complementary information to hyperspectral images (HSIs). In this article, a multiple feature-based superpixel-level decision fusion (MFSuDF) method is proposed for HSIs and LiDAR data classification. Specifically, superpixel-guided kernel principal component analysis (KPCA) is first designed and applied to HSIs to both reduce the dimensions and compress the noise impact. Next, 2-D and 3-D Gabor filters are, respectively, employed on the KPCA-reduced HSIs and LiDAR data to obtain discriminative Gabor features, and the magnitude and phase information are both taken into account. Three different modules, including the raw data-based feature cube (concatenated KPCA-reduced HSIs and LiDAR data), the Gabor magnitude feature cube, and the Gabor phase feature cube (concatenation of the corresponding Gabor features extracted from the KPCA-reduced HSIs and LiDAR data), can be, thus, achieved. After that, random forest (RF) classifier and quadrant bit coding (QBC) are introduced to separately accomplish the classification task on the aforementioned three extracted feature cubes. Alternatively, two superpixel maps are generated by utilizing the multichannel simple noniterative clustering (SNIC) and entropy rate superpixel segmentation (ERS) algorithms on the combined HSIs and LiDAR data, which are then used to regularize the three classification maps. Finally, a weighted majority voting-based decision fusion strategy is incorporated to effectively enhance the joint use of the multisource data. The proposed approach is, thus, named MFSuDF. A series of experiments are conducted on three real-world data sets to demonstrate the effectiveness of the proposed MFSuDF approach. The experimental results show that our MFSuDF can achieve the overall accuracy of 73.64%, 93.88%, and 74.11% for Houston, Trento, and Missouri University and University of Florida (MUUFL) Gulport data sets, respectively, when there are only three samples per class for training.

Learning from a limited number of labeled samples (pixels) remains a key challenge in the hyperspectral image (HSI) classification. To address this issue, we propose a deep metric learning-based feature embedding model, which can meet the tasks both for same- and cross-scene HSI classifications. In the first task, when only a few labeled samples are available, we employ ideas from metric learning based on deep embedding features and make a similarity learning between pairs of samples. In this case, the proposed model can learn well to compare whether two samples belong to the same class. In another task, when an HSI image (target scene) that needs to be classified is not labeled at all, the embedding model can learn from another similar HSI image (source scene) with sufficient labeled samples and then transfer to the target model by using an unsupervised domain adaptation technique, which not only employs the adversarial approach to make the embedding features from the source and target samples indistinguishable but also encourages the target scene's embeddings to form similar clusters with the source scene one. After the domain adaptation between the HSIs of the two scenes is finished, any traditional HSI classifier can be used. In a simple manner, the nearest neighbor (NN) algorithm is selected as the classifier for the classification tasks throughout this article. The experimental results from a series of popular HSIs demonstrate the advantages of the proposed model both in the same- and cross-scene classification tasks.

As a typical graph-based semisupervised learning technique, the label propagation (LP) approach has gained much attention in recent years. The key to LP algorithms is the propagation capability and efficiency of the similarity matrix, which describes the similarity between two data points. Concerning hyperspectral image which often contains hundreds of thousands of pixels, the corresponding similarity matrix is particularly huge and thus the LP procedure is intractable. Fortunately, superpixel, which can effectively characterize the spatial semantic information of surface objects, provides a reasonable way to solve this problem. In this article, we propose an elaborate superpixel-based weighted LP approach, abbreviated as SuWLP, for hyperspectral image classification. First, the hyperspectral image is oversegmented by the entropy rate segmentation (ERS) method, and the internal consistency of each superpixel can be achieved. Second, a new similarity measure is designed to estimate the similarity between two superpixels, and a superpixel-based similarity matrix can be thus established. Third, after the training samples have been expanded based on the superpixel distribution, a weighted LP technique is designed to propagate the sample label at the superpixel level without any parameter tuning. Finally, the label of each superpixel maps back to the contained pixels. We compared our proposed SuWLP method with several state-of-the-art ones, and experimental results on three real hyperspectral data sets certify the effectiveness and efficiency of the superpixel-level LP strategy.

A 3-D Gabor wavelet provides an effective way to obtain the spectral-spatial-fused features for hyperspectral image, which has shown advantageous performance for material classification and recognition. In this paper, instead of separately employing the Gabor magnitude and phase features, which, respectively, reflect the intensity and variation of surface materials in local area, a cascade superpixel regularized Gabor feature fusion (CSRGFF) approach has been proposed. First, the Gabor filters with particular orientation are utilized to obtain Gabor features (including magnitude and phase) from the original hyperspectral image. Second, a support vector machine (SVM)-based probability representation strategy is developed to fully exploit the decision information in SVM output, and the achieved confidence score can make the following fusion with Gabor phase more effective. Meanwhile, the quadrant bit coding and Hamming distance metric are applied to encode the Gabor phase features and measure sample similarity in sequence. Third, the carefully defined characteristics of two kinds of features are directly combined together without any weighting operation to describe the weight of samples belonging to each class. Finally, a series of superpixel graphs extracted from the raw hyperspectral image with different numbers of superpixels are employed to successively regularize the weighting cube from over-segmentation to under-segmentation, and the classification performance gradually improves with the decrease in the number of superpixels in the regularization procedure. Four widely used real hyperspectral images have been conducted, and the experimental results constantly demonstrate the superiority of our CSRGFF approach over several state-of-the-art methods.

Convolutional neural network (CNN) is a powerful visual model which has a significant performance in various visual recognition issues, and attracted considerable attention in recent years. Due to the strong correlation between bands and small sample set (3S) problem, hyperspectral image classification issue remains a challenging problem. In this paper, we construct a novel multi-task framework with embedded convolutional neural network to obtain stronger discriminative capability for hyperspectral image classification. A variance of CNN, which is named Network in Network (NIN), is used to construct three sub-model. Here 1 × 1 convolutional filter is adopted and average pooling layer is employed to replace the full connect layer. The input of the first sub-model is spatial-spectral schroedinger Eigenmapes (SSSE) which can provide fused information of spatial and spectral information. Meanwhile, due to the significant capability of extracting spatial texture of the uniform local binary (ULBP), the histogram feature extracted from ULBP is used as the second sub-model input. Raw hyperspectral data is input to the last sub-model for supplying external information that SSSE and ULBP have lost. Experimental results demonstrate the effectiveness of the proposed model.

Hyperspectral images (HSIs) has practical applications in many fields. In practical scenarios, machine learning often fails to handle changes between training (source) and testing (target) input distributions due to domain shifts. A big challenge in hyperspectral image classification is the small size of labeled data for classifier learning. We always face the situation that an HSI scene is not labeled all or with very limited number of labeled pixels, but we have sufficient labeled pixels in another HSI scene with similar land cover classes. In this paper, we propose a simple and effective method for domain adaptation called statistical fusion to minimize domain shifts by aligning the second-order and fourth-order statistics of source and target distributions. After two hyperspectral scenes are transformed into the similar property-space, any traditional HSI classification approaches can be used, and experimental results have validated the generalization of the proposed method.

In virtue of the spatial structural characteristic of surface materials, the performance of the hyperspectral image classification can be boosted by incorporating texture information. Normally, the spatial structure can be extracted by predefined operators, including the popular extended multiattribute profiles (EMAPs) and the Gabor filters. Recently, superpixel segmentation, which reflects the homogeneous regularity of objects, has drawn much attention in the field. In this paper, a collaborative representation-based multiscale superpixel fusion (CRMSF) approach has been proposed for the hyperspectral image classification. First, after obtaining the EMAPs from the raw hyperspectral image, a group of predesigned 3-D Gabor wavelet filters is convolved with the EMAP features, and the EMAP-Gabor features can, thus, be achieved. Second, the collaborative representation-based classification (CRC) is employed to fully and efficiently make use of the huge amount of extracted EMAP-Gabor features. Third, multiscale superpixel maps are generated from the EMAP features that are utilized to regularize the classification map obtained by CRC. A heuristic strategy has been specially devised to automatically decide the number of extracted superpixels in multiple scales, which can be perfectly compatible with hyperspectral images having various spatial sizes and spatial resolutions. This is the most important contribution of the developed CRMSF approach. Finally, the classification task is accomplished by fusing the multiple regularized classification maps. The CRMSF approach has been evaluated on four popular hyperspectral image data sets, and the experimental results show the advantages of CRMSF, particularly for a hyperspectral image with high spatial resolution.

Since the spatial distribution of surface materials is usually regular and locally continuous, it is reasonable to utilize the spectral and spatial information for the hyperspectral image classification. In this paper, a spectral-spatial Gabor surface feature (GSF) fusion approach has been proposed for hyperspectral image classification. First, Gabor magnitude pictures (GMPs) are extracted by applying a set of predefined 2-D Gabor filters to hyperspectral images. Second, the GSF has been extended to the spectral-spatial domains to comply with the 3-D structure of hyperspectral imagery, called 3-DGSF, which utilizes the first-order derivative of GMPs. Meanwhile, a classic superpixel segmentation method, called simple linear iterative clustering (SLIC), is adopted to divide the original hyperspectral image into disjoint superpixels. Third, principal component analysis is adopted to reduce the dimensionality of each extracted 3-DGSF feature cube. Next, a support vector machine classifier is applied on each reduced 3-DGSF features, and the majority voting strategy is used to obtain the classification results. Finally, the superpixel map obtained by SLIC is used to regularize the classification map, and thus, the proposed approach is named as S3-DGSF. Extensive experiments on three real hyperspectral data sets have demonstrated the higher performance of the proposed S3-DGSF approach over several state-of-the-art methods in the literature.

Hyperspectral remote sensing imagery provides valuable and rich information to distinguish the characteristics of materials. However, this advantage of hyperspectral imagery often encounters the problem of a limited amount of training samples, which is caused by the difficulty of manually labeling. Fortunately, the spatial distribution of surface objects can be integrated with the spectral signature to improve the discriminative ability. In this paper, a 3-D Gaussian-Gabor feature extraction and selection framework has been proposed for hyperspectral image classification. First, a bank of 3-D Gaussian-Gabor filters are convolved with the concatenated data of both extended multi-attribute profile (EMAP) features and raw hyperspectral data. Second, an improved fast density peak clustering (IFDPC) method is introduced to select the most representative features from each extracted 3-D Gaussian-Gabor feature cube. Finally, the retained features are combined together to accomplish the classification task. The proposed method is thus named as GG-IFDPC. Three real hyperspectral imagery data sets have been utilized, and the experiments demonstrate the advantages of the proposed GG-IFDPC approach over the compared ones.

Hyperspectral image acquired by a hyperspectral sensor contains hundred of narrow contiguous spectral bands, since the spatial distribution of surface materials generally exhibits high regularity and local continuity, spatial texture information should be introduced to improve the classification accuracy of hyperspectral image. The extended morphological profiles (EMP) have been created from the raw hyperspectral image, which has proven to be effective and robust of reflecting the spatial structural features of hyperspectral data. In the meanwhile, because the three-dimensional (3D) Gabor wavelets have been introduced to exploit the joint spectral-spatial features of hyperspectral image. In this paper, for the purpose of combining the advantages of the EMP operator and Gabor wavelet transform together, an extended morphological profile-based Gabor wavelets, which is named as EMP-Gabor, has been proposed for hyperspectral image classification. Definitely, to compute principal components of the hyperspectral image, the most significant principal components are used as base images for an extended morphological profile, and the EMP features can be thus obtained. Secondly, 3D Gabor wavelets with particular orientations are directly convolved with the EMP feature cube. Finally, support vector machine (SVM) classifier is utilized to carry out the classification task. Experimental results on two real hyperspectral data sets have demonstrated the effectiveness of the proposed EMP-Gabor framework for hyperspectral image classification over several state-of-the-art methods.

In this paper, we propose a new efficient superpixel-based feature extraction and fusion method on hyperspectral and LiDAR data. Such important factor that the adjacent pixels belong to the same class with high probability is taken into consideration in our method, which means each superpixel can be regarded as a small region consisting of a number of pixels with similar spectral characteristics. In order to represent each superpixel well, we use our Gabor-wavelet-based feature extraction approach instead of morphological APs. A feature selection and fusion process has also been used to reduce the redundancy among Gabor features and make the fused feature more discriminative. The results on the several real dataset indicate that the proposed method provides state-of-the-art classification results, respectively, even when only few samples, i.e., only three samples per class, are labeled.

Classification of a hyperspectral image (HSI) is a very active topic in remote sensing, which has practical applications in many fields, In this paper, a multitask sparse logistic regression method based on multi-feature and decision fusion approach (MTSLR-MPGF) is proposed for cross-scene hyperspectral image classification. Specifically, the Gabor Features with certain orientations and morphology feature are utilized and used for hyperspectral imagery classification. Next, we used feature fusion methods in order to produce an accurate thematic map based on the remote sensed hyperspectral image classification. The extensive experiments on two real hyperspectral data sets have demonstrated superior performance of the proposed MTSLR-MPGF approach over the state-of-the-art methods in terms of overall accuracy and average accuracy.

In recent years, it has been found that the fusion processing of remote sensing data produced by multiple sensors is often effective for material classification. Specifically, the joint use of hyperspectral image (HSI) and Light Detection And Ranging (LiDAR) data for classification has been an active topic of research in remote sensing field. Since hyperspectral and LiDAR data provide complementary information (spectral reflectance, and vertical structure, respectively), one promising and challenging approach is to fuse these data in the information extraction procedure. In this paper, we propose an efficient feature extraction and fusion method based on Gabor wavelet, leading to a fusion of the spectral, spatial and elevation data. The core idea of the proposed fusion approach is stacking elevation and intensity data of LiDAR as additional channels to spectral bands. Our strategies are based on the Gabor feature stack structure, which are natural and effective. The features extracted by Gabor wavelets have proved to be discriminant features when considered for thematic classification in remote sensing applications especially when dealing with hyperspectral images due to their ability to extract joint spatial and spectrum information from HSI. Experimental results on the real hyperspectral image data have shown the better discriminative power of our approach for classification.

As manual labeling is very difficult and time-consuming, the labeled samples used to train a supervised classifier are generally limited, which become one of the biggest challenge for hyperspectral imagery classification. In order to tackle this issue, a recent trend is to exploit the structure information of materials, as which reflects the region homogeneity in the spatial domain and offers an invaluable complement to the spectral information. In this respect, 3-D Gabor wavelets have been introduced to extract joint spectral-spatial features for hyperspectral images. One the one hand, the features extracted by 3-D Gabor wavelets lead to very good performance for classification. On the other hand, its drawbacks, i.e., big number of features and high computational cost, limit its applicability. In this paper, a 3-D Gabor-wavelet-based phase coding and Hamming distance-based matching (3DGPC-HDM) framework is developed for hyperspectral imagery classification. The proposed method, instead of taking into account the large volume of Gabor magnitude features, exploits the Gabor phase features with certain orientations (i.e., the direction parallel to the spectral axis), which are then encoded by a simple quadrant bit coding scheme. After that, a normalized Hamming distance matching (HDM) method is adopted to determine the similarity of two samples, and the nearest neighbor classifier is routinely utilized for pixelwise recognition. Finally, experiments on three real hyperspectral data sets show that the proposed 3DGPC-HDM leads to very good performance. Comparisons with the state-of-the-art methods in the literature, in terms of both classifier complexity and generalization ability from very small training sets, are also included.

Since it is usually difficult and time-consuming to obtain sufficient training samples by manually labeling, feature extraction, which investigates the characteristics of hyperspectral images (HSIs), such as spectral continuity and spatial locality of surface objects, to achieve the most discriminative feature representation, is very important for HSI classification. Meanwhile, due to the spatial regularity of surface materials, it is desirable to improve the classification performance of HSIs from the superpixel viewpoint. In this paper, we propose a novel local binary pattern (LBP)-based superpixel-level decision fusion method for HSI classification. The proposed framework employs uniform LBP (ULBP) to extract local image features, and then, a support vector machine is utilized to formulate the probability description of each pixel belonging to every class. The composite image of the first three components extracted by a principal component analysis from the HSI data is oversegmented into many homogeneous regions by using the entropy rate segmentation method. Then, a region merging process is applied to make the superpixels obtained more homogeneous and agree with the spatial structure of materials more precisely. Finally, a probability-oriented classification strategy is applied to classify each pixel based on superpixel-level guidance. The proposed framework “ULBP-based superpixel-level decision fusion framework” is named ULBP-SPG. Experimental results on two real HSI data sets have demonstrated that the proposed ULBP-SPG framework is more effective and powerful than several state-of-the-art methods.

Due to the 3-D nature of hyperspectral images, as well as the spatial properties (such as regularity and continuity) of land covers, many 3-D feature extraction operators have been designed to fully exploit the joint spatial-spectral information. However, the large amount of obtained features can suffer from the “curse of dimensionality” problem, especially for the small training sample set. Moreover, various spatial-spectral features can represent the characteristics of the hyperspectral image from different aspects. In this paper, a multiple 3-D feature fusion framework (M3DF3) has been proposed for hyperspectral image classification. First, we extend the 2-D Gabor surface feature into 3-D (3DSF) domains to comply with the spatial-spectral structure of the hyperspectral image, which is directly applied on the original hyperspectral image instead of the Gabor features. Second, three 3-D feature extraction methods, including the 3-D morphological profile, the 3-D local binary pattern, and the proposed 3DSF, that, respectively, characterize the hyperspectral image from three different angles, i.e., morphology, local dependence, and shape smoothness, are fused under a multitask sparse representation framework to take full advantage of the multiple 3-D features together. The proposed M3DF3approach was fully tested on three real-world hyperspectral image data, i.e., the widely used Indian Pines, Pavia University, and Houston University. The results show that our method can achieve as high as 68.22%, 79.44%, and 72.84% accuracies, respectively, even when only few samples, i.e., three samples per class, are used for training.

Hyperspectral imagery acquired by a hyperspectral sensor contains hundreds of narrow contiguous spectral bands, providing the opportunity to identify the various materials present on the surface. Due to the three-dimensional (3D) nature of hyperspectral data, 3D filters that could extract joint spatial-spectral features have been recently considered in the literature. In this paper, after the 3D Gabor features with certain orientations have been extracted from the raw hyperspectral image data, both the Gabor magnitude and phase features have been used for hyperspectral imagery classification, which is named as Gabor-MP. Specifically, the confidence score of each test sample is computed by support vector machine for each Gabor magnitude feature cube, while the Hamming distance is calculated based on the quadrant bit coding of each Gabor phase feature cube. Then the label of the test sample is identified by simple calculation between the confidence scores and Hamming distance values. Experimental results on two real hyperspectral data have demonstrated the effectiveness of the proposed Gabor feature fusion framework for hyperspectral imagery classification.

In this paper, a three-dimensional (3D) Gabor phase coding and Hamming distance matching approach, called 3DGPC-HDM, is proposed for hyperspectral imagery classification. Specifically, the Gabor phase features with certain orientations are utilized, which are then encoded by a simple quadrant bit coding scheme. Next, a normalized Hamming distance matching method has been introduced to determine the similarity of two samples, and the nearest neighbor classifier is routinely used for recognition. The extensive experiments on two real hyper-spectral data sets have demonstrated superior performance of the proposed 3DGPC-HDM approach over the state-of-the-art methods in the literature.

Discriminative dictionary learning aims to learn a dictionary from training samples in order to improve the discriminative ability of their coding vectors. Gabor wavelets have recently been successfully applied for hyperspectral image (HSI) classification due to their ability to extract joint spatial and spectrum information. Due to the high discriminative power of Gabor features, an efficient method, called Gabor feature based Support Vector Guided Dictionary Learning (GSVGDL), has been proposed in this paper for HSI classification. After Gabor features have been extracted from the hyperspectral image, the augmented Gabor feature matrix is used to construct the initial dictionary. The dictionary learning model formulates the discrimination term as the weighted summation of the squared distances between all pairs of coding vectors, which can greatly improve the discriminative ability of the dictionary. The structure of the dictionary and the corresponding linear classifier are obtained simultaneously by dictionary learning. Experimental results on two real hyperspectral image data have shown that the proposed GSVGDL approach could achieve better performance than several state-of-the-art methods.

Due to the high spectral dimensionality of hyperspectral images as well as the difficult and time-consuming process of collecting sufficient labeled samples in practice, the small sample size scenario is one crucial problem and a challenging issue for hyperspectral image classification. Fortunately, the structure information of materials, reflecting region of homogeneity in the spatial domain, offers an invaluable complement to the spectral information. Assuming some spatial regularity and locality of surface materials, it is reasonable to segment the image into different homogeneous parts in advance, called superpixel, which can be used to improve the classification performance. In this paper, a superpixel-based multitask learning framework has been proposed for hyperspectral image classification. Specifically, a set of 2-D Gabor filters are first applied to hyperspectral images to extract discriminative features. Meanwhile, a superpixel map is generated from the hyperspectral images. Second, a superpixel-based spatial-spectral Schroedinger eigenmaps (S4E) method is adopted to effectively reduce the dimensions of each extracted Gabor cube. Finally, the classification is carried out by a support vector machine (SVM)-based multitask learning framework. The proposed approach is thus termed Gabor S4E and SVM-based multitask learning (GS4E-MTLSVM). A series of experiments is conducted on three real hyperspectral image data sets to demonstrate the effectiveness of the proposed GS4E-MTLSVM approach. The experimental results show that the performance of the proposed GS4E-MTLSVM is better than those of several state-of-the-art methods, while the computational complexity has been greatly reduced, compared with the pixel-based spatial-spectral Schroedinger eigenmaps method.

The local binary pattern (LBP) is a simple and efficient texture descriptor for image processing. Recently, LBP has been introduced for feature extraction of hyperspectral imagery. Specifically, the LBP codes are extracted from the 2-D band images to capture the spatial correlation among neighboring pixels, and then the statistical histogram features from all bands, which could estimate the underlying distribution in local area, are concatenated together for pixel-wise classification. However, since hyperspectral imagery contains rich spectral and spatial information, which is actually a 3-D data cube, the 2-D LBP (2-DLBP) model cannot fully exploit the joint spectral-spatial structure. In this paper, the 2-DLBP has been extended into 3-D LBP (3-DLBP) model through forming a 3-D regular octahedral frame to characterize the spectral-spatial relationship. In order to reflect the local continuous property of hyperspectral data in both the spectral and spatial domains, while ensuring the rotational invariance of the 3-DLBP model, the code patterns of 3-DLBP model have been divided into eight groups (including seven groups of “dense” patterns and one group of “nondense” patterns) based on the consistency of spectral-spatial topology structure. Specifically, the patterns in seven “dense” groups correspond to the microstructures in the 3-D domains (such as spots, edges, and flat areas), which has a high percentage in all the 3-DLBP patterns, while the rest patterns are aggregated and treated as the “nondense” patterns. The proposed method is thus called 3-D dense LBP (3-D2LBP) model. Moreover, instead of taking zero as the hard threshold, a slack variable has been introduced to enable the difference between the central pixel and the neighboring ones varying in a small interval, which could greatly decrease the impact of spectral variability and noise, and the discriminative power of the features has been further boosted. The slack threshold-based 3-D2LBP model is named ST-3-D2LBP. A series of experiments is conducted on three real hyperspectral imageries to demonstrate the effectiveness of the proposed two 3-D2LBP-based methods. The experimental results show that the performance of the proposed ST-3-D2LBP is significantly superior to that of 2-DLBP, which is also better than the 3-D2LBP model and several state-of-the-art hyperspectral classification methods.

Local binary patterns (LBP) features extracted from hyperspectral imagery (HSI) have gained impressive performance in hyperspectral classification tasks, for which LBP got considerable attention. However, existing LBP-based hyperspectral imagery classification methods utilized two-dimensional LBP (2DLBP) that could capture gray variation signal in space, which did not excavate the contextual information that hides in spectral-spatial structure considering that hyperspectral imagery characterizes by three dimension. Aimed at this problem, this paper presents a three-dimensional LBP-based (3DLBP) hyperspectral imagery classification method where 2DLBP textures histogram on three orthogonal planes are concatenated to form 3DLBP texture features to be classified by sparse representation. A serial of experiments are conducted on the Pavia university dataset, and the experimental results show that the performance of 3DLBP is significantly superior to that of 2DLBP.

Sparse representation-based classification (SRC) assigns a test sample to the class with minimal representation error via a sparse linear combination of all the training samples, which has successfully been applied to hyperspectral imagery (HSI). Meanwhile, spatial information, that means the adjacent pixels belong to the same class with a high probability, is a valuable complement to the spectral information. In this paper, we propose an efficient method for HSI classification by using superpixel based sparse representation-based classification (SP-SRC). One superpixel can be regarded as a small region consisting of a number of pixels with similar spectral characteristics. The novel method utilizes superpixel to exploit spatial information which can greatly improve classification accuracy. Specifically, SRC is firstly used to classifier the HSI. Then an efficient segmentation algorithm is adopted to divide the HSI into disjoint superpixels. Finally, each superpixel is used to fuse the results of the SRC classifier. Experimental results on the widely-used Indian Pines hyperspectral imagery have shown that the proposed SP-SRC approach could achieve better performance than the pixel-wise SRC method.

The large amount of spectral and spatial information contained in hyperspectral imagery has provided great opportunity to effectively characterize and identify the surface materials of interest. As a novel feature extraction technique, a series of Gabor wavelet filters with different scales and frequencies was applied on hyperspectral data to extract spectral-spatial-combined features, which produced impressive performance on pixel-oriented classification. However, the incredibly large number of Gabor features could cause too much burden for onboard computation, limiting the efficiency of the method. To make matters worse, due to the nonhomogeneous spatial distribution of materials as well as the different characteristics of the constructed Gabor filters, some Gabor features could have a smaller or even negative impact on material representation, deteriorating the classification accuracy eventually. In this paper, a Gabor cube selection based multitask joint sparse representation approach, abbreviated as GS-MTJSRC, was proposed for hyperspectral image classification. First, based on the Fisher discrimination criterion, the most representative Gabor cubes for each class were picked out. Next, under multitask joint sparse representation framework, a coefficient vector could be obtained for each test sample with the selected Gabor cube features, which could be directly used for the following residual-based classification. Experimental results on three real hyperspectral data sets with different characteristics and spatial resolutions demonstrated the feasibility and efficiency of the proposed method.

Through imaging the same spatial area by hyperspectral sensors at different spectral wavelengths simultaneously, the acquired hyperspectral imagery often contains hundreds of band images, which provide the possibility to accurately analyze and identify a ground object. However, due to the difficulty of obtaining sufficient labeled training samples in practice, the high number of spectral bands unavoidably leads to the problem of a “dimensionality disaster” (also called the Hughes phenomenon), and dimensionality reduction should be applied. Concerning band (or feature) selection, conventional methods choose the representative bands by ranking the bands with defined metrics (such as non-Gaussianity) or by formulating the band selection problem as a clustering procedure. Because of the different but complementary advantages of the two kinds of methods, it can be beneficial to use both methods together to accomplish the band selection task. Recently, a fast density-peak-based clustering (FDPC) algorithm has been proposed. Based on the computation of the local density and the intracluster distance of each point, the product of the two factors is sorted in decreasing order, and cluster centers are recognized as points with anomalously large values; hence, the FDPC algorithm can be considered a ranking-based clustering method. In this paper, the FDPC algorithm has been enhanced to make it suitable for hyperspectral band selection. First, the ranking score of each band is computed by weighting the normalized local density and the intracluster distance rather than equally taking them into account. Second, an exponential-based learning rule is employed to adjust the cutoff threshold for a different number of selected bands, where it is fixed in the FDPC. The proposed approach is thus named the enhanced FDPC (E-FDPC). Furthermore, an effective strategy, which is called the isolated-point-stopping criterion, is developed to automatically determine the appropriate number of bands to be selected. That is, the clustering process will be stopped by the emergence of an isolated point (the only point in one cluster). Experimental results on three real hyperspectral data demonstrate that the bands selected by our E-FDPC approach could achieve higher classification accuracy than the FDPC and other state-of-the-art band selection techniques, whereas the isolated-point-stopping criterion is a reasonable way to determine the preferable number of bands to be selected.

Hundreds of narrow contiguous spectral bands collected by a hyperspectral sensor has provided the opportunity to identify the various materials present on the surface. Spatial information, that means the adjacent pixels belong to the same class with a high probability, is a valuable complement to the spectral information. In this paper, by decomposing each pixel and the spatial neighborhood into a low-rank form, the spatial information can be efficiently integrated into the spectral signatures. Meanwhile, in order to describe the low-rank structure of the decomposed data more precisely, an ℓ1/2norm regularization is introduced and a discrete algorithm is proposed to solve the combined optimization problem. Experimental results on real hyperspectral data have demonstrated the effectiveness and versatility of the proposed spatial information-fused approach for hyperspectral imagery classification.

Multiple kinds of features extracted from hyperspectral imagery (HSI) have shown great potential for pixel-oriented classification. However, two difficulties can be encountered during the classification process. Firstly, it is time consuming to directly utilize the large amount of features. Secondly, because each kind of feature is usually processed individually, the high-level relationship among different features is not completely configured, decreasing the performance eventually. In this paper, a new strategy to fuse the features and exploit dictionary learning for HSI classification is proposed. Based on the high-level relationship, the extracted Gabor features have been integrated into a more compact and more discriminative representation through a Fisher-based criterion. Experimental results have shown that the fused features can not only produce competitive performance for HSI classification, but also greatly reduce the computational complexity.

Recently, a fast density peak-based clustering algorithm, namely FDPC, has demonstrated its power on nonspherical clustering problems. In this paper, we propose an enhanced fast density peak-based clustering, namely E-FDPC, for hy-perspectral band selection. The main contributions of the proposed E-FDPC, in comparison with the original FDPC are two folds. First, we introduce a parameter to control the weight between the normalized local density and intra-cluster distance. The other aspect is that, we present an exponential-based learning rule to adjust the cut-off threshold for different number of selected bands, where it is empirically defined in FDPC. Furthermore, an effective strategy, called isolated-point-stopping criterion, is developed to automatically determine the appropriate number of bands. That is, the clustering process will be stopped by the emergence of the isolated point (the only point in one cluster). Experimental results on real hyperspectral data demonstrate that E-FDPC approach could achieve higher overall classification accuracies than FDPC and other state-of-the-art band selection techniques.

Sparse-representation-based classification (SRC) assigns a test sample to the class with minimum representation error via a sparse linear combination of all the training samples, which has successfully been applied to several pattern recognition problems. According to compressive sensing theory, the l1-norm minimization could yield the same sparse solution as the l0norm under certain conditions. However, the computational complexity of the l1-norm optimization process is often too high for large-scale high-dimensional data, such as hyperspectral imagery (HSI). To make matter worse, a large number of training data are required to cover the whole sample space, which is difficult to obtain for hyperspectral data in practice. Recent advances have revealed that it is the collaborative representation but not the l1-norm sparsity that makes the SRC scheme powerful. Therefore, in this paper, a 3-D Gabor feature-based collaborative representation (3GCR) approach is proposed for HSI classification. When 3-D Gabor transformation could significantly increase the discrimination power of material features, a nonparametric and effective l2-norm collaborative representation method is developed to calculate the coefficients. Due to the simplicity of the method, the computational cost has been substantially reduced; thus, all the extracted Gabor features can be directly utilized to code the test sample, which conversely makes the l2-norm collaborative representation robust to noise and greatly improves the classification accuracy. The extensive experiments on two real hyperspectral data sets have shown higher performance of the proposed 3GCR over the state-of-the-art methods in the literature, in terms of both the classifier complexity and generalization ability from very small training sets.

In this paper, an effective l2-norm collaborative representation algorithm based on 3D discrete wavelet transform (3D-DWT) features, called CR_DWT, is proposed for hyperspec-tral image classification. By using the discriminative 3D-DWT features extracted from the original spectral space, a non-parametric and efficient l2-norm CR method is developed to calculate the representation coefficients. Due to the simplicity of the method, the computational cost has been substantially reduced, thus all the extracted 3D-DWT texture features can be directly utilized to code the test sample, which greatly improves the classification accuracy of the l2-norm CR mechanism. The extensive experiments on two real hy-perspectral data sets have shown higher performance of the proposed CR_DWT approach over the state-of-the-art methods in the literature, in terms of both the accuracy and classifier complexity.

Dimensionality reduction is often adopted for hyperspectral imagery in advance to improve the efficiency of following processing, such as classification and identification. Affinity propagation (AP), which is a clustering algorithm, has shown the ability to automatically pick out the representative bands from the hyperspectral imagery. Several distance measures have been proposed to construct the similarity matrix, which is an important issue for AP, but the spatial structural information of the image is not considered. In this paper, a structural approach used to evaluate the image quality, called complex wavelet structural similarity (CW-SSIM) index, is developed to build the similarity between band images. The CW-SSIM index could capture the spatial structural information of compared images. Experiments on the real Kennedy Space Center (KSC) hyperspectral data set has demonstrated the efficacy of the proposed distance criterion for AP.

Noise reduction for hyperspectral imagery (HSI) is an indispensable step before further processes such as object detection and classification. In this paper, we propose a noise reduction method for HSI based on non-local strategy and tensor factorization. Based on the observation that natural images are always locally self-repetitive, we divide the whole HSI into small sub-blocks and cluster similar blocks into groups. Since similar blocks share the same underlying structure, the redundancy can be utilized to remove noise of the blocks jointly. We stack the similar blocks to construct a fourth-order tensor from each group. Noise is reduced by finding the lower dimensional approximation of each of the fourth-order tensors via Tucker factorization. The experimental results indicate that the proposed method has a good quality of restoring the true signal from the noisy observation.

Displaying the abundant information contained in a hyperspectral image is a challenging task. Previous visualization approach focused only on preserving the structure in the original images. They ended up with presenting pseudo-color images and stopped short of adjusting the color of the images to retrieve more desirable visual effects. In this paper, a new visualization algorithm is proposed. It can be modeled as a two stage approach. At the first stage, Laplacian Eigenmaps algorithm is applied to reduce the dimension of the hyperspectral image. In this way we obtain a three dimensional image with pseudo-color. At the second stage, we transfer the natural color of a panchromatic image to the image obtained by the first step via manifold alignment. Experimental results show that the visualized image not only retains the structure of the hyperspectral image but also possesses natural colors.

Three-dimensional Gabor wavelets have recently been successfully applied for hyperspectral image classification due to their ability to extract joint spatial and spectrum information. However, the dimension of the extracted Gabor feature is incredibly huge. In this letter, we propose a symmetrical-uncertainty-based and Markov-blanket-based approach to select informative and nonredundant Gabor features for hyperspectral image classification. The extracted Gabor features with large dimension are first ranked by their information contained for classification and then added one by one after investigating the redundancy with already selected features. The proposed approach was fully tested on the widely used Indian Pine site data. The results show that the selected features are much more efficient and can achieve similar performance with previous approach using only hundreds of features.

Hyperspectral unmixing is a crucial preprocessing step for material classification and recognition. In the last decade, nonnegative matrix factorization (NMF) and its extensions have been intensively studied to unmix hyperspectral imagery and recover the material end-members. As an important constraint for NMF, sparsity has been modeled making use of the$L_{1}$regularizer. Unfortunately, the$L_{1}$regularizer cannot enforce further sparsity when the full additivity constraint of material abundances is used, hence limiting the practical efficacy of NMF methods in hyperspectral unmixing. In this paper, we extend the NMF method by incorporating the$L_{1/2}$sparsity constraint, which we name$L_{1/2}$-NMF. The$L_{1/2}$regularizer not only induces sparsity but is also a better choice among$L_{q}(0 < q < 1)$regularizers. We propose an iterative estimation algorithm for$L_{1/2}$-NMF, which provides sparser and more accurate results than those delivered using the$L_{1}$norm. We illustrate the utility of our method on synthetic and real hyperspectral data and compare our results to those yielded by other state-of-the-art methods.

The rich information available in hyperspectral imagery not only poses significant opportunities but also makes big challenges for material classification. Discriminative features seem to be crucial for the system to achieve accurate and robust performance. In this paper, we propose a 3-D Gabor-wavelet-based approach for pixel-based hyperspectral imagery classification. A set of complex Gabor wavelets with different frequencies and orientations is first designed to extract signal variances in space, spectrum, and joint spatial/spectral domains. The magnitude of the response at each sampled location (x, y) for spectral band b contains rich information about the signal variances in the local region. Each pixel can be well represented by the rich information extracted by Gabor wavelets. A feature selection and fusion process has also been developed to reduce the redundancy among Gabor features and make the fused feature more discriminative. The proposed approach was fully tested on two real-world hyperspectral data sets, i.e., the widely used Indian Pine site and Kennedy Space Center. The results show that our method achieves as high as 96.04% and 95.36% accuracies, respectively, even when only few samples, i.e., 5% of the total samples per class, are labeled.

Nonnegative matrix factorization is used extensively for feature extraction and clustering analysis. Recently many sparsity/sparseness constraints, such as L1penalty, are introduced for sparse nonnegative matrix factorization. Inspired by sparsity measures from linear regression model, this paper proposes to integrate nonnegative matrix factorization with another sparsity constraint, the elastic net. The experimental results of clustering analysis on three gene expression datasets demonstrate the effectiveness of the proposed method.

This paper presents a novel affinity propagation (AP) based memetic band selection method (APMA) for hyperspectral imagery classification. The method incorporates AP based local search and genetic algorithm (GA) based global search to take advantage of both. Particularly, the AP based local search fine-tunes the GA individuals by adding relevant bands and eliminating irrelevant/redundant bands. A comparison study to the filters methods (including ReliefF, AP based method, and FCBF) and the counterpart wrapper GA feature selection on two hyperspectral imagery datasets demonstrates that APMA is capable of attaining competitive or better classification accuracy with fewer selected bands, which suggests APMA searches the band subset space more efficiently and identify better band subsets.

In this paper, we not only extend the temporal hierarchical alternating least squares (HALS) to spatial domain, but also incorporate two necessary characteristics of material abundances, full additivity and sparsity, to unmix hyperspectral data. The new algorithm is abbreviated as HALSSC (HALS with Sparsity Constraint). Different from the other endmember extraction approaches, the proposed algorithm does not need the existence assumption of pure pixel of each endmember in the scene. Experimental results on highly mixed synthetic data and real hyperspectral data from Washington DC mall confirm the accuracy of the developed algorithm.

Remotely sensed data by sensors on satellite or airborne platform, is becoming more and more important in monitoring the local, regional and global resources and environment. In this paper, we utilize the regularized logistic regression model for change detection of large scale remotely sensed bi-temporal multispectral images. Change detection methods base on classification schemes under this kind of condition should put more emphasis on the model's simplicity and efficiency in addition to the detection accuracy. The simple linear classifier is solved by recent proposed “Pathwise Coordinate Descent”. When applied on the L1-regularized regression problem, the algorithm can handle large problems in a comparatively very low timing cost. Through computing the solutions for a decreasing sequence of regularization parameters, the algorithm also combines model selection procedure into itself. We experiment the logistic regression with elastic-net convex penalty. Experimental results from a real data set demonstrate that, models obtained by Pathwise Coordinate Descent algorithm only need very low computational costs. The achieved remarkable efficiency indicates that regularized logistic regression via Pathwise Coordinate Descent is a promising method for large scale change detection problem in remote sensing.

Due to the enormous amounts of data contained in hyperspectral imagery, the main challenge for hyperspectral image classification is to improve the accuracy with less computation complexity. Hence, dimensionality reduction (DR) is often adopted, which includes two different kinds of methods, feature extraction and feature selection. In this paper, discrete wavelet transform (DWT) and affinity propagation (AP), which belong to feature extraction and feature selection respectively, are combined together to accomplish the DR task. Firstly, DWT-based features are extracted from the original hyperspectral data; secondly, AP is applied to select representative features from the obtained ones. Experimental results demonstrate that, compared with some other DR methods which only make use of feature extraction or feature selection, the features acquired by the hybrid technique make the classification results more accurate.

Hyperspectral unmixing is a process to identify the constituent materials and estimate the corresponding fractions from the mixture. During the last few years, nonnegative matrix factorization (NMF), as a suitable candidate for the linear spectral mixture model, has been applied to unmix hyperspectral data. Unfortunately, the local minima caused by the nonconvexity of the objective function makes the solution nonunique, thus only the nonnegativity constraint is not sufficient enough to lead to a well-defined problem. Therefore, in this paper, two inherent characteristics of hyperspectral data, piecewise smoothness (both temporal and spatial) of spectral data and sparseness of abundance fraction of every material, are introduced to NMF. The adaptive potential function from discontinuity adaptive Markov random field model is used to describe the smoothness constraint while preserving discontinuities in spectral data. At the same time, two NMF algorithms, nonsmooth NMF and NMF with sparseness constraint, are used to quantify the degree of sparseness of material abundances. A gradient-based optimization algorithm is presented, and the monotonic convergence of the algorithm is proved. Three important facts are exploited in our method: First, both the spectra and abundances are nonnegative; second, the variation of the material spectra and abundance images is piecewise smooth in wavelength and spatial spaces, respectively; third, the abundance distribution of each material is almost sparse in the scene. Experiments using synthetic and real data demonstrate that the proposed algorithm provides an effective unsupervised technique for hyperspectral unmixing.

Hyperspectral unmixing, which decomposes pixel spectra into a collection of constituent spectra, is a preprocessing step for hyperspectral applications like target detection and classification. It can be considered as a blind source separation (BSS) problem. Independent component analysis, which is a widely used method for performing BSS, models a mixed pixel as a linear mixture of its constituent spectra weighted by the correspondent abundance fractions (sources). The sources are assumed to be independent and stationary. However, in many instances, this assumption is not valid. In this paper, a complexity-based BSS algorithm is introduced, which studies the complexity of sources instead of the independence. We extend the 1-D temporal complexity, which is called complexity pursuit that was proposed by Stone, to the 2-D spatial complexity, which is named spatial complexity BSS (SCBSS), to describe the spatial autocorrelation of each abundance fraction. Further, the temporal complexity of spectrum is combined into SCBSS to account for the spectral smoothness, which is termed spectral and spatial complexity BSS. More importantly, a strict theoretic interpretation is given, showing that the complexity-based BSS is very suitable for hyperspectral unmixing. Experimental results on synthetic and real hyperspectral data demonstrate the advantages of the proposed two algorithms with respect to other methods.

Feature extraction techniques are widely used in many applications to pre-process data in order to reduce the complexity of subsequent processes. A group of kernel-based Fisher discriminant analysis (KFDA) algorithms has attracted much attention due to their high performance. In this paper, the inherent limitations of those KFDA algorithms have been discussed and a novel algorithm is proposed to effectively overcome those limitations. Experimental results on face recognition suggest that this proposed algorithm is superior to the existing methods in terms of correct classification rate.