Recently, the selective structured state space model (S6) built upon the Mamba architecture has attracted widespread attention for its outstanding performance in long-range modeling. However, existing Mamba-based hyperspectral image (HSI) classification methods suffer from certain limitations in extracting edge information. To address this problem, a new HSI classification framework called Pinwheel and Wavelet-based Spatial-Spectral Mamba (S 2 PW-Mamba) is proposed in this paper. The input to our S 2 PW-Mamba is the complete, unsegmented image, which preserves the correlation between pixels in the HSI while leveraging convolution to enhance the extraction of local information. Specifically, two S6 modules are employed along different scanning directions to extract the spatial-spectral features from the HSI, acting on the spectral and spatial domains, respectively. Meanwhile, a GateFusion module is designed to adaptively guide the fusion of spatial and spectral features by discerning their relative importance. More precisely, our S 2 PW-Mamba first extracts both edge and central spatial features using a Pinwheel-shaped convolution and a four-directional spatial scanning module, effectively capturing spatial-contextual relationships. Subsequently, by integrating the wavelet transform with the S6 architecture, the model is capable of capturing both global contextual dependencies and local texture details within the spectral domain. Comprehensive experiments on three benchmark HSI datasets show that our approach outperforms existing state-of-the-art methods.

Low-rank tensor decomposition (LRTD) has demonstrated significant efficacy in multidimensional image reconstruction. Indeed, LRTD driven by nonlinear relationship can capture the underlying low-rank structure more accurately, since real-world data often exhibits complex nonlinear interactions. However, the existing nonlinear LRTD methods do not to investigate the inherent nonlinear interactions in spatial neighborhoods and spectral or temporal models. To address these challenges, we propose a novel deep nonlinear low-rank tensor decomposition (DNLRTD). Specifically, we design a deep nonlinear transform network (DNTN) using multiple convolutional layers and channel attention modules to form a deep nonlinear transform (DNT). The custom-designed DNT effectively captures nonlinear interactions within spatial neighborhoods while paying attention to the nonlinear interactions of spectral or temporal dimensions, consequently achieving a lower-rank representation. By integrating DNT into the low-tubal-rank decomposition framework, we induce the deep tubal-rank and form the DNLRTD. Also, we design a customized DNLRTD optimization strategy to make it flexible for different multidimensional image reconstruction tasks. Based on DNLRTD, we construct two multidimensional image reconstruction models and develop corresponding algorithms based on the alternating direction method of multipliers (ADMM) to solve them. Extensive experimental results on spectral compressive imaging and dynamic magnetic resonance image (MRI) reconstruction verify the superior performance of the proposed method.

Cross-scene hyperspectral image classification aims to identify a new scene in target domain via learned knowledge from source domain using limited training samples. Existing cross-scene alignment approaches focus on aligning the global feature distribution between the source and target domains while overlooking the fine-grained alignment at different levels. Moreover, they mainly use Transformer architectures to model long-range dependencies across different channels but confront efficiency challenges due to their quadratic complexity, which limits classification performance in unsupervised domain adaptation tasks. To address these issues, a new domain-adaptive Mamba (DAMamba) is proposed for cross-scene hyperspectral image classification. First, a spectral-spatial Mamba is developed to extract high-order semantic features from the input data. Then, a domain-invariant prototype alignment method is proposed from three perspectives, i.e., intra-domain, inter-domain, and mini-batch, to produce reliable pseudo-labels and mitigate the spectral shift between the source and target domains. Finally, a fully connected layer is applied to the aligned features in the target domain to obtain the final classification results. Extensive evaluations across diverse cross-scene datasets demonstrate that our DAMamba outperforms existing state-of-the-art methods in classification accuracy and computing time. The code of this paper is available at https://github.com/PuhongDuan/DAMamba.

Hyperspectral imaging (HSI) captures abundant spectral information of land covers while light detection and ranging (LiDAR) provides elevation and structural characteristics. Joint classification of HSI and LiDAR data can effectively merge spectral and elevation information to enhance the outcome of land cover classification. Current HSI and LiDAR joint classification approaches mainly employ a three-layer deep network to extract high-order features, followed by a concatenation or weighted fusion scheme which cannot fully exploit the unique properties of different data modalities. Meanwhile, these methods usually require high computational resources. To alleviate these issues, this paper proposes a masked self-attention fusion network (MSAF) for joint HSI and LiDAR classification, where a cascaded cross-attention fusion framework is designed to fully merge different stages of features. First, a mobile convolution block is developed to extract multi-modal data features. Then, a multi-view sequence embedding method is proposed to effectively integrate elevation information and spectral-spatial information so as to obtain token sequences. Finally, an effective masked self-attention mechanism is designed to fuse token sequences. Experimental results on multiple datasets indicate that the proposed framework significantly outperforms other advanced multi-modal fusion methods in terms of classification performance and computing efficiency. The code of this manuscript is available on https://github.com/lulushh/MSAF.

Deep learning (DL) has attracted considerable attention in the field of hyperspectral image classification (HSIC). However, most DL methods still suffer from two problems: overfitting and oversmoothing, particularly when dealing with scarce labeled samples. A major challenge is that they do not make full use of the relationships among a large number of unlabeled samples and multi-scale information in structural relationships, resulting in the loss of multi-scale information. Moreover, prior information such as labels is not used to explicitly learn and modify the graph structure (including nodes, the sparsity of connections, and edge weights). To address these issues, we propose a novel deep fusion paradigm for multi-scale superpixel graphs (DFSG). Our new DFSG integrates multi-scale graphs (at both the graph-level and the feature-level) to reduce information loss while the re-segmentation based graph correction module adaptively learns new graph structures during the active learning (AL) process. In our proposed iterative updating mechanism, AL and our multi-scale methods help each other, forming a symbiotic unified DFSG-AL framework. Experiments on five real hyperspectral image (HSI) datasets demonstrate that our DFSG-AL can achieve remarkable performance in few-sample HSIC. The source codes are available at https://github.com/yulong112/DFSG.

Processing high-dimensional data cubes and developing high-performance classifiers are core objectives in the field of hyperspectral image classification (HSIC). Superpixel-based methods are widely used in HSIC due to their efficacy in reducing redundant information and enhancing local features. However, imprecise segmentation, especially in complex structures and textures of hyperspectral images (HSIs), may lead to inconsistencies in the regions extracted by superpixels and the boundaries between different ground objects. Such inconsistencies significantly degrade the classification performance of HSIs. Alternatively, when parameter settings are inaccurate, edge-aware feature extraction methods often introduce sharpening artifacts at the image boundaries, resulting in a decrease in classification accuracy. To effectively address these challenges, we propose a novel probabilistic fusion method for HSIC. This method consists of the following stages. First, spatial information is extracted by a multiscale superpixel segmentation method and then probabilistically optimized by the extended random walk (ERW) method. Next, semantic-aware structural features (S2Fs) are extracted along with edge information of different objects. Lastly, a probabilistic framework is proposed to fuse the class probabilities of superpixel-based spatial information and semantic-aware structural features. Experimental results on three real datasets show state-of-the-art classification performance, even with limited training sets.

Transformers have gained significant attention in multimodal remote sensing fusion due to their strong global context modeling capability. Although Transformer-based methods excel at processing high-dimensional spectral sequences and joint spatial-spectral information, most current research remains focused on the spatial domain. Consequently, the exploration of frequency-domain features—particularly implicit frequency representations—is often neglected. Moreover, efficiently fusing multimodal data features while emphasizing more discriminative information remains a challenging task. To address these challenges, this paper proposes an Attention-guided Frequency Feature Decomposition Network (AF2DN) for Hyperspectral and LiDAR Data Classification. First, a Transformer-based Frequency Feature Decomposition(TFFD) method is proposed, employing window attention to capture distinct directional frequency components from multimodal remote sensing data. Through this approach, low-frequency components are utilized to characterize global structural information, while various high-frequency components are employed to extract diverse texture and edge features. Second, an Attention Frequency Modulation(AFM) module is developed, incorporating a weight learning matrix in the frequency domain. This matrix is designed to selectively amplify and suppress different frequency components, thereby reducing data redundancy resulting from frequency feature decomposition. Finally, an adaptive Multimodal Same-Frequency Feature Fusion (AMSF3) module is designed to achieve cross-modal feature integration at identical frequency bands. Extensive experiments are conducted on three benchmark datasets, and the results demonstrate that the proposed framework outperforms existing state-of-the-art methods while exhibiting stronger adaptability in complex environments.

Infrared small target detection (IRSTD) has progressed significantly in spatial-domain learning. However, single-frame images’ limited spatial semantics impair discrimination between targets and similar noise while complicating integrity detection of large-scale target. To address these, we propose Global Spatial-Frequency Attention Network (GSFANet), which enhances the distribution difference between targets and noise from a frequency-domain perspective while preserving spatial information integrity. The core innovations consist of three modules: 1) Parametric Wavelet Downsampling (PWD), preserving small target details during frequency refinement to prevent feature fragmentation; 2) Hierarchical Gated Kernel Attention (HGKA), capturing cross-level frequency relationships through Cross-channel Kernel Attention (C2K) and maintaining spatial coherence via Cross-spatial Gate Attention (CSG), effectively bridging semantic gaps across layers; 3) Adaptive Frequency-Decoupled Fusion (AdaFD), dynamically fusing target-associated frequency components while suppressing noise. We further develop AdaFL Loss to balance multi-scale target gradients and stabilize training. Experiments on three benchmark datasets demonstrate GSFANet’s superior detection performance and enhanced segmentation robustness in complex scenarios compared to state-of-the-art methods. Our code will be made public at https://github.com/dengfa02/GSFANet_IRSTD.

Arbitrary-resolution hyperspectral (ARHS) pansharpening seeks to enhance low-resolution hyperspectral (LRHS) images to any target spatial resolutions by collaborating with the connected panchromatic (PAN) images. The ARHS pansharpening task is significantly different from standard hyperspectral (HS) pansharpening, where ARHS pansharpening has to deal with the challenge that how to pansharpen upcoming LRHS images to any desired scales far beyond the training scales. Neural operators (NOs) form mapping between functional spaces and build continuous function relations, offering an alternative potential path to deal with ARHS pansharpening task. In this article, we are dedicated to the design of pansharpening NOs (PNOs) by the postulation that LRHS and PAN are observations of the input functional space while target high-resolution HS (HRHS) images are observed from the output functional space. Our PNOs are built with a paradigm comprising prepansharpening encoding, inverse discretization, and functional mapping subnetwork. More specifically, by use of kernel projection of frequency domain-related design and Galerkin design, we develop Fourier PNO (Fourier-PNO) and Galerkin PNO (Galerkin-PNO). In addition, we propose the concept of dynamic spectral-spatial Green’s function and accordingly develop an enhanced Fourier-PNO (eFourier-PNO), and further introduce a PAN-guided test space tactic to develop an enhanced Galerkin-PNO (eGalerkin-PNO). Finally, we discuss and explain some details such as the scale generalization ability of our PNOs. Our four PNOs are tested on three datasets, and the experimental results verify their excellent performances.

The multisource remote sensing (RS) data have yielded promising results in target detection and classification tasks. However, most existing methods primarily focus on the spatial features inherent in spectral information, while the continuous spectral characteristics are generally neglected. This oversight leads to insufficient extraction of spectral information, thereby limiting detection performance. Recently, the Mamba architecture, based on state space models (SSMs), integrates the advantages of long-range sequence modeling and linear computational efficiency, demonstrating significant potential in low-dimensional scenarios. Inspired by this, we propose Slice Mamba for multisource RS data fusion classification. Specifically, we design two scanning methods: lateral slice scanning (LatSS) and longitudinal slice scanning (LonSS), which construct sequences from lateral and longitudinal perspectives to facilitate information interaction between pixels. In conjunction with the Mamba architecture, we develop the lateral slice Mamba block (LatSMB) and the longitudinal slice Mamba block (LonSMB) to capture continuous spatial-spectral features. Based on this, we establish the slice feature extraction (SFE) module for extracting spatial-spectral feature information and design the cross-information fusion (CIF) module to form a complementary structure for effectively modeling spatial-spectral features, thereby achieving the fusion and classification of multisource heterogeneous features. Experimental results on three benchmark datasets demonstrate that Slice Mamba outperforms existing advanced methods in fusion classification performance and exhibits greater robustness when applied to multispectral datasets.

Hyperspectral image (HSI) classification is fundamental to numerous remote sensing applications, enabling detailed analysis of material properties and environmental conditions. Recent Mamba built upon selective state space models (S6) have demonstrated exceptional advantages in long-range sequence modeling with linear computational efficiency, while Transformer based on self-attention mechanisms is particularly adept at capturing short-range dependencies. To leverage the complementary strengths of these models, this paper introduces a novel hybrid Mamba-Transformer framework (HSI-MFormer), effectively exploring the multiscale properties of hyperspectral data for HSI classification. Initially, a Multiscale Token Generation module (MTG) is developed, which converts the HSI cube into multiple spatial-spectral token groups across different scales. To adequately capture fine-grained multiscale spatial-spectral patterns, an Inner-scale Transformer Expert (ITE) is designed, which incorporates grouped self-attention operations to perform short-range sequence modeling within token groups at each scale. Meanwhile, a Cross-scale Mamba Expert (CME) is introduced, which integrates a cross-scale serialization mechanism and bidirectional Mamba block for long-range sequence modeling, further exploring the interactions and complementarity between token groups across different scales. Several hybrid strategies for integrating the ITE and CME are investigated to maximize their complementarity, including parallel, interval, and serial structures. Extensive experiments demonstrate that the propsed HSI-MFormer significantly out-performs the state-of-the-art Transformer-based and Mamba-based HSI classification methods. The code is available at https://github.com/tubingnuist/HSI-MFormer.

Arbitrary-resolution hyperspectral (ARHS) pansharpening seeks to enhance hyperspectral (HS) images to any desired spatial resolutions by integrating HS images with their connected panchromatic (PAN) images, which is obviously different from standard HS pansharpening where HS data are merely pansharpened to the spatial resolutions of PANs. Therefore, ARHS pansharpening has a great potential to broaden the applications of traditional pansharpening technique. However, ARHS pansharpening is faced with some inherent obstacles, including how to produce HS images with arbitrary resolutions beyond the training scales and how to maintain high spatial-spectral fidelities at any pansharpening scales. Implicit neural representations (INRs) is able to parametrically tune neural networks to accommodate unanalytical continuous characteristics of real-world scenes, and thus offer natural representations for ARHS pansharpening. The core of pansharpening INR design is to construct suitable implicit spectral-spatial functions (ISSFs). In this work, we propose two innovatively novel transformer-ensemble based ISSFs, i.e. pre-integrated ISSF (preISSF) and post-integrated ISSF (postISSF), which use suitable coordinate operation, novel ensemble feature extraction and coordinate query mapping to build strong spectral-spatial reconstruction capabilities, where some new tactics such as mode-1 transformer decomposition and explicit-implicit positional encoding system are involved. We test our preISSF and postISSF on three datasets. The experimental results verify the excellent performance of our methods.

Traditional supervised deep learning (DL) methods for hyperspectral image (HSI) classification are severely limited by the quality and quantity of labels. Furthermore, existing feature extraction methods generally lack the fusion of multiscale feature information, struggling to handle complex scenarios. To counter these problems, this work investigates a feature extraction module based on self-supervised graph masked autoencoders (SGMAEs). It innovatively employs graph masked autoencoders to achieve self-supervised label-free feature extraction for the complete set of samples, utilizing a multiscale graph convolutional network encoder (MGCNE) and cross correlation decoder (CCD) to extract and fuse multiscale spatial-spectral features of HSI data, respectively. Specifically, the HSI data is first converted into an edge-masked perturbed graph to label-freely extract multiscale feature representations of all pixel samples, and then fed into the MGCNE to obtain multilayer feature vectors for the pixel nodes. To reconstruct the masked edges for the fusion of multiscale features, the CCD applies cross correlation calculations to the nodes of the true edges at the masked positions and the fake edges at the random positions. The contrastive learning loss function is proposed for training of the autoencoder, which calculates the loss for the existence estimates of edges generated by cross correlation calculations. The pretrained MGCNE possesses an efficient self-supervised multiscale spatial-spectral feature extraction capability, along with strong generalizability, which significantly improves the accuracy of various mainstream models in downstream classification tasks. Extensive experiments and analyses on multiple HSI datasets demonstrate that our proposed SGMAE significantly enhances the model performance of various supervised classifiers and achieves superior performance in comparison to mainstream models.

Landslides are one of the most frequent and destructive geological disasters, often causing significant threats to human life and infrastructure. Landslide susceptibility assessment (LSA) plays a vital role in disaster prevention and risk management. Graph convolutional networks (GCNs) have demonstrated strong potential in LSA due to their ability to model graph-structured data. However, existing GCN-based models often suffer from feature aggregation issues, where the neighborhood aggregation of landslide conditioning factors (LCFs) leads to information loss and reduced predictive performance. To address this limitation, we propose an internal and external decoupled GCN (IED-GCN) based on superpixel segmentation. First, the simple linear iterative clustering (SLIC) algorithm is employed to segment the study area into superpixels, creating more meaningful spatial units. Then, the proposed model applies internal graph convolution (IGC) to capture local feature dependencies and external graph convolution (EGC) to model global feature relationships. Furthermore, edge decoupling is incorporated into the EGC to mitigate overfitting and enhance the model’s generalization capability. The effectiveness of IDE-GCN is validated using a dataset comprising historical landslide occurrences and 13 LCFs in the Three Gorges Reservoir area. The dataset is divided into 70% of the data for training and 30% for testing. Experimental results demonstrate that our IED-GCN significantly outperforms conventional models such as convolutional neural networks (CNN), residual convolutional networks (ResNet), dense convolutional networks (DenseNet), and miniGCN, achieving F1 score, Kappa coefficient, and AUC values of 0.9839, 0.9675, 0.9838. In summary, our IED-GCN effectively addresses the challenge of feature aggregation in LSA tasks, offering superior performance in geological disaster prediction with complex spatial data.

Building change detection remains challenging for urban development, disaster assessment, and military reconnaissance. While foundation models like Segment Anything Model (SAM) show strong segmentation capability, they exhibit limited performance in building change detection due to the domain gap between natural and remote sensing images. Existing adapter-based fine-tuning methods struggle with the imbalanced distribution of changed buildings, resulting in suboptimal detection of building changes exhibiting sparse distribution, small scale, or weak contrast. Additionally, bi-temporal alignment methods, including optical flow, are vulnerable to background noise interference. To address these limitations, we propose the SAM-based Network with Distribution-Aware Fourier Adaptation and Edge-Constrained Warping (FAEWNet) for building change detection. Unlike previous adapters overlook the imbalanced distribution of changed buildings, our proposed Distribution-Aware Fourier Aggregation Adapter not only addresses the domain gap issue, but also models the distribution of changed buildings. Furthermore, to mitigate noise interference and misalignment caused by registration errors, we design Multiscale Aware Flow Aggregation module that refines building edge extraction and enhances the perception of changed buildings. The results on the LEVIR-CD, S2Looking and WHU-CD datasets highlight the effectiveness of FAEWNet. Specifically, FAEWNet achieves the best performance on the LEVIR-CD with 91.29%Rc, 92.41%F1, and 85.89%IoU. On the S2Looking dataset, our method achieves improvements of 3.09% inRc, 1.07% inF1, and 1.23% inIoUcompared to the second-best method. On the WHU-CD dataset, it achieves the highest overall performance, with aRcof 94.20%, anF1 of 94.99%, and anIoUof 90.45%. The code is available at https://github.com/SUPERMAN123000/FAEWNet.

Hyperspectral image (HSI) classification is crucial for remote sensing research, while its high-dimensional features make traditional algorithms difficult to cope with. Despite the breakthroughs in deep learning, the high computational complexity and energy consumption limit its application in resource-limited environments. Spiking neural networks (SNNs), mimicking the brain’s information processing with low power consumption, have emerged as a promising alternative for edge computing. However, SNNs struggle with complex tasks due to the nondifferentiability of spike signals, which complicates training and exhibits limitations in extracting deep features and modeling long-range dependencies. In this article, we propose a novel SNN framework that addresses these challenges by enhancing feature extraction and efficiently capturing dependencies in hyperspectral data. Our framework integrates an adaptive refocusing convolutional layer with a spike self-attention (SSA) mechanism. The adaptive refocusing convolutional layer employs learnable parameters to dynamically adjust the convolutional kernel’s response to input spike data, improving feature representation. The adaptive refocusing convolutional layer uses learnable parameters to dynamically adjust kernel responses to input spike data, enhancing feature representation. Experimental results show that this model achieves over 96% classification accuracy in a single time step, significantly surpassing current methods and effectively solving the problem of low accuracy at short time steps in SNNs. Additionally, this framework reduces computational energy consumption by approximately$12.5\times $compared to similar, offering new potential for edge intelligence applications.

Convolutional Neural Networks (CNNs) and Transformers have made remarkable achievements in hyperspectral image classification (HSIC). Unfortunately, CNN-based methods struggle to capture the contextual dependencies between pixels in HSIs, while Transformer-based methods suffer from quadratic computational complexity. Recently, the Mamba model has shown great potential as it can describe long-range dependencies between HSI pixels with linear computational complexity. Yet, Mamba still faces significant challenges in terms of global modeling. Inspired by the Mamba model framework and Transformers, a novel Dual-Branch Mamba-Like Linear Attention (DBMLLA) network is proposed for HSIC, achieving efficient global dependency modeling. Specifically, the proposed DBMLLA combines an embedding module, a Spatial-Spectral Mamba-Like Linear Attention (SS-MLLA) module, and a fusion module. In the embedding module, an absolute position embedding module is introduced for better extraction of global features. In the SS-MLLA module, we design the Spatial Mamba-Like Linear Attention (SpaMLLA) block and the Spectral Mamba-Like Linear Attention (SpeMLLA) block to exploit the spatial and spectral information of the HSI. In addition, SS-MLLA is improved by utilising Depthwise Separable Convolution (DSC) to enhance the model’s ability to extract deeper local feature information. Through experiments conducted on four public hyperspectral datasets, it is demonstrated that the proposed model consistently outperforms state-of-the-art approaches.

Transformer models have achieved remarkable success in hyperspectral image classification (HSIC) owing to their strong global modeling capability. However, their quadratic complexity significantly limits their computational efficiency. Recently, Mamba has been applied to HSIC because of its linear complexity, yet it still suffers from an imbalance between global and local modeling. To overcome these challenges, this paper proposes a novel Learnable Kernel and Mamba with Spatial-Spectral Attention Fusion (LKMA) framework, which enables the extraction of global-local spatial-spectral features (SSF) while enhancing edge feature representation. For local feature extraction, the proposed Multi-Scale Spatial-Spectral Feature Generation (MSSFG) module captures local SSF by employing multi-scale learnable dilation convolutions for spatial features and multi-scale dilation convolutions for spectral features. For global feature extraction, a Global Hidden Mixing Mamba (GHMM) module is introduced, which projects hyperspectral image (HSI) features from the feature space to the hidden state space via a hidden state mixing mechanism. This enables the model to capture contextual semantic information and local details from the HSI. To further explore the synergistic effect between spatial and spectral information, the Spatial-Spectral Attention Fusion (SSAF) module integrates semantic information across multiple feature groups by combining Semantic Grouped Spatial Attention (SGSA) and Progressive Spectral Self-Attention (PSSA), enhancing spatial-spectral representations. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art approaches for HSIC.

With the development of infrared technology, infrared small target detection (IRSTD) is widely applied in fields such as environmental monitoring, marine rescue, and forest fire prevention. Existing IRSTD methods are often based on spatial domain approaches, which preserve target features in the spatial domain but overlook the characteristics of infrared small targets in the frequency domain. In frequency domain methods, infrared small targets are typically considered as high-frequency components, while the continuous background is regarded as low-frequency components. However, infrared small targets often have complex backgrounds, strong edges, and noise generated during imaging, all of which are also reflected as high-frequency components, leading to false detections. To overcome this issue and fully explore the potential of IRSTD in the frequency domain, we propose a novel network, SFDTNet, which integrates frequency-domain attention and U-Structure for IRSTD. In the encoding phase, spatial feature extraction is applied to the infrared small target. In the decoding stage, global-scale spatial features are modeled in the frequency domain to achieve more precise reconstruction of small targets while reducing the interference of background high-frequency clutter. Frequency domain self-attention (FDSA) introduces an attention mechanism to model global information in the frequency domain and capture the importance of different frequency components. Adaptive frequency selection network (AFSN) incorporates learnable masks to adaptively modulate high- and low-frequency components in the frequency domain. Finally, a deep supervision strategy is employed to help the network learn features more effectively. Experimental results demonstrate that it effectively retains the shape and contours of small targets while achieving a very low false detection rate. Compared with existing state-of-the-art methods, our approach shows superior performance and better robustness.

The “black-box” nature of machine learning (ML) and deep learning (DL) models has raised concerns about the trustworthiness of landslide susceptibility mapping (LSM) results among users. Existing studies have applied many techniques to interpret LSM models, but they predominantly focused on individual model interpretations, lacked comparisons of interpretation results across different models, and failed to fully explore the potential of explainable artificial intelligence (AI) techniques in LSM. This study develops an innovative model interpretation framework based on the Shapley additive explanation (SHAP) method and different ML and DL models, to analyze the decision mechanisms differences and discuss the geospatial heterogeneity of landslide conditioning factors (LCFs). A geospatial database is constructed, including historical landslides, 16 common LCFs, and three earthquake-related LCFs for two study areas: Zigui and Jiuzhaigou. The data are then divided into training and testing sets in a 7:3 ratio for four models: random forest (RF), extreme gradient boosting decision tree (XGBoost), residual network, and densely connected convolutional networks (DenseNets). Finally, global and local interpretations are provided using the SHAP method. The analysis indicates that: 1) XGBoost consistently outperforms the other models in both study areas, achieving Kappa coefficient (Kappa), overall accuracy (OA), and area under the receiver operating characteristic curve (AUC) values of 0.9416, 0.9738, and 0.9757 for Zigui, and 0.8525, 0.9337, and 0.9312 for Jiuzhaigou and 2) the global interpretation shows that the same LCFs play different roles in the XGBoost and DenseNet, reflecting different decision mechanisms among LSM models. Moreover, local interpretations demonstrate that the same LCFs contribute differently in the two areas, highlighting the geospatial heterogeneity in LSM.

Local and global spectral and spatial information is crucial for hyperspectral image (HSI) classification. However, modeling the global context has been challenging due to the limitations of receptive fields and quadratic complexity. Mamba’s ability to leverage long-range dependencies with linear computational complexity offers an effective approach to alleviate this issue; however, it does lead to the loss of local detail information. To address this challenge, we propose a novel local-to-global Mamba for HSI classification, termed MambaLG. MambaLG consists of a dual-branch strategy, comprising two core modules: a local and global spatial modeling module (SpaM) and a short- and long-range spectral dynamic perception module (SpeM). In the SpaM, the local and global spatial information is sequentially extracted and integrated, aiming to capture global spatial semantics while preserving the integrity of local 2-D spatial structures. In the SpeM, we utilize local spectral extraction, spectral grouping, and spectral dynamic correlation clustering (SDCC) modules, leveraging Mamba’s strengths in exploring long-range dependencies for more precise short- and long-range spectral feature modeling. Additionally, we introduce a gate attention unit into MambaLG and design a more efficient and interpretable manner for merging spatial and spectral features. Experimental results across multiple datasets (encompassing urban and agricultural scenes) indicate that MambaLG surpasses state-of-the-art algorithms regarding classification accuracy (CA) and inference speed. Comprehensive ablation studies substantiate the advantages of MambaLG in modeling local and global spatial context, enhancing short- and long-range spectral perception, and fusing spatial and spectral information. The codes will be openly available athttps://github.com/danfenghong/IEEE_TGRS_MambaLGto facilitate the reproduction of experimental results.

Hyperspectral unmixing, an essential and fundamental task in remote sensing, focuses on estimating endmembers (spectrally pure components) and their fractional abundances within each mixed pixel of a hyperspectral image. With the advent of deep learning (DL), the field of hyperspectral unmixing has made significant progress. Among DL approaches, autoencoder-based models have shown promising results. However, most unmixing methods estimate the endmembers by the weights of the linear layers in the decoder of their networks, making their performance highly dependent on weight initialization. Moreover, noise is not explicitly accounted for in most recent methods that use spectral angle distance (SAD) loss. To avoid the initialization problems, we developed an innovative inversion strategy to directly estimate the endmembers. Moreover, to optimally account for noise, an end-to-end network is proposed that integrates both denoising and unmixing. Finally, for an improved feature extraction, a novel spectral-spatial attention module is integrated in the network. Extensive experiments on a synthetic and three real datasets show that the proposed method significantly and consistently outperforms the compared state-of-the-art methods. The full code is available at https://github.com/xuanwentao for public evaluation.

Accurately obtaining the optical/microphysical characteristics of aerosols from satellite data is important for environmental protection and air quality monitoring. In this study, using Sentinel-2 satellite data, we propose a high-resolution satellite aerosol retrieval algorithm (CKAN) via convolutional neural network (CNN) and Kolmogorov-Arnold network (KAN). Unlike traditional retrieval algorithms that require the construction of physical models, the CKAN algorithm relies entirely on deep learning. This algorithm focuses on extracting high-dimensional information from the data through CNN and learning the potential nonlinear relationships between the data through the powerful fitting ability of KAN. Compared with the existing algorithms, the CKAN algorithm is characterized by simplicity and accuracy, and does not require a large amount of auxiliary meteorological data (e.g., relative humidity and ground air pressure) enables retrieval of various aerosol parameters, including Aerosol Optical Depth (AOD), Fine-mode AOD (FAOD), Coarse-mode AOD (CAOD), and Single Scattering Albedo (SSA). To demonstrate the effectiveness of the algorithm, we retrieved aerosol optical/microphysical characteristics from Sentinel-2 imagery for four study areas. Results indicate that both the AOD and FAOD retrieved by the CKAN algorithm exhibit a high degree of correlation ( R > 0.90) with AERONET products. For CAOD and SSA, although a few poor retrievals resulted in a low overall correlation, CKAN’s retrievals show good agreement with AERONET products. The CKAN algorithm has good accuracy in high-resolution satellite aerosol retrievals, and is expected to be widely used in urban-scale aerosol monitoring.

Hyperspectral target detection (HTD) can provide detailed information about the objects and materials within a scene and holds significant importance in remote sensing image analysis. Traditional HTD methods often suffer from low detection accuracy when applied to hyperspectral images (HSIs) with low spatial resolution, where the target only occupies a few pixels. This can be addressed by exploiting the higher spatial resolution of multispectral images (MSIs). In addition, many cloud-based HTD methods, which rely on the distributed processing capability of cloud computing to cope with large-scale datasets, may result in long transmission delays that cannot meet real-time requirements. This article suggests a cloud-edge collaborative HTD approach based on the fusion of remotely sensed HSIs and MSIs. We first introduce an HTD algorithm that employs low-rank matrix decomposition and hierarchical constraint energy minimization (hCEM) to fuse a low-resolution HSI (LR-HIS) and a high-resolution MSI (HR-MSI). Aiming at a continuous shooting scenario, we further present a cloud-edge implementation of the HTD algorithm through the collaboration of a cloud cluster and edge servers deployed close to data acquisition devices. The overall processing flow of remotely sensed data fusion in the cloud-edge environment is formulated as a flowshop scheduling-like optimization problem. We develop a co-optimization scheduling algorithm to explore the best resource allocation solutions to the formulated problem. Experimental results on both general-purpose and real-world datasets show that the newly proposed HTD algorithm leads to significant improvements in detection accuracy over traditional methods, and the cloud-edge collaborative approach further enhances computational efficiency.

In recent years, graph convolutional networks (GCNs) have gained increasing attention in hyperspectral image (HSI) classification due to their good ability to model the pairwise relationships between two pixels. However, it is difficult to effectively model more complex relationships among multiple pixels with simple graphs. To solve this problem, we propose a novel tensorized high-order hypergraph convolutional network (TH2GCN) for HSI classification. Specifically, the hypergraph structure is employed to effectively model complex spatial relationships between pixels in HSIs, and we propose a new tensor-based algebraic representation of hypergraphs as a powerful strategy for describing the high-order interaction structures of the hypergraph. Besides, by extending the adjacency matrix-based GCN to the tensor domain and exploiting the tensor decomposition, the TH2GCN method is designed to efficiently extract high-order discriminative information from the hypergraph at low complexity for improving HSI classification performance. Furthermore, the construction of the adjacency tensor on all the data requires a huge amount of memory, especially for large-scale remote sensing images. To this end, the TH2GCN is trained and tested for HSI data in a minibatch fashion. Experimental results on three HSI datasets prove that the performance of the proposed method outperforms the comparison methods.

Active learning (AL) has achieved great success in remotely sensed hyperspectral image classification due to its ability to select highly informative training samples. An appropriate query strategy is the core of a successful AL method. In this paper, we develop a new graph signal sampling (GS)-based AL strategy which aims to query the most globally optimal and informative pixels for hyperspectral image classification. We first demonstrate the significance of smooth graph structures for GS. Then, we propose a multi-view fused graph-based GS method (GS-FusG) for active learning. Our method combines connection information from multiple perspectives to improve the sampling effectiveness and the performance of graph signal reconstruction. Besides, we present a new reconstruction method for multidimensional graph signals, which integrates the results of multi-view graph signal reconstruction. Last but not least, we develop three variants of GS-FusG based on different multi-view compositions and signal reconstruction methods. Our experimental results with real hyperspectral images demonstrate that our proposed GS-FusG can greatly enhance AL effectiveness and classification performance.

Deep learning-based methods for hyperspectral image classification (HSIC) have been widely utilized in recent years. However, existing HSIC methods do not adequately account for illumination variations in HSIs, particularly in urban areas where shadows created by complex ground objects result in significant variations that cannot be ignored. Additionally, when dealing with limited labeled data, most deep learning methods are prone to overfitting, resulting in poor classification performance. To address these challenges, we propose a new Retinex-based dual-branch feature extraction network (RDFEN) for HSIC. First, by incorporating Retinex theory, we propose a hyperspectral Retinex (HyperRetinex) module to extract illumination attributes and reflectance attributes. Then, we propose a dual-branch feature extraction network, which consists of two submodules: illumination attributes feature extraction (IAFE) module and reflectance attributes feature extraction (RAFE) module. Finally, an illumination-reflectance attributes interaction attention fusion (IRAIAF) module is strategically designed to integrate distinct features. Experiments on four benchmark HSI datasets demonstrate that the proposed method outperforms other state-of-the-art HSIC methods, achieving up to 96.85% overall accuracy on the Pavia University dataset, thereby highlighting its effectiveness and robustness in HSIC. For reproducibility, the code is available at https://github.com/JT-shen/Code.

Graph convolutional network (GCN) has garnered significant attention in hyperspectral image (HSI) classification due to their ability to model non-Euclidean structured data. Compared with convolutional neural network (CNN), GCN can perform convolutions over irregular image regions and learn global dependencies among pixels in the whole image. Most existing GCN-based methods in the HSI community rely on average aggregation or weighted average aggregation strategies to aggregate neighboring node features. This process tends to obscure the differences between the nodes. However, for HSI classification tasks with obvious intra-class variability, average aggregation is a suboptimal choice. Moreover, the quality of the initial graph structure plays a crucial role in the model’s capacity to represent spectral relationships effectively. To mitigate these issues, we propose a neighborhood adaptive graph isomorphism network (NAGIN) for HSI classification to ensure that the diversified spectra representation of land-cover can be effectively captured. The neighborhood adaptive block (NAB) enhances spectral discriminability between land-cover classes via spectral reconstruction, enabling more precise removal of anomalous pixels in neighboring nodes. The graph isomorphism network (GIN) aggregates the features of neighboring nodes in an isomorphic manner to obtain multiple spectral expressions of the same type of land-cover, ensuring that the spectral features of different land-cover classes can be accurately distinguished. The Kolmogorov-Arnold network (KAN) leverages its ability to learn adaptive activation functions to better extract and refine the spectral features aggregated by GIN. Experimental results demonstrate that NAB can effectively improve the quality of the graph structure, the GIN aggregation method is competitive in HSI classification, and the proposed NAGIN outperforms the state-of-the-art methods on several public HSI datasets.

Existing hyperspectral image classification (HSIC) models based on the Mamba architecture predominantly center on characterizing the original spectral and spatial domains, with limited exploration of time–frequency analysis. In this study, we propose a novel wavelet decomposition-based spectral–spatial Mamba network for HSIC, dubbed “WD-SSMamba.” This model incorporates both 1-D and 2-D wavelet decompositions to extract spectral and spatial features in the frequency domain, respectively. Specifically, we design an innovative frequency feature extraction (FE) block, which comprises a spectral wavelet convolution (SWC) module for spectral FE and a wavelet separable convolution (WSC) module for spatial FE. Furthermore, to address the challenges of integrating spectral–spatial features in traditional Mamba models for HSIC, we devise a dual-branch Mamba block featuring a cross-fusion structure, termed the “HyperMamba” block, to efficiently extract and fuse spectral and spatial features. Comprehensive experiments were conducted on four publicly available hyperspectral image (HSI) datasets, namely, Pavia University, WHU-Hi-LongKou, WHU-Hi-HongHu, and Houston 2013. The results show that the WD-SSMamba model achieves overall accuracies (OAs) of 92.48%, 97.91%, 89.98%, and 87.00% on these datasets, respectively, with fewer than 20 training samples per class, surpassing those of other competing models across all tested datasets. Moreover, it significantly reduces the number of parameters to less than 50k and floating-point operations (FLOPs) to less than 3.02M, thereby fully showcasing the immense potential of frequency analysis and the Mamba structure in HSIC.

Hyperspectral image change detection (HSI-CD) is a technique that intelligently checks the changed details in bitemporal hyperspectral images (Bi-HSIs). Deep learning (DL), with the ability to model nonlinear changing features, has achieved promising results in HSI-CD, but the feature mining mechanism is unclear and the architecture design lacks transparency in such DL models. To alleviate this problem, this paper proposes a new low-rank and sparse representation-based deep unfolding network (LRSRNet) for HSI-CD. For feature mining mechanism, the LRSRNet adopts a low-rank and sparse subnetwork (LRSnet) and a change detection sub-network (CDnet). The former is responsible for extracting low-rank features with valuable information and suppressing sparse features containing interference information, while the latter aims to obtain change information from low-rank features. For architecture design, the LRSnet formulates the HSI as a low-rank estimation, sparse estimation, and hyperspectral reconstruction in a low-rank and sparse model, and iteratively optimizes and updates the above sub-problems through deep networks. A new CDnet is designed as a concise convolutional architecture to extract change information from representative Bi-HSIs features. Experiments on three real datasets demonstrate the performance superiority of the proposed LRSRNet method over nine model-driven, datadriven, and model-data-joint-driven HSI-CD algorithms in both qualitative and quantitative evaluations. The proposed LRSRNet is available online: https://github.com/chengle-zhou/LRSRNet.

Hyperspectral image anomaly detection faces the challenge of difficulty in annotating anomalous targets. Autoencoder(AE)-based methods are widely used due to their excellent image reconstruction capability. However, traditional grid-based image representation methods struggle to capture long-range dependencies and model non-Euclidean structures. To address these issues, this paper proposes a self-supervised Masked Graph AutoEncoder (MGAE) for hyperspectral anomaly detection. MGAE utilizes a Graph Attention Network (GAT) autoencoder to reconstruct the background of hyperspectral images and identifies anomalies by comparing the reconstructed features with the original features. Specifically, we constructs a topological graph structure of the hyperspectral image, which is then input into the GAT autoencoder for reconstruction, leveraging the multi-head attention mechanism to learn spatial and spectral features. To prevent the decoder from learning trivial solutions, we introduce a re-masking strategy that randomly masks both the input features and hidden representations during training, forcing the model to learn and reconstruct features under limited information, thereby improving detection performance. Additionally, the proposed loss function with graph Laplacian regularization (Twice Loss) minimizes variations in feature representations, leading to more consistent background reconstruction. Experimental results on several real-world hyperspectral datasets demonstrate that MGAE outperforms existing methods.

Autoencoders (AEs) have received extensive attention in hyperspectral anomaly detection (HAD) due to their capability to separate the background from the anomaly based on the reconstruction error. However, the existing AE methods routinely fail to adequately exploit spatial information and may precisely reconstruct anomalies, thereby affecting the detection accuracy. To address these issues, this study proposes a novel Multi-scale Autoencoder Suppression Strategy (MASS). The underlying principle of MASS is to prioritize the reconstruction of background information over anomalies. In the encoding stage, the Local Feature Extractor, which integrates Convolution and Omni-Dimensional Dynamic Convolution (ODConv), is combined with the Global Feature Extractor based on Transformer to effectively extract multi-scale features. Furthermore, a Self-Attention Suppression module (SAS) is devised to diminish the influence of anomalous pixels, enabling the network to focus more intently on the precise reconstruction of the background. During the process of network learning, a mask derived from the test outcomes of each iteration is integrated into the loss function computation, encompassing only the positions with low anomaly scores from the preceding detection round. Experiments on eight datasets demonstrate that the proposed method is significantly superior to several traditional methods and deep learning methods in terms of performance.

Accurately distinguishing between background and anomalous objects within hyperspectral images poses a significant challenge. The primary obstacle lies in the inadequate modeling of prior knowledge, leading to a performance bottleneck in hyperspectral anomaly detection (HAD). In response to this challenge, we put forth a groundbreaking coupling paradigm that combines model-driven low-rank representation (LRR) methods with data-driven deep learning techniques by learning disentangled priors (LDP). LDP seeks to capture complete priors for effectively modeling the background, thereby extracting anomalies from hyperspectral images more accurately. LDP follows a model-driven deep unfolding architecture, where the prior knowledge is separated into the explicit low-rank prior formulated by expert knowledge and implicit learnable priors by means of deep networks. The internal relationships between explicit and implicit priors within LDP are elegantly modeled through a skip residual connection. Furthermore, we provide a mathematical proof of the convergence of our proposed model. Our experiments, conducted on multiple widely recognized datasets, demonstrate that LDP surpasses most of the current advanced HAD techniques, exceling in both detection performance and generalization capability.

Convolutional long short-term memory (ConvLSTM) possesses a remarkable capability of encoding spatial information and capturing long-range dependencies in sequential data. As a result, ConvLSTM has garnered success in hyperspectral image (HSI) classification. Nonetheless, the design of the special gate structures and convolution operations contributes to a high model complexity, making it challenging to deploy in resource-constrained environments. In this article, we propose a fully tensorized ConvLSTM model for HSI spatial-spectral classification under the premise of low complexity. First, we devise a novel and efficient tensor-sequenced convolution in the tensor train (TT) format, called ETTConv. ETTConv can reduce the number of parameters and computations in the standard convolutional layer by tensorizing the convolution kernels and mapping them to a series of smaller ones. Building upon this innovation, we present a novel ETTConvLSTM unit, formed by jointly compressing all weight tensors within the recurrent units. Using it as the fundamental unit, we construct the lightweight a efficient tensor train ConvLSTM 2-D neural network (ETTCL2DNN) model, characterized by reduced complexity without compromised classification performance. Furthermore, to better preserve the joint spatial-spectral structure of HSI data, we extend the ETTConv layer and the ETTConvLSTM unit to their 3-D versions, resulting in a new lightweight a efficient tensor train ConvLSTM 3-D neural network (ETTCL3DNN) model. Extensive quantitative experimental results on three widely used HSI datasets demonstrate the superiority of the proposed methods, exhibiting enhanced classification performance with reduced model complexity.

Vision transformer (ViT) has recently been a popular topic in the foundation model field, taking advantage of its strong scalability and outstanding representation capabilities. As a deep model, ViT introduces a new architecture for achieving hyperspectral image (HSI) unmixing. However, traditional ViTs overlook pixel-level spatial continuity by partitioning the input image into nonoverlapping fixed-size patches. This approach disrupts local structural relationships and hinders the model's ability to capture fine-grained spatial dependencies, resulting in suboptimal feature representation for dense prediction tasks in unmixing. To address these challenges, this article proposes the development of a self-supervised relation-aware ViT (SRViT). SRViT incorporates a self-embedded module comprising encoders, a pixel-level position encoder (PLPE), a self-supervised contrastive mechanism (SCM), and a decoder. The self-embedded module and PLPE preserve local correlations in HSI across different views, facilitating cross-view learning through SCM to ensure generalization. In addition, the decoder incorporates Kronecker-factored approximate curvature (K-FAC) to capture the local geometric structure of spectral information. Ultimately, SRViT learns endmembers and fractional abundance as the unmixing result. The effectiveness and competitiveness of SRViT have been systematically validated through comparative experiments, demonstrating its superior performance. The source code is available at the following link: https://github.com/yuanchaosu/TNNLS-SRViT.

Graph theory-based techniques have recently been adopted for anomaly detection in hyperspectral images (HSIs). However, these methods rely excessively on the relational structure within the constructed graphs and tend to downplay the importance of spectral features in the original HSI. To address this issue, we introduce graph frequency analysis to hyperspectral anomaly detection (HAD), which can serve as a natural tool for integrating graph structure and spectral features. We treat anomaly detection as a problem of graph frequency location, achieved by constructing a beta distribution-based graph wavelet space, where the optimal wavelet can be identified adaptively for anomaly detection. Initially, a high-dimensional, undirected, unweighted graph is built using the pixels in the HSI as vertices. By leveraging the observation of energy shifting to higher frequencies caused by anomalies, we can dynamically pinpoint the specific Beta wavelet associated with the anomalies' high-frequency content to accurately extract anomalies in the context of HSIs. Furthermore, we introduce a novel entropy definition to address the frequency location problem in an adaptive manner. Experimental results from seven real HSIs validate the remarkable detection performance of our newly proposed approach when compared to various state-of-the-art anomaly detection methods.

In this paper, a representative set of state-of-the-art methods for hyperspectral pansharpening, comprising both model- and deep learning-based ones, are reviewed and compared on four datasets from the PRISMA mission. The experimental analysis has been carried out using the most credited pansharpening quality indexes, complemented by a subjective visual inspection of sample results. The obtained outcomes have provided us a preview of the strengths and weaknesses of the latest solutions to the problem at hand, paving the way for future research lines from both the methodological and quality assessment perspectives.

This paper presents a novel self-distillation based deep robust hash for fast remote sensing (RS) image retrieval. Specifically, there are two primary processes in our proposed model: teacher learning (TL) and student learning (SL). Two transformed samples are produced from one sample image through nuanced and signalized transformations, respectively. Transformed samples are fed into both the TL and the SL flows. To reduce discrepancies in the processed samples and guarantee a consistent hash code, the parameters are shared by the two modules during the training stage. Then, a resilient module is employed to enhance the image features in order to ensure more dependable hash code production. Lastly, a three-component loss function is developed to train the entire model. Comprehensive experiments are conducted on two common RS datasets: UCMerced and AID. The experimental results validate that the proposed method has competitive performance against other RS image hashing methods.

Global climate change leads to increasing frequency and severity of urban flood disasters, which restricts human sustainable development. So far, many studies have explored the potential of fusing remote sensing images and social media data for monitoring urban flood disasters. However, the SAR flooded characteristics of different land cover in complex urban environment are generally ignored. In this paper, we develop a new method for the fusion of heterogeneous SAR images and social media data by using domain adaptation and land cover information. The 2017 Houston flood event is taken as a case study for evaluation. According to our experiments, compared with traditional methods of optimal transport and geographic optimal transport, the proposed method can align the geo-tagged tweets with location uncertainty to nearby flooded areas of low-, medium-, and high-intensity developed areas with more remarkable performance.

Feature extraction is a prevalent technique in hyperspectral remote sensing. Various tasks require this technique as a pre-processing step, including image classification, anomaly detection, image denoising, and so on. Edge-preserving filtering based methods have been extensively utilized for this purpose. However, these methods do not take the inherent structural and textural information into account, leading to poor performance in classifying hyperspectral images (HSIs). In this letter, a new structural and textural-aware feature extraction method is proposed that preserves the relevant structural information and removes useless textures. First, structural and textural-aware recursive filtering features (STRFs) are extracted along with an exponential form of windowed inherent variance (eWIV). Then, multi-scale STRFs are integrated by the principal component analysis (PCA) method to obtain more discriminative features (MSTRF). Finally, the fused features are fed into a pixel-wise classifier to obtain the final results. The main difference between the MSTRF method and other feature extraction methods is that the MSTRF method can make full use of the proposed eWIV map, which can help to properly characterize structure and texture in HSIs. Experimental results on several public data sets indicate that our method leads to state-of-the-art classification performance, especially in the presence of very small training set.

The foundation model has recently garnered significant attention due to its potential to revolutionize the field of visual representation learning in a self-supervised manner. While most foundation models are tailored to effectively process RGB images for various visual tasks, there is a noticeable gap in research focused on spectral data, which offers valuable information for scene understanding, especially in remote sensing (RS) applications. To fill this gap, we created for the first time a universal RS foundation model, named SpectralGPT, which is purpose-built to handle spectral RS images using a novel 3D generative pretrained transformer (GPT). Compared to existing foundation models, SpectralGPT 1) accommodates input images with varying sizes, resolutions, time series, and regions in a progressive training fashion, enabling full utilization of extensive RS Big Data; 2) leverages 3D token generation for spatial-spectral coupling; 3) captures spectrally sequential patterns via multi-target reconstruction; and 4) trains on one million spectral RS images, yielding models with over 600 million parameters. Our evaluation highlights significant performance improvements with pretrained SpectralGPT models, signifying substantial potential in advancing spectral RS Big Data applications within the field of geoscience across four downstream tasks: single/multi-label scene classification, semantic segmentation, and change detection.

The widespread availability of remotely sensed datasets establishes a cornerstone for comprehensive image retrieval within the realm of remote sensing (RS). In response, the investigation into hashing-driven retrieval methods garners significance, enabling proficient image acquisition within such extensive data magnitudes. Nevertheless, the used datasets in practical applications are invariably less desirable and with long-tailed distribution. The primary hurdle pertains to the substantial discrepancy in class volumes. Moreover, commonly utilized RS datasets for hashing tasks encompass approximately two–three dozen classes. However, real-world datasets exhibit a randomized number of classes, introducing a challenging variability. This article proposes a new centripetal intensive attention hashing (CIAH) mechanism based on intensive attention features for long-tailed distribution RS image retrieval. Specifically, an intensive attention module (IAM) is adopted to enhance the significant features to facilitate the subsequent generation of representative hash codes. Furthermore, to deal with the inherent imbalance of long-tailed distributed datasets, the utilization of a centripetal loss function is introduced. This endeavor constitutes the inaugural effort toward long-tailed distributed RS image retrieval. In pursuit of this objective, a collection of long-tail datasets is meticulously curated using four widely recognized RS datasets, subsequently disseminated as benchmark datasets. The selected fundamental datasets contain 7, 25, 38, and 45 land-use classes to mimic different real RS datasets. Conducted experiments demonstrate that the proposed methodology attains a performance benchmark that surpasses currently existing methodologies.

In recent years, the rapid development of remote sensing (RS) technology has led to a drastic increase in the availability of RS images. This calls for the need to develop new methods able to effectively and efficiently retrieve the required instances from a massive amount of RS imagery. In retrieval tasks, finding the nearest-neighbor sample of the retrieval query is a fundamental research topic. Exhaustive comparison is the simplest method to accomplish this task. However, due to the involved computational complexity and memory limitations, this solution is no longer feasible in large data retrieval tasks. As an important branch of approximate nearest-neighbor retrieval (NNR), hash algorithms transform high-dimensional data into low-bit expressions (hash codes) with elements of 0 and 1 to reduce storage and computational costs. Hash algorithms aim to preserve the same nearest-neighbor relationship between the learned hash codes and the original data. Existing hash algorithms are divided into two classes: shallow and deep methods. Furthermore, deep hash algorithms can be divided into (semi-) supervised and unsupervised algorithms. In this article, representative hash-based RS image retrieval (HBRSIR) methods are reviewed, studying the application of hashing in other areas of the RS community and introducing available datasets and evaluation metrics for RS image retrieval (RSIR). The performance of representative and cross-modal hashing methods is validated using two common RSIR datasets (UCMerced and AID) and a cross-modal dataset (DSRSID). Prospects of future work summarizing HBRSIR are also provided.

Standard hyperspectral (HS) pansharpening utilizes panchromatic (PAN) images to improve the connected low-resolution HS (LRHS) images to the spatial resolutions of PANs; while arbitrary-resolution hyperspectral (ARHS) pansharpening aims to use PANs to enhance LRHS images to any desired spatial resolutions. For the challenging task of ARHS pansharpening, one of major obstacles is how to generalize the single pansharpening model learned under predetermined training scales to any pansharpening scales for future data. As implicit neural representations (INRs) have a potential to approximate continuous functions, they offer a possible alternative way to naturally resolve ARHS pansharpening. In this paper, we develop two spectral-spatial INRs for ARHS pansharpening: one is a naive pansharpening INR (NaivePINR); the other is a dynamic pansharpening INR (DynamicPINR). The former builds a novel spectral-spatial encoding to produce spectral-spatial priors of observed scenes and uses a spectral-spatial query mapping to reconstruct fine spectral details and spatial details. The latter establishes a innovative two-fold tuning mechanism to dynamically adjust both the spectral-spatial encoding and the spectral-spatial query mapping. Experimental results on serval datasets verify the excellent performances of the proposed pansharpening INRs.

Hyperspectral image (HSI) classification has garnered substantial attention in remote sensing fields. Recent mamba architectures built upon the selective state-space models (S6) have demonstrated enormous potential in long-range sequence modeling. However, the high dimensionality of hyperspectral data and information redundancy pose challenges to the application of S6 in HSI classification, suffering from suboptimal performance and computational efficiency. In light of this, this article investigates a lightweight interval group spatial-spectral mamba framework (IGroupSS-Mamba) for HSI classification, which allows for multidirectional and multiscale global spatial-spectral information extraction in a grouping and hierarchical manner. Technically, an interval group S6 mechanism (IGSM) is developed as the core component, which partitions high-dimensional features into multiple nonoverlapping groups at intervals, and then integrates a unidirectional S6 for each group with a specific scanning direction to achieve nonredundant sequence modeling. Compared with conventional applying multidirectional scanning to all bands, this grouping strategy leverages the complementary strengths of different scanning directions while decreasing computational costs. To adequately capture the spatial-spectral contextual information, an interval group spatial-spectral block (IGSSB) is introduced, in which two IGSM-based spatial and spectral operators are cascaded to characterize the global spatial-spectral relationship along the spatial and spectral dimensions, respectively. IGroupSS-Mamba is constructed as a hierarchical structure stacked by multiple IGSSB blocks, integrating a pixel aggregation-based downsampling strategy for multiscale spatial-spectral semantic learning from shallow to deep stages. Extensive experiments demonstrate that IGroupSS-Mamba significantly outperforms the state-of-the-art methods in classification accuracy and achieves lower model parameters and floating point operations (FLOPs). The code is available athttps://github.com/IIP-Team/IGroupSS-Mamba.

Hyperspectral image (HSI) classification constitutes a significant foundation for remote sensing analysis. Transformer architecture establishes long-range dependencies with a self-attention mechanism (SA), which exhibits advantages in HSI classification. However, most existing transformer-based methods are inadequate in exploring the multiscale properties of hybrid spatial and spectral information inherent in HSI data. To countermeasure this problem, this work investigates a hybrid multiscale spatial–spectral framework (HMSSF). It innovatively models global dependencies across multiple scales from both spatial and spectral domains, which allows for cooperatively capturing hybrid multiscale spatial and spectral characteristics for HSI classification. Technically, a spatial–spectral token generation (SSTG) module is first designed to generate the spatial tokens and spectral tokens. Then, a multiscale SA (MSSA) is developed to achieve multiscale attention modeling by constructing different dimensional attention heads per attention layer. This mechanism is adaptively integrated into both spatial and spectral branches for hybrid multiscale feature extraction. Furthermore, a spatial–spectral attention aggregation (SSAA) module is introduced to dynamically fuse the multiscale spatial and spectral features to enhance the classification robustness. Experimental results and analysis demonstrate that the proposed method outperforms the state-of-the-art methods on several public HSI datasets.

Hyperspectral image (HSI) classification constitutes the fundamental research in remote sensing fields. Convolutional neural networks (CNNs) and Transformers have demonstrated impressive capability in capturing spectral-spatial contextual dependencies. However, these architectures suffer from limited receptive fields and quadratic computational complexity, respectively. Fortunately, recent Mamba architectures built upon the state space models (SSMs) integrate the advantages of long-range sequence modeling and linear computational efficiency, exhibiting substantial potential in low-dimensional scenarios. Motivated by this, we propose a novel 3D-spectral-spatial mamba (3DSS-Mamba) framework for HSI classification, allowing for global spectral-spatial relationship modeling with greater computational efficiency. Technically, a spectral-spatial token generation (SSTG) module is designed to convert the HSI cube into a set of 3-D spectral-spatial tokens. To overcome the limitations of traditional Mamba, which is confined to modeling causal sequences and inadaptable to high-dimensional scenarios, a 3D-spectral-spatial selective scanning (3DSS) mechanism is introduced, which performs pixel-wise selective scanning on 3-D hyperspectral tokens along the spectral and spatial dimensions. Five scanning routes are constructed to investigate the impact of dimension prioritization. The 3DSS scanning mechanism combined with conventional mapping operations forms the 3D-spectral-spatial mamba block (3DMB), enabling the extraction of global spectral-spatial semantic representations. Experimental results and analysis demonstrate that the proposed method outperforms the state-of-the-art methods on HSI classification benchmarks. The code is available athttps://github.com/IIP-Team/3DSS-Mamba.

Hyperspectral images (HSIs) provide a large amount of spatial and spectral information to characterize ground objects. However, they also contain a lot of redundant information, which makes it difficult to extract complex local and global spatial-spectral features. Considering that HSIs present multi-scale similarity and anisotropic image features, multi-scale and multi-angle information can be used to effectively model local and global features and reduce the complexity of self-attention. This paper proposes a new multi-scale multi-angle attention network (MMAN) for HSI classification that models the internal relationship between image features at local and global scales. Firstly, three spectral-spatial feature extraction modules (at different scales) are constructed to extract the low-level features of the image. These modules are first used by a 3D convolutional layer for spectral feature extraction, and then input to a 2D convolutional layer for spatial feature extraction. Next, the serialized tokens are input to the multi-angle attention module. Finally, the learnable labels are identified through a linear layer, and the features of different scales are fused through a fully connected layer to realize the classification of samples. Experimental results on four standard datasets show that the proposed exhibits comparable or superior classification performance than other state-of-the-art methods.

Transformer holds significance in deep learning (DL) research. Node embedding (NE) and positional encoding (PE) are usually two indispensable components in a Transformer. The former can excavate hidden correlations from the data, while the latter can store locational relationships between nodes. Recently, the Transformer has been applied for hyperspectral image (HSI) classification because the model can capture long-range dependencies to aggregate global features for representation learning. In an HSI, adjacent pixels tend to be homogeneous, while the NE does not identify the positional information of pixels. Therefore, PE is crucial for Transformers to understand locational relationships between pixels. However, in this area, most Transformer-based methods randomly generate PEs without considering their physical meaning, which leads to weak representations. This article proposes a new graph generative structure-aware Transformer (GraphGST) to solve the above-mentioned PE problem when implementing HSI classification. In our GraphGST, a new absolute PE (APE) is established to acquire pixels’ absolute positional sequences (APSs) and is integrated into the Transformer architecture. Moreover, a generative mechanism with self-supervised learning is developed to achieve cross-view contrastive learning (CL), aiming to enhance the representation learning of the Transformer. The proposed GraphGST model can capture local-to-global correlations, and the extracted APSs can complement the spectral features of pixels to assist in NE. Several experiments with real HSIs are conducted to evaluate the effectiveness of our GraphGST. The proposed method demonstrates very competitive performance compared with other state-of-the-art (SOTA) approaches. Our source codes will be provided in the following linkhttps://github.com/yuanchaosu/TGRS-graphGST.

Current hyperspectral anomaly detection (HAD) benchmark datasets suffer from low resolution, simple background, and small size of the anomalies. These factors also limit the performance of the well-known low-rank representation (LRR) models in terms of robustness on the separation of background and target features and the reliance on manual parameter selection. To this end, we build a new HAD benchmark dataset for improving the robustness in complex scenarios, AIR-HAD for short, and propose an interpretable network with deep unfolding a binary subspace learning, named LRR-Net+, which is capable of spectrally decoupling the background structure and object properties in a more generalized fashion and eliminating the bias introduced by vital interference targets simultaneously. In addition, LRR-Net+ integrates the solution process of the alternating direction method of multipliers (ADMM) optimizer with the deep network, guiding its search process and imparting a level of interpretability to parameter optimization. Additionally, the integration of physical models with DL techniques eliminates the need for manual parameter tuning. The manually tuned parameters are seamlessly transformed into trainable parameters for deep neural networks, facilitating a more efficient and automated optimization process. Extensive experiments conducted on the AIR-HAD dataset show the superiority of our LRR-Net+ in terms of detection performance and generalization ability, compared to top-performing competitors. Furthermore, our AIR-HAD benchmark datasets will be made available freely and openly athttps://github.com/danfenghong/IEEE_TGRS_LRR-Net.

The integration of remote sensing and citizen science offers an unprecedented opportunity for observing the Earth and human activities. Recently, heterogeneous data fusion models have been proposed to align representations and geolocations of remote sensing imagery and social media data, such as optimal transport (OT) and geographic OT (GOT). However, these models generally ignore the differences in remote sensing features of the same geographical phenomenon for different land cover types, which not only affects the fusion accuracy but also leads to the loss of fusion information. In this study, we develop a general model for heterogeneous SAR imagery and social media data fusion based on OT and land cover information, namely, land cover information-constraint OT (LCIOT). Taking the 2017 Houston flood event as a case study, the experimental findings demonstrate that the proposed LCIOT can align 99% of geotagged Twitter data to flooded areas with an average transport distance of 710 m, outperforming the state-of-the-art models, i.e., OT and GOT. By combining the supervised information obtained by LCIOT with the SAR imagery, the overall accuracy (OA) and Kappa of urban flood mapping in the three study areas ranges from 0.77 to 0.82 and 0.51 to 0.61, respectively. Overall, the proposed LCIOT provides a new perspective to solve the issues of discrepancies in data distribution and geolocation uncertainty in the context of heterogeneous data fusion.

Oriented object detection is an important research topic in remote sensing. The detection of oriented objects in remote sensing images remains a daunting challenge due to their complex backgrounds, various sizes, diverse aspect ratios, and especially arbitrary orientations. In recent years, keypoint-based anchor-free object detectors have demonstrated outstanding performance in this field. However, in current anchor-free detectors, object keypoints are primarily generated using the Gaussian kernel function, which assumes a circular form. This representation falls short in accurately conveying an object’s size and orientation. To address the aforementioned issue, this paper proposes a keypoint-based oriented object detector called MRSDet, which innovatively adopts the Tricube kernel, scales and rotates it, to better generate the center keypoint heatmap of the object. Besides, to improve the model’s detection performance on oriented objects and improve its ability to perceive object keypoints and boundary boxes, we also design a large receptive field mask module (LRFM), which is based on large convolution kernel decomposition and semantic segmentation masks. Taking the BBAVectors method as a baseline, we conduct experiments on multiple types of remote sensing datasets such as HRSC2016, UCAS-AOD and SSDD+ to verify the effectiveness and generalizability of the proposed method.

In recent years, deep learning (DL) has accelerated the development of hyperspectral image (HSI) processing, expanding the range of applications further. As a typical model of unsupervised DL, the autoencoder framework has been extensively applied for spectral unmixing due to its strong representation ability and scalability. Nowadays, most DL-based unmixing approaches adopt the linear mixture model (LMM) to estimate pure spectral signatures (endmembers) and their corresponding abundance fractions. However, since sunlight scattering is an inevitable physical phenomenon, the spectral mixture problem is inherently nonlinear. Moreover, most existing nonlinear unmixing approaches focus exclusively on spectral information, neglecting the spatial distribution of materials and the intrinsic correlation between pixels, making it challenging to explore latent features. To address these issues, this article develops a new deep autoencoder-based augmented network (DAAN). The proposed DAAN employs the multilinear mixture model (MLMM) to handle the nonlinear influence caused by multiple scattering. Meanwhile, the proposed DAAN constraints homogenous smoothing in the autoencoder architecture, enabling the aggregation of intrinsic correlations by means of spatial relationships to enhance the performance of abundance estimation. We achieve unsupervised nonlinear hyperspectral unmixing by combining spectral and spatial information. The effectiveness and advantages of DAAN are confirmed by several experiments with synthetic and real HSI datasets. The results indicate that the proposed method outperforms other DL-based unmixing approaches. The source codes of the proposed DAAN will be provided in the following linkhttps://github.com/yuanchaosu/TGRS-daan.

Cross-modal hashing plays a pivotal role in large-scale remote sensing (RS) ship image retrieval. RS ship images often exhibit similar overall appearance with subtle differences. Existing hashing methods typically employ feature non-interaction strategies to generate common hash codes, which may not effectively capture the correlations between cross-modal ship images to reduce intermodality discrepancies. To address this issue, we propose a novel cross-modal hashing approach based on feature semi-interaction and semantic ranking (FSISR) for RS ship image retrieval. Our FSISR approach not only captures intricate correlations between different ship image modalities, but also enables the construction of hash tables for large-scale retrieval. FSISR comprises a feature semi-interaction module and a semantic ranking objective function. The semi-interaction module utilizes clustering centers from one modality to learn the correlations between two modalities and generate robust shared representations. The objective function optimizes these representations in a common Hamming space, consisting of a shared semantic alignment loss and a margin-free ranking loss. The alignment loss employs a shared semantic layer to preserve label-level similarity, while the ranking loss incorporates hard examples to establish a margin-free loss that captures similarity ranking relationships. We evaluate the performance of our method on benchmark datasets and demonstrate its effectiveness for cross-modal RS ship image retrieval.https://github.com/sunyuxi/FSISR.

Hyperspectral unmixing is a crucial technique in remote sensing data processing that aims to estimate component information from mixed pixels in hyperspectral images. Most existing deep learning-based hyperspectral unmixing models employ autoencoder (AE) networks to reconstruct hyperspectral images and estimate abundance maps. Here, the weight between the reconstructed and the softmax layers is used to extract/estimate endmember signatures. However, AEs are heavily dependent on initial weights, which introduces inherent randomness, potentially compromising unmixing accuracy. To address this issue, in this article, we present a new dual-feature fusion network (DFFN) for enhanced hyperspectral unmixing. Our DFFN mainly consists of four modules: 1) a feature fusion module (FFM); 2) an abundance estimation module (AEM); 3) an endmember estimation module (EEM); and 4) a reconstruction module (RM). First, FFM calculates spectral and spatial similarities and then enhances the hyperspectral image by matrix multiplications with similarity matrices. Second, AEM takes the enhanced hyperspectral image as input and uses convolutional layers to estimate abundances and reconstruct the image. Next, the reconstructed image is fed into EEM to automatically estimate endmembers. RE performs the final reconstruction through matrix multiplication of the estimated endmembers and abundances. Experiments on synthetic and real hyperspectral datasets, together with a comparison with state-of-the-art techniques, demonstrate the superiority of our newly proposed DFFN. The full code is released athttps://github.com/xuanwentaofor public evaluation.

Hyperspectral unmixing is a vibrant research field that focuses on the task of decomposing mixed pixels into a collection of pure spectral signatures, known as endmembers, along with their corresponding fractional abundances. Conventional unmixing algorithms often need to combine two techniques, namely endmember extraction and abundance estimation, to accomplish the unmixing task. Recently, deep learning (DL) has succeeded in the field of hyperspectral unmixing due to its strong feature learning and data-fitting capabilities. By extracting the output and weight of a particular layer as abundance maps and endmember signatures, available DL methods can directly unmix hyperspectral images. However, in order to improve the performance of spectral unmixing, such available DL methods frequently employ the results of endmember extraction algorithms –in most cases, the well-known vertex component analysis (VCA)– as the initial weights, which leads to significant limitations in their performance: a) the unmixing results are heavily dependent on the initialization given by VCA, and b) the randomness of VCA is passed to the unmixing network. In this paper, we design a new method called abundance-guided spectral and spatial network (A2SN) which not only skips the weights to extract endmember features directly from the network, but also estimates the abundance maps and reconstructs images directly. In particular, the proposed A2SN employs different kernels to capture spectral and spatial information. We also propose an abundance-guided attention spectral and spatial attention network (A2SAN) for hyperspectral unmixing by integrating attention mechanisms into A2SN. As a result, A2SAN is a completely innovative unmixing method that employs attention and reconstruction directly for hyperspectral unmixing, rather than just as modules for information extraction. Most importantly, both A2SN and A2SAN use a weighted summation of the feature maps to reconstruct the image and increase the noise immunity of the network. Experimental results, conducted on both synthetic and real datasets, demonstrate the effectiveness and superiority of A2SN and A2SAN over state-of-the-art unmixing methods. Our full code is released at https://github.com/xuanwentao/A2SN-and-A2SAN for public evaluation.

Constrained by the performance of edge devices and real time (RT) processing technology, the existing hyperspectral target detection algorithms often struggle to rapidly distinguish targets from complex background pixels during real-time detection. To address this issue, this article proposes a new real-time cloud-edge selective background energy constrained (CE-SBEC) hyperspectral target detection algorithm. This algorithm aims to obtain detection results in real-time after capturing new data. Moreover, it conducts in-depth analysis based on existing detection results and updates the algorithm’s internal data to enhance its capabilities in terms of global background annihilation (GBA) and complex background suppression (CBS). Consequently, it improves the accuracy of subsequent real-time detection results. To enhance the resource utilization, this article deploys various task nodes of the algorithm separately on both the cloud and the edge, enabling collaborative execution of the CE-SBEC algorithm. In our context, edge devices are airborne equipment designed for the rapid acquisition and processing of data at the site of data collection, while cloud computing devices refer to high-performance computing clusters situated at a significant distance from the data collection site. Experimental results demonstrate that compared with existing detection algorithms, our newly proposed method achieves more accurate detection results while ensuring real-time performance.

Hyperspectral anomaly detection (HAD) aims to identify anomalous objects that deviate from surrounding backgrounds in an unlabeled hyperspectral image (HSI). Most available neural networks that make use of the reconstruction error to perform HAD tend to fit both backgrounds and anomalies, resulting in small reconstruction errors for both and not being effective in separating targets from background. To address this issue, we develop DirectNet, a new background reconstruction network for HAD that seamlessly integrates a sliding dual-window model into a blind-block architecture. Concretely, DirectNet establishes an inner window within the network’s receptive field by erasing the center block information, so that the content of the inner window remains invisible during the reconstruction of the central pixel. Additionally, the depth of our reconstruction network is adaptive to the size of the input image patch, ensuring that the network’s receptive field aligns with the dimensions of the input patch. The receptive field outside the inner window is considered an outer window. This weakens the impact of anomalies on the reconstruction process, causing the reconstructed pixels to converge towards the background distribution in the outer window region. Consequently, the reconstructed HSI can be regarded as a pure background HSI, leading to further amplification of reconstruction errors for anomalous targets. This enhancement improves the discriminatory ability of DirectNet. Specifically, DirectNet solely utilizes the outer window information to predict/reconstruct the central pixel. As a result, when reconstructing pixels inside anomalous targets of different sizes, the targets primarily fall within the inner window. Comprehensive experiments (conducted on four datasets) demonstrate that DirectNet achieves competitive performance compared to other state-of-the-art detectors.

Hyperspectral anomaly detection, which aims to search and localize potential targets, is a research area with extensive application prospects and profound implications. In recent years, the emergence of unsupervised and self-supervised deep learning for image reconstruction has provided inspiring solutions for hyperspectral anomaly detection. However, due to sensor-induced and environmental effects, the full-image detection networks inevitably reconstruct anomalies along with the background. Existing detectors indirectly mitigate anomaly reconstruction by imposing constraints on hidden features or loss functions, but they provide unsatisfactory performance in large target detection scenarios. This work straightforwardly addresses this issue from the input source, i.e., introducing the concept of masked autoencoders (MAEs) into fully convolutional networks and further developing a sparse-inspired masked convolutional network (SMCNet) consisting of three mutually supportive components: 1) a hierarchical encoder; 2) a sparse projection layer; and 3) a hierarchical decoder. The encoder employs an adaptive potential anomaly masking strategy, leveraging sparse convolution for extracting multidimensional features of the remaining background. Meanwhile, a sparse-guided projection layer is created by discarding the positional embedding technique to populate the uncoded region and guide the background recovery without introducing anomalies. Finally, the decoder couples the hierarchical structure and a hybrid attention mechanism (local-middle–global and spatial-spectral) to refine the background during image recovery, whereas anomalies in the residual map are highlighted. Extensive experiments using ten typical competitors on six different types of datasets validate the effectiveness and generalization ability of the newly proposed SMCNet method.

Remote sensing building change detection (RSBCD) detects changes in the spatial distribution of buildings which is of great significance for urban planning and construction. Existing deep learning-based RSBCD methods usually suffer from low object completeness and erroneous detection problem, mainly due to insufficient utilization of difference information between bi-temporal images. To address the above issues, this article proposed a new Siamese ensemble difference attention network (SEDANet) for RSBCD tasks in very-high resolution (VHR) images. Firstly, the key module ensemble difference attention module (EDAM) is designed to effectively extract difference representation between the bi-temporal features and filter out irrelevant changes. EDAM calculates difference map of bi-temporal features and transforms the extracted change information into trainable difference attention weights. The output weights from EDAM works as a guidance for both spatial and channel visual attention process, which enables the network to focus on foreground building changes and further resolve erroneous attention problems in existing RSBCD methods. The Siamese structure is adopted to better represent bi-temporal features, and convolutional blocks are replaced with residual convolution blocks (RCBs) to speed up network fitting and prevent gradient explosion or descent. We conduct comprehensive experiments on three benchmark datasets. Both visual and quantitative results show that our proposed SEDANet is superior to other eight state-of-the-art networks. Especially on GZ-CD dataset, SEDANet outperforms other comparison methods by 3%-8%. In addition, the effectiveness of EDAM module is also discussed through a series of ablation studies.

The mining of meaningful features and effective fusion of multisource remote sensing (RS) data have always been the challenging research problems in the joint classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) data. In this paper, we propose a Multi-Frequency Graph Convolutional Network with Cross-modality Mutual Enhancement (MFGCN-CME) for multisource RS data classification. Specifically, we design an adaptive multi-frequency graph feature learning module to capture the low- and high-frequency multiscale features of HSI and LiDAR in parallel and further adaptively aggregate them. Then, we propose a bipartite graph enhancement learning module to obtain the spatial-enhanced HSI features and spectral-enhanced LiDAR features by propagating inter-modality information. To the best of our knowledge, the bipartite graph is first used to multisource RS data classification task. Furthermore, compared with traditional fusion methods, a gated fusion module is used to fully explore the complementarity of two data sources. Finally, a joint loss function combing a classification loss and a semi-supervised contrastive loss is developed to improve the model robustness. Comprehensive experiments on different HSI and LiDAR datasets demonstrate that our proposed method can yield better performance compared with several state-of-the-art multisource RS data classification methods.

Supervised hyperspectral image classification suffers from the overfitting problem when limited labels are available. Graph-based semisupervised classifiers can tackle this problem by building connections between labeled and unlabeled samples. In this work, we prove the following two propositions for an optimal graph: 1) the interclass connection weights must be 0 and 2) for a given class, a subset must contain labeled samples or be connected to the remaining subset. In a semisupervised scenario, it is very difficult to ensure that the aforementioned propositions hold. Here, we introduce a new discriminant superpixel graph (dSPG) to build a suboptimal graph, which combines a newly proposed within-superpixel graph, aimed at disconnecting pixels belonging to different classes in a superpixel (so as to decrease interclass connection weights) and a between-superpixel graph that connects spectral adjacent superpixels (to increase the intraclass subset connections). We further propose a dSPG regularizer for hyperspectral image classification and a dSPG-guided graph convolutional network (dSPGCN) to extract discriminant features. Experimental results on real hyperspectral datasets demonstrate the good performance of our newly proposed dSPG for semisupervised hyperspectral image classification. The source codes for this study are available athttps://github.com/yulong112/dSPG.

Hyperspectral imaging enables advanced change detection but struggles with extensive redundant data across spatial and spectral dimensions. This bloats model size and computational loads. To address this problem, we propose a new gated spectral–spatial–temporal attention network with spectral similarity filtering (HyGSTAN) with a lightweight yet accurate architectural design. Specifically, our HyGSTAN introduces three innovative modules: 1) spectral similarity filtering to reduce spectral redundancy via cosine similarity; 2) gated spectral-spatial attention to capture intra-image spatial features using single-head weak self-attention and gated mechanisms; and 3) gated spectral–spatial–temporal attention to extract inter-image temporal changes. Experiments on three benchmark datasets demonstrate HyGSTAN’s ability to balance accuracy, model complexity, and computational efficiency. The proposed attention mechanisms extract more discriminative information without sacrificing performance. The source code of this work will be released at https://github.com/Welcome-to-LISA/HyGSTAN.

Recently, Vision Transformer (ViT)-based deep learning models have achieved remarkable performance gains in hyperspectral image classification (HSIC) due to their abilities to model long-range dependencies and extract global spatial features. However, ViT is built with a stack of Transformer blocks and faces the challenge of learning a large number of parameters when processing hyperspectral data. Besides, the inherent modeling of global correlation in Transformer ignores the effective representation of local spatial and spectral features. To address these issues, we propose a lightweight ViT network known as Groupwise Separable Convolutional Vision Transformer (GSC-ViT). Firstly, a Groupwise Separable Convolution (GSC) module, which is a combination of grouped pointwise convolution and group convolution, is designed to significantly decrease the number of convolutional kernel parameters, and effectively capture local spectral-spatial information in hyperspectral image. Secondly, a Groupwise Separable Multi-Head Self-Attention (GSSA) module is employed to substitute the conventional Multi-Head Self-Attention (MSA) in ViT, in which the Groupwise Self-Attention(GSA) provides local spatial feature extraction, and the Pointwise Self-Attention(PWSA) provides global spatial feature extraction. Thirdly, a simple pointwise layer with enhanced skip connection mechanism is employed to substitute the Multi-Layer Perceptron (MLP) layer in all Transformer blocks of ViT, so as to eliminate unnecessary nonlinear transformations and facilitate the fusion of features derived from GSC and GSSA modules. Extensive experiments on four benchmark hyperspectral datasets reveal that our GSC-ViT can achieve surprising classification performance with relatively few training samples as compared with some existing HSIC approaches. The source code is available at https://github.com/flyzzie/TGRS-GSC-VIT.

Hyperspectral image (HSI) super-resolution, which aims at improving the spatial quality of HSIs by fusing a low spatial resolution HSI (LR-HSI) with a high spatial resolution multispectral image (HR-MSI), has drawn significant attention. Numerous LR-HSI and HR-MSI (HSI-MSI) fusion algorithms have emerged in recent times, yet they suffer from a lack of generality and integration, which hampers their usability for non-expert users. Moreover, these algorithms encounter significant challenges due to the exponential increase in remote sensing data volume. In this study, we propose a unified cloud-based framework for HSI-MSI fusion based on the general distributed alternating direction method of multipliers that incorporates nonlocal principles and tensor decomposition. The framework not only provides end-users with visualization modeling capabilities equipped with standard and comprehensive components, but also enhances the parallel processing capabilities of cloud computing. We employ a new proposed nonlocal adaptive low-rank coupled tensor canonical polyadic (CP) decomposition algorithm as a case study to evaluate the performance of this framework. Specifically, we establish the LR-HSIs and HR-MSIs relationship using order-4 coupled tensor CP decomposition and suggest an adaptive CP rank estimation method for achieving better super-resolution results. Experimental results on publicly available datasets demonstrate that the proposed parallel distributed optimization algorithm can achieve significant speedup with guaranteed accuracy. The proposed framework enables convenient and efficient processing of large-scale remote sensing data, effectively addressing the challenges associated with handling large data volumes. The source code of our method is released and available online at https://github.com/ZpWaitingForSunshine/DNAC4TCP/.

Hyperspectral videos (HSVs) play an important role in the monitoring domain, as they can provide more information than RGB videos about the movement of interesting objects from the perspective of material interpretation. However, the acquisition of HSV data is expensive and time-consuming, whereas RGB videos are readily available. In order to obtain HSV data from its corresponding RGB data, this paper proposes a lightweight frequency-spectrum unfolding network (FSUF-Net) for spectral super-resolution (SSR) of RGB video data. Specifically, the proposed FSUF-Net method belongs to a data-knowledge-driven joint paradigm, which is an interpretable SSR model instead of an end-to-end black-box architecture. The FSUF-Net consists of five main steps. First, the conversion representation of RGB video data to HSV data is derived into an initial recovery term, a data term, and a prior term according to a variable splitting method. Second, the spectral response function between hyperspectral images (HSIs) and RGB images is utilized to achieve the initial recovery term. Third, a convolutional neural network (CNN)-based frequency-domain subnetwork (called F-Net) is designed to solve the data subproblem for recovering the spatial detail information from the HSI, and a Transformer-based spectrum-domain subnetwork (called S-Net) is developed to solve the prior subproblem for reconstructing the spectral information of the HSI. Fourth, two network modules are employed to conduct parametric self-learning. Finally, the HSV data can be obtained in a fixed number of iterations, including alternately solving the above data subproblem and the prior subproblem. Experiments performed on several real datasets demonstrated that the FSUF-Net can effectively reconstruct HSV from RGB videos as compared to traditional and state-of-the-art SSR methods. The proposed method is available online: https://github.com/chengle-zhou/HSV-SSR_FSUF-Net.

Most existing techniques consider hyperspectral anomaly detection (HAD) as background modeling and anomaly search problems in the spatial domain. In this article, we model the background in the frequency domain and treat anomaly detection as a frequency-domain analysis problem. We illustrate that spikes in the amplitude spectrum correspond to the background, and a Gaussian low-pass filter performing on the amplitude spectrum is equivalent to an anomaly detector. The initial anomaly detection map is obtained by the reconstruction with the filtered amplitude and the raw phase spectrum. To further suppress the nonanomaly high-frequency detailed information, we illustrate that the phase spectrum is critical information to perceive the spatial saliency of anomalies. The saliency-aware map obtained by phase-only reconstruction (POR) is used to enhance the initial anomaly map, which realizes a significant improvement in background suppression. In addition to the standard Fourier transform (FT), we adopt the quaternion FT (QFT) for conducting multiscale and multifeature processing in a parallel way, to obtain the frequency domain representation of the hyperspectral images (HSIs). This helps with robust detection performance. Experimental results on four real HSIs validate the remarkable detection performance and excellent time efficiency of our proposed approach when compared to some state-of-the-art anomaly detection methods.

Anomaly detection is a fundamental task in hyperspectral image (HSI) processing. However, most existing methods rely on pixel feature vectors and overlook the relational structure information between pixels, limiting the detection performance. In this article, we propose a novel approach to hyperspectral anomaly detection that characterizes the HSI data using a vertex- and edge-weighted graph with the pixels as vertices. The constructed graph encodes rich structural information in an affinity matrix. A crucial innovation of our method is the ability to obtain internal relations between pixels at multiple topological scales by processing different powers of the affinity matrix. This power processing is viewed as a graph evolution, which enables anomaly detection using vertex extraction formulated as a quadratic programming problem on graphs of varying topological scales. We also design a hierarchical guided filtering architecture to fuse multiscale detection results derived from graph evolution, which significantly reduces the false alarm rate. Our approach effectively characterizes the topological properties of HSIs, leveraging the structural information between pixels to improve anomaly detection accuracy. Experimental results on four real HSIs demonstrate the superior detection performance of our proposed approach compared to some state-of-the-art hyperspectral anomaly detection methods.

Hyperspectral target detection (HTD) aims to detect fine targets in hyperspectral images (HSIs). The traditional HTD method in low-resolution hyperspectral image (LR-HSI) is incapable of detecting small targets, clearly and precisely. Accordingly, in this paper, we propose a hyperspectral and multispectral image fusion target detection method based on cloud-edge collaboration. In this method, LR-HSI is first employed for coarse detection with the output of some suspicious target areas. Afterwards, the hyperspectral images and multispectral images (HSI-MSI) fusion is performed on these areas for precise target detection. In order to ensure the efficiency of HTD, we intend to accelerate our method in parallel based on the cloud-edge collaborative architecture. Furthermore, we establish an optimization model and design a greedy strategy to achieve the optimal deployment for minimizing the shortest runtime on the cloud-edge collaborative architecture. The experimental results demonstrate that our proposed method can significantly improve the computational efficiency while ensuring the accuracy.

With the recent development of remote sensing technology, large image repositories have been collected. In order to retrieve the desired images of massive remote sensing data sets effectively and efficiently, we propose a novel central cohesion gradual hashing (CCGH) mechanism for remote sensing image retrieval. First, we design a deep hashing model based on ResNet-18 which has a shallow architecture and extracts features of remote sensing imagery effectively and efficiently. Then, we propose a new training model by minimizing a central cohesion loss which guarantees that remote-sensing hash codes are as close to their hash code centers as possible. We also adopt a quantization loss which promotes that outputs are binary values. The combination of both loss functions produces highly discriminative hash codes. Finally, a gradual sign-like function is used to reduce quantization errors. By means of the aforementioned developments, our CCGH achieves state-of-the-art accuracy in the task of remote sensing image retrieval. Extensive experiments are conducted on two public remote sensing image data sets. The obtained results support the fact that our newly developed CCGH is competitive with other existing deep hashing methods.

Multispectral remote sensing is very effective for the detection of different types of land cover. Satisfactory results have been obtained in the task of monitoring different types of vegetation covers using satellite images, including areas conquered by invasive aquatic plants such as water hyacinth. Several countries around the world are suffering from negative effects caused by the quick spread of these plants. For instance, several strategies have been carried out for their control and elimination in the Guadiana river, Spain. In this letter, we develop a new methodology capable of automatically finding the geo-location of the most frequent areas of accumulation of invasive aquatic plants in the Guadiana river. Our strategy exploits multispectral time series acquired by ESA’s Sentinel-2 satellite. Once the invasive plants have been detected using deep learning (a convolutional neural network), a subsequent analysis is carried out using geographic information systems to map the areas where water hyacinth patches are most frequently found. In this way, we demonstrate that the management of invasive aquatic plants in the Guadiana river can be successfully carried out.

Hyperspectral image super-resolution aims to fuse a low spatial resolution hyperspectral image (LR-HSI) and a high spatial resolution multispectral image (HR-MSI) to obtain a high-resolution hyperspectral image (HR-HSI). Tensor-based methods have demonstrated their outstanding ability in constructing the relationship between the LR-HSI and the HR-MSI. This paper introduces a nonlocal hierarchical Tucker decomposition (HTD) model for hyperspectral and multispectral image (HSI-MSI) fusion. First, similar nonlocal patch tensors are clustered according to their similarity in the HR-MSI. Next, the spatial/spectral relationship between the LR-HSI and the HR-MSI is extracted through HTD. The alternating direction method of multipliers (ADMM) is employed to solve the proposed model. Furthermore, to overcome the high computational complexity of the model solver, we propose an efficient distributed and parallel method to accelerate the fusion process. Experimental results demonstrate that the proposed method not only substantially outperforms state-of-the-art HSI-MSI fusion methods, but also achieves a significant acceleration rate.

Spatial-spectral classification (SSC) has become a trend for hyperspectral image (HSI) classification. However, most SSC methods mainly consider local information, so that some correlations may not be effectively discovered when they appear in regions that are not contiguous. Although many SSC methods can acquire spatial-contextual characteristics via spatial filtering, they lack the ability to consider correlations in non-Euclidean spaces. To address the aforementioned issues, we develop a new semisupervised HSI classification approach based on normalized spectral clustering with kernel-based learning (NSCKL), which can aggregate local-to-global correlations to achieve a distinguishable embedding to improve HSI classification performance. In this work, we propose a normalized spectral clustering (NSC) scheme that can learn new features under a manifold assumption. Specifically, we first design a kernel-based iterative filter (KIF) to establish vertices of the undirected graph, aiming to assign initial connections to the nodes associated with pixels. The NSC first gathers local correlations in the Euclidean space and then captures global correlations in the manifold. Even though homogeneous pixels are distributed in noncontiguous regions, our NSC can still aggregate correlations to generate new (clustered) features. Finally, the clustered features and a kernel-based extreme learning machine (KELM) are employed to achieve the semisupervised classification. The effectiveness of our NSCKL is evaluated by using several HSIs. When compared with other state-of-the-art (SOTA) classification approaches, our newly proposed NSCKL demonstrates very competitive performance. The codes will be available at https://github.com/yuanchaosu/TCYB-nsckl.

Subspace learning has been widely applied for feature extraction of hyperspectral images (HSIs) and achieved great success. However, the current methods still leave two problems that need to be further investigated. First, those methods mainly focus on finding one or multiple projection matrices for mapping the high-dimensional data into a low-dimensional subspace, which can only capture the information from each direction of high-order hyperspectral data separately. Second, the performance of feature extraction is barely satisfactory when the hyperspectral data is severely corrupted by noise. To address these issues, this article presents a t-linear tensor subspace learning (tLTSL) model for robust feature extraction of HSIs based on t-product projection. In the model, t-product projection is a new defined tensor transformation way similar to linear transformation in vector space, which can maximally capture the intrinsic structure of tensor data. The integrated tensor low-rank and sparse decomposition can effectively remove the noise corruption and the learned t-product projection can directly transform the high-order hyperspectral data into a subspace with information from all modes comprehensively considered. Moreover, a proposition related to tensor rank is proofed for interpreting the meaning of the tLTSL model. Extensive experiments are conducted on two different kinds of noise (i.e., simulated and real noise) corrupted HSI data, which validate the effectiveness of tLTSL.

Recent years have witnessed the flourishing of deep learning-based methods in hyperspectral anomaly detection (HAD). However, the lack of available supervision information persists throughout. In addition, existing unsupervised learning/semisupervised learning methods to detect anomalies utilizing reconstruction errors not only generate backgrounds but also reconstruct anomalies to some extent, complicating the identification of anomalies in the original hyperspectral image (HSI). In order to train a network able to reconstruct only background pixels (instead of anomalous pixels), in this article, we propose a new blind-spot self-supervised learning network (called BS3LNet) that generates training patch pairs with blind spots from a single HSI and trains the network in self-supervised fashion. The BS3LNet tends to generate high reconstruction errors for anomalous pixels and low reconstruction errors for background pixels due to the fact that it adopts a blind-spot architecture, i.e., the receptive field of each pixel excludes the pixel itself and the network reconstructs each pixel using its neighbors. The above characterization suits the HAD task well, considering the fact that spectral signatures of anomalous targets are significantly different from those of neighboring pixels. Our network can be considered a superb background generator, which effectively enhances the semantic feature representation of the background distribution and weakens the feature expression for anomalies. Meanwhile, the differences between the original HSI and the background reconstructed by our network are used to measure the degree of the anomaly of each pixel so that anomalous pixels can be effectively separated from the background. Extensive experiments on two synthetic and three real datasets reveal that our BS3LNet is competitive with regard to other state-of-the-art approaches.

Image stitching aims to mosaic individual images together to build a broader panorama, wherein image registration is the most critical step. In this step, all subimages are normally brought into alignment by warping functions, each of which is often parameterized by rotation, scale, and translation factors. Existing methods usually have to estimate such parameters simultaneously, which incur the problems of unstable and time-consuming computations, especially for stitching large number of unmanned aerial vehicle (UAV) images. In this article, a novel stitching method using vector shape preserving (VSP)-based warping is proposed, which is especially suitable for stitching many UAV images. We innovatively construct warping vectors and involve them to measure registration error to preserve vector shapes, which allows separately dealing with a re-plane stage and a translation stage. Additionally, a novel scale regularization processing connected to the warping is designed for tractable computation. Our method is able to achieve low computational cost, high alignment accuracy, and meanwhile valid real-world interpretation. Experimental results on four real-world UAV image datasets validate the excellent performance of our method.

Hyperspectral (HS) pansharpening seeks to integrate low spatial resolution HS (LRHS) images with connected panchromatic (PAN) images to produce high spatial resolution HS (HRHS) images. Traditional pansharpening convolutional neural networks (CNNs) directly map LRHS and PAN images into HRHS images under fixed network parameters, which imply static pansharpening rules. However, real-world HS data are often characterized by spatial variations, and intuitively, the pansharpening rules should be dynamic. To deal with the dilemma, in this article, we develop dynamic HS pansharpening CNNs. We first specify the concepts of dynamic pansharpening and static pansharpening. Then, we propose a learn-to-learn-oriented pansharpening CNN paradigm, which aims to learn a how-to-learn rule to produce spatially adaptive pansharpening rules and comprises three stages of preliminary fusion, scene-sensitive modulation, and spectral reconstruction. Finally, following the paradigm, we design two groups of dynamic pansharpening CNNs (DyPNNs), i.e., internal-connection-based and external-connection-based. They involve various spatial modulations, including spatial affine transform (AT), spatial dynamic convolution (DC), or improved spatial attention (SA), and, thus, consist of six specific DyPNNs: IC-AT-DyPNN, IC-DC-DyPNN, IC-SA-DyPNN, EC-AT-DyPNN, EC-DC-DyPNN, and EC-SA-DyPNN. Experimental results on several HS datasets verify the effectiveness of the proposed DyPNNs in terms of both the spatial reconstruction and spectral fidelity.

Deep learning models, such as convolutional neural networks (CNNs), have made significant progress in hyperspectral image (HSI) classification. However, these models require a large number of parameters, which occupy a lot of storage space and suffer from overfitting, thus resulting in performance loss. To solve the above problems, in this article, we propose a new compression network [namely, a Hybrid Fully Connected Tensorized Compression Network (HybridFCTCN)] by considering the high dimensionality of HSI data. First, using the low-rank fully connected tensor network decomposition (FCTND), three novel units, i.e., FCTN-FC, FCTNConv2D, and FCTNConv3D, are designed to compress the weight tensor of standard fully connected (FC) layer and kernel tensor of convolutional layer, reducing their parameters. In the novel units, the intrinsic correlation of the decomposed factors is adequately exploited by the FC structures, which enhances their feature extraction and classification abilities. Then, benefiting from the hybrid network backbone composed of the FCTNConv3D and FCTNConv2D units, HybridFCTCN can extract more discriminative features with fewer parameters, while it has great generalization capability and robustness, enabling better HSI classification. Finally, the rank of above-designed units is defined, and its determination is discussed to facilitate the application of the proposed model. Extensive experiments on three widely used HSI datasets reveal that the proposed model achieves state-of-the-art classification performance for different training sample sizes with a very small number of parameters.

As the spectral library continues to expand, sparse hyperspectral unmixing methods have been developed to solve the mixing problem without the need for end-member extraction or generation. These methods leverage the intrinsic spectral and spatial information to enhance the accuracy of fractional abundance estimation. However, their effectiveness is limited by rigid spatial regularization and insufficient utilization of spectral spatial information, which hampers the improvement of unmixing performance. To overcome this limitation, we present a novel algorithm named Local Spectral Similarity Guided Sparse Hyperspectral Unmixing with Spatial Graph Regularization (SGSU). In SGSU, we introduce a spatial graph regularization to enforce the inter-pixel correlation within spatial clusters and assign them to corresponding abundance vectors. To reduce the computational cost, we employ an adaptive superpixel-based spatial grouping strategy to segment the hyperspectral image, which translates the intrinsic geometry into constraints on abundance. Furthermore, we introduce a weighting factor with two components into the sparse unmixing framework. One component is based on the row sparsity of the estimated abundances, indicating the presence of active end-members; the other component is based on the similarity between neighboring pixels, which promotes piecewise smoothness of the estimated abundances. To obtain a more robust solution, we adopt a double-loop scheme based on the alternating direction method of multipliers (ADMM) algorithm to solve the SGSU model. Experimental results on both simulated and real hyperspectral datasets demonstrate that the proposed SGSU algorithm outperforms state-of-the-art sparse unmixing methods in terms of both accuracy of abundance estimation and end-member identification from spectral libraries. Our algorithm achieves superior unmixing results, which indicates its potential for practical applications in hyperspectral imaging.

Convolutional neural networks (CNNs) have been widely employed for hyperspectral image (HSI) classification due to their powerful ability to extract local spatial features. However, CNN-based methods cannot establish long-range dependencies among sequences of pixels. Transformers offer significant advantages when processing sequential data and can establish global relationships, but they still encounter a number of challenges, such as their limited spatial feature extraction ability, or their high computational cost. In order to address the aforementioned issues, we develop a new fast HSI classification approach combining transformers and SimAM-based CNNs. The latter are utilized to extract better spatial features, where the complex spatial characteristics of HSIs are retrieved using an improved hierarchical 2D dense network structure. A dual attention unit (DAU) mechanism is then utilized to direct the model’s attention to discriminative spatial pixel characteristics and effective feature map channels, while suppressing information that is irrelevant for classification purposes. Regarding the spectral features, after extracting hierarchical local characteristics from various convolutional layers (using the hierarchical dense network structure), a squeezed-enhanced axial transformer is employed to establish global long-range dependencies whilst enhancing the ability of the model to extract local detail features in the HSI. Besides, a new Lion optimizer is utilized to improve the classification performance of our model. Our quantitative and comparative experiments on four benchmark datasets demonstrate the effectiveness of the proposed approach provides better classification results than other state-of-the-art approaches. Moreover, our FTSCN also achieves better classification results than other methods in practical scenarios.

Deep learning (DL) techniques have shown remarkable progress in remotely sensed hyperspectral image (HSI) classification tasks. The performance of DL-based models highly relies on the quality and quantity of labeled data. However, manual labeling is a laborious and expensive process that requires substantial efforts from human experts. Active learning (AL) techniques have been developed to alleviate the burden of manual annotation by selecting the most informative and uncertain samples for labeling. In this paper, we propose a new class-wise graph embedding-based AL (CGE-AL) framework implemented by a class-wise graph convolutional network (CGCN). First, we train a classifier with labeled data and infer latent features from labeled and unlabeled samples with the trained parameter. Then, we group the labeled data into multiple one-label sets by category. In a class-wise manner, we initialize the nodes of the graph with one-label and unlabeled features, which are then fed into CGCN. By updating the graph parameters with binary loss, CGCNs measure the uncertainty between labeled nodes and unlabeled nodes. To select the most valuable sample for labeling, we adopt the class minimum uncertainty to query the unlabeled nodes with higher overall uncertainty. We repeat this process with the updated labeled set to retrain our classification model and CGCNs. Extensive experiments demonstrate the outstanding performance of our method compared to other state-of-the-art AL-based approaches.

In remote sensing image fusion, pansharpening is a type of remote sensing image fusion method that aims to fuse panchromatic (PAN) images and multispectral (MS) images to produce high-resolution multispectral (HRMS) images. Deep learning based pansharpening technology offers a series of advanced unsupervised algorithms. However, there are several challenges: (1) The existing unsupervised pansharpening methods only consider the fusion of single-scale features; (2) for the fusion of MS and PAN image feature branches, the existing pansharpening methods are implemented directly by concatenation and summation, without paying attention to critical features or suppressing redundant features; (3) the semantic gap in the long skip connections of the network architecture will create unexpected results. In this paper, we design a multiscale unsupervised architecture based on generative adversarial networks (GANs) for remote sensing image pansharpening (Mun-GAN), which consists of a generator and two discriminators. The generator includes a multi-scale feature extractor (MFE), a self-adaptation weighted fusion (SWF) module, and a nest feature aggregation (NFA) module. First, the MFE is utilized to extract multiscale feature information from the input images and to then pass this information to the SWF module for adaptive weight fusion. Then, multiscale features are reconstructed by the NFA module to obtain HRMS images. The two discriminators are spectral and spatial discriminators used against the generator. Moreover, we design a hybrid loss function to aggregate the multiscale spectral and spatial feature information. Compared with other state-of-the-art methods using QuickBird, GaoFen-2 and WorldView-3 images, which demonstrate that the Mun-GAN yields better fusion results.

Dimensionality reduction (DR) is important for feature extraction and classification of hyperspectral images (HSIs). Recently proposed superpixel-based DR models have shown promising performance, where superpixel segmentation techniques were applied to segment an HSI and then DR models like principal component analysis (PCA) or linear discriminant analysis (LDA) were employed to extract the local and/or global features. However, superpixelwise PCA based local features are unsatisfactory because PCA aims to extract features with high variance, which could be inefficient in superpixels with mixed objects or strong noise/outliers. In addition, superpixelwise unsupervised LDA based global features may neglect local (spatial-contextual) information. To address these issues, we propose a new spectral-spatial and superpixelwise unsupervised LDA (S3-ULDA) model for unsupervised feature extraction from HSIs. Specifically, the HSI is first segmented into various superpixels with pseudo labels. Then, superpixel based local reconstruction for HSI denoising is conducted. Next, superpixelwise unsupervised LDA (SuperULDA) is performed on both the original HSI and locally reconstructed data to extract global features. Then, superpixelwise unsupervised local Fisher discriminant analysis (SuperULFDA) is developed for local feature extraction, where each superpixel and its adjacent superpixels (along with their pseudo-labels) are fed into local Fisher discriminant analysis (LFDA) to extract local features. The superpixel-level local manifold structures can be effectively modeled by the proposed SuperULFDA. Finally, by fusing the extracted global and local features, novel global-local and spectral-spatial features can be obtained. Our experimental results on several benchmark HSIs demonstrate the superiority of the proposed method over state-of-the-art methods. The code of the proposed model is available at https://github.com/XinweiJiang/S3-ULDA.

Convolutional models have provided outstanding performance in the analysis of hyperspectral images (HSIs). These architectures are carefully designed to extract intricate information from non-linear features for classification tasks. Notwithstanding their results, model architectures are manually engineered and further optimized for generalized feature extraction. In general terms, deep architectures are time consuming for complex scenarios since they require fine tuning. Neural architecture search (NAS) has emerged as a suitable approach to tackle this shortcoming. In parallel, modern attention-based methods have boosted the recognition of sophisticated features. The search for optimal neural architectures combined with attention procedures motivates the development of this work. This paper develops a new method to automatically design and optimize convolutional neural networks (CNNs) for HSI classification using channel-based attention mechanisms. Specifically, one-dimensional (1D) and spectral-spatial (3D) classifiers are considered to handle the large amount of information contained in HSIs from different perspectives. Furthermore, the proposed AAtt-CNN method meets the requirement to lower the large computational overheads associated with architectural search. It is compared with current state-of-the-art (SOTA) classifiers. Our experiments, conducted using a wide range of HSI images, demonstrate that AAtt-CNN succeeds in finding optimal architectures for classification, leading to SOTA results.

Hyperspectral images (HSIs) comprise plenty of information in the spatial and spectral domain, which is highly beneficial for performing classification tasks in a very accurate way. Recently, attention mechanisms have been widely used in HSI classification due to their ability to extract relevant spatial and spectral features. Notwithstanding their positive results, most of the attentional strategies usually introduce a significant number of parameters to be trained, making the models more complex and increasing the computational load. In this paper, we develop a new parameter-free attention network for HSI classification. The main advantage of our model is that it does not add parameters to the original network (as opposed to other state-of-the-art approaches), whilst providing higher classification accuracies. Extensive experimental validations and quantitative comparisons are conducted –using different benchmark HSIs– to illustrate these advantages. Code is available on https://github.com/mhaut/Free2Resnet.

Vision transformers (ViTs) have been trending in image classification tasks due to their promising performance when compared to convolutional neural networks (CNNs). As a result, many researchers have tried to incorporate ViTs in hyperspectral image (HSI) classification tasks. To achieve satisfactory performance, close to that of CNNs, transformers need fewer parameters. ViTs and other similar transformers use an external classification (CLS) token which is randomly initialized and often fails to generalize well, whereas other sources of multimodal datasets, such as light detection and ranging (LiDAR) offer the potential to improve these models by means of a CLS. In this paper, we introduce a new multimodal fusion transformer (MFT) network which comprises a multihead cross patch attention (mCrossPA) for HSI land-cover classification. Our mCrossPA utilizes other sources of complementary information in addition to the HSI in the transformer encoder to achieve better generalization. The concept of tokenization is used to generate CLS and HSI patch tokens, helping to learn a distinctive representation in a reduced and hierarchical feature space. Extensive experiments are carried out on widely used benchmark datasets i.e., the University of Houston, Trento, University of Southern Mississippi Gulfpark (MUUFL), and Augsburg. We compare the results of the proposed MFT model with other state-of-the-art transformers, classical CNNs, and conventional classifiers models. The superior performance achieved by the proposed model is due to the use of multihead cross patch attention. The source code will be made available publicly at https://github.com/AnkurDeria/MFT.

In recent years, convolutional neural networks (CNNs) have drawn significant attention for the classification of hyperspectral images (HSIs). Due to their self-attention mechanism, the vision transformer (ViT) provides promising classification performance compared to CNNs. Many researchers have incorporated ViT for HSI classification purposes. However, its performance can be further improved because the current version does not use spatial–spectral features. In this article, we present a new morphological transformer (morphFormer) that implements a learnable spectral and spatial morphological network, where spectral and spatial morphological convolution operations are used (in conjunction with the attention mechanism) to improve the interaction between the structural and shape information of the HSI token and theCLStoken. Experiments conducted on widely used HSIs demonstrate the superiority of the proposed morphFormer over the classical CNN models and state-of-the-art transformer models. The source will be made available publicly athttps://github.com/mhaut/morphFormer.

Deep learning (DL) has brought many new trends for hyperspectral image classification (HIC). Graph neural networks (GNNs) are models that fuse DL and structured data. Although GNN-based methods have focused on modeling relations, most of them are susceptible to noise, being adverse to capturing hidden correlations from data. Moreover, existing related approaches typically adopt changeless graph structures, which might lead to poor generalization. To solve the problems mentioned above, this paper develops an adaptive cuckoo refinement-based graph transfer network (ACGT-Net) that introduces a meta-heuristic optimization strategy to refine the graph structure. Specifically, we first pre-train a graph convolutional network (GCN) to learn transferable weight parameters. In the undirected graph, nodes are associated with pixels, and edges correspond to similarities between nodes. Afterward, we integrate a cuckoo search strategy (CSS) into the trained GCN to adaptively refine the graph structure. The graph structure refinement (GSR) with the CSS can pay more attention to significant channels by global optimization to improve the generalization of the GNN. Several experiments with real datasets verify the effectiveness and competitiveness of our ACGT-Net compared with other state-of-the-art (SOTA) methods.

Cross-view geo-localization is a critical task in various applications, such as smart city management and disaster monitoring. Current methods typically divide a satellite image into patches and use these patches to identify the geographic location of a query image. However, these methods can only provide the location of an image rather than the location of a specific object of interest. This makes it difficult to link these methods to GeoDatabases to obtain detailed information about a target object, such as its name and construction time. To overcome this limitation, we propose a novel problem of cross-view object geo-localization in a local region with high-resolution satellite images. This problem includes two main challenges: accurately identifying the location of an object and distinguishing the target object from others in satellite images. To address these challenges, we present a new Detection-based Geo-localization method called DetGeo, which consists of an object detection-based framework with a two-branch encoder and a query-aware cross-view fusion module. DetGeo uses cross-view images as input to the detector to provide object-level geo-localization. The fusion module employs cross-view spatial attention to focus on relevant areas of target objects during cross-view feature fusion. To evaluate our method, we constructed a new Cross-View Object Geo-Localization dataset called CVOGL, which comprises ground-view or drone-view images as query images and satellite-view images as geo-tagged reference images. Comprehensive experiments are conducted to demonstrate the effectiveness of our method on CVOGL. https://github.com/sunyuxi/DetGeo.

Cross-modal hashing aims to retrieve similar images from large-scale Earth Observation (EO) data archives, which typically contain multiple satellite sources of remote sensing (RS) images. However, existing cross-modal hashing methods primarily focus on dual-source RS images and often face two main limitations when retrieving multi-source RS images. Firstly, these methods exhibit significant redundancy as they require handling all possible dual-source combinations in multi-source RS images. Secondly, they often rely on pairwise or triplet image sources to construct objective functions, which are not significantly effective in reducing the discrepancies among multiple RS image sources. To address these limitations, we propose a novel Consistency Center-based deep cross-modal Hashing method called C2Hash for multi-source RS image retrieval. Our C2Hash employs a multi-branch hashing network to directly encode multi-source RS images into unified hash codes, thereby offering higher processing efficiency. Furthermore, C2Hash introduces consistency centers to construct a novel objective function. The consistency center represents the shared semantic features among similar multi-source RS images and is generated by a label hashing network. The objective function encourages similar multi-source RS images to approach the same consistency center to align all image sources in a unified Hamming space. Our method can effectively reduce the discrepancies across multiple image sources and generate unified hash codes. To evaluate its effectiveness, we construct a new Multi-Source RS Image dataset called MSRSI, comprising five different types of image sources. We conduct comprehensive experiments to demonstrate the superior performance of our method on the MSRSI dataset. https://github.com/sunyuxi/C2Hash.

Deep neural networks play a significant role in hyperspectral image (HSI) processing, yet they can be easily fooled when trained with adversarial samples (generated by adding tiny perturbations to clean samples). These perturbations are invisible to the human eye, but can easily lead to misclassification by the deep learning model. Recent research on defense against adversarial samples in HSI classification has improved the robustness of deep networks by exploiting global contextual information. However, available methods do not distinguish between different classes of contextual information, which makes the global context unreliable and increases the success rate of attacks. To solve this problem, we propose a robust context-aware network able to defend against adversarial samples in HSI classification. The proposed model generates a global contextual representation by aggregating the features learned via dilated convolution, and then explicitly models intraclass and interclass contextual information by constructing a class context-aware learning module (including affinity loss) to further refine the global context. The module helps pixels obtain more reliable long-range dependencies and improves the overall robustness of the model against adversarial attacks. Experiments on several benchmark HSI datasets demonstrate that the proposed method is more robust and exhibits better generalization than other advanced techniques.

Hyperspectral anomaly detection (HAD) aims to identify anomalous targets that deviate from the surrounding background in unlabeled hyperspectral images (HSIs). Most existing deep networks that exploit reconstruction errors to detect anomalies are prone to fit anomalous pixels, thus yielding small reconstruction errors for anomalies, which is not favorable for separating targets from HSIs. In order to achieve a superior background reconstruction network for HAD purposes, this paper proposes a self-supervised blind-block network (termed BockNet) with a guard window. BockNet creates a blind-block (guard window) in the center of the network’s receptive field, rendering it unable to see the information inside the guard window when reconstructing the central pixel. This process seamlessly embeds a sliding dual-window model into our BockNet, in which the inner window is the guard window and the outer window is the receptive field outside the guard window. Naturally, BockNet utilizes only the outer window information to predict/reconstruct the central pixel of the perceptive field. During the reconstruction of pixels inside anomalous targets of varying sizes, the targets typically fall into the guard window, weakening the contribution of anomalies to the reconstruction results so that those reconstructed pixels converge to the background distribution of the outer window area. Accordingly, the reconstructed HSI can be deemed as a pure background HSI, and the reconstruction error of anomalous pixels will be further enlarged, thus improving the discrimination ability of the BockNet model for anomalies. Extensive experiments on four datasets illustrate the competitive and satisfactory performance of our BockNet compared to other state-of-the-art detectors.

Autoencoder-based hyperspectral anomaly detectors have received significant attention. The core of these detectors is to reconstruct backgrounds by optimizing autoencoders so that anomalies can be detected by reconstruction residuals. Nevertheless, existing methods are flawed in two aspects: 1) most of them reconstruct the background along with the anomalies, resulting in undesired performance for large target detection in complex backgrounds; 2) they only focus on the encoder optimization part, ignoring the decoder reconstruction quality of the background. Given the above, this paper proposes a background-guided deformable convolutional autoencoder (DCAE) network with three mutually supportive parts, including encoder, decoder, and background guidance modules. In the encoder, deformable convolution is introduced into regular convolution to build the adaptive spatial feature extractor to fit complex spatial structures, whilst a non-local convolution is introduced to build an external feature extractor to capture global spatial relationships. Further, a mask is designed to filter potential anomalous information, curbing the representation of high-frequency anomalies to focus on widespread backgrounds. In the decoder, a background guidance module (considering the physical meaning of linear reconstruction) is built, guiding the proposed network learning via two strategies. One is initializing the weight of the decoder, and another is adding a loss term. Notably, both the number of output channels of the encoder and the decoder construction are determined by the background guidance module, which creates a bridge between the network design and practical situations. A profound analysis demonstrates the outstanding performance of the proposed method, which outperforms traditional and deep learning methods, proving that the novel designs introduced in the network architecture are extremely effective.

Nowadays, convolutional neural network (CNN)-based deep learning models have been popularized in hyperspectral image classification (HSIC) and achieved significant accuracy gains, which is due to their hierarchical and nonlinear feature learning patterns. However, too deeper network structures may induce a huge amount of parameters and excessive computing overhead, leading to the need for plenty of labeled samples for training. Besides, highly abstract semantic features may not be the most suitable for hyperspectral land-cover classification tasks. To address these issues, we propose a fairly lightweight network model for HSIC, which is built on a type of exquisitely designed convolution module, namelygrouped separable convolution. Compared with the standard convolution, the designed grouped separable convolution module combines grouped convolution with point-wise convolution, which not only greatly reduces the number of parameters of convolution kernels, but also caters to the inherent 3D cube style of hyperspectral image data. Moreover, Gabor filters are introduced to modulate the grouped separable convolution kernels, so as to further use relatively few convolution kernels with additional prior orientation and scale information for feature extraction. The experiments are carried out on four real hyperspectral datasets, and the experimental results reveal that the proposed model has low training cost and memory overhead. Compared with some existing deep network models that have been applied to HSIC, our proposed model can achieve competitive classification accuracy with fewer training samples.

Spatio-temporal fusion is a feasible way to generating the synthetic remote sensing data with high spatial resolution and high temporal resolution simultaneously by blending the fine and coarse resolution satellite images. To date, dozens of spatio-temporal fusion approaches have been developed. A basic rule of these approaches is the bands of coarse and fine images must be corresponding, which means the quality of fused images depends on that of both fine and coarse images. In the literature, the MODIS images are the most wildly used coarse images in spatio-temporal fusion. However, the MODIS images may suffer from serious stripe noises in the short-wave infrared-1 and short-wave infrared-2 bands, which will lead to undesired results of spatio-temporal fusion. To address this problem, we develop a noise proof strategy in this paper, which takes advantage of the spectral correlation of base fine image to remove the stripe noises of the base MODIS image, then the spatial correlation of base MODIS image is exploited to restore the MODIS image of the predicted time. Finally, the reconstructed MODIS images are fused with the base fine image to predict the missing fine images. The strategy is tested via real Landsat and MODIS images, and the experimental result demonstrates it is not only effective in removing the stripe noises of MDOIS short-wave infrared-1 and short-wave infrared-2 bands, but also able to improve the fusion accuracy.

Hyperspectral unmixing aims at extracting pure spectral signatures and estimating their corresponding abundances at each pixel. Traditional unmixing algorithms consider end-member extraction and abundance estimation as two separate steps, and the completion of abundance estimation requires results from other endmember extraction algorithms. Considering that convolutional neural networks (CNNs) have powerful learning and data fitting capabilities, some techniques based on deep learning (DL) have been proposed in the literature. Most of them only utilize spectral information and neglect spatial information. In addition, existing unmixing methods based on DL usually extract the weight and output of a specific activation layer as endmembers and abundances, respectively. In our work, we exploit 3D convolution to propose a new 3D convolution unmixing network (3DCUN) for hyperspectral unmixing. Two types of real data, i.e., Samson and Jasper, are used to evaluate the performance of our proposed 3DCUN in endmember extraction and abundance estimation. The experimental results reflect that our proposed 3DCUN gets accurate results in estimating endmembers and abundances.

The performance of hyperspectral anomaly detection based on representation models is importantly related to the corresponding dictionary. A good dictionary can optimally model background to detect anomalies. To realize adaptively background reconstruction, this paper constructs global-local dictionaries for collaborative representation detector by using adaptive-shape (SA-CRD). Specifically, robust principal component analysis (RPCA) is used to separate background and anomalies preliminarily. Then adaptive-shape neighbor is adopted to build local dictionaries for robust background region, and the robust background region is clustered to construct a global dictionary for potential anomaly region. Finally, global-local dictionaries are used in the collaborative representation model to finish anomaly detection. Obtained results over two real data sets indicate that the proposed method can improve the accuracy of anomaly detection intensively compared to other state-of-art methods.

Deep metric learning methods have recently drawn significant attention in the field of remote sensing (RS), owing to their prominent capabilities for modeling relations among RS images based on their semantic contents. In the context of scene classification and large-scale image retrieval, one of the most prominent deep metric learning methods is the scalable neighborhood component analysis (SNCA), which has demonstrated excellent performance on the locality neighborhood structure in the metric space. However, the standard SNCA has important constraints on separating the hard positive and other negative images in the metric space, and this may become a major limitation when dealing with the large-scale variance problem inherent to RS data. To address this issue, we propose a novel deep metric learning formulation that introduces a new margin parameter to enforce the compactness of the within-class feature embeddings. Based on this innovative scheme, we propose two novel loss functions: 1) T-SNCA-c, where the parameter is based on the cosine similarity, and 2) T-SNCA-a, where the parameter is based on the angular distance. Besides, we exploit memory bank optimization to further enhance the semantic diversity during training. Our experimental results, conducted using three downstream applications ($K$-NN classification, clustering, and image retrieval) and two large-scale RS benchmark datasets, demonstrate that the proposed approach can achieve superior performance when compared to current state-of-the-art deep metric learning methods. The codes of this work will be made available online (https://github.com/jiankang1991/GRSL_TSNCA).

Convolutional neural networks (CNNs) are relevant tools for remote sensing data processing in the last few years. Kernels process and integrate the spatial information of remotely sensed hyperspectral images (HSIs) accurately enough, so as to reduce the noise and spectral variations present in the data. Despite the great efficiency of the CNNs with HSI data, kernels need to be composed of a large number of layers and parameters to be able to obtain good performance, forcing the model to process a large amount of information from the datasets and making the model prone to overfitting, due to the limited number of labels in some cases. On the other hand, with that huge amount of data to consume, a large number of kernels are needed, making the model less efficient due to computational complexity. To overcome these challenges, this letter presents a new technique to reduce the computational cost and increase the model accuracy [based on adaptable rectangular convolutions (ARCs)], leading to a substantial reduction of the number of parameters and improving the model, in order to achieve better results in the context of HSI semantic segmentation. Thus, the model learns through the convolution the dimensions and offsets of the kernel of this Adaptable layer, performing the average operation based on integral image works to achieve better results with fewer parameters, reducing the risk of overfitting and computational cost. The source code can be found in the repository available at the link. [https://github.com/jlgs96/segHSI]

Convolutional neural networks (CNNs) have proven to be very efficient for the analysis of remote sensing (RS) images. Due to the inherent complexity of extracting features from these images, along with the increasing amount of data to be processed (and the diversity of applications), there is a clear tendency to develop and employ increasingly deep and complex CNNs. In this regard, graphics processing units (GPUs) are frequently used to optimize their execution, both for the training and inference stages, optimizing the performance of neural models through their many-core architecture. Hence, the efficient use of the GPU resources should be at the core of optimizations. This letter analyzes the possibilities of using a new family of CNNs, denoted as TResNets, to provide an efficient solution to the RS scene classification problem. Moreover, the considered models have been combined with mixed precision to enhance their training performance. Our experimental results, conducted over three publicly available RS data sets, show that the proposed networks achieve better accuracy and more efficient use of GPU resources than other state-of-the-art networks. Source code is available athttps://github.com/mhaut/GPUfriendlyRS.

Most state-of-the-art deep learning-based methods for extraction of building footprints are aimed at designing proper convolutional neural network (CNN) architectures or loss functions able to effectively predict building masks from remote sensing (RS) images. To properly train such CNN models, large-scale and pixel-level building annotations are required. One common approach to obtain scalable benchmark data sets for the segmentation of buildings is to register RS images with auxiliary geospatial information data, such as those available from OpenStreetMaps (OSM). However, due to land-cover changes, urban construction, and delayed geospatial information updating, some building annotations may be missing in the corresponding ground-truth building mask layers. This will likely introduce confusion in the training of CNN models for discriminating between background and building pixels. To solve this important issue, we first formulate the problem as a long-tailed classification one. Then, we introduce a new joint loss function based on three terms: 1) logit adjusted cross entropy (LACE) loss, aimed at discriminating between building and background pixels from a long-tailed label distribution; 2) weighted dice loss, aimed at increasing the$F_{1}$scores of the predicted building masks; and 3) boundary (BD) alignment loss, which is optimized for preserving the fine-grained structure of building boundaries. Our experiments, conducted on two benchmark building segmentation data sets, validate the effectiveness of our newly proposed loss with respect to other state-of-the-art losses commonly used for extracting building footprints. The codes of this letter will be publicly available fromhttps://github.com/jiankang1991/GRSL_BFE_MA.

Land-cover information is of paramount importance in a wide range of environmental and socioeconomic applications. Deep learning (DL) provides a large variety of potential models for extracting useful information from raw images. However, remote sensing image (RSI) classification remains a challenging goal due to the intrinsic features of the data, such as the high sample variability and lack of labeled data. This provides a challenge to the reliability of deep classifiers. In particular, convolution-based models are greatly affected by overfitting and vanishing gradient problems. To overcome these drawbacks, this letter presents a new attention-based architecture, including attention modular blocks. These blocks divide their input feature maps into several groups and split them along the channel dimension and then combine them to create an attention mask encoding global contextual information. The mask is applied to obtain a refined feature representation, strengthening those features that affect most significantly the classification and attenuating the rest. Our new method reduces significantly the number of trainable parameters. Our results, obtained using several widely used RSIs, demonstrate that the new method exhibits higher classification performance when compared to several state-of-the-art methods.

Endmember estimation consists of two tasks, that is, determining the number of pure spectral constituents (endmembers) and extracting their spectral signatures. We present a new geometric distance-based method for endmember estimation from hyperspectral images (HSIs), which does not need to know the number of endmembers in advance. Our strategy optimizes the widely used maximum distance analysis (MDA) method from two viewpoints. First, the traditional MDA method performs endmember estimation by computing the maximum distances between any pixel and one specific pixel, line, plane, or affine hull (AH) composed by the endmembers that have been formerly extracted. Instead, our new strategy only requires computing the maximum distance between any pixel and one specific AH. This operation provides a simpler way than MDA to estimate endmembers. Second, our strategy exploits a new distance computation between any pixel and an AH and just needs the normal vector (compared to the traditional MDA method, which uses the normal vector and offset). The new distance computation in our method is much more efficient than that in the traditional MDA method.

Image segmentation is a key task in computer vision and image processing with important applications such as scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, and image compression, among others, and numerous segmentation algorithms are found in the literature. Against this backdrop, the broad success of deep learning (DL) has prompted the development of new image segmentation approaches leveraging DL models. We provide a comprehensive review of this recent literature, covering the spectrum of pioneering efforts in semantic and instance segmentation, including convolutional pixel-labeling networks, encoder-decoder architectures, multiscale and pyramid-based approaches, recurrent networks, visual attention models, and generative models in adversarial settings. We investigate the relationships, strengths, and challenges of these DL-based segmentation models, examine the widely used datasets, compare performances, and discuss promising research directions.

Traditional hyperspectral (HS) pansharpening aims at fusing a HS image with its panchromatic (PAN) counterpart, to bring the spatial resolution of the HS image to that of the PAN image. However, in many practical applications, arbitrary resolution HS (ARHS) pansharpening is required, where the HS and PAN images need to be integrated to generate a pansharpened HS image with arbitrary resolution (usually higher than that of the PAN image). Such an innovative task brings forth new challenges for the pansharpening technique, mainly including how to reconstruct HS images beyond the training scale and how to guarantee spectral fidelity at any spatial resolutions. To tackle the challenges, we present a novel convolutional neural network (CNN)-based method for ARHS pansharpening called ARHS-CNN. It is based on a two-step relay optimization process, which is associated with a multilevel enhancement subnetwork and a rescaling subnetwork. With a careful design following the thread, our ARHS-CNN is able to pansharpen HS images to any spatial resolutions using just a single CNN model trained on a limited number of scales while meantime to keep spectral fidelity at those resolutions, which wins an obvious advantage over traditional pansharpening methods. Experimental results obtained on several datasets verify the excellent performance of our ARHS-CNN method.

Sparse unmixing (SU) has been widely applied to remotely sensed hyperspectral images interpretation. Compared with traditional unmixing algorithms, SU does not need to extract pure signatures (endmembers) from the image. The endmember matrix is constructed by directly selecting spectra from a known library which is used to estimate the fractional abundances associated with endmembers. This avoids the problem of extracting virtual endmembers without physical meaning. However, SU does not generally include spatial information, which may limit its performance. In order to address this limitation and include local spatial information, low-rank and sparse features in local regions can be exploited. In this paper, we include spatial information in the traditional SU algorithm by extracting low rank and spatial information based on superpixels, and further propose an algorithm named superpixel-based collaborative sparse and low-rank regularization for sparse unmixing (SCLRSU) to improve the performance of the traditional spatial regularization-based SU methods. In our proposed method, we combine superpixel segmentation and structural sparsity. Experiments are carried out on two simulated datasets and two real hyperspectral image datasets, and our results are compared with those obtained by traditional SU methods. Our results indicate that our newly proposed method provides very competitive performance.

While deep learning algorithms have achieved good results in hyperspectral image (HSI) classification, several supervised classification algorithms rely on a large number of labeled samples to get adequate performance. Collecting a large number of labeled samples is expensive in many real applications. To address this issue, a novel semisupervised HSI classification framework called deep representative feature learning (DRFL) with virtual adversarial training (DRFL-VAT) is developed in this article. By embedding the local manifold learning (LML) into the fully connected layers of a convolutional neural network (CNN), our newly developed DRFL can learn representative features. The VAT regularization is adopted to exploit the prediction label distribution of training samples and addresses the overfitting problem. Finally, the objective function of DRFL-VAT is solved by a customized algorithm. We test our method on three widely public HSI datasets and our results show that our method is competitive when compared to other state-of-the-art approaches.

Advanced remote-sensing instruments produce massively large scenes from the surface of the earth, with very high spatial resolution and dimensionality. Developing methods for efficiently localizing specific objects in a large-scale scene presents a significant challenge, mainly because of the high computational requirements involved. To tackle this issue, we propose a new hashing for localization (HfL) framework that efficiently searches for specific objects in the large-scale scene. It begins by dividing the scene into a large number of overlapping local patches. A lightweight deep hash model, referred to as a tiny hashing network (THNet), encodes the local patches into hash codes. The Hamming distances between the hash code of an object image, i.e., an image containing the specific class of objects to be localized in the scene, and those of all local patches are computed. Small values of the Hamming distance indicate local patches that are similar to the object image. The positions of these local patches in the large-scale scene reflect the regional locations of the specific objects. The hash codes are binary and do not take up much space, and the Hamming distance carries very low-computational overheads. Further, we exploit a class center loss as the THNet training objective, which can comprehensively manage multiple object classes. These features mean that the HfL framework can localize specific objects very quickly, regardless of the size of the scene. Extensive experiments validate the effectiveness and efficiency of the framework. For instance, HfL can find objects in a remote-sensing image of 19584$\times$19584 pixels in only 4.388 s (on a single RTX2080ti), with remarkable localization results. The source codes and datasets are available athttps://github.com/lrhan/HfL, together providing a baseline for fast localizing objects in a large-scale scene.

Standard hyperspectral (HS) pansharpening rely on fusion to enhance low-resolution HS (LRHS) images to the resolution of their matching panchromatic (PAN) images, whose practical implementation is normally under a stipulation of scale invariance of model across the training phase and the pansharpening phase. By contrast, arbitrary resolution HS (ARHS) pansharpening seeks to pansharpen LRHS images to any user-customized resolutions. For such a new HS pansharpening task, it is not feasible to train and store CNN models for all possible candidate scales, which implies the single model acquired from the training phase should be capable of being generalized to yield HS images with any resolutions in the pansharpening phase. To address the challenge, a novel variable sub-pixel convolution (VSPC)-based CNN (VSPC-CNN) method following our arbitrary upsampling CNN (AU-CNN) framework is developed for ARHS pansharpening. The VSPC-CNN method comprises a two-stage elevating thread. The first stage is to improve the spatial resolution of input HS image to that of the PAN image through a pre-pansharpening module and then a VSPC-encapsulated arbitrary scale attention upsampling (ASAU) module is cascaded for arbitrary resolution adjustment. After training with given scales, it can be generalized to pansharpen HS image to arbitrary scales under the spatial patterns invariance across the training and pansharpening phases. Experimental results from several specific VSPC-CNNs on both simulated and real HS datasets show the superiority of the proposed method.

Hyperspectral (HS) images are characterized by approximately contiguous spectral information, enabling the fine identification of materials by capturing subtle spectral discrepancies. Owing to their excellent locally contextual modeling ability, convolutional neural networks (CNNs) have been proven to be a powerful feature extractor in HS image classification. However, CNNs fail to mine and represent the sequence attributes of spectral signatures well due to the limitations of their inherent network backbone. To solve this issue, we rethink HS image classification from a sequential perspective with transformers, and propose a novel backbone network called \ul{SpectralFormer}. Beyond band-wise representations in classic transformers, SpectralFormer is capable of learning spectrally local sequence information from neighboring bands of HS images, yielding group-wise spectral embeddings. More significantly, to reduce the possibility of losing valuable information in the layer-wise propagation process, we devise a cross-layer skip connection to convey memory-like components from shallow to deep layers by adaptively learning to fuse "soft" residuals across layers. It is worth noting that the proposed SpectralFormer is a highly flexible backbone network, which can be applicable to both pixel- and patch-wise inputs. We evaluate the classification performance of the proposed SpectralFormer on three HS datasets by conducting extensive experiments, showing the superiority over classic transformers and achieving a significant improvement in comparison with state-of-the-art backbone networks. The codes of this work will be available at https://github.com/danfenghong/IEEE_TGRS_SpectralFormer for the sake of reproducibility.

Convolutional long short-term memory (ConvLSTM) has received much attention for hyperspectral image (HSI) classification due to its ability of modeling long-range correlations, which, however, is vulnerable to too many parameters and insufficient training, limiting its classification accuracy, especially for small samples. Different from it, traditional hand-crafted methods extract the features with basic attributes of HSIs, which can provide the lack of details and interpretability of deep semantic features. However, existing methods fail to incorporate their complementarity for HSI classification. As such, a Pseudo complex-valued (CV) Deformable ConvLSTM Neural Network with mutual Attention learning (APDCLNN) is proposed, providing a new way to realize the collaborative learning of hand-crafted and deep features for HSI classification. First, a 2-D pseudo CV deformable ConvLSTM (PDConvLSTM2D) cell is designed using deformable convolution and complex operations, with which a spatial–spectral PDConvLSTM2D neural network (SSPDCL2DNN) is built to extract scale- and spectral-enhanced deep spatial–spectral features. Then, 3-D Gabor filter is used to extract hand-crafted features, and a mutual attention-based multimodality feature learning and fusion (MAMLF) module is designed to integrate them into deep features for training and optimization of SSPDCL2DNN. Finally, an attention loss subnetwork is designed to refine the classification results. As we know, this is the first attempt to apply the idea of mutual attention learning to fuse hand-crafted and deep features for HSI classification. Extensive experiments on three widely used HSI datasets show the advantages of our model over other deep methods in terms of both quantitative and visual quality.

Endowing convolutional neural networks (CNNs) with the rotation-invariant capability is important for characterizing the semantic contents of remote sensing (RS) images since they do not have typical orientations. Most of the existing deep methods for learning rotation-invariant CNN models are based on the design of proper convolutional or pooling layers, which aims at predicting the correct category labels of the rotated RS images equivalently. However, a few works have focused on learning rotation-invariant embeddings in the framework of deep metric learning for modeling the fine-grained semantic relationships among RS images in the embedding space. To fill this gap, we first propose a rule that the deep embeddings of rotated images should be closer to each other than those of any other images (including the images belonging to the same class). Then, we propose to maximize the joint probability of the leave-one-out image classification and rotational image identification. With the assumption of independence, such optimization leads to the minimization of a novel loss function composed of two terms: 1) a class-discrimination term and 2) a rotation-invariant term. Furthermore, we introduce a penalty parameter that balances these two terms and further propose a final loss to Rotation-invariant Deep embedding for RS images, termed RiDe. Extensive experiments conducted on two benchmark RS datasets validate the effectiveness of the proposed approach and demonstrate its superior performance when compared to other state-of-the-art methods. The codes of this article will be publicly available athttps://github.com/jiankang1991/TGRS_RiDe.

Synthetic aperture radar (SAR) images provide all-weather and all-time capabilities for Earth observation, which becomes highly beneficial in the field of intelligent remote sensing (RS) image interpretation. Due to these advantages, SAR images have been widely exploited in automatic building segmentation tasks under poor weather conditions, especially when disasters happen. However, compared to optical images, the semantics inherent to SAR images are less rich and interpretable due to factors such as speckle noise and imaging geometry. In this scenario, most state-of-the-art methods are focused on designing advanced network architectures or loss functions for building footprint extraction. However, few works have been oriented toward improving segmentation performance through knowledge transfer from optical images. In this article, we propose a novel method based on theDisOptNetnetwork, which can distill the useful semantic knowledge from optical images into a network only trained with SAR data. Specifically, we first analyze the multilevel feature discrepancies between multiple stages of the networks pretrained on the two image modalities. We observe that feature discrepancies start to increase as the encoding stage gradually changes from low level to high level. Based on such observation, we reuse the early stage features and construct parallel convolutional neural network (CNN) branches that are responsible for capturing high-level domain-specific knowledge for each image modality. The optical branch is aimed at mimicking feature generation at the optical pretrained network given the input SAR images. Then, an aggregation module is introduced to calibrate and fuse the features from different modalities while generating the building segments. Extensive experiments were conducted on a large-scale multisensor all-weather building segmentation dataset with state-of-the-art methods used for comparison. Our experimental results validate the effectiveness ofDisOptNet, which demonstrates great potential in the task of weather-independent building footprint generation under real scenarios. The codes of this article will be made publicly available athttps://github.com/jiankang1991/TGRS_DisOptNet.

Hyperspectral unmixing is a critical step to process hyperspectral images (HSIs). Nonnegative matrix factorization (NMF) has drawn extensive attention in remotely sensed hyperspectral unmixing since it does not require prior knowledge about the pure spectral constituents (endmembers) in the scene. However, this approach is normally implemented as a single-layer procedure, which does not allow for a refinement of the obtained endmember abundances. In addition, HSIs suffer from the interference of sparse noise (besides Gaussian noise), which brings challenges when pursuing efficient hyperspectral unmixing. To address these issues, we propose a new self-supervised robust deep matrix factorization (SSRDMF) model for hyperspectral unmixing, which consists of two parts:encoderanddecoder. In theencoder, a multilayer nonlinear structure is designed to directly map the observed HSI data to the corresponding abundances. The abundances are then decoded by thedecoder, in which the connected weights are treated as the extracted endmembers. By modeling the sparse noise explicitly, the proposed method can reduce the effect caused by both Gaussian and sparse noise. Furthermore, a self-supervised constraint is included for exploring the spectral information, which is beneficial to further improve unmixing performance. To validate our method, we have conducted extensive experiments on both synthetic and real datasets. Our experiments reveal that our newly developed SSRDMF achieves superior unmixing performance compared to other state-of-the-art methods.

Spatiotemporal fusion (STF) aims at generating remote-sensing data with both high spatial and temporal resolution. In the literature, one of the most widely used strategies to accomplish this goal is to fuse high temporal resolution images collected by the Moderate Resolution Imaging Spectroradiometer (MODIS) with images with finer spatial resolution than those provided by MODIS (e.g., those collected by other satellite instruments such as Landsat or Sentinel-2). Current STF methods generally fuse an upsampled MODIS image with finer spatial resolution images. This leads to two main problems. First of all, the model uncertainty errors (resulting from the ill-posed upsampling problem) will be propagated into the fusion results, leading to spatial and spectral distortion. Furthermore, the spatial details of the upsampled MODIS image may be significantly different from those of the finer spatial resolution images, making the STF problem even more challenging. In order to tackle these issues, in this work, we develop a new linear regression-based STF strategy (LiSTF), which performs the reconstruction from a MODIS-like image (instead of from an upsampled MODIS image), thus reducing the model uncertainty errors and preserving better the spatial information. The MODIS-like images are built from the finer spatial resolution images via downsampling. Our experimental results, conducted using two publicly available datasets of Landsat–MODIS image pairs and one publicly available dataset of Sentinel–MODIS image pairs, reveal that our newly proposed LiSTF approach can significantly enhance the quantitative and qualitative performance of STF, particularly in terms of preserving the spatial information.

Convolutional neural networks (CNNs) are widely used in hyperspectral image (HSI) classification. However, the network architecture of CNNs is often designed manually, which requires careful fine-tuning. Recently, many techniques for neural architecture search (NAS) have been proposed to design the network automatically but most of the methods are only concerned with the overall classification accuracy and ignore the balance between the floating point operations per second (FLOPs) and the number of parameters. In this paper, we propose a new multi-objective optimization (MO) method called MO-CNN to automatically design the structure of CNNs for HSI classification. First, a MO method based on continuous particle swarm optimization (CPSO) is constructed, where the overall accuracy, floating point operations (FLOPs) and the number of parameters are considered, to obtain an optimal architecture from the Pareto front. Then, an auxiliary skip connection strategy is added (together with a partial connection strategy) to avoid performance collapse and to reduce memory consumption. Furthermore, an end-to-end band selection network (BS-Net) is used to reduce redundant bands and to maintain spectral-spatial uniformity. To demonstrate the performance of our newly proposed MO-CNN in scenarios with limited training sets, a quantitative and comparative analysis (including ablation studies) is conducted. Our optimization strategy is shown to improve the classification accuracy, reduce memory and obtain an optimal structure for CNNs based on unbiased datasets.

For deep networks, accurate image similarities cannot be well characterized with limited iterations, so the latent relationships between images can be embedded to enhance image retrieval performance. In this article, we propose a method named DFLLR to learn deep image features and accurate image similarities for remote sensing image retrieval (RSIR) simultaneously. First, the AlexNet is employed to extract high-level semantic features. Second, to obtain accurate image similarities, latent relationships between images are constructed with manifold learning and embedded in the AlexNet model with fully connected layers; in this way, the latent relationships and image features can be jointly learned. Third, to boost the RSIR performance further, the constraints of central and margin for jointly learning latent relationships and image features are integrated into our DFLLR. The central constraint is used to reduce the discrepancy of the latent relationships at the intraclass level and enhance the accuracies of image features. Moreover, the margin constraint is designed to enhance the accuracies of the latent relationships by maximizing the manifold margin between the latent relationships at the intraclass and interclass levels. To validate our method, we perform comprehensive experiments on three publicly available remote sensing image datasets, and the results demonstrate that it significantly outperforms other state-of-the-art methods.

Nowadays, a variety of different sources can be combined together to measure and monitor maritime human activities. Reliable data fusion techniques are essential to associate the targets from different systems for maritime surveillance. In particular, the fusion of data from Sentinel-2 satellites and the Automatic Identification System (AIS) has attracted wide attention due to their public availability and complementarity. However, most traditional methods for target association are not suitable for this particular case, due to the time lag phenomenon of Sentinel-2 data. In this study, we first construct two new datasets for the detection of moving ships and their wakes based on Sentinel-2 images. Combined with the detection results obtained by the You Only Look Once (YOLOv5) model, the position and course information of the detected ships are first extracted. After carefully analyzing the time lag phenomenon of Sentinel-2 data, we develop a new domain adaptation-based method for target association based on the fusion of Sentinel-2 and AIS data, called Moving Ship Optimal Association (MSOA). Different from standard domain adaptation methods only for representation alignment, the proposed MSOA is able to align representation, time and position simultaneously. A case study is provided in which the newly proposed method is tested over the Port of Long Beach, USA. Experimental results demonstrate that both moving ships and wakes are well detected. Specifically, our newly proposed MSOA exhibits more accurate and robust performance when compared to traditional methods, and the detected ships without corresponding AIS tracks can also be detected by our MSOA. Moreover, the real sensing time and time lag of Sentinel-2 data are deduced with high accuracy. Overall, it can be concluded that our MSOA provides a new perspective for accurate target association based on heterogeneous data fusion.

Deep learning methods have demonstrated excellent performance in hyperspectral image (HSI) classification. However, these methods mainly focus on improving the classification accuracy while ignoring their high complexity. By considering that the data formats of both HSIs and network weights can be represented in the form of tensors, we develop a new lightweight tensorized neural network for HSI classification that takes advantage of low-rank tensor decomposition techniques to reduce complexity. Firstly, inspired by tensor train (TT)-based tensorized convolutional layers, a new tensorized 2D convolutional layer based on chain calculation (with better expression ability) is introduced. Based on this innovation, a new lightweight 2D tensorized neural network (2D-TNN) is designed for HSI classification. Furthermore, to better preserve the intrinsic structure of HSI data, a new lightweight 3D tensorized neural network (3D-TNN) is proposed by extending the tensorized 2D convolutional layers to their 3D versions. Quantitative and comparative experiments on three widely used data sets show that the proposed models are able to achieve state-of-the-art performance (with a low number of model parameters) for different training sample sizes, especially for very small training sets.

Convolutional neural networks (CNNs) have achieved excellent feature extraction capabilities in remotely sensed hyperspectral image (HSI) classification. This is due to their ability to learn representative spatial and spectral features. However, it is difficult for conventional computers to classify HSIs quickly enough for practical use in many applications, mainly because of the large number of calculations and parameters needed by deep learning-based methods. Although several weight quantization methods achieved remarkable results in network compression, the network acceleration effect is still not significant because a full exploration of the potential of network acceleration brought by network weight quantization is still absent from the literature. In this article, a new step activation quantization method is proposed to constrain the input of the network layer of the CNN so that the data can be represented by low-bit integers. As a result, floating-point operations can be replaced with integer operations to greatly accelerate the forward (inference) step of the network. Specifically, nonlinear uniform quantization is adopted in this work to restrain the input of the CNN in the forward inference of the step activation quantization layer, and two functions (constant and tanh-like) are used in the backpropagation step to avoid gradient vanishing and noise. Our newly proposed step activation quantization acceleration method is applied to a CNN for HSI with two well-known benchmark data sets and the experimental results demonstrate that the proposed method is very effective in terms of both memory savings and computation acceleration, with only a slight decrease in classification accuracy. Specifically, our method reduces memory requirements in$13.6\times $and obtains around$10\times {}$speedup with regard to the original real-valued network version.

Convolutional Neural Networks (CNNs) have been extensively utilized for Hyperspectral (HSI) as well as Light Detection and Ranging (LiDAR) data Classification. However, CNNs have not been much explored for joint HSI and LiDAR image classification. Therefore, this article proposes a joint feature learning (HSI and LiDAR) and fusion mechanism using CNN and Spatial Morphological blocks which generates highly accurate land-cover maps. The CNN model comprises three Conv3D layers and is directly applied to the HSIs for extracting discriminative spectral-spatial feature representation. On the contrary, the spatial morphological block is able to capture the information relevant to the height or shape of the different land-cover regions from LiDAR data. The LiDAR features are extracted using morphological dilation and erosion layers which increase the robustness of the proposed model by considering elevation information as an additional feature. Finally, both the obtained features from CNNs and spatial morphological blocks are combined using an additive operation prior to the classification. Extensive experiments are shown with widely used HSIs and LiDAR datasets, i.e., University of Houston (UH), Trento, and MUUFL Gulfport scene. The reported results show that the proposed model significantly outperforms traditional methods and other state-of-the-art deep learning models. The source code for the proposed model will be made available publicly at https://github.com/AnkurDeria/HSI+LiDAR.

Recently, convolutional neural networks (CNNs) have exhibited commendable performance for hyperspectral image (HSI) classification. Generally, an important number of samples are needed for each class to properly train CNNs. However, existing HSI data sets suffer from a significant class imbalance problem, where many classes do not have enough samples to characterize the spectral information. The performance of existing CNN models is biased toward the majority classes, which possess more samples for the training. This article addresses this issue of imbalanced data in HSI classification. In particular, a new3D-HyperGAMOmodel is proposed, which uses generative adversarial minority oversampling. The proposed3D-HyperGAMOautomatically generates more samples for minority classes at training time, using the existing samples of that class. The samples are generated in the form of a 3-D hyperspectral patch. A different classifier from the generator and the discriminator is used in the3D-HyperGAMOmodel, which is trained using both original and generated samples to determine the classes of newly generated samples to which they actually belong. The generated data are combined classwise with the original training data set to learn the network parameters of the class. Finally, the trained 3-D classifier network validates the performance of the model using the test set. Four benchmark HSI data sets, namely, Indian Pines (IP), Kennedy Space Center (KSC), University of Pavia (UP), and Botswana (BW), have been considered in our experiments. The proposed model shows outstanding data generation ability during the training, which significantly improves the classification performance over the considered data sets. The source code is available publicly athttps://github.com/mhaut/3D-HyperGAMO.

The hyperspectral images are composed of a variety of textures across the different bands which increase the spectral similarity and make it difficult to predict the pixel-wise labels without inducing additional complexity at the feature level. To extract robust and discriminative features from the different regions of land cover, the hyperspectral research community is still seeking such type of convolutions which can efficiently deal with fine-grained texture information during the feature extraction phase, which often overlook this aspect by vanilla convolution. To overcome the above shortcoming, this article proposes a generalized gradient centralized 3D convolution (G2C-Conv3D) operation, which is a weighted combination between the vanilla and gradient centralized 3D convolutions (GC-Conv3D) to extract both theintensity-levelsemantic information andgradient-levelinformation. This can be easily plugged into the existing HSI feature extraction networks to boost the performance of accurate prediction for land-cover types. To validate the feasibility of the proposedG2C-Conv3D, we have considered the existing CNN3D, MS3DNet, ContextNet, and SSRN feature extraction models and as well as CAE3D, VAE3D, and SAE3D autoencoder (AE) networks, respectively. All these networks are embedded withG2C-Conv3Dconvolution to implement both generalized gradient centralized feature extraction networks (G2C-FE) and generalized gradient centralized AE networks (G2C-AE) for fine-grained spectral–spatial feature learning. In addition,G2C-Conv2Dis also considered with few networks. The extensive experiments are conducted on four most widely used hyperspectral datasets i.e., IP, KSC, UH, and UP, respectively, and compared with the nine methods. The results demonstrate that the proposedG2C-Conv3Dcan effectively enhance the feature learning ability of the existing networks and both the qualitative and quantitative results show the superiority and effectiveness of the proposedG2C-Conv3D. The source codes will be publicly available athttps://github.com/danfenghong/G2C-Conv3D-HSI.

Unsupervised hashing for remote sensing (RS) image retrieval first extracts image features and then use these features to construct supervised information (e.g., pseudo-labels) to train hashing networks. Existing methods usually regard RS images as natural images to extract unisource features. However, these features only contain partial information about ground objects and cannot produce reliable pseudo-labels. In addition, existing methods only generate a pseudo single-label to annotate each RS image, which cannot accurately represent multiple scenes in a RS image. To address these drawbacks, this paper proposes a new Multisource data reconstruction-based deep unsupervised Hashing method, called MrHash, which explores the characteristics of RS images to construct reliable pseudo-labels. In particular, we first use geographic coordinates to obtain different satellite images and develop a novel autoencoder network to extract multisource features from these images. Then pseudo multi-labels are designed to deal with the coexistence of multiple scenes in a single image. These labels are generated by a custom probability function with extracted multisource features. Finally, we propose a novel multi-semantic hash loss by using the Kull-back–Leibler (KL) divergence to preserve the semantic similarity of these pseudo multi-labels in Hamming space. Our newly developed MrHash only uses multisource images to construct supervised information, and hash code generation still relies on a unisource input image. Experiments on benchmark datasets clearly show the superiority of the proposed method over state-of-the-art baselines. https://github.com/sunyuxi/MrHash.

Spectral unmixing plays a vital role in hyperspectral image analysis. It mainly consists of two procedures, i.e., endmember extraction and abundance estimation. Although most algorithms for each of the two procedures may exhibit good performance, few studies have been done considering both problems simultaneously. Therefore, hyperspectral unmixing accuracy is normally achieved by exploring all possible combinations of the two types of algorithms, which renders high computational overloads. We propose a novel orthogonal projection framework to conduct fast hyperspectral unmixing. It addresses both endmember extraction and abundance estimation with orthogonal projection endmember (OPE) and orthogonal projection abundance (OPA). Especially, the pixel with the largest orthogonal projection on any pixel is considered to be an endmember. We randomly choose one pixel from the hyperspectral data to compute the orthogonal projections of all pixels and extract the pixel with the largest projection as the first endmember. To avoid extracting the same endmembers, we compute orthogonal projections of all pixels to endmembers that have been previously extracted, and the pixel with the largest projection is considered as the next endmember. In terms of abundance estimation, we also utilize the concept of orthogonal projection and search for a diagonal matrix whose multiplication with the endmember matrix is not only a square matrix but also a diagonal matrix. Then, we exploit some specific matrix operations to estimate the abundance of each endmember at every pixel. We have evaluated the proposed OPE and OPA algorithms on synthetic and real data, and the experimental results have validated their effectiveness and efficiency in hyperspectral unmixing.

Hyperspectral images (HSIs) contain abundant information in the spatial and spectral domains, allowing for a precise characterization of categories of materials. Convolutional neural networks (CNNs) have achieved great success in HSI classification, owing to their excellent ability in local contextual modeling. However, CNNs suffer from fixed filter weights and deep convolutional layers, which lead to a limited receptive field and high computational burden. The recent Vision Transformer (ViT) models long-range dependencies with a self-attention mechanism and has been an alternative backbone to the CNNs traditionally used in HSI classification. However, such transformer-based architectures designate all input pixels of the receptive field as feature tokens in terms of feature embedding and self-attention, which inevitably limits the ability for learning multi-scale features and increases the computational cost. To overcome this issue, we propose a local semantic feature aggregation-based transformer (LSFAT) architecture which allows transformers to represent long-range dependencies of multi-scale features more efficiently. We introduce the concept of the homogeneous region into the transformer by considering a pixel aggregation strategy and further propose neighborhood aggregation-based embedding (NAE) and attention (NAA) modules, which are able to adaptively form multi-scale features and capture locally spatial semantics among them in a hierarchical transformer architecture. A reusable classification token is included together with the feature tokens in the attention calculation. In the last stage, a fully connected layer is employed to perform classification on the reusable token after transformer encoding. We verify the effectiveness of the NAE and NAA modules compared with the traditional ViT through extensive experiments. Our results demonstrate the excellent classification performance of the proposed method in comparison with other state-of-the-art approaches on several public HSIs.

Anomaly detection is an important technique for hyperspectral image processing. It aims to find pixels that are markedly different from the background when the target spectrum is unavailable. Many anomaly detection methods have been proposed over the past years, among which graph-based ones have attracted extensive attention. And they usually just consider the spectral information to build the adjacency matrix of the graph, which does not think over the effect of spatial information in this process. This paper proposes a new anomaly detection method using the Spectral-Spatial Graph (SSG) that considers both the spatial and spectral information. Thus, the spatial adjacency matrix and spectral adjacency matrix are constructed from the spatial and spectral dimensions, respectively. To obtain a spectral-spatial graph with more discriminant characteristics, and two different local neighborhood detection strategies are used to measure the similarity of the SSG. Furthermore, global anomaly detection results on hyperspectral images were obtained by the graph Laplacian anomaly detection method and the global and local anomaly detection results were optimized by the differential fusion method. Compared with other anomaly detection algorithms on several synthetic and real data sets, the proposed algorithm shows superior detection performance.

In hyperspectral anomaly detection, anomalies are rare targets that exhibit distinct spectral signatures from the background. Thus, anomalies are with low probabilities of occurrence in hyperspectral images. In this article, we develop a new technique for hyperspectral anomaly detection that adopts a new information theory perspective, to fully utilize the aforementioned concepts. Our goal is to transform system entropy into quantitative metrics of anomaly conspicuousness of pixels. To do so, two tasks are first completed: first, the construction of occurrence probability of pixels based on the density peak clustering algorithm, and second, the valid system definitions for pixels in specific anomaly detection problems with multiviews. Specifically, three types of systems are separately established by pixel pairs to conform to the definitions of three entropy definitions in information theory, i.e., Shannon entropy, joint entropy, and relative entropy. Then, three individual entropy-based metrics that assess the anomaly conspicuousness are defined. In addition, we design a standard deviation-based ensemble strategy for the integrated representation of the three individual metrics, which considers both logic “OR” and “AND” operations to simultaneously improve the detection rate and reduce the false alarm rate. Our experimental results obtained on two publicly available datasets with anomalies of different sizes and shapes demonstrate the superiority of our newly proposed anomaly detection method.

Fisher vectors (FVs) can capture multiple order information from superpixels (SPs) in synthetic aperture radar (SAR) images. Existing FV-based ship detectors mainly exploit the local contrast of FVs (LCFVs) but do not consider their global density features. This may lead to degraded performance in terms of discrimination between ship targets and the complex sea clutter. In this article, two new global cues from FVs are designed based on the fact that target FVs exhibit much lower densities than those of clutter FVs and also have large distances to the latter. Our two new global cues can suppress the sea clutter and significantly enhance ship targets throughout the SAR image. We also design an improved local cue from FVs for ship detection, in which the intensity contrast of SPs is incorporated into the existing LCFV indicator to reduce false alarms. By fusing the above two new global cues (and an improved local cue from FVs), we propose a new method for ship detection in SAR images. Experimental results based on Gaofen-3 SAR images show that the newly proposed detector provides better detection performance than other state-of-the-art detectors, especially in the presence of strong and highly heterogeneous sea clutter.

The simple linear iterative clustering (SLIC) has been shown as an efficient and widely used superpixel-based algorithm for segmenting marine synthetic aperture radar (SAR) images. However, SLIC does not consider the fact that the density of ship target pixels is significantly lower than that of sea clutter pixels, leading to a waste of computational cost and memory resources on lots of pure clutter areas and to the degradation of the compactness of superpixels. To address the aforementioned issues, we develop a new density-based SLIC (DSLIC) method for the superpixel-based segmentation of marine SAR images. In the initialization stage of our DSLIC, all the subimages in a large marine SAR image are rapidly prescreened via a new density-driven classifier, where most of the subimages only occupied by clutter pixels with comparatively high density are discarded and do not need to be segmented in the subsequent local clustering stage. The retained subimages contain both the clutter and potential target areas. This prescreening operation results in higher computation efficiency and memory savings. In the local clustering stage of DSLIC, besides the intensity proximity and the spatiality proximity (used in SLIC), the sparsity proximity (measured by density distances) is considered to reduce the coexistence of sparse target pixels with low density and nonsparse clutter pixels with high density within superpixels. Our theoretical and experimental results show that the proposed DSLIC method is faster and requires less memory than SLIC and other state-of-the-art superpixel-based segmentation methods for marine SAR images with similar or better segmentation accuracy.

Anomaly detection has become an important remote sensing application due to the abundant spectral and spatial information contained in hyperspectral images. Recently, hyperspectral anomaly detection methods based on collaborative representation model have attracted significant attention. Nevertheless, these methods have to face two main challenges: (1) all features (spectral signatures) are constrained to share the same representation coefficient, which ignores the differences among features; (2) existing dictionaries for pixel-by-pixel detection model are usually not reliable. To address these issues, this paper proposes a new relaxed collaborative representation detector for hyperspectral anomaly detection by using a novel non-global dictionary. The proposed detector conducts collaborative representation on each feature dimension of the pixel under test, and simultaneously constrains the coding vectors of different features to be similar. To the best of our knowledge, this is the first time that a detection model is built from each feature dimension. To adjust the contributions of each feature, an adaptive feature weight constrained version of the method is also proposed. The non-global dictionary is constructed by combining the k-nearest neighbor method and an existing global dictionary, which is more reliable and practical than the widely used dual windows dictionary. In addition, this paper also designs a band selection strategy for the proposed method. Experiments on five real datasets indicate that the proposed method suppresses background well and outperforms other classical and state-of-the-art methods.

Remotely sensed hyperspectral images contain several bands (at about adjoining frequencies) for a similar zone on the surface of the Earth. Hyperspectral unmixing is a significant method for breaking down hyperspectral images into the components (endmembers) that conform each (potentially mixed) pixel and their abundance maps. Nonnegative matrix factorization (NMF) has attracted huge consideration because of the way that it can address mixed pixel scenarios. Most existing NMF unmixing techniques do not include spatial information in the analysis. An ongoing trend is to fuse the spatial and the spectral information contained in hyperspectral scenes to improve the solution. In this article, we build up another hyperspectral unmixing technique named spectral–spatial weighted sparse NMF (SSWNMF), in which two weighting factors are acquainted into the NMF model to upgrade the sparsity of the solution and capture the piecewise smooth structure of the data. We adopt a multiplicative iterative strategy to implement the proposed SSWNMF model. Our experimental results, conducted with both synthetic and real hyperspectral data, uncover that the proposed SSWNMF strategy can get accurate unmixing results over those gave by other unmixing strategies, with less parameter tuning.

Hyperspectral image (HSI) classification is essential in remote sensing image analysis. The classification methods based on deep learning have attracted more and more attention. However, classification accuracy is seriously affected by the quantity of labeled data and redundant information. Therefore, a deep semisupervised shared subspace learning (DS4L) model is developed to overcome these problems in this article. DS4L is composed of two parts. First, the basic feature extraction (BFE) network is constructed to preliminary extract high-dimensional-space features of multiscale data and fusion them to one shared subspace. Then, a deep shared subspace learning (DSSL) network is proposed to obtain a deeper and more representative low-dimensional subspace. Moreover, to obtain a more representative subspace and alleviate dependence on labeled samples, the regular, irregular constraint, and cross-entropy (CE) loss are integrated into the model. The regular constraint is adopted to reconstruct the multiscale patches to ensure the quality of the subspace in an unsupervised manner. The irregular constraint can well embed labeled and unlabeled samples into the procedure of subspace learning (SL). Then, the CE loss is used to extract more discriminative subspace using the limited labeled samples. Finally, we perform experiments on three widely used HSI datasets. Compared with the basic SL model, the DS4L’s classification accuracy on the popular Salinas, Indian Pines, and PaviaU datasets are increased by 4.68%, 4.25%, and 1.57%, respectively.

Object detection in remote sensing images (RSIs) poses great difficulties due to arbitrary orientations, various scales and dense location of the targets over the ground. Recent evidence suggests that encoding the orientation information is of great use for training an accurate object detector for oriented object detection (OOD). In this paper, we propose a new frequency-domain orientation learning (FDOL) module with two main components: the frequency domain feature extraction (FFE) network and an orientation enhanced self-attention layer (OES-Layer). The FFE network models the interactions among spatial locations in the frequency domain to determine the frequency of spatial features. Then, these features are fed into our OES-Layer to learn the orientation information. Moreover, the orientation weights are adopted to guide the feature selection in a self-attention architecture, using them as a control gate to emphasize the spatial responses of target instances. Considering that the original similarity weights (calculated by the self-attention algorithm) do not distinctly model the orientation variation, the considered orientation weights provide an efficient asset to emphasize the orientation of objects. Extensive experiments on the DOTA and HRSC2016 datasets demonstrate that our method achieves state-of-the-art performance among single-scale methods, while achieving competitive performance over multi-scale methods.

The problem of effectively exploiting the information multiple data sources has become a relevant but challenging research topic in remote sensing. In this article, we propose a new approach to exploit the complementarity of two data sources: hyperspectral images (HSIs) and light detection and ranging (LiDAR) data. Specifically, we develop a new dual-channel spatial, spectral and multiscale attention convolutional long short-term memory neural network (called dual-channel$A^{3}$CLNN) for feature extraction and classification of multisource remote sensing data. Spatial, spectral, and multiscale attention mechanisms are first designed for HSI and LiDAR data in order to learn spectral- and spatial-enhanced feature representations and to represent multiscale information for different classes. In the designed fusion network, a novel composite attention learning mechanism (combined with a three-level fusion strategy) is used to fully integrate the features in these two data sources. Finally, inspired by the idea of transfer learning, a novel stepwise training strategy is designed to yield a final classification result. Our experimental results, conducted on several multisource remote sensing data sets, demonstrate that the newly proposed dual-channel$A^{\,3}$CLNN exhibits better feature representation ability (leading to more competitive classification performance) than other state-of-the-art methods.

Manual and automatic annotation of Remote Sensing (RS) scenes are rather complex tasks which may unavoidably introduce some degree of mislabeled data in large-scale archives. In this regard, noisy annotations become an important constraint for deep metric learning-based RS characterization methods since most of them are trained in a supervised way. To address this problem, here we investigate the use of deep metric learning for characterizing RS scenes with noisy labels. Specifically, we consider the Normalized Softmax Loss and develop a robust extension, i.e., the Robust Normalized Softmax Loss (RNSL), in order to effectively capture the semantic relationships among RS scenes with mislabeled ground-truth information. The conducted experiments, using the K-NN classifier and two benchmark RS image archives, show the potential of the proposed approach with respect to other state-of-the-art methods.

Deep metric learning has recently become a prominent technology for the semantic understanding of remote sensing (RS) scenes due to its great potential for characterizing visual semantics. However, state-of-the-art deep metric learning models are often constrained in RS by the use of single-label annotations, which eventually reduce their capacity to characterize complex aerial scenes. Additionally, many of the existing works are specialized in particular RS applications which constrains the study of their associated metric spaces from a multi-task perspective. In this paper, we propose a new unified deep metric learning approach for both single- and multi-label RS scene characterization while also taking into account different downstream RS applications. Specifically, we extend the Scalable Neighborhood Component Analysis (SNCA) to the multi-label case and propose its generalized version, i.e., GSNCA. Extensive experiments on single- and multi-label RS benchmark datasets have been conducted to evaluate the effectiveness of the proposed method for RS image classification, clustering and retrieval.

Social media data generated from individuals provides a unique opportunity to gain valuable insight on information flow, especially for emergency response. However, the inherent limitations associated to these data (particularly, the spatial bias) restrict its precise application. Existing research on spatial bias correction of social media data mainly face two issues: 1) the geographic extent in target domains may be underestimated, and 2) source elements may be transported within inappropriate distance. In this paper, we take 2013 Boulder, Colorado flood event as a case study, and present a new method called subspace optimal transport (SOT). Our proposed SOT aims at transporting biased tweets from dry to real flooded areas with a relatively close distance. Specifically, a comparison between our newly developed SOT and the traditional optimal transport (OT) and geographic optimal transport (GOT) is performed. Experimental results demonstrate that our new SOT method is able to correct the spatially biased geo-referenced tweets, with high precision and excellent computing performance.

José Manuel Bioucas-Dias was an outstanding expert in many different IEEE-related areas, including inverse problems in imaging, signal and image processing, pattern recognition, optimization, and remote sensing. He authored or coauthored more than 250 publications, including more than 100 journal papers (66 of which published in IEEE journals) and over 200 peer-reviewed international conference papers and book chapters. His contributions have been extremely influential in many different fields, namely phase estimation and unwrapping, convex optimization and Bayesian inference for imaging inverse problems, with a special emphasis on remote sensing, including synthetic aperture radar (SAR), hyperspectral unmixing, fusion, super-resolution, classification, and segmentation. In this paper, we provide an overview of his outstanding contributions to remote sensing image processing.

Land cover classification of remotely sensed images is an extremely important and challenging task. During the last two decades, several methods have been proposed to deal with this problem. In particular, convolutional neural network (CNN)-based methods for land cover classification have enjoyed high popularity due to their strong feature extraction and characterization abilities. However, most CNNs-based methods use relatively small kernels (usually, 3 x 3 pixels in size). Increasing the size of the kernel introduces a lot of parameters and renders considerable computational overloads. To address this issue and allow for the processing of large image datasets, the pyramidal convolution (PyConv) network has been adopted. PyConv network contains several levels of kernels with varying scales and depths, and shows significant improvements in the task of visual recognition. In this paper, we evaluate the performance of the PyConv network on the UCMERCED dataset. Our experimental results reveal that the considered approach exhibits good performance and high efficiency in the task of land cover classification.

Convolutional neural networks (CNNs) have demonstrated excellent performance in hyperspectral image (HSI) classification. However, tuning some critical hyperparameters of a CNN-such as the receptive field (RF) size-presents a major challenge due to the presence of features with different scales in HSIs. Contrary to the conventional design of CNNs, which fixes the RF size, it has been proven that the RF size is modulated by the stimulus and hence, depends on the scene being considered. Such a dilemma has been rarely considered in CNN design. In this letter, a new multibranch selective kernel network (MSKNet) is introduced, in which the input image is convolved using different RF sizes to create multiple branches so that the effect of each branch is adjusted by an attention mechanism according to the input contrast. As a result, our newly developed MSKNet is capable of modeling different scales. Our experimental results, conducted on three widely used HSIs, reveal that the MSKNet can outperform state-of-the-art CNNs in the context of HSI classification problems. The source code of our newly developed MSKNet is available from: https://github.com/mhaut/MSKNet-HSI.

Unsupervised hashing methods have attracted considerable attention in large-scale remote sensing (RS) image retrieval, due to their capability for massive data processing with significantly reduced storage and computation. Although existing unsupervised hashing methods are suitable for operational applications, they exhibit limitations when accurately modeling the complex semantic content present in RS images using binary codes (in an unsupervised manner). To address this problem, in this letter, we introduce a novel unsupervised hashing method that takes advantage of the generative nature of probabilistic topic models to encapsulate the hidden semantic patterns of the data into the final binary representation. Specifically, we introduce a new probabilistic latent semantic hashing (pLSH) model to effectively learn the hash codes using three main steps: 1) data grouping, where the input RS archive is clustered into several groups; 2) topic computation, where the pLSH model is used to uncover highly descriptive hidden patterns from each group; and 3) hash code generation, where the data probability distributions are thresholded to generate the final binary codes. Our experimental results, obtained on two benchmark archives, reveal that the proposed method significantly outperforms state-of-the-art unsupervised hashing methods.

High-resolution digital surface models (DSMs) provide valuable height information about the Earth's surface, which can be successfully combined with other types of remotely sensed data in a wide range of applications. However, the acquisition of DSMs with high spatial resolution is extremely time-consuming and expensive with their estimation from a single optical image being an ill-possed problem. To overcome these limitations, this letter presents a new unpaired approach to obtain DSMs from optical images using deep learning techniques. Specifically, our new deep neural model is based on variational autoencoders (VAEs) and generative adversarial networks (GANs) to perform image-to-image translation, obtaining DSMs from optical images. Our newly proposed method has been tested in terms of photographic interpretation, reconstruction error, and classification accuracy using three well-known remotely sensed data sets with very high spatial resolution (obtained over Potsdam, Vaihingen, and Stockholm). Our experimental results demonstrate that the proposed approach obtains satisfactory reconstruction rates that allow enhancing the classification results for these images. The source code of our method is available from: https://github.com/mhaut/UIMG2DSM.

As a newly emerging technology, deep learning (DL) is a very promising field in big data applications. Remote sensing often involves huge data volumes obtained daily by numerous in-orbit satellites. This makes it a perfect target area for data-driven applications. Nowadays, technological advances in terms of software and hardware have a noticeable impact on Earth observation applications, more specifically in remote sensing techniques and procedures, allowing for the acquisition of data sets with greater quality at higher acquisition ratios. This results in the collection of huge amounts of remotely sensed data, characterized by their large spatial resolution (in terms of the number of pixels per scene), and very high spectral dimensionality, with hundreds or even thousands of spectral bands. As a result, remote sensing instruments on spaceborne and airborne platforms are now generating data cubes with extremely high dimensionality, imposing several restrictions in terms of both processing runtimes and storage capacity. In this article, we provide a comprehensive review of the state of the art in DL for remote sensing data interpretation, analyzing the strengths and weaknesses of the most widely used techniques in the literature, as well as an exhaustive description of their parallel and distributed implementations (with a particular focus on those conducted using cloud computing systems). We also provide quantitative results, offering an assessment of a DL technique in a specific case study (source code available: https://github.com/mhaut/cloud-dnn-HSI). This article concludes with some remarks and hints about future challenges in the application of DL techniques to distributed remote sensing data interpretation problems. We emphasize the role of the cloud in providing a powerful architecture that is now able to manage vast amounts of remotely sensed data due to its implementation simplicity, low cost, and high efficiency compared to other parallel and distributed architectures, such as grid computing or dedicated clusters.

The large data volume and high algorithm complexity of hyperspectral image (HSI) problems have posed big challenges for efficient classification of massive HSI data repositories. Recently, cloud computing architectures have become more relevant to address the big computational challenges introduced in the HSI field. This article proposes an acceleration method for HSI classification that relies on scheduling metaheuristics to automatically and optimally distribute the workload of HSI applications across multiple computing resources on a cloud platform. By analyzing the procedure of a representative classification method, we first develop its distributed and parallel implementation based on the MapReduce mechanism on Apache Spark. The subtasks of the processing flow that can be processed in a distributed way are identified as divisible tasks. The optimal execution of this application on Spark is further formulated as a divisible scheduling framework that takes into account both task execution precedences and task divisibility when allocating the divisible and indivisible subtasks onto computing nodes. The formulated scheduling framework is an optimization procedure that searches for optimized task assignments and partition counts for divisible tasks. Two metaheuristic algorithms are developed to solve this divisible scheduling problem. The scheduling results provide an optimized solution to the automatic processing of HSI big data on clouds, improving the computational efficiency of HSI classification by exploring the parallelism during the parallel processing flow. Experimental results demonstrate that our scheduling-guided approach achieves remarkable speedups by facilitating the automatic processing of HSI classification on Spark, and is scalable to the increasing HSI data volume.

Convolutional neural networks (CNNs) have been attracting increasing attention in hyperspectral (HS) image classification due to their ability to capture spatial-spectral feature representations. Nevertheless, their ability in modeling relations between the samples remains limited. Beyond the limitations of grid sampling, graph convolutional networks (GCNs) have been recently proposed and successfully applied in irregular (or nongrid) data representation and analysis. In this article, we thoroughly investigate CNNs and GCNs (qualitatively and quantitatively) in terms of HS image classification. Due to the construction of the adjacency matrix on all the data, traditional GCNs usually suffer from a huge computational cost, particularly in large-scale remote sensing (RS) problems. To this end, we develop a new minibatch GCN (called miniGCN hereinafter), which allows to train large-scale GCNs in a minibatch fashion. More significantly, our miniGCN is capable of inferring out-of-sample data without retraining networks and improving classification performance. Furthermore, as CNNs and GCNs can extract different types of HS features, an intuitive solution to break the performance bottleneck of a single model is to fuse them. Since miniGCNs can perform batchwise network training (enabling the combination of CNNs and GCNs), we explore three fusion strategies: additive fusion, elementwise multiplicative fusion, and concatenation fusion to measure the obtained performance gain. Extensive experiments, conducted on three HS data sets, demonstrate the advantages of miniGCNs over GCNs and the superiority of the tested fusion strategies with regard to the single CNN or GCN models. The codes of this work will be available at https://github.com/danfenghong/IEEE_TGRS_GCN for the sake of reproducibility.

Most deep metric learning-based image characterization methods exploit supervised information to model the semantic relations among the remote sensing (RS) scenes. Nonetheless, the unprecedented availability of large-scale RS data makes the annotation of such images very challenging, requiring automated supportive processes. Whether the annotation is assisted by aggregation or crowd-sourcing, the RS large-variance problem, together with other important factors [e.g., geo-location/registration errors, land-cover changes, even low-quality Volunteered Geographic Information (VGI), etc.] often introduce the so-called label noise, i.e., semantic annotation errors. In this article, we first investigate the deep metric learning-based characterization of RS images with label noise and propose a novel loss formulation, named robust normalized softmax loss (RNSL), for robustly learning the metrics among RS scenes. Specifically, our RNSL improves the robustness of the normalized softmax loss (NSL), commonly utilized for deep metric learning, by replacing its logarithmic function with the negative Box–Cox transformation in order to down-weight the contributions from noisy images on the learning of the corresponding class prototypes. Moreover, by truncating the loss with a certain threshold, we also propose a truncated robust normalized softmax loss (t-RNSL) which can further enforce the learning of class prototypes based on the image features with high similarities between them, so that the intraclass features can be well grouped and interclass features can be well separated. Our experiments, conducted on two benchmark RS data sets, validate the effectiveness of the proposed approach with respect to different state-of-the-art methods in three different downstream applications (classification, clustering, and retrieval). The codes of this article will be publicly available fromhttps://github.com/jiankang1991.

Convolutional neural networks (CNNs) have achieved great success when characterizing remote sensing (RS) images. However, the lack of sufficient annotated data (together with the high complexity of the RS image domain) often makes supervised and transfer learning schemes limited from an operational perspective. Despite the fact that unsupervised methods can potentially relieve these limitations, they are frequently unable to effectively exploit relevant prior knowledge about the RS domain, which may eventually constrain their final performance. In order to address these challenges, this article presents a new unsupervised deep metric learning model, called spatially augmented momentum contrast (SauMoCo), which has been specially designed to characterize unlabeled RS scenes. Based on the first law of geography, the proposed approach defines spatial augmentation criteria to uncover semantic relationships among land cover tiles. Then, a queue of deep embeddings is constructed to enhance the semantic variety of RS tiles within the considered contrastive learning process, where an auxiliary CNN model serves as an updating mechanism. Our experimental comparison, including different state-of-the-art techniques and benchmark RS image archives, reveals that the proposed approach obtains remarkable performance gains when characterizing unlabeled scenes since it is able to substantially enhance the discrimination ability among complex land cover categories. The source codes of this article will be made available to the RS community for reproducible research.

Due to the proliferation of large-scale remote-sensing (RS) archives with multiple annotations, multilabel RS scene classification and retrieval are becoming increasingly popular. Although some recent deep learning-based methods are able to achieve promising results in this context, the lack of research on how to learn embedding spaces under the multilabel assumption often makes these models unable to preserve complex semantic relations pervading aerial scenes, which is an important limitation in RS applications. To fill this gap, we propose a new graph relation network (GRN) for multilabel RS scene categorization. Our GRN is able to model the relations between samples (or scenes) by making use of a graph structure which is fed into network learning. For this purpose, we define a new loss function called scalable neighbor discriminative loss with binary cross entropy (SNDL-BCE) that is able to embed the graph structures through the networks more effectively. The proposed approach can guide deep learning techniques (such as convolutional neural networks) to a more discriminative metric space, where semantically similar RS scenes are closely embedded and dissimilar images are separated from a novel multilabel viewpoint. To achieve this goal, our GRN jointly maximizes a weighted leave-one-out K-nearest neighbors ( KNN) score in the training set, where the weight matrix describes the contributions of the nearest neighbors associated with each RS image on its class decision, and the likelihood of the class discrimination in the multilabel scenario. An extensive experimental comparison, conducted on three multilabel RS scene data archives, validates the effectiveness of the proposed GRN in terms of KNN classification and image retrieval. The codes of this article will be made publicly available for reproducible research in the community.

The fusion of heterogeneous remote sensing and social media data can fill the gaps in satellite image collections and improve the spatiotemporal resolution of the available data sets. As a result, it is being gradually adopted in multimodal data analytics. Generally, the fusion of heterogeneous geographic data faces the following issues: 1) the probability density functions may differ from different data sources and 2) the geolocations may not be well aligned. The former one can be generally solved by performing an alignment of representations in the source and target domains using, for instance, domain adaptation. The latter issue is seldom considered in the fusion of heterogeneous geographic data. In this article, we present a new method called geographic optimal transport (GOT), which aims at aligning representations and geolocations in a simultaneous fashion. A flood event that took place in 2013 in Boulder, CO, USA, is taken as a case study to evaluate our GOT method. Here, we consider two remote sensing features derived from water indicators, i.e., the normalized difference vegetation index (NDVI) and the normalized difference water index (NDWI), for the fusion of Landsat 8 imagery and Twitter data. A comparison between our newly developed GOT and the traditional optimal transport (OT) is performed. Experimental results demonstrate that the proposed GOT can accurately align spatially biased georeferenced tweets to the flood phenomena, leading to the conclusion that GOT can effectively fuse heterogeneous remote sensing and social media data.

Hyperspectral imaging (HSI) is a competitive remote sensing technique in several fields, from Earth observation to health, robotic vision, and quality control. Each HSI scene contains hundreds of (narrow) contiguous spectral bands. The amount of data generated by HSI devices is often both a solution and a problem for a given application. Extracting information from HSI data cubes is a complex and computationally demanding problem. To tackle this challenge, convolutional neural networks (CNNs) have been widely applied to HSI classification. Despite their success, CNNs are computationally demanding algorithms with high memory requirements due to their large number of internal parameters. The recent interest in using HSI devices in mobile and embedded systems for air and spaceborne platforms turned the attention to computationally lightweight CNN architectures with good classification accuracy. In this article, we present a contribution in that direction. The proposed method combines the ghost-module architecture with a CNN-based HSI classifier to reduce the computational cost and, simultaneously, achieves an efficient classification method with high performance. Our new method is evaluated against nine standard HSI classifiers, and five improved deep-CNN architectures, over five commonly used HSI data sets for algorithm benchmarking. Conducted experiments show that the proposed method exhibits similar or better performance than the other classifiers, achieving top values in the considered performance metrics—even for very limited training sets—and, most importantly, with a fraction of the computational cost. Our novel approach for HSI classification is a strong candidate for implementation on systems with limited computational resources.

Convolutional neural networks (CNNs) have proven to be a powerful tool for the classification of hyperspectral images (HSIs). The CNN kernels are able to naturally include spatial information to smooth out the spectral variability and the noise present in HSI data. However, these kernels are composed of a large number of learning parameters that must be correctly adjusted to achieve good performance. This forces the model to consume a large amount of training data, being prone to overfitting when limited labeled samples are available. In addition, the execution of kernels is computationally very expensive, increasing quadratically with respect to the size of the convolution filter. This significantly reduces the performance of the model. To overcome the aforementioned limitations, this work presents a new few-parameter CNN (based on shift operations) for HSI classification that dramatically reduces both the number of parameters and the computational complexity of the model in terms of floating-point operations (FLOPs). The operational module combines a shift kernel (which adjusts the input data in particular directions without involving any parameters nor FLOPs) with pointwise convolutions that perform the feature extraction stage. The newly developed shift-based CNN has been employed to conduct HSI classification over five widely used and challenging data sets, achieving very promising results in terms of computational performance and classification accuracy.

Hyperspectral unmixing is an important problem for remotely sensed data interpretation. It amounts at estimating the spectral signatures of the pure spectral constituents in the scene (endmembers) and their corresponding subpixel fractional abundances. Although the unmixing problem is inherently nonlinear (due to multiple scattering), the nonlinear unmixing of hyperspectral data has been a very challenging problem. This is because nonlinear models require detailed knowledge about the physical interactions between the sunlight scattered by multiple materials. In turn, bilinear mixture models (BMMs) can reach good accuracy with a relatively simple model for scattering. In this article, we develop a new BMM and a corresponding unsupervised unmixing approach which consists of two main steps. In the first step, a deep autoencoder is used to linearly estimate the endmember signatures and their associated abundance fractions. The second step refines the initial (linear) estimates using a bilinear model, in which another deep autoencoder (with a low-rank assumption) is adapted to model second-order scattering interactions. It should be noted that in our developed BMM model, the two deep autoencoders are trained in a mutually interdependent manner under the multitask learning framework, and the relative reconstruction error is used as the stopping criterion. The effectiveness of the proposed method is evaluated using both synthetic and real hyperspectral data sets. Our experimental results indicate that the proposed approach can reasonably estimate the nature of nonlinear interactions in real scenarios. Compared with other state-of-the-art unmixing algorithms, the proposed approach demonstrates very competitive performance.

As a powerful coding strategy for superpixels in synthetic aperture radar (SAR) images, Fisher vector (FV) lies in a low-dimensional subspace and can be sparsely represented as a linear combination of training samples. The existing ship detection methods based on FVs often consider the Euclidean distances between target FVs and clutter FVs, where the subspace features of FVs are generally not exploited. In this article, we propose a new ship detection algorithm based on nonnegative sparse locality-representation (NSLR) to exploit the subspace features of FVs. The proposed NSLR method is based on the assumption that FVs of superpixels in SAR images are sparsely represented by the dictionary of background sea clutter only under a null hypothesis. In addition, we propose two FV-based filters to enhance the robustness of our newly developed NSLR to heterogeneous sea clutter environments by further exploiting the intrinsic features of ship targets in terms of intensity and spatiality. The experimental results based on Gaofen-3 SAR images demonstrate that the proposed NSLR detection method provides higher target-to-clutter contrast and achieves better detection performance than other commonly used ship detection algorithms.

Recently, many convolutional neural network (CNN) methods have been designed for hyperspectral image (HSI) classification since CNNs are able to produce good representations of data, which greatly benefits from a huge number of parameters. However, solving such a high-dimensional optimization problem often requires a large number of training samples in order to avoid overfitting. In addition, it is a typical nonconvex problem affected by many local minima and flat regions. To address these problems, in this article, we introduce the naive Gabor networks or Gabor-Nets that, for the first time in the literature, design and learn CNN kernels strictly in the form of Gabor filters, aiming to reduce the number of involved parameters and constrain the solution space and, hence, improve the performances of CNNs. Specifically, we develop an innovative phase-induced Gabor kernel, which is trickily designed to perform the Gabor feature learning via a linear combination of local low-frequency and high-frequency components of data controlled by the kernel phase. With the phase-induced Gabor kernel, the proposed Gabor-Nets gains the ability to automatically adapt to the local harmonic characteristics of the HSI data and, thus, yields more representative harmonic features. Also, this kernel can fulfill the traditional complex-valued Gabor filtering in a real-valued manner, hence making Gabor-Nets easily perform in a usual CNN thread. We evaluated our newly developed Gabor-Nets on three well-known HSIs, suggesting that our proposed Gabor-Nets can significantly improve the performance of CNNs, particularly with a small training set.

In this paper, we propose an improved superpixel segmentation algorithm for ship target detection in synthetic aperture radar (SAR) images, called adaptive Fisher vector-based simple linear iterative clustering (AFVSLIC). Compared with existing algorithms, three new features produced by Fisher vectors, i.e., zero-order, first-order and second-order features, are exploited by the proposed AFVSLIC algorithm to enhance segmentation performance. Besides, AFVSLIC adaptively adjusts the weights of the features to maintain the segmentation performance in different signal-to-clutter ratio (SCR) scenarios. Experimental results demonstrate that the proposed AFVSLIC algorithm outperforms existing, commonly used algorithms for superpixel segmentation and (accordingly) improves the performance of ship target detection.

In recent years, deep learning models have shown great advantages for hyperspectral images (HSIs) classification, in which long short-term memory (LSTM) has attracted plenty of attentions for its characteristic of modeling long-range dependencies. However, for the 2-D extended architecture of it (namely 2-D convolutional LSTM, ConvLSTM2D), it is the special gate structures of ConvLSTM2D that leads to a large number of training parameters and high requirements for device storage. To address this shortcoming, in this paper, a lightweight ConvLSTM2D cell is developed by using tensor-train decomposition (TTD) for the compression of training parameters, which is named TT-ConvLSTM2D and further applied to two state-of-the-art ConvLSTM2D-based HSI classification models for verifying its superiority. Experiments on a widely-used Indian Pines HSI data set are conducted, whose results demonstrate that the proposed TT-ConvLSTM2D cell can effectively reduce the number of the parameters and memory requirements of the whole models within a small range of accuracy degradation.

Social media data have shown great potential for disaster response. However, the inherent limitations associated to these data (particu-larly, the spatial bias) restrict its precise application. In this work, we present a new spatial bias correction method based on remote sensing knowledge and spatio-temporal fusion, named locally optimal transport (LOT). Our method is first tested using a case study (2013 Boulder, Colorado flood event). Then, we apply our method to a 2016 Wuhan flood event to test its accuracy in a data deficient region. Our results show that combining remote sensing features and spatio-temporal fusion can help to address problems with a lack of prior data and limited disaster period data. According to the random ground verification points collected from news, pictures and videos, our new LOT method is able to accurately relocate spatially biased social media data to inundated areas, which are dangerous for users.

Hyperspectral imaging (HSI) gathers hundreds of images along the electromagnetic spectrum for the same area on the surface of the Earth, collecting a rich amount of spatial and spectral information. Deep learning classifiers have achieved significantly high precision results when analyzing HSI data. In particular, capsule networks (CapsNets) can provide robust classification results, overcoming the limitations of traditional convolutional neural networks (CNNs) by enriching the feature presentation capability and applying dynamic routing mechanisms. As a result, CapsNets are now widely regarded as the state-of-the-art within deep learning field. However, as it is the case for CNNs, the performance of CapsNets strongly depends on the quantity and quality of the available training samples, which in HSI tends to be scarce and noisy. Moreover, obtaining labeled data is expensive and time-consuming, and the high dimensionality of HSI data makes it difficult to accurately design classifiers based on limited training samples. This is mainly due to the strong intra-class variability present in the HSI data. Active learning (AL) can alleviate the aforementioned problems by selecting a small set of highly-representative labeled samples from a pool of unlabeled data, in iterative fashion. This paper presents a new AL-based approach for HSI data classification that integrates the spectral and the spatial information contained in the HSI data and enhances the performance of CapsNets when very limited training samples are available. Code: https://github.com/mhaut/AL-CapsNet-HSI.

In this paper, we describe a method for radiometric calibration of Fengyun-3D (FY-3D) MERSI-II TIR, bands 24 and 25 that combines vicarious and cross-calibration of the corresponding bands of Aqua MODIS, bands 31 and 32. A field campaign was conducted on lake Qinghai, China, on August 18, 2019. The surface measurements were performed before and after the passing time of Terra MODIS and FY-3D MERSI-II. Path radiance and transmittance were then calculated via the radiative transfer code MODTRAN 4.0, and the difference caused by the relative spectral response (RSR) of the sensors was eliminated by the spectral matching. Our experimental results indicate that the obtained radiometric calibration accuracy of FY-3D MERSI-II is stable during on-orbit operation.

High-resolution precipitation data are crucial to monitor disasters in urban areas, especially in cases with abundant precipitation. Based on the spatiotemporal, non-stationary relationship between precipitation and normalized differential vegetation index (NDVI), in this paper we introduce a geographically and temporally weighted regression (GTWR) model and further evaluate it in a case study in Guangdong province, China, in the summer of 2015. Our results indicate that there is a mainly negative correlation between precipitation and NDVI in the summer. Our GTWR downscaling model performs better than a previously available geographically weighted regression (GWR) model, providing more accurate downscaled precipitation estimations. This suggests that considering the spatiotemporal, non-stationarity relationship between NDVI and precipitation, our GTWR downscaling model can provide high-spatial resolution precipitation estimates, with more details in urban areas with abundant precipitation.

Hyperspectral unmixing aims to decompose a hyperspectral image (HSI) into a collection of constituent materials, or end-members, and their corresponding abundance fractions. Recently, nonnegative tensor factorization (NTF)-based spectral unmixing methods have attracted significant attention owing to their outstanding performance when representing an HSI without any information loss. However, tensor factorization-based HSI methods do not fully exploit the spatial contextual information present in the scene. Besides, these approaches are sensitive to low signal-to-noise ratio (SNR) in HSIs. To address this limitation, we propose a new spectral-spatial weighted sparse nonnegative tensor factorization (SSWNTF) method to preserve the spatial details in the abundance maps via the spectral and spatial weighting factors. Our experiments with simulated data sets certified that the proposed method outperforms other advanced methods.

This letter presents a novel hyperspectral endmember extraction approach that integrates a tensor-based decomposition scheme with a probabilistic framework in order to take advantage of both technologies when uncovering the signatures of pure spectral constituents in the scene. On the one hand, statistical unmixing models are generally able to provide accurate endmember estimates by means of rather complex optimization algorithms. On the other hand, tensor decomposition techniques are very effective factorization tools which are often constrained by the lack of physical interpretation within the remote sensing field. In this context, this letter develops a new hybrid endmember extraction approach based on the decomposition of the probabilistic tensor moments of the hyperspectral data. Initially, the input image reflectance values are modeled as a collection of multinomial distributions provided by a family of Dirichlet generalized functions. Then, the unmixing process is effectively conducted by the tensor decomposition of the third-order probabilistic tensor moments of the multivariate data. Our experiments, conducted over four hyperspectral data sets, reveal that the proposed approach is able to provide efficient and competitive results when compared to different state-of-the-art endmember extraction methods.

Deep neural networks (DNNs) exhibit great performance in the task of hyperspectral image (HSI) classification. However, these models are usually overparameterized and require large amounts of training data in order to properly avoid the curse of dimensionality and the variability of spectral signatures, thus suffering from overfitting problems when very few training samples are available, due to poor generalization ability in this particular case. The traditional regularization dropout (DO) strategy has been shown to be effective in fully connected DNNs but not in convolutional-based ones. This is mainly due to the way these architectures manage the spatial information. In this letter, we introduce a new approach to improve the generalization of convolutional-based models for HSI classification. Specifically, we develop a neighboring region DO technique that selectively cuts off certain neighboring outputs, creating spatial dropped regions. Our experimental results with two well-known HSIs reveal that the newly proposed method helps to achieve better classification accuracy than the traditional DO strategy, with a low computational cost.

With the development of convolutional neural networks (CNNs), the semantic understanding of remote sensing (RS) scenes has been significantly improved based on their prominent feature encoding capabilities. While many existing deep-learning models focus on designing different architectures, only a few works in the RS field have focused on investigating the performance of the learned feature embeddings and the associated metric space. In particular, two main loss functions have been exploited: the contrastive and the triplet loss. However, the straightforward application of these techniques to RS images may not be optimal in order to capture their neighborhood structures in the metric space due to the insufficient sampling of image pairs or triplets during the training stage and to the inherent semantic complexity of remotely sensed data. To solve these problems, we propose a new deep metric learning approach, which overcomes the limitation on the class discrimination by means of two different components: 1) scalable neighborhood component analysis (SNCA) that aims at discovering the neighborhood structure in the metric space and 2) the cross-entropy loss that aims at preserving the class discrimination capability based on the learned class prototypes. Moreover, in order to preserve feature consistency among all the minibatches during training, a novel optimization mechanism based on momentum update is introduced for minimizing the proposed loss. An extensive experimental comparison (using several state-of-the-art models and two different benchmark data sets) has been conducted to validate the effectiveness of the proposed method from different perspectives, including: 1) classification; 2) clustering; and 3) image retrieval. The related codes of this article will be made publicly available for reproducible research by the community.

Fast and accurate remote sensing image retrieval from large data archives has been an important research topic in the remote sensing research literature. Recently, hashing-based remote sensing image retrieval has attracted extreme attention because of its efficient search capabilities. Especially, deep remote sensing image hashing algorithms have been developed based on convolutional neural networks (CNNs) and have shown effective retrieval performance. However, implementing a deep hashing network tends to be highly expensive in terms of storage space and computing resources to be suitable for on-orbit remote sensing image retrieval, which usually operates on resource-limited devices such as satellites and unmanned aerial vehicles (UAVs). To address this limitation, we propose to hash a deep network that in turn hashes remote sensing images. Specifically, we develop a quantized deep learning to hash (QDLH) framework for large-scale remote sensing image retrieval. The weights and activation functions in the QDLH framework are binarized to low-bit representations, which require comparatively much less storage space and computing resources. The QDLH results in a lightweight deep neural network for effective remote sensing image hashing. We conduct extensive experiments on two public remote sensing image data sets by incorporating several state-of-the-art network architectures into our QDLH methodology for remote sensing image hashing. The experimental results demonstrate that the proposed QDLH is effective in saving hardware resources in terms of both storage and computation. Moreover, superior remote sensing image retrieval performance is also achieved by our QDLH, compared with state-of-the-art deep remote sensing image hashing methods.

Denoising is a common preprocessing step prior to the analysis and interpretation of hyperspectral images (HSIs). However, the vast majority of methods typically adopted for HSI denoising exploit architectures originally developed for grayscale or RGB images, exhibiting limitations when processing high-dimensional HSI data cubes. In particular, traditional methods do not take into account the high spectral correlation between adjacent bands in HSIs, which leads to unsatisfactory denoising performance as the rich spectral information present in HSIs is not fully exploited. To overcome this limitation, this article considers deep learning models-such as convolutional neural networks (CNNs)-to perform spectral-spatial HSI denoising. The proposed model, called HSI single denoising CNN (HSI-SDeCNN), efficiently takes into consideration both the spatial and spectral information contained in HSIs. Experimental results on both synthetic and real data demonstrate that the proposed HSI-SDeCNN outperforms other state-of-the-art HSI denoising methods. Source code: https://github.com/mhaut/HSI-SDeCNN.

Advances in deep learning (DL) have allowed for the development of more complex and powerful neural architectures. The adoption of deep convolutional-based architectures with residual learning [residual networks (ResNets)] has reached the state-of-the-art performance in hyperspectral image (HSI) classification. Traditionally, ResNets have been considered as stacks of discrete layers, where each one obtains a hidden state of the input data. This formulation must deal with very deep networks, which suffer from an important data degradation as they become deeper. Moreover, these complex models exhibit significant requirements in terms of memory due to the amount of parameters that need to be fine tuned. This leads to inadequate generalization and loss of accuracy. In order to address these issues, this article redesigns the ResNet as a continuous-time evolving model, where hidden representations (or states) are obtained with respect to time (understood as the depth of the network) through the evaluation of an ordinary differential equation (ODE), which is combined with a deep neural architecture. Our experimental results, conducted with four well-known HSI data sets, indicate that redefining deep networks as continuous systems through ODEs offers flexibility when processing and classifying these kinds of remotely sensed data, achieving significant performance even when a very few training samples are available.

Most existing endmember extraction techniques require prior knowledge about the number of endmembers in a hyperspectral image. The number of endmembers is normally estimated by a separate procedure, whose accuracy has a large influence on the endmember extraction performance. In order to bridge the two seemingly independent but, in fact, highly correlated procedures, we develop a new endmember estimation strategy that simultaneously counts and extracts endmembers. We consider a hyperspectral image as a hyperspectral pixel set and define the subset of pixels that are most different from one another as the divergent subset (DS) of the hyperspectral pixel set. The DS is characterized by the condition that any additional pixel would increase the likeness within the DS and, thus, reduce its divergent degree. We use the DS as the endmember set, with the number of endmembers being the subset cardinality. To render a practical computation scheme for identifying the DS, we reformulate it in terms of a quadratic optimization problem with a numerical solution. In addition to operating as an endmember estimation algorithm by itself, the DS method can also co-operate with existing endmember extraction techniques by transforming them into a novel and more effective schemes. Experimental results validate the effectiveness of the DS methodology in simultaneously counting and extracting endmembers not only as an individual algorithm but also as a foundation algorithm for improving existing methods. Our full code is released for public evaluation.

Hyperspectral anomaly detection is one of the most active topics in hyperspectral image (HSI) analysis. The fine spectral information of HSIs allows us to uncover anomalies with very high accuracy. Recently, an intrinsic image decomposition (IID) model has been introduced for low-rank IID (LRIID) in multispectral images. Inspired by the LRIID, which is able to effectively recover the reflectance and shading components of the multispectral image, this article adapts the LRIID for obtaining the reflectance component of HSIs (which is the key feature for the discrimination of different objects). In order to exploit the reflectance component, we also propose a new dual window density (DWD)-based detector for anomaly detection, which is based on the idea that anomalies are usually rare pixels and, thus, exhibit low density in the image. The density analysis of DWD is intended not only to circumvent the Gaussian assumption regarding the distribution of HSI data, but also to mitigate the contamination of background statistics caused by anomalies. The dual window operation of our DWD is specifically designed to adaptively calculate the density of each pixel under test, so as to identify anomalies with nonspecific sizes. Our experimental results, obtained on a database of real HSIs including Airport, Beach, and Urban scenes, demonstrate the superiority of the proposed method in terms of detection performance when compared to other widely used anomaly detection methods.

Classification is an important technique for remotely sensed hyperspectral image (HSI) exploitation. Often, the presence of wrong (noisy) labels presents a drawback for accurate supervised classification. In this article, we introduce a new framework for noisy label detection that combines a superpixel-to-pixel weighting distance (SPWD) and density peak clustering. The proposed method is able to accurately detect and remove noisy labels in the training set before HSI classification. It considers two weak assumptions when exploiting the spectral-spatial information contained in the HSI: 1) all the pixels in a superpixel belong to the same class and 2) close pixels in spectral space have the same label. The proposed method consists of the following steps. First, a superpixel segmentation step is used to obtain self-adaptive spatial information for each training sample. Then, a metric is utilized to measure the spectral distance information between each superpixel and pixel. Meanwhile, in order to overcome the first weak assumption, we use K nearest neighbors to obtain the closest neighborhoods of pixels around each superpixel, and a Gaussian weight is employed to mitigate the second weak assumption by adapting the original distance information. Next, the noisy labels in the original training set are removed by a density threshold-based decision function. Finally, the support vector machine (SVM) classifier is employed to evaluate the effectiveness of the proposed SPWD detection method in terms of classification accuracy. Experiments performed on several real HSI data sets demonstrate that the method can effectively improve the performance of classifiers trained with noisy training sets in terms of classification accuracy.

Hyperspectral unmixing (HU) is an active research topic in the remote-sensing community. It aims at modeling mixed pixels using a collection of pure constituent materials (endmembers) weighted by their corresponding fractional abundances. Among existing unmixing schemes, nonnegative matrix factorization (NMF) has drawn significant attention due to its unsupervised nature, as well as its capacity to obtain both endmembers and fractional abundances simultaneously. In this article, we present a new blind unmixing method based on the generalized morphological component analysis (GMCA) framework, in which an additional constraint is introduced into the standard NMF model to represent the sparsity and morphological diversity of the abundance maps associated with each endmember. More specifically, we take into account the fact that different ground categories in a hyperspectral scene generally exhibit various spatial distributions and morphological characteristics. As a result, when providing a specific dictionary basis for these categories, their corresponding abundance maps (referred to as sources) can be sparsely represented. In addition, due to the low correlation between different sources, their sparse representations will not share the same most significant coefficients. With this observation in mind, we can further promote source discrimination and separation in the unmixing process. Moreover, in order to obtain a stable solution of the involved optimization problem, we adopt an alternate iterative constrained algorithm with a threshold descent strategy. Our experiments, carried out on both synthetic and real hyperspectral scenes, reveal that our newly developed GMCA-based unmixing method obtains very promising results with fast convergence speed and requiring significantly less parameter tuning. This confirms the advantage of the proposed spatial morphological component approach for HU purposes.

This paper proposes a novel end-to-end learning model, called skip-connected covariance (SCCov) network, for remote sensing scene classification (RSSC). The innovative contribution of this paper is to embed two novel modules into the traditional convolutional neural network (CNN) model, i.e., skip connections and covariance pooling. The advantages of newly developed SCCov are twofold. First, by means of the skip connections, the multi-resolution feature maps produced by the CNN are combined together, which provides important benefits to address the presence of large-scale variance in RSSC data sets. Second, by using covariance pooling, we can fully exploit the second-order information contained in such multi-resolution feature maps. This allows the CNN to achieve more representative feature learning when dealing with RSSC problems. Experimental results, conducted using three large-scale benchmark data sets, demonstrate that our newly proposed SCCov network exhibits very competitive or superior classification performance when compared with the current state-of-the-art RSSC techniques, using a much lower amount of parameters. Specifically, our SCCov only needs 10% of the parameters used by its counterparts.

The probabilistic Latent Semantic Analysis (pLSA) model has recently shown a great potential to uncover highly descriptive semantic features from limited amounts of remote sensing data. Nonetheless, the high computational cost of this algorithm often constraints its operational application for land cover categorization tasks. In this scenario, this paper presents an Open Multi-Processing (OpenMP) implementation of the pLSA algorithm for unsupervised Synthetic Aperture Radar (SAR) and Multi-Spectral Imaging (MSI) image categorization. The experimental results suggest that multi-core systems are an important architecture for the efficient processing of both SAR and MSI datasets. Specifically, the proposed approach is able to cover a real scenario exhibiting good results in both accuracy and performance terms.

Convolutional neural networks (CNNs) have recently shown remarkable performance in remote sensing scene image classification. However, long-range dependencies (e.g. non-local similarities) within the scene are often ignored by CNNs. To address this issue, in this paper we develop a new high-order self-attention network (HoSA) for remote sensing scene classification. Specifically, we embed two novel modules, i.e., a self-attention module and a high order pooling module, into off-the-shelf CNN models and then fine-tune the whole network. The advantages of our newly proposed HoSA network are twofold. Firstly, with the self-attention module, the HoSA can capture long-range dependencies within the scenes for high-level semantic feature extraction. Secondly, by means of its high-order pooling mechanism, our newly developed HoSA can further explore high-order information contained in the features. Our experiments with a widely used remote sensing scene data set demonstrate that the proposed HoSA network exhibits better classification performance than the baseline and several well-known methods.

In some remote sensing applications such as change detection, satellite images with both high spatial and high temporal resolution are required. However, no single satellite sensor can currently provide such images due to technical specifications. To solve this problem, spatio-temporal fusion provides a cost-effective solution. In this paper, we propose a new spatio-temporal fusion approach, based on convolutional neural networks (CNNs), for Landsat and MODIS image fusion. Specifically, the proposed approach utilizes CNNs to model the heterogeneity of fine pixels from the coarse MODIS images. Here, the heterogeneity of fine pixels is defined as the difference between the reflectance changes obtained from the two types of images. After that, two transition-predicted images can be obtained using the trained CNNs, which are then fused in order to obtain a fi-nal prediction. In our newly proposed approach, CNNs are only used to learn the heterogeneity of fine pixels rather than the whole images, thus providing a more stable and less time-consuming strategy as compared to other available approaches. We evaluated the proposed approach on a public spatio-temporal fusion dataset and the obtained results suggest that our newly developed method achieves state-of-the-art performance.

This work proposes a new collaborative active and semi-supervised learning approach, named accessibility-free active learning (AFAL), for hyperspectral imaging classification. The proposed approach aims to tackle an existing problem in traditional active learning methods, that is, the fact that some selected samples are not accessible by oracles for assigning them pseudo labels, i.e., confident predictions for the classifier. The proposal specifically addresses this problem using superpixels in a self-training context. Specifically, AFAL first generates a set of candidates locally around the labeled pixels and then expands them to other subregions via a density peak-based augmentation strategy, in order to guarantee the confidence of pseudo labels. Our experimental results, obtained on two real and well-used hyperspectral images, reveal that the proposed scheme can lead to state-of-the-art performance.

Deep neural networks (DNNs) have revolutionized the way remotely sensed hyperspectral image (HSI) data are managed and processed. For instance, residual networks (ResNets) have achieved high classification accuracy by applying sequential transformations (layer by layer) on the input HSI data, obtaining highly discriminative data representations. However, these models are quite complex, with significant requirements in terms of memory resulting from the large number of parameters that they need to learn, which also leads to potential overfitting issues. In this work, we specifically address the aforementioned problem by re-interpreting a DNN (the ResNet) as a continuous transformation, instead of the traditional (discrete) step-by-step approach. To achieve this, we combine ordinary differential equations (ODEs) with DNN architectures for the first time in the HSI data classification literature. This allows us to perform remotely sensed HSI data classification in an efficient way in terms of number of parameters. Our experimental results, conducted using two well-known HSI data sets, indicate that the inclusion of ODEs in the architecture of DNNs offers significant advantages when processing and classifying this kind of high-dimensional data, achieving better performance even with less training data.

Nonlinear unmixing of hyperspectral images has been a very challenging research problem, as it needs to consider the physical interactions between the sunlight scattered by multiple materials. In this paper, we propose a new approach for nonlinear unmixing which is based on multi-task learning (MTL) with low-rank matrix factorization (LRMF). The proposed approach establishes two tasks to conduct the unmixing problem under a nonlinear mixing model. In the first task, we employ LRMF to obtain endmember signatures and their corresponding abundance fractions simultaneously. Then, the second task uses LRMF to solve interactions from multiple scattering. The effectiveness of the proposed method is verified by using real hyperspectral data. Compared with other state-of-the-art nonlinear unmixing algorithms, the proposed approach demonstrates very competitive performance.

Sparse representation-based approaches have been successfully applied to remotely sensed hyperspectral image unmixing. In recent years, sparse unmixing techniques have incorporated spatial information into the sparse unmixing model, achieving improved fractional abundance results. Most spatial-based sparse unmixing methods utilize regular-shaped neighborhoods (e.g., a cross or a square window) to characterize the spatial-contextual information around each pixel. However, the spatial characteristics of natural scenes are not always uniform, but vary according to the observed objects. Therefore, assuming uniform spatial neighborhoods may not be consistent with real spatial structures in the scene. Super-pixels offer a good solution to this problem since they can better characterize such spatial structures. Based on this observation, in this paper we develop a new superpixel-guided sparse unmixing (SPGSU) method for hyperspectral scenes. The proposed SPGSU includes the spatial correlation through a superpixel-based technique rather than assuming predefined pixel grids. Each superpixel can be regarded as a small spatial region, whose shape and size can be adaptively changed to accommodate different spatial structures. Our experimental results, conducted using simulated data sets, quantitatively indicate that our newly proposed method produces better results than other advanced spectral unmixing methods.

Soil can be used as a damage indicator of landslides and flooding, which expose soil from vegetation canopy. It can also be used as an indirect indicator of illegal tunnel digging activity. This letter presents a sparsity-based approach to soil detection using multispectral satellite images, where both original and synthetic bands have been used. Spatial and spectral information has then been jointly used in soil detection. Extensive experiments clearly demonstrated the feasibility of our approach.

This letter presents a novel intersensor vegetation estimation framework, which aims at combining Sentinel-2 (S2) spatial resolution with Sentinel-3 (S3) spectral characteristics in order to generate fused vegetation maps. On the one hand, the multispectral instrument (MSI), carried by S2, provides high spatial resolution images. On the other hand, the Ocean and Land Color Instrument (OLCI), one of the instruments of S3, captures the Earth's surface at a substantially coarser spatial resolution but using smaller spectral bandwidths, which makes the OLCI data more convenient to highlight specific spectral features and motivates the development of synergetic fusion products. In this scenario, the approach presented here takes advantage of the proposed constrained probabilistic latent semantic analysis (CpLSA) model to produce intersensor vegetation estimations, which aim at synergically exploiting MSI's spatial resolution and OLCI's spectral characteristics. Initially, CpLSA is used to uncover the MSI reflectance patterns, which are able to represent the OLCI-derived vegetation. Then, the original MSI data are projected onto this higher abstraction-level representation space in order to generate a high-resolution version of the vegetation captured in the OLCI domain. Our experimental comparison, conducted using four data sets, three different regression algorithms, and two vegetation indices, reveals that the proposed framework is able to provide a competitive advantage in terms of quantitative and qualitative vegetation estimation results.

This letter presents a novel intersensor registration framework specially designed to register Sentinel-3 (S3) operational data using the Sentinel-2 (S2) instrument as a reference. The substantially higher resolution of the Multispectral Instrument (MSI), on-board S2, with respect to the Ocean and Land Color Instrument (OLCI), carried by S3, makes the former sensor a suitable spatial reference to finely adjust OLCI products. Nonetheless, the important spectral-spatial differences between both instruments may constrain traditional registration mechanisms to effectively align data of such different nature. In this context, the proposed registration scheme advocates the use of a topic model-based embedding approach to conduct the intersensor registration task within a common multispectral semantic space, where the input imagery is represented according to their corresponding spectral feature patterns instead of the low-level attributes. Thus, the OLCI products can be effectively registered to the MSI reference data by aligning those hidden patterns that fundamentally express the same visual concepts across the sensors. The experiments, conducted over four different S2 and S3 operational data collections, reveal that the proposed approach provides performance advantages over six different intersensor registration counterparts.

Convolutional neural networks have emerged as an excellent tool for remotely sensed hyperspectral image (HSI) classification. Nonetheless, the high computational complexity and energy requirements of these models typically limit their application in on-board remote sensing scenarios. In this context, low-power consumption architectures are promising platforms that may provide acceptable on-board computing capabilities to achieve satisfactory classification results with reduced energy demand. For instance, the new NVIDIA Jetson Tegra TX2 device is an efficient solution for on-board processing applications using deep-learning (DL) approaches. So far, very few efforts have been devoted to exploiting this or other similar computing platforms in on-board remote sensing procedures. This letter explores the use of low-power consumption architectures and DL algorithms for HSI classification. The conducted experimental study reveals that the NVIDIA Jetson Tegra TX2 device offers a good choice in terms of performance, cost, and energy consumption for on-board HSI classification tasks.

This letter introduces a novel remote sensing singleimage superresolution (SR) architecture based on a deep efficient compendium model. The current deep learning-based SR trend stands for using deeper networks to improve the performance. However, this practice often results in the degradation of visual results. To address this issue, the proposed approach harmonizes several different improvements on the network design to achieve state-of-the-art performance when superresolving remote sensing imagery. On the one hand, the proposal combines residual units and skip connections to extract more informative features on both local and global image areas. On the other hand, it makes use of parallelized 1×1 convolutional filters (network in network) to reconstruct the superresolved result while reducing the information loss through the network. Our experiments, conducted using seven different SR methods over the well-known UC Merced remote sensing data set, and two additional GaoFen-2 test images, show that the proposed model is able to provide competitive advantages.

Convolutional neural networks (CNNs) have become a powerful tool for remotely sensed hyperspectral image (HSI) classification due to their great generalization ability and high accuracy.However, owing to the huge amount of parameters that need to be learned and to the complex nature of HSI data itself, these approaches must deal with the important problem of overfitting, which can lead to inadequate generalization and loss of accuracy.In order to mitigate this problem, in this letter we adopt random occlusion, a recently developed data augmentation (DA) method for training CNNs in which the pixels of different rectangular spatial regions in the HSI are randomly occluded, generating training images with various levels of occlusion and reducing the risk of overfitting.Our results with two well-known HSIs reveal that the proposed method helps to achieve better classification accuracy with low computational cost.

Since the 1960s, remote sensing (as an innovative, comprehensive, and interdisciplinary academic area) has been adopted in a wide range of disciplines related to Earth observation, including hydrology, ecology, oceanography, glaciology, geology, military, intelligence, business, economy, and planning [1]-[3]. The constant development of the remote sensing image acquisition technology now allows for the collection of a wide variety of images with different characteristics and resolutions, obtained by remote sensing instruments mounted on spacecraft or aircraft platforms. These images record some type of signal or energy measured from the Earth's surface, which depends on the type of sensor used.

Remote sensing has evolved into a multidisciplinary field involving many different areas such as sensor technology, computing, and advanced applications. Information extraction now plays a fundamental role in the exploitation of the massive amount of data collected by earth observation instruments. In this Point of View, the authors analyze the evolution of this field, identifying three main phases in its development. The first period, which was marked by advances in digital signal processing, led to a significant development of statistical processing methods. The second phase was based on advances in physical models and brought an era of quantitative remote sensing which lasted until the first decade of this century. In the third and current period, information extraction techniques are gradually adopting advanced artificial intelligence models in an effort to cope with the tremendous increase in data volume. This article describes some of these recent advances and addresses challenges caused by the 4Vs (volume, velocity, variety, and veracity) of big data. Finally, the authors offer insight into future directions in this multidisciplinary field.

In the last decades, the problem of target detection has received considerable attention in remote sensing applications. When this problem is tackled using hyperspectral images with hundreds of bands, the use of high-performance computing (HPC) is essential. One of the most popular algorithms in the hyperspectral image analysis community for this purpose is the automatic target detection and classification algorithm (ATDCA). Previous research has already investigated the mapping of ATDCA on HPC platforms such as multicore processors, graphics processing units (GPUs), and field-programmable gate arrays (FPGAs), showing impressive speedup factors (after careful fine-tuning) that allow for its exploitation in time-critical scenarios. However, the lack of standardization resulted in most implementations being too specific to a given architecture, eliminating (or at least making extremely difficult) code reusability across different platforms. In order to address this issue, we present a portability study of an implementation of ATDCA developed using the open computing language (OpenCL). We focus on cross-platform parameters such as performance, energy consumption, and code design complexity, as compared to previously developed (hand-tuned) implementations. Our portability study analyzes different strategies to expose data parallelism as well as enable the efficient exploitation of complex memory hierarchies in heterogeneous devices. We also conduct an assessment of energy consumption and discuss metrics to analyze the quality of our code. The conducted experiments-using synthetic and real hyperspectral data sets collected by the Hyperspectral Digital Imagery Collection Experiment (HYDICE) and NASA's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS)-demonstrate, for the first time in the literature, that portability across different HPC platforms can be achieved for real-time target detection in hyperspectral missions.

The current trend in remote sensing image superresolution (SR) is to use supervised deep learning models to effectively enhance the spatial resolution of airborne and satellite-based optical imagery. Nonetheless, the inherent complexity of these architectures/data often makes these methods very difficult to train. Despite these recent advances, the huge amount of network parameters that must be fine-tuned and the lack of suitable high-resolution remotely sensed imagery in actual operational scenarios still raise some important challenges that may become relevant limitations in the existent earth observation data production environments. To address these problems, we propose a new remote sensing SR approach that integrates a visual attention mechanism within a residual-based network design in order to allow the SR process to focus on those features extracted from land-cover components that require more computations to be superresolved. As a result, the network training process is significantly improved because it aims at learning the most relevant high-frequency information while the proposed architecture allows neglecting the low-frequency features extracted from spatially uninformative earth surface areas by means of several levels of skip connections. Our experimental assessment, conducted using the University of California at Merced and GaoFen-2 remote sensing image collections, three scaling factors, and eight different SR methods, demonstrates that our newly proposed approach exhibits competitive performance in the task of superresolving remotely sensed imagery.

Advances in remote sensing hardware have led to a significantly increased capability for high-quality data acquisition, which allows the collection of remotely sensed images with very high spatial, spectral, and radiometric resolution. This trend calls for the development of new techniques to enhance the way that such unprecedented volumes of data are stored, processed, and analyzed. An important approach to deal with massive volumes of information is data compression, related to how data are compressed before their storage or transmission. For instance, hyperspectral images (HSIs) are characterized by hundreds of spectral bands. In this sense, high-performance computing (HPC) and high-throughput computing (HTC) offer interesting alternatives. Particularly, distributed solutions based on cloud computing can manage and store huge amounts of data in fault-tolerant environments, by interconnecting distributed computing nodes so that no specialized hardware is needed. This strategy greatly reduces the processing costs, making the processing of high volumes of remotely sensed data a natural and even cheap solution. In this paper, we present a new cloud-based technique for spectral analysis and compression of HSIs. Specifically, we develop a cloud implementation of a popular deep neural network for non-linear data compression, known as autoencoder (AE). Apache Spark serves as the backbone of our cloud computing environment by connecting the available processing nodes using a master-slave architecture. Our newly developed approach has been tested using two widely available HSI data sets. Experimental results indicate that cloud computing architectures offer an adequate solution for managing big remotely sensed data sets.

Deep neural networks (DNNs), including convolutional neural networks (CNNs) and residual networks (ResNets) models, are able to learn abstract representations from the input data by considering a deep hierarchy of layers that perform advanced feature extraction. The combination of these models with visual attention techniques can assist with the identification of the most representative parts of the data from a visual standpoint, obtained through more detailed filtering of the features extracted by the operational layers of the network. This is of significant interest for analyzing remotely sensed hyperspectral images (HSIs), characterized by their very high spectral dimensionality. However, few efforts have been conducted in the literature in order to adapt visual attention methods to remotely sensed HSI data analysis. In this paper, we introduce a new visual attention-driven technique for the HSI classification. Specifically, we incorporate attention mechanisms to a ResNet in order to better characterize the spectral-spatial information contained in the data. Our newly proposed method calculates a mask that is applied to the features obtained by the network in order to identify the most desirable ones for classification purposes. Our experiments, conducted using four widely used HSI data sets, reveal that the proposed deep attention model provides competitive advantages in terms of classification accuracy when compared to other state-of-the-art methods.

The classification of hyperspectral images (HSIs) using convolutional neural networks (CNNs) has recently drawn significant attention. However, it is important to address the potential overfitting problems that CNN-based methods suffer when dealing with HSIs. Unlike common natural images, HSIs are essentially three-order tensors which contain two spatial dimensions and one spectral dimension. As a result, exploiting both spatial and spectral information is very important for HSI classification. This paper proposes a new hand-crafted feature extraction method, based on multiscale covariance maps (MCMs), that is specifically aimed at improving the classification of HSIs using CNNs. The proposed method has the following distinctive advantages. First, with the use of covariance maps, the spatial and spectral information of the HSI can be jointly exploited. Each entry in the covariance map stands for the covariance between two different spectral bands within a local spatial window, which can absorb and integrate the two kinds of information (spatial and spectral) in a natural way. Second, by means of our multiscale strategy, each sample can be enhanced with spatial information from different scales, increasing the information conveyed by training samples significantly. To verify the effectiveness of our proposed method, we conduct comprehensive experiments on three widely used hyperspectral data sets, using a classical 2-D CNN (2DCNN) model. Our experimental results demonstrate that the proposed method can indeed increase the robustness of the CNN model. Moreover, the proposed MCMs+2DCNN method exhibits better classification performance than other CNN-based classification strategies and several standard techniques for spectral-spatial classification of HSIs.

Convolutional neural networks (CNNs) have recently exhibited an excellent performance in hyperspectral image classification tasks. However, the straightforward CNN-based network architecture still finds obstacles when effectively exploiting the relationships between hyperspectral imaging (HSI) features in the spectral-spatial domain, which is a key factor to deal with the high level of complexity present in remotely sensed HSI data. Despite the fact that deeper architectures try to mitigate these limitations, they also find challenges with the convergence of the network parameters, which eventually limit the classification performance under highly demanding scenarios. In this paper, we propose a new CNN architecture based on spectral-spatial capsule networks in order to achieve a highly accurate classification of HSIs while significantly reducing the network design complexity. Specifically, based on Hinton's capsule networks, we develop a CNN model extension that redefines the concept of capsule units to become spectral-spatial units specialized in classifying remotely sensed HSI data. The proposed model is composed by several building blocks, called spectral-spatial capsules, which are able to learn HSI spectral-spatial features considering their corresponding spatial positions in the scene, their associated spectral signatures, and also their possible transformations. Our experiments, conducted using five well-known HSI data sets and several state-of-the-art classification methods, reveal that our HSI classification approach based on spectral-spatial capsules is able to provide competitive advantages in terms of both classification accuracy and computational time.

Convolutional neural networks (CNNs) exhibit good performance in image processing tasks, pointing themselves as the current state-of-the-art of deep learning methods. However, the intrinsic complexity of remotely sensed hyperspectral images still limits the performance of many CNN models. The high dimensionality of the HSI data, together with the underlying redundancy and noise, often makes the standard CNN approaches unable to generalize discriminative spectral-spatial features. Moreover, deeper CNN architectures also find challenges when additional layers are added, which hampers the network convergence and produces low classification accuracies. In order to mitigate these issues, this paper presents a new deep CNN architecture specially designed for the HSI data. Our new model pursues to improve the spectral-spatial features uncovered by the convolutional filters of the network. Specifically, the proposed residual-based approach gradually increases the feature map dimension at all convolutional layers, grouped in pyramidal bottleneck residual blocks, in order to involve more locations as the network depth increases while balancing the workload among all units, preserving the time complexity per layer. It can be seen as a pyramid, where the deeper the blocks, the more feature maps can be extracted. Therefore, the diversity of high-level spectral-spatial attributes can be gradually increased across layers to enhance the performance of the proposed network with the HSI data. Our experiments, conducted using four well-known HSI data sets and 10 different classification techniques, reveal that our newly developed HSI pyramidal residual model is able to provide competitive advantages (in terms of both classification accuracy and computational time) over the state-of-the-art HSI classification methods.

Spectral unmixing is a technique for remotely sensed image interpretation that expresses each (possibly mixed) pixel as a combination of pure spectral signatures (endmembers) and their fractional abundances. In this paper, we develop a new technique for unsupervised unmixing which is based on a deep autoencoder network (DAEN). Our newly developed DAEN consists of two parts. The first part of the network adopts stacked autoencoders (SAEs) to learn spectral signatures, so as to generate a good initialization for the unmixing process. In the second part of the network, a variational autoencoder (VAE) is employed to perform blind source separation, aimed at obtaining the endmember signatures and abundance fractions simultaneously. By taking advantage from the SAEs, the robustness of the proposed approach is remarkable as it can unmix data sets with outliers and low signal-to-noise ratio. Moreover, the multihidden layers of the VAE ensure the required constraints (nonnegativity and sum-to-one) when estimating the abundances. The effectiveness of the proposed method is evaluated using both synthetic and real hyperspectral data. When compared with other unmixing methods, the proposed approach demonstrates very competitive performance.

The large amount of data produced by satellites and airborne remote sensing instruments has posed important challenges to efficient and scalable processing of remotely sensed data in the context of various applications. In this paper, we propose a new big data framework for processing massive amounts of remote sensing images on cloud computing platforms. In addition to taking advantage of the parallel processing abilities of cloud computing to cope with large-scale remote sensing data, this framework incorporates task scheduling strategy to further exploit the parallelism during the distributed processing stage. Using a computation- and data-intensive pan-sharpening method as a study case, the proposed approach starts by profiling a remote sensing application and characterizing it into a directed acyclic graph (DAG). With the obtained DAG representing the application, we further develop an optimization framework that incorporates the distributed computing mechanism and task scheduling strategy to minimize the total execution time. By determining an optimized solution of task partitioning and task assignments, high utilization of cloud computing resources and accordingly a significant speedup can be achieved for remote sensing data processing. Experimental results demonstrate that the proposed framework achieves promising results in terms of execution time as compared with the traditional (serial) processing approach. Our results also show that the proposed approach is scalable with regard to the increasing scale of remote sensing data.

Fine-tuning of pretrained convolutional neural networks (CNNs) has been proven to be an effective strategy for remote sensing image scene classification, particularly when a limited number of labeled data sets are available for training purposes. However, such a fine-tuning process often needs that the input images are resized into a fixed size to generate input vectors of the size required by fully connected layers (FCLs) in the pretrained CNN model. Such a resizing process often discards key information in the scenes and thus deteriorates the classification performance. To address this issue, in this paper, we introduce a scale-free CNN (SF-CNN) for remote sensing scene classification. Specifically, the FCLs in the CNN model are first converted into convolutional layers, which not only allow the input images to be of arbitrary sizes but also retain the ability to extract discriminative features using a traditional sliding-window-based strategy. Then, a global average pooling (GAP) layer is added after the final convolutional layer so that input images of arbitrary size can be mapped to feature maps of uniform size. Finally, we utilize the resulting feature maps to create a new FCL that is fed to a softmax layer for final classification. Our experimental results conducted using several real data sets demonstrate the superiority of the proposed SF-CNN method over several well-known classification methods, including pretrained CNN-based ones.

Land-cover classification with hyperspectral imagery has been an active topic in the remote sensing community. It aims at relating a unique class label to each pixel in the scene, so that it can be well defined by a given land cover type. In this paper, we explore the intrinsic characteristics of hyperspectral imagery from a subpixel-level perspective and propose a new subpixel component analysis (SCA) approach for feature extraction and land-cover classification. The core idea of SCA is that we extract a subpixel attribute component feature from the abundance maps. Compared with the abundance maps, the extracted subpixel feature image shows higher signal-to-noise level and clearer spatial distribution details. In order to deal with spectral variability, as well as obtain representative image endmember signatures and their corresponding abundance maps, we adopt a regional clustering-based spatial preprocessing (RCSPP) strategy for endmember identification, and a partial unmixing model based on mixture tuned matched filtering (MTMF) for abundance estimation. Furthermore, to highlight the spatial distribution details as well as eliminate the noise disturbance in the derived abundance maps, we perform sparse image decomposition on the obtained abundance maps, thus achieving a new subpixel feature representation for classification. Our experimental results reveal that the proposed SCA approach can obtain feature representation with explicit physical meaning, clear spatial distribution details, and better noise robustness, leading to state-of-the-art classification results.

The relatively recent availability of operational products from Sentinel-2 and Sentinel-3 missions gives widespread opportunities to combine data collected from different sensors in order to provide products of a higher processing level. Nonetheless, the availability of these products may be affected by multiple factors, such as cloud occlusions, band saturation, geolocation errors or even misaligned detectors. All these anomalies affecting remote sensing data may eventually limit the accessibility to fused products because some of the required information may become partially unavailable for specific areas of interest. In this scenario, the work presented here aims at analyzing the effectiveness of several state-of-the-art regression models in order to restore Sentinel-3 products with partial anomalies from Sentinel-2 integral data. In particular this work investigates three regression methods, two linear-regression method and a non-linear artificial neural networks based method. Obtained results prove that the nonlinear approach and linear RIDGE method are able to carry out a good estimation of S3 from S2 data.

During recent years, many regularization techniques have been proposed to deal with ill-posed problems related to hyperspectral image classification, in which the limited number of training samples contrasts with the very high spectral dimensionality. However, the intrinsic structure of a hyperspectral image often depends on the specific scene and spectrometer, although regularizers like Ridge, LASSO, etc, have been widely used in practical applications. Instead of imposing these regularizers to the probabilistic output of a classifier, this work evaluates the use of extreme learning machines (ELM) with output weights of a single-hidden layer feed-forward neural network (SLFN) regularized with Ridge and LASSO priors, respectively. Experimental results with several real hyperspectral images are conducted to compare the performance and adaptation of these two regularizers with the the original ELM in classification scenarios.

In this paper, a covariance matrix based feature fusion (CMF-F) framework is proposed to combine two low-level visual features i.e., the Gabor feature and color feature for scene classification. Generally, the proposed method consists of following three steps. Firstly, the Gabor feature and color feature are extracted from original image and stacked together. Then, a covariance matrix is extracted to fuse these two low-level visual features. Each nondiagonal entry in the covariance matrix stands for the correlation of two different feature dimensions. Finally, the obtained covariance matrix is handled by a kernel linear discriminative analysis algorithm followed with nearest neighboring classifier for label assignment. The proposed method is tested on a public 21-classes UC Merced land use data set and compared with mid-level visual feature oriented method and the high-level feature oriented methods. The experimental results demonstrate that the proposed CMFF framework can not only improve the classification performance of the low-level visual feature (the Gabor feature and the color feature), but also can outperform the conventional mid-level visual feature oriented methods.

Computational advances have allowed for the development of deep learning (DL) applied to remote sensing data and, particularly, to hyperspectral image (HSI) classification. Deeper architectures are able to establish a better separation of the characteristics of the data, allowing for a better and accurate performance. However, it is known that employing very deep architectures with many abstraction levels can result in a loss of information due to the fact that deep networks often normalize each data individually, without considering the set of adjacent data. To address this issue, this paper implements a self-normalizing neural network (SNN) in order to extract high-level abstract representations without losing information due to the data initialization. The selected activation function (scaled exponential linear units or SELU) normalizes the data considering their neighborhood's information and a special dropout technique (a-dropout), obtaining good classification performance while maintaining the data characteristics across the successive layers. Obtained results show that the proposal improves the performance with few training samples.

In this paper, we propose a deep auto-encoder network for the unmixing for hyperspectral data with outliers and low signal to noise ratio. The proposed deep auto-encoder network composes of two parts. The first part of the network adopts stacked non-negative sparse auto-encoder to learn the spectral signatures such that to generate a good initialization for the network. In the second part of the network, a variational auto-encoder is employed to perform unmixing, aiming at the endmember signatures and abundance fractions. The effectiveness of the proposed method is verified by using a synthetic data set. In our comparison with other state-of-the-art unmixing methods, the proposed approach demonstrates highly competitive performance.

Solving the mixture problem in remotely sensed hyperspectral images remains a challenging task. In particular, solutions are needed in order to obtain a response for applications with real-time constraints. In the last decade, several efforts have been developed, many of them using graphics processing units (GPUs) and focused on the exploitation of spectral information alone. However, a few spectral unmixing chains have been developed using other architectures such as multicore processors, field programmable gate arrays, or Intel Xeon Phi coprocessors. In this letter, we develop a new parallel unmixing chain for multicore processors. Compared with other approaches, the proposed spatial-spectral alternative takes advantage of the complementary information provided by the spatial correlation of the pixels in the image in addition to the spectral information. Our implementation has been optimized using the application program interface OpenMP and the Intel Math Kernel Library on two multicore architectures, and using real analysis scenarios. The results reveal competitive real-time performance compared with another compute unified device architecture implementation previously developed for GPUs.

Probabilistic topic models have recently shown a great potential in the remote sensing image fusion field, which is particularly helpful in land-cover categorization tasks. This letter first studies the application of probabilistic latent semantic analysis (pLSA) and latent Dirichlet allocation to remote sensing synthetic aperture radar (SAR) and multispectral imaging (MSI) unsupervised land-cover categorization. Then, a novel pLSA-based image fusion approach is presented, which pursues to uncover multimodal feature patterns from SAR and MSI data in order to effectively fuse and categorize Sentinel-1 and Sentinel-2 remotely sensed data. Experiments conducted over two different data sets reveal the advantages of the proposed approach for unsupervised land-cover categorization tasks.

In this paper, a novel local covariance matrix (CM) representation method is proposed to fully characterize the correlation among different spectral bands and the spatial-contextual information in the scene when conducting feature extraction (FE) from hyperspectral images (HSIs). Specifically, our method first projects the HSI into a subspace, using the maximum noise fraction method. Then, for each test pixel in the subspace, its most similar neighboring pixels (within a local spatial window) are clustered using the cosine distance measurement. The test pixel and its neighbors are used to calculate a local CM for FE purposes. Each nondiagonal entry in the matrix characterizes the correlation between different spectral bands. Finally, these matrices are used as spatial-spectral features and fed to a support vector machine for classification purposes. The proposed method offers a new strategy to characterize the spatial-spectral information in the HSI prior to classification. Experimental results have been conducted using three publicly available hyperspectral data sets for classification, indicating that the proposed method can outperform several state-of-the-art techniques, especially when the training samples available are limited.

Hyperspectral unmixing is an important processing step for many hyperspectral applications, mainly including: 1) estimation of pure spectral signatures (endmembers) and 2) estimation of the abundance of each endmember in each pixel of the image. In recent years, nonnegative matrix factorization (NMF) has been highly attractive for this purpose due to the nonnegativity constraint that is often imposed in the abundance estimation step. However, most of the existing NMF-based methods only consider the information in a single layer while neglecting the hierarchical features with hidden information. To alleviate such limitation, in this paper, we propose a new sparsity-constrained deep NMF with total variation (SDNMF-TV) technique for hyperspectral unmixing. First, by adopting the concept of deep learning, the NMF algorithm is extended to deep NMF model. The proposed model consists ofpretraining stageandfine-tuning stage, where the former pretrains all factors layer by layer and the latter is used to reduce the total reconstruction error. Second, in order to exploit adequately the spectral and spatial information included in the original hyperspectral image, we enforce two constraints on the abundance matrix. Specifically, the$L_{1/2}$constraint is adopted, since the distribution of each endmember is sparse in the 2-D space. The TV regularizer is further introduced to promote piecewise smoothness in abundance maps. For the optimization of the proposed model, multiplicative update rules are derived using the gradient descent method. The effectiveness and superiority of the SDNMF-TV algorithm are demonstrated by comparing with other unmixing methods on both synthetic and real data sets.

This paper presents a novel approach for spectral unmixing of remotely sensed hyperspectral data. It exploits probabilistic latent topics in order to take advantage of the semantics pervading the latent topic space when identifying spectral signatures and estimating fractional abundances from hyperspectral images. Despite the contrasted potential of topic models to uncover image semantics, they have been merely used in hyperspectral unmixing as a straightforward data decomposition process. This limits their actual capabilities to provide semantic representations of the spectral data. The proposed model, called dual-depth sparse probabilistic latent semantic analysis (DEpLSA), makes use of two different levels of topics to exploit the semantic patterns extracted from the initial spectral space in order to relieve the ill-posed nature of the unmixing problem. In other words, DEpLSA defines a first level of deep topics to capture the semantic representations of the spectra, and a second level of restricted topics to estimate endmembers and abundances over this semantic space. An experimental comparison in conducted using the two standard topic models and the seven state-of-the-art unmixing methods available in the literature. Our experiments, conducted using four different hyperspectral images, reveal that the proposed approach is able to provide competitive advantages over available unmixing approaches.

Super-resolution (SR) brings an excellent opportunity to improve a wide range of different remote sensing applications. SR techniques are concerned about increasing the image resolution while providing finer spatial details than those captured by the original acquisition instrument. Therefore, SR techniques are particularly useful to cope with the increasing demand remote sensing imaging applications requiring fine spatial resolution. Even though different machine learning paradigms have been successfully applied in SR, more research is required to improve the SR process without the need of external high-resolution (HR) training examples. This paper proposes a new convolutional generator model to super-resolve low-resolution (LR) remote sensing data from an unsupervised perspective. That is, the proposed generative network is able to initially learn relationships between the LR and HR domains throughout several convolutional, downsampling, batch normalization, and activation layers. Then, the data are symmetrically projected to the target resolution while guaranteeing a reconstruction constraint over the LR input image. An experimental comparison is conducted using 12 different unsupervised SR methods over different test images. Our experiments reveal the potential of the proposed approach to improve the resolution of remote sensing imagery.

Hyperspectral imaging is a widely used technique in remote sensing in which an imaging spectrometer collects hundreds of images (at different wavelength channels) for the same area on the surface of the earth. In the last two decades, several methods (unsupervised, supervised, and semisupervised) have been proposed to deal with the hyperspectral image classification problem. Supervised techniques have been generally more popular, despite the fact that it is difficult to collect labeled samples in real scenarios. In particular, deep neural networks, such as convolutional neural networks (CNNs), have recently shown a great potential to yield high performance in the hyperspectral image classification. However, these techniques require sufficient labeled samples in order to perform properly and generalize well. Obtaining labeled data is expensive and time consuming, and the high dimensionality of hyperspectral data makes it difficult to design classifiers based on limited samples (for instance, CNNs overfit quickly with small training sets). Active learning (AL) can deal with this problem by training the model with a small set of labeled samples that is reinforced by the acquisition of new unlabeled samples. In this paper, we develop a new AL-guided classification model that exploits both the spectral information and the spatial-contextual information in the hyperspectral data. The proposed model makes use of recently developed Bayesian CNNs. Our newly developed technique provides robust classification results when compared with other state-of-the-art techniques for hyperspectral image classification.

This paper proposes a new method, called multilayer stacked covariance pooling (MSCP), for remote sensing scene classification. The innovative contribution of the proposed method is that it is able to naturally combine multilayer feature maps, obtained by pretrained convolutional neural network (CNN) models. Specifically, the proposed MSCP-based classification framework consists of the following three steps. First, a pretrained CNN model is used to extract multilayer feature maps. Then, the feature maps are stacked together, and a covariance matrix is calculated for the stacked features. Each entry of the resulting covariance matrix stands for the covariance of two different feature maps, which provides a natural and innovative way to exploit the complementary information provided by feature maps coming from different layers. Finally, the extracted covariance matrices are used as features for classification by a support vector machine. The experimental results, conducted on three challenging data sets, demonstrate that the proposed MSCP method can not only consistently outperform the corresponding single-layer model but also achieve better classification performance than other pretrained CNN-based scene classification methods.

Superpixels are a powerful device to characterize the spatial-contextual information in remotely sensed hyperspectral image (HSI) interpretation. However, the exploitation of superpixels in classification problems is not straightforward, often leading to unbearable NP-hard discrete integer optimization problems. In this paper, we attack this hurdle by leveraging on a convex relaxation of the original integer optimization problem, which opens the door to include oversegmented superpixel-based regularizers. Specifically, we develop a new method for generating oversegmented superpixels. Then, we introduce a family of convex regularizers in the form of graph total variation, which promotes the same labeling in each superpixel. Vectorial total variation is also included in order to promote piecewise smoothness and align discontinuities along the class boundaries. The solution of the obtained convex optimization problem is computed with the split-augmented Lagrangian shrinkage algorithm. Experiments on HSIs yield classification maps with precise boundaries and inner consistency inside oversegmented superpixels, leading to the state-of-the-art classification accuracies.

Accurately estimating the elements in Earth observations is crucial when assessing specific features such as air quality index, water pollution, or urbanization process behavior. Moreover, physical-chemical composition can be retrieved from hyperspectral images when proper spectral unmixing architectures are employed. Specifically, when linear and nonlinear combinations of endmembers (pure spectral components) are accurately characterized, hyperspectral unmixing plays a key role in understanding and quantifying phenomena occurring over the instantaneous field-of-view. Thus, reliable detection of nonlinear reflectance behavior can play a key role in enhancing hyperspectral unmixing performance. In this paper, two new methods for adaptive design of mixture models for hyperspectral unmixing are introduced. One of the methods relies on exploiting geometrical features of hyperspectral signatures in terms of nonorthogonal projections onto the space induced by the endmembers' spectra. Then, an iterative process aims at understanding the order of local nonlinearity that is displayed by each endmember over every pixel. An improved version of an artificial neural network-based approach for nonlinearity order information is also considered and compared. Experimental results show that the proposed approaches are actually able to retrieve thorough information on the nature of the nonlinear effects over the image, while providing excellent performance in reconstructing the given data sets.

In this paper, a new joint spectral-spatial subpixel mapping model is proposed for hyperspectral remotely sensed imagery. Conventional approaches generally use an intermediate step based on the derivation of fractional abundance maps obtained after a spectral unmixing process, and thus the rich spectral information contained in the original hyperspectral data set may not be utilized fully. In this paper, a concept of subpixel abundance map, which calculates the abundance fraction of each subpixel to belong to a given class, was introduced. This allows us to directly connect the original (coarser) hyperspectral image with the final subpixel result. Furthermore, the proposed approach incorporates the spectral information contained in the original hyperspectral imagery and the concept of spatial dependence to generate a final subpixel mapping result. The proposed approach has been experimentally evaluated using both synthetic and real hyperspectral images, and the obtained results demonstrate that the method achieves better results when compared to other seven subpixel mapping methods. The numerical comparisons are based on different indexes such as the overall accuracy and the CPU time. Moreover, the obtained results are statistically significant at 95% confidence.

Spectral unmixing aims at estimating the fractional abundances of a set of pure spectral materials (endmembers) in each pixel of a hyperspectral image. The wide availability of large spectral libraries has fostered the role of sparse regression techniques in the task of characterizing mixed pixels in remotely sensed hyperspectral images. A general solution for sparse unmixing methods consists of using the l2regularizer to control the sparsity, resulting in a very promising performance but also suffering from sensitivity to large and small sparse coefficients. A recent trend to address this issue is to introduce weighting factors to penalize the nonzero coefficients in the unmixing solution. While most methods for this purpose focus on analyzing the hyperspectral data by considering the pixels as independent entities, it is known that there exists a strong spatial correlation among features in hyperspectral images. This information can be naturally exploited in order to improve the representation of pixels in the scene. In order to take advantage of the spatial information for hyperspectral unmixing, in this paper, we develop a new spectral-spatial weighted sparse unmixing (S2WSU) framework, which uses both spectral and spatial weighting factors, further imposing sparsity on the solution. Our experimental results, conducted using both simulated and real hyperspectral data sets, illustrate the good potential of the proposed S2WSU, which can greatly improve the abundance estimation results when compared with other advanced spectral unmixing methods.

Spectral unmixing is an important technique for remotely sensed hyperspectral image interpretation, of which the goal is to decompose the image into a set of pure spectral components (endmembers) and their abundance fractions in each pixel of the scene. Sparse-representation-based approaches have been widely studied for remotely sensed hyperspectral unmixing. A recent trend is to incorporate the spatial information to improve the spectral unmixing results. Those methods generally assume that the abundances of the pixels are piecewise smooth and fall into a homogeneous region occupied by the same endmembers and their corresponding fractional abundances. However, in real scenarios, abundances may vary abruptly from pixel to pixel. Therefore, the former assumption in most spatial models does not hold. To address this limitation, we propose a new strategy to preserve the spatial details in the abundance maps via a spatial discontinuity weight. Our experimental results, conducted with both simulated and real hyperspectral data sets, illustrate the good potential of our discontinuity-preserving strategy for sparse unmixing, which can greatly improve the abundance estimation results.

The synergistic analysis of light detection and ranging (LiDAR) and hyperspectral data is attracting a significant interest in recent years due to the complementary nature of these two sources of remote sensing data. In this paper, we propose a new spectral-spatial classification method able to jointly exploit these two kinds of data. Our work is based on three innovative components: 1) a superpixel generation method aimed at multivariate image spatial partitioning, 2) a multi-source framework for feature extraction, and 3) a convex framework used to approach the solutions of the resulted image labeling problem associated with vectorial total variation and superpixel-based graph total variation regularizers. Our experimental results, conducted with a hyperspectral data set collected by the Compact Airborne Spectrographic Imager (CASI) spectrometer over the city of Houston in 2013 and a corresponding LiDAR data set, illustrate the effectiveness of the proposed framework.

This work introduces a cloud shadow removal method for hyperspectral images (HSIs) based on hyperspectral unmixing. The shading is modeled by a spectral offset and a spectral-dependent attenuation. The offset and the attenuation are estimated by solving a nonconvex optimization problem, which exploits the linear mixing model (LMM). The mixing matrix of the LMM is estimated from the unshadowed image areas. The effectiveness of the proposed method is assessed from classification results of the Houston 2013 (Compact Airborne Spectrographic Imager (CASI) spectrometer VHR HS), whose shadowed areas were removed with the proposed method. The obtained results indicate classification performances in the shadowed areas very close to those of the unshadowed ones, thus providing evidence of the effectiveness of the proposed shading removal technique.

Artificial neural networks (ANNs) have been widely used in the analysis of remotely sensed imagery. In particular, convolutional neural networks (CNNs) are gaining more and more attention. Unlike traditional CNNs methods, where the relevant information to classify the elements of a remotely sensed image is extracted only from the last fully-connected layer, the new adaptive deep pyramid matching (ADPM) model [1] takes advantage of the features from all of the convolutional layers. This model allows the optimal fusing weights for different convolutional layers be learned from the data itself. In addition, the combination of CNNs with spatial pyramid pooling (SPP-net) to create the basic deep network allows the use of images with multiple scales, which results in better learning process thanks to the complementary information. The original ADPM method is divided in two parts: the multi-scale deep feature extraction and the ADPM core. In this paper we present a computational improvement of the ADPM core, coding a parallel-multicore version. This strategy is shown to significantly enhance performance in the analysis of remotely sensed data.

The finer spatial, spectral and radiometric resolutions of current and planned sensors are rendering increasingly-high data rates which, coupled with limited on-board storage, downlink bandwidth and receiving ground station availability, make high-throughput, high-performance data-reduction techniques essential in forthcoming missions. On this paper we describe an algorithm well suited to high-dimensional data as those produced by multispectral and hyperspectral sensors, both highly relevant in a broad range of Earth Observation activities with the latter becoming increasingly available and delivering the highest data rates. The performance of parallel implementations of the algorithm on multi-core and GPU architectures is also evaluated.

Endmember extraction is a fundamental task in spectral unmixing of remotely sensed hyperspectral images. In this work, we develop a new robust algorithm for endmember extraction which is based on a nonnegative sparse autoencoder. The proposed approach is based on two main steps. First, it uses an automatic sampler approach with local outlier factor and affinity propagation to intelligently gather a set of training samples. Then, a set of endmember signatures are extracted from the selected training samples by the nonnegative sparse autoencoder. Taking advantage from both automatic sampling and nonnegative sparse autoencoding, the proposed method can tackle problems with outliers. The effectiveness of the proposed method is verified by using simulated data. In our comparison with other state-of-the-art endmember extraction methods, the proposed approach demonstrates highly competitive performance.

Abundance information has been recently used to assist hyperspectral image classification by combining the information coming from classification and unmixing. The fact that classes are usually inconsistent with endmembers makes it a crucial issue to find possible connections between classification and unmixing. This paper describes a new class-based endmember extraction and sparse unmixing approach aimed at establishing the correspondence between endmembers and classes. The proposed approach is exploited in a semisupervised classification framework that combines classification and unmixing with active learning (AL). During the AL process, the class probabilities and abundance information are exploited simultaneously to select the most informative unlabeled samples for classification purposes. Our approach adopts a well-established discriminative probabilistic classifier, the multinomial logistic regression (MLR), to learn the class posterior probabilities. The effectiveness of the proposed method is evaluated using real hyperspectral data set collected by the NASA Jet Propulsion Laboratory's Airborne Visible Infrared Imaging Spectrometer (AVIRIS) over the Indian Pines region, Indiana.

Sparse unmixing of hyperspectral data is an important technique which aims at estimating the fractional abundances of endmembers (pure spectral components). It is well known that enforcing sparseness becomes a necessary process in sparse unmixing methods. To better exploit the sparsity in hyperspectral imagery, a double reweighted sparse unmixing algorithm has been proposed. However, it focusses on analyzing the hyperspectral data without fully incorporating the spatial information. To address this limitation, a spatial weighted sparse unmixing (SWSU) algorithm is proposed in this paper, which can take full advantage of the spatial information and further enhance the sparsity of the abundances. This is done by incorporating local neighborhood weights into the double reweighted sparse unmixing formulation. Experimental results on simulated hyperspectral data sets illustrate the good potential of the spatial weighted strategy for sparse unmixing introduced in this paper, which can greatly improve abundance estimation results.

Morphological attribute profiles (MAPs) are one of the most effective methodologies to characterize the spatial information in remote sensing images. This technique extracts components able to accurately describe objects in the surface of the Earth. In this work, we present a new method for impervious surface extraction from multispectral images using morphological attribute profiles. The proposed method first uses morphological profiles to extend Landsat ETM+ images with additional features. Then, we adopt a vegetation-impervious surface-soil (V-I-S) model and extract three pure classes (endmembers) from these images (i.e. vegetation, impervious surface and soil) using the vertex component algorithm (VCA). Finally, linear spectral mixture analysis (SMA) is conducted to extract the impervious surface percentage (ISP). To test the performance of the proposed method, more than 300 test samples including business districts, residential areas and urban roads are randomly selected from QuickBird imagery with very high resolution. The coefficient of determination R2is 0.7571, which significantly outperformed other standard techniques in the literature. The obtained experimental results demonstrate that the proposed approach based on morphological attribute profiles can lead to very good extraction and characterization of impervious surfaces.

Over the last few years, several new strategies for spectral unmixing of remotely sensed hyperspectral data have been proposed. Many of them have been developed to solve the most time-consuming and relevant step: endmember extraction. However, unmixing algorithms can be computationally very expensive in terms of processing time and energy consumption, a fact that compromises their use in applications under real-time and energy/power constraints. In this letter, we present a new parallel simplex growing algorithm (SGA) for hyperspectral data which exploits the memory hierarchy with operations in single-precision floating point. Those optimizations accelerate the most time-consuming parts of this method using the open computing language (OpenCL) standard. We have evaluated the performance versus energy consumption using the same open standard for parallel programming over a diverse set of heterogeneous platforms. Experiments have been conducted using real hyperspectral images collected by NASA's Airborne Visible Infrared Imaging Spectrometer and a collection of 24 synthetic hyperspectral images simulated with different sizes and number of endmembers (10-30). Considering the power consumption and OpenCL across all the proposed devices, the analysis presented indicates that the SGA can now be executed in computationally efficient fashion, which was not possible before introducing the parallel implementation described in this letter.

Oil spill monitoring in optical remote sensing (RS) images is a challenging task due to the complexity of target discrimination in an oil spill scenario. Differently from traditional oil spill detection methods that are mainly carried out in a monotemporal image, in this letter, a novel solution is given in a multitemporal domain by investigating potential capability of change detection (CD) techniques, and it mainly contributes to an unsupervised, semiautomatic, and efficient approach. It opens a new perspective for solving an oil spill detection problem. In particular, a coarse-to-fine multitemporal change analysis procedure is designed to investigate the spectral–temporal variation of change targets present in the scenario. Changes relevant and irrelevant to suspected oil spills are identified and discriminated according to a binary and a multiple CD process, respectively. The proposed approach provides a quick yet effective oil spill detection solution, which is valuable and important in practical applications. The proposed method was validated on two real multitemporal RS data sets presenting the oil spill event in northern Gulf of Mexico in 2010. Experimental results confirmed its effectiveness.

Spectral unmixing is an important technique in hyperspectral image applications. Recently, sparse regression has been widely used in hyperspectral unmixing, but its performance is limited by the high mutual coherence of spectral libraries. To address this issue, a new sparse unmixing algorithm, called double reweighted sparse unmixing and total variation (TV), is proposed in this letter. Specifically, the proposed algorithm enhances the sparsity of fractional abundances in both spectral and spatial domains through the use of double weights, where one is used to enhance the sparsity of endmembers in spectral library, and the other is introduced to improve the sparsity of fractional abundances. Moreover, a TV-based regularization is further adopted to explore the spatial-contextual information. As such, the simultaneous utilization of both double reweighted l1minimization and TV regularizer can significantly improve the sparse unmixing performance. Experimental results on both synthetic and real hyperspectral data sets demonstrate the effectiveness of the proposed algorithm both visually and quantitatively.

Robust techniques such as sparse subspace clustering (SSC) have been recently developed for hyperspectral images (HSIs) based on the assumption that pixels belonging to the same land-cover class approximately lie in the same subspace. In order to account for the spatial information contained in HSIs, SSC models incorporating spatial information have become very popular. However, such models are often based on a local averaging constraint, which does not allow for a detailed exploration of the spatial information, thus limiting their discriminative capability and preventing the spatial homogeneity of the clustering results. To address these relevant issues, in this letter, we develop a new and effective ℓ2-norm regularized SSC algorithm which adds a four-neighborhood ℓ2-norm regularizer into the classical SSC model, thus taking full advantage of the spatial-spectral information contained in HSIs. The experimental results confirm the potential of including the spatial information (through the newly added ℓ2-norm regularization term) in the SSC framework, which leads to a significant improvement in the clustering accuracy of SSC when applied to HSIs.

The convergence of remote sensing technologies with social media, coupled with advances in other location-aware technologies such as WiFi and smartphones, is moving us on a fast track to a situation in which we can readily know, at any time, where everybody and everything are located on the surface of the Earth, and to exploit the power of social media in different contexts. Remote sensing technology involves the use of systems and algorithms to record information about the surface of the Earth from a remote location [1]. Although reliable as a data source, remote sensing data may not always be available. However, these data can be complemented by other sources of data, such as geographic information systems (GIS) and social media [2], in order to address time-critical applications. For instance, relating publicly available social media information with remote sensing or GIS data can lead to a more efficient management of emergency response (which refers to applications in which real/near-real-time response is needed, such as natural disasters). Social media are now playing a more relevant role in our daily lives and provide a unique opportunity to gain valuable insight on information flow and social networking within the society. As a result, the integration of social media data with other consolidated technologies such as remote sensing or GIS is of great importance.

The rapid development of social media data and the associated growth in volume, velocity, and variety has fostered the idea of using these data to guide traditional remote sensing image retrieval and information extraction tasks. Although important progress has been made in recent years in harvesting spatial and temporal data from social media, the exploitation of these data for decision making still needs further investigation, particularly in the context of its integration with remote sensing and geographic information systems. In this paper, we first discuss the relation between localization techniques and spatial technologies, pointing out their similarities and differences. Then, we provide a discussion on location analysis of social media data, and the fusion of multiple data sources, with specific attention to the integration of social media content (including localization) with remote sensing-based spatial technologies. Next, we provide specific examples addressing the use of social media data to perform information extraction from large remote sensing data repositories. Although significant possibilities for the integration of localization and spatial technologies can be seen in the examples provided, our survey suggests that the convergence of remote sensing and social media data will continue to deeply transform these technologies.

The significant development of social media over the past decade has been complemented by the rise of spatial technologies to provide new mapping mechanisms that allow users engage with online information services and also with each other in an unprecedented way. Users of these technologies now provide a comprehensive geosocial overlay of the physical environment of the planet.

Spectral-spatial classification of remotely sensed hyperspectral images has attracted a lot of attention in recent years. Although Gabor filtering has been used for feature extraction from hyperspectral images, its capacity to extract relevant information from both the spectral and the spatial domains of the image has not been fully explored yet. In this paper, we present a new discriminative low-rank Gabor filtering (DLRGF) method for spectral-spatial hyperspectral image classification. A main innovation of the proposed approach is that our implementation is accomplished by decomposing the standard 3-D spectral-spatial Gabor filter into eight subfilters, which correspond to different combinations of low-pass and bandpass single-rank filters. Then, we show that only one of the subfilters (i.e., the one that performs low-pass spatial filtering and bandpass spectral filtering) is actually appropriate to extract suitable features based on the characteristics of hyperspectral images. This allows us to perform spectral-spatial classification in a highly discriminative and computationally efficient way, by significantly decreasing the computational complexity (from cubic to linear order) compared with the 3-D spectral-spatial Gabor filter. In order to theoretically prove the discriminative ability of the selected subfilter, we derive an overall classification risk bound to evaluate the discriminating abilities of the features provided by the different subfilters. Our experimental results, conducted using different hyperspectral images, indicate that the proposed DLRGF method exhibits significant improvements in terms of classification accuracy and computational performance when compared with the 3-D spectral-spatial Gabor filter and other state-of-the-art spectral-spatial classification methods.

In order to provide reliable information about the instantaneous field of view considered in hyperspectral images through spectral unmixing, understanding the kind of mixture that occurs over each pixel plays a crucial role. In this paper, in order to detect nonlinear mixtures, a method for fast identification of linear mixtures is introduced. The proposed method does not need statistical information and performs an a priori test on the spectral linearity of each pixel. It uses standard least squares optimization to achieve estimates of the likelihood of occurrence of linear combinations of endmembers by taking advantage of the geometrical properties of hyperspectral signatures. Experimental results on both real and synthetic data sets show that the aforesaid algorithm is actually able to deliver a reliable and thorough assessment of the kind of mixtures present in the pixels of the scene.

The availability of diverse data captured over the same region makes it possible to develop multisensor data fusion techniques to further improve the discrimination ability of classifiers. In this paper, a new sparse and low-rank technique is proposed for the fusion of hyperspectral and light detection and ranging (LiDAR)-derived features. The proposed fusion technique consists of two main steps. First, extinction profiles are used to extract spatial and elevation information from hyperspectral and LiDAR data, respectively. Then, the sparse and low-rank technique is utilized to estimate the low-rank fused features from the extracted ones that are eventually used to produce a final classification map. The proposed approach is evaluated over an urban data set captured over Houston, USA, and a rural one captured over Trento, Italy. Experimental results confirm that the proposed fusion technique outperforms the other techniques used in the experiments based on the classification accuracies obtained by random forest and support vector machine classifiers. Moreover, the proposed approach can effectively classify joint LiDAR and hyperspectral data in an ill-posed situation when only a limited number of training samples are available.

Sparse coding can exploit the intrinsic sparsity of hyperspectral images (HSIs) by representing it as a group of sparse codes. This strategy has been shown to be effective for HSI denoising. However, how to effectively exploit the structural information within the sparse codes (structured sparsity) has not been widely studied. In this paper, we propose a new method for HSI denoising, which uses structured sparse coding and intracluster filtering. First, due to the high spectral correlation, the HSI is represented as a group of sparse codes by projecting each spectral signature onto a given dictionary. Then, we cast the structured sparse coding into a covariance matrix estimation problem. A latent variable-based Bayesian framework is adopted to learn the covariance matrix, the sparse codes, and the noise level simultaneously from noisy observations. Although the considered strategy is able to perform denoising through accurately reconstructing spectral signatures, an inconsistent recovery of sparse codes may corrupt the spectral similarity in each spatial homogeneous cluster within the scene. To address this issue, an intracluster filtering scheme is further employed to restore the spectral similarity in each spatial cluster, which results in better denoising results. Our experimental results, conducted using both simulated and real HSIs, demonstrate that the proposed method outperforms several state-of-the-art denoising methods.

We present a new framework, called multisensor coupled spectral unmixing (MuCSUn), that solves unmixing problems involving a set of multisensor time-series spectral images in order to understand dynamic changes of the surface at a subpixel scale. The proposed methodology couples multiple unmixing problems based on regularization on graphs between the time-series data to obtain robust and stable unmixing solutions beyond data modalities due to different sensor characteristics and the effects of nonoptimal atmospheric correction. Atmospheric normalization and cross calibration of spectral response functions are integrated into the framework as a preprocessing step. The proposed methodology is quantitatively validated using a synthetic data set that includes seasonal and trend changes on the surface and the residuals of nonoptimal atmospheric correction. The experiments on the synthetic data set clearly demonstrate the efficacy of MuCSUn and the importance of the preprocessing step. We further apply our methodology to a real time-series data set composed of 11 Hyperion and 22 Landsat-8 images taken over Fukushima, Japan, from 2011 to 2015. The proposed methodology successfully obtains robust and stable unmixing results and clearly visualizes class-specific changes at a subpixel scale in the considered study area.

Many algorithms for spectral unmixing have been proposed in the last years applied on hyperspectral imaging. This process is composed by three stages where the extraction of endmembers is the most consuming step. However, endmember extraction algorithms (EEAs) can be computationally very expensive and its acceleration on parallel architectures is still an interesting and open problem. In this paper, we present a parallel implementation of the simplex growing algorithm for hyperspectral unmixing called P-SGA on different platforms using the OpenCL framework. The proposed implementation exploits the memory hierarchy to accelerate the parts of this method which are more time-consuming. The proposed algorithm is evaluated in terms of both accuracy and computational performance through Monte Carlo simulations using the following architectures: multi-core Xeon CPU, NVidia GeForce GTX 980 GPU and Intel Xeon Phi accelerator. Experiments are conducted using real hyperspectral data set revealing considerable acceleration factors, which satisfies the real-time constraints given by the data acquisition rate.

The use of spatial information prior to spectral unmixing of hyperspectral data is a very active research line in recent years. There are many approximations that consider spatial characteristics of the data in order to guide the endmember identification/extraction procedure. In particular, the spatial preprocessing (SPP) algorithm can be used prior to most existing spectral-based endmember identification techniques, thus promoting the selection of endmembers in spatially representative parts of the scene. The main concern regarding SPP and this kind of preprocessing techniques is that they are computational expensive, adding a significant burden to the spectral unmixing process which should be alleviated. In this paper we revisit and enhance a previously developed implementation of SPP for graphical processing units (GPUs) in order to increase its performance by exhaustively using the level one (L1)-cache level of the GPU. The performance of the proposed implementation is evaluated using an NVidiaTMGeForce GTX 580. Our experimental validation reveals that real-time processing performance can be obtained for real hyperspectral data sets collected by the Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS).

Change detection by unmixing has been shown to provide enhanced change detection performance for hyperspectral images with respect to more traditional approaches, especially when the temporal images contain sub-pixel level changes. In a recent paper, change detection by spectral unmixing was investigated in detail and the advantages that can be gained by using such an approach were systematically presented through various experimental studies. However, the utilized unmixing-based change detection approach relied solely on spectral information and disregarded the spatial distribution in the scene, which inevitably limits the performance that can be achieved. In this paper, superpixels are used to integrate the spatial information in the image into the unmixing process, which in turn enhances the change detection performance with respect to spectral unmixing based change detection.

Sparse hyperspectral unmixing is a widely used technique in remote sensing data characterization. It aims at inferring, from a large spectral library, the pure spectral signatures (endmembers) present in each pixel of a hyperspectral image, jointly with their corresponding abundances. The input to sparse unmixing is represented, thus, by a hyperspectral image acquired from a platform flying at high altitude and a spectral library compiled using laboratory measurements. The reflectance datacube results from a complex ensemble of algorithms which translate the digital numbers stored by the sensor to meaningful ground reflectance, including the removal of atmospheric influence. A recurrent question in the research community does not have an answer yet: how does the atmospheric composition at the time of the flight influence the fractional abundances retrieved via sparse unmixing? This is a fundamental question, as the atmospheric parameters are subject to uncertainties, being very difficult to know them in all pixels. In this paper, we investigate how the uncertainty in two atmospheric parameters: water vapor content and visibility range, propagates to the final abundance maps via atmospheric correction of the sensed image. Our experiments reveal that sparse unmixing is more robust to uncertainty in those parameters and performs better in terms of accuracy than unmixing with image-based endmembers.

Linear spectral unmixing consists on the identification of spectrally pure constituents, called endmembers and their corresponding proportions or abundances using a linear model. Traditionally, most of the attention has been focussed on the exploitation of spectral information when identifying a set of endmembers and, only recently, some techniques try to take advantage of complementary information such as the one provided by the spatial correlation of the pixels in the image. Computational complexity represents a major problem in most of these spatial-spectral based techniques, as hyperspectral images provide very rich information in both the spatial and the spectral domain. In this paper we provide a computationally efficient implementation of a spatial-spectral processing (SSPP) algorithm which can be used prior to endmember identification and spectral unmixing. Specifically we present an implementation optimized for commodity graphics processing units (GPUs), which is evaluated using two different GPU architectures from NVidia: GeForce GTX580 and GeForce GT740. Our experimental validation reveals that significant speedups can be achieved when processing hyperspectral images of different sizes.

Supervised classification of hyperspectral images is a challenging task due to the relatively low ratio between the number of training samples and the number of spectral channels. Subspace-based classification methods deal with this difficulty by assuming that feature vectors lie in a low-dimensional subspace. Based on the fact that a class in a hyperspectral image may be composed of a number of different groups of materials and mixture of spectral features, we suggest to estimate several lower dimensional random subspaces for the samples within each class. For subspace learning and classification, we propose to exploit the union of random subspaces in a Gaussian Mixture Model. Experimental results, conducted on two real hyperspectral data sets, indicate that the proposed method provides competitive classification results in comparison with other state-of-the-art approaches.

The superpixels provided by an unsupervised segmentation algorithm are sets of neighboring pixels homogeneous in some sense. Therefore it is very likely that, in a classification problem, most pixels in a superpixel belong to the same class, namely if the homogeneity criterion is compatible with the class statistics. Superpixels are, therefore, a powerful device to express spatial contextual information. However, the exploitation of superpixels in a principled way is not straightforward. Recent efforts attack this problem under a discrete optimization framework, by including regularization terms promoting consistence of the labels in the superpixels and computing approximate labelings with graph-cut algorithms. The well known hardness of integer optimization problems is a major limitation of this line of attack. In this paper, we introduce a new strategy, based on convex relaxation, to include the spatial information provided by superpixels in classification problems. The convex relaxation of an integer optimization problem opens a door to include extra information, such as spatial partitioning information given by over-segmented superpixels. The convex optimization problem thus obtained is solved by using SALSA algorithm. Experimental results with the ROSIS Pavia University dataset illustrate the effectiveness of the proposed framework.

In order to provide reliable information on the instantaneous field-of-view considered in hyperspectral images through spectral unmixing, understanding the kind of mixture that occurs over each pixel plays a crucial role. In this paper, a new method for fast detection of linear mixtures is introduced. The proposed method does not need statistical information and performs an a priori test on the spectral linearity of each pixel. It uses standard least squares optimization to achieve estimates of the likelihood of occurrence of linear combinations of endmembers by taking advantage of geometrical properties of hyperspectral signatures. Experimental results on synthetic datasets show how the aforesaid algorithm is actually able to deliver a reliable and thorough assessment of the kind of mix on the scene.

In this paper, we joint autoencoder with active learning for hyperspectral imagery classification. Specifically, we learn the classifier via autoencoder, where the most informative samples are acitvely selected through the interaction between the autoencoder and active learning. Experimental results, conducted using both the Kennedy Space Center and the Indian Pines hyperspectral images, show that driven by active learning, the performance of autoencoder can be greatly improved.

This paper presents a new classification framework for the fusion of hyperspectral and LiDAR data. The proposed approach aims at exploiting the complementarity of the features, i.e., textural features in the hyperspectral data and the height features in the LiDAR data, respectively. In this work, we use a morphological component analysis (MCA) method for textural feature extraction. The classification is then executed by a multinomial logistic regression classifier (MLR). Our obtained experimental results reveal that the proposed feature fusion method can lead to very good classification results.

Spectral unmixing pursues the identification of spectrally pure constituents, called endmembers, and their corresponding abundances in each pixel of a hyperspectral image. Most unmixing techniques have focused on the exploitation of spectral information alone. Recently, some techniques have been developed to take advantage of the complementary information provided by the spatial correlation of the pixels in the image. Computational complexity represents a major problem in these spatial-spectral techniques, as hyperspectral images contain very rich information in both the spatial and spectral domains. In this letter, we develop a computationally efficient implementation of a spatial-spectral processing algorithm that has been successfully applied prior to the spectral unmixing of the hyperspectral data. Our implementation has been optimized for the commodity graphics processing units (GPUs) and is evaluated (using both synthetic and real data) using different GPU architectures. Significant speedups can be achieved when processing hyperspectral images of different sizes. This allows for the inclusion of the proposed parallel preprocessing module in a full hyperspectral unmixing chain able to operate in real time.

Spectral unmixing is an important technique for hyperspectral data exploitation. In order to solve the unmixing problem using a collection of previously available spectral signatures (i.e., a spectral library), sparse unmixing aims at finding the optimal subset of endmembers to represent the pixels in a hyperspectral image. The classic collaborative unmixing globally assumes that all pixels in a hyperspectral scene share the same active set of endmembers. This assumption rarely holds in practice, as endmembers tend to appear localized in spatially homogeneous areas rather than spread over the whole image. To address this limitation, in this letter, we introduce a new strategy to preserve local collaborativity for sparse hyperspectral unmixing. The proposed approach, which is called local collaborative sparse unmixing, considers the fact that endmember signatures generally appear distributed in local spatial regions instead of uniformly distributed throughout the scene. The proposed approach, which includes spatial information in the standard collaborative formulation, has been experimentally validated using both simulated and real hyperspectral data sets.

Every day a large number of Earth observation (EO) spaceborne and airborne sensors from many different countries provide a massive amount of remotely sensed data. Those data are used for different applications, such as natural hazard monitoring, global climate change, urban planning, etc. The applications are data driven and mostly interdisciplinary. Based on this it can truly be stated that we are now living in the age of big remote sensing data. Furthermore, these data are becoming an economic asset and a new important resource in many applications. In this paper, we specifically analyze the challenges and opportunities that big data bring in the context of remote sensing applications. Our focus is to analyze what exactly does big data mean in remote sensing applications and how can big data provide added value in this context. Furthermore, this paper describes the most challenging issues in managing, processing, and efficient exploitation of big data for remote sensing problems. In order to illustrate the aforementioned aspects, two case studies discussing the use of big data in remote sensing are demonstrated. In the first test case, big data are used to automatically detect marine oil spills using a large archive of remote sensing data. In the second test case, content-based information retrieval is performed using high-performance computing (HPC) to extract information from a large database of remote sensing images, collected after the terrorist attack to the World Trade Center in New York City. Both cases are used to illustrate the significant challenges and opportunities brought by the use of big data in remote sensing applications.

Remotely sensed images with very high spatial resolution provide a detailed representation of the surveyed scene with a geometrical resolution that, at the present, can be up to 30 cm (WorldView-3). A set of powerful image processing operators have been defined in the mathematical morphology framework. Among those, connected operators [e.g., attribute filters (AFs)] have proven their effectiveness in processing very high resolution images. AFs are based on attributes which can be efficiently implemented on tree-based image representations. In this paper, we considered the definition of min, max, direct, and subtractive filter rules for the computation of AFs over the tree-of-shapes representation. We study their performance on the classification of remotely sensed images. We compare the classification results over the tree of shapes with the results obtained when the same rules are applied on the component trees. The random forest is used as a baseline classifier, and the experiments are conducted using multispectral data sets acquired by QuickBird and IKONOS sensors over urban areas.

In recent years, the support vector machines (SVMs) have been very successful in remote sensing image classification, particularly when dealing with high-dimensional data and limited training samples. Nevertheless, the vector-based feature alignment of the SVM can lead to an information loss in representation of hyperspectral images, which intrinsically have a tensor-based data structure. In this paper, a new multiclass support tensor machine (STM) is specifically developed for hyperspectral image classification. Our newly proposed STM processes the hyperspectral image as a data cube and then identifies the information classes in tensor space. The multiclass STM is developed from a set of binary STM classifiers using the one-against-one parallel strategy. As a part of our tensor-based processing chain, a multilinear principal component analysis (MPCA) is used for preprocessing, in order to reduce the tensorial data redundancy and, at the same time, preserve the tensorial structure information in sparse and high-order subspaces. As a result, the contributions of this work are twofold: a new multiclass STM model for hyperspectral image classification is developed, and a tensorial image interpretation framework is constructed, which provides a system consisting of tensor-based feature representation, feature extraction, and classification. Experiments with four hyperspectral data sets, covering agricultural and urban areas, are conducted to validate the effectiveness of the proposed framework. Our experimental results show that the proposed STM and MPCA-STM can achieve better results than traditional SVM-based classifiers.

Spectral unmixing is an important technique for remotely sensed hyperspectral data exploitation. It amounts to identifying a set of pure spectral signatures, which are called endmembers, and their corresponding fractional, draftrulesabun-dances in each pixel of the hyperspectral image. Over the last years, different algorithms have been developed for each of the three main steps of the spectral unmixing chain: 1) estimation of the number of endmembers in a scene; 2) identification of the spectral signatures of the endmembers; and 3) estimation of the fractional abundance of each endmember in each pixel of the scene. However, few algorithms can perform all the stages involved in the hyperspectral unmixing process. Such algorithms are highly desirable to avoid the propagation of errors within the chain. In this paper, we develop a new algorithm, which is termed robust collaborative nonnegative matrix factorization (R-CoNMF), that can perform the three steps of the hyperspectral unmixing chain. In comparison with other conventional methods, R-CoNMF starts with an overestimated number of endmembers and removes the redundant endmembers by means of collaborative regularization. Our experimental results indicate that the proposed method provides better or competitive performance when compared with other widely used methods.

This paper presents a new approach for accurate spatial-spectral classification of hyperspectral images, which consists of three main steps. First, a pixelwise classifier, i.e., the probabilistic-kernel collaborative representation classification (PKCRC), is proposed to obtain a set of classification probability maps using the spectral information contained in the original data. This is achieved by means of a kernel extension based on collaborative representation (CR) classification. Then, an adaptive weighted graph (AWG)-based postprocessing model is utilized to include the spatial information by refining the obtained pixelwise probability maps. Furthermore, to deal with scenarios dominated by limited training samples, we modify the postprocessing model by fixing the probabilistic outputs of training samples to integrate the spatial and label information. The proposed approach is able to cover different analysis scenarios by means of a fully adaptive processing chain (based on three steps) for hyperspectral image classification. All the techniques that integrate the proposed approach have a closed-form analytic solution and are easy to be implemented and calculated, exhibiting potential benefits for hyperspectral image classification under different conditions. Specifically, the proposed method is experimentally evaluated using two real hyperspectral imagery data sets, exhibiting good classification performance even when the number of training samples available a priori is very limited.

Remote sensing images exhibit significant contrast and intensity regions and edges, which makes them highly suitable for using different texture features to properly represent and classify the objects that they contain. In this paper, we present a new technique based on multiple morphological component analysis (MMCA) that exploits multiple textural features for decomposition of remote sensing images. The proposed MMCA framework separates a given image into multiple pairs of morphological components (MCs) based on different textural features, with the ultimate goal of improving the signal-to-noise level and the data separability. A distinguishing feature of our proposed approach is the possibility to retrieve detailed image texture information, rather than using a single spatial characteristic of the texture. In this paper, four textural features: content, coarseness, contrast, and directionality (including horizontal and vertical), are considered for generating the MCs. In order to evaluate the obtained MCs, we conduct classification by using both remotely sensed hyperspectral and polarimetric synthetic aperture radar (SAR) scenes, showing the capacity of the proposed method to deal with different kinds of remotely sensed images. The obtained results indicate that the proposed MMCA framework can lead to very good classification performances in different analysis scenarios with limited training samples.

Cloud covers, which generally appear in optical remote sensing images, limit the use of collected images in many applications. It is known that removing these cloud effects is a necessary preprocessing step in remote sensing image analysis. In general, auxiliary images need to be used as the reference images to determine the true ground cover underneath cloud-contaminated areas. In this paper, a new cloud removal approach, which is called multitemporal dictionary learning (MDL), is proposed. Dictionaries of the cloudy areas (target data) and the cloud-free areas (reference data) are learned separately in the spectral domain. The removal process is conducted by combining coefficients from the reference image and the dictionary learned from the target image. This method could well recover the data contaminated by thin and thick clouds or cloud shadows. Our experimental results show that the MDL method is effective in removing clouds from both quantitative and qualitative viewpoints.

Cloud removal is an important goal for enhancing the utilization of optical remote sensing satellite images. Clouds dynamically affect the signal transmission due to their different shapes, heights, and distribution. In the case of thick opaque clouds, pixel replacement has been commonly adopted. For thin clouds, pixel correction techniques allow the effects of thin clouds to be removed while retaining the remaining information in the contaminated pixels. In this paper, we develop a new method based on signal transmission and spectral mixture analysis for pixel correction which makes use of a cloud removal model that considers not only the additive reflectance from the clouds but also the energy absorption when solar radiation passes through them. Data correction is achieved by subtracting the product of the cloud endmember signature and the cloud abundance and rescaling according to the cloud thickness. The proposed method has no requirement for meteorological data and does not rely on reference images. Our experimental results indicate that the proposed approach is able to perform effective removal of thin clouds in different scenarios.

A novel method for anomaly detection in hyperspectral images (HSIs) is proposed based on low-rank and sparse representation. The proposed method is based on the separation of the background and the anomalies in the observed data. Since each pixel in the background can be approximately represented by a background dictionary and the representation coefficients of all pixels form a low-rank matrix, a low-rank representation is used to model the background part. To better characterize each pixel's local representation, a sparsity-inducing regularization term is added to the representation coefficients. Moreover, a dictionary construction strategy is adopted to make the dictionary more stable and discriminative. Then, the anomalies are determined by the response of the residual matrix. An important advantage of the proposed algorithm is that it combines the global and local structure in the HSI. Experimental results have been conducted using both simulated and real data sets. These experiments indicate that our algorithm achieves very promising anomaly detection performance.

The Segmentation Parameter Tuner (SPT) is a tool designed for automatic tuning of segmentation parameters. In SPT, the goodness of a set of parameter values is given by the level of agreement between the segmentation result and a given reference (representing the desired outcome) quantified by a metric selected by the user (empirical discrepancy methods). This metric is used as the fitness function of an optimization algorithm that searches the parameter space for the minimum value, which is expected to correspond to the segmentation outcome most similar to the reference. SPT 3.1 offers many interesting features such as: five segmentation algorithms (for Optical, Hyperspectral and SAR images), four optimization algorithms (stochastic and direct search optimization methods) and seven discrepancy metrics (pixel and object-based). This paper describes the optimization procedure underlying SPT 3.1, the features added to this version as well as an experiment that illustrates the operation of the tool.

Advances in remote sensors are providing exceptional quantities of large-scale data with increasing spatial, spectral and temporal resolutions, raising new challenges in its analysis, e.g. those presents in classification processes. This work presents the architecture of the InterIMAGE Cloud Platform (ICP): Data Mining Package; a tool able to perform supervised classification procedures on huge amounts of data, on a distributed infrastructure. The architecture is implemented on top of the MapReduce framework. The tool has four classification algorithms implemented taken from WEKA's machine learning library, namely: Decision Trees, Naïve Bayes, Random Forest and Support Vector Machines. The SVM classifier was applied on datasets of different sizes (2 GB, 4 GB and 10 GB) for different cluster configurations (5, 10, 20, 50 nodes). The results show the tool as a potential approach to parallelize classification processes on big data.

This paper presents a new parallel implementation of a previously hyperspectral coded aperture (HYCA) algorithm for compressive sensing on graphics processing units (GPUs). HYCA method combines the ideas of spectral unmixing and compressive sensing exploiting the high spatial correlation that can be observed in the data and the generally low number of endmembers needed in order to explain the data. The proposed implementation exploits the GPU architecture at low level, thus taking full advantage of the computational power of GPUs using shared memory and coalesced accesses to memory. The proposed algorithm is evaluated not only in terms of reconstruction error but also in terms of computational performance using two different GPU architectures by NVIDIA: GeForce GTX 590 and GeForce GTX TITAN. Experimental results using real data reveals signficant speedups up with regards to serial implementation.

The integration of spatial information into spectral unmixing process has attracted much attention in recent years. Several approaches have been developed to incorporate spatial considerations into the endmember extraction/estimation procedure. Spatial preprocessing algorithms are one of the most commonly adopted techniques to guide endmember identification algorithms in terms of the spatial characteristics of the hyperspectral data. Particularly, spatial preprocessing algorithm (SPP) consists on a preprocessing technique that can be used prior to most of existing spectral-based endmember extraction process, thus promoting the selection of endmem-bers from the most spatially homogeneous regions of the data set. This paper presents a parallel implementation of SPP algorithm which is tested over two different graphic processing units (GPUs) architectures: NVidiaTMGeForce GTX 580 and NVidiaTMGeForce GTX 870M. Experimental validation using a hyperspectral data set collected by AVIRIS sensor shows that it is possible to achieve real-time performance.

Light Detection and Ranging (LiDAR) is a technology used in different topic (mapping, urban land cover, agriculture, forestry, etc.). The great potential of LiDAR data lies in its high accuracy in the measurement of heights. Hyperspectral images, which comprise hundreds of (nearly contiguous) spectral channels, can also have spatial resolution of up to 1-5 meters per pixel. In this work, we propose to integrate both hyperspectral and LiDAR data by adding the LiDAR information to the hyperspectral data cube and correcting the geometric distortions. After arranging both data sets in the same format, we analyzed the errors obtained for each data source in order to determine if the final resolution adopted was the most appropriate one for performing data fusion. Our experimental results, in an area of Extremadura, indicate improvements in the classification after integrating the hyperspectral and LiDAR data.

Hyperspectral imaging is a new technique in remote sensing that collects hundreds of images at differents wavelength values for the same area of the Earth. For instance the Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) sensor of NASA capable to obtain 224 spectral channels in a wavelength range between 40 and 250 nanometers. As a result each pixel of the image can be represented as a spectral signature. Image segmentation is the process of dividing a digital image into groups of pixels or objects. Hyperspectral image classification is an important and active area dedicated to identifying each pixel in the image with an exclusive material/object class. Several efforts had been done in this field using spectral and spatial information separately or simultaneously in order to improve the performance of the classification techniques. In this work we have developed a new technique that uses a segmentation algorithm to post-process the classification results obtained using a widely used classifier such as the support vector machine (SVM). Experimental results with a real hyperspectral data set collected over the city of Pavia, Italy, are provided.

Spectral unmixing has been a popular technique for analyzing remotely sensed hyperspectral images. The goal of unmixing is to find a collection of pure spectral signatures (called endmembers) that can explain each (possibly mixed) pixel of the scene as a combination of endmembers, weighted by their coverage fractions in the pixel or abundances. Over the last years, many algorithms have been presented to address the different parts of the spectral unmixing chain. These algorithms can be categorized in estimation of the number of end-members, identification of the endmember signatures, and estimation of the endmember's abundances on each pixel. This work presents a tool that integrates many efficient implementations of different methods in order to build more complex processing graphical chains. Also, the tool offers compatibility with the graphical processing unit (GPU), so that algorithms are automatically executed in parallel by the GPU (if available) in order to improve performance. The tool is available online from http://hypercomphypermix.blogspot.com.es and has been validated with both synthetic and real hyper-spectral scenes providing state-of-the-art unmixing results.

The light detection and ranging (LiDAR) data provides very valuable information about the height of the surveyed area which can be used as a source of complementary information for the classification of hyperspectral data, in particular when it is difficult to separate complex classes. In this work, we suggest to exploit the generalized composite kernel strategy for fusion and classification of hyperspectral and LiDAR data. Our experimental results, conducted using a hyperspectral image and a LiDAR derived intensity image collected over a rural area in Extremadura, Spain, indicate that the proposed framework for the fusion of hyperspactral and LiDAR data provides significant classification results.

Principal component analysis (PCA) is an important method for feature extraction of hyperspectral remote sensing image. With the development of hyperspectral sensors, the magnitude of hyperspectral data grows quickly, and it is a challenging task to efficiently reduce the data dimension and compress massive data volumes in hyperspectral imaging. In this paper, a distributed parallel optimization of PCA algorithm (PCA_DP) is presented on cloud computing architecture. The realization of the proposed method using Apache Hadoop and MapReduce model is described and evaluated. The experiments conducted on real hyperspectral images of different sizes, demonstrate significant acceleration factor of PCA_DP. It is efficient for massive hyperspectral data processing.

This paper presents a new approach for class-oriented spectral partitioning for hyperspectral image classification. First, without empirical information, we automatically search the spectral bands that correspond to a specific class by using different band selection approaches. Then, the obtained class-oriented spectral partitions are used respectively as the input of a group of classifiers, the results of which are combined together to generate a final one by a multiple classifier system. Our experimental results, conducted with the well-known Indians Pines test site hyperspectral image collected by the Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) in NW Indiana, suggest that our presented spectral partitioning method leads to competitive results when compared with other state-of-the-art approaches.

Compressive Sensing has raised as a very useful way to save costs in the acquisition equipment due to the fact that with this technique we can measure the signal in an already compressed form. This is very interesting in hyperspectral applications due to the large amount of data that the hyper-spectral sensors collect, store and transmit to the ground stations. Over the last years many compressive sensing methods have been applied to hyperspectral images, and others have been proposed for exploiting the unique features of this kind of images. Over the last years, many techniques have been proposed to perform compressive sensing in hyperspectral imaging. One of them is the Hyperspectral Coded Aperture (HYCA), which exploits two characteristics of hyper-spectral imagery: 1) the hyperspectral vectors belong to a low dimensional subspace, and 2) the data cube components exhibit very high correlation in the spatial and in the spectral domains. However, HYCA requires the knowledge of the subspace in advance, which, very often, may compromise its applicability. In this paper it is presented a new technique similar to HYCA which does not require the knowledge of the subspace in advance; the proposed technique is termed blind HYCA (B-HYCA) and it performs a form of blind hyperspectral compressed sensing.

In this letter, we introduce an efficient algorithm to estimate the noise correlation matrix in the initial stage of the hyperspectral signal identification by minimum error (HySime) method, commonly used for signal subspace identification in remotely sensed hyperspectral images. Compared with the current implementations of this stage, the new algorithm for noise estimation relies on the reliable QR factorization, producing correct results even when operating with single-precision arithmetic. Additionally, our algorithm exhibits a lower computational cost, and it is highly parallel. The experiments on a multicore server, using two real hyperspectral scenes, expose that these theoretical advantages carry over to the practical results.

Applying spectral unmixing on a series of multitemporal hyperspectral images for change detection has the potential to reveal important subpixel-level information, such as the abundance variation of each underlying material in a given location or the change in the distribution of materials throughout the scene, with time or resulting from significant events such as a natural disaster. However, change detection by spectral unmixing for hyperspectral images has not been extensively studied up to now, and most studies have been limited to specific cases and data sets. This is caused by the scarcity of real multitemporal hyperspectral data and the inherent difficulties in applying unmixing to multitemporal hyperspectral data in a coherent way. In this letter, we investigate change detection for hyperspectral images by spectral unmixing and systematically present the advantages that can be gained by using such an approach, supported by experimental studies conducted on carefully prepared synthetic data sets and also with real data sets.

Hyperspectral image classification has been a very active area of research in recent years. It faces challenges related with the high dimensionality of the data and the limited availability of training samples. In order to address these issues, subspace-based approaches have been developed to reduce the dimensionality of the input space in order to better exploit the (limited) training samples available. An example of this strategy is a recently developed subspace-projection-based multinomial logistic regression technique able to characterize mixed pixels, which are also an important concern in the analysis of hyperspectral data. In this letter, we extend the subspace-projection-based concept to support vector machines (SVMs), a very popular technique for remote sensing image classification. For that purpose, we construct the SVM nonlinear functions using the subspaces associated to each class. The resulting approach, called SVMsub, is experimentally validated using a real hyperspectral data set collected using the National Aeronautics and Space Administration's Airborne Visible/Infrared Imaging Spectrometer. The obtained results indicate that the proposed algorithm exhibits good performance in the presence of very limited training samples.

Spectral unmixing has been a popular technique for analyzing remotely sensed hyperspectral images. The goal of unmixing is to find a collection of pure spectral constituents (called endmembers) that can explain each (possibly mixed) pixel of the scene as a combination of endmembers, weighted by their coverage fractions in the pixel or abundances. Over the last years, many algorithms have been presented to address the three main parts of the spectral unmixing chain: 1) estimation of the number of endmembers; 2) identification of the endmember signatures; and 3) estimation of the per-pixel fractional abundances. However, to date, there is no standardized tool that integrates these algorithms in a unified framework. In this letter, we present HyperMix, an open-source tool for spectral unmixing that integrates different approaches for spectral unmixing and allows building unmixing chains in graphical fashion, so that the end-user can define one or several spectral unmixing chains in fully configurable mode. HyperMix provides efficient implementations of most of the algorithms used for spectral unmixing, so that the tool automatically recognizes if the computer has a graphics processing unit (GPU) available and optimizes the execution of these algorithms in the GPU. This allows for the execution of spectral unmixing chains on large hyperspectral scenes in computationally efficient fashion. The tool is available online from http://hypercomphypermix. blogspot.com.es and has been validated with real hyperspectral scenes, providing state-of-the-art unmixing results.

Content-based image retrieval (CBIR) systems have gained significant importance in the remotely sensed hyperspectral imaging community due to the increasing availability of hyperspectral data collected from different instruments. Spectral unmixing has been a popular technique for not only interpreting hyperspectral images but also retrieving them precisely from databases based on information content. This is due to the fact that the information provided by unmixing (i.e., the spectrally pure components of the scene or endmembers, and their corresponding abundance fractions) provides a very intuitive way to describe the content of the scene in both the spectral and the spatial sense. In this letter, we present a new computationally efficient CBIR system for hyperspectral data (available online: http://hypercomp. es/repositorySparse) which uses sparse unmixing concepts to retrieve hyperspectral scenes, based on their content, from large repositories. The search is guided by a spectral library, which is used as a guide to retrieve the data in a robust and efficient way. Given the large size of libraries and the sparsity of the unmixing solutions, the incorporation of sparse unmixing to the CBIR engine brings significant advantages. To optimize its performance in computational terms, the system has been implemented in parallel by taking advantage of the computational power of commodity graphics processing units. The proposed system is validated using a collection of synthetic and real hyperspectral images, exhibiting state-of-the-art performance.

In this letter, we present an efficient parallel implementation of composite kernels in support vector machines (SVMs) for hyperspectral image (HSI) classification. Our implementation makes effective use of commodity graphics processing units (GPUs). Specifically, we port the calculation of composite kernels to GPUs, perform intensive computations based on NVidia's compute unified device architecture, and execute the rest of the operations related with control and small data calculations in the CPU. Our experimental results, conducted using real hyperspectral data sets and NVidia GPU platforms, indicate significant improvements in terms of computational effectiveness, achieving near-real-time performance of spatial-spectral HSI classification for the first time in the literature.

In this letter, a real-time implementation of the logistic regression via variable splitting and augmented Lagrangian (LORSAL) algorithm for sparse multinomial logistic regression is presented on commodity graphics processing units (GPUs) using Nvidia's compute unified device architecture. The proposed parallel method properly exploits the GPU architecture at the low level, including its shared memory, and takes full advantage of the computational power of GPUs to achieve real-time classification performance of hyperspectral images for the first time in the hyperspectral imaging literature. Our experimental results reveal remarkable acceleration factors and real-time performance, while retaining exactly the same classification accuracy with regard to the serial and multicore versions of the classifier.

Linear spectral unmixing aims at estimating the number of pure spectral substances, also calledendmembers, their spectral signatures, and their abundance fractions in remotely sensed hyperspectral images. This paper describes a method for unsupervised hyperspectral unmixing called minimum volume simplex analysis (MVSA) and introduces a new computationally efficient implementation. MVSA approaches hyperspectral unmixing by fitting a minimum volume simplex to the hyperspectral data, constraining the abundance fractions to belong to the probability simplex. The resulting optimization problem, which is computationally complex, is solved in this paper by implementing a sequence of quadratically constrained subproblems using the interior point method, which is particularly effective from the computational viewpoint. The proposed implementation (available online: www.lx.it.pt/%7ejun/DemoMVSA.zip) is shown to exhibit state-of-the-art performance not only in terms of unmixing accuracy, particularly in nonpure pixel scenarios, but also in terms of computational performance. Our experiments have been conducted using both synthetic and real data sets. An important assumption of MVSA is that pure pixels may not be present in the hyperspectral data, thus addressing a common situation in real scenarios which are often dominated by highly mixed pixels. In our experiments, we observe that MVSA yields competitive performance when compared with other available algorithms that work under the nonpure pixel regime. Our results also demonstrate that MVSA is well suited to problems involving a high number of endmembers (i.e., complex scenes) and also for problems involving a high number of pixels (i.e., large scenes).

Classification and spectral unmixing are two important techniques for hyperspectral data exploitation. Traditionally, these techniques have been exploited independently. In this paper, we propose a new technique that exploits their complementarity. Specifically, we develop a new framework for semisupervised hyperspectral image classification that naturally integrates the information provided by discriminative classification and spectral unmixing. The idea is to assign more confidence to the information provided by discriminative classification for those pixels that can be easily catalogued due to their spectral purity. For those pixels that are more highly mixed in nature, we assign more confidence to the information provided by spectral unmixing. In this case, we use a traditional spectral unmixing chain to produce the abundance fractions of the pure signatures (endmembers) that model the mixture information at a subpixel level. The decision on which source of information is prioritized in the process is taken adaptively, when new unlabeled samples are selected and included in our semisupervised framework. In this regard, the proposed approach can adaptively integrate these two sources of information without the need to establish any weight parameters, thus exploiting the complementarity of classification and unmixing and selecting the most appropriate source of information in each case. In order to test our concept, which has similar computational complexity as traditional semisupervised classification strategies, we have used two different hyperspectral data sets with different characteristics and spatial resolution. In our experiments, we consider two different discriminative classifiers: multinomial logistic regression and probabilistic support vector machine. The obtained results indicate that the proposed approach, which jointly exploits the features provided by classification and spectral unmixing in adaptive fashion, offers an effective solution to improve classification performance in hyperspectral scenes containing mixed pixels.

Abstract—Hyperspectral image classification has been an active topic of research in recent years. In the past, many different types of features have been extracted (using both linear and nonlinear strategies) for classification problems. On the one hand, some approaches have exploited the original spectral information or other features linearly derived from such information in order to have classes which are linearly separable. On the other hand, other techniques have exploited features obtained through nonlinear transformations intended to reduce data dimensionality, to better model the inherent nonlinearity of the original data (e.g., kernels) or to adequately exploit the spatial information contained in the scene (e.g., using morphological analysis). Special attention has been given to techniques able to exploit a single kind of features, such as composite kernel learning or multiple kernel learning, developed in order to deal with multiple kernels. However, few

Spectral unmixing is a popular technique for analyzing remotely sensed hyperspectral data sets with subpixel precision. Over the last few years, many algorithms have been developed for each of the main processing steps involved in spectral unmixing (SU) under the LMM assumption: 1) estimation of the number of endmembers; 2) identification of the spectral signatures of the endmembers; and 3) estimation of the abundance of endmembers in the scene. Although this general processing chain has proven to be effective for unmixing certain types of hyperspectral images, it also has some drawbacks. The first one comes from the fact that the output of each stage is the input of the following one, which favors the propagation of errors within the unmixing chain. A second problem is the huge variability of the results obtained when estimating the number of endmembers of a hyperspectral scene with different state-of-the-art algorithms, which influences the rest of the process. A third issue is the computational complexity of the whole process. To address the aforementioned issues, this paper develops a novel negative abundance-oriented SU algorithm that covers, for the first time in the literature, the main steps involved in traditional hyperspectral unmixing chains. The proposed algorithm can also be easily adapted to a scenario in which the number of endmembers is known in advance and two additional variations of the algorithm are provided to deal with high-noise scenarios and to significantly reduce its execution time, respectively. Our experimental results, conducted using both synthetic and real hyperspectral scenes, indicate that the presented method is highly competitive (in terms of both unmixing accuracy and computational performance) with regard to other SU techniques with similar requirements, while providing a fully self-contained unmixing chain without the need for any input parameters.

Hyperspectral imaging relies on sophisticated acquisition and data processing systems able to acquire, process, store, and transmit hundreds or thousands of image bands from a given area of interest. In this paper, we exploit the high correlation existing among the components of the hyperspectral data sets to introduce a new compressive sensing methodology, termed hyperspectral coded aperture (HYCA), which largely reduces the number of measurements necessary to correctly reconstruct the original data. HYCA relies on two central properties of most hyperspectral images, usually termed data cubes: 1) the spectral vectors live on a low-dimensional subspace; and 2) the spectral bands present high correlation in both the spatial and the spectral domain. The former property allows to represent the data vectors using a small number of coordinates. In this paper, we particularly exploit the high spatial correlation mentioned in the latter property, which implies that each coordinate is piecewise smooth and thus compressible using local differences. The measurement matrix computes a small number of random projections for every spectral vector, which is connected with coded aperture schemes. The reconstruction of the data cube is obtained by solving a convex optimization problem containing a data term linked to the measurement matrix and a total variation regularizer. The solution of this optimization problem is obtained by an instance of the alternating direction method of multipliers that decomposes very hard problems into a cyclic sequence of simpler problems. In order to address the need to set up the parameters involved in the HYCA algorithm, we also develop a constrained version of HYCA (C-HYCA), in which all the parameters can be automatically estimated, which is an important aspect for practical application of the algorithm. A series of experiments with simulated and real data shows the effectiveness of HYCA and C-HYCA, indicating their potential in real-world applications.

This paper presents a new multiple feature learning approach for accurate spectral-spatial classification of hyperspec-tral images. The proposed method integrates multiple features based on the logarithmic opinion pool. We consider subspace multinomial logistic regression for classification as it exhibits a flexible structure for the combination of multiple features through the posterior probability. At the same time, it is able to cope with highly mixed hyperspectral data and with the presence of limited training samples. In this work, we considered lowpass filtering and morphological attribute profiles for spatial feature extraction. Our experimental results with a real hyperspectral images collected by the NASA Jet Propulsion Laboratory's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) indicate that the proposed method exhibits state-of-the-art classification performance.

In this paper, we present a new approach for spectral partitioning which is intended to deal with ill-posed problems in hyperspectral image classification. First, we use adaptive affinity propagation (AAP) to intelligently group the original spectral bands. Such grouping strategy not only allows us to reduce the number of spectral bands, but also to provide a different perspective on the original hyperspectral data. Then, a multiple classifier system (MCS) based on multinomial logistic regression (MLR) is applied. The system is trained using different band subsets resulting from the previously conducted intelligent grouping, and the results are combined to produce a final classification result. Our experimental results, conducted using the well-known hyperspectral scenes collected by the Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the Indian Pines region in NW Indiana, indicate that the proposed method can provide important advantages in terms of classification, in particular, when the number of training samples available a priori is very low.

Over the last few years, several new strategies have been proposed for spectral-spatial classification of remotely sensed image data, for cases when high spatial and spectral resolutions are available. In this letter, we focus on the possibility of performing advanced spectral-spatial classification of remote sensing images with limited spectral resolution (often called multispectral). A new strategy is proposed, where the spectral dimensionality of the multispectral data is first expanded by using nonlinear feature extraction with kernel methods such as kernel principal component analysis. Then, extended multiattribute profiles (EMAPs), built on the expanded set of spectral features, are used to include spatial information. This strategy allows us to first decompose different spectral clusters into different spectral features and further improve the spatial discrimination. The resulting EMAPs are used for classification using advanced classifiers such as support vector machines and random forests. We test our proposed methodology with different multispectral data sets obtaining state-of-the-art classification results.

In this letter, we propose a multinomial-logistic-regression method for pixelwise hyperspectral classification. The feature vectors are formed by the energy of the spectral vectors projected on class-indexed subspaces. In this way, we model not only the linear mixing process that is often present in the hyperspectral measurement process but also the nonlinearities that are separable in the feature space defined by the aforementioned feature vectors. Our experimental results have been conducted using both simulated and real hyperspectral data sets, which are collected using NASA's Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and the Reflective Optics System Imaging Spectrographic (ROSIS) system. These results indicate that the proposed method provides competitive results in comparison with other state-of-the-art approaches.

This letter presents a new parallel method for hyperspectral unmixing composed by the efficient combination of two popular methods: vertex component analysis (VCA) and sparse unmixing by variable splitting and augmented Lagrangian (SUNSAL). First, VCA extracts the endmember signatures, and then, SUNSAL is used to estimate the abundance fractions. Both techniques are highly parallelizable, which significantly reduces the computing time. A design for the commodity graphics processing units of the two methods is presented and evaluated. Experimental results obtained for simulated and real hyperspectral data sets reveal speedups up to 100 times, which grants real-time response required by many remotely sensed hyperspectral applications.

Spectral unmixing amounts to automatically finding the signatures of pure spectral components (called endmembers in the hyperspectral imaging literature) and their associated abundance fractions in each pixel of the hyperspectral image. Many algorithms have been proposed to automatically find spectral endmembers in hyperspectral data sets. Perhaps one of the most popular ones is the pixel purity index (PPI), which is available in the ENVI software from Exelis Visual Information Solutions. This algorithm identifies the endmembers as the pixels with maxima projection values after projections onto a large randomly generated set of random vectors (called skewers). Although the algorithm has been widely used in the spectral unmixing community, it is highly time consuming as its precision asymptotically increases. Due to its high computational complexity, the PPI algorithm has been recently implemented in several high-performance computing architectures, including commodity clusters, heterogeneous and distributed systems, field programmable gate arrays, and graphics processing units (GPUs). In this letter, we present an improved GPU implementation of the PPI algorithm, which provides real-time performance for the first time in the literature.

This paper develops several new strategies for remote sensing image classification postprocessing (CPP) and conducts a systematic study in this area. CPP is defined as a refinement of the labeling in a classified image in order to enhance its original classification accuracy. The current mainstream classification methods (preprocessing) extract additional spatial features in order to complement spectral information and enhance classification using spectral responses alone. On the other hand, however, the CPP methods, providing a new solution to improve classification accuracy by refining the initial result, have not received sufficient attention. They have potential for achieving comparable accuracy to the preprocessing methods but in a more direct and succinct way. In this paper, we consider four groups of CPP strategies: filtering; random field; object-based voting; and relearning. In addition to the state-of-the-art CPP algorithms, we also propose a series of new ones, e.g., anisotropic probability diffusion and primitive cooccurrence matrix. In experiments, a number of multisource remote sensing data sets are used for evaluation of the considered CPP algorithms. It is shown that all the CPP strategies are capable of providing more accurate results than the raw classification. Among them, the relearning approaches achieve the best results. In addition, our relearning algorithms are compared with the state-of-the-art spectral-spatial classification. The results obtained further verify the effectiveness of CPP in different remote sensing applications.

Sparse unmixing has been recently introduced in hyperspectral imaging as a framework to characterize mixed pixels. It assumes that the observed image signatures can be expressed in the form of linear combinations of a number of pure spectral signatures known in advance (e.g., spectra collected on the ground by a field spectroradiometer). Unmixing then amounts to finding the optimal subset of signatures in a (potentially very large) spectral library that can best model each mixed pixel in the scene. In this paper, we present a refinement of the sparse unmixing methodology recently introduced which exploits the usual very low number of endmembers present in real images, out of a very large library. Specifically, we adopt the collaborative (also called “multitask” or “simultaneous”) sparse regression framework that improves the unmixing results by solving a joint sparse regression problem, where the sparsity is simultaneously imposed to all pixels in the data set. Our experimental results with both synthetic and real hyperspectral data sets show clearly the advantages obtained using the new joint sparse regression strategy, compared with the pixelwise independent approach.

Spectral unmixing aims at finding the spectrally pure constituent materials (also called endmembers) and their respective fractional abundances in each pixel of a hyperspectral image scene. In recent years, sparse unmixing has been widely used as a reliable spectral unmixing methodology. In this approach, the observed spectral vectors are expressed as linear combinations of spectral signatures assumed to be known a priori and presented in a large collection, termed spectral library or dictionary, usually acquired in laboratory. Sparse unmixing has attracted much attention as it sidesteps two common limitations of classic spectral unmixing approaches, namely, the lack of pure pixels in hyperspectral scenes and the need to estimate the number of endmembers in a given scene, which are very difficult tasks. However, the high mutual coherence of spectral libraries, jointly with their ever-growing dimensionality, strongly limits the operational applicability of sparse unmixing. In this paper, we introduce a two-step algorithm aimed at mitigating the aforementioned limitations. The algorithm exploits the usual low dimensionality of the hyperspectral data sets. The first step, which is similar to the multiple signal classification array signal processing algorithm, identifies a subset of the library elements, which contains the endmember signatures. Because this subset has cardinality much smaller than the initial number of library elements, the sparse regression we are led to is much more well conditioned than the initial one using the complete library. The second step applies collaborative sparse regression, which is a form of structured sparse regression, exploiting the fact that only a few spectral signatures in the library are active. The effectiveness of the proposed approach, termed MUSIC-CSR, is extensively validated using both simulated and real hyperspectral data sets.

Remotely sensed hyperspectral image classification is a very challenging task. This is due to many different aspects, such as the presence of mixed pixels in the data or the limited information available a priori. This has fostered the need to develop techniques able to exploit the rich spatial and spectral information present in the scenes while, at the same time, dealing with mixed pixels and limited training samples. In this paper, we present a new spectral–spatial classifier for hyperspectral data that specifically addresses the issue of mixed pixel characterization. In our presented approach, the spectral information is characterized both locally and globally, which represents an innovation with regard to previous approaches for probabilistic classification of hyperspectral data. Specifically, we use a subspace-based multinomial logistic regression method for learning the posterior probabilities and a pixel-based probabilistic support vector machine classifier as an indicator to locally determine the number of mixed components that participate in each pixel. The information provided by local and global probabilities is then fused and interpreted in order to characterize mixed pixels. Finally, spatial information is characterized by including a Markov random field (MRF) regularizer. Our experimental results, conducted using both synthetic and real hyperspectral images, indicate that the proposed classifier leads to state-of-the-art performance when compared with other approaches, particularly in scenarios in which very limited training samples are available.

In recent years, sparse representations have been widely studied in the context of remote sensing image analysis. In this paper, we propose to exploit sparse representations of morphological attribute profiles for remotely sensed image classification. Specifically, we use extended multiattribute profiles (EMAPs) to integrate the spatial and spectral information contained in the data. EMAPs provide a multilevel characterization of an image created by the sequential application of morphological attribute filters that can be used to model different kinds of structural information. Although the EMAPs' feature vectors may have high dimensionality, they lie in class-dependent low-dimensional subpaces or submanifolds. In this paper, we use the sparse representation classification framework to exploit this characteristic of the EMAPs. In short, by gathering representative samples of the low-dimensional class-dependent structures, any given sample may by sparsely represented, and thus classified, with respect to the gathered samples. Our experiments reveal that the proposed approach exploits the inherent low-dimensional structure of the EMAPs to provide state-of-the-art classification results for different multi/hyperspectral data sets.

The binary partition tree (BPT) is a hierarchical region-based representation of an image in a tree structure. The BPT allows users to explore the image at different segmentation scales. Often, the tree is pruned to get a more compact representation and so the remaining nodes conform an optimal partition for some given task. Here, we propose a novel BPT construction approach and pruning strategy for hyperspectral images based on spectral unmixing concepts. Linear spectral unmixing consists of finding the spectral signatures of the materials present in the image (endmembers) and their fractional abundances within each pixel. The proposed methodology exploits the local unmixing of the regions to find the partition achieving a global minimum reconstruction error. Results are presented on real hyperspectral data sets with different contexts and resolutions.

The Binary Partition Tree (BPT) is a hierarchical region-based representation of an image in a tree structure. BPT allows users to explore the image at different segmentation scales, from fine partitions close to the leaves to coarser partitions close to the root. Often, the tree is pruned so the leaves of the resulting pruned tree conform an optimal partition given some optimality criterion. Here, we propose a novel BPT construction approach and pruning strategy for hyperspectral images based on spectral unmixing concepts. The proposed methodology exploits the local unmixing of the regions to find the partition achieving a global minimum reconstruction error. We successfully tested the proposed approach on the well-known Cuprite hyperspectral image collected by NASA Jet Propulsion Laboratory's Airborne Visible/Infrared Imaging Spectrometer (AVIRIS). This scene is considered as a standard benchmark to validate spectral unmixing algorithms.

In this paper, a new parallel method for sparse spectral unmixing of remotely sensed hyperspectral data on commodity graphics processing units (GPUs) is presented. A semi-supervised approach is adopted, which relies on the increasing availability of spectral libraries of materials measured on the ground instead of resorting to endmember extraction methods. This method is based on the spectral unmixing by splitting and augmented Lagrangian (SUNSAL) that estimates the material's abundance fractions. The parallel method is performed in a pixel-by-pixel fashion and its implementation properly exploits the GPU architecture at low level, thus taking full advantage of the computational power of GPUs. Experimental results obtained for simulated and real hyperspectral datasets reveal significant speedup factors, up to 164 times, with regards to optimized serial implementation.

Recently, signal unmixing was proposed in remote sensing with the goal to infer physical parameters of materials of interest, such as vegetation, on the ground. The typical approach uses large collections of pure spectra, called spectral libraries, in which many possible states of the vegetation are modeled by simulated or on-site acquired spectra. Spectra randomly selected from these libraries are used as input to dedicated unmixing methods, such as Multiple Endmember Spectral Mixture Analysis (MESMA). The spectra leading to the lowest reconstruction error are considered to be representative for the materials present in the pixel, such that the physical parameters of the ground vegetation can be inferred. However, the large number of spectra in the library imposes limits to the performance of this combinatorial approach, mainly related to running time constraints. In this paper, we propose the inclusion of a pre-processing step in the processing chain, based on the group lasso, which has the goal of selecting groups of signatures likely to be present in the mixtures. In this sense, the Group Sparse Unmixing via variable Splitting and Augmented Lagrangian (GSUnSAL) algorithm is used. The signatures contained in the groups selected by GSUnSAL are then used as input for MESMA. Our experiments using a real dataset acquired by an ASD spectrometer in a South-African orchard show that the proposed approach introduces important improvements in the signal unmixing solutions.

This paper presents a new multiple-classifier approach for accurate spectral-spatial classification of hyperspectral images, where the spectral information is exploited by combining probabilistic support vector machines (SVM) and subspace-based multinomial logistic regression (MLRsub) and the spatial information is exploited by means of a Markov random field (MRF) regularizer. The proposed approach is based on the decision fusion of global posterior probability distributions and local probabilities which result from the whole image and the class combinations map respectively. With respect to the SVM or MLRsub algorithms, the proposed method greatly improves the classification accuracy. Our experimental results with real hyperspectral images collected by the NASA Jet Propulsion Laboratory's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) and the Reflective Optics Spectrographic Imaging System (ROSIS), indicate that the proposed multiple-classifier system leads to state-of-the-art classification performance for cases with very limited number of training samples.

Remote sensors capture large scenes that are conventionally split in smaller patches before being stored and analyzed. Traditionally, this has been done by dividing the scene in rectangular windows. Such windowing methodology could provoke the separation of spectrally homogeneous areas or objects of interest into two or more patches. This is due to the presence of objects of interest in correspondence to windows' borders, or because the fixed size of the windows does not adapt well to the scale of the objects. To alleviate this issue, the windows can be arranged in an overlapping way, incurring in some data redundancy storage. Recently, tree representations have been used as an alternative to windowing in order to structure and store large amounts of remote sensing data. In this work we explore the benefits of using Binary Partition Trees (BPT) instead of windowing to store hyperspectral large scenes. We are particularly interested in storing the information resulting of local spectral unmixing processes running over a large real hyperspectral scene. We show that under similar conditions BPT allows a better storage of the unmixing information in terms of reconstruction error.

We describe the minimum volume simplex enclosure problem (MVSEP), which is known to be a global optimization problem, and further investigate its multimodality. The problem is a basis for several (unmixing) methods that estimate so-called endmembers and fractional values in a linear mixing model. We describe one of the estimation methods based on MVSEP. We show numerically that using nonlinear optimization local search leads to the estimation results aimed at. This is done using examples, designing instances and comparing the outcomes with a maximum volume enclosing simplex approach which is used frequently in unmixing data.

In this letter, we propose a new semisupervised learning (SSL) algorithm for remotely sensed hyperspectral image classification. Our main contribution is the development of a new soft sparse multinomial logistic regression model which exploits both hard and soft labels. In our terminology, these labels respectively correspond to labeled and unlabeled training samples. The proposed algorithm represents an innovative contribution with regard to conventional SSL algorithms that only assign hard labels to unlabeled samples. The effectiveness of our proposed method is evaluated via experiments with real hyperspectral images, in which comparisons with conventional semisupervised self-learning algorithms with hard labels are carried out. In such comparisons, our method exhibits state-of-the-art performance.

In this letter, we present a new parallel implementation of the vertex component analysis (VCA) algorithm for spectral unmixing of remotely sensed hyperspectral data on commodity graphics processing units. We first developed a C serial version of the VCA algorithm and three parallel versions: one using NVIDIA's Compute Unified Device Architecture (CUDA), another using CUDA basic linear algebra subroutines library CUBLAS, and the last using the CUDA linear algebra library CULA. Experimental results, based on the analysis of hyperspectral images acquired by a variety of hyperspectral imaging sensors, show the effectiveness of our implementation, which satisfies the real-time constraints given by the data acquisition rate.

The detection of (moving or static) targets in remotely sensed hyperspectral images often requires real-time responses for swift decisions that depend upon high computing performance of algorithm analysis. The automatic target detection and classification algorithm (ATDCA) has been widely used for this purpose. In this letter, we develop several optimizations for accelerating the computational performance of ATDCA. The first one focuses on the use of the Gram-Schmidt orthogonalization method instead of the orthogonal projection process adopted by the classic algorithm. The second one is focused on the development of a new implementation of the algorithm on commodity graphics processing units (GPUs). The proposed GPU implementation properly exploits the GPU architecture at low level, including shared memory, and provides coalesced accesses to memory that lead to very significant speedup factors, thus taking full advantage of the computational power of GPUs. The GPU implementation is specifically tailored to hyperspectral imagery and the special characteristics of this kind of data, achieving real-time performance of ATDCA for the first time in the literature. The proposed optimizations are evaluated not only in terms of target detection accuracy but also in terms of computational performance using two different GPU architectures by NVIDIA: Tesla C1060 and GeForce GTX 580, taking advantage of the performance of operations in single-precision floating point. Experiments are conducted using hyperspectral data sets collected by three different hyperspectral imaging instruments. These results reveal considerable acceleration factors while retaining the same target detection accuracy for the algorithm.

Hyperspectral image processing represents a valuable tool for remote sensing of the Earth. This fact has led to the inclusion of hyperspectral sensors in different airborne and satellite missions for Earth observation. However, one of the main drawbacks encountered when dealing with hyperspectral images is the huge amount of data to be processed, in particular, when advanced analysis techniques such as spectral unmixing are used. The main contribution of this letter is the introduction of a novel preprocessing (PP) module, called SE2PP, which is based on the integration of spatial and spectral information. The proposed approach can be combined with existing algorithms for endmember extraction, reducing the computational complexity of those algorithms while providing similar figures of accuracy. The key idea behind SE2PP is to identify and select a reduced set of pixels in the hyperspectral image, so that there is no need to process a large amount of them to get accurate spectral unmixing results. Compared to previous approaches based on similar spatial and spatial-spectral PP strategies, SE2PP clearly outperforms their results in terms of accuracy and computation speed, as it is demonstrated with artificial and real hyperspectral images.

Hyperspectral imaging is an important technique in remote sensing which is characterized by high spectral resolutions. With the advent of new hyperspectral remote sensing missions and their increased temporal resolutions, the availability and dimensionality of hyperspectral data is continuously increasing. This demands fast processing solutions that can be used to compress and/or interpret hyperspectral data onboard spacecraft imaging platforms in order to reduce downlink connection requirements and perform a more efficient exploitation of hyperspectral data sets in various applications. Over the last few years, reconfigurable hardware solutions such as field-programmable gate arrays (FPGAs) have been consolidated as the standard choice for onboard remote sensing processing due to their smaller size, weight, and power consumption when compared with other high-performance computing systems, as well as to the availability of more FPGAs with increased tolerance to ionizing radiation in space. Although there have been many literature sources on the use of FPGAs in remote sensing in general and in hyperspectral remote sensing in particular, there is no specific reference discussing the state-of-the-art and future trends of applying this flexible and dynamic technology to such missions. In this work, a necessary first step in this direction is taken by providing an extensive review and discussion of the (current and future) capabilities of reconfigurable hardware and FPGAs in the context of hyperspectral remote sensing missions. The review covers both technological aspects of FPGA hardware and implementation issues, providing two specific case studies in which FPGAs are successfully used to improve the compression and interpretation (through spectral unmixing concepts) of remotely sensed hyperspectral data. Based on the two considered case studies, we also highlight the major challenges to be addressed in the near future in this emerging and fast growing research area.

In this paper, we propose a new framework for spectral-spatial classification of hyperspectral image data. The proposed approach serves as an engine in the context of which active learning algorithms can exploit both spatial and spectral information simultaneously. An important contribution of our paper is the fact that we exploit the marginal probability distribution which uses the whole information in the hyperspectral data. We learn such distributions from both the spectral and spatial information contained in the original hyperspectral data using loopy belief propagation. The adopted probabilistic model is a discriminative random field in which the association potential is a multinomial logistic regression classifier and the interaction potential is a Markov random field multilevel logistic prior. Our experimental results with hyperspectral data sets collected using the National Aeronautics and Space Administration's Airborne Visible Infrared Imaging Spectrometer and the Reflective Optics System Imaging Spectrometer system indicate that the proposed framework provides state-of-the-art performance when compared to other similar developments.

Remotely sensed hyperspectral imaging allows for the detailed analysis of the surface of the Earth using advanced imaging instruments which can produce high-dimensional images with hundreds of spectral bands. Supervised hyperspectral image classification is a difficult task due to the unbalance between the high dimensionality of the data and the limited availability of labeled training samples in real analysis scenarios. While the collection of labeled samples is generally difficult, expensive, and time-consuming, unlabeled samples can be generated in a much easier way. This observation has fostered the idea of adopting semisupervised learning techniques in hyperspectral image classification. The main assumption of such techniques is that the new (unlabeled) training samples can be obtained from a (limited) set of available labeled samples without significant effort/cost. In this paper, we develop a new approach for semisupervised learning which adapts available active learning methods (in which a trained expert actively selects unlabeled samples) to a self-learning framework in which the machine learning algorithm itself selects the most useful and informative unlabeled samples for classification purposes. In this way, the labels of the selected pixels are estimated by the classifier itself, with the advantage that no extra cost is required for labeling the selected pixels using this machine–machine framework when compared with traditional machine–human active learning. The proposed approach is illustrated with two different classifiers: multinomial logistic regression and a probabilistic pixelwise support vector machine. Our experimental results with real hyperspectral images collected by the National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible–Infrared Imaging Spectrometer and the Reflective Optics Spectrographic Imaging System indicate that the use of self-learning represents an effective and promising strategy in the context of hyperspectral image classification.

This paper presents a new framework for the development of generalized composite kernel machines for hyperspectral image classification. We construct a new family of generalized composite kernels which exhibit great flexibility when combining the spectral and the spatial information contained in the hyperspectral data, without any weight parameters. The classifier adopted in this work is the multinomial logistic regression, and the spatial information is modeled from extended multiattribute profiles. In order to illustrate the good performance of the proposed framework, support vector machines are also used for evaluation purposes. Our experimental results with real hyperspectral images collected by the National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible/Infrared Imaging Spectrometer and the Reflective Optics Spectrographic Imaging System indicate that the proposed framework leads to state-of-the-art classification performance in complex analysis scenarios.

In this paper, we develop a new algorithm for hyperspectral unmixing which can provide suitable endmembers (and their corresponding abundances) in a single step. Hence, the algorithm does not require a previous subspace identification step to estimate the number of endmembers as it can cope with the two most likely scenarios in practice (i.e., the number of endmembers is correctly determined or overestimated a priori). The proposed approach, termed collaborative NMF (CoNMF), uses a collaborative regularization prior which forces the abundances corresponding to the overestimated endmembers to zero, such that it is guaranteed that only the true endmembers have fractional abundance contributions and the estimation of the number of endmembers is not required in advance. The obtained experimental results demonstrate that the proposed method exhibits very good performance in case the number of endmember is not available a priori.

Vertex component analysis (VCA) has become a very popular and useful tool to linear unmix large hyperspectral datasets without the use of any a priori knowledge of the constituent spectra. Although VCA is fast method, many hyperspectral imagery applications require a response in real time or near-real time.

Spectral unmixing is an important tast for remotely sensed hyperspectral imaging. Spectral unmixing algorithms can be computationally expensive. In this paper, we develop two efficient implementations of a full hyperspectral unmixing chain on two different kinds of high performance computing architectures: graphics processing units (GPUs) and multi-core processors. These platforms are inter-compared in the context of hyperspectral unmixing. Our experimental results reveal that both GPUs and multi-core processors can provide real-time unmixing performance when properly implemented in this kind of parallel platforms.

In this paper, we develop a new framework for semi-supervised learning which exploits active learning for unlabeled sample selection in hyperspectral data classification. Specifically, we use active learning to select the most informative unlabeled training samples with the ultimate goal of systematically achieving noticeable improvements in classification results with regard to those found by randomly selected training sets of the same size. Our experimental results, conducted with an urban hyperspectral scene collected by the Reflective Optics Spectrographic Imaging Instrument (ROSIS) of the Deutschen Zentrum for Luftund Raumfahrt (DLR, the German Aerospace Agency) over the city of Pavia, Italy, indicate that using active learning for unlabeled sample selection represents an effective and promising strategy in the context of urban hyperspectral data classification.

In our previous research, we have proposed band-similarity-based unsupervised band selection approaches, which are proven to be very efficient. In this paper, we propose to use a collaborative sparse model for further improvement. Specifically, the pre-selected bands using the fast method, called NFINDR+LP, are further refined using a collaborative sparse model. It not only requires that the linear regression coefficients are sparse, but also requires that the same set of active bands is shared by all the bands to be removed. With the collaborative sparseness constraint being relaxed, the final selected bands can be further improved, that is, the band subset with the same number of bands can provide better classification accuracy. Based on the preliminary result, the proposed sparse model is also capable of finding the minimum number of bands to be selected.

Sparse unmixing aims at estimating the constituent materials (endmembers) and their respective fractional abundances in each pixel of a hyperspectral image by assuming that the endmembers are present in a large collection of pure spectral signatures (spectral library), known a priori. In this paper, we propose a refinement of the sparse unmixing approach by taking into account the fact that all the pixels of the image share the same set of endmembers, thus lying in a lower dimensional subspace. Our idea is based on the collaborative lasso, which enforces sparsity across the pixels. The goal of this line of attack is to obtain higher accuracy of the estimated fractional abundances, at the same time with a decrease in the number of endmembers used to explain the observed data. The experimental results, obtained with both simulated and real data, confirm the potential of the proposed approach in the unmixing problem.

In this paper, we present a new open source system for evaluating and inter-comparing new spectral unmixing applications. The proposed tool, called HyperMix, comprises several open source implementations of algorithms for end member identification and spectral unmixing. The tool also includes a database of synthetic hyperspectral images (generated using fractals to simulate natural patterns) which can be used to evaluate the precision of the algorithms for end member identification and abundance estimation which are already incorporated in the tool. The paper also presents an exhaustive inter-comparison of algorithms for end member extraction and abundance estimation using the proposed tool.

In this paper, we develop a new lossy compression framework for hyperspectral images, termed hyperspectral coded aperture (HYCA), which combines the ideas of spectral unmixing and compressive sensing. It takes advantage of two main properties of hyperspectral data, namely the high spatial correlation that can be observed in the data and the generally low number of endmembers needed in order to explain the data. In other words, our proposed approach intends to exploit the fact that the high dimensional hyperspectral data lives in a subspace of much lower dimension due to the mixing phenomenon. Our experimental results, conducted with synthetic hyperspectral data, indicate that the proposed approach represents a promising new strategy.

In this paper, we develop a new parallel implementation of the iterative error analysis (IEA) algorithm for lossy hyperspectral image compression on graphics processing units (GPUs), an inexpensive parallel computing platform that has recently become very popular in hyperspectral imaging applications. The proposed GPU implementation is tested on several different architectures from NVidia, the main GPU vendor worldwide, and is shown to exhibit real-time performance in the analysis of AVIRIS data sets. The GPU implementation of the IEA represents a step forward towards real-time onboard (lossy) compression of hyperspectral data where the quality of the compression can be also adjusted in real-time.

In a Mediterranean semiarid area in Central Spain, with dominant rainfed agriculture, hyperspectral airborne data, supported by field spectroscopy have been obtained, allowing a spectral identification of bare soil with the corresponding erosion stages. The definition of soil erosion stages is based on a spectral characterization supported by morphological, physical and chemical features of contrasted soil surfaces as a result of soil loss. Different soil erosion stages were defined within two bare-soil sites representing the soil variability of the area, and where such stages are spatially represented. The validation of selected image derived endmebers was a key step to carry out a partial unmixing for determining soil erosion stages. A preliminary spatial distribution of advanced and intermediate erosion stages was obtained for the most representative soil types.

Hyperspectral remote sensing attempts to identify features in the surface of the Earth using sensors that generally provide large amounts of data. The data are usually collected by a satellite or an airborne instrument and sent to a ground station that processes it. The main bottleneck of this approach is the (often reduced) bandwidth connection between the satellite and the station, which drastically limits the information that can be sent and processed in real time. A possible way to overcome this problem is to include onboard computing resources able to preprocess the data, reducing its size by orders of magnitude. Reconfigurable field-programmable gate arrays (FPGAs) are a promising platform that allows hardware/software codesign and the potential to provide powerful onboard computing capability and flexibility at the same time. Since FPGAs can implement custom hardware solutions, they can reach very high performance levels. Moreover, using run-time reconfiguration, the functionality of the FPGA can be updated at run time as many times as needed to perform different computations. Hence, the FPGA can be reused for several applications reducing the number of computing resources needed. One of the most popular and widely used techniques for analyzing hyperspectral data is linear spectral unmixing, which relies on the identification of pure spectral signatures via a so-called endmember extraction algorithm. In this paper, we present the first FPGA design for N-FINDR, a widely used endmember extraction algorithm in the literature. Our system includes a direct memory access module and implements a prefetching technique to hide the latency of the input/output communications. The proposed method has been implemented on a Virtex-4 XC4VFX60 FPGA (a model that is similar to radiation-hardened FPGAs certified for space operation) and tested using real hyperspectral data collected by NASA's Earth Observing-1 Hyperion (a satellite instrument) and the Airborne Visible Infra-Red Imaging Spectrometer over the Cuprite mining district in Nevada and the Jasper Ridge Biological Preserve in California. Experimental results demonstrate that our hardware version of the N-FINDR algorithm can significantly outperform an equivalent software version and is able to provide accurate results in near real time, which makes our reconfigurable system appealing for onboard hyperspectral data processing.

Spectral unmixing is an important technique for hyperspectral data exploitation, in which a mixed spectral signature is decomposed into a collection of spectrally pure constituent spectra, called endmembers, and a set of correspondent fractions, or abundances, that indicate the proportion of each endmember present in the mixture. Over the last years, several algorithms have been developed for automatic or semiautomatic endmember extraction. Some available approaches assume that the input data set contains at least one pure spectral signature for each distinct material and further conduct a search for the most spectrally pure signatures in the high-dimensional space spanned by the hyperspectral data. Among these approaches, those aimed at maximizing the volume of the simplex that can be formed using available spectral signatures have found wide acceptance. However, the presence of spectrally pure constituents is unlikely in remotely sensed hyperspectral scenes due to spatial resolution, mixing phenomena, and other considerations. In order to address this issue, other available algorithms have been developed to generate virtual endmembers (not necessarily present among the input data samples) by finding the simplex with minimum volume that encloses all available observations. In this paper, we discuss maximum-volume versus minimum-volume enclosing solutions and further develop a novel algorithm in the latter category which incorporates the fractional abundance estimation as an internal step of the endmember searching process (i.e., it does not require an external method to produce endmember fractional abundances). The method is based on iteratively enclosing the observations in a lower dimensional space and removing observations that are most likely not to be enclosed by the simplex of the endmembers to be estimated. The performance of the algorithm is investigated and compared to that of other algorithms (with and without the pure pixel assumption) using synthetic and real hyperspectral data sets collected by a variety of hyperspectral imaging instruments.

Spectral unmixing aims at estimating the fractional abundances of pure spectral signatures (also called endmembers) in each mixed pixel collected by a remote sensing hyperspectral imaging instrument. In recent work, the linear spectral unmixing problem has been approached in semisupervised fashion as a sparse regression one, under the assumption that the observed image signatures can be expressed as linear combinations of pure spectra, known a priori and available in a library. It happens, however, that sparse unmixing focuses on analyzing the hyperspectral data without incorporating spatial information. In this paper, we include the total variation (TV) regularization to the classical sparse regression formulation, thus exploiting the spatial-contextual information present in the hyperspectral images and developing a new algorithm called sparse unmixing via variable splitting augmented Lagrangian and TV. Our experimental results, conducted with both simulated and real hyperspectral data sets, indicate the potential of including spatial information (through the TV term) on sparse unmixing formulations for improved characterization of mixed pixels in hyperspectral imagery.

This paper introduces a new supervised segmentation algorithm for remotely sensed hyperspectral image data which integrates the spectral and spatial information in a Bayesian framework. A multinomial logistic regression (MLR) algorithm is first used to learn the posterior probability distributions from the spectral information, using a subspace projection method to better characterize noise and highly mixed pixels. Then, contextual information is included using a multilevel logistic Markov-Gibbs Markov random field prior. Finally, a maximum a posteriori segmentation is efficiently computed by the min-cut-based integer optimization algorithm. The proposed segmentation approach is experimentally evaluated using both simulated and real hyperspectral data sets, exhibiting state-of-the-art performance when compared with recently introduced hyperspectral image classification methods. The integration of subspace projection methods with the MLR algorithm, combined with the use of spatial-contextual information, represents an innovative contribution in the literature. This approach is shown to provide accurate characterization of hyperspectral imagery in both the spectral and the spatial domain.

Onboard processing of remotely sensed hyper spectral data is a highly desirable goal in many applications. For this purpose, compact reconfigurable hardware modules such as field programmable gate arrays (FPGAs) are widely used. In this paper, we develop a new implementation of an automatic target generation process (ATGP) for hyper spectral images. Our implementation is based on a design methodology that starts from a high-level description in Matlab (or alternative C/C++) and obtains a register transfer level (RTL) description that can be ported to FPGAs. In order to validate our new implementation, we develop a quantitative and comparative study using two different FPGA architectures: Xilinx Virtex-5 and Altera Stratix-III Altera. Experimental results have been obtained in the context of a real application focused on the detection of mineral components over the Cup rite mining district (Nevada), using hyper spectral data collected by NASA's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS). Our experimental results indicate that the proposed implementation can achieve peak frequency designs above 200MHz in the considered FPGAs, in addition to satisfactory results in terms of target detection accuracy and parallel performance. This represents a step forward towards the design of real-time onboard implementations of hyper spectral image analysis algorithms.

The development of parallel implementations is an important task for hyper spectral data exploitation. In most cases, real-time or nearly real-time processing of hyper spectral images is required for swift decisions which depend upon high computing performance of algorithm analysis. A popular algorithm in hyper spectral image interpretation is the automatic target generation process (ATGP). In this paper, we develop a new parallel version of this algorithm, which is routinely applied in many application domains, including defence and intelligence, precision agriculture, geology, or forestry. We improve considerably the computational cost of this algorithm, and also improve its detection accuracy by incorporating a new method for calculating the orthogonal projection process in which the algorithm is based using the Gram-Schmidt method. Our proposed strategy reduces the computational cost over the a previous implementation of the same algorithm which uses the pseudoinverse operation to compute the orthogonal projection. Our parallel algorithm is implemented on a multi-core cluster system made up of of sixteen nodes, with two CPUs of four cores per node, and quantitatively evaluated using hyper spectral data collected by NASA's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the World Trade Center (WTC) in New York and over the Cup rite mining district, Nevada, United States.

Hyperspectral instruments acquire electromagnetic energy scattered within their ground instantaneous field view in hundreds of spectral channels with high spectral resolution. Very often, however, owing to low spatial resolution of the scanner or to the presence of intimate mixtures (mixing of the materials at a very small scale) in the scene, the spectral vectors (collection of signals acquired at different spectral bands from a given pixel) acquired by the hyperspectral scanners are actually mixtures of the spectral signatures of the materials present in the scene. Given a set of mixed spectral vectors, spectral mixture analysis (or spectral unmixing) aims at estimating the number of reference materials, also called endmembers, their spectral signatures, and their fractional abundances. Spectral unmixing is, thus, a source separation problem. This paper presents an overview of the principal research directions in hyperspectral unmixing. The paper is organized into six main topics: i) mixing models, ii) signal subspace identification, iii) geometrical-based spectral unmixing, iv) statistical-based spectral unmixing, v) sparse regression based unmixing, and vi) spatial-contextual information. For each topic, we summarize what is the mathematical problem involved and give relevant pointers to state-of-the-art algorithms to address these problems.

Low spatial resolution is a major limitation for remote sensing classification, especially in a urban environment. In this work, we will focus on the simulation of urban area environment at a low spatial resolution, comparable to the new hyperspectral sensors that will be launched in the next few years. The aim is to better understand the possibility offered by the new sensors, in a challenging scenario like the one represented by a highly mixed image. Particular attention is placed on the characteristics of the sensor EnMap, produced by DLR. The experiments conducted on a real data set confirm the challenges posed by low spatial resolution when analyzing a urban environment.

Sparse unmixing has been recently introduced as a mechanism to characterize mixed pixels in remotely sensed hyper-spectral images. It assumes that the observed image signatures can be expressed in the form of linear combinations of a number of pure spectral signatures known in advance (e.g., spectra collected on the ground by a field spectroradiometer). Unmixing then amounts to finding the optimal subset of signatures in a (potentially very large) spectral library that can best model each mixed pixel in the scene. In available spectral libraries, it is observed that the spectral signatures appear organized in groups (e.g. different alterations of a single mineral in the U.S. Geological Survey spectral library). In this paper, we explore the potential of the sparse group lasso technique in solving hyperspectral unmixing problems. Our introspection in this work is that, when the spectral signatures appear in groups, this technique has the potential to yield better results than the standard sparse regression approach. Experimental results with both synthetic and real hyperspectral data are given to investigate this issue.

In this work, we present a new subspace discriminant analysis classification algorithm for remotely sensed hyperspectral image data. Our motivation for including subspace projection as a distinctive feature of our work is to better model noise and mixed pixels present in hyperspectral images. Two different dimensionality reduction techniques are considered: principal component analysis (PCA) and the hyperspectral signal identification by minimum error (HySime) algorithm. Experimental results indicate that the proposed method can provide competitive classification results (in the presence of very limited training data sets) with regards to those achieved by other state-of-the-art methods, such as linear discriminant analysis (LDA), subspace LDA, support vector machines (SVMs), and subspace SVMs using PCA and HySime for dimensionality reduction purposes.

This paper develops a noise-robust spatial preprocessing module which can be used prior to spectral unmixing of remotely sensed hyperspectral images. The method first derives a spatial homogeneity index which is relatively insensitive to the noise present in the original hyperspectral data. Then, it fuses this index with a spectral-based classification, obtaining a set of pure regions which are used to guide the unmixing process. An experimental comparison of the proposed method with other spatial-spectral unmixing approaches is conducted using both synthetic and real hyperspectral data collected by the Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS). Our experiments indicate that spectral unmixing can benefit from the proposed pre-processing approach, in particular, when the noise level present in the original hypespectral scene is relatively high.

Spectral unmixing is an important task for hyperspectral data exploitation. It generally consists of two steps: identification of pure spectral signatures (endmembers) and estimation of the fractional abundance of each endmember in each pixel of the scene. A successful algorithm to perform both tasks in simultaneous fashion is the iterative error analysis (IEA) algorithm, which applies an iterative process in which the next endmember to be detected depends on the set of previously extracted ones, which can be computationally expensive for hyperspectral images with a large number of endmembers. In this paper, we propose a new parallel implementation of the IEA algorithm for graphics processing units (GPUs). The proposed implementation is tested on three different GPUs fromNVidia™, and is shown to exhibit real-time performance in the analysis of an Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) data set collected over the Cuprite mining district in Nevada.

Abstract Hyperspectral imaging instruments are capable of collecting hundreds of images, corresponding to different wavelength channels, for the same area on the surface of the Earth. One of the main problems in the analysis of hyperspectral data cubes is the presence of mixed pixels, which arise when the spatial resolution of the sensor is not enough to separate spectrally distinct materials. Hyperspectral unmixing is one of the most popular techniques to analyze hyperspectral data. It comprises two stages: (i) automatic identification of pure spectral signatures (endmembers) and (ii) estimation of the fractional abundance of each endmember in each pixel. The spectral unmixing process is quite expensive in computational terms, mainly due to the extremely high dimensionality of hyperspectral data cubes. Although this process maps nicely to high performance systems such as clusters of computers, these systems are generally expensive and difficult to adapt to real‐time data processing requirements introduced by several applications, such as wildland fire tracking, biological threat detection, monitoring of oil spills, and other types of chemical contamination. In this paper, we develop an implementation of the full hyperspectral unmixing chain on commodity graphics processing units (GPUs). The proposed methodology has been implemented, using the CUDA (compute device unified architecture), and tested on three different GPU architectures: NVidia Tesla C1060, NVidia GeForce GTX 275, and NVidia GeForce 9800 GX2, achieving near real‐time unmixing performance in some configurations tested when analyzing two different hyperspectral images, collected over the World Trade Center complex in New York City and the Cuprite mining district in Nevada. Copyright © 2011 John Wiley & Sons, Ltd.

Supervised classification of hyperspectral images is a very challenging task due to the generally unfavorable ratio between the number of spectral bands and the number of training samples available a priori, which results in the Hughes phenomenon. For this purpose, several feature extraction methods have been investigated in order to reduce the dimensionality of the data to the right subspace without significant loss of the original information that allows for the separation of classes. In this letter, we explore the use of spectral unmixing for feature extraction prior to supervised classification of hyperspectral data using support vector machines. The proposed feature extraction strategy has been implemented in the form of four different unmixing chains and evaluated using two different scenes collected by National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible/Infrared Imaging Spectrometer. The experiments suggest competitive results but also show that the definition of the unmixing chains plays an important role in the final classification accuracy. Moreover, differently from most feature extraction techniques available in the literature, the features obtained using linear spectral unmixing are potentially easier to interpret due to their physical meaning.

Hyperspectral data lossy compression has not yet achieved global acceptance in the remote sensing community, mainly because it is generally perceived that using compressed images may affect the results of posterior processing stages. This possible negative effect, however, has not been accurately characterized so far. In this letter, we quantify the impact of lossy compression on two standard approaches for hyperspectral data exploitation: spectral unmixing, and supervised classification using support vector machines. Our experimental assessment reveals that different stages of the linear spectral unmixing chain exhibit different sensitivities to lossy data compression. We have also observed that, for certain compression techniques, a higher compression ratio may lead to more accurate classification results. Even though these results may seem counterintuitive, this work explains these observations in light of the spatial regularization and/or whitening that most compression techniques perform and further provides recommendations on best practices when applying lossy compression prior to hyperspectral data classification and/or unmixing.

Linear spectral unmixing is an important task in remotely sensed hyperspectral data exploitation. This approach first identifies a collection of spectrally pure constituent spectra, called endmembers, and then expresses the measured spectrum of each mixed pixel as a combination of endmembers weighted by fractions or abundances that indicate the proportion of each endmember in the pixel. Over the last decade, several algorithms have been developed for automatic extraction of spectral endmembers from hyperspectral data, with many of them relying exclusively on the spectral information. In this letter, we develop a novel unsupervised spatial preprocessing (SPP) module which adopts a region-based approach for the characterization of each endmember class prior to endmember identification using spectral information. The proposed approach can be combined with any spectral-based endmember extraction technique. Our method is validated using both synthetic scenes constructed using fractals and a real hyperspectral data set collected by NASA's Airborne Visible Infrared Imaging Spectrometer over the Cuprite Mining District in Nevada and further compared with previous efforts in the same direction such as the spatial-spectral endmember extraction, automatic morphological endmember extraction, or SPP methods.

In this letter, we discuss the use of multicore processors in the acceleration of endmember extraction algorithms for hyperspectral image unmixing. Specifically, we develop computationally efficient versions of two popular fully automatic endmember extraction algorithms: orthogonal subspace projection and N-FINDR. Our experimental results, based on the analysis of hyperspectral data collected by the National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible InfraRed Imaging Spectrometer, indicate that endmember extraction algorithms can significantly benefit from these inexpensive high-performance computing platforms, which can offer real-time response with some programming effort.

Linear spectral unmixing is a popular tool in remotely sensed hyperspectral data interpretation. It aims at estimating the fractional abundances of pure spectral signatures (also called as endmembers) in each mixed pixel collected by an imaging spectrometer. In many situations, the identification of the end-member signatures in the original data set may be challenging due to insufficient spatial resolution, mixtures happening at different scales, and unavailability of completely pure spectral signatures in the scene. However, the unmixing problem can also be approached in semisupervised fashion, i.e., by assuming that the observed image signatures can be expressed in the form of linear combinations of a number of pure spectral signatures known in advance (e.g., spectra collected on the ground by a field spectroradiometer). Unmixing then amounts to finding the optimal subset of signatures in a (potentially very large) spectral library that can best model each mixed pixel in the scene. In practice, this is a combinatorial problem which calls for efficient linear sparse regression (SR) techniques based on sparsity-inducing regularizers, since the number of endmembers participating in a mixed pixel is usually very small compared with the (ever-growing) dimensionality (and availability) of spectral libraries. Linear SR is an area of very active research, with strong links to compressed sensing, basis pursuit (BP), BP denoising, and matching pursuit. In this paper, we study the linear spectral unmixing problem under the light of recent theoretical results published in those referred to areas. Furthermore, we provide a comparison of several available and new linear SR algorithms, with the ultimate goal of analyzing their potential in solving the spectral unmixing problem by resorting to available spectral libraries. Our experimental results, conducted using both simulated and real hyperspectral data sets collected by the NASA Jet Propulsion Laboratory's Airborne Visible Infrared Imaging Spectrometer and spectral libraries publicly available from the U.S. Geological Survey, indicate the potential of SR techniques in the task of accurately characterizing the mixed pixels using the library spectra. This opens new perspectives for spectral unmixing, since the abundance estimation process no longer depends on the availability of pure spectral signatures in the input data nor on the capacity of a certain endmember extraction algorithm to identify such pure signatures.

This paper introduces a new supervised Bayesian approach to hyperspectral image segmentation with active learning, which consists of two main steps. First, we use a multinomial logistic regression (MLR) model to learn the class posterior probability distributions. This is done by using a recently introduced logistic regression via splitting and augmented Lagrangian algorithm. Second, we use the information acquired in the previous step to segment the hyperspectral image using a multilevel logistic prior that encodes the spatial information. In order to reduce the cost of acquiring large training sets, active learning is performed based on the MLR posterior probabilities. Another contribution of this paper is the introduction of a new active sampling approach, called modified breaking ties, which is able to provide an unbiased sampling. Furthermore, we have implemented our proposed method in an efficient way. For instance, in order to obtain the time-consuming maximum a posteriori segmentation, we use the α-expansion min-cut-based integer optimization algorithm. The state-of-the-art performance of the proposed approach is illustrated using both simulated and real hyperspectral data sets in a number of experimental comparisons with recently introduced hyperspectral image analysis methods.

The 19 papers in this special issue focus on the state-of-the-art and most recent developments in the area of spectral unmixing of remotely sensed data.

Remotely sensed hyperspectral imaging instruments provide high-dimensional data containing rich information in both the spatial and the spectral domain. In many surveillance applications, detecting objects (targets) is a very important task. In particular, algorithms for detecting (moving or static) targets, or targets that could expand their size (such as propagating fires) often require timely responses for swift decisions that depend upon high computing performance of algorithm analysis. In this paper, we develop parallel versions of a target detection algorithm based on orthogonal subspace projections. The parallel implementations are tested in two types of parallel computing architectures: a massively parallel cluster of computers called Thunderhead and available at NASA's Goddard Space Flight Center in Maryland, and a commodity graphics processing unit (GPU) of NVidia GeForce GTX 275 type. While the cluster-based implementation reveals itself as appealing for information extraction from remote sensing data already transmitted to Earth, the GPU implementation allows us to perform near real-time anomaly detection in hyperspectral scenes, with speedups over 50x with regards to a highly optimized serial version. The proposed parallel algorithms are quantitatively evaluated using hyperspectral data collected by the NASA's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) system over the World Trade Center (WTC) in New York, five days after the attacks that collapsed the two main towers in the WTC complex.

Hyper spectral imaging is a new technique in remote sensing that generates hundreds of images, corresponding to different wavelength channels, for the same area on the surface of the Earth. In previous work, we have explored the application of morphological operations to integrate both spatial and spectral responses in hyper spectral data analysis. These operations rely on ordering pixel vectors in spectral space, but there is no unambiguous means of defining the minimum and maximum values between two vectors of more than one dimension. Our original contribution in this paper is to examine the impact of different vector ordering strategies on the definition of multi-channel morphological operations. Our focus is on morphological unmixing, which decomposes each pixel vector in the hyper spectral scene into a combination of pure spectral signatures (called end members) and their associated abundance fractions, allowing sub-pixel characterization. Experiments are conducted using real hyper spectral data sets collected by NASA/JPL's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) system.

Spectral unmixing is an important task for remotely sensed hyperspectral data exploitation. Linear spectral unmixing relies on two main steps: 1) identification of pure spectral constituents (endmembers), and 2) end member abundance estimation in mixed pixels. One of the main problems concerning the identification of spectral endmembers is the lack of pure spectral signatures in real hyperspectral data due to spatial resolution and mixture phenomena happening at different scales. In this paper, we present a new method for endmember estimation which does not assume the presence of pure pixels in the input data. The method minimizes the volume of an enclosing simplex in the reduced space while estimating the fractional abundance of vertices in simultaneous fashion, as opposed to other volume-based approaches such as N-FINDR which inflate the simplex of maximumvolume that can be formed using available image pixels. Our experimental results and comparisons to other endmember extraction algorithms indicate promising performance of the method in the task of extracting endmembers from real hyperspectral data. In our experiments, we use laboratory-simulated forest scenes with known endmembers and fractional abundances due to their acquisition in a controlled environment using a real hyperspectral imaging instrument.

This paper explores the applicability of new sparse algorithms to perform spectral unmixing of hyperspectral images using available spectral libraries instead of resorting to well-known end member extraction techniques widely available in the literature. Our main assumption is that it is unlikely to find pure pixels in real hyperspectral images due to available spatial resolution and mixing phenomena happening at different scales. The algorithms analyzed in our study rely on different principles, and their performance is quantitatively assessed using both simulated and real hyperspectral data sets. The experimental validation of sparse techniques conducted in this work indicates promising results of this new approach to attack the spectral unmixing problem in remotely sensed hyperspectral images.

In this paper, we develop a new spatial preprocessing strategy which can be applied prior to a spectral-based endmember extraction process for unmixing of hyperspectral data. Our proposed approach directs the endmember searching process to regions which are both spectrally pure and spatially homogeneous in the scene. Our experimental results, conducted using simulated hyperspectral data sets with known endmembers and fractional abundances, reveal that the proposed approach can successfully integrate the spatial and spectral information in the search for more relevant endmembers.

Endmember extraction is an important technique in the context of spectral unmixing of remotely sensed hyperspectral data. Winter's N-FINDR algorithm is one of the most widely used and successfully applied methods for endmember extraction from remotely sensed hyperspectral images. Depending on the dimensionality of the hyperspectral data, the algorithm can be time consuming. In this paper, we propose a new parallel implementation of the N-FINDR algorithm. The proposed implementation is quantitatively assessed in terms of both endmember extraction accuracy and parallel efficiency, using two different generations of commercial graphical processing units (GPUs) from NVidia. Our experimental results indicate that the parallel implementation performs better with latest-generation GPUs, thus taking advantage of the increased processing power of such units.

In this paper, we investigate the use of the watershed transformation for integrating spatial and spectral information in the process of endmember extraction for spectral unmixing of hyperspectral images. The proposed approach is presented as a preprocessing module designed to automatically select a small subset of pixels containing potentially relevant candidates from both spatial and spectral point of view. Dimensionality reduction is required. The idea is to use the morphological watershed transformation to guide the endmember searching process to spatially homogeneous and spectrally “purer” areas. Here the main assumption is that such areas can be located at the local minima of the catchment basins, and far away from watershed lines that define the transition areas between different regions, expected to contain mixed pixels. Experimental results, conducted using a database of 28 simulated hyperspectral data sets obtained through manipulation of a real hyperspectral image acquired by the Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over a mixed scenario including agricultural, vegetation, and urban areas, suggests a promising trade-off between percentage of endmember candidates retained and degree of spectral purity of predominant endmembers.

Abstract The purpose of content‐based image retrieval (CBIR) is to retrieve, from real data stored in a database, information that is relevant to a query. In remote sensing applications, the wealth of spectral information provided by latest‐generation (hyperspectral) instruments has quickly introduced the need for parallel CBIR systems able to effectively retrieve features of interest from ever‐growing data archives. To address this need, this paper develops a new parallel CBIR system that has been specifically designed to be run on heterogeneous networks of computers (HNOCs). These platforms have soon become a standard computing architecture in remote sensing missions due to the distributed nature of data repositories. The proposed heterogeneous system first extracts an image feature vector able to characterize image content with sub‐pixel precision using spectral mixture analysis concepts, and then uses the obtained feature as a search reference. The system is validated using a complex hyperspectral image database, and implemented on several networks of workstations and a Beowulf cluster at NASA's Goddard Space Flight Center. Our experimental results indicate that the proposed parallel system can efficiently retrieve hyperspectral images from complex image databases by efficiently adapting to the underlying parallel platform on which it is run, regardless of the heterogeneity in the compute nodes and communication links that form such parallel platform. Copyright © 2009 John Wiley & Sons, Ltd.

In this letter, we address the use of artificial neural networks for spectral mixture analysis of hyperspectral scenes. We specifically focus on the issue of how to effectively train neural network architectures in the context of spectral mixture analysis applications. To address this issue, a multilayer perceptron neural architecture is combined with techniques for intelligent selection and labeling of training samples directly obtained from the input data, thus maximizing the information that can be obtained from those samples while reducing the need fora prioriinformation about the scene. The proposed approach is compared to unconstrained and fully constrained linear mixture models using hyperspectral data sets acquired (in the laboratory) from artificial forest scenes, using the compact airborne spectrographic imaging system. The Spreading of Photons for Radiation INTerception (SPRINT) canopy model, which assumes detailed knowledge about object geometry, was employed to evaluate the results obtained by the different methods. Our results show that the proposed approach, when trained with both pure and mixed training samples (generated automatically withoutpriorinformation) can provide similar results to those provided by SPRINT, using very few labeled training samples. An application to real airborne data using a set of hyperspectral images collected at different altitudes by the digital airborne imaging spectrometer 7915 and the reflective optics system imaging spectrometer, operating simultaneously at multiple spatial resolutions, is also presented and discussed.

This paper presents a new semisupervised segmentation algorithm, suited to high-dimensional data, of which remotely sensed hyperspectral image data sets are an example. The algorithm implements two main steps: 1) semisupervised learning of the posterior class distributions followed by 2) segmentation, which infers an image of class labels from a posterior distribution built on the learned class distributions and on a Markov random field. The posterior class distributions are modeled using multinomial logistic regression, where the regressors are learned using both labeled and, through a graph-based technique, unlabeled samples. Such unlabeled samples are actively selected based on the entropy of the corresponding class label. The prior on the image of labels is a multilevel logistic model, which enforces segmentation results in which neighboring labels belong to the same class. The maximum a posteriori segmentation is computed by the α-expansion min-cut-based integer optimization algorithm. Our experimental results, conducted using synthetic and real hyperspectral image data sets collected by the Airborne Visible/Infrared Imaging Spectrometer system of the National Aeronautics and Space Administration Jet Propulsion Laboratory over the regions of Indian Pines, IN, and Salinas Valley, CA, reveal that the proposed approach can provide classification accuracies that are similar or higher than those achieved by other supervised methods for the considered scenes. Our results also indicate that the use of a spatial prior can greatly improve the final results with respect to a case in which only the learned class densities are considered, confirming the importance of jointly considering spatial and spectral information in hyperspectral image segmentation.

We develop a new approach for characterization of mixed pixels in remotely sensed hyperspectral images. The proposed method first performs joint spatial-spectral pixel characterization via extended morphological transformations, and then automatically extracts pure spectral signatures (called endmembers) using volume optimization and convex geometry concepts. The proposed method outperforms other widely used approaches in the analysis of a real hyperspectral scene collected by the NASA's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the Cuprite mining district in Nevada. Ground-truth information available from U.S. Geological Survey is used to substantiate our findings.

We have explored in this paper a framework to test in a quantitative manner the stability of different endmember extraction and spectral unmixing algorithms based on the concept of Consensus Clustering. The idea is to investigate if the sensibility of those algorithms to the number of endmembers can be used to estimate this parameter itself. Preliminary results on synthetic data reveal that the proposed scheme, which can be implemented efficiently in parallel, can compete with state-of-the-art schemes.

Finding an accurate sparse approximation of a spectral vector described by a linear model, when there is available a library of possible constituent signals (called endmembers or atoms), is a hard combinatorial problem which, as in other areas, has been increasingly addressed. This paper studies the efficiency of the sparse regression techniques in the spectral unmixing problem by conducting a comparison between four different approaches: Moore-Penrose Pseudoinverse, Orthogonal Matching Pursuit (OMP), Iterative Spectral Mixture Analysis (ISMA) and l2- l1sparse regression techniques, which are of widespread use in compressed sensing. We conclude that the l2-l1sparse regression techniques, implemented here by Iterative Shrinkage/Thresholding (TwIST) algorithm, yield the state-of-the-art in the hyperspectral unmixing area.

This paper introduces a new semi-supervised classification and segmentation approach tailored to hyperspectral images. The posterior distributions of the classes are modeled by the multinomial logistic regression. The contextual information inherent to the spatial configuration of the image pixels is modeled by a Multi-Level Logistic (MLL) Markov-Gibbs random field. The multinomial logistic regressors, assumed to be random vectors with independent Lapla-cian components, are learned using the recently introduced LOR-SAL algorithm. The maximum a posteriori (MAP) segmentation is computed via the α-Expansion algorithm, a powerful graph cut based approach to integer optimization. The effectiveness of the proposed methodology is illustrated by classifying simulated and real data sets. Comparisons with state-of-art methods are also included.

Over the last decade, several algorithms have been developed for automatic or semi-automatic extraction of spectral endmembers from hyperspectral image data. In this paper, we present a thorough analytical comparison of endmember extraction methods which include spatial information in the search of spectral endmembers versus a few algorithms which are exclusively based on spectral information. Our quantitative and comparative assessment of algorithm accuracy and computational performance, conducted using both synthetic and real hyperspectral data, provides interesting findings about the potential benefits that can be obtained after incorporating spatial information into the design of endmember extraction algorithms.

The main purpose of this paper is to describe available (HPC)-based implementations of remotely sensed hyperspectral image processing algorithms on multi-computer clusters, heterogeneous networks of computers, and specialized hardware architectures such as field programmable gate arrays (FPGAs) and graphic processing units (GPUs). Combined, the revision of existing techniques conducted in this paper, along with the description of performance results for a parallel hyperspectral processing chain on different architectures, delivers an excellent snapshot of the state-of-the-art in the area of HPC-based hyperspectral image processing and a thoughtful perspective of the potential and emerging challenges of applying HPC paradigms to hyperspectral imaging problems.

The Hyperspectral Imaging Network (HYPER-I-NET) is a four-year Marie Curie Research Training Network project designed to build an interdisciplinary European research community focusing on hyperspectral imaging activities. The network is currently formed by a multidisciplinary team composed of fifteen highly experienced European partner organizations. In this paper, we outline the activities that have been carried out in the four main areas covered by HYPER-I-NET: 1) hyperspectral sensor specification, 2) processing chain definition and implementation, 3) calibration, validation and definition of standardization mechanisms, and 4) science applications. Along with the description of the progress made in the four main areas listed above, this paper also describes some of the training and transfer of knowledge activities carried out during the first two years of the project.

In previous work, we have reported that the scalability of parallel processing algorithms for hyperspectral image analysis is affected by the amount of data to exchanged through the communication network of the parallel system. However, large messages are common in hyperspectral imaging applications since processing algorithms are often pixel-based, and each pixel vector to be exchanged through the communication network is made up of hundreds of spectral values. Thus, decreasing the amount of data to be exchanged could improve the scalability and parallel performance. In this paper, we propose a new framework based on intelligent utilization of data compression techniques for improving the scalability of a standard spectral unmixin-based parallel hyperspectral processing chain on heterogeneous networks of workstations. Our experimental results indicate that adaptive, wavelet-based lossy compression can lead to improvements in the scalability of the parallel algorithms without significantly sacrificing algorithm analysis accuracy.

Spectral mixture analysis is an important task for remotely sensed hyperspectral data interpretation. In spectral unmixing, both the determination of spectrally pure signatures (endmembers) and the unmixing process that interprets mixed pixels as combinations of endmembers are computationally expensive procedures. An exciting recent development in the field of commodity computing is the emergence of programmable graphics processing units (GPUs), which are now increasingly being used address the ever-growing computational requirements introduced by hyperspectral imaging applications. In this paper, we develop three new GPU-based implementations of endmember extraction algorithms: the pixel purity index (PPI), a kernel version of the PPI (KPPI), and the automatic morphological endmember extraction (AMEE) algorithm. We also provide a GPU-based implementation of the fully constrained linear spectral unmixing algorithm.

The N-FINDR algorithm is one of the most widely used and successfully applied methods for automatically determining endmembers in hyperspectral image data without usinga prioriinformation. The algorithm attempts to automatically find the simplex of maximum volume that can be inscribed within the hyperspectral data set. Due to the intrinsic complexity of remotely sensed scenes, the final volume-based solution provided by N-FINDR may be not the global maximum. In addition, the final results provided by the algorithm are typically dependent of its initialization. In this letter, we explore the aforementioned issues and conduct a quantitative and comparative analysis of different (available and new) strategies for the implementation of N-FINDR. Our experimental evaluation and comparison are conducted using two well-known hyperspectral scenes collected by the National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible Infrared Imaging Spectrometer.

Endmember extraction is the process of selecting a collection of pure signature spectra of the materials present in a remotely sensed hyperspectral scene. These pure signatures are then used to decompose the scene into abundance fractions by means of a spectral unmixing algorithm. Most techniques available in the endmember extraction literature rely on exploiting the spectral properties of the data alone. As a result, the search for endmembers in a scene is conducted by treating the data as a collection of spectral measurements with no spatial arrangement. In this paper, we propose a novel strategy to incorporate spatial information into the traditional spectral-based endmember search process. Specifically, we propose to estimate, for each pixel vector, a scalar spatially derived factor that relates to the spectral similarity of pixels lying within a certain spatial neighborhood. This scalar value is then used to weigh the importance of the spectral information associated to each pixel in terms of its spatial context. Two key aspects of the proposed methodology are given as follows: 1) No modification of existing image spectral-based endmember extraction methods is necessary in order to apply the proposed approach. 2) The proposed preprocessing method enhances the search for image spectral endmembers in spatially homogeneous areas. Our experimental results, which were obtained using both synthetic and real hyperspectral data sets, indicate that the spectral endmembers obtained after spatial preprocessing can be used to accurately model the original hyperspectral scene using a linear mixture model. The proposed approach is suitable for jointly combining spectral and spatial information when searching for image-derived endmembers in highly representative hyperspectral image data sets.

In this paper, we describe a first approximation to the relevant issue of defining a part of the hyperspectral processing chain in a flexible manner. An ultimate goal of our study is to objectively quantify the impact of different (standard and new) processing stages on the generation of a realistic, user-oriented product in the context of an urban land cover mapping problem by means of hyperspectral data, selected in this work as an application case study for demonstration purposes. Although the proposed study is linked to a specific application domain, our experimental results reveal interesting considerations that may help image analysts in defining customized processing chains based on parameters which can be identified and objectively evaluated a priori, such as available sensor resolution or ancillary information. In addition, our study also demonstrates the importance of incorporating information related to both the spatial and the spectral domain in the different steps that comprise the hyperspectral processing chain; particularly when such chain can take advantage of the combined use of both sources of information as it is the case in the considered urban characterization application.

This paper develops several parallel algorithms for target detection in hyperspectral imagery, considered to be a crucial goal in many remote sensing applications. In order to illustrate parallel performance of the proposed parallel algorithms, we consider a massively parallel Beowulf cluster at NASA's Goddard Space Flight Center. Experimental results, collected by the AVIRIS sensor over the World Trade Center, just five days after the terrorist attacks, indicate that commodity cluster computers can be used as a viable tool to increase computational performance of hyperspectral target detection applications.

Hyperspectral image processing has been a very active area in remote sensing and other application domains in recent years. Despite the availability of a wide range of advanced processing techniques for hyperspectral data analysis, many techniques for hyperspectral data classification are based on the consideration of spectral information separately from spatial information information, and thus the two types of information are not treated simultaneously. In this paper, we develop a new technique for joint spatial-spectral classification of hyperspectral image data which makes use of opening and closing by reconstruction, a kind of mathematical morphology operations which are extended here to hyperspectral images. A high performance parallel implementation of the proposed technique is also developed to satisfy time-critical constraints in remote sensing applications, using NASA's Thunderhead Beowulf cluster computer for demonstration purposes.

Abstract The rapid development of space and computer technologies allows for the possibility to store huge amounts of remotely sensed image data, collected using airborne and satellite instruments. In particular, NASA is continuously gathering high‐dimensional image data with Earth observing hyperspectral sensors such as the Jet Propulsion Laboratory's airborne visible–infrared imaging spectrometer (AVIRIS), which measures reflected radiation in hundreds of narrow spectral bands at different wavelength channels for the same area on the surface of the Earth. The development of fast techniques for transforming massive amounts of hyperspectral data into scientific understanding is critical for space‐based Earth science and planetary exploration. Despite the growing interest in hyperspectral imaging research, only a few efforts have been devoted to the design of parallel implementations in the literature, and detailed comparisons of standardized parallel hyperspectral algorithms are currently unavailable. This paper compares several existing and new parallel processing techniques for pure and mixed‐pixel classification in hyperspectral imagery. The distinction of pure versus mixed‐pixel analysis is linked to the considered application domain, and results from the very rich spectral information available from hyperspectral instruments. In some cases, such information allows image analysts to overcome the constraints imposed by limited spatial resolution. In most cases, however, the spectral bands collected by hyperspectral instruments have high statistical correlation, and efficient parallel techniques are required to reduce the dimensionality of the data while retaining the spectral information that allows for the separation of the classes. In order to address this issue, this paper also develops a new parallel feature extraction algorithm that integrates the spatial and spectral information. The proposed technique is evaluated (from the viewpoint of both classification accuracy and parallel performance) and compared with other parallel techniques for dimensionality reduction and classification in the context of three representative application case studies: urban characterization, land‐cover classification in agriculture, and mapping of geological features, using AVIRIS data sets with detailed ground‐truth. Parallel performance is assessed using Thunderhead, a massively parallel Beowulf cluster at NASA's Goddard Space Flight Center. The detailed cross‐validation of parallel algorithms conducted in this work may specifically help image analysts in selection of parallel algorithms for specific applications. Copyright © 2008 John Wiley & Sons, Ltd.

This paper discusses the application and adaptation of two existing operational algorithms for land surface emissivity ($\varepsilon$) retrieval from different operational satellite/airborne sensors with bands in the visible and near-infrared (VNIR) and thermal IR (TIR) regions: 1) the temperature and emissivity separation algorithm, which retrieves$\varepsilon$only from TIR data and 2) the normalized-difference vegetation index thresholds method, in which$\varepsilon$is retrieved from VNIR data.

This paper explains some of the the goals and objectives of the newly started HYPER-I-NET Marie Curie Research and Training Network. In particular, the requirements related to the definition and implementation of an efficient, adequate and sufficiently general data processing chain for hyperspectral data analysis are considered. Some of the research lines that are expected to play a central role in the activities of this network are also presented and briefly discussed.

The task of image registration can be divided into two major components, i.e., the extraction of control points or features from images, and the search among the extracted features for the matching pairs that represent the same feature in the images to be matched. Manual extraction of control features can be subjective and extremely time consuming, and often results in few usable points. On the other hand, automated feature extraction allows using invariant target features such as edges, corners, and line intersections as relevant landmarks for registration purposes. In this paper, we present an extension of a recently developed morphological approach for automatic extraction of landmark chips and corresponding windows in a fully unsupervised manner for the registration of multispectral images. Once a set of chip-window pairs is obtained, a (hierarchical) robust feature matching procedure, based on a multiresolution overcomplete wavelet decomposition scheme, is used for registration purposes. The proposed method is validated on a pair of remotely sensed scenes acquired by the Advanced Land Imager (ALI) multispectral instrument and the Hyperion hyperspectral instrument aboard NASA’s Earth Observing-I satellite.

Abstract—This paper addresses the main goals and objec-tives of the Hyperspectral Imaging Network (HYPER-I-NET), a recently started Marie Curie Research Training Network. The project is designed to build an interdisciplinary research community focusing on hyperspectral imaging activities. The core strategy of the network is to create a powerful interdisciplinary synergy between different domains of expertise closely related to hyperspectral imaging activities in Europe, ranging from sensor design and flight operation to data collection, processing, interpretation, and dissemination. Our main goals in this paper are to present the project to the Geoscience and Remote Sensing community and to provide an overview of the planned activities in each sub-activity covered by the network.

Many available techniques for spectral mixture analysis involve the separation of mixed pixel spectra collected by imaging spectrometers into pure component (endmember) spectra, and the estimation of abundance values for each end- member. Although linear mixing models generally provide a good abstraction of the mixing process, several naturally occurring situations exist where nonlinear models may provide the most accurate assessment of endmember abundance. In this paper, we propose a combined linear/nonlinear mixture model which makes use of linear mixture analysis to provide an initial model estimation, which is then thoroughly refined using a multi-layer neural network coupled with intelligent algorithms for automatic selection of training samples. Three different algorithms for automatic selection of training samples, such as border training algorithm (BTA), mixed signature algorithm (MSA) and mophological erosion algorithm (MEA) are developed for this purpose. The proposed model is evaluated in the context of a real application which involves the use of hyperspectral data sets, collected by the Digital Airborne (DAIS 7915) and Reflective Optics System (ROSIS) imaging spectrometers of DLR, operating simultaneously at multiple spatial resolutions.

Heterogeneous networks of workstations have rapidly become a cost-effective computing solution in many application areas. This paper develops several highly innovative parallel algorithms for target detection in hyperspectral imagery, considered to be a crucial goal in remote sensing-based homeland security and defense applications. In order to illustrate parallel performance, we consider four (partially and fully) heterogeneous networks of workstations distributed among different locations at University of Maryland, and also a massively parallel Beowulf cluster at NASA's Goddard Space Flight Center. Experimental results indicate that heterogeneous networks can be used as a viable low-cost alternative to homogeneous parallel systems in many on-going and planned remote sensing missions