Deep learning (DL) has attracted considerable attention in the field of hyperspectral image classification (HSIC). However, most DL methods still suffer from two problems: overfitting and oversmoothing, particularly when dealing with scarce labeled samples. A major challenge is that they do not make full use of the relationships among a large number of unlabeled samples and multi-scale information in structural relationships, resulting in the loss of multi-scale information. Moreover, prior information such as labels is not used to explicitly learn and modify the graph structure (including nodes, the sparsity of connections, and edge weights). To address these issues, we propose a novel deep fusion paradigm for multi-scale superpixel graphs (DFSG). Our new DFSG integrates multi-scale graphs (at both the graph-level and the feature-level) to reduce information loss while the re-segmentation based graph correction module adaptively learns new graph structures during the active learning (AL) process. In our proposed iterative updating mechanism, AL and our multi-scale methods help each other, forming a symbiotic unified DFSG-AL framework. Experiments on five real hyperspectral image (HSI) datasets demonstrate that our DFSG-AL can achieve remarkable performance in few-sample HSIC. The source codes are available at https://github.com/yulong112/DFSG.

Scene classification of remote sensing images plays a vital role in Earth observation applications. Among various challenges, occlusion is a prevalent and critical issue in practical applications, particularly when dealing with large-area occlusions caused by clouds, shadows, and man-made structures. Current methods, whether based on occlusion recovery or occlusion-robust feature extraction, generally show limited performance when processing extensive occluded regions due to ignoring the inconsistency in feature representation caused by multiscale occlusions. To address the occlusion challenge, this letter proposes a novel contrastive learning-based multiscale occlusion framework with three key components: 1) a pretext task module that distinguishes between small and large occlusions to enable occlusion-invariant feature learning; 2) a multibranch feature extraction network based on ResNet-50’s shared-weight convolutional layers for consistent feature extraction across occlusion levels; and 3) a joint loss function that adaptively balances contrastive feature learning and classification. Extensive experiments were evaluated on the DIOR-Occ and LEVIR-Occ benchmark datasets, demonstrating significant improvements in classification accuracy across different occlusion scenarios. Compared with existing approaches, the proposed framework achieves superior robustness and generalization capabilities, with notable advantages in the analysis of highly occluded data. Future research will explore the adaptation of this framework to detection and segmentation tasks.

Processing high-dimensional data cubes and developing high-performance classifiers are core objectives in the field of hyperspectral image classification (HSIC). Superpixel-based methods are widely used in HSIC due to their efficacy in reducing redundant information and enhancing local features. However, imprecise segmentation, especially in complex structures and textures of hyperspectral images (HSIs), may lead to inconsistencies in the regions extracted by superpixels and the boundaries between different ground objects. Such inconsistencies significantly degrade the classification performance of HSIs. Alternatively, when parameter settings are inaccurate, edge-aware feature extraction methods often introduce sharpening artifacts at the image boundaries, resulting in a decrease in classification accuracy. To effectively address these challenges, we propose a novel probabilistic fusion method for HSIC. This method consists of the following stages. First, spatial information is extracted by a multiscale superpixel segmentation method and then probabilistically optimized by the extended random walk (ERW) method. Next, semantic-aware structural features (S2Fs) are extracted along with edge information of different objects. Lastly, a probabilistic framework is proposed to fuse the class probabilities of superpixel-based spatial information and semantic-aware structural features. Experimental results on three real datasets show state-of-the-art classification performance, even with limited training sets.

Class activation map (CAM) based weakly-supervised semantic segmentation (WSSS) of remote sensing (RS) images has attracted extensive research interests for its potential in reducing annotation cost. However, challenged by unconstrained activation issue, existing methods struggle to delineate object boundaries clearly, making them particularly difficult to separate multiple densely packed objects, which are common in RS images. By conducting an in-depth analysis of RS image characteristics, we observed a strong correlation between object shapes and their semantics. Inspired by this finding, we propose an Intrinsic Shape Activation Network (ISANet) to learn the category-relevant shape priors as geometry constraints for target-focused region activation in WSSS of RS images. The key idea is to distill the intrinsic shape priors from the hybrid features that are deterministic in classification. Specifically, we adopt a dual-branch architecture to decouple the learning of shape and texture features and leverage a shape awareness alignment module to generate boundary-clear CAMs for computing pseudo labels. In this way, CAMs are generated with perception of target shapes, which increases the completeness of activation regions and alleviates the ultrarange responses. Extensive experiments demonstrates the superiority of our method in delineating densely-packed objects with clear contours, which is especially beneficial for separating multiple targets in RS images. Our method improves the mIoU of the state-of-the-art method by 7.9% and 3.3% on the NWPU VHR-10 and iSAID dataset respectively.

Hyperspectral unmixing is a technique in hyperspectral image processing that decomposes the spectra of mixed pixels into pure spectral components (endmembers) and their corresponding contributions (abundances). When dealing with complex mixed-terrain scenes, such as urban areas, significant challenges arise due to the complexity of the environment. Urban areas feature intricate geometric structures in individual pixels, including diverse 2-D and 3-D structures and the composite use of various building materials, resulting in highly complex scenarios. To address these challenges, this work exploits urban auxiliary information in the framework of an adaptive multitask autoencoder (AE) unmixing model, utilizing graph associations. The framework enhances the information in hyperspectral images by utilizing urban auxiliary data. Specifically, it performs superpixel segmentation to subdivide complex urban environments into simpler units. Subsequently, different AE-based unmixing methods are applied to these segmented results. Graph associations are employed to identify similar blocks in the image, incorporating this additional information into the unmixing process. In the experiments conducted for this work, two hyperspectral unmixing datasets were prepared, along with their corresponding urban auxiliary data. The results demonstrate that the proposed method achieves robust performance, even in complex urban environments.

Presents corrections to the paper, (Corrections to “Adaptive Multitask Autoencoder-Based Hyperspectral Unmixing Exploiting Auxiliary Data via Graph Associations”).

Transformers have gained significant attention in multimodal remote sensing fusion due to their strong global context modeling capability. Although Transformer-based methods excel at processing high-dimensional spectral sequences and joint spatial-spectral information, most current research remains focused on the spatial domain. Consequently, the exploration of frequency-domain features—particularly implicit frequency representations—is often neglected. Moreover, efficiently fusing multimodal data features while emphasizing more discriminative information remains a challenging task. To address these challenges, this paper proposes an Attention-guided Frequency Feature Decomposition Network (AF2DN) for Hyperspectral and LiDAR Data Classification. First, a Transformer-based Frequency Feature Decomposition(TFFD) method is proposed, employing window attention to capture distinct directional frequency components from multimodal remote sensing data. Through this approach, low-frequency components are utilized to characterize global structural information, while various high-frequency components are employed to extract diverse texture and edge features. Second, an Attention Frequency Modulation(AFM) module is developed, incorporating a weight learning matrix in the frequency domain. This matrix is designed to selectively amplify and suppress different frequency components, thereby reducing data redundancy resulting from frequency feature decomposition. Finally, an adaptive Multimodal Same-Frequency Feature Fusion (AMSF3) module is designed to achieve cross-modal feature integration at identical frequency bands. Extensive experiments are conducted on three benchmark datasets, and the results demonstrate that the proposed framework outperforms existing state-of-the-art methods while exhibiting stronger adaptability in complex environments.

Recently, Convolutional Neural Networks (CNNs) and Transformer have achieved considerable success in Hyperspectral Image (HSI) classification tasks. However, existing methods not only lack the study of spectral variability of samples from the same land class, but also struggle to mine local-global spectral information and spatial structure information of HSI at different granularities effectively. To mitigate these limitations, this article proposes a multigranularity spatial-spectral convolution Transformer network (MSSCFormer), which can reduce the intra-class spectral differences of samples and extract multigranularity spatial-spectral features from a local-global-local perspective. Specifically, MSSCFormer consists of three components: intra-class spectral attention (ICSA), spatial-spectral feature extractor (SSFE), and global-local convolutional Transformer (GLCT). Firstly, ICSA redistributes the spectral weights of samples of the same land class by establishing an attention mapping between spectral channels within the class to reduce the intra-class spectral differences. Second, SSFE extracts shallow spatial features and multigranularity spectral features of HSI with reassigned spectral weights samples from a local perspective. Finally, GLCT takes advantage of CNNs and Transformer, it uses MHSA to model global spectral features and utilizes local context feature block (LCFB) to capture local spatial features in a multigranularity way. Experimental results on three benchmark datasets show that MSSCFormer exhibits excellent performance on the HSI datasets and outperforms state-of-the-art HSI classification algorithms.

Arbitrary-resolution hyperspectral (ARHS) pansharpening seeks to enhance low-resolution hyperspectral (LRHS) images to any target spatial resolutions by collaborating with the connected panchromatic (PAN) images. The ARHS pansharpening task is significantly different from standard hyperspectral (HS) pansharpening, where ARHS pansharpening has to deal with the challenge that how to pansharpen upcoming LRHS images to any desired scales far beyond the training scales. Neural operators (NOs) form mapping between functional spaces and build continuous function relations, offering an alternative potential path to deal with ARHS pansharpening task. In this article, we are dedicated to the design of pansharpening NOs (PNOs) by the postulation that LRHS and PAN are observations of the input functional space while target high-resolution HS (HRHS) images are observed from the output functional space. Our PNOs are built with a paradigm comprising prepansharpening encoding, inverse discretization, and functional mapping subnetwork. More specifically, by use of kernel projection of frequency domain-related design and Galerkin design, we develop Fourier PNO (Fourier-PNO) and Galerkin PNO (Galerkin-PNO). In addition, we propose the concept of dynamic spectral-spatial Green’s function and accordingly develop an enhanced Fourier-PNO (eFourier-PNO), and further introduce a PAN-guided test space tactic to develop an enhanced Galerkin-PNO (eGalerkin-PNO). Finally, we discuss and explain some details such as the scale generalization ability of our PNOs. Our four PNOs are tested on three datasets, and the experimental results verify their excellent performances.

The multisource remote sensing (RS) data have yielded promising results in target detection and classification tasks. However, most existing methods primarily focus on the spatial features inherent in spectral information, while the continuous spectral characteristics are generally neglected. This oversight leads to insufficient extraction of spectral information, thereby limiting detection performance. Recently, the Mamba architecture, based on state space models (SSMs), integrates the advantages of long-range sequence modeling and linear computational efficiency, demonstrating significant potential in low-dimensional scenarios. Inspired by this, we propose Slice Mamba for multisource RS data fusion classification. Specifically, we design two scanning methods: lateral slice scanning (LatSS) and longitudinal slice scanning (LonSS), which construct sequences from lateral and longitudinal perspectives to facilitate information interaction between pixels. In conjunction with the Mamba architecture, we develop the lateral slice Mamba block (LatSMB) and the longitudinal slice Mamba block (LonSMB) to capture continuous spatial-spectral features. Based on this, we establish the slice feature extraction (SFE) module for extracting spatial-spectral feature information and design the cross-information fusion (CIF) module to form a complementary structure for effectively modeling spatial-spectral features, thereby achieving the fusion and classification of multisource heterogeneous features. Experimental results on three benchmark datasets demonstrate that Slice Mamba outperforms existing advanced methods in fusion classification performance and exhibits greater robustness when applied to multispectral datasets.

Hyperspectral image (HSI) classification is fundamental to numerous remote sensing applications, enabling detailed analysis of material properties and environmental conditions. Recent Mamba built upon selective state space models (S6) have demonstrated exceptional advantages in long-range sequence modeling with linear computational efficiency, while Transformer based on self-attention mechanisms is particularly adept at capturing short-range dependencies. To leverage the complementary strengths of these models, this paper introduces a novel hybrid Mamba-Transformer framework (HSI-MFormer), effectively exploring the multiscale properties of hyperspectral data for HSI classification. Initially, a Multiscale Token Generation module (MTG) is developed, which converts the HSI cube into multiple spatial-spectral token groups across different scales. To adequately capture fine-grained multiscale spatial-spectral patterns, an Inner-scale Transformer Expert (ITE) is designed, which incorporates grouped self-attention operations to perform short-range sequence modeling within token groups at each scale. Meanwhile, a Cross-scale Mamba Expert (CME) is introduced, which integrates a cross-scale serialization mechanism and bidirectional Mamba block for long-range sequence modeling, further exploring the interactions and complementarity between token groups across different scales. Several hybrid strategies for integrating the ITE and CME are investigated to maximize their complementarity, including parallel, interval, and serial structures. Extensive experiments demonstrate that the propsed HSI-MFormer significantly out-performs the state-of-the-art Transformer-based and Mamba-based HSI classification methods. The code is available at https://github.com/tubingnuist/HSI-MFormer.

Arbitrary-resolution hyperspectral (ARHS) pansharpening seeks to enhance hyperspectral (HS) images to any desired spatial resolutions by integrating HS images with their connected panchromatic (PAN) images, which is obviously different from standard HS pansharpening where HS data are merely pansharpened to the spatial resolutions of PANs. Therefore, ARHS pansharpening has a great potential to broaden the applications of traditional pansharpening technique. However, ARHS pansharpening is faced with some inherent obstacles, including how to produce HS images with arbitrary resolutions beyond the training scales and how to maintain high spatial-spectral fidelities at any pansharpening scales. Implicit neural representations (INRs) is able to parametrically tune neural networks to accommodate unanalytical continuous characteristics of real-world scenes, and thus offer natural representations for ARHS pansharpening. The core of pansharpening INR design is to construct suitable implicit spectral-spatial functions (ISSFs). In this work, we propose two innovatively novel transformer-ensemble based ISSFs, i.e. pre-integrated ISSF (preISSF) and post-integrated ISSF (postISSF), which use suitable coordinate operation, novel ensemble feature extraction and coordinate query mapping to build strong spectral-spatial reconstruction capabilities, where some new tactics such as mode-1 transformer decomposition and explicit-implicit positional encoding system are involved. We test our preISSF and postISSF on three datasets. The experimental results verify the excellent performance of our methods.

Traditional supervised deep learning (DL) methods for hyperspectral image (HSI) classification are severely limited by the quality and quantity of labels. Furthermore, existing feature extraction methods generally lack the fusion of multiscale feature information, struggling to handle complex scenarios. To counter these problems, this work investigates a feature extraction module based on self-supervised graph masked autoencoders (SGMAEs). It innovatively employs graph masked autoencoders to achieve self-supervised label-free feature extraction for the complete set of samples, utilizing a multiscale graph convolutional network encoder (MGCNE) and cross correlation decoder (CCD) to extract and fuse multiscale spatial-spectral features of HSI data, respectively. Specifically, the HSI data is first converted into an edge-masked perturbed graph to label-freely extract multiscale feature representations of all pixel samples, and then fed into the MGCNE to obtain multilayer feature vectors for the pixel nodes. To reconstruct the masked edges for the fusion of multiscale features, the CCD applies cross correlation calculations to the nodes of the true edges at the masked positions and the fake edges at the random positions. The contrastive learning loss function is proposed for training of the autoencoder, which calculates the loss for the existence estimates of edges generated by cross correlation calculations. The pretrained MGCNE possesses an efficient self-supervised multiscale spatial-spectral feature extraction capability, along with strong generalizability, which significantly improves the accuracy of various mainstream models in downstream classification tasks. Extensive experiments and analyses on multiple HSI datasets demonstrate that our proposed SGMAE significantly enhances the model performance of various supervised classifiers and achieves superior performance in comparison to mainstream models.

Lithologic unit classification is essential for resource surveys and infrastructure planning. Remote sensing provides a rapid and scalable alternative to traditional field surveys but faces challenges in vegetation-covered areas due to limited lithological information. To address this, we propose AMSNet, an attention-based multi-scale a priori knowledge-guided lithologic classification framework, which integrates Multi-Scale Enhanced Cross-Spatial Attention (MSECA) and Wavelet-Enhanced Crisscross Attention (WCCA). MSECA extracts lithologic features by aggregating multi-scale neighborhood information and introducing cross-spatial learning, while WCCA enhances long-range contextual understanding using wavelet transforms to mitigate information loss during key-query generation. Experiments on the Qichun and Tieshan datasets demonstrate that AMSNet achieves an overall accuracy (OA) of 69.03% and 83.77%, a mean intersection over union (mIoU) of 48.41% and 48.18%, and a Macro-F1 score of 63.65% and 60.68%, outperforming baseline models. Ablation studies further confirm the effectiveness of MSECA and WCCA in improving lithologic feature extraction. These findings demonstrate the potential of AMSNet to improve lithologic classification in complex terrains, contributing to advancements in geoscience applications.

Building change detection remains challenging for urban development, disaster assessment, and military reconnaissance. While foundation models like Segment Anything Model (SAM) show strong segmentation capability, they exhibit limited performance in building change detection due to the domain gap between natural and remote sensing images. Existing adapter-based fine-tuning methods struggle with the imbalanced distribution of changed buildings, resulting in suboptimal detection of building changes exhibiting sparse distribution, small scale, or weak contrast. Additionally, bi-temporal alignment methods, including optical flow, are vulnerable to background noise interference. To address these limitations, we propose the SAM-based Network with Distribution-Aware Fourier Adaptation and Edge-Constrained Warping (FAEWNet) for building change detection. Unlike previous adapters overlook the imbalanced distribution of changed buildings, our proposed Distribution-Aware Fourier Aggregation Adapter not only addresses the domain gap issue, but also models the distribution of changed buildings. Furthermore, to mitigate noise interference and misalignment caused by registration errors, we design Multiscale Aware Flow Aggregation module that refines building edge extraction and enhances the perception of changed buildings. The results on the LEVIR-CD, S2Looking and WHU-CD datasets highlight the effectiveness of FAEWNet. Specifically, FAEWNet achieves the best performance on the LEVIR-CD with 91.29%Rc, 92.41%F1, and 85.89%IoU. On the S2Looking dataset, our method achieves improvements of 3.09% inRc, 1.07% inF1, and 1.23% inIoUcompared to the second-best method. On the WHU-CD dataset, it achieves the highest overall performance, with aRcof 94.20%, anF1 of 94.99%, and anIoUof 90.45%. The code is available at https://github.com/SUPERMAN123000/FAEWNet.

Hyperspectral image (HSI) classification is crucial for remote sensing research, while its high-dimensional features make traditional algorithms difficult to cope with. Despite the breakthroughs in deep learning, the high computational complexity and energy consumption limit its application in resource-limited environments. Spiking neural networks (SNNs), mimicking the brain’s information processing with low power consumption, have emerged as a promising alternative for edge computing. However, SNNs struggle with complex tasks due to the nondifferentiability of spike signals, which complicates training and exhibits limitations in extracting deep features and modeling long-range dependencies. In this article, we propose a novel SNN framework that addresses these challenges by enhancing feature extraction and efficiently capturing dependencies in hyperspectral data. Our framework integrates an adaptive refocusing convolutional layer with a spike self-attention (SSA) mechanism. The adaptive refocusing convolutional layer employs learnable parameters to dynamically adjust the convolutional kernel’s response to input spike data, improving feature representation. The adaptive refocusing convolutional layer uses learnable parameters to dynamically adjust kernel responses to input spike data, enhancing feature representation. Experimental results show that this model achieves over 96% classification accuracy in a single time step, significantly surpassing current methods and effectively solving the problem of low accuracy at short time steps in SNNs. Additionally, this framework reduces computational energy consumption by approximately$12.5\times $compared to similar, offering new potential for edge intelligence applications.

Convolutional Neural Networks (CNNs) and Transformers have made remarkable achievements in hyperspectral image classification (HSIC). Unfortunately, CNN-based methods struggle to capture the contextual dependencies between pixels in HSIs, while Transformer-based methods suffer from quadratic computational complexity. Recently, the Mamba model has shown great potential as it can describe long-range dependencies between HSI pixels with linear computational complexity. Yet, Mamba still faces significant challenges in terms of global modeling. Inspired by the Mamba model framework and Transformers, a novel Dual-Branch Mamba-Like Linear Attention (DBMLLA) network is proposed for HSIC, achieving efficient global dependency modeling. Specifically, the proposed DBMLLA combines an embedding module, a Spatial-Spectral Mamba-Like Linear Attention (SS-MLLA) module, and a fusion module. In the embedding module, an absolute position embedding module is introduced for better extraction of global features. In the SS-MLLA module, we design the Spatial Mamba-Like Linear Attention (SpaMLLA) block and the Spectral Mamba-Like Linear Attention (SpeMLLA) block to exploit the spatial and spectral information of the HSI. In addition, SS-MLLA is improved by utilising Depthwise Separable Convolution (DSC) to enhance the model’s ability to extract deeper local feature information. Through experiments conducted on four public hyperspectral datasets, it is demonstrated that the proposed model consistently outperforms state-of-the-art approaches.

Transformer models have achieved remarkable success in hyperspectral image classification (HSIC) owing to their strong global modeling capability. However, their quadratic complexity significantly limits their computational efficiency. Recently, Mamba has been applied to HSIC because of its linear complexity, yet it still suffers from an imbalance between global and local modeling. To overcome these challenges, this paper proposes a novel Learnable Kernel and Mamba with Spatial-Spectral Attention Fusion (LKMA) framework, which enables the extraction of global-local spatial-spectral features (SSF) while enhancing edge feature representation. For local feature extraction, the proposed Multi-Scale Spatial-Spectral Feature Generation (MSSFG) module captures local SSF by employing multi-scale learnable dilation convolutions for spatial features and multi-scale dilation convolutions for spectral features. For global feature extraction, a Global Hidden Mixing Mamba (GHMM) module is introduced, which projects hyperspectral image (HSI) features from the feature space to the hidden state space via a hidden state mixing mechanism. This enables the model to capture contextual semantic information and local details from the HSI. To further explore the synergistic effect between spatial and spectral information, the Spatial-Spectral Attention Fusion (SSAF) module integrates semantic information across multiple feature groups by combining Semantic Grouped Spatial Attention (SGSA) and Progressive Spectral Self-Attention (PSSA), enhancing spatial-spectral representations. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art approaches for HSIC.

Implicit neural representations are increasingly recognized as a transformative approach for arbitrary-scale image Super-Resolution (SR), demonstrating capabilities that transcend traditional upscaling methods. However, the high complexity of urban structures poses significant challenges for existing methods, which mainly operate in the spatial domain and often fail to recover fine-grained textures and high-frequency details. To overcome this limitation, in this work we consider the frequency domain as a complementary representation, enabling the extraction of high-frequency information that enhances fine-grained detail reconstruction. Specifically, this study proposes a new Dual-domain Implicit Super-Resolution (DISR) framework that simultaneously extracts and fuses features from both the spatial and frequency domains, enabling continuous representation learning for remote sensing images. The DISR architecture comprises three key components: (1) The Dual-domain Feature Extraction (DFE) module, which extracts and enhances complementary features encompassing both local details and global structural information; (2) The Dual-domain Cross Attention Fusion (DCAF) mechanism, which dynamically integrates frequency-domain cues with spatial features to improve texture fidelity and structural coherence; and (3) The Dual-path Adaptive Implicit Parser (DAIP), which establishes a nonlinear mapping from coordinate positions to pixel values through parallel global-local processing pathways. Experimental results demonstrate that our new DISR framework not only improves other methods in terms of overall reconstruction quality metrics, but also excels at restoring high-frequency textures and fine structural details. The code is available at https://github.com/Liuyx-max/DISR.

With the development of infrared technology, infrared small target detection (IRSTD) is widely applied in fields such as environmental monitoring, marine rescue, and forest fire prevention. Existing IRSTD methods are often based on spatial domain approaches, which preserve target features in the spatial domain but overlook the characteristics of infrared small targets in the frequency domain. In frequency domain methods, infrared small targets are typically considered as high-frequency components, while the continuous background is regarded as low-frequency components. However, infrared small targets often have complex backgrounds, strong edges, and noise generated during imaging, all of which are also reflected as high-frequency components, leading to false detections. To overcome this issue and fully explore the potential of IRSTD in the frequency domain, we propose a novel network, SFDTNet, which integrates frequency-domain attention and U-Structure for IRSTD. In the encoding phase, spatial feature extraction is applied to the infrared small target. In the decoding stage, global-scale spatial features are modeled in the frequency domain to achieve more precise reconstruction of small targets while reducing the interference of background high-frequency clutter. Frequency domain self-attention (FDSA) introduces an attention mechanism to model global information in the frequency domain and capture the importance of different frequency components. Adaptive frequency selection network (AFSN) incorporates learnable masks to adaptively modulate high- and low-frequency components in the frequency domain. Finally, a deep supervision strategy is employed to help the network learn features more effectively. Experimental results demonstrate that it effectively retains the shape and contours of small targets while achieving a very low false detection rate. Compared with existing state-of-the-art methods, our approach shows superior performance and better robustness.

Pansharpening of multispectral images aims to merge multispectral (MS) and panchromatic (PAN) images to produce high-resolution multispectral (HRMS) images. Unsupervised pansharpening algorithms, which are widely used in deep learning for pansharpening tasks, commonly employ generative adversarial networks (GANs). However, existing unsupervised methods based on GANs have some limitations: 1) restricted ability for joint preservation of spatial and spectral information, and 2) the receptive field of general convolutions restricts the extraction of long-range dependencies. To address these issues, we propose a frequency-guided generative adversarial networks for unsupervised pansharpening (FG-GAN). Our unsupervised framework uses high-frequency and low-frequency information as prior constraints to guide the training of the FG-GAN’s generator and discriminator networks, thereby enhancing the joint preservation of spatial and spectral details. Furthermore, a graph convolution-based generator network is designed, in which long-range edge dependencies are extracted and propagated by learning the relationships between distant edge feature nodes. Extensive experiments on the Quickbird and Gaofen-2 datasets demonstrate the effectiveness of our design: our method enhances spatial details and reduces the spatial distortion index (Ds) by 44.8%, while achieving the fidelity of spatial-spectral information with a HQNR score of 0.99.

Deep learning-based methods have demonstrated promising performance in hyperspectral image (HSI) classification. However, the black-box nature of deep learning poses a significant challenge in designing effective network architectures for HSI classification. To overcome this issue, this article presents a representation model-inspired interpretable deep unfolding network (NSR-Net). First, we formulate a deep-constrained nonnegative sparse representation (NSR) model with enhanced generalization ability to address the limitations of the prior-constrained NSR, i.e., its reliance on manual priors and specific assumptions. Second, the solving process for deep-constrained NSR is unfolded into a deep network, with each component of the network corresponding directly to a specific step. Finally, following the principle of representation model-based classification, a subdictionary reconstruction module (SDRM) is designed to determine the class label. In SDRM, each subdictionary is learned through a context-integrated training process, resulting in superior discriminative capability. In addition, to better guide NSR-Net optimization, we introduce a new composite loss function, which consists of constraint loss and residual loss, aiming to effectively recover representation coefficients and reconstruct data from the subdictionary. Experiments conducted on four distinct HSI datasets illustrate the superiority and generalization performance of the proposed method compared with advanced representation model-based and deep learning-based methods, with overall accuracy (OA) improvements of 0.72%–9.80%, 1.39%–8.09%, 0.40%–5.85%, and 0.56%–6.84% for Indian Pines, Salinas, LongKou, and Loukia, respectively. The source code will be available at:https://github.com/ZhaohuiXue/NSR-Net.

Accurately obtaining the optical/microphysical characteristics of aerosols from satellite data is important for environmental protection and air quality monitoring. In this study, using Sentinel-2 satellite data, we propose a high-resolution satellite aerosol retrieval algorithm (CKAN) via convolutional neural network (CNN) and Kolmogorov-Arnold network (KAN). Unlike traditional retrieval algorithms that require the construction of physical models, the CKAN algorithm relies entirely on deep learning. This algorithm focuses on extracting high-dimensional information from the data through CNN and learning the potential nonlinear relationships between the data through the powerful fitting ability of KAN. Compared with the existing algorithms, the CKAN algorithm is characterized by simplicity and accuracy, and does not require a large amount of auxiliary meteorological data (e.g., relative humidity and ground air pressure) enables retrieval of various aerosol parameters, including Aerosol Optical Depth (AOD), Fine-mode AOD (FAOD), Coarse-mode AOD (CAOD), and Single Scattering Albedo (SSA). To demonstrate the effectiveness of the algorithm, we retrieved aerosol optical/microphysical characteristics from Sentinel-2 imagery for four study areas. Results indicate that both the AOD and FAOD retrieved by the CKAN algorithm exhibit a high degree of correlation ( R > 0.90) with AERONET products. For CAOD and SSA, although a few poor retrievals resulted in a low overall correlation, CKAN’s retrievals show good agreement with AERONET products. The CKAN algorithm has good accuracy in high-resolution satellite aerosol retrievals, and is expected to be widely used in urban-scale aerosol monitoring.

Hyperspectral unmixing (HU) is a fundamental task in hyperspectral image analysis, aiming to decompose mixed pixels into endmember spectra and their corresponding fractional abundances. With the development of deep learning (DL), numerous DL-based unmixing methods have been proposed. However, most of these methods fail to integrate geometric priors (e.g., minimum volume constraints), limiting their performance in non-pure pixel scenarios (i.e., when no pure pixels are present in the scene). Current minimum volume constraint strategies based on the distance between endmembers and the data centroid may yield biased estimations under non-uniform data distributions. To overcome these limitations, we propose a new blind unmixing network with minimum volume constraint, named MVCUN. By jointly constraining the simplex volume through a total variation (TV) term related with the endmembers and a volume upper bound term, MVCUN achieves improved unmixing performance in non-pure pixel scenarios and exhibits robustness against non-uniform data distributions. Furthermore, we design a multi-scale global context attention (MSGCA) module that fuses local and global channel information through multi-scale convolutions and fully connected layers, thereby improving the network’s ability to estimate abundances. Experimental results on one simulated dataset and three real datasets demonstrate that our MVCUN significantly outperforms existing approaches in terms of both endmember extraction and abundance estimation accuracy.

Object detection is a crucial task in computer vision for remote sensing applications. However, the reliance of traditional methods on predefined and trained object categories limits their applicability in open-world scenarios. A key challenge in open-vocabulary object detection lies in accurately identifying unseen objects. Existing approaches often focus solely on detecting object locations, struggling to recognize the categories of previously unseen targets. To address this issue, we propose a novel benchmark where models are trained on known base classes and evaluated on their performance in detecting and recognizing unseen or novel classes. To this end, we introduce llama-Unidetector, a universal framework that incorporates textual information into a closed-set detector, enabling the generalization to open-set scenarios. Our llama-Unidetector leverages a decoupled learning strategy that separates localization and recognition. In the first stage, a class-agnostic detector identifies objects, distinguishing only between foreground and background. In the second stage, the detected foreground objects are passed through TerraOV-LLM, a multimodal large language model, for recognition, utilizing the strong generalization capabilities of large language models to infer the correct categories. We propose a self-built Vision Question Answering (VQA) remote sensing dataset, TerraVQA, and conduct extensive experiments on the NWPU-VHR10, DOTA1.0, and DIOR datasets. The llama-Unidetector achieves impressive results, with a performance of 75.46% AP, 50.22% AP and 51.38% AP on the zero-shot detection benchmarks for the NWPU-VHR10, DOTA1.0 and DIOR datasets, respectively. Our source code is available at: https://github.com/ChloeeGrace/LLaMA-Unidetector.

Active learning (AL) has achieved great success in remotely sensed hyperspectral image classification due to its ability to select highly informative training samples. An appropriate query strategy is the core of a successful AL method. In this paper, we develop a new graph signal sampling (GS)-based AL strategy which aims to query the most globally optimal and informative pixels for hyperspectral image classification. We first demonstrate the significance of smooth graph structures for GS. Then, we propose a multi-view fused graph-based GS method (GS-FusG) for active learning. Our method combines connection information from multiple perspectives to improve the sampling effectiveness and the performance of graph signal reconstruction. Besides, we present a new reconstruction method for multidimensional graph signals, which integrates the results of multi-view graph signal reconstruction. Last but not least, we develop three variants of GS-FusG based on different multi-view compositions and signal reconstruction methods. Our experimental results with real hyperspectral images demonstrate that our proposed GS-FusG can greatly enhance AL effectiveness and classification performance.

Recent advances in natural-domain multi-modal large language models (MLLMs) have demonstrated effective spatial reasoning through visual and textual prompting. However, their direct transfer to remote sensing (RS) is hindered by heterogeneous sensing physics, diverse modalities, and unique spatial scales. Existing RS MLLMs are mainly limited to optical imagery and plain language interaction, preventing flexible and scalable real-world applications. In this article, EarthGPT-X is proposed, the first flexible spatial MLLM that unifies multi-source RS imagery comprehension and accomplishes both coarse-grained and fine-grained visual tasks under diverse visual prompts in a single framework. Distinct from prior models, EarthGPT-X introduces: 1) a dual-prompt mechanism combining text instructions with various visual prompts (i.e., point, box, and free-form) to mimic the versatility of referring in human life; 2) a comprehensive multi-source multi-level prompting dataset, the model advances beyond holistic image understanding to support hierarchical spatial reasoning, including scene-level understanding and fine-grained object attributes and relational analysis; 3) a cross-domain one-stage fusion training strategy, enabling efficient and consistent alignment across modalities and tasks. Extensive experiments demonstrate that EarthGPT-X substantially outperforms prior natural and RS MLLMs, establishing the first framework capable of multi-source, multi-task, and multi-level interpretation using visual prompting in RS scenarios. The code and dataset are available athttps://github.com/wivizhang/EarthGPT-X.

Recent advances in prompt learning have allowed users to interact with artificial intelligence (AI) tools in multiturn dialog, enabling an interactive understanding of images. However, it is difficult and inefficient to deliver information in complicated remote sensing (RS) scenarios using plain language instructions alone, which would severely hinder deep comprehension of the latent content in imagery. Besides, existing prompting strategies in natural scenes are hard to apply to interpret the RS data due to significant domain differences. To address these challenges, the first visual prompting-based multimodal large language model (MLLM) named EarthMarker is proposed in the RS domain. EarthMarker is capable of interpreting RS imagery at the image, region, and point levels by levering visual prompts (i.e., boxes and points). Specifically, a shared visual encoding method is developed to establish the spatial pattern interpretation relationships between the multiscale representations of input images and various visual prompts. Subsequently, the mixed visual-spatial representations are associated with language instructions to construct joint prompts, enabling the interpretation of intricate content of RS imagery. Furthermore, to bridge the domain gap between natural and RS data, and effectively transfer domain-level knowledge from natural scenes to the RS domain, a cross-domain learning strategy is developed to facilitate the RS imagery understanding. In addition, to tackle the lack of RS visual prompting data, a dataset named RSVP featuring multimodal multigranularity visual prompts instruction-following is constructed. Extensive experiments are conducted to demonstrate the competitive performance of the EarthMarker. The proposed EarthMarker represents a significant advance in multigranularity RS imagery interpretation under the visual prompting learning framework. Our code and dataset are available athttps://github.com/wivizhang/EarthMarker.

Graph convolutional network (GCN) has garnered significant attention in hyperspectral image (HSI) classification due to their ability to model non-Euclidean structured data. Compared with convolutional neural network (CNN), GCN can perform convolutions over irregular image regions and learn global dependencies among pixels in the whole image. Most existing GCN-based methods in the HSI community rely on average aggregation or weighted average aggregation strategies to aggregate neighboring node features. This process tends to obscure the differences between the nodes. However, for HSI classification tasks with obvious intra-class variability, average aggregation is a suboptimal choice. Moreover, the quality of the initial graph structure plays a crucial role in the model’s capacity to represent spectral relationships effectively. To mitigate these issues, we propose a neighborhood adaptive graph isomorphism network (NAGIN) for HSI classification to ensure that the diversified spectra representation of land-cover can be effectively captured. The neighborhood adaptive block (NAB) enhances spectral discriminability between land-cover classes via spectral reconstruction, enabling more precise removal of anomalous pixels in neighboring nodes. The graph isomorphism network (GIN) aggregates the features of neighboring nodes in an isomorphic manner to obtain multiple spectral expressions of the same type of land-cover, ensuring that the spectral features of different land-cover classes can be accurately distinguished. The Kolmogorov-Arnold network (KAN) leverages its ability to learn adaptive activation functions to better extract and refine the spectral features aggregated by GIN. Experimental results demonstrate that NAB can effectively improve the quality of the graph structure, the GIN aggregation method is competitive in HSI classification, and the proposed NAGIN outperforms the state-of-the-art methods on several public HSI datasets.

A unified object detector needs to integrate various abilities for adapting to different remote sensing object detection tasks. However, there is a lack of a feasible way to integrate multigrained object detection requirements i.e., horizontal bounding box (HBB), oriented bounding box (OBB), and instance segmentation (InSeg) into a unified detection way. Then, it often has to design specific parametric learning ways and their corresponding architectures, which cannot be finely adaptive to various kinds of object detection tasks. Therefore, in this article, a new benchmark is set up to integrate multigrained object detection requirements of HBB, OBB, and InSeg into one challenging task of arbitrary-shaped object contour detection. At the same time, a unified object contour detector (UniconDet) is proposed for achieving multigrained object detection from complicated remote sensing scenes. First, a Fourier contour parametric modeling (FCPM) is defined to project arbitrary-shaped object contours from the spatial domain into the frequency domain. Then, it can unify spatial parametric representations of HBB, OBB, and InSeg as frequency coefficient representations, which can be used for realizing a more generic and robust parametric regression. Second, a multiview cross-attention (MVCA) feature extraction way is designed at each scale of the regression layer, which can assist UniconDet in perceiving Fourier contour parameters by exploring the coupled relations between different discrete contour sampling periods of each object. Third, a center-contour enhancing regression layer (C2-ERL) is designed to generate regional guidance and cascade contour propagation, which can ensure a more accurate center point prediction and Fourier contour parameter regression. Finally, extensive experiments are carried out on benchmarks of HBB, OBB, InSeg, and new multigrained object detection, and the results indicate that our proposed UniconDet can obtain superior performance. The source code is available athttps://github.com/ZhAnGToNG1/UniconDet.

Few-shot object detection (FSOD) has to learn classification and localization information for unseen object detection under very low-data resource regimes. However, when deficient samples are adopted for model training, it is hard to build powerful location-aware and identification abilities for well coping with agnostic bias from diverse testing scenarios; at the same time, the overfitting phenomenon is easily occurring. Therefore, in this article, a controllable generative knowledge-driven FSOD called CGK-FSOD is proposed for unseen object detection from optical remote sensing imagery. Specifically, to enrich the learnable data space of scarce samples for preventing incomplete agnostic-bias learning, while avoiding the overfitting phenomenon, a visual-textual prompt-based controllable data generation is designed to generate high-quality object detection data based on pretrained foundational models [i.e., the stable diffusion (SD) and contrastive language-image pre-training (CLIP)], which not only can introduce the generalized domain-level knowledge into the remote sensing domain but also sets up an all-round data space to support complete learning of potential agnostic bias. Furthermore, with respect to the denoising generative process of SD, a series of cross-modality generative features in latent representation space are reused for few-shot fine-tuning by the designed cross-modality feature embedding (CMFE), which not only can bring diverse generative abilities into the feature fusion step of the detector but also gracefully sets up feature representation scalability to make the detector better adapt to agnostic bias from diverse testing scenarios of FSOD. Finally, extensive experiments are executed on two public remote sensing datasets (e.g., DIOR and NWPUVHR-10), and the results indicate that the proposed CGK-FSOD is very effective and flexible for FSOD.

Hyperspectral image anomaly detection faces the challenge of difficulty in annotating anomalous targets. Autoencoder(AE)-based methods are widely used due to their excellent image reconstruction capability. However, traditional grid-based image representation methods struggle to capture long-range dependencies and model non-Euclidean structures. To address these issues, this paper proposes a self-supervised Masked Graph AutoEncoder (MGAE) for hyperspectral anomaly detection. MGAE utilizes a Graph Attention Network (GAT) autoencoder to reconstruct the background of hyperspectral images and identifies anomalies by comparing the reconstructed features with the original features. Specifically, we constructs a topological graph structure of the hyperspectral image, which is then input into the GAT autoencoder for reconstruction, leveraging the multi-head attention mechanism to learn spatial and spectral features. To prevent the decoder from learning trivial solutions, we introduce a re-masking strategy that randomly masks both the input features and hidden representations during training, forcing the model to learn and reconstruct features under limited information, thereby improving detection performance. Additionally, the proposed loss function with graph Laplacian regularization (Twice Loss) minimizes variations in feature representations, leading to more consistent background reconstruction. Experimental results on several real-world hyperspectral datasets demonstrate that MGAE outperforms existing methods.

Autoencoders (AEs) have received extensive attention in hyperspectral anomaly detection (HAD) due to their capability to separate the background from the anomaly based on the reconstruction error. However, the existing AE methods routinely fail to adequately exploit spatial information and may precisely reconstruct anomalies, thereby affecting the detection accuracy. To address these issues, this study proposes a novel Multi-scale Autoencoder Suppression Strategy (MASS). The underlying principle of MASS is to prioritize the reconstruction of background information over anomalies. In the encoding stage, the Local Feature Extractor, which integrates Convolution and Omni-Dimensional Dynamic Convolution (ODConv), is combined with the Global Feature Extractor based on Transformer to effectively extract multi-scale features. Furthermore, a Self-Attention Suppression module (SAS) is devised to diminish the influence of anomalous pixels, enabling the network to focus more intently on the precise reconstruction of the background. During the process of network learning, a mask derived from the test outcomes of each iteration is integrated into the loss function computation, encompassing only the positions with low anomaly scores from the preceding detection round. Experiments on eight datasets demonstrate that the proposed method is significantly superior to several traditional methods and deep learning methods in terms of performance.

Graph theory-based techniques have recently been adopted for anomaly detection in hyperspectral images (HSIs). However, these methods rely excessively on the relational structure within the constructed graphs and tend to downplay the importance of spectral features in the original HSI. To address this issue, we introduce graph frequency analysis to hyperspectral anomaly detection (HAD), which can serve as a natural tool for integrating graph structure and spectral features. We treat anomaly detection as a problem of graph frequency location, achieved by constructing a beta distribution-based graph wavelet space, where the optimal wavelet can be identified adaptively for anomaly detection. Initially, a high-dimensional, undirected, unweighted graph is built using the pixels in the HSI as vertices. By leveraging the observation of energy shifting to higher frequencies caused by anomalies, we can dynamically pinpoint the specific Beta wavelet associated with the anomalies' high-frequency content to accurately extract anomalies in the context of HSIs. Furthermore, we introduce a novel entropy definition to address the frequency location problem in an adaptive manner. Experimental results from seven real HSIs validate the remarkable detection performance of our newly proposed approach when compared to various state-of-the-art anomaly detection methods.

Deep learning has been widely used in the field of hyperspectral image (HSI) classification, but existing classification methods generally require a large number of labels. With rarely labeled samples, most deep learning methods have the problem of overfitting. Although few-shot learning has developed in this direction in recent years, many methods are weak in exploring the relationships between samples in the current scene. Graph-based methods have advantages in this respect, but there are some problems in graph learning, such as lack of label information guidance and multi-scale information loss. To solve these problems, in this paper, we propose a new multi-scale superpixel graph fusion method (MSSPG), and the graph structure is dynamically optimized by combining active learning (MSSPG-AL). By incorporating class relationships and multi-scale information, false connections in the graph structure can be iteratively eliminated. Experiments on three real hyperspectral data demonstrate that our method can achieve remarkable performance in the few-shot HSI classification.

Urban hyperspectral image super-resolution can rapidly acquire high-quality data with rich spatial details and spectral fidelity through technical means for urban development. However, conventional super-resolution methods for multi-spectral and natural images struggle to meet the aforementioned data requirements simultaneously. this paper proposes an urban hyperspectral super-resolution method that combines subpixel mapping and interpolation. This method aims to ensure spectral information and minimize time consumption through a high-precision surface interpolation method based on curve theory. Additionally, by utilizing subpixel mapping to introduce urban unmixing information. Finally, by employing wavelet transformation, the method integrates the effective information from both approaches, obtaining urban hyperspectral images with enhanced spatial detail and spectral fidelity. This method has been subjected to comprehensive experimentation, affirming that our proposed method surpasses the current state-of-the-art super-resolution in terms of performance and effectiveness.

Multispectral and hyperspectral image fusion has emerged as a highly effective technique for obtaining images with both high spatial and spectral resolution. This is an ill-posed problem that poses significant challenges to the optimization solution, which is often mitigated by incorporating low-rank and smooth priors to restrict the solution space. Traditionally, these priors are combined additively using nuclear norm and total variation (TV) regularization. However, the intricate interactions between these priors make it difficult to accurately characterize the prior structure using an additive approach. Moreover, their influence is heavily dependent on the trade-off parameter between the regularization terms. To address this issue, we propose a novel fusion method leveraging a joint low-rank and smooth tensor prior (LRST). The LRST method introduces a tensor nuclear norm on the gradient maps of various dimensions of the target image capitalizing on the analogous manifold structures shared between the original image and its gradient map, which seamlessly integrates the two priors into a unified regularization framework. This facilitates the precise and convenient exploitation of spatial and spectral correlations inherent in the desired hyperspectral image. Experimental findings demonstrate that compared to state-of-the-art fusion methods, the LRST approach yields finely fused images.

Normalized Difference Vegetation Index (NDVI) data from optical satellites have been widely used in the field of remote sensing. However, due to cloud cover and other extreme weather reasons, NDVI time series are often missing. In order to fill the observation gap under severe weather conditions, an NDVI time series reconstruction method that fuses optical and synthetic aperture radar (SAR) images has been developed. However, optical images often suffer from long-term data missing, severely limiting the usability of this reconstruction method. To solve this problem, this paper proposes a NDVI-like image strategy to avoid reliance on optical data. Specifically, the strategy obtains NDVI-like images by fitting the multivariate linear relationship between VV, VH polarization bands of SAR images and NDVI images. Then, the optical image information is represented by NDVI-like images, and the SAR and NDVI-like time series are input into the time series reconstruction network to reconstruct the NDVI time series in cloudy areas. Experimentation has revealed that the NDVI-like image strategy can effectively supplements optical image information for the NDVI reconstruction network, reduce the the influence of SAR image speckle noise on reconstruction outcomes, and obtain better reconstruction effects.

Global climate change leads to increasing frequency and severity of urban flood disasters, which restricts human sustainable development. So far, many studies have explored the potential of fusing remote sensing images and social media data for monitoring urban flood disasters. However, the SAR flooded characteristics of different land cover in complex urban environment are generally ignored. In this paper, we develop a new method for the fusion of heterogeneous SAR images and social media data by using domain adaptation and land cover information. The 2017 Houston flood event is taken as a case study for evaluation. According to our experiments, compared with traditional methods of optimal transport and geographic optimal transport, the proposed method can align the geo-tagged tweets with location uncertainty to nearby flooded areas of low-, medium-, and high-intensity developed areas with more remarkable performance.

Graph-based methods have excellent performance in hyperspectral image (HSI) classification because of their strong ability to explore the relationship between labeled and unlabeled samples. However, most graph-based methods do not take sufficient account of label information and more discriminant features to establish graph connections, which will lead to a lot of improper connections, and then produce over-smooth or noisy classification results. To solve this issue, we propose a posterior probability fused superpixel graph (PPSPG), which exploits the fitting ability of supervised models to encode information from labels and discriminant features into the posterior probabilities. By measuring the weights between the posterior probabilities of any two superpixels, the proposed PPSPG can fully integrate label information into graph connetions, and alleviate the over-smoothing between superpixels. Experiments on real hyperspectral data show that our method has outstanding accuracies and can obtain quite distinct classification boundaries.

The fusion of remote sensing data and social media data effectively enhances the spatiotemporal resolution of available datasets, providing assistance in addressing specific issues. For the integration of heterogeneous data, domain adaptation and Geographic Optimal Transport (GOT) methods have offered significant support. However, within the process of data fusion, the climatic environment of the data collection areas is often less considered, resulting in the omission of certain climate characteristics under remote sensing imagery, which typically significantly impact algorithm precision. In this paper, building upon the GOT, we further explore the impact of variations in different remote sensing parameters on data fusion accuracy under specific climatic conditions, leading to modifications in the GOT formula. In experiments conducted for flood scenarios, we have enhanced the accuracy of heterogeneous data fusion in this context by emphasizing the MNDWI term and removing the NDVI term from the GOT equation.

Feature extraction is a prevalent technique in hyperspectral remote sensing. Various tasks require this technique as a pre-processing step, including image classification, anomaly detection, image denoising, and so on. Edge-preserving filtering based methods have been extensively utilized for this purpose. However, these methods do not take the inherent structural and textural information into account, leading to poor performance in classifying hyperspectral images (HSIs). In this letter, a new structural and textural-aware feature extraction method is proposed that preserves the relevant structural information and removes useless textures. First, structural and textural-aware recursive filtering features (STRFs) are extracted along with an exponential form of windowed inherent variance (eWIV). Then, multi-scale STRFs are integrated by the principal component analysis (PCA) method to obtain more discriminative features (MSTRF). Finally, the fused features are fed into a pixel-wise classifier to obtain the final results. The main difference between the MSTRF method and other feature extraction methods is that the MSTRF method can make full use of the proposed eWIV map, which can help to properly characterize structure and texture in HSIs. Experimental results on several public data sets indicate that our method leads to state-of-the-art classification performance, especially in the presence of very small training set.

Hyperspectral sensors can rapidly acquire high-quality spectral data, very useful for urban monitoring applications. Unfortunately, their spatial detail is not fine enough, and methods to enhance this resolution are required. However, conventional super-resolution (SR) methods for multispectral data do not match the requirements needed to maintain high spectral fidelity. Therefore, this article proposes a hyperspectral SR method that combines subpixel mapping and interpolation, and whose main aim is to enhance urban monitoring. This method aims to guarantee spectral quality and minimize computational time through a high-precision surface interpolation method based on curve theory. Moreover, unmixing-based subpixel mapping is exploited to introduce unmixing information. Finally, using wavelet transforms, the method integrates the effective information from the two previous approaches, obtaining urban hyperspectral images with enhanced spatial details and spectral fidelity. This method has been subjected to a comprehensive experimentation, affirming that the proposed method surpasses the current state-of-the-art SR results in terms of performance and effectiveness.

Standard hyperspectral (HS) pansharpening utilizes panchromatic (PAN) images to improve the connected low-resolution HS (LRHS) images to the spatial resolutions of PANs; while arbitrary-resolution hyperspectral (ARHS) pansharpening aims to use PANs to enhance LRHS images to any desired spatial resolutions. For the challenging task of ARHS pansharpening, one of major obstacles is how to generalize the single pansharpening model learned under predetermined training scales to any pansharpening scales for future data. As implicit neural representations (INRs) have a potential to approximate continuous functions, they offer a possible alternative way to naturally resolve ARHS pansharpening. In this paper, we develop two spectral-spatial INRs for ARHS pansharpening: one is a naive pansharpening INR (NaivePINR); the other is a dynamic pansharpening INR (DynamicPINR). The former builds a novel spectral-spatial encoding to produce spectral-spatial priors of observed scenes and uses a spectral-spatial query mapping to reconstruct fine spectral details and spatial details. The latter establishes a innovative two-fold tuning mechanism to dynamically adjust both the spectral-spatial encoding and the spectral-spatial query mapping. Experimental results on serval datasets verify the excellent performances of the proposed pansharpening INRs.

Hyperspectral image (HSI) classification has garnered substantial attention in remote sensing fields. Recent mamba architectures built upon the selective state-space models (S6) have demonstrated enormous potential in long-range sequence modeling. However, the high dimensionality of hyperspectral data and information redundancy pose challenges to the application of S6 in HSI classification, suffering from suboptimal performance and computational efficiency. In light of this, this article investigates a lightweight interval group spatial-spectral mamba framework (IGroupSS-Mamba) for HSI classification, which allows for multidirectional and multiscale global spatial-spectral information extraction in a grouping and hierarchical manner. Technically, an interval group S6 mechanism (IGSM) is developed as the core component, which partitions high-dimensional features into multiple nonoverlapping groups at intervals, and then integrates a unidirectional S6 for each group with a specific scanning direction to achieve nonredundant sequence modeling. Compared with conventional applying multidirectional scanning to all bands, this grouping strategy leverages the complementary strengths of different scanning directions while decreasing computational costs. To adequately capture the spatial-spectral contextual information, an interval group spatial-spectral block (IGSSB) is introduced, in which two IGSM-based spatial and spectral operators are cascaded to characterize the global spatial-spectral relationship along the spatial and spectral dimensions, respectively. IGroupSS-Mamba is constructed as a hierarchical structure stacked by multiple IGSSB blocks, integrating a pixel aggregation-based downsampling strategy for multiscale spatial-spectral semantic learning from shallow to deep stages. Extensive experiments demonstrate that IGroupSS-Mamba significantly outperforms the state-of-the-art methods in classification accuracy and achieves lower model parameters and floating point operations (FLOPs). The code is available athttps://github.com/IIP-Team/IGroupSS-Mamba.

Hyperspectral image (HSI) classification constitutes a significant foundation for remote sensing analysis. Transformer architecture establishes long-range dependencies with a self-attention mechanism (SA), which exhibits advantages in HSI classification. However, most existing transformer-based methods are inadequate in exploring the multiscale properties of hybrid spatial and spectral information inherent in HSI data. To countermeasure this problem, this work investigates a hybrid multiscale spatial–spectral framework (HMSSF). It innovatively models global dependencies across multiple scales from both spatial and spectral domains, which allows for cooperatively capturing hybrid multiscale spatial and spectral characteristics for HSI classification. Technically, a spatial–spectral token generation (SSTG) module is first designed to generate the spatial tokens and spectral tokens. Then, a multiscale SA (MSSA) is developed to achieve multiscale attention modeling by constructing different dimensional attention heads per attention layer. This mechanism is adaptively integrated into both spatial and spectral branches for hybrid multiscale feature extraction. Furthermore, a spatial–spectral attention aggregation (SSAA) module is introduced to dynamically fuse the multiscale spatial and spectral features to enhance the classification robustness. Experimental results and analysis demonstrate that the proposed method outperforms the state-of-the-art methods on several public HSI datasets.

Hyperspectral image (HSI) classification constitutes the fundamental research in remote sensing fields. Convolutional neural networks (CNNs) and Transformers have demonstrated impressive capability in capturing spectral-spatial contextual dependencies. However, these architectures suffer from limited receptive fields and quadratic computational complexity, respectively. Fortunately, recent Mamba architectures built upon the state space models (SSMs) integrate the advantages of long-range sequence modeling and linear computational efficiency, exhibiting substantial potential in low-dimensional scenarios. Motivated by this, we propose a novel 3D-spectral-spatial mamba (3DSS-Mamba) framework for HSI classification, allowing for global spectral-spatial relationship modeling with greater computational efficiency. Technically, a spectral-spatial token generation (SSTG) module is designed to convert the HSI cube into a set of 3-D spectral-spatial tokens. To overcome the limitations of traditional Mamba, which is confined to modeling causal sequences and inadaptable to high-dimensional scenarios, a 3D-spectral-spatial selective scanning (3DSS) mechanism is introduced, which performs pixel-wise selective scanning on 3-D hyperspectral tokens along the spectral and spatial dimensions. Five scanning routes are constructed to investigate the impact of dimension prioritization. The 3DSS scanning mechanism combined with conventional mapping operations forms the 3D-spectral-spatial mamba block (3DMB), enabling the extraction of global spectral-spatial semantic representations. Experimental results and analysis demonstrate that the proposed method outperforms the state-of-the-art methods on HSI classification benchmarks. The code is available athttps://github.com/IIP-Team/3DSS-Mamba.

Spatiotemporal fusion is an important means to reconstruct the medium spatial resolution remote sensing image series. Presently, many spatiotemporal fusion approaches have been developed and adopted in research on agriculture, ecology, environment, and so on. Although these approaches have achieved remarkable performance in experiments and applications, most of them are designed to fuse all involved bands using the same model with the same parameters, which ignores the band difference. The ignorance may limit the fusion quality for some bands. To address this problem, we propose a novel spatiotemporal data fusion approach based on parameter selection (PSDFA) in this article. The core idea of the newly proposed PSDFA is producing the synthetic image pairs using available data via three means first and then selecting a similar image pair for each band to provide the parameters that are needed for their fusion. The PSDFA can not only be applied in local computers, and its simplified version can also be implemented in Google Earth Engine (GEE), which is a powerful and widely used cloud platform for remote sensing data computing. To test the PSDFA, we conduct two experiments, one in local computers and another in GEE. In local computers, the PSDFA is compared with five state-of-the-art fusion methods on two public Landsat–Moderate Resolution Imaging Spectroradiometer (MODIS) datasets. In GEE, it is used to produce the monthly 30-m image series in two study sites in the USA and compared with another GEE-based fusion approach. The experimental results demonstrate the outstanding performance of the proposed PSDFA in both local computers and GEE.

The integration of remote sensing and citizen science offers an unprecedented opportunity for observing the Earth and human activities. Recently, heterogeneous data fusion models have been proposed to align representations and geolocations of remote sensing imagery and social media data, such as optimal transport (OT) and geographic OT (GOT). However, these models generally ignore the differences in remote sensing features of the same geographical phenomenon for different land cover types, which not only affects the fusion accuracy but also leads to the loss of fusion information. In this study, we develop a general model for heterogeneous SAR imagery and social media data fusion based on OT and land cover information, namely, land cover information-constraint OT (LCIOT). Taking the 2017 Houston flood event as a case study, the experimental findings demonstrate that the proposed LCIOT can align 99% of geotagged Twitter data to flooded areas with an average transport distance of 710 m, outperforming the state-of-the-art models, i.e., OT and GOT. By combining the supervised information obtained by LCIOT with the SAR imagery, the overall accuracy (OA) and Kappa of urban flood mapping in the three study areas ranges from 0.77 to 0.82 and 0.51 to 0.61, respectively. Overall, the proposed LCIOT provides a new perspective to solve the issues of discrepancies in data distribution and geolocation uncertainty in the context of heterogeneous data fusion.

Multilabel remote sensing (RS) image classification aims to predict multiple semantic labels from an RS image. Previous methods [e.g., graph convolution networks (GCNs)] focus on mining the relationships of multiple labels, neglecting that the scene information is closely related to labels. To remedy this deficiency, in this article we propose a novel end-to-end deep neural network for multilabel RS image classification. In the proposed network, we use the GCN as the base model and introduce several new components to improve the classification performance. First, we explore hybrid contrastive learning (CL), including supervised transformation-based CL and unsupervised mix-based CL, to explicitly learn discriminative scene representations. Then, we apply the GCN-based classifier to the learned scene representations to obtain initial label prediction scores. Meanwhile, we pass the scene representations to a softmax layer to predict the probability that each image belongs to each specific scene class and use the scene-to-label information with the law of total probability to calibrate the initial label prediction scores. Finally, we incorporate CL, scene classification, and multilabel classification into a unified learning framework using uncertainty to weigh different losses. Experimental results on two benchmark RS datasets demonstrate the superiority of our proposed network for multilabel image classification.

Hyperspectral image (HSI) and LiDAR data are complementary to each other, which can be combined to improve the classification performance. However, existing deep network models do not sufficiently consider their complementarity to design the network structure and loss functions. Moreover, there lacks a hierarchical mutual-assistance learning mechanism that leverages the modality-shared features to enhance the modality-specific ones and vice versa. In view of these, we propose a novel height information guided hierarchical fusion-and-separation network (HFSNet) for joint classification of HSI and LiDAR data. HFSNet consists of three major components, i.e., dual-structure feature encoders (DSFEs), feature fusion-and-separation blocks (F2SBs), and an edge decoder (ED). Specifically, the transformer and convolutional neural network are introduced in DSFEs to encode the spectral and spatial information of HSI and LiDAR data, respectively. In F2SBs, the deformable convolution-based height information guided fusion module and the modality separation refinement module are proposed to sequentially extract modality-shared and modality-specific features. Additionally, the ED is incorporated into our model to predict the LiDAR edge map from the HSI feature to improve the model’s generalization ability. As such, the learned features from HSI and LiDAR data are deeply fused and mutually enhanced. Experiments on three benchmark datasets show the superiority of HFSNet to the state-of-the-art methods for jointly classifying HSI and LiDAR data with limited training samples.

Supervised hyperspectral image classification suffers from the overfitting problem when limited labels are available. Graph-based semisupervised classifiers can tackle this problem by building connections between labeled and unlabeled samples. In this work, we prove the following two propositions for an optimal graph: 1) the interclass connection weights must be 0 and 2) for a given class, a subset must contain labeled samples or be connected to the remaining subset. In a semisupervised scenario, it is very difficult to ensure that the aforementioned propositions hold. Here, we introduce a new discriminant superpixel graph (dSPG) to build a suboptimal graph, which combines a newly proposed within-superpixel graph, aimed at disconnecting pixels belonging to different classes in a superpixel (so as to decrease interclass connection weights) and a between-superpixel graph that connects spectral adjacent superpixels (to increase the intraclass subset connections). We further propose a dSPG regularizer for hyperspectral image classification and a dSPG-guided graph convolutional network (dSPGCN) to extract discriminant features. Experimental results on real hyperspectral datasets demonstrate the good performance of our newly proposed dSPG for semisupervised hyperspectral image classification. The source codes for this study are available athttps://github.com/yulong112/dSPG.

Multimodal remote sensing image recognition aims to identify a category of land cover for every pixel with consistency and complementary information provided by different modalities. Most existing methods perform land cover recognition in a supervised manner with explicit label guidance. It is challenging to perform recognition without label guidance due to the complex spatial distribution and modality incompatibility, especially for large-scale data. In this article, we propose a dual graph learning affinity propagation (DGLAP) method for multimodal remote sensing image clustering. Based on the consistent spatial distribution from local regions, the proposed method learns an$N \times M$consensus anchor graph from N denoised pixels and M anchors by adaptive weighting different modalities along with projection learning. Meanwhile, an optimal$M \times M$compressed consensus anchor graph is learned from the updated anchors in different modalities with diverse adaptive contributions and connectivity constraint. Since$M \ll N$, clustering results can be efficiently obtained according to affinity propagation from the pseudolabeled anchors to the pixels without additional steps. An alternating optimization algorithm is devised to solve the proposed formulation. This is the first attempt to propose a ultraefficient graph-based clustering method with linear time complexity$\mathcal {O}(N)$and low time cost for large-scale multimodal remote sensing data. Extensive experiments on three datasets demonstrate the superiority of the proposed method over the state-of-the-art methods in both efficacy and efficiency. The code is released athttps://github.com/ZhangYongshan/DGLAP.

Hyperspectral image (HSI) super-resolution, which aims at improving the spatial quality of HSIs by fusing a low spatial resolution HSI (LR-HSI) with a high spatial resolution multispectral image (HR-MSI), has drawn significant attention. Numerous LR-HSI and HR-MSI (HSI-MSI) fusion algorithms have emerged in recent times, yet they suffer from a lack of generality and integration, which hampers their usability for non-expert users. Moreover, these algorithms encounter significant challenges due to the exponential increase in remote sensing data volume. In this study, we propose a unified cloud-based framework for HSI-MSI fusion based on the general distributed alternating direction method of multipliers that incorporates nonlocal principles and tensor decomposition. The framework not only provides end-users with visualization modeling capabilities equipped with standard and comprehensive components, but also enhances the parallel processing capabilities of cloud computing. We employ a new proposed nonlocal adaptive low-rank coupled tensor canonical polyadic (CP) decomposition algorithm as a case study to evaluate the performance of this framework. Specifically, we establish the LR-HSIs and HR-MSIs relationship using order-4 coupled tensor CP decomposition and suggest an adaptive CP rank estimation method for achieving better super-resolution results. Experimental results on publicly available datasets demonstrate that the proposed parallel distributed optimization algorithm can achieve significant speedup with guaranteed accuracy. The proposed framework enables convenient and efficient processing of large-scale remote sensing data, effectively addressing the challenges associated with handling large data volumes. The source code of our method is released and available online at https://github.com/ZpWaitingForSunshine/DNAC4TCP/.

Most existing techniques consider hyperspectral anomaly detection (HAD) as background modeling and anomaly search problems in the spatial domain. In this article, we model the background in the frequency domain and treat anomaly detection as a frequency-domain analysis problem. We illustrate that spikes in the amplitude spectrum correspond to the background, and a Gaussian low-pass filter performing on the amplitude spectrum is equivalent to an anomaly detector. The initial anomaly detection map is obtained by the reconstruction with the filtered amplitude and the raw phase spectrum. To further suppress the nonanomaly high-frequency detailed information, we illustrate that the phase spectrum is critical information to perceive the spatial saliency of anomalies. The saliency-aware map obtained by phase-only reconstruction (POR) is used to enhance the initial anomaly map, which realizes a significant improvement in background suppression. In addition to the standard Fourier transform (FT), we adopt the quaternion FT (QFT) for conducting multiscale and multifeature processing in a parallel way, to obtain the frequency domain representation of the hyperspectral images (HSIs). This helps with robust detection performance. Experimental results on four real HSIs validate the remarkable detection performance and excellent time efficiency of our proposed approach when compared to some state-of-the-art anomaly detection methods.

Anomaly detection is a fundamental task in hyperspectral image (HSI) processing. However, most existing methods rely on pixel feature vectors and overlook the relational structure information between pixels, limiting the detection performance. In this article, we propose a novel approach to hyperspectral anomaly detection that characterizes the HSI data using a vertex- and edge-weighted graph with the pixels as vertices. The constructed graph encodes rich structural information in an affinity matrix. A crucial innovation of our method is the ability to obtain internal relations between pixels at multiple topological scales by processing different powers of the affinity matrix. This power processing is viewed as a graph evolution, which enables anomaly detection using vertex extraction formulated as a quadratic programming problem on graphs of varying topological scales. We also design a hierarchical guided filtering architecture to fuse multiscale detection results derived from graph evolution, which significantly reduces the false alarm rate. Our approach effectively characterizes the topological properties of HSIs, leveraging the structural information between pixels to improve anomaly detection accuracy. Experimental results on four real HSIs demonstrate the superior detection performance of our proposed approach compared to some state-of-the-art hyperspectral anomaly detection methods.

A novel InSAR-based time-series prediction framework was proposed to analyze ground subsidence in underground coal mines in Yuzhou City. The study utilized Sentinel-1A descending track data and employed a deep learning model. Initially, the SBAS-InSAR technique was employed to examine the spatiotemporal evolution patterns of ground subsidence in the designated area. The wavelet transform algorithm was applied to extract the trend term displacement and periodic term displacement. Subsequently, a univariate LSTM network was utilized to predict the trend term displacement, while a multivariate CNN-LSTM network was employed to predict the periodic displacement. The preliminary results demonstrated a high level of agreement between the time-series predictions obtained through this framework and the InSAR monitoring values.

Due to its unique geographical location and development potential, the Yangtze River Economic Belt (YEB) has emerged as one of the most economically developed regions in China. The study employs remote sensing data to conduct a coupled coordination analysis of the ecological environment and urbanization processes of the three major urban agglomerations in the YEB using the Coupled Coordination Model (CCD). The results show that (1) the mean RSEI values of the three urban agglomerations initially declined, then fluctuated upwards; (2) the CNLI values of all three urban agglomerations maintained a continuous increase, with the urbanization level of the Yangtze River Delta urban agglomeration being significantly higher than that of the other two urban agglomerations; and (3) The degree of coupling and coordination of the Yangtze River Delta urban agglomeration was significantly higher than that of the Chengdu-Chongqing urban agglomeration and the middle reaches of the Yangtze River urban agglomeration.

Surface deformation is a complex geological phenomenon with potential threats to urban construction and human safety. Time-series prediction of surface deformation has an important role in mitigating the impact of such hazards. This study utilizes the small baseline subset interferometric synthetic aperture radar method (LiCSBAS) to monitor the long-time surface deformation in the main urban area of Kunming, and constructs a deep neural network model (TCN-GRU) to predict typical surface deformation points in the study area. The results show that Kunming city experienced a surface deformation rate of -44.23-17.65 mm/y from March 2018 to July 2022, along with the presence of five subsidence funnels. The TCN-GRU model demonstrates the best short-term prediction performance on different datasets, and significantly outperforms other traditional deep learning models. The results of this paper can serve as a valuable reference for the study of urban surface deformation.

Natural hazards frequently threaten human life, ecosystem, and economy all around the world, while landslides are the most destructive ones. Landslide susceptibility mapping (LSM) is an effective strategy to determine the probability of future landslide events by involving a comprehensive analysis of various factors, including geological environmental, historical landslides and the landslide physical laws. As artificial intelligence techniques are becoming more popular in LSM, it is important to understand how decisions are made by these models. This study aims to use representative ML (ML) and deep learning (DL) models (include random forest (RF), support vector machine (SVM), residual neural networks (ResNet) and Densely connected convolutional networks (DenseNet)) to map the landslide susceptibility of the study area in Zigui, and then visualize the decision-making process of the model through an explainable artificial intelligence (XAI) technology, so as to provide more transparency and reliability about the occurrence of landslides.

Remote sensing change detection (RSCD) can recognize large-scale spatial building distribution changes, saving a lot of manpower and material resources compared to field surveys or manual visual interpretation. The existing deep learning based change detection (CD) networks are commonly improved from segmentation networks. Although the overall accuracy of CD tasks has been greatly improved, there are still noticeable shortcomings in terms of the object completeness and edge details. For this issue, we proposed a Siamese Residual Network (SRNet) based on U-Net for VHR images CD tasks. We first constructed a Siamese network for better utilization of bi-temporal images information. And then we added residual connection to form a convolutional block for faster model convergence and easier training difficulties. We conduct comprehensive experiments on two benchmark datasets. Both the visual and quantitative results show that our proposed SRNet is superior to the other five state-of-the-art networks.

High-quality landslide susceptibility mapping (LSM) is an important step in landslide disaster prevention and mitigation. In order to explore and evaluate the practical performance of a convolutional and transformer hybrid model in landslide susceptibility mapping, this study selected 202 historical landslides and seven conditioning factors to construct a geographical spatial dataset for LSM. The hybrid model called the compact convolutional transformer (CCT) model was used to perform LSM in the Three Gorges Reservoir area in China, and it was compared with vision transformer (ViT), convolutional neural network (CNN), and residual network (ResNet). The results show that the CCT model has the best overall performance, achieving the highest accuracy in multiple statistical metrics, and improving the accuracy of LSM. This study provides a new approach for obtaining high-quality LSM using a convolutional and transformer hybrid model.

Spatiotemporal fusion (STF) can produce synthetic remote sensing data with both high spatial and high temporal resolution by blending the temporally sparse fine resolution data and temporally dense coarse resolution data, for which it has been adopted in many fields, such as agriculture, environment and ecology. In the literature, Moderate Resolution Imaging Spectroradiometer (MODIS) images are the most frequently employed as the coarse resolution images in STF due to their high quality and availability. However, the MODIS images contain significant strip noise sometimes, which will lead to unfavorable fusion results. In this letter, a MODIS strip noise removal strategy for STF (NRSTF) is suggested. Specifically, the NRSTF first removes the strip noise of MODIS image of the base time utilizing the bands relationship from the base fine resolution image, then reconstructs that of the predicted time via an adaptive convolution-based noise estimation approach. Finally, the reconstructed, noise-free MODIS images are fused with the base fine resolution image to predict the missing fine resolution images. Four wildly used STF approaches, including STARFM, FSDAF, Fit-FC, and RASDF are selected to test the NRSTF on an actual Landsat-MODIS dataset, whose MODIS images suffers from serious strip noise in two short wave infrared (SWIR) bands. The experiment results demonstrate that the proposed NRSTF can not only effectively eliminate strip noise, but also significantly enhance the accuracy and applicability of the fusion results. The source code of NRSTF is available at https://github.com/lijli26/NRSTF.git.

The salient object detection of optical remote sensing images (ORSI-SOD) is an important research direction in ORSI processing, which has achieved promising results in the last few years. Many recent works heavily rely on feature learning of regions for the improvement of the detection accuracy, while neglecting the concrete role of edge and skeleton information in the calculation. In this work, we propose a two-stage edge and skeleton guidance network (ESGNet) for ORSI-SOD in a coarse-to-fine way, and further demonstrate that the fused features of edge and skeleton are essential for ORSI-SOD. In the first stage, we construct the spatial graph attention (SGA) module for saliency features to generate an initial saliency map, and apply the spatial self-optimization (SSO) to enhance edge and skeleton features. The multi-level interactive fusion (MIF) module is used for the adequate integration of edge and skeleton features into saliency features. In the second stage, with the aim to accomplish better prediction of salient object localization and shape, the feature enhancement integration operation is introduced to recover object details from the learned edge and skeleton features. Extensive experiments on three public ORSI-SOD datasets demonstrate that our ESGNet achieves competitive performance with the state-of-the-art methods and also confirms the importance of edge and skeleton information for ORSI-SOD. Meanwhile, generalizability experiments on natural image datasets show that our method is competent for many types of SOD tasks. The code and results of our method are available athttps://github.com/aoao0206/ESGNet.

Hyperspectral unmixing is a crucial task in hyperspectral image processing and analysis. It aims to decompose mixed pixels into pure spectral signatures and their associated abundances. However, most current unmixing methods ignore the reality that the same pixel of a hyperspectral image has many different reflections simultaneously. To address this issue, we propose a multi-task autoencoding model for multiple reflections, which can improve the algorithm’s robustness in complex environments. Our proposed framework uses 3D-CNN-based networks to jointly learn spectral-spatial priors and adapt to different pixels by complementing the advantages of other unmixing methods. The proposed method can quantitatively evaluate each area of data, which helps improve the algorithm’s interpretability. This paper presents MAHUM (Multi-tasks Autoencoder Hyperspectral Unmixing Model), which stacks multiple models to deal with various reflections of complex terrain. We also perform sensitivity analysis on some parameters and show experimental results demonstrating our method’s ability to express the adaptability of different materials in different methods quantitatively.

Multispectral image pan-sharpening aims to increase the spatial details of multispectral images by fusing multispectral and panchromatic images. Existing component substitution-based deep learning pan-sharpening is generally regarded as a black box and fails to mine the image interaction relation with physical significance in each step of pan-sharpening, which not only limits the improvement of image resolution, but also ignores the physical interpretability of the models. To improve this situation, according to the traditional component substitution-based detail injection pan-sharpening model, we consider the matrix calculation in each step as the transformation between image pixel values and carry out linear transformations, and therefore the pan-sharpened multispectral image is represented as the sum of two multispectral images. Then given the spatial and spectral heterogeneity, the two summed images are decomposed based on the fact that any real number can be expressed as the product of two real numbers. Ultimately, the multispectral image pan-sharpening model can be constructed as the sum of two Hadamard products. We design a dual-branch network with attention mechanisms that merges the sum and the Hadamard products into a concise formulation. This method not only enhances physical interpretability but also improves spatial resolution. Experiments on five real-world datasets validate that the proposed multispectral image pan-sharpening model can improve performance.

Image stitching aims to mosaic individual images together to build a broader panorama, wherein image registration is the most critical step. In this step, all subimages are normally brought into alignment by warping functions, each of which is often parameterized by rotation, scale, and translation factors. Existing methods usually have to estimate such parameters simultaneously, which incur the problems of unstable and time-consuming computations, especially for stitching large number of unmanned aerial vehicle (UAV) images. In this article, a novel stitching method using vector shape preserving (VSP)-based warping is proposed, which is especially suitable for stitching many UAV images. We innovatively construct warping vectors and involve them to measure registration error to preserve vector shapes, which allows separately dealing with a re-plane stage and a translation stage. Additionally, a novel scale regularization processing connected to the warping is designed for tractable computation. Our method is able to achieve low computational cost, high alignment accuracy, and meanwhile valid real-world interpretation. Experimental results on four real-world UAV image datasets validate the excellent performance of our method.

Hyperspectral (HS) pansharpening seeks to integrate low spatial resolution HS (LRHS) images with connected panchromatic (PAN) images to produce high spatial resolution HS (HRHS) images. Traditional pansharpening convolutional neural networks (CNNs) directly map LRHS and PAN images into HRHS images under fixed network parameters, which imply static pansharpening rules. However, real-world HS data are often characterized by spatial variations, and intuitively, the pansharpening rules should be dynamic. To deal with the dilemma, in this article, we develop dynamic HS pansharpening CNNs. We first specify the concepts of dynamic pansharpening and static pansharpening. Then, we propose a learn-to-learn-oriented pansharpening CNN paradigm, which aims to learn a how-to-learn rule to produce spatially adaptive pansharpening rules and comprises three stages of preliminary fusion, scene-sensitive modulation, and spectral reconstruction. Finally, following the paradigm, we design two groups of dynamic pansharpening CNNs (DyPNNs), i.e., internal-connection-based and external-connection-based. They involve various spatial modulations, including spatial affine transform (AT), spatial dynamic convolution (DC), or improved spatial attention (SA), and, thus, consist of six specific DyPNNs: IC-AT-DyPNN, IC-DC-DyPNN, IC-SA-DyPNN, EC-AT-DyPNN, EC-DC-DyPNN, and EC-SA-DyPNN. Experimental results on several HS datasets verify the effectiveness of the proposed DyPNNs in terms of both the spatial reconstruction and spectral fidelity.

Convolutional neural networks (CNNs) have been widely employed for hyperspectral image (HSI) classification due to their powerful ability to extract local spatial features. However, CNN-based methods cannot establish long-range dependencies among sequences of pixels. Transformers offer significant advantages when processing sequential data and can establish global relationships, but they still encounter a number of challenges, such as their limited spatial feature extraction ability, or their high computational cost. In order to address the aforementioned issues, we develop a new fast HSI classification approach combining transformers and SimAM-based CNNs. The latter are utilized to extract better spatial features, where the complex spatial characteristics of HSIs are retrieved using an improved hierarchical 2D dense network structure. A dual attention unit (DAU) mechanism is then utilized to direct the model’s attention to discriminative spatial pixel characteristics and effective feature map channels, while suppressing information that is irrelevant for classification purposes. Regarding the spectral features, after extracting hierarchical local characteristics from various convolutional layers (using the hierarchical dense network structure), a squeezed-enhanced axial transformer is employed to establish global long-range dependencies whilst enhancing the ability of the model to extract local detail features in the HSI. Besides, a new Lion optimizer is utilized to improve the classification performance of our model. Our quantitative and comparative experiments on four benchmark datasets demonstrate the effectiveness of the proposed approach provides better classification results than other state-of-the-art approaches. Moreover, our FTSCN also achieves better classification results than other methods in practical scenarios.

Deep learning (DL) techniques have shown remarkable progress in remotely sensed hyperspectral image (HSI) classification tasks. The performance of DL-based models highly relies on the quality and quantity of labeled data. However, manual labeling is a laborious and expensive process that requires substantial efforts from human experts. Active learning (AL) techniques have been developed to alleviate the burden of manual annotation by selecting the most informative and uncertain samples for labeling. In this paper, we propose a new class-wise graph embedding-based AL (CGE-AL) framework implemented by a class-wise graph convolutional network (CGCN). First, we train a classifier with labeled data and infer latent features from labeled and unlabeled samples with the trained parameter. Then, we group the labeled data into multiple one-label sets by category. In a class-wise manner, we initialize the nodes of the graph with one-label and unlabeled features, which are then fed into CGCN. By updating the graph parameters with binary loss, CGCNs measure the uncertainty between labeled nodes and unlabeled nodes. To select the most valuable sample for labeling, we adopt the class minimum uncertainty to query the unlabeled nodes with higher overall uncertainty. We repeat this process with the updated labeled set to retrain our classification model and CGCNs. Extensive experiments demonstrate the outstanding performance of our method compared to other state-of-the-art AL-based approaches.

Due to the complex and highly heterogeneous land cover in urban areas, the single-temporal pixel-wise and parcel-wise classification cannot realize high-precision recognition of ground objects. Semantic segmentation of satellite image time series (SITS), can distinguish objects with similar spectral reflection and temporal evolution. But optical SITS have problems of uneven time-frequency distribution and incomplete, which makes it impossible to directly use existing models to carry out time series semantic segmentation. This study proposes a semantic segmentation network that combines optical and radar SITS, named Multi-Source Temporal Attention Fusion-Based Temporal-Spatial Transformer (MTAF-TST), to achieve high-precision land cover classification in urban areas. Firstly, MTAF-TST uses the Transformer spatial semantic segmentation module to extract the spatial context information of ground objects to realize pixel-level land cover classification, which relieves the salt-and-pepper phenomenon that is easy to occur in traditional pixel-by-pixel classification in complex scenes. Secondly, MTAF-TST uses the Transformer time feature extraction module to mine long-range time-dependent and high-level semantic information, overcoming the drawbacks of traditional convolutional and recurrent neural networks that cannot mine long-range time-dependent features of SITS. Finally, MTAF-TST uses a multi-source temporal attention fusion module to fuse the depth features of optical and radar SITS, which overcomes the shortcomings of traditional direct feature stitching methods that cannot make full use of time-correlated features, achieving high-precision land cover classification. The experimental results show that the MTAF-TST can realize the complementarity of radar and optical SITS in terms of timing integrity, color, texture, etc., and effectively improve the accuracy of SITS classification.

Nowadays, convolutional neural network (CNN)-based deep learning models have been popularized in hyperspectral image classification (HSIC) and achieved significant accuracy gains, which is due to their hierarchical and nonlinear feature learning patterns. However, too deeper network structures may induce a huge amount of parameters and excessive computing overhead, leading to the need for plenty of labeled samples for training. Besides, highly abstract semantic features may not be the most suitable for hyperspectral land-cover classification tasks. To address these issues, we propose a fairly lightweight network model for HSIC, which is built on a type of exquisitely designed convolution module, namelygrouped separable convolution. Compared with the standard convolution, the designed grouped separable convolution module combines grouped convolution with point-wise convolution, which not only greatly reduces the number of parameters of convolution kernels, but also caters to the inherent 3D cube style of hyperspectral image data. Moreover, Gabor filters are introduced to modulate the grouped separable convolution kernels, so as to further use relatively few convolution kernels with additional prior orientation and scale information for feature extraction. The experiments are carried out on four real hyperspectral datasets, and the experimental results reveal that the proposed model has low training cost and memory overhead. Compared with some existing deep network models that have been applied to HSIC, our proposed model can achieve competitive classification accuracy with fewer training samples.

Lithological unit classification (LUC) refers to the classification of different types of rocks within an area, and it has been widely used in many fields, such as resource surveys and infrastructure planning. However, traditional field surveys require a lot of resources and time. Since remote sensing technology can rapidly acquire information without regional limitations, many researchers have focused on classifying lithological units with remote sensing images. However, in an area covered by vegetation, the beneficial information directly provided by remote sensing images is limited. Moreover, lithological interpretation often requires geological prior knowledge for guidance, which cannot be provided by remote sensing images. Thus, this study designed a dual-branch deep learning model to extract geological prior knowledge from geological information, and improve the accuracy of LUC. In the process of feature transmission of the model, a Dense Attention residual - Atrous Spatial Pyramid Pooling (DA-ASPP) module was proposed to maximize the preservation of lithological units’ features. The DA-ASPP integrates the idea of dense connection into ASPP for multiscale object feature preservation and adds residual structure into the channel attention mechanism to screen out the representative features of lithological units. The study area was located in southeastern Hubei Province, China, with seven categories of lithological units. A total of seven deep-learning networks were compared. The proposed method achieved a mean Intersection over Union (IOU) of 44.61% with a Macro-F1 of 56.54%, which were better than those of comparison models. Moreover, the visualization results demonstrated the superiority of the proposed model in LUC.

Land cover classification in mining areas (LCMA) is essential for the environmental assessment of mines and plays a crucial role in their sustainable development. The shapes of mine land occupation elements are irregular, and the overall proportion of their area is relatively small. Therefore, their features may be easily lost during feature extraction, which limits the interpretation accuracy in mining areas. This study attempts to address these issues. We propose a model named EG-UNet to enhance the features of elements with few samples and to capture long-range information. The proposed EG-UNet includes two main modules. First, the edge feature enhancement module, the edges of elements of mine land occupation contain more information than other spatial locations. Hence, during the feature extraction of elements, a Sobel operator is used to extract the object boundary, which increases the weight of these features before the pooling operation for their preservation. Second, the long-range information extraction module, long-range information helps extract tiny objects, such as dumping grounds in the mining area. We present a graph convolutional network (GCN) to capture the long-range features and apply convolutional neural networks to learn the graph construction. A total of ten deep-learning networks were compared using the LCMA semantic segmentation dataset. Our model exhibited the best performance, especially in classifying classes with few samples. Furthermore, to evaluate the general ability of EG-UNet, a benchmark-Gaofen Image Dataset (GID) was used, and the result still reflected the superiority of our method.

Hyperspectral (HS) imaging has been widely used in various real application problems. However, due to the hardware limitations, the obtained HS images usually have low spatial resolution, which could obviously degrade their performance. Through fusing a low spatial resolution HS image with a high spatial resolution auxiliary image (e.g., multispectral, RGB or panchromatic image), the so-called HS image fusion has underpinned much of recent progress in enhancing the spatial resolution of HS image. Nonetheless, a corresponding well registered auxiliary image cannot always be available in some real situations. To remedy this issue, we propose in this paper a newly single HS image super-resolution method based on a novel knowledge-driven deep unrolling technique. Precisely, we first propose a maximum a posterior based energy model with implicit priors, which can be solved by alternating optimization to determine an elementary iteration mechanism. We then unroll such iteration mechanism with an ingenious Transformer embedded convolutional recurrent neural network in which two structural designs are integrated. That is, the vision Transformer and 3D convolution learn the implicit spatial-spectral priors, and the recurrent hidden connections over iterations model the recurrence of the iterative reconstruction stages. Thus, an effective knowledge-driven, end-to-end and data-dependent HS image super-resolution framework can be successfully attained. Extensive experiments on three HS image datasets demonstrate the superiority of the proposed method over several state-of-the-art HS image super-resolution methods.

Spatio-temporal fusion is a feasible way to generating the synthetic remote sensing data with high spatial resolution and high temporal resolution simultaneously by blending the fine and coarse resolution satellite images. To date, dozens of spatio-temporal fusion approaches have been developed. A basic rule of these approaches is the bands of coarse and fine images must be corresponding, which means the quality of fused images depends on that of both fine and coarse images. In the literature, the MODIS images are the most wildly used coarse images in spatio-temporal fusion. However, the MODIS images may suffer from serious stripe noises in the short-wave infrared-1 and short-wave infrared-2 bands, which will lead to undesired results of spatio-temporal fusion. To address this problem, we develop a noise proof strategy in this paper, which takes advantage of the spectral correlation of base fine image to remove the stripe noises of the base MODIS image, then the spatial correlation of base MODIS image is exploited to restore the MODIS image of the predicted time. Finally, the reconstructed MODIS images are fused with the base fine image to predict the missing fine images. The strategy is tested via real Landsat and MODIS images, and the experimental result demonstrates it is not only effective in removing the stripe noises of MDOIS short-wave infrared-1 and short-wave infrared-2 bands, but also able to improve the fusion accuracy.

Since traditional convolutional neural network (CNN) is used to extract the spectral-spatial features of hyperspectral image (HSI) will result in lots of spatial information redundancy. Octave convolution is used to replace the traditional CNN to reduce spatial redundancy and expand the receptive field. However, the methods based on 3D octave convolution may cause many parameters and the model to be complicated. To address these issues, we propose an HSI classification approach based on a double-branch multi-scale spectral-spatial convolution network (DBMS) in this paper. Firstly, We utilize 2D octave convolution and 3D DenseNet sub-networks with different kernels sizes to extract complex spatial features and spectral features, respectively. Furthermore, a channel attention module and a spectral attention module are employed in this two sub-network respectively, to highlight the important feature areas and specific spectral bands that consist of significant information for the classification. Compared with several other state-of-the-art methods, our proposed method can achieve competitive performance on Salinas Valley (SV) HSI dataset.

In this letter, we propose a new phase-induced Gabor-based multiview active learning (MVAL) (PGMVAL) approach for hyperspectral image (HSI) classification. Our main contribution is to explore the potential of the phase offset term$P$in hand-crafted Gabor feature extraction, which is rarely exploited in previous works. The Gabor filters with$P$added, named as the phase-induced Gabor filters, are able to adjust their frequency response characteristics through$P$. Specifically, we utilize the phase-induced Gabor filtering for view generation purposes under a MVAL framework. As a result, PGMVAL is capable to exploit the complementary information residing in the phase-induced Gabor features corresponding to different$P\text{s}$and simultaneously avoids high memory consumption and a large number of training samples required caused by introducing a new parameter. The experimental results obtained on two benchmark HSI data sets show that the proposed PGMVAL approach using phase-induced Gabor filtering could achieve better classification results with limited training samples.

Traditional hyperspectral (HS) pansharpening aims at fusing a HS image with its panchromatic (PAN) counterpart, to bring the spatial resolution of the HS image to that of the PAN image. However, in many practical applications, arbitrary resolution HS (ARHS) pansharpening is required, where the HS and PAN images need to be integrated to generate a pansharpened HS image with arbitrary resolution (usually higher than that of the PAN image). Such an innovative task brings forth new challenges for the pansharpening technique, mainly including how to reconstruct HS images beyond the training scale and how to guarantee spectral fidelity at any spatial resolutions. To tackle the challenges, we present a novel convolutional neural network (CNN)-based method for ARHS pansharpening called ARHS-CNN. It is based on a two-step relay optimization process, which is associated with a multilevel enhancement subnetwork and a rescaling subnetwork. With a careful design following the thread, our ARHS-CNN is able to pansharpen HS images to any spatial resolutions using just a single CNN model trained on a limited number of scales while meantime to keep spectral fidelity at those resolutions, which wins an obvious advantage over traditional pansharpening methods. Experimental results obtained on several datasets verify the excellent performance of our ARHS-CNN method.

Salient object detection (SOD) for optical remote sensing images (RSIs) aims at locating and extracting visually distinctive objects/regions from the optical RSIs. Despite some saliency models were proposed to solve the intrinsic problem of optical RSIs (such as complex background and scale-variant objects), the accuracy and completeness are still unsatisfactory. To this end, we propose a relational reasoning network with parallel multi-scale attention for SOD in optical RSIs in this paper. The relational reasoning module that integrates the spatial and the channel dimensions is designed to infer the semantic relationship by utilizing high-level encoder features, thereby promoting the generation of more complete detection results. The parallel multi-scale attention module is proposed to effectively restore the detail information and address the scale variation of salient objects by using the low-level features refined by multi-scale attention. Extensive experiments on two datasets demonstrate that our proposed RRNet outperforms the existing state-of-the-art SOD competitors both qualitatively and quantitatively.

Urban informal settlements (UIS) are high-density population settlements with low standards of living and supply. UIS semantic segmentation, which identifies pixels corresponding to informal settlements in remote sensing images, is crucial to the estimation of poor communities, urban management, resource allocation, and future planning, particularly in megacities. However, most studies on informal settlement mapping are either based on parcels (image classification) or pixels (semantic segmentation). Few studies utilize object information to improve UIS mapping. Since informal settlements are formed by buildings (objects), utilizing object information can improve UIS semantic segmentation. Furthermore, current UIS mapping studies mainly focus on using single-modality remote sensing images, and there is a lack of related research on using multimodal data. Due to the spatial heterogeneity of informal settlements, using only a single modality of remote sensing image features limits the effectiveness and accuracy of informal settlements semantic segmentation. Aiming at achieving fine-scale UIS mapping results, this paper proposes a UIS semantic segmentation method, namely UisNet, that utilizes a transformer-based block to receive multimodal data, including high-spatial-resolution remote sensing images (parcel- and pixel-level) and building polygon data (object-level) to identify UIS. The experiments were conducted in Shenzhen City, and they confirmed the superior performance of UisNet, which achieved an overall accuracy (OA) of 94.80% and a mean intersection over union (mIoU) of 85.51% in the testing set of the manually labeled UIS semantic segmentation dataset (UIS-Shenzhen dataset) and outperformed the best models on semantic segmentation tasks. Besides, we add a set of experiments on a public dataset (GID dataset) and compare our method with the current state-of-the-art semantic segmentation methods. Experiments show that the proposed UisNet improves mIoU by 1.64% to 7.58% compared to other methods. This work will be available at https://github.com/RunyuFan/.

Urban Villages (UV) refer to areas of urban informal settlements lagging behind the rapid urbanization process. Recent studies focus on using satellite images to classify UV. However, satellite images only capture objects from a bird-eye perspective, thus cannot obtain complex spatial relationships between objects. In UV areas, buildings and objects are usually dense, small in size, and obscure each other. Therefore, it is challenging to classify UV accurately using only satellite images with bird-eye perspectives. In this paper, to solve this problem, we proposed a novel method that uses satellite images combined with streetview images to classify UV. Specifically, we propose a novel multilevel spatial-channel feature fusion network, namely FusionMixer, that integrates CNN-based feature extraction modules and a multilevel spatial-channel feature fusing layer to make an optimal UV classification. Experiments were conducted in Shenzhen City (the RsSt-ShenzhenUV dataset) and a public UV dataset (theS2UVdataset). The proposed FusionMixer achieved an increase of OA by 8.83% and 8.84%, and improves Kappa by 0.1765 and 0.1770 in the validation set and testing set, compared to the second-best fusion models in RsSt-ShenzhenUV dataset. Experiments in theS2UVdataset show that the proposed FusionMixer improves OA by 1.82% and Kappa by 0.04 compared to other methods. We also added a set of experiments on a public dataset (Houston dataset) and compare our method with the current state-of-the-art multimodal fusion methods to prove the generalization of the proposed FusionMixer in fusing other multimodality data. These experiments confirmed the superior performance of the proposed FusionMixer.

Geological remote sensing interpretation can extract elements of interest from multiple types of images, which is vital in geological survey and mapping, especially in inaccessible regions. However, due to numerous classes, high interclass similarities, complex distributions, and sample imbalances of geological elements, the interpretation results of machine-learning (ML)-based methods are understandably worse than manual visual interpretation. Additionally, scholars in remote sensing have mainly carried out their works to interpret a single geological element category, such as mineral, lithological, soil and structure. The interpretation of multiple geological elements is missing, which is more in line with the open world. To improve the interpretation results of ML-based methods and reduce the labor cost in geological survey and mapping, we propose a deep-learning (DL)-feature-based adaptive multi-source data fusion network (AMSDFNet) for the efficient interpretation of multiple geological remote sensing elements. The AMSDFNet has two branches for learning valuable spatial and spectral information from two kinds of data sources, wherein the atrous spatial pyramid pooling operation and an attention block are applied to adaptively extract and fuse multi-scale informative features. A hard example mining algorithm was also added to select important training examples to address sample imbalance. A large-scale region in western China with sufficient geological elements was set as the research area. The proposed model improved the two critical metrics by more than 2% in the experiment section. As far as we know, this research work is the first time DL features and multi-source remote sensing images have been utilized to simultaneously interpret geological elements of lithology, soil, surface water, and glaciers. The extensive experimental results demonstrated the superiority of DL features and our model in geological remote sensing interpretation.

Video satellite can generate video image sequences with rich dynamic information, thus providing a new way for monitoring moving objects. However, to maintain high temporal resolution, video satellite images usually sacrifice their spatial resolution. Therefore, super-resolution (SR) plays a vital role in improving the quality of video satellite images. In this article, we propose a multiframe video SR neural network (MVSRnet) for video satellite image SR reconstruction. The proposed MVSRnet consists of three main subnetworks: an optical flow estimation subnetwork (OFEnet), an upscaling subnetwork (Upnet) and an attention-based residual learning subnetwork (ARLnet). The OFEnet aims to estimate low-resolution (LR) optical flow of multiple image frames. Upnet is then constructed to enhance the resolution of both input frames and the estimated LR optical flows. Motion compensation is subsequently performed according to the high-resolution (HR) optical flows. Finally, the compensated HR cube is fed to the ARLnet to generate SR results. Different from existing video satellite image SR methods, the proposed MVSRnet is a multiframe-based method with an attention mechanism, which can merge the motion information among adjacent frames and highlight the importance of extracted features. Experiments conducted on Jilin-1 and OVS-1 video satellite images demonstrate that the proposed MVSRnet significantly outperforms some state-of-the-art SR methods.

Standard hyperspectral (HS) pansharpening rely on fusion to enhance low-resolution HS (LRHS) images to the resolution of their matching panchromatic (PAN) images, whose practical implementation is normally under a stipulation of scale invariance of model across the training phase and the pansharpening phase. By contrast, arbitrary resolution HS (ARHS) pansharpening seeks to pansharpen LRHS images to any user-customized resolutions. For such a new HS pansharpening task, it is not feasible to train and store CNN models for all possible candidate scales, which implies the single model acquired from the training phase should be capable of being generalized to yield HS images with any resolutions in the pansharpening phase. To address the challenge, a novel variable sub-pixel convolution (VSPC)-based CNN (VSPC-CNN) method following our arbitrary upsampling CNN (AU-CNN) framework is developed for ARHS pansharpening. The VSPC-CNN method comprises a two-stage elevating thread. The first stage is to improve the spatial resolution of input HS image to that of the PAN image through a pre-pansharpening module and then a VSPC-encapsulated arbitrary scale attention upsampling (ASAU) module is cascaded for arbitrary resolution adjustment. After training with given scales, it can be generalized to pansharpen HS image to arbitrary scales under the spatial patterns invariance across the training and pansharpening phases. Experimental results from several specific VSPC-CNNs on both simulated and real HS datasets show the superiority of the proposed method.

Spatio-temporal fusion is a feasible way to provide synthetic satellite images with high spatial and high temporal resolution simultaneously. Due to its practicability, spatio-temporal fusion has gotten increasing attention, for which many spatio-temporal fusion approaches have been developed. Most spatio-temporal fusion methods follow the “base fine image guided” (BFIG) fusion mode, resulting in the fact that their fusion results are similar to the base fine images. Therefore, these methods can perform well in areas with limited surface changes due to high similarity between the base and the predicted fine images. However, they might not be applicable in areas with intense surface changes. In this article, we develop a pansharpening-based spatio-temporal fusion model (PSTFM) by introducing the pansharpening fusion mode, which is “coarse image guided” (CIG), into spatio-temporal fusion. PSTFM first trains a pansharpening convolutional neural network (CNN), which then fuses the coarse images and reconstructed panchromatic (Pan) images of the predicted time to recover the missing fine images. The newly proposed PSTFM is compared with three representative BFIG spatio-temporal fusion methods on two Landsat–Moderate Resolution Imaging Spectroradiometer (MODIS) datasets, both of which contain intense surface changes. After that, the experimental results are analyzed and discussed in detail. The experiments and the analysis demonstrate that the newly proposed PSTFM has remarkably qualitative and quantitative performance in predicting the intense surface changes while it is mediocre in areas with low surface change intensity.

Spatiotemporal fusion (STF) aims at generating remote-sensing data with both high spatial and temporal resolution. In the literature, one of the most widely used strategies to accomplish this goal is to fuse high temporal resolution images collected by the Moderate Resolution Imaging Spectroradiometer (MODIS) with images with finer spatial resolution than those provided by MODIS (e.g., those collected by other satellite instruments such as Landsat or Sentinel-2). Current STF methods generally fuse an upsampled MODIS image with finer spatial resolution images. This leads to two main problems. First of all, the model uncertainty errors (resulting from the ill-posed upsampling problem) will be propagated into the fusion results, leading to spatial and spectral distortion. Furthermore, the spatial details of the upsampled MODIS image may be significantly different from those of the finer spatial resolution images, making the STF problem even more challenging. In order to tackle these issues, in this work, we develop a new linear regression-based STF strategy (LiSTF), which performs the reconstruction from a MODIS-like image (instead of from an upsampled MODIS image), thus reducing the model uncertainty errors and preserving better the spatial information. The MODIS-like images are built from the finer spatial resolution images via downsampling. Our experimental results, conducted using two publicly available datasets of Landsat–MODIS image pairs and one publicly available dataset of Sentinel–MODIS image pairs, reveal that our newly proposed LiSTF approach can significantly enhance the quantitative and qualitative performance of STF, particularly in terms of preserving the spatial information.

Nowadays, a variety of different sources can be combined together to measure and monitor maritime human activities. Reliable data fusion techniques are essential to associate the targets from different systems for maritime surveillance. In particular, the fusion of data from Sentinel-2 satellites and the Automatic Identification System (AIS) has attracted wide attention due to their public availability and complementarity. However, most traditional methods for target association are not suitable for this particular case, due to the time lag phenomenon of Sentinel-2 data. In this study, we first construct two new datasets for the detection of moving ships and their wakes based on Sentinel-2 images. Combined with the detection results obtained by the You Only Look Once (YOLOv5) model, the position and course information of the detected ships are first extracted. After carefully analyzing the time lag phenomenon of Sentinel-2 data, we develop a new domain adaptation-based method for target association based on the fusion of Sentinel-2 and AIS data, called Moving Ship Optimal Association (MSOA). Different from standard domain adaptation methods only for representation alignment, the proposed MSOA is able to align representation, time and position simultaneously. A case study is provided in which the newly proposed method is tested over the Port of Long Beach, USA. Experimental results demonstrate that both moving ships and wakes are well detected. Specifically, our newly proposed MSOA exhibits more accurate and robust performance when compared to traditional methods, and the detected ships without corresponding AIS tracks can also be detected by our MSOA. Moreover, the real sensing time and time lag of Sentinel-2 data are deduced with high accuracy. Overall, it can be concluded that our MSOA provides a new perspective for accurate target association based on heterogeneous data fusion.

Convolutional neural networks (CNNs) have achieved excellent feature extraction capabilities in remotely sensed hyperspectral image (HSI) classification. This is due to their ability to learn representative spatial and spectral features. However, it is difficult for conventional computers to classify HSIs quickly enough for practical use in many applications, mainly because of the large number of calculations and parameters needed by deep learning-based methods. Although several weight quantization methods achieved remarkable results in network compression, the network acceleration effect is still not significant because a full exploration of the potential of network acceleration brought by network weight quantization is still absent from the literature. In this article, a new step activation quantization method is proposed to constrain the input of the network layer of the CNN so that the data can be represented by low-bit integers. As a result, floating-point operations can be replaced with integer operations to greatly accelerate the forward (inference) step of the network. Specifically, nonlinear uniform quantization is adopted in this work to restrain the input of the CNN in the forward inference of the step activation quantization layer, and two functions (constant and tanh-like) are used in the backpropagation step to avoid gradient vanishing and noise. Our newly proposed step activation quantization acceleration method is applied to a CNN for HSI with two well-known benchmark data sets and the experimental results demonstrate that the proposed method is very effective in terms of both memory savings and computation acceleration, with only a slight decrease in classification accuracy. Specifically, our method reduces memory requirements in$13.6\times $and obtains around$10\times {}$speedup with regard to the original real-valued network version.

Deep learning networks have facilitated the automated scene recognition of landforms based on geomorphogenesis. However, current genetic landform classification methods do not consider regional geological context, which can more accurately reflect the formation and evolution mechanism of geomorphic landforms than local ones. Therefore, this study proposes a multimodal, deep learning landform recognition framework based on a joint contextual geological and channel attention module (GCMENET). First, the multibranch feature extraction network of DenseNet121 is used to extract the respective features from the target scene and the contextual geological scene. Second, the features similar to the landform features of the target scene are extracted from the contextual geological features based on the cosine method and then combined with geomorphic features of the target scene. Third, channel attention mechanism is used to reduce the interference caused by redundant contextual geological information after fusion of data. To measure the classification accuracy of GCMENET, we establish a fine geomorphogenic dataset consisting of remote sensing images of six landform types with a$64\times64$-pixel size and 10-m resolution (JOS10m). During the training process of two geomorphogenic datasets, the feature extraction network without batchnorm2d (batch normalization) could preserve the distribution and spatial alignment of data from the components. Using different training-to-validation data ratios and combinations of input components, the results of the GCMENET supplemented with the joint contextual geological and channel attention module exhibited greater accuracy than those obtained without the module. This observation confirms the importance of contextual geological information in automated geomorphogenic landforms.

Due to the limitation of technology and budget, it is often difficult for sensors of a single remote sensing satellite to have both high temporal resolution and high spatial (HTHS) resolution at the same time. In this paper, we proposed a new Multi-level Feature Fusion with Generative Adversarial Network (MLFF-GAN) for generating fusion HTHS images. MLFF-GAN mainly uses U-net-like architecture and its generator is composed of three stages: feature extraction, feature fusion, and image reconstruction. In feature extraction and reconstruction stage, the generator employs the encoding and decoding structure to extract three groups of multi-level features, which can cope with the huge difference of resolution between high-resolution images and low-resolution images. In the feature fusion stage, Adaptive Instance Normalization (AdaIN) block is designed to learn the global distribution relationship between multi-temporal images, and an attention module (AM) is used to learn the local information weights for the change of small areas. The proposed MLFF-GAN was tested on two Landsat and MODIS datasets. Some state-of-the-art algorithms are comprehensively compared with MLFF-GAN. We also carried on the ablation experiment to test the effectiveness of different sub-module in MLFF-GAN. The experiment results and ablation analysis show the better performances of the proposed method when compared with other methods. The code is available at https://github.com/songbingze/MLFF-GAN.

Recurrent neural networks (RNNs) have been widely used for hyperspectral image (HSI) classification via sequence modeling. However, most of the RNN methods focus on modeling long-range dependencies along the spectral direction, without fully exploring multi-directional dependencies in the joint spectral-spatial domain. To tackle this issue, we propose MSLAN, a two-branch multi-directional spectral-spatial long short-term memory (LSTM) attention network, for HSI classification. In particular, we employ LSTMs to extract six-directional spatial-spectral features which simultaneously capture the spectral-spatial dependencies along different directions. We then design an attention-based feature fuse module to integrate these directional features, followed by a fully connected layer with cross-entropy loss for classification. Additionally, we incorporate an auxiliary branch into our model to enhance the generalization capability. In this branch, random spatial shuffle and a cosine loss are explored for feature consistency learning by taking into account the varying spatial distributions. The resulting two branch networks, sharing the same network structure and weights, are incorporated into a unified deep learning architecture for training. Experiments show the superiority of MSLAN to the state-of-the-art methods for HSI classification with limited training samples.

Anomaly detection is an important technique for hyperspectral image processing. It aims to find pixels that are markedly different from the background when the target spectrum is unavailable. Many anomaly detection methods have been proposed over the past years, among which graph-based ones have attracted extensive attention. And they usually just consider the spectral information to build the adjacency matrix of the graph, which does not think over the effect of spatial information in this process. This paper proposes a new anomaly detection method using the Spectral-Spatial Graph (SSG) that considers both the spatial and spectral information. Thus, the spatial adjacency matrix and spectral adjacency matrix are constructed from the spatial and spectral dimensions, respectively. To obtain a spectral-spatial graph with more discriminant characteristics, and two different local neighborhood detection strategies are used to measure the similarity of the SSG. Furthermore, global anomaly detection results on hyperspectral images were obtained by the graph Laplacian anomaly detection method and the global and local anomaly detection results were optimized by the differential fusion method. Compared with other anomaly detection algorithms on several synthetic and real data sets, the proposed algorithm shows superior detection performance.

In hyperspectral anomaly detection, anomalies are rare targets that exhibit distinct spectral signatures from the background. Thus, anomalies are with low probabilities of occurrence in hyperspectral images. In this article, we develop a new technique for hyperspectral anomaly detection that adopts a new information theory perspective, to fully utilize the aforementioned concepts. Our goal is to transform system entropy into quantitative metrics of anomaly conspicuousness of pixels. To do so, two tasks are first completed: first, the construction of occurrence probability of pixels based on the density peak clustering algorithm, and second, the valid system definitions for pixels in specific anomaly detection problems with multiviews. Specifically, three types of systems are separately established by pixel pairs to conform to the definitions of three entropy definitions in information theory, i.e., Shannon entropy, joint entropy, and relative entropy. Then, three individual entropy-based metrics that assess the anomaly conspicuousness are defined. In addition, we design a standard deviation-based ensemble strategy for the integrated representation of the three individual metrics, which considers both logic “OR” and “AND” operations to simultaneously improve the detection rate and reduce the false alarm rate. Our experimental results obtained on two publicly available datasets with anomalies of different sizes and shapes demonstrate the superiority of our newly proposed anomaly detection method.

The Advanced Hyperspectral Imager (AHSI) is the second hyperspectral imager of China, which is also one of the most important payloads onboard the ZY1E satellite. In order to monitor the radiometric calibration status since its launch on 12 Sep. 2019, and thus provide supplementary on-board radiometric calibration, this paper conducts several vicarious radiometric calibration experiments for the ZY-1E AHSI. Five satellite observations over Dunhuang radiometric calibration site, one of the most relevant China Radiometric Calibration Sites (CRCS), were used. Surface reflectance, radiosonde data, aerosol optical depth (AOD) loading, and water vapor content were used to simulate the top of atmosphere (TOA) radiance at the entrance pupil of the satellite via MODerate resolution atmospheric TRANsmission (MODTRAN). Our results show that the vicarious radiometric calibration coefficients are relatively constant with the official coefficients: the mean relative differences are 5.55%, 5.64%, 7.02%, 5.07%, and 5.30% on 11 Jan, 17 Jan, 12 Feb, 07 May, and 23 Nov, 2021, respectively. The vicarious radiometric calibration coefficients were found in good agreement with the official ones according to the validation analysis based on different surface types, i.e., water and vegetation. Uncertainties of vicarious radiometric calibration due to AOD assumptions, AOD measurements, water vapor measurements, radiosonde data measurements, relative spectral response (RSR) shifts and surface reflectance measurements are also discussed in detail. The obtained results show that the ZY-1E AHSI exhibits good on-orbit radiometric status.

High spatial resolution precipitation data are important for hydrological modeling and meteorological applications, especially at regional scales. Statistical downscaling methods for satellite precipitation products using the normalized difference vegetation index (NDVI) have been carried out in many regions to provide high spatial resolution precipitation. These methods generally use NDVI and precipitation at the same time, assuming that there is a real-time response of vegetation to precipitation. However, this assumption does not hold in many scenarios. It is known that different vegetation types exhibit different response times to precipitation, i.e., there is a possible lag in the response of vegetation to precipitation depending on the vegetation/landcover type. Therefore, it is not appropriate to estimate precipitation using NDVI collected at the same time. To better represent the relationship between precipitation and vegetation, this article develops a new vegetation index based on adaptive lag phase (VIAL) estimated from a new growth rate that is adaptive to landcover type. Based on VIAL, a new local precipitation downscaling method called LPVIAL is proposed, which essentially considers the nonstationary relationship between precipitation and VIAL. The performance of LPVIAL is assessed by downscaling Integrated Multi-satellitE Retrievals for Global Precipitation Measurement (IMERG) from 0.1° to 1-km spatial resolution over the Pearl River Basin in Southern China from 2010 to 2017 at 16-day temporal resolution, and the downscaled products are validated against ground observations. Results indicate that the high-resolution precipitation data obtained from the new downscaling approach perform well, and the accuracy is higher than traditional approaches. With the enhancement of spatial resolution, LPVIAL downscaled products show more detailed spatial information of precipitation with smooth distribution, and the downscaled products have slightly higher accuracy compared with IMERG. It is, therefore, suggested that the adaptive lag phase should be considered in the satellite precipitation product downscaling process.

Remotely sensed hyperspectral images contain several bands (at about adjoining frequencies) for a similar zone on the surface of the Earth. Hyperspectral unmixing is a significant method for breaking down hyperspectral images into the components (endmembers) that conform each (potentially mixed) pixel and their abundance maps. Nonnegative matrix factorization (NMF) has attracted huge consideration because of the way that it can address mixed pixel scenarios. Most existing NMF unmixing techniques do not include spatial information in the analysis. An ongoing trend is to fuse the spatial and the spectral information contained in hyperspectral scenes to improve the solution. In this article, we build up another hyperspectral unmixing technique named spectral–spatial weighted sparse NMF (SSWNMF), in which two weighting factors are acquainted into the NMF model to upgrade the sparsity of the solution and capture the piecewise smooth structure of the data. We adopt a multiplicative iterative strategy to implement the proposed SSWNMF model. Our experimental results, conducted with both synthetic and real hyperspectral data, uncover that the proposed SSWNMF strategy can get accurate unmixing results over those gave by other unmixing strategies, with less parameter tuning.

The problem of effectively exploiting the information multiple data sources has become a relevant but challenging research topic in remote sensing. In this article, we propose a new approach to exploit the complementarity of two data sources: hyperspectral images (HSIs) and light detection and ranging (LiDAR) data. Specifically, we develop a new dual-channel spatial, spectral and multiscale attention convolutional long short-term memory neural network (called dual-channel$A^{3}$CLNN) for feature extraction and classification of multisource remote sensing data. Spatial, spectral, and multiscale attention mechanisms are first designed for HSI and LiDAR data in order to learn spectral- and spatial-enhanced feature representations and to represent multiscale information for different classes. In the designed fusion network, a novel composite attention learning mechanism (combined with a three-level fusion strategy) is used to fully integrate the features in these two data sources. Finally, inspired by the idea of transfer learning, a novel stepwise training strategy is designed to yield a final classification result. Our experimental results, conducted on several multisource remote sensing data sets, demonstrate that the newly proposed dual-channel$A^{\,3}$CLNN exhibits better feature representation ability (leading to more competitive classification performance) than other state-of-the-art methods.

In recent years, Gabor filtering has been successfully applied in spectral-spatial hyperspectral image (HSI) classification tasks due to its strong power to characterize surface materials. A standard Gabor filter involves the real and imaginary parts. A common way to jointly use both the two parts is in the form of Gabor magnitude features. However, this combination form might weaken some unique characteristics of the two parts, therefore leading to limited improvement when compared to the real parts only. To solve this problem, we propose an integrated Gabor-based decision fusion (IG-DF) method for HSI classification. As the term suggests, our approach explores the integration of the two parts of Gabor features by means of a decision-level fusion strategy, where an extensive number of Gabor cubes generated with the real and imaginary parts of Gabor filters as well as different frequencies and orientations are directly fed into each classifier to yield a set of probability outputs. Afterward, the integrated Gabor-based decision fusion strategy is adopted to generate the final classification results from these probability outputs. Our experimental results, conducted on two commonly-used HSI datasets, demonstrate that the proposed IG-DF approach exhibits good improvements when compared with the real-part based and the magnitude feature based methods.

Spatio-temporal fusion has been developed to generate the synthetic remote sensing data with high spatial resolution and high temporal resolution simultaneously. To date, a number of spatio-temporal fusion methods have been developed and most of them are remarkable. However, most the methods are designed to achieve better fusion performance and higher fusion accuracy, while the fusion speed is always ignored. As a matter of fact, most current spatio-temporal fusion methods are time-consuming. To address this defect of spatiotemporal fusion, in this paper we propose a extremely fast spatiotemporal fusion method. The core idea of this method is extracting the spatial information from the prior high-spatial-resolution images and embedding that into the low-spatial-resolution images by local normalization to predict the missing high-spatial-resolution images. In the experiments, two dataset, including a Landsat-MODIS dataset and a Sentienl-MODIS dataset, are adopted to testing this method. The experimental results demonstrate this method can achieve great performance with extremely fast speed.

Social media data generated from individuals provides a unique opportunity to gain valuable insight on information flow, especially for emergency response. However, the inherent limitations associated to these data (particularly, the spatial bias) restrict its precise application. Existing research on spatial bias correction of social media data mainly face two issues: 1) the geographic extent in target domains may be underestimated, and 2) source elements may be transported within inappropriate distance. In this paper, we take 2013 Boulder, Colorado flood event as a case study, and present a new method called subspace optimal transport (SOT). Our proposed SOT aims at transporting biased tweets from dry to real flooded areas with a relatively close distance. Specifically, a comparison between our newly developed SOT and the traditional optimal transport (OT) and geographic optimal transport (GOT) is performed. Experimental results demonstrate that our new SOT method is able to correct the spatially biased geo-referenced tweets, with high precision and excellent computing performance.

José Manuel Bioucas-Dias was an outstanding expert in many different IEEE-related areas, including inverse problems in imaging, signal and image processing, pattern recognition, optimization, and remote sensing. He authored or coauthored more than 250 publications, including more than 100 journal papers (66 of which published in IEEE journals) and over 200 peer-reviewed international conference papers and book chapters. His contributions have been extremely influential in many different fields, namely phase estimation and unwrapping, convex optimization and Bayesian inference for imaging inverse problems, with a special emphasis on remote sensing, including synthetic aperture radar (SAR), hyperspectral unmixing, fusion, super-resolution, classification, and segmentation. In this paper, we provide an overview of his outstanding contributions to remote sensing image processing.

Despite the wide availability of remote sensing big data from numerous different Earth Observation (EO) instruments, the limitations in the spatial and temporal resolution of such EO sensors (as well as atmospheric opacity and other kinds of interferers) have led to many situations in which using only remote sensing data cannot fully meet the requirements of applications in which a (near) real-time response is needed. Examples of these applications include floods, earthquakes, and other kinds of natural disasters, such as typhoons. To address this issue, social media data have gradually been adopted to fill possible gaps in the analysis when remote sensing data are lacking or incomplete. In this case, the fusion of heterogeneous big data streams from multiple data sources introduces significant demands from a computational viewpoint. In order to meet these challenges, distributed computing is increasingly viewed as a feasible solution to parallelize the analysis of massive data coming from different sources (e.g., remote sensing and social media data). In this article, we provide an overview of available and new distributed strategies to address the computational challenges brought by massive heterogeneous data processing and fusion for real-time environmental monitoring and decision-making. The 2013 Boulder (Colorado) flood event is taken as a case study to evaluate several new distributed data fusion frameworks. Experimental results demonstrate that the proposed distributed frameworks are suitable in terms of response time and computational requirements for fusing large-volume heterogeneous data sources.

The large data volume and high algorithm complexity of hyperspectral image (HSI) problems have posed big challenges for efficient classification of massive HSI data repositories. Recently, cloud computing architectures have become more relevant to address the big computational challenges introduced in the HSI field. This article proposes an acceleration method for HSI classification that relies on scheduling metaheuristics to automatically and optimally distribute the workload of HSI applications across multiple computing resources on a cloud platform. By analyzing the procedure of a representative classification method, we first develop its distributed and parallel implementation based on the MapReduce mechanism on Apache Spark. The subtasks of the processing flow that can be processed in a distributed way are identified as divisible tasks. The optimal execution of this application on Spark is further formulated as a divisible scheduling framework that takes into account both task execution precedences and task divisibility when allocating the divisible and indivisible subtasks onto computing nodes. The formulated scheduling framework is an optimization procedure that searches for optimized task assignments and partition counts for divisible tasks. Two metaheuristic algorithms are developed to solve this divisible scheduling problem. The scheduling results provide an optimized solution to the automatic processing of HSI big data on clouds, improving the computational efficiency of HSI classification by exploring the parallelism during the parallel processing flow. Experimental results demonstrate that our scheduling-guided approach achieves remarkable speedups by facilitating the automatic processing of HSI classification on Spark, and is scalable to the increasing HSI data volume.

The recent availability of high-resolution multiview ZY-3 satellite images, with angular information, can provide an opportunity to capture 3-D structural features for classification. In high-resolution image classification over urban areas, objects with diverse vertical structures make urban landscape more heterogeneous in 3-D space and consequently can make the classification challenging. In this article, a novel multiangle gray-level cooccurrence tensor feature is proposed based on the multiview bands of the ZY-3 imagery, namely, GLCMMA–T. The GLCMMA–Tfeature captures the distributions of the gray-level spatial variation under different viewing angles, which can depict the 3-D textures and structures of urban objects. The spectral and GLCMMA–Ttensor features are interpreted by two 3-D convolutional neural network (CNN) streams and then concatenated as the input to the fully connected layer. This novel multispectral and multiangle 3-D convolutional neural network (M2-3-DCNN) combines the spectral and angular information, and the fused feature has the potential to provide a comprehensive description of urban objects with complex vertical structures. The experimental results on ZY-3 multiview images from four test areas indicate that the proposed method can significantly improve the classification accuracy when compared with several state-of-the-art multiangle features and deep-learning-based image classification methods.

The fusion of heterogeneous remote sensing and social media data can fill the gaps in satellite image collections and improve the spatiotemporal resolution of the available data sets. As a result, it is being gradually adopted in multimodal data analytics. Generally, the fusion of heterogeneous geographic data faces the following issues: 1) the probability density functions may differ from different data sources and 2) the geolocations may not be well aligned. The former one can be generally solved by performing an alignment of representations in the source and target domains using, for instance, domain adaptation. The latter issue is seldom considered in the fusion of heterogeneous geographic data. In this article, we present a new method called geographic optimal transport (GOT), which aims at aligning representations and geolocations in a simultaneous fashion. A flood event that took place in 2013 in Boulder, CO, USA, is taken as a case study to evaluate our GOT method. Here, we consider two remote sensing features derived from water indicators, i.e., the normalized difference vegetation index (NDVI) and the normalized difference water index (NDWI), for the fusion of Landsat 8 imagery and Twitter data. A comparison between our newly developed GOT and the traditional optimal transport (OT) is performed. Experimental results demonstrate that the proposed GOT can accurately align spatially biased georeferenced tweets to the flood phenomena, leading to the conclusion that GOT can effectively fuse heterogeneous remote sensing and social media data.

Hyperspectral unmixing is an important problem for remotely sensed data interpretation. It amounts at estimating the spectral signatures of the pure spectral constituents in the scene (endmembers) and their corresponding subpixel fractional abundances. Although the unmixing problem is inherently nonlinear (due to multiple scattering), the nonlinear unmixing of hyperspectral data has been a very challenging problem. This is because nonlinear models require detailed knowledge about the physical interactions between the sunlight scattered by multiple materials. In turn, bilinear mixture models (BMMs) can reach good accuracy with a relatively simple model for scattering. In this article, we develop a new BMM and a corresponding unsupervised unmixing approach which consists of two main steps. In the first step, a deep autoencoder is used to linearly estimate the endmember signatures and their associated abundance fractions. The second step refines the initial (linear) estimates using a bilinear model, in which another deep autoencoder (with a low-rank assumption) is adapted to model second-order scattering interactions. It should be noted that in our developed BMM model, the two deep autoencoders are trained in a mutually interdependent manner under the multitask learning framework, and the relative reconstruction error is used as the stopping criterion. The effectiveness of the proposed method is evaluated using both synthetic and real hyperspectral data sets. Our experimental results indicate that the proposed approach can reasonably estimate the nature of nonlinear interactions in real scenarios. Compared with other state-of-the-art unmixing algorithms, the proposed approach demonstrates very competitive performance.

Recently, many convolutional neural network (CNN) methods have been designed for hyperspectral image (HSI) classification since CNNs are able to produce good representations of data, which greatly benefits from a huge number of parameters. However, solving such a high-dimensional optimization problem often requires a large number of training samples in order to avoid overfitting. In addition, it is a typical nonconvex problem affected by many local minima and flat regions. To address these problems, in this article, we introduce the naive Gabor networks or Gabor-Nets that, for the first time in the literature, design and learn CNN kernels strictly in the form of Gabor filters, aiming to reduce the number of involved parameters and constrain the solution space and, hence, improve the performances of CNNs. Specifically, we develop an innovative phase-induced Gabor kernel, which is trickily designed to perform the Gabor feature learning via a linear combination of local low-frequency and high-frequency components of data controlled by the kernel phase. With the phase-induced Gabor kernel, the proposed Gabor-Nets gains the ability to automatically adapt to the local harmonic characteristics of the HSI data and, thus, yields more representative harmonic features. Also, this kernel can fulfill the traditional complex-valued Gabor filtering in a real-valued manner, hence making Gabor-Nets easily perform in a usual CNN thread. We evaluated our newly developed Gabor-Nets on three well-known HSIs, suggesting that our proposed Gabor-Nets can significantly improve the performance of CNNs, particularly with a small training set.

Social media data have shown great potential for disaster response. However, the inherent limitations associated to these data (particu-larly, the spatial bias) restrict its precise application. In this work, we present a new spatial bias correction method based on remote sensing knowledge and spatio-temporal fusion, named locally optimal transport (LOT). Our method is first tested using a case study (2013 Boulder, Colorado flood event). Then, we apply our method to a 2016 Wuhan flood event to test its accuracy in a data deficient region. Our results show that combining remote sensing features and spatio-temporal fusion can help to address problems with a lack of prior data and limited disaster period data. According to the random ground verification points collected from news, pictures and videos, our new LOT method is able to accurately relocate spatially biased social media data to inundated areas, which are dangerous for users.

In this paper, we describe a method for radiometric calibration of Fengyun-3D (FY-3D) MERSI-II TIR, bands 24 and 25 that combines vicarious and cross-calibration of the corresponding bands of Aqua MODIS, bands 31 and 32. A field campaign was conducted on lake Qinghai, China, on August 18, 2019. The surface measurements were performed before and after the passing time of Terra MODIS and FY-3D MERSI-II. Path radiance and transmittance were then calculated via the radiative transfer code MODTRAN 4.0, and the difference caused by the relative spectral response (RSR) of the sensors was eliminated by the spectral matching. Our experimental results indicate that the obtained radiometric calibration accuracy of FY-3D MERSI-II is stable during on-orbit operation.

High-resolution precipitation data are crucial to monitor disasters in urban areas, especially in cases with abundant precipitation. Based on the spatiotemporal, non-stationary relationship between precipitation and normalized differential vegetation index (NDVI), in this paper we introduce a geographically and temporally weighted regression (GTWR) model and further evaluate it in a case study in Guangdong province, China, in the summer of 2015. Our results indicate that there is a mainly negative correlation between precipitation and NDVI in the summer. Our GTWR downscaling model performs better than a previously available geographically weighted regression (GWR) model, providing more accurate downscaled precipitation estimations. This suggests that considering the spatiotemporal, non-stationarity relationship between NDVI and precipitation, our GTWR downscaling model can provide high-spatial resolution precipitation estimates, with more details in urban areas with abundant precipitation.

Hyperspectral unmixing aims to decompose a hyperspectral image (HSI) into a collection of constituent materials, or end-members, and their corresponding abundance fractions. Recently, nonnegative tensor factorization (NTF)-based spectral unmixing methods have attracted significant attention owing to their outstanding performance when representing an HSI without any information loss. However, tensor factorization-based HSI methods do not fully exploit the spatial contextual information present in the scene. Besides, these approaches are sensitive to low signal-to-noise ratio (SNR) in HSIs. To address this limitation, we propose a new spectral-spatial weighted sparse nonnegative tensor factorization (SSWNTF) method to preserve the spatial details in the abundance maps via the spectral and spatial weighting factors. Our experiments with simulated data sets certified that the proposed method outperforms other advanced methods.

In active learning (AL)-based remote sensing (RS) image classification tasks, the acquisition of labeled data depends not only on the informativeness and representativeness measured in feature space but also on the spatial distributions and relations in an image plane. However, very few studies have investigated the advantages of integrating spatial constraints into the AL paradigm. Hence, under the basic assumption “instances that are difficult to classify are usually located around edges between different objects or land-cover types,” edge gradient information was integrated into the conventional AL paradigm using popular uncertainty and diversity measurements. The experimental results with two real hyperspectral images confirmed the advantages of the proposed edge gradient-based AL (EGAL) approach from the aspects of fast convergence and computationally efficient operation.

Spectral-spatial Gabor filtering, which is based on 3-D local harmonic analysis, has been a powerful spectral-spatial feature extraction tool for hyperspectral image (HSI) classification. However, existing spectral-spatial Gabor approaches are prone to oversmoothing, neglecting the existences of edges and negatively affecting the classification. In this article, we propose a new HSI Gabor filtering concept, called spectral-spatial-range Gabor filtering, which intends to restrain edge interference from disturbing local spectral-spatial harmonic components. Contributions and novelties of our work can be identified as follows: 1) an HSI filtering framework is created, which can accommodate various Gabor filtering procedures and hence offer the potential to guide the design of new Gabor filters; 2) following such a unified filtering framework and taking into consideration both local spectral-spatial harmonic characteristics and range domain variations, we develop a new concept of spectral-spatial-range Gabor filtering; and 3) utilizing this proposed Gabor prototype and elaborating mathematical derivations, we achieve a novel discriminative spectral-spatial-range Gabor filtering method, which can deal with discriminative local harmonics and edge interference simultaneously along the spectral-spatial-range domain, obtaining highly discriminative Gabor features while yielding linear computational complexity. Our novel method is evaluated on four real HSI data sets and achieves excellent performances.

Hyperspectral unmixing (HU) is an active research topic in the remote-sensing community. It aims at modeling mixed pixels using a collection of pure constituent materials (endmembers) weighted by their corresponding fractional abundances. Among existing unmixing schemes, nonnegative matrix factorization (NMF) has drawn significant attention due to its unsupervised nature, as well as its capacity to obtain both endmembers and fractional abundances simultaneously. In this article, we present a new blind unmixing method based on the generalized morphological component analysis (GMCA) framework, in which an additional constraint is introduced into the standard NMF model to represent the sparsity and morphological diversity of the abundance maps associated with each endmember. More specifically, we take into account the fact that different ground categories in a hyperspectral scene generally exhibit various spatial distributions and morphological characteristics. As a result, when providing a specific dictionary basis for these categories, their corresponding abundance maps (referred to as sources) can be sparsely represented. In addition, due to the low correlation between different sources, their sparse representations will not share the same most significant coefficients. With this observation in mind, we can further promote source discrimination and separation in the unmixing process. Moreover, in order to obtain a stable solution of the involved optimization problem, we adopt an alternate iterative constrained algorithm with a threshold descent strategy. Our experiments, carried out on both synthetic and real hyperspectral scenes, reveal that our newly developed GMCA-based unmixing method obtains very promising results with fast convergence speed and requiring significantly less parameter tuning. This confirms the advantage of the proposed spatial morphological component approach for HU purposes.

In some remote sensing applications such as change detection, satellite images with both high spatial and high temporal resolution are required. However, no single satellite sensor can currently provide such images due to technical specifications. To solve this problem, spatio-temporal fusion provides a cost-effective solution. In this paper, we propose a new spatio-temporal fusion approach, based on convolutional neural networks (CNNs), for Landsat and MODIS image fusion. Specifically, the proposed approach utilizes CNNs to model the heterogeneity of fine pixels from the coarse MODIS images. Here, the heterogeneity of fine pixels is defined as the difference between the reflectance changes obtained from the two types of images. After that, two transition-predicted images can be obtained using the trained CNNs, which are then fused in order to obtain a fi-nal prediction. In our newly proposed approach, CNNs are only used to learn the heterogeneity of fine pixels rather than the whole images, thus providing a more stable and less time-consuming strategy as compared to other available approaches. We evaluated the proposed approach on a public spatio-temporal fusion dataset and the obtained results suggest that our newly developed method achieves state-of-the-art performance.

This work proposes a new collaborative active and semi-supervised learning approach, named accessibility-free active learning (AFAL), for hyperspectral imaging classification. The proposed approach aims to tackle an existing problem in traditional active learning methods, that is, the fact that some selected samples are not accessible by oracles for assigning them pseudo labels, i.e., confident predictions for the classifier. The proposal specifically addresses this problem using superpixels in a self-training context. Specifically, AFAL first generates a set of candidates locally around the labeled pixels and then expands them to other subregions via a density peak-based augmentation strategy, in order to guarantee the confidence of pseudo labels. Our experimental results, obtained on two real and well-used hyperspectral images, reveal that the proposed scheme can lead to state-of-the-art performance.

Nonlinear unmixing of hyperspectral images has been a very challenging research problem, as it needs to consider the physical interactions between the sunlight scattered by multiple materials. In this paper, we propose a new approach for nonlinear unmixing which is based on multi-task learning (MTL) with low-rank matrix factorization (LRMF). The proposed approach establishes two tasks to conduct the unmixing problem under a nonlinear mixing model. In the first task, we employ LRMF to obtain endmember signatures and their corresponding abundance fractions simultaneously. Then, the second task uses LRMF to solve interactions from multiple scattering. The effectiveness of the proposed method is verified by using real hyperspectral data. Compared with other state-of-the-art nonlinear unmixing algorithms, the proposed approach demonstrates very competitive performance.

Sparse representation-based approaches have been successfully applied to remotely sensed hyperspectral image unmixing. In recent years, sparse unmixing techniques have incorporated spatial information into the sparse unmixing model, achieving improved fractional abundance results. Most spatial-based sparse unmixing methods utilize regular-shaped neighborhoods (e.g., a cross or a square window) to characterize the spatial-contextual information around each pixel. However, the spatial characteristics of natural scenes are not always uniform, but vary according to the observed objects. Therefore, assuming uniform spatial neighborhoods may not be consistent with real spatial structures in the scene. Super-pixels offer a good solution to this problem since they can better characterize such spatial structures. Based on this observation, in this paper we develop a new superpixel-guided sparse unmixing (SPGSU) method for hyperspectral scenes. The proposed SPGSU includes the spatial correlation through a superpixel-based technique rather than assuming predefined pixel grids. Each superpixel can be regarded as a small spatial region, whose shape and size can be adaptively changed to accommodate different spatial structures. Our experimental results, conducted using simulated data sets, quantitatively indicate that our newly proposed method produces better results than other advanced spectral unmixing methods.

This letter introduces a novel remote sensing singleimage superresolution (SR) architecture based on a deep efficient compendium model. The current deep learning-based SR trend stands for using deeper networks to improve the performance. However, this practice often results in the degradation of visual results. To address this issue, the proposed approach harmonizes several different improvements on the network design to achieve state-of-the-art performance when superresolving remote sensing imagery. On the one hand, the proposal combines residual units and skip connections to extract more informative features on both local and global image areas. On the other hand, it makes use of parallelized 1×1 convolutional filters (network in network) to reconstruct the superresolved result while reducing the information loss through the network. Our experiments, conducted using seven different SR methods over the well-known UC Merced remote sensing data set, and two additional GaoFen-2 test images, show that the proposed model is able to provide competitive advantages.

Convolutional neural networks (CNNs) have become a powerful tool for remotely sensed hyperspectral image (HSI) classification due to their great generalization ability and high accuracy.However, owing to the huge amount of parameters that need to be learned and to the complex nature of HSI data itself, these approaches must deal with the important problem of overfitting, which can lead to inadequate generalization and loss of accuracy.In order to mitigate this problem, in this letter we adopt random occlusion, a recently developed data augmentation (DA) method for training CNNs in which the pixels of different rectangular spatial regions in the HSI are randomly occluded, generating training images with various levels of occlusion and reducing the risk of overfitting.Our results with two well-known HSIs reveal that the proposed method helps to achieve better classification accuracy with low computational cost.

Since the 1960s, remote sensing (as an innovative, comprehensive, and interdisciplinary academic area) has been adopted in a wide range of disciplines related to Earth observation, including hydrology, ecology, oceanography, glaciology, geology, military, intelligence, business, economy, and planning [1]-[3]. The constant development of the remote sensing image acquisition technology now allows for the collection of a wide variety of images with different characteristics and resolutions, obtained by remote sensing instruments mounted on spacecraft or aircraft platforms. These images record some type of signal or energy measured from the Earth's surface, which depends on the type of sensor used.

Remote sensing has evolved into a multidisciplinary field involving many different areas such as sensor technology, computing, and advanced applications. Information extraction now plays a fundamental role in the exploitation of the massive amount of data collected by earth observation instruments. In this Point of View, the authors analyze the evolution of this field, identifying three main phases in its development. The first period, which was marked by advances in digital signal processing, led to a significant development of statistical processing methods. The second phase was based on advances in physical models and brought an era of quantitative remote sensing which lasted until the first decade of this century. In the third and current period, information extraction techniques are gradually adopting advanced artificial intelligence models in an effort to cope with the tremendous increase in data volume. This article describes some of these recent advances and addresses challenges caused by the 4Vs (volume, velocity, variety, and veracity) of big data. Finally, the authors offer insight into future directions in this multidisciplinary field.

Nowadays, many methods of spatial-spectral classification have been developed and achieved good results for classification with high-resolution remotely sensed images, especially superpixel-based methods. However, these methods generally consider a superpixel as a group of pixels instead of one entity, ignoring the spectral-spatial entirety in the third-order RSI data cube. In order to fully exploit the third-order spectral-spatial information, in this paper, we propose a superpixel-based tensor model for RSI classification, where a multiattribute superpixel tensor (MAST) model is constructed on the top of multiattribute superpixel maps based on the concept of extended morphological profiles (EMAPs). In order to manage the adaptive spatial nature of superpixels, we develop an increment strategy to augment all superpixels with filling up their own envelop rectangles including three different ways, i.e., 0 vector, mean vector of all the pixels within the superpixel, or original pixels. Then, we use CANDECOMP/PARAFAC (CP) decomposition to obtain the features of the unified dimension from the MASTs of various sizes. Especially, CP decomposition can deal with missing data, so we also got a fourth means of constructing the MAST. Finally, base kernels calculated, respectively, from the original spectral feature, EMAP features and MAST features are learned by multiple kernel learning methods, with the optimal kernel fed to a support vector machine to complete the classification task. The experiments conducted on four real RSIs and compared with several well-known methods demonstrate the effectiveness of the proposed model.

Deep neural networks (DNNs), including convolutional neural networks (CNNs) and residual networks (ResNets) models, are able to learn abstract representations from the input data by considering a deep hierarchy of layers that perform advanced feature extraction. The combination of these models with visual attention techniques can assist with the identification of the most representative parts of the data from a visual standpoint, obtained through more detailed filtering of the features extracted by the operational layers of the network. This is of significant interest for analyzing remotely sensed hyperspectral images (HSIs), characterized by their very high spectral dimensionality. However, few efforts have been conducted in the literature in order to adapt visual attention methods to remotely sensed HSI data analysis. In this paper, we introduce a new visual attention-driven technique for the HSI classification. Specifically, we incorporate attention mechanisms to a ResNet in order to better characterize the spectral-spatial information contained in the data. Our newly proposed method calculates a mask that is applied to the features obtained by the network in order to identify the most desirable ones for classification purposes. Our experiments, conducted using four widely used HSI data sets, reveal that the proposed deep attention model provides competitive advantages in terms of classification accuracy when compared to other state-of-the-art methods.

Convolutional neural networks (CNNs) have recently exhibited an excellent performance in hyperspectral image classification tasks. However, the straightforward CNN-based network architecture still finds obstacles when effectively exploiting the relationships between hyperspectral imaging (HSI) features in the spectral-spatial domain, which is a key factor to deal with the high level of complexity present in remotely sensed HSI data. Despite the fact that deeper architectures try to mitigate these limitations, they also find challenges with the convergence of the network parameters, which eventually limit the classification performance under highly demanding scenarios. In this paper, we propose a new CNN architecture based on spectral-spatial capsule networks in order to achieve a highly accurate classification of HSIs while significantly reducing the network design complexity. Specifically, based on Hinton's capsule networks, we develop a CNN model extension that redefines the concept of capsule units to become spectral-spatial units specialized in classifying remotely sensed HSI data. The proposed model is composed by several building blocks, called spectral-spatial capsules, which are able to learn HSI spectral-spatial features considering their corresponding spatial positions in the scene, their associated spectral signatures, and also their possible transformations. Our experiments, conducted using five well-known HSI data sets and several state-of-the-art classification methods, reveal that our HSI classification approach based on spectral-spatial capsules is able to provide competitive advantages in terms of both classification accuracy and computational time.

Spectral unmixing is a technique for remotely sensed image interpretation that expresses each (possibly mixed) pixel as a combination of pure spectral signatures (endmembers) and their fractional abundances. In this paper, we develop a new technique for unsupervised unmixing which is based on a deep autoencoder network (DAEN). Our newly developed DAEN consists of two parts. The first part of the network adopts stacked autoencoders (SAEs) to learn spectral signatures, so as to generate a good initialization for the unmixing process. In the second part of the network, a variational autoencoder (VAE) is employed to perform blind source separation, aimed at obtaining the endmember signatures and abundance fractions simultaneously. By taking advantage from the SAEs, the robustness of the proposed approach is remarkable as it can unmix data sets with outliers and low signal-to-noise ratio. Moreover, the multihidden layers of the VAE ensure the required constraints (nonnegativity and sum-to-one) when estimating the abundances. The effectiveness of the proposed method is evaluated using both synthetic and real hyperspectral data. When compared with other unmixing methods, the proposed approach demonstrates very competitive performance.

Land-cover classification with hyperspectral imagery has been an active topic in the remote sensing community. It aims at relating a unique class label to each pixel in the scene, so that it can be well defined by a given land cover type. In this paper, we explore the intrinsic characteristics of hyperspectral imagery from a subpixel-level perspective and propose a new subpixel component analysis (SCA) approach for feature extraction and land-cover classification. The core idea of SCA is that we extract a subpixel attribute component feature from the abundance maps. Compared with the abundance maps, the extracted subpixel feature image shows higher signal-to-noise level and clearer spatial distribution details. In order to deal with spectral variability, as well as obtain representative image endmember signatures and their corresponding abundance maps, we adopt a regional clustering-based spatial preprocessing (RCSPP) strategy for endmember identification, and a partial unmixing model based on mixture tuned matched filtering (MTMF) for abundance estimation. Furthermore, to highlight the spatial distribution details as well as eliminate the noise disturbance in the derived abundance maps, we perform sparse image decomposition on the obtained abundance maps, thus achieving a new subpixel feature representation for classification. Our experimental results reveal that the proposed SCA approach can obtain feature representation with explicit physical meaning, clear spatial distribution details, and better noise robustness, leading to state-of-the-art classification results.

Feature extraction (FE) is a crucial research area in hyperspectral image (HSI) processing. Recently, due to the powerful ability of deep learning (DL) to extract spatial and spectral features, DL-based FE methods have shown great potentials for HSI processing. However, most of the DL-based FE methods are supervised, and the training of them suffers from the absence of labeled samples in HSIs severely. The training issue of supervised DL-based FE methods limits their application on HSI processing. To address this issue, in this paper, a novel modified generative adversarial network (GAN) is proposed to train a DL-based feature extractor without supervision. The designed GAN consists of two components, which are a generator and a discriminator. The generator can focus on the learning of real probability distributions of data sets and the discriminator can extract spatial-spectral features with superior invariance effectively. In order to learn upsampling and downsampling strategies adaptively during FE, the proposed generator and discriminator are designed based on a fully deconvolutional subnetwork and a fully convolutional subnetwork, respectively. Moreover, a novel min-max cost function is designed for training the proposed GAN in an end-to-end fashion without supervision, by utilizing the zero-sum game relationship between the generator and discriminator. Besides, the proposed modified GAN replaces the original Jensen-Shannon divergence with the Wasserstein distance, aiming to mitigate the unstability and difficulty of the training of GAN frameworks. Experimental results on three real data sets validate the effectiveness of the proposed method.

This paper presents a subpixel spatial-spectral feature mining approach for hyperspectral image classification. First, a regional clustering-based spatial preprocessing (RCSPP) strategy is introduced to identify the endmember signatures from the original image. Then, a partial unmixing model of mixture tuned matched filtering (MTMF) is adopted to estimate the abundance maps. Finally, the morphological component analysis (MCA) is adopted to decompose the abundance map into different spatial morphological components, and the smoothness components are chosen for classification. The experimental results reveal that the obtained subpixel spatial-spectral feature can lead to very good classification accuracies.

Hyperspectral image (HSI) denoising has become an important research topic in the research community due to its significance improvements in many applications (e.g. classification). In this paper, we introduce a Monte Carlo non-local means (MCNLM) method for noise reduction of the HSI. Each band of the HSI is processed by the MCNLM, which is a randomized algorithm suitable for large-scale patch-based image (e.g. HSI) filtering. More specifically, the MCNLM is achieved by randomly choosing a fraction of the similarity weights to obtain an approximated result. Compared to the classical non-local means (NLM), the MCNLM consumes less time while achieves comparable performance. Experimental results on the real hyperspectral data set demonstrate the promising performance of the MCNLM for HSI denoising.

In this paper, we propose a wide contextual residual network (WCRN) with active learning (AL) for remote sensing image (RSI) classification. Although ResNets have achieved great success in various applications (e.g. RSI classification), its performance is limited by the requirement of abundant labeled samples. As it is very difficult and expensive to obtain class labels in real world, we integrate the proposed WCRN with AL to improve its generalization by using the most informative training samples. Specifically, we first design a wide contextual residual network for RSI classification. We then integrate it with AL to achieve good machine generalization with limited number of training sampling. Experimental results on the University of Pavia and Flevoland datasets demonstrate that the proposed WCRN with AL can significantly reduce the needs of samples.

In this paper, we propose a deep auto-encoder network for the unmixing for hyperspectral data with outliers and low signal to noise ratio. The proposed deep auto-encoder network composes of two parts. The first part of the network adopts stacked non-negative sparse auto-encoder to learn the spectral signatures such that to generate a good initialization for the network. In the second part of the network, a variational auto-encoder is employed to perform unmixing, aiming at the endmember signatures and abundance fractions. The effectiveness of the proposed method is verified by using a synthetic data set. In our comparison with other state-of-the-art unmixing methods, the proposed approach demonstrates highly competitive performance.

This paper presents an inspiring approach for accurate estimation of impervious surfaces, which exploits the strength of two kind of heterogeneous features, i.e., physical features derived from satellite images and social features derived from social media datasets, respectively. On the one hand, we use a morphological attribute profiles guided spectral mixture analysis model to achieve estimates of physical features. On the other hand, we mine the social features from textual information of social media datasets. Then, a multivariable linear regression model is conducted to obtain the impervious surfaces. Experiment results, conducted with multi-spectral images collected by LANDSAT-8 and social media datasets scraped from Sina Weibo of Guangzhou city, suggest that our approach could lead to reliable and good estimation of the imperviousness.

As an unsupervised learning tool, autoencoder has been widely applied in many fields. In this letter, we propose a new robust unmixing algorithm that is based on stacked nonnegative sparse autoencoders (NNSAEs) for hyperspectral data with outliers and low signal-to-noise ratio. The proposed stacked autoencoders network contains two main steps. In the first step, a series of NNSAE is used to detect the outliers in the data. In the second step, a final autoencoder is performed for unmixing to achieve the endmember signatures and abundance fractions. By taking advantage from nonnegative sparse autoencoding, the proposed approach can well tackle problems with outliers and low noise-signal ratio. The effectiveness of the proposed method is evaluated on both synthetic and real hyperspectral data. In comparison with other unmixing methods, the proposed approach demonstrates competitive performance.

Lunar impact craters are the most typical geomorphic feature on the moon and are of great importance in studies of lunar terrain features. This paper presents a crater detection algorithm (CDA) that is based on terrain analysis and mathematical morphology methods. The proposed CDA is applied to digital elevation models (DEMs) to identify the boundaries of impact craters. The topographic and morphological characteristics of impact craters are discussed, and detailed steps are presented to detect different types of craters, such as dispersal craters, connective craters, and con-craters. The DEM from the Lunar Reconnaissance Orbiter, which has a resolution of 100 m, is used to verify the proposed CDA. The results show that the boundaries of impact craters can be detected. The results enable increased understanding of surface processes through the characterization of crater morphometry and the use of crater size-frequency distributions to estimate the ages of planetary surfaces.

Hyperspectral unmixing is an important processing step for many hyperspectral applications, mainly including: 1) estimation of pure spectral signatures (endmembers) and 2) estimation of the abundance of each endmember in each pixel of the image. In recent years, nonnegative matrix factorization (NMF) has been highly attractive for this purpose due to the nonnegativity constraint that is often imposed in the abundance estimation step. However, most of the existing NMF-based methods only consider the information in a single layer while neglecting the hierarchical features with hidden information. To alleviate such limitation, in this paper, we propose a new sparsity-constrained deep NMF with total variation (SDNMF-TV) technique for hyperspectral unmixing. First, by adopting the concept of deep learning, the NMF algorithm is extended to deep NMF model. The proposed model consists ofpretraining stageandfine-tuning stage, where the former pretrains all factors layer by layer and the latter is used to reduce the total reconstruction error. Second, in order to exploit adequately the spectral and spatial information included in the original hyperspectral image, we enforce two constraints on the abundance matrix. Specifically, the$L_{1/2}$constraint is adopted, since the distribution of each endmember is sparse in the 2-D space. The TV regularizer is further introduced to promote piecewise smoothness in abundance maps. For the optimization of the proposed model, multiplicative update rules are derived using the gradient descent method. The effectiveness and superiority of the SDNMF-TV algorithm are demonstrated by comparing with other unmixing methods on both synthetic and real data sets.

Hyperspectral imaging is a widely used technique in remote sensing in which an imaging spectrometer collects hundreds of images (at different wavelength channels) for the same area on the surface of the earth. In the last two decades, several methods (unsupervised, supervised, and semisupervised) have been proposed to deal with the hyperspectral image classification problem. Supervised techniques have been generally more popular, despite the fact that it is difficult to collect labeled samples in real scenarios. In particular, deep neural networks, such as convolutional neural networks (CNNs), have recently shown a great potential to yield high performance in the hyperspectral image classification. However, these techniques require sufficient labeled samples in order to perform properly and generalize well. Obtaining labeled data is expensive and time consuming, and the high dimensionality of hyperspectral data makes it difficult to design classifiers based on limited samples (for instance, CNNs overfit quickly with small training sets). Active learning (AL) can deal with this problem by training the model with a small set of labeled samples that is reinforced by the acquisition of new unlabeled samples. In this paper, we develop a new AL-guided classification model that exploits both the spectral information and the spatial-contextual information in the hyperspectral data. The proposed model makes use of recently developed Bayesian CNNs. Our newly developed technique provides robust classification results when compared with other state-of-the-art techniques for hyperspectral image classification.

Imaging spectroscopy, also known as hyperspectral imaging, has been transformed in the last four decades from being a sparse research tool into a commodity product available to a broad user community. Specially, in the last 10 years, a large number of new techniques able to take into account the special properties of hyperspectral data have been introduced for hyperspectral data processing, where hyperspectral image classification, as one of the most active topics, has drawn massive attentions. Spectral-spatial hyperspectral image classification can achieve better classification performance than its pixel-wise counterpart, since the former utilizes not only the information of spectral signature but also that from spatial domain. In this paper, we provide a comprehensive overview on the methods belonging to the category of spectral-spatial classification in a relatively unified context. First, we develop a concept of spatial dependency system that involves pixel dependency and label dependency, with two main factors: neighborhood covering and neighborhood importance. In terms of the way that the neighborhood information is used, the spatial dependency systems can be classified into fixed, adaptive, and global systems, which can accommodate various kinds of existing spectral-spatial methods. Based on such, the categorizations of single-dependency, bilayer-dependency, and multiple-dependency systems are further introduced. Second, we categorize the performings of existing spectral-spatial methods into four paradigms according to the different fusion stages wherein spatial information takes effect, i.e., preprocessing-based, integrated, postprocessing-based, and hybrid classifications. Then, typical methodologies are outlined. Finally, several representative spectral-spatial classification methods are applied on real-world hyperspectral data in our experiments.

Multitask learning (MTL) has recently yielded impressive results for classification of remotely sensed data due to its ability to incorporate shared information across multiple tasks. However, it remains a challenging issue to achieve robust classification results in the case that the data are from nonlinear subspaces. In this paper, we propose a kernel low-rank MTL (KL-MTL) method to handle multiple features from the 2-D variational mode decomposition (2-D-VMD) domain for multi-/hyperspectral classification. On the one hand, a nonrecursive 2-D-VMD method is applied to extract various features [i.e., intrinsic mode functions (IMFs)] of the original data concurrently. Compared with the existing 2-D empirical mode decomposition, 2-D-VMD has much stronger mathematical foundation and does not need any recursive sifting process. On the other hand, KL-MTL is proposed for classification by taking the extracted IMFs as features of multiple tasks. In KL-MTL, the low-rank representation formulated by nuclear norm can capture global structure of multiple tasks, while the kernel tricks are utilized for nonlinear extension of the low-rank MTL. Moreover, the optimization problem in KL-MTL is solved by the inexact augmented Lagrangian method. Compared with several state-of-the-art feature extraction and classification methods, the experimental results using both multi-/hyperspectral images demonstrate that the proposed method has satisfactory classification performance.

With the sparse unmixing becoming increasingly popular recently, some advanced regularization algorithms have been proposed for settling this problem. However, they are limited by their “decision ahead of solution” attribute, i.e., the regularization parameters must be preset before the solution is obtained. In this paper, the sparse unmixing problem is first formulated as a two-phase multiobjective problem. The first phase simultaneously minimizes the unmixing residuals and the number of estimated endmembers for automatically finding the real active endmembers from the spectral library. A decomposition-based endmember selection algorithm considering the gene exchange in the population is specially designed for better and quicker search of the decision space. This algorithm can obtain a set of nondominated solutions for better decision of the active endmembers, which are important for the subsequent calculation of the abundance matrix. The second phase concurrently minimizes the unmixing residuals and the total variation term for estimating a preferable abundance matrix. A local search strategy based on the multiplicative update rule is designed in the evolution process for better approximation of the Pareto front. The experimental results on the synthetic as well as the real data reveal that the proposed framework has a better performance in finding the real active endmembers and estimating their corresponding abundances than some advanced regularization algorithms.

Superpixels are a powerful device to characterize the spatial-contextual information in remotely sensed hyperspectral image (HSI) interpretation. However, the exploitation of superpixels in classification problems is not straightforward, often leading to unbearable NP-hard discrete integer optimization problems. In this paper, we attack this hurdle by leveraging on a convex relaxation of the original integer optimization problem, which opens the door to include oversegmented superpixel-based regularizers. Specifically, we develop a new method for generating oversegmented superpixels. Then, we introduce a family of convex regularizers in the form of graph total variation, which promotes the same labeling in each superpixel. Vectorial total variation is also included in order to promote piecewise smoothness and align discontinuities along the class boundaries. The solution of the obtained convex optimization problem is computed with the split-augmented Lagrangian shrinkage algorithm. Experiments on HSIs yield classification maps with precise boundaries and inner consistency inside oversegmented superpixels, leading to the state-of-the-art classification accuracies.

Active learning (AL) has obtained a great success in supervised remotely sensed hyperspectral image classification, since it is able to select highly informative training samples. As an intrinsically biased sampling approach, AL generally favors the selection of samples following discriminative distributions, which are located in low-density areas. However, hyperspectral data are often highly class-mixed, i.e., most samples fluctuate in the overlapping regions of distributions of different classes. In this case, the potential of AL to select effective training samples is more limited. As AL strongly depends on the features, a possibility to increase its capabilities is to transfer the data into a highly discriminative feature space, in which the mixture of distributions that different classes of data follow tends to reduce. Based on this observation, in this paper, we introduce the concept of feature-driven AL, namely, the sample selection is going to be conducted in a given optimized feature space whose superiority is measured by an overall error probability. For illustrative purposes, we used Gabor filtering and morphological profiles for instantiation. Our experimental results, obtained on three real hyperspectral data sets, indicate that the proposed approach can significantly improve the potential of AL for hyperspectral image classification.

In remote sensing image classification, active learning aims to learn a good classifier as best as possible by choosing the most valuable (informative and representative) training samples. Multiview is a concept that regards analyzing the same object from multiple different views. Generally, these views show diversity and complementarity of features. In this paper, we propose a new multiview active learning (MVAL) framework for hyperspectral image classification. First, we generate multiple views by extracting different attribute components from the same image data. Specifically, we adopt the multiple morphological component analysis to decompose the original image into multiple pairs of attribute components, including content, coarseness, contrast, and directionality, and the smooth component from each pair is chosen as one single view. Second, we construct two multiview intensity-based query strategies for active learning. On the one hand, we exploit the intensity differences of multiple views along with the samples' uncertainty to choose the most informative candidates. On the other hand, we consider the clustering distribution of all unlabeled samples, and query the most representative candidates in addition to the highly informative ones. Our experiments are performed on four benchmark hyperspectral image data sets. The obtained results show that the proposed MVAL framework can lead to better classification performance than the traditional, single-view active learning schemes. In addition, compared with the conventional disagree-based MVAL scheme, the proposed query selection strategies show competitive classification accuracy.

In this paper, a new joint spectral-spatial subpixel mapping model is proposed for hyperspectral remotely sensed imagery. Conventional approaches generally use an intermediate step based on the derivation of fractional abundance maps obtained after a spectral unmixing process, and thus the rich spectral information contained in the original hyperspectral data set may not be utilized fully. In this paper, a concept of subpixel abundance map, which calculates the abundance fraction of each subpixel to belong to a given class, was introduced. This allows us to directly connect the original (coarser) hyperspectral image with the final subpixel result. Furthermore, the proposed approach incorporates the spectral information contained in the original hyperspectral imagery and the concept of spatial dependence to generate a final subpixel mapping result. The proposed approach has been experimentally evaluated using both synthetic and real hyperspectral images, and the obtained results demonstrate that the method achieves better results when compared to other seven subpixel mapping methods. The numerical comparisons are based on different indexes such as the overall accuracy and the CPU time. Moreover, the obtained results are statistically significant at 95% confidence.

Spectral unmixing aims at estimating the fractional abundances of a set of pure spectral materials (endmembers) in each pixel of a hyperspectral image. The wide availability of large spectral libraries has fostered the role of sparse regression techniques in the task of characterizing mixed pixels in remotely sensed hyperspectral images. A general solution for sparse unmixing methods consists of using the l2regularizer to control the sparsity, resulting in a very promising performance but also suffering from sensitivity to large and small sparse coefficients. A recent trend to address this issue is to introduce weighting factors to penalize the nonzero coefficients in the unmixing solution. While most methods for this purpose focus on analyzing the hyperspectral data by considering the pixels as independent entities, it is known that there exists a strong spatial correlation among features in hyperspectral images. This information can be naturally exploited in order to improve the representation of pixels in the scene. In order to take advantage of the spatial information for hyperspectral unmixing, in this paper, we develop a new spectral-spatial weighted sparse unmixing (S2WSU) framework, which uses both spectral and spatial weighting factors, further imposing sparsity on the solution. Our experimental results, conducted using both simulated and real hyperspectral data sets, illustrate the good potential of the proposed S2WSU, which can greatly improve the abundance estimation results when compared with other advanced spectral unmixing methods.

Spectral unmixing is an important technique for remotely sensed hyperspectral image interpretation, of which the goal is to decompose the image into a set of pure spectral components (endmembers) and their abundance fractions in each pixel of the scene. Sparse-representation-based approaches have been widely studied for remotely sensed hyperspectral unmixing. A recent trend is to incorporate the spatial information to improve the spectral unmixing results. Those methods generally assume that the abundances of the pixels are piecewise smooth and fall into a homogeneous region occupied by the same endmembers and their corresponding fractional abundances. However, in real scenarios, abundances may vary abruptly from pixel to pixel. Therefore, the former assumption in most spatial models does not hold. To address this limitation, we propose a new strategy to preserve the spatial details in the abundance maps via a spatial discontinuity weight. Our experimental results, conducted with both simulated and real hyperspectral data sets, illustrate the good potential of our discontinuity-preserving strategy for sparse unmixing, which can greatly improve the abundance estimation results.

In this paper, we propose a kernel low-rank multitask learning (KL-MTL) method to handle multiple features from the variational mode decomposition (VMD) domain for hyperspectral (HSI) classification. Core ideas of the proposed method are twofold: 1) a non-recursive VMD method is applied to extract various features (i.e. intrinsic mode functions (IMFs)) of the original data concurrently; 2) KL-MTL is proposed for classification by taking the extracted IMFs as multiple tasks. In KL-MTL, the low-rank representation formulated by nuclear norm can capture global structure of multiple tasks while the kernel tricks are utilized for nonlinear extension of the low-rank multitask learning (MTL). Experimental results using the real hyperspectral data demonstrate that the proposed methods have satisfactory classification performance.

Discriminative dictionary learning aims to learn a dictionary from training samples in order to improve the discriminative ability of their coding vectors. Gabor wavelets have recently been successfully applied for hyperspectral image (HSI) classification due to their ability to extract joint spatial and spectrum information. Due to the high discriminative power of Gabor features, an efficient method, called Gabor feature based Support Vector Guided Dictionary Learning (GSVGDL), has been proposed in this paper for HSI classification. After Gabor features have been extracted from the hyperspectral image, the augmented Gabor feature matrix is used to construct the initial dictionary. The dictionary learning model formulates the discrimination term as the weighted summation of the squared distances between all pairs of coding vectors, which can greatly improve the discriminative ability of the dictionary. The structure of the dictionary and the corresponding linear classifier are obtained simultaneously by dictionary learning. Experimental results on two real hyperspectral image data have shown that the proposed GSVGDL approach could achieve better performance than several state-of-the-art methods.

The synergistic analysis of light detection and ranging (LiDAR) and hyperspectral data is attracting a significant interest in recent years due to the complementary nature of these two sources of remote sensing data. In this paper, we propose a new spectral-spatial classification method able to jointly exploit these two kinds of data. Our work is based on three innovative components: 1) a superpixel generation method aimed at multivariate image spatial partitioning, 2) a multi-source framework for feature extraction, and 3) a convex framework used to approach the solutions of the resulted image labeling problem associated with vectorial total variation and superpixel-based graph total variation regularizers. Our experimental results, conducted with a hyperspectral data set collected by the Compact Airborne Spectrographic Imager (CASI) spectrometer over the city of Houston in 2013 and a corresponding LiDAR data set, illustrate the effectiveness of the proposed framework.

This work introduces a cloud shadow removal method for hyperspectral images (HSIs) based on hyperspectral unmixing. The shading is modeled by a spectral offset and a spectral-dependent attenuation. The offset and the attenuation are estimated by solving a nonconvex optimization problem, which exploits the linear mixing model (LMM). The mixing matrix of the LMM is estimated from the unshadowed image areas. The effectiveness of the proposed method is assessed from classification results of the Houston 2013 (Compact Airborne Spectrographic Imager (CASI) spectrometer VHR HS), whose shadowed areas were removed with the proposed method. The obtained results indicate classification performances in the shadowed areas very close to those of the unshadowed ones, thus providing evidence of the effectiveness of the proposed shading removal technique.

Convolutional neural networks (CNNs) have shown great potential for remote sensing image classification. As the features obtained from a deep CNN generally exhibit high generalization capacity, the subsequent classifier is normally able to provide good results without the need for careful optimization. However it is well-known that, in the pursuit of high classification results, it is generally difficult to acquire a large number of training samples for the learning stage of the CNN. Therefore, it is important to exploit not only the feature generalization ability, but also the machine generalization ability for accurate classification with limited training samples. In this work, we introduce a new two-fold sparse regularization method (based on the deep CNN learning framework) for remote sensing image classification. Our proposed method, called TFCNN (for two-fold CNN), exploits the fact that the sparseness of features leads to increased linear class-separability. It also relies on the fact that the sparseness of the activation function collaborates with those features in a straightforward manner. As a result, the proposed TFCNN naturally achieves (for the first time in the literature) both feature and machine generalization, in complementary fashion. Our experimental results are conducted with three types of remote sensing data: hyperspectral images, multispectral images and synthetic aperture radar (SAR), suggesting that the proposed framework achieves excellent classification performance in all cases.

Endmember extraction is a fundamental task in spectral unmixing of remotely sensed hyperspectral images. In this work, we develop a new robust algorithm for endmember extraction which is based on a nonnegative sparse autoencoder. The proposed approach is based on two main steps. First, it uses an automatic sampler approach with local outlier factor and affinity propagation to intelligently gather a set of training samples. Then, a set of endmember signatures are extracted from the selected training samples by the nonnegative sparse autoencoder. Taking advantage from both automatic sampling and nonnegative sparse autoencoding, the proposed method can tackle problems with outliers. The effectiveness of the proposed method is verified by using simulated data. In our comparison with other state-of-the-art endmember extraction methods, the proposed approach demonstrates highly competitive performance.

Sparse unmixing of hyperspectral data is an important technique which aims at estimating the fractional abundances of endmembers (pure spectral components). It is well known that enforcing sparseness becomes a necessary process in sparse unmixing methods. To better exploit the sparsity in hyperspectral imagery, a double reweighted sparse unmixing algorithm has been proposed. However, it focusses on analyzing the hyperspectral data without fully incorporating the spatial information. To address this limitation, a spatial weighted sparse unmixing (SWSU) algorithm is proposed in this paper, which can take full advantage of the spatial information and further enhance the sparsity of the abundances. This is done by incorporating local neighborhood weights into the double reweighted sparse unmixing formulation. Experimental results on simulated hyperspectral data sets illustrate the good potential of the spatial weighted strategy for sparse unmixing introduced in this paper, which can greatly improve abundance estimation results.

Morphological attribute profiles (MAPs) are one of the most effective methodologies to characterize the spatial information in remote sensing images. This technique extracts components able to accurately describe objects in the surface of the Earth. In this work, we present a new method for impervious surface extraction from multispectral images using morphological attribute profiles. The proposed method first uses morphological profiles to extend Landsat ETM+ images with additional features. Then, we adopt a vegetation-impervious surface-soil (V-I-S) model and extract three pure classes (endmembers) from these images (i.e. vegetation, impervious surface and soil) using the vertex component algorithm (VCA). Finally, linear spectral mixture analysis (SMA) is conducted to extract the impervious surface percentage (ISP). To test the performance of the proposed method, more than 300 test samples including business districts, residential areas and urban roads are randomly selected from QuickBird imagery with very high resolution. The coefficient of determination R2is 0.7571, which significantly outperformed other standard techniques in the literature. The obtained experimental results demonstrate that the proposed approach based on morphological attribute profiles can lead to very good extraction and characterization of impervious surfaces.

Spectral unmixing is an important technique in hyperspectral image applications. Recently, sparse regression has been widely used in hyperspectral unmixing, but its performance is limited by the high mutual coherence of spectral libraries. To address this issue, a new sparse unmixing algorithm, called double reweighted sparse unmixing and total variation (TV), is proposed in this letter. Specifically, the proposed algorithm enhances the sparsity of fractional abundances in both spectral and spatial domains through the use of double weights, where one is used to enhance the sparsity of endmembers in spectral library, and the other is introduced to improve the sparsity of fractional abundances. Moreover, a TV-based regularization is further adopted to explore the spatial-contextual information. As such, the simultaneous utilization of both double reweighted l1minimization and TV regularizer can significantly improve the sparse unmixing performance. Experimental results on both synthetic and real hyperspectral data sets demonstrate the effectiveness of the proposed algorithm both visually and quantitatively.

The convergence of remote sensing technologies with social media, coupled with advances in other location-aware technologies such as WiFi and smartphones, is moving us on a fast track to a situation in which we can readily know, at any time, where everybody and everything are located on the surface of the Earth, and to exploit the power of social media in different contexts. Remote sensing technology involves the use of systems and algorithms to record information about the surface of the Earth from a remote location [1]. Although reliable as a data source, remote sensing data may not always be available. However, these data can be complemented by other sources of data, such as geographic information systems (GIS) and social media [2], in order to address time-critical applications. For instance, relating publicly available social media information with remote sensing or GIS data can lead to a more efficient management of emergency response (which refers to applications in which real/near-real-time response is needed, such as natural disasters). Social media are now playing a more relevant role in our daily lives and provide a unique opportunity to gain valuable insight on information flow and social networking within the society. As a result, the integration of social media data with other consolidated technologies such as remote sensing or GIS is of great importance.

The rapid development of social media data and the associated growth in volume, velocity, and variety has fostered the idea of using these data to guide traditional remote sensing image retrieval and information extraction tasks. Although important progress has been made in recent years in harvesting spatial and temporal data from social media, the exploitation of these data for decision making still needs further investigation, particularly in the context of its integration with remote sensing and geographic information systems. In this paper, we first discuss the relation between localization techniques and spatial technologies, pointing out their similarities and differences. Then, we provide a discussion on location analysis of social media data, and the fusion of multiple data sources, with specific attention to the integration of social media content (including localization) with remote sensing-based spatial technologies. Next, we provide specific examples addressing the use of social media data to perform information extraction from large remote sensing data repositories. Although significant possibilities for the integration of localization and spatial technologies can be seen in the examples provided, our survey suggests that the convergence of remote sensing and social media data will continue to deeply transform these technologies.

Remote sensing is a powerful technology for Earth observation (EO), and it plays an essential role in many applications, including environmental monitoring, precision agriculture, resource managing, urban characterization, disaster and emergency response, etc. However, due to limitations in the spectral, spatial, and temporal resolution of EO sensors, there are many situations in which remote sensing data cannot be fully exploited, particularly in the context of emergency response (i.e., applications in which real/near-real-time response is needed). Recently, with the rapid development and availability of social media data, new opportunities have become available to complement and fill the gaps in remote sensing data for emergency response. In this paper, we provide an overview on the integration of social media and remote sensing in time-critical applications. First, we revisit the most recent advances in the integration of social media and remote sensing data. Then, we describe several practical case studies and examples addressing the use of social media data to improve remote sensing data and/or techniques for emergency response.

The significant development of social media over the past decade has been complemented by the rise of spatial technologies to provide new mapping mechanisms that allow users engage with online information services and also with each other in an unprecedented way. Users of these technologies now provide a comprehensive geosocial overlay of the physical environment of the planet.

Spectral-spatial classification of remotely sensed hyperspectral images has attracted a lot of attention in recent years. Although Gabor filtering has been used for feature extraction from hyperspectral images, its capacity to extract relevant information from both the spectral and the spatial domains of the image has not been fully explored yet. In this paper, we present a new discriminative low-rank Gabor filtering (DLRGF) method for spectral-spatial hyperspectral image classification. A main innovation of the proposed approach is that our implementation is accomplished by decomposing the standard 3-D spectral-spatial Gabor filter into eight subfilters, which correspond to different combinations of low-pass and bandpass single-rank filters. Then, we show that only one of the subfilters (i.e., the one that performs low-pass spatial filtering and bandpass spectral filtering) is actually appropriate to extract suitable features based on the characteristics of hyperspectral images. This allows us to perform spectral-spatial classification in a highly discriminative and computationally efficient way, by significantly decreasing the computational complexity (from cubic to linear order) compared with the 3-D spectral-spatial Gabor filter. In order to theoretically prove the discriminative ability of the selected subfilter, we derive an overall classification risk bound to evaluate the discriminating abilities of the features provided by the different subfilters. Our experimental results, conducted using different hyperspectral images, indicate that the proposed DLRGF method exhibits significant improvements in terms of classification accuracy and computational performance when compared with the 3-D spectral-spatial Gabor filter and other state-of-the-art spectral-spatial classification methods.

A novel method for anomaly detection in hyperspectral images is proposed. The method is based on two ideas. First, compared with the surrounding background, objects with anomalies usually appear with small areas and distinct spectral signatures. Second, for both the background and the objects with anomalies, pixels in the same class are usually highly correlated in the spatial domain. In this paper, the pixels with specific area property and distinct spectral signatures are first detected with attribute filtering and a Boolean map-based fusion approach in order to obtain an initial pixel-wise detection result. Then, the initial detection result is refined with edge-preserving filtering to make full use of the spatial correlations among adjacent pixels. Compared with other widely used anomaly detection methods, the experimental results obtained on real hyperspectral data sets including airport, beach, and urban scenes demonstrate that the performance of the proposed method is quite competitive in terms of computing time and detection accuracy.

Regularization has appeared explicitly in hyperspectral image (HSI) classification community, which serves as a promising paradigm for leveraging labeled and unlabeled information, computer's automation and user's interaction, spectral and spatial information, and so on. Graph-based regularization is capable of modeling the nonlinear structures embedded in high-dimensional space, with the great potential for HSI classification. However, traditional methods exhibit low capacity when facing noisy and large-scale data, thus posing a big challenge for their successful use in this community. In this paper, we present two novel sparse graph regularization methods, SGR and SGR with total variation (TV-SGR). In SGR, the labels of large unknown data are propagated based on the fraction matrix and the prediction function, where the fraction matrix is obtained using an effective sparse representation (SR) algorithm with respect to the dictionary, and the prediction function is estimated by optimizing a typical graph-based regularization problem. In contrast, TV-SGR is an extension of SGR by considering spatial information modeled by total variation in SR. Propagating the prediction function from dictionary to large unknown data using the fraction matrix is the essence of the paradigm. SGR and TV-SGR can be equipped with semisupervised learning, active learning, and spectral-spatial classification with large flexibility. The experimental results with two popular hyperspectral data sets indicate that the proposed methods outperform some state-of-the-art approaches in terms of computational efficacy, classification accuracy, and robustness to noise.

Most blind hyperspectral unmixing methods exploit convex geometry properties of hyperspectral data. The minimum volume simplex analysis (MVSA) is one of such methods, which, as many others, estimates the minimum volume (MV) simplex where the measured vectors live. MVSA was conceived to circumvent the matrix factorization step often implemented by MV-based algorithms and also to cope with outliers, which compromise the results produced by MV algorithms. Inspired by the recently proposed robust MV enclosing simplex (RMVES) algorithm, we herein introduce the robust MVSA (RMVSA), which is a version of MVSA robust to noise. As in RMVES, the robustness is achieved by employing chance constraints, which control the volume of the resulting simplex. RMVSA differs, however, substantially from RMVES in the way optimization is carried out. In this paper, we develop a linearization relaxation of the nonlinear chance constraints, which can greatly lighten the computational complex of chance constraint problems. The effectiveness of RMVSA is illustrated by comparing its performance with the state of the art.

In this paper, we propose a multi-way projections-based reconstruction method for noise reduction of hyperspectral image (HSI). Core ideas of the proposed method are twofold: 1) the original HSI is partitioned into many small three-dimensional (3D) patches. Each of the patch is taken as a third-order tensor, on which compressive multi-way measurements are performed; 2) denoised patches are produced by the approximate low multilinear-rank reconstructions, and the final denoised HSI can be obtained by putting the denoised patches back to where they are in the original HSI. Experiments conducted on the real hyperspectral data set demonstrate the promising performance of the proposed method.

Active learning has obtained a great success in supervised remotely sensed hyperspectral image classification, since it can be used to select highly informative training samples. As an intrinsically biased sampling approach, it generally favors the selection of samples following discriminative distributions, i.e., those located in low density areas in feature space. However, the hyperspectral data are often highly mixed, i.e., most samples fluctuate in a local density areas. In this case, the potential of active learning for effective training sample selection is more limited. In order to address this relevant issue, we develop a new Gabor-based active learning approach for hyperspectral image classification, which consists of two main steps. First, we use a Gabor filter for feature extraction, which aims at bringing the data into a discriminative space. Then, we perform active learning to find the most informative training samples in the low density areas prior to the final classification. Our experimental results, conducted using two real hyperspectral datasets, indicate that the proposed Gabor-based approach can greatly improve the potential of active learning for classification purposes.

Supervised classification of hyperspectral images is a challenging task due to the relatively low ratio between the number of training samples and the number of spectral channels. Subspace-based classification methods deal with this difficulty by assuming that feature vectors lie in a low-dimensional subspace. Based on the fact that a class in a hyperspectral image may be composed of a number of different groups of materials and mixture of spectral features, we suggest to estimate several lower dimensional random subspaces for the samples within each class. For subspace learning and classification, we propose to exploit the union of random subspaces in a Gaussian Mixture Model. Experimental results, conducted on two real hyperspectral data sets, indicate that the proposed method provides competitive classification results in comparison with other state-of-the-art approaches.

The superpixels provided by an unsupervised segmentation algorithm are sets of neighboring pixels homogeneous in some sense. Therefore it is very likely that, in a classification problem, most pixels in a superpixel belong to the same class, namely if the homogeneity criterion is compatible with the class statistics. Superpixels are, therefore, a powerful device to express spatial contextual information. However, the exploitation of superpixels in a principled way is not straightforward. Recent efforts attack this problem under a discrete optimization framework, by including regularization terms promoting consistence of the labels in the superpixels and computing approximate labelings with graph-cut algorithms. The well known hardness of integer optimization problems is a major limitation of this line of attack. In this paper, we introduce a new strategy, based on convex relaxation, to include the spatial information provided by superpixels in classification problems. The convex relaxation of an integer optimization problem opens a door to include extra information, such as spatial partitioning information given by over-segmented superpixels. The convex optimization problem thus obtained is solved by using SALSA algorithm. Experimental results with the ROSIS Pavia University dataset illustrate the effectiveness of the proposed framework.

In this paper, we joint autoencoder with active learning for hyperspectral imagery classification. Specifically, we learn the classifier via autoencoder, where the most informative samples are acitvely selected through the interaction between the autoencoder and active learning. Experimental results, conducted using both the Kennedy Space Center and the Indian Pines hyperspectral images, show that driven by active learning, the performance of autoencoder can be greatly improved.

This paper presents a new classification framework for the fusion of hyperspectral and LiDAR data. The proposed approach aims at exploiting the complementarity of the features, i.e., textural features in the hyperspectral data and the height features in the LiDAR data, respectively. In this work, we use a morphological component analysis (MCA) method for textural feature extraction. The classification is then executed by a multinomial logistic regression classifier (MLR). Our obtained experimental results reveal that the proposed feature fusion method can lead to very good classification results.

This paper proposes a new supervised classification method for hyperspectral images combining the spectral and spatial information. The main contribution is presented by combining subspace-based support vector machine (SVMsub) and Markov random field (MRF). A SVM classifier integrated with a subspace projection is first used to model the posterior distributions of the classes from the spectral information. Then, the spatial information is modeled by a multilevel MRF. Finally, the maximum posterior probability classification is computed via the α-Expansion graph-cut-based optimization algorithm. The proposed method, abbreviated as SVMsub-MRF, is validated using a real typical hyperspectral data set. The results indicate that the proposed method exhibits better performance on accuracy and computational cost compared to other related classical hyperspectral image classification methods.

The spatial information has been proved to be effective in improving the performance of spectral-based classification. However, it is difficult to describe different image scenes by using monofeature owing to complexity of the geospatial scenes. In this letter, a novel framework is developed to combine the multiple spectral and spatial features based on the Markov random field (MRF). Specifically, the pixels in an image are separated into reliable and unreliable ones according to the decision of multifeature classifications. The labels of the reliable pixels can be conveniently determined, but the unreliable pixels are then classified by fusing the multifeature classification results and reducing the classification uncertainties based on the MRF optimization. Experiments are conducted on three multispectral high-resolution images to verify the effectiveness of the proposed method. Several state-of-the-art multifeature classification methods are also achieved for the purpose of comparison. Moreover, three classifiers (i.e., multinomial logistic regression, support vector machines, and random forest) are used to test the performance of the proposed framework. It is shown that the proposed method can effectively integrate multiple features, yield promising results, and outperform other approaches compared.

Spectral unmixing is an important technique for hyperspectral data exploitation. In order to solve the unmixing problem using a collection of previously available spectral signatures (i.e., a spectral library), sparse unmixing aims at finding the optimal subset of endmembers to represent the pixels in a hyperspectral image. The classic collaborative unmixing globally assumes that all pixels in a hyperspectral scene share the same active set of endmembers. This assumption rarely holds in practice, as endmembers tend to appear localized in spatially homogeneous areas rather than spread over the whole image. To address this limitation, in this letter, we introduce a new strategy to preserve local collaborativity for sparse hyperspectral unmixing. The proposed approach, which is called local collaborative sparse unmixing, considers the fact that endmember signatures generally appear distributed in local spatial regions instead of uniformly distributed throughout the scene. The proposed approach, which includes spatial information in the standard collaborative formulation, has been experimentally validated using both simulated and real hyperspectral data sets.

In this paper, we propose a fast 3-D empirical mode decomposition (fTEMD) method for hyperspectral images (HSIs) to achieve class-oriented multitask learning (cMTL). The major steps of the proposed method are twofold: 1) fTEMD and 2) cMTL. On the one hand, the traditional empirical mode decomposition is extended to its 3-D version, which naturally treats the HSI as a cube and effectively decomposes the HSI into several 3-D intrinsic mode functions (TIMFs). To accelerate the fTEMD, 3-D Delaunay triangulation is adopted to determine the distances of extrema, whereas separable filters are implemented to generate the envelopes. On the other hand, cMTL is performed on the TIMFs by taking those TIMFs as features of different tasks. The proposed cMTL learns the representation coefficients by taking advantage of the class labels and fully exploiting the information contained in each TIMF. Experiments conducted on three benchmark data sets demonstrate the effectiveness of the proposed method.

Spectral unmixing is an important technique for remotely sensed hyperspectral data exploitation. It amounts to identifying a set of pure spectral signatures, which are called endmembers, and their corresponding fractional, draftrulesabun-dances in each pixel of the hyperspectral image. Over the last years, different algorithms have been developed for each of the three main steps of the spectral unmixing chain: 1) estimation of the number of endmembers in a scene; 2) identification of the spectral signatures of the endmembers; and 3) estimation of the fractional abundance of each endmember in each pixel of the scene. However, few algorithms can perform all the stages involved in the hyperspectral unmixing process. Such algorithms are highly desirable to avoid the propagation of errors within the chain. In this paper, we develop a new algorithm, which is termed robust collaborative nonnegative matrix factorization (R-CoNMF), that can perform the three steps of the hyperspectral unmixing chain. In comparison with other conventional methods, R-CoNMF starts with an overestimated number of endmembers and removes the redundant endmembers by means of collaborative regularization. Our experimental results indicate that the proposed method provides better or competitive performance when compared with other widely used methods.

This paper presents a new approach for accurate spatial-spectral classification of hyperspectral images, which consists of three main steps. First, a pixelwise classifier, i.e., the probabilistic-kernel collaborative representation classification (PKCRC), is proposed to obtain a set of classification probability maps using the spectral information contained in the original data. This is achieved by means of a kernel extension based on collaborative representation (CR) classification. Then, an adaptive weighted graph (AWG)-based postprocessing model is utilized to include the spatial information by refining the obtained pixelwise probability maps. Furthermore, to deal with scenarios dominated by limited training samples, we modify the postprocessing model by fixing the probabilistic outputs of training samples to integrate the spatial and label information. The proposed approach is able to cover different analysis scenarios by means of a fully adaptive processing chain (based on three steps) for hyperspectral image classification. All the techniques that integrate the proposed approach have a closed-form analytic solution and are easy to be implemented and calculated, exhibiting potential benefits for hyperspectral image classification under different conditions. Specifically, the proposed method is experimentally evaluated using two real hyperspectral imagery data sets, exhibiting good classification performance even when the number of training samples available a priori is very limited.

Remote sensing images exhibit significant contrast and intensity regions and edges, which makes them highly suitable for using different texture features to properly represent and classify the objects that they contain. In this paper, we present a new technique based on multiple morphological component analysis (MMCA) that exploits multiple textural features for decomposition of remote sensing images. The proposed MMCA framework separates a given image into multiple pairs of morphological components (MCs) based on different textural features, with the ultimate goal of improving the signal-to-noise level and the data separability. A distinguishing feature of our proposed approach is the possibility to retrieve detailed image texture information, rather than using a single spatial characteristic of the texture. In this paper, four textural features: content, coarseness, contrast, and directionality (including horizontal and vertical), are considered for generating the MCs. In order to evaluate the obtained MCs, we conduct classification by using both remotely sensed hyperspectral and polarimetric synthetic aperture radar (SAR) scenes, showing the capacity of the proposed method to deal with different kinds of remotely sensed images. The obtained results indicate that the proposed MMCA framework can lead to very good classification performances in different analysis scenarios with limited training samples.

A novel method for anomaly detection in hyperspectral images (HSIs) is proposed based on low-rank and sparse representation. The proposed method is based on the separation of the background and the anomalies in the observed data. Since each pixel in the background can be approximately represented by a background dictionary and the representation coefficients of all pixels form a low-rank matrix, a low-rank representation is used to model the background part. To better characterize each pixel's local representation, a sparsity-inducing regularization term is added to the representation coefficients. Moreover, a dictionary construction strategy is adopted to make the dictionary more stable and discriminative. Then, the anomalies are determined by the response of the residual matrix. An important advantage of the proposed algorithm is that it combines the global and local structure in the HSI. Experimental results have been conducted using both simulated and real data sets. These experiments indicate that our algorithm achieves very promising anomaly detection performance.

The light detection and ranging (LiDAR) data provides very valuable information about the height of the surveyed area which can be used as a source of complementary information for the classification of hyperspectral data, in particular when it is difficult to separate complex classes. In this work, we suggest to exploit the generalized composite kernel strategy for fusion and classification of hyperspectral and LiDAR data. Our experimental results, conducted using a hyperspectral image and a LiDAR derived intensity image collected over a rural area in Extremadura, Spain, indicate that the proposed framework for the fusion of hyperspactral and LiDAR data provides significant classification results.

Principal component analysis (PCA) is an important method for feature extraction of hyperspectral remote sensing image. With the development of hyperspectral sensors, the magnitude of hyperspectral data grows quickly, and it is a challenging task to efficiently reduce the data dimension and compress massive data volumes in hyperspectral imaging. In this paper, a distributed parallel optimization of PCA algorithm (PCA_DP) is presented on cloud computing architecture. The realization of the proposed method using Apache Hadoop and MapReduce model is described and evaluated. The experiments conducted on real hyperspectral images of different sizes, demonstrate significant acceleration factor of PCA_DP. It is efficient for massive hyperspectral data processing.

This paper presents a new approach for class-oriented spectral partitioning for hyperspectral image classification. First, without empirical information, we automatically search the spectral bands that correspond to a specific class by using different band selection approaches. Then, the obtained class-oriented spectral partitions are used respectively as the input of a group of classifiers, the results of which are combined together to generate a final one by a multiple classifier system. Our experimental results, conducted with the well-known Indians Pines test site hyperspectral image collected by the Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) in NW Indiana, suggest that our presented spectral partitioning method leads to competitive results when compared with other state-of-the-art approaches.

Recently, a fast density peak-based clustering algorithm, namely FDPC, has demonstrated its power on nonspherical clustering problems. In this paper, we propose an enhanced fast density peak-based clustering, namely E-FDPC, for hy-perspectral band selection. The main contributions of the proposed E-FDPC, in comparison with the original FDPC are two folds. First, we introduce a parameter to control the weight between the normalized local density and intra-cluster distance. The other aspect is that, we present an exponential-based learning rule to adjust the cut-off threshold for different number of selected bands, where it is empirically defined in FDPC. Furthermore, an effective strategy, called isolated-point-stopping criterion, is developed to automatically determine the appropriate number of bands. That is, the clustering process will be stopped by the emergence of the isolated point (the only point in one cluster). Experimental results on real hyperspectral data demonstrate that E-FDPC approach could achieve higher overall classification accuracies than FDPC and other state-of-the-art band selection techniques.

In this work, we propose a new multiple morphological component analysis (MMCA) based decomposition framework for remote sensing image classification. The proposed MMCA framework aims at exploiting relevant textural characteristics present in a scene such as content, coarseness, contrast or directionality. Specifically, MMCA decomposes an image into a pair of morphological components (for each textural characteristic), which can be associated to a smooth and a textural components. The extracted features are then used for classification with a multinomial logistic regression (MLR). The experimental results, conducted using both a hyperspectral and a synthetic aperture radar (SAR) images, reveal that the proposed scheme can lead to state-of-the-art classification accuracy.

Hyperspectral image classification has been a very active area of research in recent years. It faces challenges related with the high dimensionality of the data and the limited availability of training samples. In order to address these issues, subspace-based approaches have been developed to reduce the dimensionality of the input space in order to better exploit the (limited) training samples available. An example of this strategy is a recently developed subspace-projection-based multinomial logistic regression technique able to characterize mixed pixels, which are also an important concern in the analysis of hyperspectral data. In this letter, we extend the subspace-projection-based concept to support vector machines (SVMs), a very popular technique for remote sensing image classification. For that purpose, we construct the SVM nonlinear functions using the subspaces associated to each class. The resulting approach, called SVMsub, is experimentally validated using a real hyperspectral data set collected using the National Aeronautics and Space Administration's Airborne Visible/Infrared Imaging Spectrometer. The obtained results indicate that the proposed algorithm exhibits good performance in the presence of very limited training samples.

In this letter, we present an efficient parallel implementation of composite kernels in support vector machines (SVMs) for hyperspectral image (HSI) classification. Our implementation makes effective use of commodity graphics processing units (GPUs). Specifically, we port the calculation of composite kernels to GPUs, perform intensive computations based on NVidia's compute unified device architecture, and execute the rest of the operations related with control and small data calculations in the CPU. Our experimental results, conducted using real hyperspectral data sets and NVidia GPU platforms, indicate significant improvements in terms of computational effectiveness, achieving near-real-time performance of spatial-spectral HSI classification for the first time in the literature.

In this letter, a real-time implementation of the logistic regression via variable splitting and augmented Lagrangian (LORSAL) algorithm for sparse multinomial logistic regression is presented on commodity graphics processing units (GPUs) using Nvidia's compute unified device architecture. The proposed parallel method properly exploits the GPU architecture at the low level, including its shared memory, and takes full advantage of the computational power of GPUs to achieve real-time classification performance of hyperspectral images for the first time in the hyperspectral imaging literature. Our experimental results reveal remarkable acceleration factors and real-time performance, while retaining exactly the same classification accuracy with regard to the serial and multicore versions of the classifier.

Linear spectral unmixing aims at estimating the number of pure spectral substances, also calledendmembers, their spectral signatures, and their abundance fractions in remotely sensed hyperspectral images. This paper describes a method for unsupervised hyperspectral unmixing called minimum volume simplex analysis (MVSA) and introduces a new computationally efficient implementation. MVSA approaches hyperspectral unmixing by fitting a minimum volume simplex to the hyperspectral data, constraining the abundance fractions to belong to the probability simplex. The resulting optimization problem, which is computationally complex, is solved in this paper by implementing a sequence of quadratically constrained subproblems using the interior point method, which is particularly effective from the computational viewpoint. The proposed implementation (available online: www.lx.it.pt/%7ejun/DemoMVSA.zip) is shown to exhibit state-of-the-art performance not only in terms of unmixing accuracy, particularly in nonpure pixel scenarios, but also in terms of computational performance. Our experiments have been conducted using both synthetic and real data sets. An important assumption of MVSA is that pure pixels may not be present in the hyperspectral data, thus addressing a common situation in real scenarios which are often dominated by highly mixed pixels. In our experiments, we observe that MVSA yields competitive performance when compared with other available algorithms that work under the nonpure pixel regime. Our results also demonstrate that MVSA is well suited to problems involving a high number of endmembers (i.e., complex scenes) and also for problems involving a high number of pixels (i.e., large scenes).

Classification and spectral unmixing are two important techniques for hyperspectral data exploitation. Traditionally, these techniques have been exploited independently. In this paper, we propose a new technique that exploits their complementarity. Specifically, we develop a new framework for semisupervised hyperspectral image classification that naturally integrates the information provided by discriminative classification and spectral unmixing. The idea is to assign more confidence to the information provided by discriminative classification for those pixels that can be easily catalogued due to their spectral purity. For those pixels that are more highly mixed in nature, we assign more confidence to the information provided by spectral unmixing. In this case, we use a traditional spectral unmixing chain to produce the abundance fractions of the pure signatures (endmembers) that model the mixture information at a subpixel level. The decision on which source of information is prioritized in the process is taken adaptively, when new unlabeled samples are selected and included in our semisupervised framework. In this regard, the proposed approach can adaptively integrate these two sources of information without the need to establish any weight parameters, thus exploiting the complementarity of classification and unmixing and selecting the most appropriate source of information in each case. In order to test our concept, which has similar computational complexity as traditional semisupervised classification strategies, we have used two different hyperspectral data sets with different characteristics and spatial resolution. In our experiments, we consider two different discriminative classifiers: multinomial logistic regression and probabilistic support vector machine. The obtained results indicate that the proposed approach, which jointly exploits the features provided by classification and spectral unmixing in adaptive fashion, offers an effective solution to improve classification performance in hyperspectral scenes containing mixed pixels.

Abstract—Hyperspectral image classification has been an active topic of research in recent years. In the past, many different types of features have been extracted (using both linear and nonlinear strategies) for classification problems. On the one hand, some approaches have exploited the original spectral information or other features linearly derived from such information in order to have classes which are linearly separable. On the other hand, other techniques have exploited features obtained through nonlinear transformations intended to reduce data dimensionality, to better model the inherent nonlinearity of the original data (e.g., kernels) or to adequately exploit the spatial information contained in the scene (e.g., using morphological analysis). Special attention has been given to techniques able to exploit a single kind of features, such as composite kernel learning or multiple kernel learning, developed in order to deal with multiple kernels. However, few

Sparse graph embedding (SGE) is a promising technique useful for the nonlinear feature extraction (FE) of hyperspectral images (HSIs). However, such images exhibit spatial variability and spectral multimodality, presenting challenges to existing FE methods, including SGE. To address this issue, this paper presents two novel SGE methods for HSI classification. One method, which is termed simultaneous SGE (SSGE), is designed to consider the spatial variability of spectral signatures by using a simultaneous sparse representation (SSR) model integrated with a shape-adaptive neighborhood building approach. In addition, a sparse graph is constructed via matrix computation based on sparse codes. Then, low-dimensional features are produced by employing linear graph embedding (LGE) based on the constructed sparse graph. The other method, which is termed simultaneous sparse multimanifold learning (SSMML), is proposed to handle the multimodality of an HSI. In SSMML, multiple views are generated to represent different modalities. Then, multiview-oriented submanifolds are produced by adopting SSGE, and they are further integrated via coregularization. SSGE is capable of modeling both local and global data structures. Furthermore, SSMML serves as a prototype that can model multimodal data structures. The proposed methods are evaluated by using sparse multinomial logistic regression for HSI classification. Experimental results with two popular hyperspectral data sets validate the good performance of the two methods in producing more representative low-dimensional features and yielding superior classification results compared with other related approaches.

This paper presents a new spectral-spatial classification method for hyperspectral images via morphological component analysis-based image separation rationale in sparse representation. The method consists of three main steps. First, the high-dimensional spectral domain of hyperspectral images is reduced into a low-dimensional feature domain by using minimum noise fraction (MNF). Second, the proposed separation method is acted on each features to generate the morphological components (MCs), i.e., the content and texture components. To this end, the dictionaries for these two components are built by using local curvelet and Gabor wavelet transforms within the randomly chosen image partitions. Then, sparse coding of one of the MCs and update of the associated dictionary are sequentially performed with the other one fixed. To better direct the separation process, an undecimated Haar wavelet with soft threshold is performed for the content component to make it smooth. This process is repeated until some stopping criterion is met. Finally, a support vector machine is adopted to obtain the classification maps based on the MCs. The experimental results with hyperspectral images collected by the National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible/Infrared Imaging Spectrometer and the Reflective Optics Spectrographic Imaging System indicate that the proposed scheme provides better performance when compared with other widely used methods.

This paper presents a new multiple feature learning approach for accurate spectral-spatial classification of hyperspec-tral images. The proposed method integrates multiple features based on the logarithmic opinion pool. We consider subspace multinomial logistic regression for classification as it exhibits a flexible structure for the combination of multiple features through the posterior probability. At the same time, it is able to cope with highly mixed hyperspectral data and with the presence of limited training samples. In this work, we considered lowpass filtering and morphological attribute profiles for spatial feature extraction. Our experimental results with a real hyperspectral images collected by the NASA Jet Propulsion Laboratory's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) indicate that the proposed method exhibits state-of-the-art classification performance.

In this paper, we present a new approach for spectral partitioning which is intended to deal with ill-posed problems in hyperspectral image classification. First, we use adaptive affinity propagation (AAP) to intelligently group the original spectral bands. Such grouping strategy not only allows us to reduce the number of spectral bands, but also to provide a different perspective on the original hyperspectral data. Then, a multiple classifier system (MCS) based on multinomial logistic regression (MLR) is applied. The system is trained using different band subsets resulting from the previously conducted intelligent grouping, and the results are combined to produce a final classification result. Our experimental results, conducted using the well-known hyperspectral scenes collected by the Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the Indian Pines region in NW Indiana, indicate that the proposed method can provide important advantages in terms of classification, in particular, when the number of training samples available a priori is very low.

In this letter, we propose a multinomial-logistic-regression method for pixelwise hyperspectral classification. The feature vectors are formed by the energy of the spectral vectors projected on class-indexed subspaces. In this way, we model not only the linear mixing process that is often present in the hyperspectral measurement process but also the nonlinearities that are separable in the feature space defined by the aforementioned feature vectors. Our experimental results have been conducted using both simulated and real hyperspectral data sets, which are collected using NASA's Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and the Reflective Optics System Imaging Spectrographic (ROSIS) system. These results indicate that the proposed method provides competitive results in comparison with other state-of-the-art approaches.

Remotely sensed hyperspectral image classification is a very challenging task. This is due to many different aspects, such as the presence of mixed pixels in the data or the limited information available a priori. This has fostered the need to develop techniques able to exploit the rich spatial and spectral information present in the scenes while, at the same time, dealing with mixed pixels and limited training samples. In this paper, we present a new spectral–spatial classifier for hyperspectral data that specifically addresses the issue of mixed pixel characterization. In our presented approach, the spectral information is characterized both locally and globally, which represents an innovation with regard to previous approaches for probabilistic classification of hyperspectral data. Specifically, we use a subspace-based multinomial logistic regression method for learning the posterior probabilities and a pixel-based probabilistic support vector machine classifier as an indicator to locally determine the number of mixed components that participate in each pixel. The information provided by local and global probabilities is then fused and interpreted in order to characterize mixed pixels. Finally, spatial information is characterized by including a Markov random field (MRF) regularizer. Our experimental results, conducted using both synthetic and real hyperspectral images, indicate that the proposed classifier leads to state-of-the-art performance when compared with other approaches, particularly in scenarios in which very limited training samples are available.

In recent years, sparse representations have been widely studied in the context of remote sensing image analysis. In this paper, we propose to exploit sparse representations of morphological attribute profiles for remotely sensed image classification. Specifically, we use extended multiattribute profiles (EMAPs) to integrate the spatial and spectral information contained in the data. EMAPs provide a multilevel characterization of an image created by the sequential application of morphological attribute filters that can be used to model different kinds of structural information. Although the EMAPs' feature vectors may have high dimensionality, they lie in class-dependent low-dimensional subpaces or submanifolds. In this paper, we use the sparse representation classification framework to exploit this characteristic of the EMAPs. In short, by gathering representative samples of the low-dimensional class-dependent structures, any given sample may by sparsely represented, and thus classified, with respect to the gathered samples. Our experiments reveal that the proposed approach exploits the inherent low-dimensional structure of the EMAPs to provide state-of-the-art classification results for different multi/hyperspectral data sets.

This paper presents a new multiple-classifier approach for accurate spectral-spatial classification of hyperspectral images, where the spectral information is exploited by combining probabilistic support vector machines (SVM) and subspace-based multinomial logistic regression (MLRsub) and the spatial information is exploited by means of a Markov random field (MRF) regularizer. The proposed approach is based on the decision fusion of global posterior probability distributions and local probabilities which result from the whole image and the class combinations map respectively. With respect to the SVM or MLRsub algorithms, the proposed method greatly improves the classification accuracy. Our experimental results with real hyperspectral images collected by the NASA Jet Propulsion Laboratory's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) and the Reflective Optics Spectrographic Imaging System (ROSIS), indicate that the proposed multiple-classifier system leads to state-of-the-art classification performance for cases with very limited number of training samples.

In this letter, we propose a new semisupervised learning (SSL) algorithm for remotely sensed hyperspectral image classification. Our main contribution is the development of a new soft sparse multinomial logistic regression model which exploits both hard and soft labels. In our terminology, these labels respectively correspond to labeled and unlabeled training samples. The proposed algorithm represents an innovative contribution with regard to conventional SSL algorithms that only assign hard labels to unlabeled samples. The effectiveness of our proposed method is evaluated via experiments with real hyperspectral images, in which comparisons with conventional semisupervised self-learning algorithms with hard labels are carried out. In such comparisons, our method exhibits state-of-the-art performance.

In this paper, we propose a new framework for spectral-spatial classification of hyperspectral image data. The proposed approach serves as an engine in the context of which active learning algorithms can exploit both spatial and spectral information simultaneously. An important contribution of our paper is the fact that we exploit the marginal probability distribution which uses the whole information in the hyperspectral data. We learn such distributions from both the spectral and spatial information contained in the original hyperspectral data using loopy belief propagation. The adopted probabilistic model is a discriminative random field in which the association potential is a multinomial logistic regression classifier and the interaction potential is a Markov random field multilevel logistic prior. Our experimental results with hyperspectral data sets collected using the National Aeronautics and Space Administration's Airborne Visible Infrared Imaging Spectrometer and the Reflective Optics System Imaging Spectrometer system indicate that the proposed framework provides state-of-the-art performance when compared to other similar developments.

Remotely sensed hyperspectral imaging allows for the detailed analysis of the surface of the Earth using advanced imaging instruments which can produce high-dimensional images with hundreds of spectral bands. Supervised hyperspectral image classification is a difficult task due to the unbalance between the high dimensionality of the data and the limited availability of labeled training samples in real analysis scenarios. While the collection of labeled samples is generally difficult, expensive, and time-consuming, unlabeled samples can be generated in a much easier way. This observation has fostered the idea of adopting semisupervised learning techniques in hyperspectral image classification. The main assumption of such techniques is that the new (unlabeled) training samples can be obtained from a (limited) set of available labeled samples without significant effort/cost. In this paper, we develop a new approach for semisupervised learning which adapts available active learning methods (in which a trained expert actively selects unlabeled samples) to a self-learning framework in which the machine learning algorithm itself selects the most useful and informative unlabeled samples for classification purposes. In this way, the labels of the selected pixels are estimated by the classifier itself, with the advantage that no extra cost is required for labeling the selected pixels using this machine–machine framework when compared with traditional machine–human active learning. The proposed approach is illustrated with two different classifiers: multinomial logistic regression and a probabilistic pixelwise support vector machine. Our experimental results with real hyperspectral images collected by the National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible–Infrared Imaging Spectrometer and the Reflective Optics Spectrographic Imaging System indicate that the use of self-learning represents an effective and promising strategy in the context of hyperspectral image classification.

This paper presents a new framework for the development of generalized composite kernel machines for hyperspectral image classification. We construct a new family of generalized composite kernels which exhibit great flexibility when combining the spectral and the spatial information contained in the hyperspectral data, without any weight parameters. The classifier adopted in this work is the multinomial logistic regression, and the spatial information is modeled from extended multiattribute profiles. In order to illustrate the good performance of the proposed framework, support vector machines are also used for evaluation purposes. Our experimental results with real hyperspectral images collected by the National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible/Infrared Imaging Spectrometer and the Reflective Optics Spectrographic Imaging System indicate that the proposed framework leads to state-of-the-art classification performance in complex analysis scenarios.

In this paper, we develop a new algorithm for hyperspectral unmixing which can provide suitable endmembers (and their corresponding abundances) in a single step. Hence, the algorithm does not require a previous subspace identification step to estimate the number of endmembers as it can cope with the two most likely scenarios in practice (i.e., the number of endmembers is correctly determined or overestimated a priori). The proposed approach, termed collaborative NMF (CoNMF), uses a collaborative regularization prior which forces the abundances corresponding to the overestimated endmembers to zero, such that it is guaranteed that only the true endmembers have fractional abundance contributions and the estimation of the number of endmembers is not required in advance. The obtained experimental results demonstrate that the proposed method exhibits very good performance in case the number of endmember is not available a priori.

In this paper, we develop a new framework for semi-supervised learning which exploits active learning for unlabeled sample selection in hyperspectral data classification. Specifically, we use active learning to select the most informative unlabeled training samples with the ultimate goal of systematically achieving noticeable improvements in classification results with regard to those found by randomly selected training sets of the same size. Our experimental results, conducted with an urban hyperspectral scene collected by the Reflective Optics Spectrographic Imaging Instrument (ROSIS) of the Deutschen Zentrum for Luftund Raumfahrt (DLR, the German Aerospace Agency) over the city of Pavia, Italy, indicate that using active learning for unlabeled sample selection represents an effective and promising strategy in the context of urban hyperspectral data classification.

This paper introduces a new supervised segmentation algorithm for remotely sensed hyperspectral image data which integrates the spectral and spatial information in a Bayesian framework. A multinomial logistic regression (MLR) algorithm is first used to learn the posterior probability distributions from the spectral information, using a subspace projection method to better characterize noise and highly mixed pixels. Then, contextual information is included using a multilevel logistic Markov-Gibbs Markov random field prior. Finally, a maximum a posteriori segmentation is efficiently computed by the min-cut-based integer optimization algorithm. The proposed segmentation approach is experimentally evaluated using both simulated and real hyperspectral data sets, exhibiting state-of-the-art performance when compared with recently introduced hyperspectral image classification methods. The integration of subspace projection methods with the MLR algorithm, combined with the use of spatial-contextual information, represents an innovative contribution in the literature. This approach is shown to provide accurate characterization of hyperspectral imagery in both the spectral and the spatial domain.

In this work, we present a new subspace discriminant analysis classification algorithm for remotely sensed hyperspectral image data. Our motivation for including subspace projection as a distinctive feature of our work is to better model noise and mixed pixels present in hyperspectral images. Two different dimensionality reduction techniques are considered: principal component analysis (PCA) and the hyperspectral signal identification by minimum error (HySime) algorithm. Experimental results indicate that the proposed method can provide competitive classification results (in the presence of very limited training data sets) with regards to those achieved by other state-of-the-art methods, such as linear discriminant analysis (LDA), subspace LDA, support vector machines (SVMs), and subspace SVMs using PCA and HySime for dimensionality reduction purposes.

This paper introduces a new supervised Bayesian approach to hyperspectral image segmentation with active learning, which consists of two main steps. First, we use a multinomial logistic regression (MLR) model to learn the class posterior probability distributions. This is done by using a recently introduced logistic regression via splitting and augmented Lagrangian algorithm. Second, we use the information acquired in the previous step to segment the hyperspectral image using a multilevel logistic prior that encodes the spatial information. In order to reduce the cost of acquiring large training sets, active learning is performed based on the MLR posterior probabilities. Another contribution of this paper is the introduction of a new active sampling approach, called modified breaking ties, which is able to provide an unbiased sampling. Furthermore, we have implemented our proposed method in an efficient way. For instance, in order to obtain the time-consuming maximum a posteriori segmentation, we use the α-expansion min-cut-based integer optimization algorithm. The state-of-the-art performance of the proposed approach is illustrated using both simulated and real hyperspectral data sets in a number of experimental comparisons with recently introduced hyperspectral image analysis methods.

This paper presents a new semisupervised segmentation algorithm, suited to high-dimensional data, of which remotely sensed hyperspectral image data sets are an example. The algorithm implements two main steps: 1) semisupervised learning of the posterior class distributions followed by 2) segmentation, which infers an image of class labels from a posterior distribution built on the learned class distributions and on a Markov random field. The posterior class distributions are modeled using multinomial logistic regression, where the regressors are learned using both labeled and, through a graph-based technique, unlabeled samples. Such unlabeled samples are actively selected based on the entropy of the corresponding class label. The prior on the image of labels is a multilevel logistic model, which enforces segmentation results in which neighboring labels belong to the same class. The maximum a posteriori segmentation is computed by the α-expansion min-cut-based integer optimization algorithm. Our experimental results, conducted using synthetic and real hyperspectral image data sets collected by the Airborne Visible/Infrared Imaging Spectrometer system of the National Aeronautics and Space Administration Jet Propulsion Laboratory over the regions of Indian Pines, IN, and Salinas Valley, CA, reveal that the proposed approach can provide classification accuracies that are similar or higher than those achieved by other supervised methods for the considered scenes. Our results also indicate that the use of a spatial prior can greatly improve the final results with respect to a case in which only the learned class densities are considered, confirming the importance of jointly considering spatial and spectral information in hyperspectral image segmentation.

This paper introduces a new semi-supervised classification and segmentation approach tailored to hyperspectral images. The posterior distributions of the classes are modeled by the multinomial logistic regression. The contextual information inherent to the spatial configuration of the image pixels is modeled by a Multi-Level Logistic (MLL) Markov-Gibbs random field. The multinomial logistic regressors, assumed to be random vectors with independent Lapla-cian components, are learned using the recently introduced LOR-SAL algorithm. The maximum a posteriori (MAP) segmentation is computed via the α-Expansion algorithm, a powerful graph cut based approach to integer optimization. The effectiveness of the proposed methodology is illustrated by classifying simulated and real data sets. Comparisons with state-of-art methods are also included.

This paper presents a new method of minimum volume class for hyperspectral unmixing, termed minimum volume simplex analysis (MVSA). The underlying mixing model is linear; i.e., the mixed hyperspectral vectors are modeled by a linear mixture of the endmember signatures weighted by the correspondent abundance fractions. MVSA approaches hyperspectral unmixing by fitting a minimum volume simplex to the hyperspectral data, constraining the abundance fractions to belong to the probability simplex. The resulting optimization problem is solved by implementing a sequence of quadratically constrained subproblems. In a final step, the hard constraint on the abundance fractions is replaced with a hinge type loss function to account for outliers and noise. We illustrate the state-of-the-art performance of the MVSA algorithm in unmixing simulated data sets. We are mainly concerned with the realistic scenario in which the pure pixel assumption (i.e., there exists at least one pure pixel per endmember) is not fulfilled. In these conditions, the MVSA yields much better performance than the pure pixel based algorithms.

Studying optical properties of atmospheric aerosol is important because aerosol affects people around the world significantly. These effects strongly depend on the physical and optical properties of aerosol particles. In this paper, we propose to use lidar, sun-photometer, and CALIPSO synchronously, then present combined retrieval to investigate the optical properties of aerosol. The observations were performed at Wuhan during the period of December 2007 to May 2008. The primary results show that the proposed method improved the precision of aerosol optical depth effectively. Furthermore, long-term atmospheric and aerosol data could be obtained by consecutive observations. Also these data will be useful for future understanding about their environmental and climate effects.

The stable aerosol retrieval algorithm needs a prior assumption of lidar ratio (the extinction-to-backscatter ratio), and the known aerosol type that is the prerequisite of this assumption, so how to identify the clouds and aerosol from lidar profile is fundamental to acquire atmospheric optical parameter. In this paper, we first employ the CloudSat to validate the CALISPO's classificatory results, which is released in different versions, after choosing more accurate classification, then start retrieving. Second, sun-photometer is used for verifying the CALIPSO's calibration coefficient and supplies the day time records which are relatively more accurate. Finally, aerosol characteristic in Hubei province is analyzed. All the data will supply more available information for further climate change research.

Obtaining high-accuracy optical property of atmosphere timely will lead to good results of atmospheric correction and real remote sensing image inversion. We have developed a multi-function Raman/Mie lidar system. Making use of this lidar, we can get the spatial distribution and the time evolution of many atmospheric parameters. In this paper, we studied the main factors affecting atmospheric correction, namely absorption and scattering by aerosols and several major atmospheric elements. Preliminary experimental results are described. These data are combined with sunphotometer data and another scanning Mie lidar data, integrated with the imagery from such as remote sensing of the Earth satellite to obtain the ground truth.

In this paper, a hybrid retrieval method of aerosol optical depth based on the combination of active and passive optical remote sensing is proposed. Two methods to retrieve the atmospheric optical depth are introduced: the so-called dark pixel method is used for retrieving the aerosol optical depth from MODIS; the other one is used by CALIPSO lidar data. After analyzing the two methods, the combined MODIS and CALIPSO method is applied for the aerosol optical depth, the primary experimental results show that these data are in agreement with each other in time-space evolvement trend.