Spatiotemporal fusion of remote sensing data is one of the critical techniques for Earth’s surface dynamic monitoring and analysis, which solves the limitation of spatial resolution and temporal coverage in individual sensor. In order to establish a more accurate and physically meaningful spatiotemporal fusion model, a weighted spatiotemporal fusion method via tensor collaborative representation (W-STFTCR) is proposed. Specifically, the collaborative representation (CR) constraint is incorporated into the tensor decomposition framework to prevent overfitting and enhance model robustness. Meanwhile, the superpixel segmentation strategy is adopted to partition the input difference image into superpixel blocks, facilitating block dictionary construction and clustering effectively. In addition, the normalized difference vegetation index (NDVI) and joint information entropy are introduced for weighting bands in predicting the final image, which leads to more accurate and physically meaningful outcomes. To verify the performance of the proposed method, the spatiotemporal fusion experiments on two publicly available datasets were conducted. The experiment results show that the proposed method outperforms the previous state-of-the-art (SOTA) spatiotemporal fusion algorithms, with excellent parameter robustness.

Hyperspectral images capture numerous narrow spectral bands to provide detailed information to identify and locate targets, making them highly suitable for anomaly detection tasks. In recent years, deep learning techniques have demonstrated impressive capabilities and prospects in hyperspectral anomaly detection (HAD), primarily relying on multilayer perceptrons (MLPs) and convolutional neural networks (CNNs) to extract and represent nonlinear features. However, MLPs and CNNs often require deeper network architectures when dealing with complex high-dimensional data, resulting in a constrained generalization and limited representation of features. To address this issue, and inspired by the recent Kolmogorov-Arnold network (KAN), this article introduces a novel asymmetric convolutional autoencoder (AE) network by integrating KAN and CNN, namedKACNet. Specifically, we design a spectral KAN block in the convolutional encoder and a spatial KAN block in the convolutional decoder, to simultaneously enhance the feature extraction and characterization capabilities of the network. Furthermore, to effectively utilize the limited prior information, a weight initialization mechanism based on hierarchical density-based spatial clustering of applications with noise (HDBSCAN) is developed to boost the background recovery. By combining KAN, CNN, and HDBSCAN, the proposed integration enhances the interpretability and reliability of HAD. Extensive experiments are conducted on six public datasets, demonstrating that the KAN poses remarkable performance on background reconstruction, particularly, the proposedKACNetsignificantly outperforms the other state-of-the-art methods.

A novel approach is proposed in this study that combines superpixel (SP) segmentation and multi-classifier ensemble learning to address the limited availability of labeled samples in the coastal wetland land cover classification. Firstly, the SP segmentation techniques is employed to partition unknown samples into multiple homogeneous regions, thereby facilitating the effective capture of spatial information pertaining to land cover. Subsequently, a multiclassifier ensemble learning strategy is employed within these regions to process the samples, effectively leading to a reduction in classification errors and an improvement in accuracy. To enhance the performance of semi-supervised learning (SSL), a sample iteration selection metric is introduced, optimizing the training samples based on the consistency of sample types within homogeneous regions and the results obtained from the multi-classifier ensemble, thus enhancing the reliability of pseudo-labels. Additionally, multi-scale SP segmentation is utilized to augment the ensemble strategy for samples, reducing the necessity for hyperparameter adjustments and increasing the automation and reliability of the model. The effectiveness of the proposed approach has been assessed through experiments conducted on Dafeng Natural Reserve hyperspectral images of wetlands in China.

Hyperspectral sensors carried by lunar rovers or satellites can effectively invert the mineral abundance of the lunar surface. Due to special environment of the lunar surface and small number of samples, it is a challenge to analyze typical minerals on the lunar surface using hyperspectral images. In this paper, a spectral-spatial diffusion autoencoder unmixing model (SSDiffAU) is proposed for mineral mapping. This is the first time the diffusion model is invoked in the unmixing field. The 3D-CNN is utilized as an encoder to represent deep spectral-spatial information. Additionally, the diffusion model is used to obtain high quality abundance maps, and then the endmember matrix is obtained by the decoder. Finally, the performance of the proposed algorithm was verified using popular unmixing datasets. The hyperspectral data acquired by Chang'e-3 Yutu rover are unmixed to estimate typical mineral spectra and their abundance.

In recent years, the rapid development of remote sensing (RS) technology has led to a drastic increase in the availability of RS images. This calls for the need to develop new methods able to effectively and efficiently retrieve the required instances from a massive amount of RS imagery. In retrieval tasks, finding the nearest-neighbor sample of the retrieval query is a fundamental research topic. Exhaustive comparison is the simplest method to accomplish this task. However, due to the involved computational complexity and memory limitations, this solution is no longer feasible in large data retrieval tasks. As an important branch of approximate nearest-neighbor retrieval (NNR), hash algorithms transform high-dimensional data into low-bit expressions (hash codes) with elements of 0 and 1 to reduce storage and computational costs. Hash algorithms aim to preserve the same nearest-neighbor relationship between the learned hash codes and the original data. Existing hash algorithms are divided into two classes: shallow and deep methods. Furthermore, deep hash algorithms can be divided into (semi-) supervised and unsupervised algorithms. In this article, representative hash-based RS image retrieval (HBRSIR) methods are reviewed, studying the application of hashing in other areas of the RS community and introducing available datasets and evaluation metrics for RS image retrieval (RSIR). The performance of representative and cross-modal hashing methods is validated using two common RSIR datasets (UCMerced and AID) and a cross-modal dataset (DSRSID). Prospects of future work summarizing HBRSIR are also provided.

Hyperspectral unmixing is a vibrant research field that focuses on the task of decomposing mixed pixels into a collection of pure spectral signatures, known as endmembers, along with their corresponding fractional abundances. Conventional unmixing algorithms often need to combine two techniques, namely endmember extraction and abundance estimation, to accomplish the unmixing task. Recently, deep learning (DL) has succeeded in the field of hyperspectral unmixing due to its strong feature learning and data-fitting capabilities. By extracting the output and weight of a particular layer as abundance maps and endmember signatures, available DL methods can directly unmix hyperspectral images. However, in order to improve the performance of spectral unmixing, such available DL methods frequently employ the results of endmember extraction algorithms –in most cases, the well-known vertex component analysis (VCA)– as the initial weights, which leads to significant limitations in their performance: a) the unmixing results are heavily dependent on the initialization given by VCA, and b) the randomness of VCA is passed to the unmixing network. In this paper, we design a new method called abundance-guided spectral and spatial network (A2SN) which not only skips the weights to extract endmember features directly from the network, but also estimates the abundance maps and reconstructs images directly. In particular, the proposed A2SN employs different kernels to capture spectral and spatial information. We also propose an abundance-guided attention spectral and spatial attention network (A2SAN) for hyperspectral unmixing by integrating attention mechanisms into A2SN. As a result, A2SAN is a completely innovative unmixing method that employs attention and reconstruction directly for hyperspectral unmixing, rather than just as modules for information extraction. Most importantly, both A2SN and A2SAN use a weighted summation of the feature maps to reconstruct the image and increase the noise immunity of the network. Experimental results, conducted on both synthetic and real datasets, demonstrate the effectiveness and superiority of A2SN and A2SAN over state-of-the-art unmixing methods. Our full code is released at https://github.com/xuanwentao/A2SN-and-A2SAN for public evaluation.

Hyperspectral anomaly detection, which aims to search and localize potential targets, is a research area with extensive application prospects and profound implications. In recent years, the emergence of unsupervised and self-supervised deep learning for image reconstruction has provided inspiring solutions for hyperspectral anomaly detection. However, due to sensor-induced and environmental effects, the full-image detection networks inevitably reconstruct anomalies along with the background. Existing detectors indirectly mitigate anomaly reconstruction by imposing constraints on hidden features or loss functions, but they provide unsatisfactory performance in large target detection scenarios. This work straightforwardly addresses this issue from the input source, i.e., introducing the concept of masked autoencoders (MAEs) into fully convolutional networks and further developing a sparse-inspired masked convolutional network (SMCNet) consisting of three mutually supportive components: 1) a hierarchical encoder; 2) a sparse projection layer; and 3) a hierarchical decoder. The encoder employs an adaptive potential anomaly masking strategy, leveraging sparse convolution for extracting multidimensional features of the remaining background. Meanwhile, a sparse-guided projection layer is created by discarding the positional embedding technique to populate the uncoded region and guide the background recovery without introducing anomalies. Finally, the decoder couples the hierarchical structure and a hybrid attention mechanism (local-middle–global and spatial-spectral) to refine the background during image recovery, whereas anomalies in the residual map are highlighted. Extensive experiments using ten typical competitors on six different types of datasets validate the effectiveness and generalization ability of the newly proposed SMCNet method.

In recent years, hyperspectral anomaly detection methods based on representation models has attracted much attention. However, when the dictionary is polluted by anomalous pixels, their performance is greatly affected. To adjust the contributions of different dictionary atoms, traditional methods usually predefine a distance weighting matrix and impose it on the dictionary matrix or coefficient vector, which may not be accurate enough. To solve this problem, a self-paced probabilistic collaborative representation detector (SP-ProCRD) is proposed in this article. It assigns weights for each atom loss term according to the probability that the pixel under test (PUT) belongs to the same class as each dictionary atom. Unlike the predefined weight matrix approach, a self-paced learning (SPL) strategy is used for iterative optimization, so that dictionary atoms participate in the representation from "good" to "bad" ones when solving the model. The representation residuals are utilized to accelerate the convergence. The proposed model can optimally represent each PUT using similar dictionary atoms and minimize the negative impact caused by anomalous atoms contained in the dictionary. In terms of weighting for SPL, an adaptive weighting scheme based on the polynomial self-paced (SP) regularizer is proposed to address the generalization issues of most previous weighting schemes. This scheme improves the generalization and automation of the model. Experimental results reveal that the proposed method produces more accurate result than existing methods and runs efficiently.

Graph convolutional networks (GCNs) have been skillfully employed in hyperspectral image (HSI) classification, exhibiting remarkable performance owing to their unique superiority in handling non-Euclidean graph-structured data. However, the inherent absence of predefined connections between pixels in HSI results in the underutilization of the structural and attribute information of the graph edges. Furthermore, the construction of adjacency matrices for large-scale HSI data imposes a huge computational burden on traditional GCNs. Therefore, in this article, a novel method combining relaxed collaborative representation (RCR) and GCN (RCR-GCN) for hyperspectral classification is proposed. Specifically, RCR is adopted to compute the representation coefficients of each feature, reflecting the similarity and diversity among different sample features. Meanwhile, the representation coefficients are applied as edge attributes in the graph, denoting the weights of the connections between neighboring nodes. After that, GCN is employed to classify the graph nodes. Moreover, an efficient version of the RCR-GCN method is developed to boost the computation, which constructs the graph based on superpixel nodes instead of the pixel nodes by using simple linear iterative clustering (SLIC). Extensive experiments on three HSI image datasets demonstrate that the proposed method outperforms other state-of-the-art methods and achieves more efficiency and feasibility in HSI image classification.

With the recent development of remote sensing technology, large image repositories have been collected. In order to retrieve the desired images of massive remote sensing data sets effectively and efficiently, we propose a novel central cohesion gradual hashing (CCGH) mechanism for remote sensing image retrieval. First, we design a deep hashing model based on ResNet-18 which has a shallow architecture and extracts features of remote sensing imagery effectively and efficiently. Then, we propose a new training model by minimizing a central cohesion loss which guarantees that remote-sensing hash codes are as close to their hash code centers as possible. We also adopt a quantization loss which promotes that outputs are binary values. The combination of both loss functions produces highly discriminative hash codes. Finally, a gradual sign-like function is used to reduce quantization errors. By means of the aforementioned developments, our CCGH achieves state-of-the-art accuracy in the task of remote sensing image retrieval. Extensive experiments are conducted on two public remote sensing image data sets. The obtained results support the fact that our newly developed CCGH is competitive with other existing deep hashing methods.

Hyperspectral images (HSIs) comprise plenty of information in the spatial and spectral domain, which is highly beneficial for performing classification tasks in a very accurate way. Recently, attention mechanisms have been widely used in HSI classification due to their ability to extract relevant spatial and spectral features. Notwithstanding their positive results, most of the attentional strategies usually introduce a significant number of parameters to be trained, making the models more complex and increasing the computational load. In this paper, we develop a new parameter-free attention network for HSI classification. The main advantage of our model is that it does not add parameters to the original network (as opposed to other state-of-the-art approaches), whilst providing higher classification accuracies. Extensive experimental validations and quantitative comparisons are conducted –using different benchmark HSIs– to illustrate these advantages. Code is available on https://github.com/mhaut/Free2Resnet.

Autoencoder-based hyperspectral anomaly detectors have received significant attention. The core of these detectors is to reconstruct backgrounds by optimizing autoencoders so that anomalies can be detected by reconstruction residuals. Nevertheless, existing methods are flawed in two aspects: 1) most of them reconstruct the background along with the anomalies, resulting in undesired performance for large target detection in complex backgrounds; 2) they only focus on the encoder optimization part, ignoring the decoder reconstruction quality of the background. Given the above, this paper proposes a background-guided deformable convolutional autoencoder (DCAE) network with three mutually supportive parts, including encoder, decoder, and background guidance modules. In the encoder, deformable convolution is introduced into regular convolution to build the adaptive spatial feature extractor to fit complex spatial structures, whilst a non-local convolution is introduced to build an external feature extractor to capture global spatial relationships. Further, a mask is designed to filter potential anomalous information, curbing the representation of high-frequency anomalies to focus on widespread backgrounds. In the decoder, a background guidance module (considering the physical meaning of linear reconstruction) is built, guiding the proposed network learning via two strategies. One is initializing the weight of the decoder, and another is adding a loss term. Notably, both the number of output channels of the encoder and the decoder construction are determined by the background guidance module, which creates a bridge between the network design and practical situations. A profound analysis demonstrates the outstanding performance of the proposed method, which outperforms traditional and deep learning methods, proving that the novel designs introduced in the network architecture are extremely effective.

Capsule networks have been a breakthrough in the field of automatic image analysis, opening a new frontier in the art for image classification. Nevertheless, these models were initially designed for RGB images and naively applying these techniques to remote sensing hyperspectral images (HSI) may lead to sub-optimal behaviour, blowing up the number of parameters needed to train the model or not correctly modeling the spectral relations between the different layers of the scene. To overcome this drawback, this work implements a new capsule-based architecture with attention mechanism to improve the HSI data processing. The attention mechanism is applied during the concurrent iterative routing procedure through an inverted dot-product attention.

Hyperspectral unmixing aims at extracting pure spectral signatures and estimating their corresponding abundances at each pixel. Traditional unmixing algorithms consider end-member extraction and abundance estimation as two separate steps, and the completion of abundance estimation requires results from other endmember extraction algorithms. Considering that convolutional neural networks (CNNs) have powerful learning and data fitting capabilities, some techniques based on deep learning (DL) have been proposed in the literature. Most of them only utilize spectral information and neglect spatial information. In addition, existing unmixing methods based on DL usually extract the weight and output of a specific activation layer as endmembers and abundances, respectively. In our work, we exploit 3D convolution to propose a new 3D convolution unmixing network (3DCUN) for hyperspectral unmixing. Two types of real data, i.e., Samson and Jasper, are used to evaluate the performance of our proposed 3DCUN in endmember extraction and abundance estimation. The experimental results reflect that our proposed 3DCUN gets accurate results in estimating endmembers and abundances.

The performance of hyperspectral anomaly detection based on representation models is importantly related to the corresponding dictionary. A good dictionary can optimally model background to detect anomalies. To realize adaptively background reconstruction, this paper constructs global-local dictionaries for collaborative representation detector by using adaptive-shape (SA-CRD). Specifically, robust principal component analysis (RPCA) is used to separate background and anomalies preliminarily. Then adaptive-shape neighbor is adopted to build local dictionaries for robust background region, and the robust background region is clustered to construct a global dictionary for potential anomaly region. Finally, global-local dictionaries are used in the collaborative representation model to finish anomaly detection. Obtained results over two real data sets indicate that the proposed method can improve the accuracy of anomaly detection intensively compared to other state-of-art methods.

Anomaly detection has become an important remote sensing application due to the abundant spectral and spatial information contained in hyperspectral images. Recently, hyperspectral anomaly detection methods based on collaborative representation model have attracted significant attention. Nevertheless, these methods have to face two main challenges: (1) all features (spectral signatures) are constrained to share the same representation coefficient, which ignores the differences among features; (2) existing dictionaries for pixel-by-pixel detection model are usually not reliable. To address these issues, this paper proposes a new relaxed collaborative representation detector for hyperspectral anomaly detection by using a novel non-global dictionary. The proposed detector conducts collaborative representation on each feature dimension of the pixel under test, and simultaneously constrains the coding vectors of different features to be similar. To the best of our knowledge, this is the first time that a detection model is built from each feature dimension. To adjust the contributions of each feature, an adaptive feature weight constrained version of the method is also proposed. The non-global dictionary is constructed by combining the k-nearest neighbor method and an existing global dictionary, which is more reliable and practical than the widely used dual windows dictionary. In addition, this paper also designs a band selection strategy for the proposed method. Experiments on five real datasets indicate that the proposed method suppresses background well and outperforms other classical and state-of-the-art methods.

Recently, collaborative representation classification (CRC) has attracted extensive interest for hyperspectral images (HSIs) classification. However, for collaborative representation with Tikhonov (CRT), a testing sample is collaboratively represented by training samples from all the classes, which may result in high computational cost. In this article, we select the first$k$class training samples that are nearest to the testing sample for representation, namely,$k$-nearest class CRT (KNCCRT) algorithm. In order to improve the performance of KNCCRT for HSI classification, the idea of random subspace-based KNCCRT ensemble framework is proposed. KNCCRT is adopted as base classifier and random subspace (RS) contributes to diversity by selecting feature randomly. Moreover, to further increase the classification accuracy, shape-adaptive (SA) neighborhood constraint is utilized in RS ensemble framework to incorporate spatial information. Experimental results on three real hyperspectral data sets demonstrate the effectiveness of the proposed methods for HSI classification. The combination of KNCCRT and RS framework provides a reliable accuracy for HSI classification.

Recently, low-rank representation and collaborative representation for hyperspectral anomaly detection are widely studied. In this paper, a novel anomaly detector which combines low-rank and collaborative representations for hyperspectral anomaly detection (LRCRD) is proposed. Different from existing anomaly detection methods using low-rank and collaborative representation, the proposed method divides an image into two parts: background and anomaly targets. A background dictionary is used to represent the background whose coefficient matrix is constrained by low-rank and l2norm minimization. The sparsely distributed anomalies are determined by the residual matrix which is constrained by l2,1norm minimization. Considering different similarities between a testing pixel and a dictionary atom, a distance-weighted matrix is adopted. Moreover, construction of the background dictionary avoids the pollution of abnormal pixels and makes the detection result more stable. Experimental results show that the LRCRD performs better than state-of-the-art anomaly detection methods.