Hyperspectral remote sensing images often suffer from mixed noise-Gaussian, stripe, and impulse-due to atmospheric interference, solar variability, and sensor imperfections. These noises are typically band-dependent and diverse in distribution, making unified denoising particularly challenging. Existing deep denoising methods rely on clean/noisy pairs, which are unavailable in real-world remote sensing, while traditional approaches require manual tuning and lack adaptability. We propose SpecEStop, a fully self-supervised spectral vector denoising framework that requires only a single noisy HSI for training. Leveraging a novel deep spectral prior, SpecEStop exploits the spectral bias of neural networks, which tend to learn low-frequency (clean) signal components with Gaussian noise before overfitting to non-Gaussian noise ones. An adaptive early stopping strategy halts training before non-Gaussian noise is learned, enabling effective suppression of complex noise patterns. To address remaining Gaussian noise, we purposely design the network architecture to preserve its statistical properties in the latent space, allowing the use of off-the-shelf Gaussian denoisers during inference. Without any clean supervision, SpecEStop achieves effective, stage-wise removal of mixed noise, as validated across diverse real-world scenarios. Code will be released at https://github.com/ruobing-Zhang and the permanent code repository maintained by the corresponding author at http://github.com/LinaZhuang.

Object Detection is a fundamental procedure in the interpretation of remote sensing images. In large-scale remote sensing images, it is common to observe that the interesting objects only occupy a small area. Such objects provide limited information gain and exhibit unclear edges, often named as small objects. The inherent characteristics of small objects significantly hinder the precise localization and accurate classification of deep object detection networks. In this paper, we introduce a significant challenge: the presence of similar objects among these small objects, which leads to dramatic misclassification and overall accuracy decrease. To assess this phenomenon, we propose a novel metric, Similar Category Angle (SCA), for classification discrimination, which serves to intuitively describe the network’s effectiveness in discriminating similar category objects in its final predictions. We also propose a one-stage object detection network named Similar Category Enhancement Network (SCENet), designed to tackle the challenges associated with discriminating similar objects in small object detection tasks. Specifically, we design SCA Loss guided by the SCA metric, which integrates SCA into the network training process, thereby enhances the network’s capability to discriminate between similar category objects. Meanwhile, we propose Laplacian Sobel Enhancement FPN, LSE-FPN, a module that incorporates dynamic edge extraction operators into the FPN to enhance the network’s ability to detect small objects by sharpening the explicit edges of objects in the feature map. Extensive experiments conducted on SODA-A, VisDrone2019 and FAIR1M-AIR datasets demonstrate the superiority of SCENet in the small object detection task, with significant improvements in detection results for both the mAP50 and SCA metrics. The code is available at https://github.com/weiziji01/SCENet.

Hyperspectral anomaly detection is an important task in remote sensing for identifying pixels with anomalous spectral signatures that deviate from their local background. Recently, low-rank and sparse representation-based methods have garnered significant attention in hyperspectral anomaly detection, which typically employ low-rank representation to characterize the background and sparse representation to capture anomalies. Since the background and anomalies usually exhibit complex characteristics beyond the low-rankness and sparsity, low-rank and sparse representation-based methods typically do not perform satisfactorily for complex scenarios. To address the challenge, we propose an unsupervised hyperspectral anomaly detection method from a continuous perspective, which organically integrates Continuous Background representation and deep Anomaly Representation (CBAR). Specifically, the CBAR model leverages the continuous low-rank tensor function to encapsulate both the low-rankness and smoothness of the background and the deep neural network to capture the complex geometric structure of anomalies. Moreover, to mitigate the overfitting of the background and anomalies to the observed HSI, we introduce two terms as overfit-shield by exploiting the prior knowledge of the background and anomalies. To solve the CBAR model, we develop an efficient alternating minimization algorithm. Extensive experiments on benchmark datasets (including Airport, Urban, Beach, and HYDICE) demonstrate that the proposed CBAR outperforms the state-of-the-art anomaly detection methods both qualitatively and quantitatively. For reproducibility, we will release our source code at: https://github.com/Weihao-Wu/CBAR.

Anomaly detection is one of the most popular research topics in hyperspectral remote sensing. A variety of traditional model-driven methods fail to reveal features of data with diversity due to monotonous, fixed analytical modes. This paper analyzes mathematical-statistical properties of hyperspectral images (HSIs) and proposes an interesting approach of information entropy estimation based on point-set topology (IEEPST) to resolve anomaly detection from a brand new perspective, thus eliminating the limitations caused by the data-model discrepancy. Specifically, the original HSI data are mapped into topological spaces to enable ordered arrangements, in preparation for revealing data features. Particularly, information entropy estimation is introduced for the first time in the adoption of point-set topology to adequately unravel the data arrangements in topological spaces, whereby the land cover information is efficiently extracted for detection. Experimental results demonstrate that IEEPST accommodates both detection accuracy and computational efficiency, and is highly competitive with other sophisticated and state-of-the-art methods.

Hyperspectral anomaly detection (HAD) poses the challenge of distinguishing anomalous targets from the majority of background objects without prior knowledge. Most existing deep learning (DL) models struggle to account for both local and global spatial-spectral features in the image, limiting their performance. In this letter, we introduce PUNNet, which integrates the patch-shuffle downsampling technique and nonlinear activation-free network (NAFNet) block with dilated convolution into an advanced blind-spot network for HAD. Specifically, PUNNet utilizes the patch-shuffle downsampling operation to extend its receptive field and exploits channel attention in the NAFNet block with dilated convolution to capture global contextual information in the image. Meanwhile, PUNNet satisfies the blind-spot requirement, meaning its receptive field excludes the center pixel’s information. This allows for reliable and precise background reconstruction in a self-supervised learning paradigm, further weakening anomalous feature expression and increasing the reconstruction error of anomalies. Experimental results demonstrate that PUNNet achieves a leading position in HAD performance. The code is available athttps://github.com/DegangWang97/IEEE_GRSL_PUNNet.

The detection of objects with larger aspect ratios (OLAR) is a challenging problem in a special application scenario, such as remote sensing object recognition and scene text detection. However, current object detectors perform poorly in OLAR feature extraction because they are incapable of adaptively responding to object shapes, which leads to severe misalignment between impure feature representations and region proposals. In this letter, we aim at solving this problem by proposing our shape-sensitive convolution network (SSC-Net). SSC-Net is carefully embedded with a feature enhancement module (SSC module) specifically suitable for OLAR. This module can use fewer sampling points to achieve more intelligent feature sampling area transformation, thus achieving the goal of enhancing OLAR feature representation. Extensive experiments on benchmark datasets that are rich in OLARs have proved the superiority of our method. Besides, we further verified the plug-and-play performance of the SSC module, and the experimental results show that it can significantly improve the detection performance of the detector for OLAR.

With hyperspectral remote sensors (imaging spectrometers) imaging a scene, the specificity of the target of interest is manifested in the significant differences between it and the surrounding background in terms of quantity, spatial distribution, and spectral characteristics, which provides conditions for the implementation of pixel-level diagnostics for target detection. Traditional model-driven methods utilize specific model assumptions to parse hyperspectral image (HSI) data in scenes with variability and are prone to encounter limitations due to model-data discrepancy. Most data-driven methods are limited in practical applications due to the great demand for training samples, the large number of parameters to be determined, and the costly computational complexity. To address the limitations of the existing methods, this article adopts point-set topology theories to analyze the properties of hyperspectral data at the mathematical-statistical level and seek a solution for the information retrieval task of target detection, whereby a target detection method through information entropy estimation based on point-set topology is proposed. First, parallel topological spaces are constructed to order the original HSI data to ensure that the differences in data features between various classes of land covers are reflected in intuitive properties in the topological spaces. Second, in conjunction with the priori information about the target, information entropy estimation is introduced to select optimal separable spaces for the target and the background by measuring the degree of ordering of data to achieve an accurate separation. Finally, a proper way to quantify and highlight the differences in data features between various land covers in the optimal separable spaces is explored for the algorithmic output to perform the information retrieval task. The proposed target detection through information entropy estimation based on point-set topology (TD-IEEPST) exploits an innovative combination of point set topology theories and information entropy estimation to achieve efficient extraction of land cover information for detection, ensuring both theoretical interpretability and computational efficiency. Extensive experimental results on real hyperspectral datasets verify that the proposed method is ahead of other widely used and state-of-the-art methods in terms of computational cost, detection effects, and robustness, and promising to provide technical support for detection response requirements in practical applications.

As one of the most active research hotspots in hyperspectral remote sensing, anomaly detection is widely used because it takes effect without any priori information about the target or the background. Most of the traditional model-driven methods fail to reveal features of data with diversity due to fixed analytical modes. A variety of data-driven methods encounter difficulties in practical applications due to their costly computational complexity. In this article, an innovative combination of point-set topology and information entropy theories is utilized to analyze the mathematical–statistical properties of hyperspectral images (HSIs), thus eliminating the limitations caused by the data-model discrepancy. Specifically, the original HSI data are mapped into topological spaces in a specific form to enable ordered arrangements, in preparation for revealing data features. In particular, information entropy estimation is introduced for the first time in the adoption of point-set topology to adequately unravel the data arrangements in topological spaces, whereby the land cover information is efficiently extracted for detection. Accordingly, an interesting approach of information entropy estimation based on point-set topology (IEEPST) is proposed to resolve anomaly detection from a brand new perspective, pursuing prominent detection accuracy while ensuring computational efficiency. The experimental results on benchmark HSI datasets demonstrate that IEEPST achieves detection performance with high probabilities of detection (PD) and low false alarm rates (FARs) at an inexpensive computational cost. The proposed IEEPST is highly competitive with other sophisticated and state-of-the-art methods.

Hyperspectral anomaly detection (HAD) aims to identify anomalous objects that deviate from surrounding backgrounds in an unlabeled hyperspectral image (HSI). Most available neural networks that make use of the reconstruction error to perform HAD tend to fit both backgrounds and anomalies, resulting in small reconstruction errors for both and not being effective in separating targets from background. To address this issue, we develop DirectNet, a new background reconstruction network for HAD that seamlessly integrates a sliding dual-window model into a blind-block architecture. Concretely, DirectNet establishes an inner window within the network’s receptive field by erasing the center block information, so that the content of the inner window remains invisible during the reconstruction of the central pixel. Additionally, the depth of our reconstruction network is adaptive to the size of the input image patch, ensuring that the network’s receptive field aligns with the dimensions of the input patch. The receptive field outside the inner window is considered an outer window. This weakens the impact of anomalies on the reconstruction process, causing the reconstructed pixels to converge towards the background distribution in the outer window region. Consequently, the reconstructed HSI can be regarded as a pure background HSI, leading to further amplification of reconstruction errors for anomalous targets. This enhancement improves the discriminatory ability of DirectNet. Specifically, DirectNet solely utilizes the outer window information to predict/reconstruct the central pixel. As a result, when reconstructing pixels inside anomalous targets of different sizes, the targets primarily fall within the inner window. Comprehensive experiments (conducted on four datasets) demonstrate that DirectNet achieves competitive performance compared to other state-of-the-art detectors.

In recent years, neural network-based methods have shown promising results in hyperspectral image (HSI) denoising area. Real HSIs exhibit substantial variations in noise distribution due to various factors such as different imaging techniques, camera variations, imaging environments, and hardware aging. In this paper, we develop an eigenimage plus eigennoise level map guided convolutional neural network for HSI denoising. Our main idea is to perform eigendecomposition on HSIs, utilize the low-rank property of HSIs in the spectral dimension and approximate the spectral vectors in a low-dimensional orthogonal subspace, where representation coefficients are called eigenimages. Besides eigenimages, we make use of estimated eigennoise level map as an input to guide the network for denoising. The proposed network can be constructed without restriction in the number of eigencomponents by using all eigenimages and eigennoise level maps of training noisy-clean pairs. In the inference part, the trained network can be used to remove noise in observed eigenimages without restriction in the number of eigencomponents, and an underlying clean image HSI can be estimated by performing orthogonal projection back. Experimental results on both simulated and real HSIs demonstrate the effectiveness of our trained Eigen-CNN compared with state-of-the-art HSI denoising methods. A MATLAB demo of this work is available at https://github.com/LinaZhuang/HSI-denoiser-Eigen-CNN for the sake of reproducibility.

The performance of deep learning-based denoisers highly depends on the quantity and quality of training data. However, paired noisy-clean training images are generally unavailable in hyperspectral remote sensing areas. To solve this problem, this work resorts to the self-supervised learning technique, where our proposed model can train itself to learn one part of noisy input from another part of noisy input. We study a general hyperspectral image (HSI) denoising framework, called Eigenimage2Eigenimage (E2E), which turns the HSI denoising problem into an eigenimage (i.e., the subspace representation coefficients of the HSI) denoising problem and proposes a learning strategy to generate noisy-noisy paired training eigenimages from noisy eigenimages. Consequently, the E2E denoising framework can be trained without clean data and applied to denoise HSIs without the constraint with the number of frequency bands. Experimental results are provided to demonstrate the performance of the proposed method that is better than the other existing deep learning methods for denoising HSIs. A MATLAB demo of this work is available at https://github.com/LinaZhuang/HSI-denoiser-Eigenimage2Eigenimage for the sake of reproducibility.

Without any prior information, hyperspectral anomaly detection is devoted to locating targets of interest within a specific scene by exploiting differences in spectral characteristics between various land covers. Traditional methods originated from the signal processing perspective, and most of them rely heavily on specific model assumptions. Because of the model-driven attributes, such methods cannot mine the deep-level features of data to adapt to the variability of scenes and cannot fully extract the information of land covers contained in images to accurately separate anomalies from the background. By independently designing a chessboard-shaped topological framework that avoids making any distribution assumptions but directly mines high-dimensional data features to break through the limitations of traditional detectors, this article proposes a novel chessboard topology-based anomaly detection (CTAD) method to dissect images and extract detailed information of land covers adaptively, thereby enabling highly accurate detection. Extensive experimental results on hyperspectral images (HSIs) in real scenes demonstrate that the proposed CTAD can be adapted to the variability of scenes by autonomously learning data features and exhibiting strong generalization and detection capabilities, facilitating practical applications.

Recent years have witnessed the flourishing of deep learning-based methods in hyperspectral anomaly detection (HAD). However, the lack of available supervision information persists throughout. In addition, existing unsupervised learning/semisupervised learning methods to detect anomalies utilizing reconstruction errors not only generate backgrounds but also reconstruct anomalies to some extent, complicating the identification of anomalies in the original hyperspectral image (HSI). In order to train a network able to reconstruct only background pixels (instead of anomalous pixels), in this article, we propose a new blind-spot self-supervised learning network (called BS3LNet) that generates training patch pairs with blind spots from a single HSI and trains the network in self-supervised fashion. The BS3LNet tends to generate high reconstruction errors for anomalous pixels and low reconstruction errors for background pixels due to the fact that it adopts a blind-spot architecture, i.e., the receptive field of each pixel excludes the pixel itself and the network reconstructs each pixel using its neighbors. The above characterization suits the HAD task well, considering the fact that spectral signatures of anomalous targets are significantly different from those of neighboring pixels. Our network can be considered a superb background generator, which effectively enhances the semantic feature representation of the background distribution and weakens the feature expression for anomalies. Meanwhile, the differences between the original HSI and the background reconstructed by our network are used to measure the degree of the anomaly of each pixel so that anomalous pixels can be effectively separated from the background. Extensive experiments on two synthetic and three real datasets reveal that our BS3LNet is competitive with regard to other state-of-the-art approaches.

Given a priori knowledge, hyperspectral target detection aims to locate objects of interest within specific scenes by utilizing differences in spectral characteristics among various land covers. However, for those traditional model-driven detectors with monotonic analytical mode, they perform mediocrely in the disassembly of hyperspectral image (HSI) data, failing to cope with real scenes with complexity. The discrepancy between fixed model assumptions and HSI data severely reduces detection effects, leading to the inability of such methods to mine deep-level features and adapt to the variability of imaging scenes. To overcome the limitations of traditional methods, we propose a chessboard-shaped topological framework for high-dimensional data structures to disassemble an HSI from both spatial and spectral dimensions adaptively. With hyperspectral target detection is refined into an information retrieval task in a topological space, a target detection method based on chessboard-shaped topology (CTTD) is proposed. In the topological space, latent and hidden data features of original images are presented in an intuitive way. Therefore, the differences in both spatial and spectral dimensions between the two classes of objects, namely target and background, are specifically amplified and exploited to perform the information retrieval task with superior performance. Extensive experimental results on benchmark HSI data sets demonstrate that CTTD can efficiently adapt to the variability of real scenes while extracting abundant and detailed information for accurate target localization. Moreover, both detection effects and computational efficiency exhibited by the proposed method provide a strong support for its popularization in practical applications.

The Plug-and-play (PnP) technique enables us to plug image priors into an ADMM framework for solving a regularized optimization problem. Deep image priors have shown their flexibility and robustness in solving several image inverse problems. Hyperspectral image (HSI) super-resolution problem is an ill-posed inverse problem that aims to obtain a high-resolution HSI (HR-HSI) by combining the information of low-resolution HSI (LR-HSI) and HR multispectral image simultaneously. This paper proposes a hyperspectral and multispectral image fusion framework termed E2E-fusion, plugged with a self-supervised deep learning prior calledEigenimage2Eigenimage. Firstly, the spectral low-rank structure of HSIs is exploited via subspace representations of spectra vectors. Meanwhile, benefiting from the high quality of the first eigenimage (i.e., representation coefficients), we design a self-supervised deep eigenimage guidance network image prior, E2E. By using the PnP technique, we plugged the E2E prior into the ADMM fusion framework to update the optimal objective function iteratively. The numerical experimental results both on the simulated datasets and real datasets demonstrate that the proposed method performs better than state-of-the-art fusion methods.

Hyperspectral anomaly detection (HAD) aims to identify anomalous targets that deviate from the surrounding background in unlabeled hyperspectral images (HSIs). Most existing deep networks that exploit reconstruction errors to detect anomalies are prone to fit anomalous pixels, thus yielding small reconstruction errors for anomalies, which is not favorable for separating targets from HSIs. In order to achieve a superior background reconstruction network for HAD purposes, this paper proposes a self-supervised blind-block network (termed BockNet) with a guard window. BockNet creates a blind-block (guard window) in the center of the network’s receptive field, rendering it unable to see the information inside the guard window when reconstructing the central pixel. This process seamlessly embeds a sliding dual-window model into our BockNet, in which the inner window is the guard window and the outer window is the receptive field outside the guard window. Naturally, BockNet utilizes only the outer window information to predict/reconstruct the central pixel of the perceptive field. During the reconstruction of pixels inside anomalous targets of varying sizes, the targets typically fall into the guard window, weakening the contribution of anomalies to the reconstruction results so that those reconstructed pixels converge to the background distribution of the outer window area. Accordingly, the reconstructed HSI can be deemed as a pure background HSI, and the reconstruction error of anomalous pixels will be further enlarged, thus improving the discrimination ability of the BockNet model for anomalies. Extensive experiments on four datasets illustrate the competitive and satisfactory performance of our BockNet compared to other state-of-the-art detectors.

Arbitrary-oriented object detection (AOOD) is widely used in aerial images because of its efficient object representation. However, current detectors employ the over-standardized feature extraction structure, resulting in detectors has no ability to adaptively readjust feature representations of detection units. Meanwhile, we observe that many detection units could not focus on the objects of interest in their receptive field and are easily affected by the background information and interference targets, leading to the weaking of feature expression ability. We call them sub-optimal detection units. To address this issue, we propose a novel feature enhancement module called fountain feature enhancement module (FFEM). FFEM ingeniously uses the fountain-like structure to reconstruct the features of sub-optimal detection units, generating fountain features that can automatically condense spatial regional features, which effectively enhances detectors’ overall representation ability. Then, a high-performance AOOD detector called fountain fusion net (FFN) is proposed with FFEM embedded, and many novel AOOD components are tested for their progressiveness. We validated our FFN and FFEM using three remote sensing datasets ‒ DOTA, HRSC2016, and UCAS-AOD as well as one scene text dataset‒ICDAR 2015. Extensive experiments demonstrate the effectiveness of our proposed method on improving current detectors to achieve state-of-the-art performance based on this novel idea.

A hyperspectral pushbroom sensor scans objects line-by-line using a detector array, and a cross-track illumination error (CTIE) exists in the imagery acquired in this way. When the illumination of the individual cells of the detector is not aligned well, or if some of the cells are degraded or old, the acquired images will exhibit nonuniform illumination in the cross-track direction. As additive Gaussian noise is found widely in hyperspectral images (HSIs), we develop a unified mathematical model that describes the image formation process corrupted by the CTIE and additive Gaussian noise. The CTIE produced by line-by-line scanning is replicated and modeled as an offset term with the equivalent values in the direction of flight. The main contribution of this study is the development of a hyperspectral image cross-track illumination correction (HyCIC) method, which corrects the cross-track illumination using column (along-track) mean compensation with total variation and sparsity regularizations, and attenuates the Gaussian noise by using a form of low-rank constraint. The effectiveness of the proposed method is illustrated using semireal data and real HSIs. The performance of the proposed HyCIC is found to be better than other existing methods.

Multi-shot coded aperture snapshot spectral imaging (CASSI) uses multiple measurement snapshots to encode the three-dimensional hyperspectral image (HSI). Increasing the number of snapshots will multiply the number of measurements, making CASSI system more appropriate for detailed spatial or spectrally rich scenes. However, the reconstruction algorithms still face the challenge of being ineffective or inflexible. In this paper, we propose a plug-and-play (PnP) method that uses denoiser as priors for multi-shot CASSI. Specifically, the proposed PnP method is based on the primal-dual algorithm with linesearch (PDAL), which makes it flexible and can be used for any multi-shot CASSI mechanisms. Furthermore, a new subspaced-based nonlocal reweighted low-rank (SNRL) denoiser is presented to utilize the global spectral correlation and nonlocal self-similarity priors of HSI. By integrating the SNRL denoiser into PnP-PDAL, we show the balloons ( 512×512×31 ) in CAVE dataset recovered from two snapshots compressive measurements with MPSNR above 50 dB. Experimental results demonstrate that our proposed method leads to significant improvements compared to the current state-of-the-art methods.

The decrease in the widths of spectral bands in hyperspectral imaging leads to a decrease in signal-to-noise ratio (SNR) of measurements. The decreased SNR reduces the reliability of measured features or information extracted from hyperspectral images (HSIs). Furthermore, the image degradations linked with various mechanisms also result in different types of noise, such as Gaussian noise, impulse noise, deadlines, and stripes. This article introduces a fast and parameter-free hyperspectral image mixed noise removal method (termed FastHyMix), which characterizes the complex distribution of mixed noise by using a Gaussian mixture model and exploits two main characteristics of hyperspectral data, namely, low rankness in the spectral domain and high correlation in the spatial domain. The Gaussian mixture model enables us to make a good estimation of Gaussian noise intensity and the locations of sparse noise. The proposed method takes advantage of the low rankness using subspace representation and the spatial correlation of HSIs by adding a powerful deep image prior, which is extracted from a neural denoising network. An exhaustive array of experiments and comparisons with state-of-the-art denoisers was carried out. The experimental results show significant improvement in both synthetic and real datasets. A MATLAB demo of this work is available at https://github.com/LinaZhuang for the sake of reproducibility.

Hyperspectral images (HSIs) exist in directional stripes commonly due to the failure of pushbroom acquisition. These stripes are not only vertically and horizontally oriented but also tend to be oblique. Furthermore, they can also be aperiodic and heavy. To address this problem, we propose a hyperspectral destriping algorithm, namely, GF-destriping. Taking advantage of the high sparsity and strong directionality of stripes in HSIs, Gabor filters are used to detect the stripes band by band first, and then, an advanced inpainting method, FastHyIn, is used to recover to the striped image. The numerical experiments on simulated data and real data sets show that our proposed algorithm is efficient and superior to state-of-the-art HSI destriping algorithms.

Tensor-based methods have been widely studied to attack inverse problems in hyperspectral imaging since a hyperspectral image (HSI) cube can be naturally represented as a third-order tensor, which can perfectly retain the spatial information in the image. In this article, we extend the linear tensor method to the nonlinear tensor method and propose a nonlinear low-rank tensor unmixing algorithm to solve the generalized bilinear model (GBM). Specifically, the linear and nonlinear parts of the GBM can both be expressed as tensors. Furthermore, the low-rank structures of abundance maps and nonlinear interaction abundance maps are exploited by minimizing their nuclear norm, thus taking full advantage of the high spatial correlation in HSIs. Synthetic and real-data experiments show that the low rank of abundance maps and nonlinear interaction abundance maps exploited in our method can improve the performance of the nonlinear unmixing. A MATLAB demo of this work will be available athttps://github.com/LinaZhuangfor the sake of reproducibility.

This article proposes a new denoising method for hyperspectral images (HSIs) corrupted by mixtures (in a statistical sense) of stripe noise, Gaussian noise, and impulsive noise. The proposed method has three distinctive features: 1) it exploits the intrinsic characteristics of HSIs, namely, low-rank and self-similarity; 2) the observation noise is assumed to be additive and modeled by a mixture of Gaussian (MoG) densities; 3) the inference is performed with an expectation maximization (EM) algorithm, which, in addition to the clean HSI, also estimates the mixture parameters (posterior probability of each mode and variances). Comparisons of the proposed method with state-of-the-art algorithms provide experimental evidence of the effectiveness of the proposed denoising algorithm. A MATLAB demo of this work will be available athttps://github.com/TaiXiangJiangfor the sake of reproducibility.

Recently, deep learning-based denoising methods for hyperspectral images (HSIs) have been comprehensively studied and achieved impressive performance because they can effectively extract complex and nonlinear image features. Compared with deep learning-based methods, the nonlocal similarity-based denoising methods are more suitable for images containing edges or regular textures. We propose a powerful HSI denoising method, termed NL-3DCNN, combining traditional machine learning and deep learning techniques. NL-3DCNN exploits the high spectral correlation of an HSI by using subspace representation and corresponding representation coefficients are termed eigenimages. The high spatial correlation in eigenimages is exploited by grouping nonlocal similar patches, which are denoised by a 3D convolutional neural network. The numerical and graphical denoising results of simulated and real data show that the proposed method is superior to state-of-the-art methods.

Hyperspectral imaging measures the amount of electromagnetic energy across the instantaneous field of view at a very high resolution in hundreds or thousands of spectral channels. This enables objects to be detected and the identification of materials that have subtle differences between them. However, the increase in spectral resolution often means that there is a decrease in the number of photons received in each channel, which means that the noise linked to the image formation process is greater. This degradation limits the quality of the extracted information and its potential applications. Thus, denoising is a fundamental problem in hyperspectral image (HSI) processing. As images of natural scenes with highly correlated spectral channels, HSIs are characterized by a high level of self-similarity and can be well approximated by low-rank representations. These characteristics underlie the state-of-the-art methods used in HSI denoising. However, where there are rarely occurring pixel types, the denoising performance of these methods is not optimal, and the subsequent detection of these pixels may be compromised. To address these hurdles, in this article, we introduce RhyDe (Robust hyperspectral Denoising), a powerful HSI denoiser, which implements explicit low-rank representation, promotes self-similarity, and, by using a form of collaborative sparsity, preserves rare pixels. The denoising and detection effectiveness of the proposed robust HSI denoiser is illustrated using semireal and real data.

This article proposes a smart hyperspectral sensing strategy, implemented in the spectral domain, conceived for spaceborne sensor systems, where physical space, storage resources, and communication bandwidth are extremely scarce and expensive. Smart sensing means faster and hardware-friendly imaging. Instead of acquiring all band samples in the spectral domain, we randomly select a few band samples per spatial pixel location. A periodic structure of spectral band selector array (SBSA) is designed so that we can learn a subspace basis from subsamples, which is essential to the underlying hyperspectral image (HSI) recovery algorithm. This spectral subsampling sensing strategy yields a demosaicing problem. We propose a blind hyperspectral reconstruction technique termed hyperspectral demosaicing (Hy-demosaicing) exploiting spectral low-rankness and spatial correlation of HSIs. It is blind in the sense that the signal subspace is learned from measured spectral subsamples. The subspace basis is data-adaptive and provides a more compact representation than other non-adaptive representations. This adaptiveness leads to improved image recovery as illustrated in experiments with real data.

Due to the importance in many military and civilian applications, hyperspectral anomaly detection has attracted remarkable interest. Low-rank representation (LRR)-based anomaly detectors use the low-rank property to represent background pixels, and pixels that cannot be well represented are detected as anomalies. The ability of an LRR-based detector to separate background pixels and anomalous pixels depends on the dictionary representation ability, which usually can be enhanced by designing a proper prior for dictionary representation coefficients and constructing a better dictionary. However, it is not easy to handcraft effective and meaningful regularizers for dictionary coefficients. In this article, we propose a novel anomaly detection algorithm that uses a plug-and-play prior for representation coefficients and constructs a new dictionary based on clustering. Instead of cumbersomely handcrafting a regularizer for representation coefficients, we propose solving the anomaly detection problem using the plug-and-play framework, which enables us to plug state-of-the-art priors for representation coefficients. An effective convolutional neural network (CNN) denoiser is plugged into our framework to fully exploit the spatial correlation of representation coefficients. We also propose a modified background dictionary construction method, which carefully includes background pixels and excludes anomalous pixels from clustering results. We refer to the proposed anomaly detection method as plug-and-play denoising CNN regularized anomaly detection (DeCNN-AD) method. Extensive experiments were performed on five data sets in a comparison with eight state-of-the-art anomaly detection methods. The experimental results suggest that the proposed method is effective in anomaly detection and can produce better anomaly detection results than that of the comparison methods. The codes of this work will be available athttps://github.com/FxyPdfor the sake of reproducibility.

In hyperspectral image (HSI) denoising, subspace-based denoising methods can reduce the computational complexity of the denoising algorithm. However, the existing matrix subspaces, which are generated by the unfolding matrix of the HSI tensor, cannot completely represent a tensor since the unfolding operation will destroy the tensor structure. To overcome this, we design a novel basis tensor that is directly learned from the original tensor and present a tensor subspace representation (TenSR), which is a more authentic representation for delivering the intrinsic structure of the tensor than a matrix subspace representation. Equipped with the TenSR, we then propose a TenSR-based HSI denoising (TenSRDe) model, which simultaneously considers the low-tubal rankness of the HSI tensor and the nonlocal self-similarity of the coefficient tensor. Moreover, we develop an efficient proximal alternating minimization (PAM) algorithm to solve the proposed nonconvex model and theoretically prove that the algorithm globally converges to a critical point. Experiments implemented on simulated and real data sets substantiate the denoising effect and efficiency of the proposed method.

The ever-increasing spectral resolution of hyperspectral images (HSIs) is often obtained at the cost of a decrease in the signal-to-noise ratio of the measurements, thus calling for effective denoising techniques. HSIs from the real world lie in low-dimensional subspaces and are self-similar. The low dimensionality stems from the high correlation existing among the reflectance vectors, and self-similarity is common in real-world images. In this article, we exploit the above two properties. The low dimensionality is a global property that enables the denoising to be formulated just with respect to the subspace representation coefficients, thus greatly improving the denoising performance and reducing the computational complexity during processing. The self-similarity is exploited via a low-rank tensor factorization of nonlocal similar 3-D patches. The proposed factorization hinges on the optimal shrinkage/thresholding of the singular value decomposition (SVD) singular values of low-rank tensor unfoldings. As a result, the proposed method is user friendly and insensitive to its parameters. Its effectiveness is illustrated in a comparison with state-of-the-art competitors. A MATLAB demo of this work is available athttps://github.com/LinaZhuangfor the sake of reproducibility.

Hyperspectral images (HSIs), featured by high spectral resolution over a wide range of electromagnetic spectra, have been widely used to characterize materials with subtle differences in the spectral domain. However, a large number of bands and an insufficient number of sample pixels for each class are challenging for traditional machine learning-based classifiers. As alternative tools for feature extraction, neural networks have received extensive attention. This letter proposes to combine t-distributed stochastic neighbor embedding (t-SNE) with a convolutional neural network (CNN) for HSI classification. Our framework is designed to automatically capture the potential assembly features, which are extracted from both the dimension-reduced CNN (DR-CNN) and the multiscale-CNN. Experimental results show that the proposed classification framework outperforms several state-of-the-art techniques for three real data sets.

Spectral-spatial framework has been widely applied for hyperspectral image classification task. Some well-established models, such as group sparse representation (GSR), have gained a certain advance but still mainly focus on the usage of local spatial similarity and neglect the nonlocal spatial information. Recently, nonlocal self-similarity (NLSS) has been exploited to support the spatial coherence tasks. However, current NLSS-based methods are biased toward the direct use of nonlocal spatial information as a whole, while the underlying spectral information is not well exploited. In this article, we proposed a novel method to exploit local spectral similarity through nonlocal spatial similarity, with the integration of local spatial consistency in a single framework. Specifically, the proposed approach first exploits the NLSS by searching the nonoverlapped similar patches in defined scopes. Then, spectral similarity is determined locally within the found patches. After that, the found similar data and the original data are fused in a designed pattern. Finally, the GSR-based classifier (GSRC) is applied to process the fused data characterized by the manifold learning algorithm. The experimental results based on three real hyperspectral data sets demonstrate the efficiency of the proposed method, with improvements over the other related nonlocal or local similarity-based methods.

Linear hyperspectral unmixing (HU) aims at factoring the observation matrix into an endmember matrix and an abundance matrix. Linear HU via variational minimum volume (MV) regularization has recently received considerable attention in the remote sensing and machine learning areas, mainly owing to its robustness against the absence of pure pixels. We put some popular linear HU formulations under a unifying framework, which involves a data-fitting term and an MV-based regularization term, and collectively solve it via a nonconvex optimization. As the former and the latter terms tend, respectively, to expand (reducing the data-fitting errors) and to shrink the simplex enclosing the measured spectra, it is critical to strike a balance between those two terms. To the best of our knowledge, the existing methods find such balance by tuning a regularization parameter manually, which has little value in unsupervised scenarios. In this paper, we aim at selecting the regularization parameter automatically by exploiting the fact that a too large parameter overshrinks the volume of the simplex defined by the endmembers, making many data points be left outside of the simplex and hence inducing a large data-fitting error, while a sufficiently small parameter yields a large simplex making data-fitting error very small. Roughly speaking, the transition point happens when the simplex still encloses the data cloud but there are data points on all its facets. These observations are systematically formulated to find the transition point that, in turn, yields a good parameter. The competitiveness of the proposed selection criterion is illustrated with simulated and real data.

This paper proposes a new denoising method for hyperspectral images (HSIs) corrupted by mixtures (in a statistical sense) of stripe noise, Gaussian noise, and impulsive noise. The proposed method has three distinctive features: 1) it exploits the intrinsic characteristics of HSIs, namely, low-rank and self-similarity; 2) the observation noise is assumed to be additive and modeled by a mixture of Gaussian (MoG) densities; 3) the inference is performed with an expectation maximization (EM) algorithm, which, in addition to the clean HSI, also estimates the mixture parameters (posterior probability of each mode and variances). Comparisons of the proposed method with state-of-the-art algorithms provide experimental evidence of the effectiveness of the proposed denoising algorithm.

This paper proposes a very light hyperspectral sensing strategy, implemented in the spectral domain, conceived to spaceborne sensor systems, where physical space, storage resources, and communication bandwidth are extremely scarce and expensive. Instead of acquiring all samples in spectral domain, we propose to randomly select a few samples per pixel. This subsampling sensing strategy yields a demosaicing problem. We propose a blind hyperspectral reconstruction technique termed hyperspectral demosaicing (Hy-demosaicing) exploiting low-rank and self-similarity properties of hyperspectral images. It is blind in sense that the signal subspace is learned from measured subsamples. The subspace basis is data adaptive and provides a more compact representation than other non-adaptive representations. This adaptiveness leads to improved image recovery as illustrated in experiments with real data.

Deblurring of document images is an important problem, with several relevant applications, such as camera-based document acquisition and processing systems. Consequently, considerable attention has been given to this problem, namely in the blind image deblurring (BID) scenario, where the blurring filter is (partially or fully) unknown. Traditional BID methods can be used for document images, but this is far from optimal, since those methods are tailored to natural images, that is, they rely on statistical properties of natural images. This has lead to the proposal of a few special-purpose techniques, namely by exploiting properties of text images. In fact, in document images, the most prevalent type of content is text, but in some cases, it is not the only one, with the other types being very different from text. For example, identity documents typically contain faces and/or fingerprints, which are not adequately treated by methods designed for images of text. In this work, we propose a new method for BID of documents, supported on a class-adapted dictionary-based prior (learned from one or more sets of clean images of specific classes) for the image and a sparsity-inducing prior on the (unknown) blurring filter. This approach handles document images that contain two or more image classes (e.g., text and faces) which is a main contribution of our work. Experiments with document images containing both text and faces show the competitiveness of the proposed method in terms of restoration quality. Additionally, our experiments show that the proposed method is able to handle images with strong noise, outperforming state-of-the-art methods designed for BID of text images.

The ever increasing spectral resolution of the hyperspectral images (HSIs) is often obtained at the cost of a decrease in the signal-to-noise of the measurements, thus calling for effective denoising techniques. HSIs from the real world live in low dimensional subspaces and are self-similar. The low dimensionality stems from the high correlation existing among the reflectance vectors and the self-similarity is common to images of the real world. In this paper, we exploit the above two properties. The low dimensionality is a global property, which enables the denoising to be formulated just with respect to the subspace representation coefficients, thus greatly improving the denoising performance and reducing the processing computational complexity. The self-similarity is exploited via low-rank tensor factorization of non-local similar 3D-patches. The proposed factorization hinges on optimal shrinkage/thresholding of SVD singular value of low-rank tensor unfoldings. As a result, the proposed method has no parameters, apart from the noise variance. Its effectiveness is illustrated in a comparison with state-of-the-art competitors.

Hyperspectral images (HSIs) cover hundreds of narrow spectral bands, thus yielding high spectral resolution, enabling precise identification of different materials. However, the existence of dead pixels in the light sensors produces a number of irrelevant measurements, which may compromise the usefulness of HSIs. In this paper, a new hyperspectral inpainting method, named HyInpaint, is proposed. The original HSI is represented on a low dimensional subspace and its estimation is formalized with respect to the subspace representation coefficients on a given basis. The coefficients are estimated by minimizing an objective function which, in addition to the data term, contains a regularizer based on the Criminisi's inpainting method. The optimization is carried out by an instance of the alternating direction method of multipliers (ADMM), adopting the plug-and-play methodology. The effectiveness of the proposed HyInpaint approach is illustrated on Tiangong-1 hyperspectral visible near infrared (VNIR) wavebands data.

The very high spectral resolution of Hyperspectral Images (HSIs) enables the identification of materials with subtle differences and the extraction subpixel information. However, the increasing of spectral resolution often implies an increasing in the noise linked with the image formation process. This degradation mechanism limits the quality of extracted information and its potential applications. This paper presents a new HSI denoising approach developed under the assumption that the clean HSI is low-rank and self-similar. Under these assumptions, the clean HSI admits extremely compact and sparse representations, which are exploited to derive a very fast and competitive denoising algorithm, named Fast Hyperspectral Denoising (FastHyDe), able to cope with Gaussian and Poissonian noise. In a series of experiments, the proposed approach competes with state-of-the-art methods, with much lower computational complexity.

Endmember variability is receiving growing attention in the hyperspectral image (HSI) unmixing field. As an extension of linear mixing model (LMM), normal compositional model (NCM) assumes that the pixels of the HSI are linear combinations of random endmembers (as opposed to deterministic for the LMM). NCM explains spectral differences between the observed pixels and endmembers as endmember mixtures and endmember variances, the characteristic of which makes it possible to incorporate the endmember spectral variability in the unmixing process. But the tricky issue for using NCM is the estimation of endmember variances inhering in materials. This letter presents a new approach, termed region-based stochastic expectation maximization, to learn endmember variances from spatial information. The idea is assuming that significant homogeneous regions (composed of similar materials or similar mixture) exist in the HSI, such regions usually give visual indication that spatial-based spectral variability really exists in hyperspectral data. As modeled in NCM, spectral variances in homogeneous region can be approximately linear represented by endmember variances. Hence, given region-based spectral variances, we are able to learn endmember variances. In experiments with simulated data and Moffett field data, the proposed approach competes with other unmixing methods considering endmember variability, with better endmember variance estimates.

A new hyperspectral unmixing algorithm is proposed based on the normal compositional model (NCM) to estimate the endmembers and abundance parameters jointly in this paper. The NCM considers the hyperspectral imaging as a stochastic process and interprets each pixel value as a random vector, which is linearly mixed by the endmembers. More precisely, these endmembers are also treated as random variables as opposed to deterministic values in order to capture spectral variability that is not well described by the linear mixing model (LMM). However, the higher complexity of such an unmixing model leads to more difficulty in parameter estimation. A particle swarm optimization-expectation maximization (PSO-EM) algorithm, a “winner-take-all” version of the EM, is proposed to solve the parameter estimation problem, which employs a partial E step. The main contribution of the proposed PSO-EM is making optimum use of particle swarm optimization method (PSO) in the partial E step, which solves the difficulty of the integrals in the NCM model. The performance of the proposed methodology is evaluated through synthetic and real data experiments. Our obtained results demonstrate the superior performance of PSO-EM compared to other NCM-based as well as LMM-based methods.