Hyperspectral unmixing, an essential and fundamental task in remote sensing, focuses on estimating endmembers (spectrally pure components) and their fractional abundances within each mixed pixel of a hyperspectral image. With the advent of deep learning (DL), the field of hyperspectral unmixing has made significant progress. Among DL approaches, autoencoder-based models have shown promising results. However, most unmixing methods estimate the endmembers by the weights of the linear layers in the decoder of their networks, making their performance highly dependent on weight initialization. Moreover, noise is not explicitly accounted for in most recent methods that use spectral angle distance (SAD) loss. To avoid the initialization problems, we developed an innovative inversion strategy to directly estimate the endmembers. Moreover, to optimally account for noise, an end-to-end network is proposed that integrates both denoising and unmixing. Finally, for an improved feature extraction, a novel spectral-spatial attention module is integrated in the network. Extensive experiments on a synthetic and three real datasets show that the proposed method significantly and consistently outperforms the compared state-of-the-art methods. The full code is available at https://github.com/xuanwentao for public evaluation.

Accurate estimation of the fractional abundances of intimately mixed materials from spectral reflectances is generally hard due to a highly nonlinear relationship between the measured spectrum and the composition of the material. Changes in the acquisition and the illumination conditions cause variability in the spectral reflectance, further complicating the spectral unmixing procedure. In this work, we propose a methodology for unmixing intimate mixtures that can tackle both nonlinearity and spectral variability. A supervised approach is proposed that characterizes the nonlinear data manifolds by high-dimensional Bézier surfaces. To deal with spectral variability, a manifold transformation procedure is designed. To generate Bézier surfaces, training samples are required that are uniformly distributed throughout the data manifold. For this, we recently generated a hyperspectral dataset of intimate mineral powder mixtures by homogeneously mixing five different clay powders (Kaolin, Roof clay, Red clay, mixed clay, and Calcium hydroxide) in laboratory settings. In total 330 samples (325 mixtures and five pure materials) were prepared. The ground fractional abundances of these mixtures uniformly cover the 5-D probability simplex. The spectral reflectances of these samples were acquired by multiple sensors with a large variation in sensor types, platforms, and acquisition conditions. Experiments are conducted both on simulated and real intimate mineral powder mixtures. Comparison with a number of unsupervised unmixing methods demonstrates the potential of the proposed approach.

Hyperspectral unmixing is a crucial technique in remote sensing data processing that aims to estimate component information from mixed pixels in hyperspectral images. Most existing deep learning-based hyperspectral unmixing models employ autoencoder (AE) networks to reconstruct hyperspectral images and estimate abundance maps. Here, the weight between the reconstructed and the softmax layers is used to extract/estimate endmember signatures. However, AEs are heavily dependent on initial weights, which introduces inherent randomness, potentially compromising unmixing accuracy. To address this issue, in this article, we present a new dual-feature fusion network (DFFN) for enhanced hyperspectral unmixing. Our DFFN mainly consists of four modules: 1) a feature fusion module (FFM); 2) an abundance estimation module (AEM); 3) an endmember estimation module (EEM); and 4) a reconstruction module (RM). First, FFM calculates spectral and spatial similarities and then enhances the hyperspectral image by matrix multiplications with similarity matrices. Second, AEM takes the enhanced hyperspectral image as input and uses convolutional layers to estimate abundances and reconstruct the image. Next, the reconstructed image is fed into EEM to automatically estimate endmembers. RE performs the final reconstruction through matrix multiplication of the estimated endmembers and abundances. Experiments on synthetic and real hyperspectral datasets, together with a comparison with state-of-the-art techniques, demonstrate the superiority of our newly proposed DFFN. The full code is released athttps://github.com/xuanwentaofor public evaluation.

Since many materials behave as heterogeneous intimate mixtures with which each photon interacts differently, the relationship between spectral reflectance and material composition is very complex. Quantitative validation of spectral unmixing algorithms requires high-quality ground truth fractional abundance data, which are very difficult to obtain.In this work, we generated a comprehensive hyperspectral dataset of intimate mineral powder mixtures by homogeneously mixing five different clay powders (Kaolin, Roof clay, Red clay, mixed clay, and Calcium hydroxide). In total 325 samples were prepared. Among the 325 samples, 60 mixtures were binary, 150 were ternary, 100 were quaternary, and 15 were quinary. For each mixture (and pure clay powder), reflectance spectra are acquired by 13 different sensors, with a broad wavelength range between the visible and the long-wavelength infrared regions (i.e., between 350 nm and 15385 nm) and with a large variation in sensor types, platforms, and acquisition conditions. We will make this dataset public, to be used by the community for the validation of nonlinear unmixing methodologies (https://github.com/VisionlabUA/Multisensor_datasets)

Leaf chlorophyll ab content is an important indicator of vegetation physiological status and is generally obtained from spectral reflectance. For non-destructive estimation of chlorophyll ab content, physical leaf reflectance models, such as the PROSPECT model and supervised methods have been applied. While the former generally does not perform optimal, the latter only performs well when trained on similar data. In this work, we developed a robust supervised method that overcomes this problem. The method derives a proxy for chlorophyll ab content as the relative position of a leaf reflectance spectrum on the arc spanned by the two extremes, containing high and low chlorophyll ab content. This proxy is found to be unaffected by spectral variability, caused by environmental and acquisition conditions. The relation between this proxy and the actual chlorophyll ab content is obtained by a supervised regression model, that is trained on a single leaf reflectance dataset, and that is transferable to other datasets. The proposed method is validated on seven real hyperspectral datasets.

This paper proposes a sparse unmixing technique using a convolutional neural network (SUnCNN). We reformulate the sparse unmixing problem into an optimization over the parameters of a convolutional network. Relying on a spectral library, the deep network learns in an unsuper-vised manner a mapping from a fixed input to the sparse abundances. Moreover, SUnCNN fulfills the sum-to-one constraint using a softmax activation layer. We compare SUnCNN with the state-of-the-art using a simulated and a real dataset. The experimental results show that the proposed deep learning-based unmixing method outperforms the oth-ers in terms of signal to reconstruction error. Additionally, SUnCNN is visually superior to the competing techniques. SUnCNN was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online: https://github.com/BehnoodRasti/SUnCNN.

This paper proposes a deep blind hyperspectral unmixing network for datasets without pure pixels called minimum simplex convolutional network (MiSiCNet). MiSiCNet is the first deep learning-based blind unmixing method proposed in the literature which incorporates both spatial and geometrical information of the hyperspectral data, in addition to the spectral information. The proposed convolutional encoder-decoder architecture incorporates the spatial information using convolutional filters and implicitly applying a prior on the abundances. We added a minimum simplex volume penalty term to the loss function to exploit the geometrical information. We evaluate the performance of MiSiCNet on simulated and real datasets. The experimental results confirm the robustness of the proposed method to both noise and absence of pure pixels. Additionally, MiSiCNet considerably outperforms the state-of-the-art unmixing approaches. The results are given in terms of spectral angle distance in degree for the endmember estimation, and root mean square error in percentage for the abundance estimation. MiS-iCNet was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online: https://github.com/BehnoodRasti/MiSiCNet.

In this letter, we propose a sparse unmixing technique using a convolutional neural network (SUnCNN) for hyperspectral images. SUnCNN is the first deep learning-based technique proposed for sparse unmixing. It uses a deep convolutional encoder–decoder to generate the abundances relying on a spectral library. We reformulate the sparse unmixing into an optimization over the deep network’s parameters. Therefore, the deep network learns in an unsupervised manner to map a fixed input into the sparse optimum abundances. Additionally, SUnCNN holds the sum-to-one constraint using a softmax activation layer. The proposed method is compared with the state-of-the-art using two synthetic datasets and one real hyperspectral dataset. The overall results confirm that the proposed method outperforms the other ones in terms of signal to reconstruction error (SRE). Additionally, SUnCNN shows visual superiority for both real and synthetic datasets compared with the competing techniques. The proposed method was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online:https://github.com/BehnoodRasti/SUnCNN.

Transformers have intrigued the vision research community with their state-of-the-art performance in natural language processing. With their superior performance, transformers have found their way in the field of hyperspectral image classification and achieved promising results. In this article, we harness the power of transformers to conquer the task of hyperspectral unmixing and propose a novel deep neural network-based unmixing model with transformers. A transformer network captures nonlocal feature dependencies by interactions between image patches, which are not employed in CNN models, and hereby has the ability to enhance the quality of the endmember spectra and the abundance maps. The proposed model is a combination of a convolutional autoencoder and a transformer. The hyperspectral data is encoded by the convolutional encoder. The transformer captures long-range dependencies between the representations derived from the encoder. The data are reconstructed using a convolutional decoder. We applied the proposed unmixing model to three widely used unmixing datasets, i.e., Samson, Apex, and Washington DC mall and compared it with the state-of-the-art in terms of root mean squared error and spectral angle distance. The source code for the proposed model will be made publicly available at https://github.com/preetam22n/DeepTrans-HSU.

Due to the complex interaction of light with moist soils, the soil moisture content (SMC) is hard to estimate from the soil spectral reflectance. Spectral variability, caused by variations in viewing and illumination angle and between-sensor variability, further complicates the estimation. In this work, we developed a supervised methodology to accurately estimate SMC from spectral reflectance. The method determines a proxy for the SMC of moist soil, making use of the reflectance spectra of an air-dried and saturated soil sample. The proxy is made invariant to illumination and viewing angle, and sensor type. In the next step, the proxy is normalized with respect to the ground-truth SMC of the saturated soil to make the technique less dependent on the soil type. The normalized proxy can be directly used as an estimate of SMC. Alternatively, the nonlinear relationship between the normalized proxy and the actual SMC can be learned by supervised regression. Experiments are conducted on real moist soil data. In particular, we developed datasets of moist minerals, acquired by two different sensors, an Agrispec spectrometer and an Imec snapscan shortwave infrared (SWIR) hyperspectral camera, under strictly controlled experimental settings. The proposed methodology is also validated on the available real moist soil data from the literature. Compared to state-of-the-art methods, the proposed method accurately estimates the SMC.

This paper proposes a blind nonlinear unmixing technique for intimate mixtures using the Hapke model and convolutional neural networks (HapkeCNN). We use the Hapke model and a fully convolutional encoder-decoder deep network for the nonlinear unmixing. Additionally, we propose a novel loss function that includes three terms; 1) a quadratic term based on the Hapke model, that captures the nonlinearity, 2) the reconstruction error of the reflectances, to ensure the fidelity of the reconstructed reflectance, and 3) a minimum volume total variation term that exploits the geometrical information to estimate the endmembers in the absence of pure pixels in the hyperspectral data. The proposed method is evaluated using two simulated and two real datasets. We compare the results of endmember and abundance estimation with a number of nonlinear, and projection-based linear unmixing techniques. The experimental results confirm that HapkeCNN considerably outperforms the state-of-the-art nonlinear approaches.The proposed method was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available at: https://github.com/BehnoodRasti/HapkeCNN.

In this article, we propose a minimum simplex convolutional network (MiSiCNet) for deep hyperspectral unmixing. Unlike all the deep learning-based unmixing methods proposed in the literature, the proposed convolutional encoder–decoder architecture incorporates spatial information and geometrical information of the hyperspectral data in addition to the spectral information. The spatial information is incorporated using convolutional filters and implicitly applying a prior on the abundances. The geometrical information is exploited by incorporating a minimum simplex volume penalty term in the loss function for the endmember estimation. This term is beneficial when there are no pure material pixels in the data, which is often the case in real-world applications. We generated simulated datasets, where we consider two different no-pure pixel scenarios. In the first scenario, there are no pure pixels but at least two pixels on each facet of the data simplex (i.e., mixtures of two pure materials). The second scenario is a complex case with no pure pixels and only one pixel on each facet of the data simplex. In addition, we evaluate the performance of MiSiCNet in three real datasets. The experimental results confirm the robustness of the proposed method to both noise and the absence of pure pixels. In addition, MiSiCNet considerably outperforms the state-of-the-art unmixing approaches. The results are given in terms of spectral angle distance in degree for the endmember estimation and the root mean square error in percentage for the abundance estimation. MiSiCNet was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online:https://github.com/BehnoodRasti/MiSiCNet.

In this article, we introduce a deep learning-based technique for the linear hyperspectral unmixing problem. The proposed method contains two main steps. First, the endmembers are extracted using a geometric endmember extraction method, i.e., a simplex volume maximization in the subspace of the data set. Then, the abundances are estimated using a deep image prior. The main motivation of this work is to boost the abundance estimation and make the unmixing problem robust to noise. The proposed deep image prior uses a convolutional neural network to estimate the fractional abundances, relying on the extracted endmembers and the observed hyperspectral data set. The proposed method is evaluated on simulated and three real remote sensing data for a range of SNR values (i.e., from 20 to 50 dB). The results show considerable improvements compared to state-of-the-art methods. The proposed method was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online:https://github.com/BehnoodRasti/UnDIP.

In this paper, we introduce ‘Unmixing Deep Image Prior’ (UnDIP), a deep learning-based technique for the linear hyperspectral unmixing problem. The proposed method contains two steps. First, the endmembers are extracted using a geometric endmember extraction method, i.e. a simplex volume maximization in a subspace of the dataset. Then, the abundances are estimated using a deep image prior. The proposed deep image prior uses a convolutional neural network to estimate the fractional abundances, relying on the extracted endmembers and the observed hyperspectral dataset. The results show considerable improvements compared to state-of-the-art methods.

We present an analysis of the influence of noise on the unmixing of hyperspectral data. We propose four scenarios to 1) investigate the effect of noise reduction as a preprocessing step on the performance of hyperspectral unmixing and 2) study the relation between noise and different endmembers selection strategies. Experiments are conducted on a simu-1ated and a real datasets with a wide range of signal to noise ratios (from 10 to 50 dB).

Due to the complex interaction of light with mixed materials, reflectance spectra are highly nonlinearly related to the pure material endmember spectra, making it hard to estimate the fractional abundances of the materials. Changing illumination conditions and cross-sensor situations cause spectral variability, further complicating the unmixing procedure. In this work, we propose a supervised approach to unmix mineral powder mixtures, containing endmember variability. First, the abundances are estimated by calculating the geodesic distances between the mixtures and the endmembers. It is argued and experimentally validated that the estimated geodesic abundances, although not correct, are invariant to external spectral variability. Then, a supervised approach is applied to learn a mapping from the obtained geodesic abundances to spectra that follow a linear model. To learn this mapping, groundtruth fractional abundances of a number of training samples are required. Although any nonlinear regression method can be used to learn the mapping, Gaussian process is found to be suitable when a limited number of training samples are available. The trained model is applicable to all manifolds that contain a similar nonlinear behavior as the trained manifold, e.g., when the same mixtures are measured by another sensor. Using the output spectra, a simple inversion of the linear model reveals the true abundances. Experiments are conducted on simulated and real mineral mixtures. In particular, we developed data sets of homogeneously mixed mineral powder mixtures, acquired by two different sensors, an Agrispec spectrometer and a snapscan shortwave infrared (SWIR) hyperspectral camera, under strictly controlled experimental settings. The proposed approach is compared to other supervised approaches and nonlinear mixture models.

Spectral measurements are commonly applied for the nondestructive estimation of leaf parameters, such as the concentrations of chlorophyll a and b, carotenoid, anthocyanin, brown pigment, leaf water content, and leaf mass per area for the quantification of vegetation physiology. The most popular way to estimate these parameters is by using spectral vegetation indices. The use of biochemical models allows us to use the full wavelength range (400-2500 nm) and to physically interpret the result. However, their performance is usually lower than that of supervised machine learning regression techniques. Machine learning regression techniques, on the other hand, have the disadvantage that the relationship between estimated parameters and the reflectance/transmission spectra is unclear. In this article, a hybrid between a supervised learning method and physical modeling for the estimation of leaf parameters is proposed. In this method, a machine learning regression technique is applied to learn a mapping from the true hyperspectral data set to a data set that follows the PROSPECT model. The PROSPECT model then reveals the actual leaf parameters. Two mapping methods, based on Gaussian processes (GPs) and kernel ridge regression (KRR) are proposed. As an alternative, mapping onto the leaf absorption spectra is proposed as well. The proposed methodology not only estimates the leaf parameters with a lower error but also solves the interpretation problem of the parameters estimated by the advanced machine learning regression techniques. This method is validated on the ANGERS and LOPEX data set.

As the interaction of light with the Earth surface is very complex, spectral reflectances are composed of nonlinear mixtures of the observed materials. Nonlinear mixing models have the disadvantage that not all spectra of a hyperspectral dataset necessarily follow the same particular mixing model. Moreover, most models lack a proper interpretation of the estimated parameters in terms of fractional abundances. In this paper, we present a semi-supervised nonlinear unmixing technique that overcomes these problems. In a first step, we apply a kernelized simplex volume maximization to select an overcomplete set of endmembers that precisely describe the hyperspectral data manifold. In a second step, this set is used as ground truth data in a supervised learning approach to generate fractional abundance maps from the entire dataset. For this, three methods are presented, based on kernelized sparse unmixing, feedforward neural networks, and gaussian processes. The proposed method is validated on simulated data, a dataset obtained by ray tracing, and a real hyperspectral image.

Because of the complex interaction of light with the Earth surface, a hyperspectral pixel can be composed of a highly nonlinear mixture of the reflectances of the materials on the ground. When nonlinear mixing models are applied, the estimated model parameters are usually hard to interpret and to link to the actual fractional abundances. Moreover, not all spectral reflectances in a real scene follow the same particular mixing model. In this paper, we present a supervised learning method for nonlinear spectral unmixing. In this method, a neural network is applied to learn mappings of the true spectral reflectances to the reflectances that would be obtained if the mixture was linear. A simple linear unmixing then reveals the actual abundance fractions. This technique is model-independent and allows for an easy interpretation of the obtained abundance fractions. We validate this method on several artificial datasets, a data set obtained by ray tracing, and a real dataset.