Hyperspectral unmixing, an essential and fundamental task in remote sensing, focuses on estimating endmembers (spectrally pure components) and their fractional abundances within each mixed pixel of a hyperspectral image. With the advent of deep learning (DL), the field of hyperspectral unmixing has made significant progress. Among DL approaches, autoencoder-based models have shown promising results. However, most unmixing methods estimate the endmembers by the weights of the linear layers in the decoder of their networks, making their performance highly dependent on weight initialization. Moreover, noise is not explicitly accounted for in most recent methods that use spectral angle distance (SAD) loss. To avoid the initialization problems, we developed an innovative inversion strategy to directly estimate the endmembers. Moreover, to optimally account for noise, an end-to-end network is proposed that integrates both denoising and unmixing. Finally, for an improved feature extraction, a novel spectral-spatial attention module is integrated in the network. Extensive experiments on a synthetic and three real datasets show that the proposed method significantly and consistently outperforms the compared state-of-the-art methods. The full code is available at https://github.com/xuanwentao for public evaluation.

Accurate estimation of the fractional abundances of intimately mixed materials from spectral reflectances is generally hard due to a highly nonlinear relationship between the measured spectrum and the composition of the material. Changes in the acquisition and the illumination conditions cause variability in the spectral reflectance, further complicating the spectral unmixing procedure. In this work, we propose a methodology for unmixing intimate mixtures that can tackle both nonlinearity and spectral variability. A supervised approach is proposed that characterizes the nonlinear data manifolds by high-dimensional Bézier surfaces. To deal with spectral variability, a manifold transformation procedure is designed. To generate Bézier surfaces, training samples are required that are uniformly distributed throughout the data manifold. For this, we recently generated a hyperspectral dataset of intimate mineral powder mixtures by homogeneously mixing five different clay powders (Kaolin, Roof clay, Red clay, mixed clay, and Calcium hydroxide) in laboratory settings. In total 330 samples (325 mixtures and five pure materials) were prepared. The ground fractional abundances of these mixtures uniformly cover the 5-D probability simplex. The spectral reflectances of these samples were acquired by multiple sensors with a large variation in sensor types, platforms, and acquisition conditions. Experiments are conducted both on simulated and real intimate mineral powder mixtures. Comparison with a number of unsupervised unmixing methods demonstrates the potential of the proposed approach.

Hyperspectral unmixing is a crucial technique in remote sensing data processing that aims to estimate component information from mixed pixels in hyperspectral images. Most existing deep learning-based hyperspectral unmixing models employ autoencoder (AE) networks to reconstruct hyperspectral images and estimate abundance maps. Here, the weight between the reconstructed and the softmax layers is used to extract/estimate endmember signatures. However, AEs are heavily dependent on initial weights, which introduces inherent randomness, potentially compromising unmixing accuracy. To address this issue, in this article, we present a new dual-feature fusion network (DFFN) for enhanced hyperspectral unmixing. Our DFFN mainly consists of four modules: 1) a feature fusion module (FFM); 2) an abundance estimation module (AEM); 3) an endmember estimation module (EEM); and 4) a reconstruction module (RM). First, FFM calculates spectral and spatial similarities and then enhances the hyperspectral image by matrix multiplications with similarity matrices. Second, AEM takes the enhanced hyperspectral image as input and uses convolutional layers to estimate abundances and reconstruct the image. Next, the reconstructed image is fed into EEM to automatically estimate endmembers. RE performs the final reconstruction through matrix multiplication of the estimated endmembers and abundances. Experiments on synthetic and real hyperspectral datasets, together with a comparison with state-of-the-art techniques, demonstrate the superiority of our newly proposed DFFN. The full code is released athttps://github.com/xuanwentaofor public evaluation.

Since many materials behave as heterogeneous intimate mixtures with which each photon interacts differently, the relationship between spectral reflectance and material composition is very complex. Quantitative validation of spectral unmixing algorithms requires high-quality ground truth fractional abundance data, which are very difficult to obtain.In this work, we generated a comprehensive hyperspectral dataset of intimate mineral powder mixtures by homogeneously mixing five different clay powders (Kaolin, Roof clay, Red clay, mixed clay, and Calcium hydroxide). In total 325 samples were prepared. Among the 325 samples, 60 mixtures were binary, 150 were ternary, 100 were quaternary, and 15 were quinary. For each mixture (and pure clay powder), reflectance spectra are acquired by 13 different sensors, with a broad wavelength range between the visible and the long-wavelength infrared regions (i.e., between 350 nm and 15385 nm) and with a large variation in sensor types, platforms, and acquisition conditions. We will make this dataset public, to be used by the community for the validation of nonlinear unmixing methodologies (https://github.com/VisionlabUA/Multisensor_datasets)

Current shadow-aware hyperspectral unmixing methods often suffer from noisy abundance maps and inaccurate abundance estimation of shadowed pixels, as these are characterized by low reflectance values and signal-to-noise ratio. In order to achieve a shadow-insensitive abundance estimation, in this article we propose a novel spatial-spectral shadow-aware mixing model (S3AM). The approach models shadows by considering diffuse solar illumination and secondary illumination from neighbouring pixels. Besides, spatial regularization using shadow-aware weighted Total Variation is employed. Specifically, pixels in the local neighborhood of a target pixel take simultaneously into account spectral similarity measures derived from the imagery, elevation similarity measures derived from a Digital Surface Model, and the impact of shadows. The sky view factorF, needed as input for the model, is also derived from available Digital Surface Models (DSM). The proposed approach is extensively validated and compared to state-of-the-art methods on two datasets. Results demonstrate that S3AM yields superior abundance estimation maps for real scenarios, by decreasing the noise in the results and achieving more accurate reconstructions in the presence of shadows.

Leaf chlorophyll ab content is an important indicator of vegetation physiological status and is generally obtained from spectral reflectance. For non-destructive estimation of chlorophyll ab content, physical leaf reflectance models, such as the PROSPECT model and supervised methods have been applied. While the former generally does not perform optimal, the latter only performs well when trained on similar data. In this work, we developed a robust supervised method that overcomes this problem. The method derives a proxy for chlorophyll ab content as the relative position of a leaf reflectance spectrum on the arc spanned by the two extremes, containing high and low chlorophyll ab content. This proxy is found to be unaffected by spectral variability, caused by environmental and acquisition conditions. The relation between this proxy and the actual chlorophyll ab content is obtained by a supervised regression model, that is trained on a single leaf reflectance dataset, and that is transferable to other datasets. The proposed method is validated on seven real hyperspectral datasets.

This paper proposes a sparse unmixing technique using a convolutional neural network (SUnCNN). We reformulate the sparse unmixing problem into an optimization over the parameters of a convolutional network. Relying on a spectral library, the deep network learns in an unsuper-vised manner a mapping from a fixed input to the sparse abundances. Moreover, SUnCNN fulfills the sum-to-one constraint using a softmax activation layer. We compare SUnCNN with the state-of-the-art using a simulated and a real dataset. The experimental results show that the proposed deep learning-based unmixing method outperforms the oth-ers in terms of signal to reconstruction error. Additionally, SUnCNN is visually superior to the competing techniques. SUnCNN was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online: https://github.com/BehnoodRasti/SUnCNN.

This paper proposes a deep blind hyperspectral unmixing network for datasets without pure pixels called minimum simplex convolutional network (MiSiCNet). MiSiCNet is the first deep learning-based blind unmixing method proposed in the literature which incorporates both spatial and geometrical information of the hyperspectral data, in addition to the spectral information. The proposed convolutional encoder-decoder architecture incorporates the spatial information using convolutional filters and implicitly applying a prior on the abundances. We added a minimum simplex volume penalty term to the loss function to exploit the geometrical information. We evaluate the performance of MiSiCNet on simulated and real datasets. The experimental results confirm the robustness of the proposed method to both noise and absence of pure pixels. Additionally, MiSiCNet considerably outperforms the state-of-the-art unmixing approaches. The results are given in terms of spectral angle distance in degree for the endmember estimation, and root mean square error in percentage for the abundance estimation. MiS-iCNet was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online: https://github.com/BehnoodRasti/MiSiCNet.

Hyperspectral imaging is an important technology in the field of geosciences and remote sensing.However, the highdimensional nature of hyperspectral images (HSIs) together with the limited availability of training/labeled samples challenge an efficient processing of HSIs.To alleviate these challenges, we propose a deep multi-resolution clustering network (DMC-Net) to analyze HSIs.DMC-Net, without requiring training/labeled samples for the training process, captures the non-linear intrinsic relation within data points in an HSI and analyzes the image at various resolutions by applying atrous convolutions.Furthermore, DMC-Net preserves the spectral information by directly incorporating extracted features from the original HSI into the reconstruction phase.In terms of clustering accuracy, experimental results on two real HSIs demonstrate the superior performance of DMC-Net compared to the state-of-the-art deep learning-based clustering approaches.

Transformers have intrigued the vision research community with their state-of-the-art performance in natural language processing. With their superior performance, transformers have found their way in the field of hyperspectral image classification and achieved promising results. In this article, we harness the power of transformers to conquer the task of hyperspectral unmixing and propose a novel deep neural network-based unmixing model with transformers. A transformer network captures nonlocal feature dependencies by interactions between image patches, which are not employed in CNN models, and hereby has the ability to enhance the quality of the endmember spectra and the abundance maps. The proposed model is a combination of a convolutional autoencoder and a transformer. The hyperspectral data is encoded by the convolutional encoder. The transformer captures long-range dependencies between the representations derived from the encoder. The data are reconstructed using a convolutional decoder. We applied the proposed unmixing model to three widely used unmixing datasets, i.e., Samson, Apex, and Washington DC mall and compared it with the state-of-the-art in terms of root mean squared error and spectral angle distance. The source code for the proposed model will be made publicly available at https://github.com/preetam22n/DeepTrans-HSU.

Due to the complex interaction of light with moist soils, the soil moisture content (SMC) is hard to estimate from the soil spectral reflectance. Spectral variability, caused by variations in viewing and illumination angle and between-sensor variability, further complicates the estimation. In this work, we developed a supervised methodology to accurately estimate SMC from spectral reflectance. The method determines a proxy for the SMC of moist soil, making use of the reflectance spectra of an air-dried and saturated soil sample. The proxy is made invariant to illumination and viewing angle, and sensor type. In the next step, the proxy is normalized with respect to the ground-truth SMC of the saturated soil to make the technique less dependent on the soil type. The normalized proxy can be directly used as an estimate of SMC. Alternatively, the nonlinear relationship between the normalized proxy and the actual SMC can be learned by supervised regression. Experiments are conducted on real moist soil data. In particular, we developed datasets of moist minerals, acquired by two different sensors, an Agrispec spectrometer and an Imec snapscan shortwave infrared (SWIR) hyperspectral camera, under strictly controlled experimental settings. The proposed methodology is also validated on the available real moist soil data from the literature. Compared to state-of-the-art methods, the proposed method accurately estimates the SMC.

This paper proposes a blind nonlinear unmixing technique for intimate mixtures using the Hapke model and convolutional neural networks (HapkeCNN). We use the Hapke model and a fully convolutional encoder-decoder deep network for the nonlinear unmixing. Additionally, we propose a novel loss function that includes three terms; 1) a quadratic term based on the Hapke model, that captures the nonlinearity, 2) the reconstruction error of the reflectances, to ensure the fidelity of the reconstructed reflectance, and 3) a minimum volume total variation term that exploits the geometrical information to estimate the endmembers in the absence of pure pixels in the hyperspectral data. The proposed method is evaluated using two simulated and two real datasets. We compare the results of endmember and abundance estimation with a number of nonlinear, and projection-based linear unmixing techniques. The experimental results confirm that HapkeCNN considerably outperforms the state-of-the-art nonlinear approaches.The proposed method was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available at: https://github.com/BehnoodRasti/HapkeCNN.

In this article, we propose a minimum simplex convolutional network (MiSiCNet) for deep hyperspectral unmixing. Unlike all the deep learning-based unmixing methods proposed in the literature, the proposed convolutional encoder–decoder architecture incorporates spatial information and geometrical information of the hyperspectral data in addition to the spectral information. The spatial information is incorporated using convolutional filters and implicitly applying a prior on the abundances. The geometrical information is exploited by incorporating a minimum simplex volume penalty term in the loss function for the endmember estimation. This term is beneficial when there are no pure material pixels in the data, which is often the case in real-world applications. We generated simulated datasets, where we consider two different no-pure pixel scenarios. In the first scenario, there are no pure pixels but at least two pixels on each facet of the data simplex (i.e., mixtures of two pure materials). The second scenario is a complex case with no pure pixels and only one pixel on each facet of the data simplex. In addition, we evaluate the performance of MiSiCNet in three real datasets. The experimental results confirm the robustness of the proposed method to both noise and the absence of pure pixels. In addition, MiSiCNet considerably outperforms the state-of-the-art unmixing approaches. The results are given in terms of spectral angle distance in degree for the endmember estimation and the root mean square error in percentage for the abundance estimation. MiSiCNet was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online:https://github.com/BehnoodRasti/MiSiCNet.

In this article, we introduce a deep learning-based technique for the linear hyperspectral unmixing problem. The proposed method contains two main steps. First, the endmembers are extracted using a geometric endmember extraction method, i.e., a simplex volume maximization in the subspace of the data set. Then, the abundances are estimated using a deep image prior. The main motivation of this work is to boost the abundance estimation and make the unmixing problem robust to noise. The proposed deep image prior uses a convolutional neural network to estimate the fractional abundances, relying on the extracted endmembers and the observed hyperspectral data set. The proposed method is evaluated on simulated and three real remote sensing data for a range of SNR values (i.e., from 20 to 50 dB). The results show considerable improvements compared to state-of-the-art methods. The proposed method was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online:https://github.com/BehnoodRasti/UnDIP.

In this paper, we introduce ‘Unmixing Deep Image Prior’ (UnDIP), a deep learning-based technique for the linear hyperspectral unmixing problem. The proposed method contains two steps. First, the endmembers are extracted using a geometric endmember extraction method, i.e. a simplex volume maximization in a subspace of the dataset. Then, the abundances are estimated using a deep image prior. The proposed deep image prior uses a convolutional neural network to estimate the fractional abundances, relying on the extracted endmembers and the observed hyperspectral dataset. The results show considerable improvements compared to state-of-the-art methods.

We present an analysis of the influence of noise on the unmixing of hyperspectral data. We propose four scenarios to 1) investigate the effect of noise reduction as a preprocessing step on the performance of hyperspectral unmixing and 2) study the relation between noise and different endmembers selection strategies. Experiments are conducted on a simu-1ated and a real datasets with a wide range of signal to noise ratios (from 10 to 50 dB).

Remote sensing data is contaminated with different types of noise that can severely affect the analysis of this data. Generally, in modern treatment chains of satellite and aerial data, denoising techniques are applied to atmospherically corrected images prior to further analysis (e.g., classification). However, since the noise contaminates the measured radiance at the sensor, it can influence the atmospheric correction in itself and consequently the remaining of the processing chain. In this paper, we compare the performance of a denoising technique, when applied before or after atmospheric correction. Our observations challenge the current de facto paradigm of denoising in a processing chain of spaceborne and airborne remotely sensed images.

Random Gaussian noise and striping artifacts are common phenomena in hyperspectral images (HSI). In this article, an effective restoration method is proposed to simultaneously remove Gaussian noise and stripes by merging a denoising and a destriping submodel. A denoising submodel performs a multiband denoising, i.e., Gaussian noise removal, considering Gaussian noise variations between different bands, to restore the striped HSI from the corrupted image, in which the striped HSI is constrained by a weighted nuclear norm. For the destriping submodel, we propose an adaptive anisotropy total variation method to adaptively smoothen the striped HSI, and we apply, for the first time, the truncated nuclear norm to constrain the rank of the stripes to 1. After merging the above two submodels, an ultimate image restoration model is obtained for both denoising and destriping. To solve the obtained optimization problem, the alternating direction method of multipliers (ADMM) is carefully schemed to perform an alternative and mutually constrained execution of denoising and destriping. Experiments on both synthetic and real data demonstrate the effectiveness and superiority of the proposed approach.

Due to the complex interaction of light with mixed materials, reflectance spectra are highly nonlinearly related to the pure material endmember spectra, making it hard to estimate the fractional abundances of the materials. Changing illumination conditions and cross-sensor situations cause spectral variability, further complicating the unmixing procedure. In this work, we propose a supervised approach to unmix mineral powder mixtures, containing endmember variability. First, the abundances are estimated by calculating the geodesic distances between the mixtures and the endmembers. It is argued and experimentally validated that the estimated geodesic abundances, although not correct, are invariant to external spectral variability. Then, a supervised approach is applied to learn a mapping from the obtained geodesic abundances to spectra that follow a linear model. To learn this mapping, groundtruth fractional abundances of a number of training samples are required. Although any nonlinear regression method can be used to learn the mapping, Gaussian process is found to be suitable when a limited number of training samples are available. The trained model is applicable to all manifolds that contain a similar nonlinear behavior as the trained manifold, e.g., when the same mixtures are measured by another sensor. Using the output spectra, a simple inversion of the linear model reveals the true abundances. Experiments are conducted on simulated and real mineral mixtures. In particular, we developed data sets of homogeneously mixed mineral powder mixtures, acquired by two different sensors, an Agrispec spectrometer and a snapscan shortwave infrared (SWIR) hyperspectral camera, under strictly controlled experimental settings. The proposed approach is compared to other supervised approaches and nonlinear mixture models.

Spectral measurements are commonly applied for the nondestructive estimation of leaf parameters, such as the concentrations of chlorophyll a and b, carotenoid, anthocyanin, brown pigment, leaf water content, and leaf mass per area for the quantification of vegetation physiology. The most popular way to estimate these parameters is by using spectral vegetation indices. The use of biochemical models allows us to use the full wavelength range (400-2500 nm) and to physically interpret the result. However, their performance is usually lower than that of supervised machine learning regression techniques. Machine learning regression techniques, on the other hand, have the disadvantage that the relationship between estimated parameters and the reflectance/transmission spectra is unclear. In this article, a hybrid between a supervised learning method and physical modeling for the estimation of leaf parameters is proposed. In this method, a machine learning regression technique is applied to learn a mapping from the true hyperspectral data set to a data set that follows the PROSPECT model. The PROSPECT model then reveals the actual leaf parameters. Two mapping methods, based on Gaussian processes (GPs) and kernel ridge regression (KRR) are proposed. As an alternative, mapping onto the leaf absorption spectra is proposed as well. The proposed methodology not only estimates the leaf parameters with a lower error but also solves the interpretation problem of the parameters estimated by the advanced machine learning regression techniques. This method is validated on the ANGERS and LOPEX data set.

In this work, we present a nonlinear spectral mixing model that, apart from the fractional abundances, contains two additional parameters, one accounting for multiple reflections and another accounting for shadow. The model is based on the multilinear mixing (MLM) model that we have proposed earlier. An analysis of the parameter values is performed on a close-range hyperspectral image of a building facade. The model is compared to the linear model and two models that account for only one of the two effects: the linear model with an extra shadow endmember, and the MLM model.

As the interaction of light with the Earth surface is very complex, spectral reflectances are composed of nonlinear mixtures of the observed materials. Nonlinear mixing models have the disadvantage that not all spectra of a hyperspectral dataset necessarily follow the same particular mixing model. Moreover, most models lack a proper interpretation of the estimated parameters in terms of fractional abundances. In this paper, we present a semi-supervised nonlinear unmixing technique that overcomes these problems. In a first step, we apply a kernelized simplex volume maximization to select an overcomplete set of endmembers that precisely describe the hyperspectral data manifold. In a second step, this set is used as ground truth data in a supervised learning approach to generate fractional abundance maps from the entire dataset. For this, three methods are presented, based on kernelized sparse unmixing, feedforward neural networks, and gaussian processes. The proposed method is validated on simulated data, a dataset obtained by ray tracing, and a real hyperspectral image.

In optical remote sensing, phenomena such as multiple scattering, shadowing, and spatial neighbor effects generate spectral reflectances that are nonlinear mixtures of the reflectances of the surface materials. Using hyperspectral images, the obtained spectral reflectances can be unmixed. We present a general method for creating nonlinear mixing models, based on a ray-based approximation of light and a graph-based description of the optical interactions. This results in a stochastic process which can be used to calculate path probabilities and contributions, and their weighted sum. In many cases, a closed-form equation can be obtained. We illustrate the approach by deriving several existing mixing models, such as linear, bilinear, and multilinear mixing (MLM) models popular in remote sensing, layered models for vegetation canopies, and intimate mineral mixtures. Furthermore, we use the proposed technique to derive a new mixing model, which extends the MLM model with shadowing. Experiments on artificial and real data show the positive traits of this model, which also demonstrates the power of the graphical model approach.

The high dimensionality of hyperspectral images, the limited availability of ground-truth data as well as the low spatial resolution (causing pixels to contain mixtures of materials) hinder hyperspectral image classification. In this work we propose a novel hyperspectral classification method where we combine the outcome of spectral unmixing with the outcome of a supervised classifier. In particular, we consider fractional abundances obtained from a Sparse Unmixing method along with posterior probabilities acquired from a Multinomial Logistic Regression classifier. Both sources of information are fused using a Markov Random Field framework. We conducted experiments on publicly available real hyperspectral images: Indian Pines and University of Pavia using a very limited number of training samples. Our results indicate that the proposed decision fusion approach significantly improves the classification result over using the individual sources and outperforms the state of the art methods.

We present a new method for spectral unmixing which takes both nonlinear mixing and spectral variability into account. This is accomplished by combining the multiple endmember spectral mixture analysis (MESMA) approach with the recently developed multilinear mixing model (MLM). As the traditional approach of nonlinear unmixing of all combinations is very time consuming, we investigate a second approach, where endmember model selection is linearly performed by a fast alternative for MESMA, followed by nonlinear unmixing. We show that this approach yields similar reconstruction errors (REs) as the full combinatorial approach, and hence results in a relatively fast method for nonlinear unmixing with variability.

Because of the complex interaction of light with the Earth surface, a hyperspectral pixel can be composed of a highly nonlinear mixture of the reflectances of the materials on the ground. When nonlinear mixing models are applied, the estimated model parameters are usually hard to interpret and to link to the actual fractional abundances. Moreover, not all spectral reflectances in a real scene follow the same particular mixing model. In this paper, we present a supervised learning method for nonlinear spectral unmixing. In this method, a neural network is applied to learn mappings of the true spectral reflectances to the reflectances that would be obtained if the mixture was linear. A simple linear unmixing then reveals the actual abundance fractions. This technique is model-independent and allows for an easy interpretation of the obtained abundance fractions. We validate this method on several artificial datasets, a data set obtained by ray tracing, and a real dataset.

In this paper, a new method for spatial resolution enhancement of hyperspectral images (HSI), based on the non-subsampled shearlet transform (NSST) is introduced. The proposed method integrates a high spectral resolution HSI with a high spatial resolution multispectral image (MSI) of the same scene. First, the HSI is spatially upsampled by means of a bicubic interpolation. Second, a 2D NSST is applied to each spectral band of the upsampled HSI and the MSI respectively. Third, the spectral coverage regions of HSI and MSI are matched and the detail shearlet coefficients of the HSI bands are replaced by detail shearlet information of the MSI, based on the spectral matching of both sensors. The proposed method is applied to real datasets and compared with some state-of-the-art fusion algorithms. The obtained results show that the proposed method significantly increases the spatial resolution while preserving the spectral content of the HSI.

In this paper, a new fusion method for merging the spectral and spatial contents of hyperspectral images (HSI) with the height information of light detection and ranging (LiDAR) for increasing the classification accuracy of HSI is introduced. First, 2D non-subsampled shearlet transform (NSST) is applied to each band of hyperspectral and LiDAR data separately in order to extract the spatial features. Second, principal component analysis (PCA) is applied to all shearlet subbands of HSI in order to reduce their dimension. Third, the spectral information of HSI and obtained spatial features are integrated and classified using subspace multinomial logistic regression (MLRsub). We evaluate the performance of the proposed method over University of Houston, USA and a rural one captured over Trento, Italy. The obtained results show that the proposed method can efficiently classify the joint hyperspectral and LiDAR images.

Several classes of endmember (EM) extraction algorithms based on the pure pixel assumption exist. Most of these algorithms employ some geometrical interpretation of the spectral mixing process, and use orthogonal projections, random projections, or some combination of them. Random projection based algorithms, such as pixel purity index, often find clusters of EM candidates which show high correlation, requiring a manual post-processing. Pure orthogonal projection based methods such as the simplex growing algorithm always yield a single, identical set of EMs, as the iteration process is fully deterministic. Mixed methods, such as VCA, can be highly random, and produce different sets of EMs each run. In this work, we present a new EM extraction algorithm which combines the positive aspects of orthogonal projection and random projection-based methods, resulting in a method which does not require manual intervention, possesses much less randomness than VCA, and is more flexible than fixed iterative methods. These properties are illustrated on a real data set, and compared with several different types of popular EM extraction algorithms.

Estimation of the number of endmembers (NOE) is an important first step in many hyperspectral unmixing applications. We present a new method for solving this problem, based on the statistics of the indegree distribution (IDD) of the data nearest neighbor graph. It is known that this IDD shows a high dependence on the intrinsic dimensionality (ID) of the data, and becomes skewed for increasing dimensionality. This effect is known as the hubness phenomenon, and we propose a technique that exploits this effect to derive an estimate for the NOE in a hyperspectral data set. While this number should have a trivial relation with the ID of the data set, this relation is often obscured by the large correlations that exist between endmember spectra and adjacent spectral bands. The proposed technique circumvents this problem by building representative statistics based on simulated hyperspectral data sets, and therefore performs much better than alternative techniques. Also several types of nonlinearly mixed data sets can be treated by the proposed technique, which is illustrated with bilinear data sets.

Hyperspectral images are high dimensional while the available number of training samples can be very low. For very small training sizes, classical supervised classification strategies may fail. In this work we propose an alternative, semi-supervised approach which is based on sparse unmixing. In this method, all training samples are gathered in a dictionary and serve as possible endmembers. Unmixing then reveals the relative contributions of the different training samples to an unlabeled sample. Since standard unmixing strategies as the Fully Constrained Linear Spectral Unmixing (FCLSU) typically assume only one endmember per class, we investigate the use of sparse unmixing. In this work, we apply the SunSAL algorithm. We show that this method outperforms SVM classification in the case of extremely small training sizes of only a few samples per class.

Several techniques exist for dealing with spectral variability in hyperspectral unmixing, such as multiple endmember spectral mixture analysis (MESMA) or compositional models. These algorithms are computationally very involved, and often cannot be executed on problems of reasonable size. In this work, we present a new algorithm for solving the unmixing problem when spectral variability is present. The algorithm uses a library-based approach to describe the variability present in each class, and executes an alternating optimization with respect to these libraries. The optimization problem itself is constructed as an angle minimization problem by exploiting the geometrical interpretation of the unmixing problem. This results in an algorithm which yields almost identical results as MESMA, but is computationally much more favorable.

Recent advances in airborne light detection and ranging (LiDAR) technology allow us to rapid measure the topographical information over large areas. LiDAR remote sensed data has been widely used in many applications, e.g. forest management, urban planning, disaster predictions, etc. However, extracting useful information from LiDAR data remains challenging, especially in the urban remote sensing, where many objects have the same elevation and are connected, such as road and parking lots, trees and buildings. In this work, we present a new method to extract geometric and textural information from LiDAR data by using attribute filters with partial reconstruction. The proposed method can separate the connected objects and better model the geometric and textural information than traditional connected filters (e.g. attribute filters). Experimental results on LiDAR data from the 2013 IEEE GRSS Data Fusion Contest demonstrate effectiveness of the proposed method. Compared to the methods using original LiDAR data or attribute profiles computed by traditional attribute filters, with the proposed method, overall classification accuracies were improved by 35% and 12%, respectively.

Unlike multispectral (MSI) and panchromatic (PAN) images, the spatial resolution of hyperspectral images (HSI) is limited. In many applications, HSI with a high spectral as well as spatial resolution are required. In this paper, a new method for superresolution of HSI is introduced. A high resolution HSI (HRHSI) is constructed by using the linear spectral unmixing model and making use of a MSI of the same scene. As a regularization, we apply a sparse representation of the HSI, obtained by constructing a dictionary of unrelated PAN images. Experiments show that the reconstruction quality is higher than some the well-known HSI spatial resolution enhancement methods.

In hyperspectral unmixing, bilinear and linear-quadratic models have become popular recently, and also the polynomial postnonlinear model shows promising results. These models do not consider endmember interactions involving more than two endmembers, although such interactions might compose a nontrivial part of the observed spectrum in scenarios involving bright materials and complex geometrical structures, such as vegetation and intimate mixtures. In this paper, we present an extension of these models to include an infinite number of interactions. Several technical problems, such as divergence of the resulting series, can be avoided by introducing an optical interaction probability, which becomes the only free parameter of the model in addition to the abundances. We present an unmixing strategy based on this multilinear mixing (MLM) model; present comparisons with the bilinear models and the Hapke model for intimate mixing; and show that, in several scenarios, the MLM model obtains superior results.

In hyperspectral unmixing applications, one typically assumes that a single spectrum exists for every endmember. In many scenarios, this is not the case, and one requires a set or a distribution of spectra to represent an endmember or class. This inherent spectral variability can pose severe difficulties in classical unmixing approaches. In this paper, we present a new algorithm for dealing with endmember variability in spectral unmixing, based on the geometrical interpretation of the resulting unmixing problem, and an alternating optimization approach. This alternating-angle-minimization algorithm uses sets of spectra to represent the variability present in each class and attempts to identify the subset of endmembers which produce the smallest reconstruction error. The algorithm is analogous to the popular multiple endmember spectral mixture analysis technique but has a much more favorable computational complexity while producing similar results. We illustrate the algorithm on several artificial and real data sets and compare with several other recent techniques for dealing with endmember variability.

In this paper, we present a new lossy compression method for hyperspectral images that aims to optimally compress in both spatial and spectral domains and simultaneously minimizes the effect of the compression on linear spectral unmixing performance. To achieve this, a nonnegative Tucker decomposition is applied. This decomposition is a function of three dimension parameters. By employing a link between this decomposition and the linear spectral mixing model, an optimization problem is defined to find the optimal parameters by minimizing the root-mean-square error between the abundance matrices of the original and reconstructed data sets. The resulting optimization problem is solved by a particle swarm optimization algorithm. An approximate method for fast estimation of the free parameters is introduced as well. Our simulation results show that, in comparison with well-known state-of-the-art lossy compression methods, an improved compression and spectral unmixing performance of the reconstructed hyperspectral image is obtained. It is noteworthy to mention that the superiority of our method becomes more apparent as the compression ratio grows.

In this paper, a new method is presented to solve the spectral unmixing problem. The method is based on the projection on convex sets principle, in which a simplex is considered as an intersection of a plane and half-spaces, and the abundances are obtained by alternatively projecting data onto the half-spaces using the well-known Dykstra algorithm. In this paper, every step of such a recently developed alternating projection unmixing algorithm is rephrased using distance geometry, i.e. using only the spectral distances between the data points and the endmembers. This distance geometric approach allows to use any distance metric other than the Euclidean one. The experimental validation shows that the method provides exact results for the fully constrained unmixing problem. Moreover, we demonstrate the usefulness of the method for nonlinear unmixing, using geodesic distances on the data manifold.

We present a fast alternative for the popular pixel purity index (PPI) algorithm. This multi-dimensional PPI (MDPPI) algorithm is based on iteratively identifying convex hull indices in low-dimensional random projections. The MDPPI algorithm can yield identical results as the PPI algorithm, but several orders of magnitude faster. Furthermore, we show that the PPI algorithm is simply the MDPPI algorithm used in one dimension. In this work, we focus mainly on an efficient computational implementation of the algorithm. An ENVI implementation which can be used as a drop-in replacement of the standard PPI algorithm will be made available, along with a Matlab implementation.

In this paper, we present a new hyperspectral image lossy compression method that aims to optimally compress in both spatial and spectral domains and simultaneously considers linear spectral unmixing as a target. To achieve this, a non-negative tucker decomposition is applied. This algorithm has three flexible dimension parameters. We propose an approach that, for any desired compression ratio (CR), chooses the optimal parameters by minimizing the root mean square error (RMSE) between the abundance matrices of the original and compressed datasets using fully constrained least square spectral unmixing. The resulting optimization problem is solved by a Particle Swarm Optimization algorithm. Our simulation results show that the proposed method, in comparison with well-known lossy compression methods such as 3D-SPECK and combined PCA+JPEG2000 algorithms, provides a lower RMSE and higher signal to noise ratio (SNR) for any given CR. It is noteworthy to mention that the superiority of our method becomes more apparent as the value of CR grows.

In this letter, a new geometric matched filter (MF) is proposed by combining the standard MF with concepts of convex geometry. The purpose of the method is twofold: for subpixel target detection and for partial unmixing of a hyperspectral image. In standard matched filtering, the filter is designed based on the background statistics of the entire image, which works fine for rare targets but fails when the target is frequently present throughout the whole image. In the presented method, the background is restricted to pixels that have a zero contribution to the target spectrum. These background pixels are identified based on the simplex formed by the target and other relevant endmembers of the data set. Experiments are conducted for the specific case of targets which are frequently present in an image. The presented method is shown to outperform standard matched filtering and orthogonal subspace projection for target detection, and for the estimation of the target abundances.

One of the major issues with spectral mixture analysis remains the lack of ability to properly account for the spectral variability of endmembers (EMs). EM variability is most often addressed using large spectral libraries incorporating the variability present in the image. We propose a new geometric-based methodology to efficiently evaluate different binary EM combinations. Our approach selects the best EM combination prior to unmixing, building upon the equivalence between the reconstruction error in least squares unmixing and spectral angle minimization in geometric unmixing. This geometric approach is tested on both a simulated data set based on field measurements and a HyMap image. It is demonstrated that selecting the best EM combination for a pixel based on the angle minimization provided identical results compared with using the projection distance or reconstruction error. It also has the additional benefit of reducing the computation time due to the simplicity of the angle calculations.

Hyperspectral images (HSIs) can be very noisy, and the amount of noise may differ from band to band. While some spectral bands may be dominated by low signal-independent noise levels, others have mixed noise levels, which may include high levels of Gaussian, Poisson, and Spike noises. When a denoising algorithm is globally applied to the whole data set, it usually affects the low-noise bands adversely. Therefore, it is better to use different criteria for denoising different bands. In this paper, we propose a new denoising strategy to do so. The method is based on a 2-D nonsubsampled shearlet transform, applied to each spectral band of the HSI. We propose an effective method to distinguish between bands with low levels of Gaussian noise (LGN bands) and bands with mixed noise (MN bands) based on spectral correlation. LGN bands are denoised using a thresholding technique on the shearlet coefficients. On the MN bands, a local noise reduction method is applied, in which the detail shearlet coefficients of adjacent LGN bands are employed. This targeted approach is prone to reduce spectral distortions during denoising compared with global denoising methods. This advantage is shown in experiments where the proposed method is compared with state-of-the-art denoising methods on synthetic and real hyperspectral data sets. To assess the effect of denoising, classification and spectral unmixing tasks are applied to the denoised data. Obtained results show the superiority of the proposed approach.

The inversion problem in hyperspectral unmixing involves solving a constrained least-squares problem. Several solutions have been proposed, often based on convex optimization techniques, such as alternating optimization strategies, projection onto convex sets, augmenting positively constrained optimization algorithms, or quadratic programming. One of the most popular techniques, fully-constrained least-squares unmixing, is based on extending the Lawson-Hanson non-negatively constrained least-squares algorithm with an extra weighted term that takes the sum-to-one constraint into account. In this paper, we present an alternative active-set algorithm, inspired by the Lawson-Hanson algorithm, which solves the unmixing problem exactly, and does not require any weighting parameters. The resulting algorithm always finds the correct solution, and works an order of magnitude faster than the fully-constrained least-squares algorithm.

Automated behaviour recognition is a challenging problem and it has recently gained momentum in biological behaviour studies. This paper describes a framework for tracking and automatical classification of the behaviour of multiple freely interacting Drosophila Melanogaster (fruit flies) in a low resolution video. The movements of interacting flies are recorded by Fly world, a dedicated imaging platform. Each individual fly is identified in every frame and tracked over the complete video without losing its identity. The orientation of the flies is tracked as well, by defining their head and tail positions. From the obtained tracks, temporal features for every pair of fly are derived, allowing quantitative analysis of the fly behaviour. In order to derive information of the fly social activity, we concentrate on 2 specific behaviours: 'sniffing' and 'chasing'. Experimental results show that the classifier is able to classify the correct behaviour with an average overall accuracy of 95.46%.

Due to technological constraints, hyperspectral earth observation imagery are often a mosaic of overlapping flight lines collected in different passes over the area of interest. This causes variations in aqcuisition conditions such that the reflected spectrum can vary significantly between these flight lines. Partly, this problem is solved by atmospherical correction, but residual spectral differences often remain. A probabilistic domain adaptation framework based on graph matching using Hidden Markov Random Fields was recently proposed for transforming hyperspectral data from one image to better correspond to the other. This paper investigates the use of scale and angle invariant textural features for improving the performance of the used Hidden Markov Random Field matching framework in the case of hyperspectral flight lines. These textural features are derived from the filtering of VHR optical imagery with a bank of Gabor filters with varying orientation, scale and frequency and subsequently rendering them invariant to scale and frequency by applying the 2D DFT on the filter responses in the scale and frequency space.

In this paper, we propose a method to match multitemporal sequences of hyperspectral images using Hidden Markov Random Fields. Based on the matching of the data manifold, the algorithm matches the reflectance spectra of the classes, thus allowing the reuse of labeled examples acquired on one image to classify the other. This allows valorization of spectra collected in situ to other acquisitions than the one they were acquired for, without user supervision, prior knowledge of the class reflectance in the new domain or global information about atmospheric conditions.

An important problem in hyperspectral unmixing is solving the inversion problem, which determines the abundances of each endmember in each pixel, taking the constraints on these abundances into account. In this paper, we present a new geometrical method for solving this inversion problem, based on the equivalence with the simplex projection problem, and projection onto convex sets. By writing the simplex as an intersection of a plane and convex halfspaces, an alternating projection algorithm is constructed based on the Dykstra algorithm. We show that the resulting algorithm can be used to successfully solve the spectral unmixing problem, and yields results that are comparable to those obtained with state-of-the-art methods. The runtime required is very competitive, and the very simple nature of the algorithm allows for highly efficient implementations.

One of the earliest endmember extraction algorithms employed in hyperspectral image processing is the pixel purity index (PPI) algorithm. This algorithm is still popular today but suffers from several drawbacks, such as a large computational cost. Many recent papers focus on improving the speed of the PPI algorithm with high-performance computing or combinatorial methods. In this paper, we present a computationally efficient way of calculating the PPI scores, based on the geometrical interpretation of the PPI sampling process. We first demonstrate the equivalence with Monte Carlo sampling of the polar cones of the convex hull of the data set. Next, we introduce a more efficient sampling method, where we use higher dimensional subspaces to sample these polar cones instead of 1-D skewers. The resulting algorithm can be used to quickly estimate the most important convex hull vertices of the data set, determine the corresponding PPI scores, and produce a list of endmember candidates. An unweighted version of this algorithm is introduced as well, which is simpler to implement, has a higher computational performance, and yields similar endmembers. If the subspace dimension is chosen to be one, both algorithms reduce to the PPI algorithm. We demonstrate the properties of these algorithms, such as convergence speed and accuracy, on artificial and real hyperspectral data and show that the results correspond to those obtained with PPI. The proposed algorithms, however, are up to three orders of magnitude faster and can generate representative PPI scores in less than a second on real hyperspectral data sets.

We propose a novel semisupervised local discriminant analysis method for feature extraction in hyperspectral remote sensing imagery, with improved performance in both ill-posed and poor-posed conditions. The proposed method combines unsupervised methods (local linear feature extraction methods and supervised method (linear discriminant analysis) in a novel framework without any free parameters. The underlying idea is to design an optimal projection matrix, which preserves the local neighborhood information inferred from unlabeled samples, while simultaneously maximizing the class discrimination of the data inferred from the labeled samples. Experimental results on four real hyperspectral images demonstrate that the proposed method compares favorably with conventional feature extraction methods.

The adjacency effect is a well-known phenomenon of creating path interferences between the reflectances from different ground-cover materials. The effect is caused by atmospheric scattering, hence a typical approach to its detection has been the modeling of radiation transfer and spectral correspondence at particular wavelengths. In this paper, we investigate the detection of adjacency effect as being a general unmixing problem. This means that we opt to use unmixing to separate the true signature of a pixel from the background scatter reflected from its large neighborhood. Here, we concentrate on the prevalent linear mixing, and compare this with a specialized approach for detecting the adjacency effect in turbid waters surrounded by vegetation.

We present a new method for estimating the number of end-members present in a hyperspectral data set, based on the scaling behavior of nearest-neighbor distances. We demonstrate the method on artificial data, and show that it has a low dependence on the spectral dimensionality or the size of the data set. Furthermore, the proposed technique gives consistent results over different random instances of the data, indicated by a low standard deviation. On the AVIRIS Cuprite and Indian Pines data set, this technique yields results that are comparable to those obtained via other methods.

In this article, an automatized procedure for selecting informative values of the thresholds, essential for the construction of morphological attribute profiles, is proposed. To this end, connected component analysis is performed on a preliminary supervised or unsupervised classification result that does not involve contextual information. Subsequently, after extracting the relevant attributes from each of the connected components, the threshold values are found by grouping the attribute vectors using a clustering algorithm. In our experiments, we demonstrate the effect of image scaling on the selected thresholds. In addition, we show the advantage of using our automatic threshold selection approach with respect to manual selection, by both monitoring redundancy and performing a classification experiment.

Recently, several nonlinear techniques have been proposed in hyperspectral image processing for classification and unmixing applications. A popular data-driven approach for treating nonlinear problems employs the geodesic distances on the data manifold as property of interest. These geodesic distances are approximated by the shortest path distances in a nearest neighbor graph constructed in the data cloud. Although this approach often works well in practical applications, the graph-based approximation of these geodesic distances often fails to capture correctly the true nonlinear structure of the manifold, causing deviations in the subsequent algorithms. On the other hand, several model-based nonlinear techniques have been introduced as well and have the advantage that one can, in theory, calculate the geodesic distances analytically. In this letter, we demonstrate how one can calculate the true geodesics, and their lengths, on any manifold induced by a nonlinear hyperspectral mixing model. We introduce the required techniques from differential geometry, show how the constraints on the abundances can be integrated in these techniques, and present a numerical method for finding a solution of the geodesic equations. We demonstrate this technique on the recently developed generalized bilinear model, which is a flexible model for the nonlinearities introduced by secondary reflections. As an application of the technique, we demonstrate that multidimensional scaling applied to these geodesic distances can be used as a preprocessing step to linear unmixing, yielding better unmixing results on nonlinear data when compared to principal component analysis and outperforming ISOMAP.

In this paper, a Bayesian restoration technique for multiple observations of hyperspectral (HS) images is presented. As a prototype problem, we assume that a low-spatial-resolution HS observation and a high-spatial-resolution multispectral (MS) observation of the same scene are available. The proposed approach applies a restoration on the HS image and a joint fusion with the MS image, accounting for the joint statistics with the MS image. The restoration is based on an expectation-maximization algorithm, which applies a deblurring step and a denoising step iteratively. The Bayesian framework allows to include spatial information from the MS image. To keep the calculation feasible, a practical implementation scheme is presented. The proposed approach is validated by simulation experiments for general HS image restoration and for the specific case of pansharpening. The experimental results of the proposed approach are compared with pure fusion and deconvolution results for performance evaluation.

A prevalent methodology for extracting pure pixels from hyperspectral images has been the use of linear-mixture geometry, which dictates that pure components must reside at the corners of a simplex enclosing all the remaining points (the mixtures). Recently, adaptations to popular algorithms for estimating the largest simplex (e.g. N-findr) have been proposed, aimed to reduce their number of iterations and so shorten the execution time. This paper goes a step further, by proposing to perform the simplex maximization in a streaming fashion, that is, by evaluating one pixel at a time without using large buffers or subsequent pixels. This is achieved by reformulating the simplex measurement in terms of distance-based geometry. Besides, a new streaming simplex-growing initialization procedure is proposed. Tested on several natural scenes, the proposed algorithm is found to yield results comparable to those produced by the reference methods.

In hyperspectral unmixing, one often observes that the inter actions between the endmember spectra can contain strong non-linear effects. Recently, a new endmember extraction algorithm has been proposed that is capable of dealing with a non-linearly shaped data manifold, based upon a combination of geodesic distances and a volume-maximizing search algorithm. Once the endmembers have been found, the pixels have to be decomposed into their abundances, which within the lin ear mixing assumption becomes a constrained least-squares problem. These techniques are however not fit for dealing with non-linearly mixed data. In this work, we present an algorithm that is capable of unmixing non-linearly mixed data, and which obeys the positivity and sum-to-one constraint usually imposed on the abundance vectors. The algorithm is based upon a reformulation of the recently developed SPU algorithm in terms of distance geometry. A demonstration of the algorithm on the Cuprite data set is provided.

In the remote sensing domain data from many different sources are often available. Each of these data sources are characterized by their own sensor- and platform-specific properties, i.e. spectral range, or spatial and spectral resolution. In this paper we consider a low spatial, but high spectral resolution satellite image, together with its high spatial resolution RGB color image, e.g. obtained by UAV. Spatial features are extracted from the color image by combining the three color bands R, G and B, ordering these color vectors, and presenting color mathematical morphological profiles accordingly. This way the spatial information contained in the correlation between the different bands is completely taken into account and thus also totally preserved in the feature extraction. In a classification experiment these color morphological profiles are combined with the spectral features of the hyperspectral image, and we show that the spatial characterization of the color image is improved.

We present an enhanced visualization algorithm for hyperspectral images (HSIs). The visualization is based on the projection onto color matching functions of the human vision system. A contrast enhancement procedure is introduced by the fusion of the gradient information of the individual HSI bands. Both visualization and enhancement are combined into a multiresolution framework using wavelets. The HSI is transformed into a specific representation (HSI wavelet representation), in which the enhancement is performed at the level of the wavelet detail subbands, whereas the visualization is performed at the level of the low-resolution subbands. Specific objective quality measures are applied to demonstrate that the proposed procedure provides visualization results with a high contrast. Results are compared with state-of-the-art HSI visualization techniques and with the postprocessing enhancement.

We present a new algorithm for linear spectral mixture analysis, which is capable of supervised unmixing of hyperspectral data while respecting the constraints on the abundance coefficients. This simplex-projection unmixing algorithm is based upon the equivalence of the fully constrained least squares problem and the problem of projecting a point onto a simplex. We introduce several geometrical properties of high-dimensional simplices and combine them to yield a recursive algorithm for solving the simplex-projection problem. A concrete implementation of the algorithm for large data sets is provided, and the algorithm is benchmarked against well-known fully constrained least squares unmixing (FCLSU) techniques, on both artificial data sets and real hyperspectral data collected over the Cuprite mining region. Unlike previous algorithms for FCLSU, the presented algorithm possesses no optimization steps and is completely analytical, severely reducing the required processing power.

Many high-dimensional datasets can be mapped onto lower-dimensional linear simplexes, parametrized by barycentric coordinates. We present an unsupervised algorithm that is able to find the barycentric coordinates and corresponding vertices of such a high-dimensional dataset, by combining manifold learning with a distance geometry based algorithm for finding a maximal volume inscribed simplex. The performance of the algorithm is demonstrated on a Swiss-roll dataset that is restricted to a simplex, and on the spectral unmixing of hyperspectral imagery.

In this paper, we present an unmixing algorithm that is capable to determine endmembers and their abundances in hyperspectral imagery under non-linear mixing assumptions. The algorithm is an based upon the popular N-findR method, but uses distances between points in spectral space instead of the spectral values. These distances are defined as shortest-path distances in a nearest-neighbor graph, hereby respecting the non-trivial geometry of the data manifold in the case of nonlinearly mixed pixels. This allows the algorithm to be applied under non-linear mixing conditions. A demonstration on artificial data is given.

An enhanced visualization algorithm for hyperspectral images (HSI) is presented in this paper. The visualization is based on the projection onto color matching functions of the human vision system. A contrast enhancement procedure is introduced making use of multiband gradient information. Both visualization and enhancement are combined into a multiresolution framework using wavelets. The HSI is transformed into a specific representation (Multiscale Fundamental Form representation), in which the enhancement is performed at the level of the wavelet detail subbands, while the visualization is performed at the level of the low resolution subbands. The results are compared to state of art hyperspectral visualization algorithms.

This article presents a method for acquiring habitat maps, intended for monitoring and evaluating the conservation status of heathland vegetation, starting from thematic land cover maps. The procedure is a modified kernel-based reclassification technique, that fits into a complete habitat quality assessment framework. Part one of the procedure shifts a small square kernel over the land cover map and assigns a habitat type to each position that complies with a single set of expert rules, related to the land cover composition in that position. Part two fills the gaps, by assigning a habitat type to any of the map positions that don't conform to any of the rules, or to more than one set of rules, by using a distance measure. The technique is tested on real data from a heathland site and shows some promising results.

We study the retrieval of coloured textures from a database. In a statistical framework we model the heavy-tailed wavelet histograms through a generalized Gaussian distribution (GGD). We choose the Kullback-Leibler divergence (KLD) as a similarity measure and we obtain a closed-form expression for the KLD between two zero-mean bivariate GGDs. This allows us to take into account the rich correlation structure between the colour bands two by two. We show that this results in a considerably improved retrieval rate and, in addition, we demonstrate the superior performance of the bivariate GGD, in comparison with the bivariate Gaussian.

In this paper, a restoration technique for hyperspectral images is presented. The technique requires a low spatial resolution hyperspectral image and a high spatial resolution multispectral image of the same scene. The proposed approach applies a restoration on the hyperspectral image, while accounting for the joint statistics with the multispectral image. The restoration is based on an Expectation-Maximization algorithm, which applies a deconvolution step and a denoising step iteratively. A practical implementation scheme is presented. Simulation experiments are conducted for performance evaluation.

In this paper, we propose a technique to incorporate spatial features in the classification of hyperspectral data by means of a prior segmentation of the dataset. The key idea of the technique is that each pixel is not classified individually, but that the regions obtained from the prior segmentation are classified as a whole. The proposed technique is validated on a hyperspectral dataset of a heathland area in Belgium. Experimental results show that we can achieve larger and spatially smoothed regions, while the overall classification success rate is comparable to the pure spectral classification results.

This article deals with a method for acquiring vegetation maps, suitable for monitoring and evaluating the conservation status of heathland vegetation from hyperspectral data. The applied method is a recursive supervised segmentation algorithm based on a Tree-structured Markov Random Field (TS-MRF), capable of incorporating structural dependencies in the classification process. To this end, a tree structure is used that is built upon structural dependencies that are present in the field. The classification results from this TS-MRF with extended tree are compared to pixel-based classification results, results from a simple smoothing post-processing, and the result from the original binary TS-MRF technique.

In this paper, we present a combined image restoration and fusion approach to enhance the spatial resolution of hyper-spectral (HS) images, using a low spatial resolution HS observation and a high spatial resolution multispectral (MS) observation of the same scene. The proposed approach is based on an iterative Expectation-Maximization restoration algorithm, improving the spatial resolution of the HS observation by knowledge of the images point spread function, combined with a Bayesian fusion approach, using the MS observation at a higher spatial resolution scale as an auxiliary. A practical implementation scheme is presented. Simulation experiments are conducted for performance evaluation.

In this paper, we present a technique for the restoration of multispectral images. The presented procedure is based on an expectation-maximization (EM) algorithm, which applies iteratively a deconvolution and a denoising step. The restoration is performed in a multispectral way instead of band-by-band. The deconvolution technique is a generalization of the EM-based grayscale-image restoration and allows for the reconstruction of spatial as well as spectral blurring. The denoising step is performed in wavelet domain. To account for interband correlations, a multispectral probability density model for the wavelet coefficients is chosen. Rather than using a multinormal model, we opted for a Gaussian scale mixture model, which is a heavy-tailed model. Also in this paper, the framework is extended to include an auxiliary image of the same scene to improve the restoration. Experiments on Landsat and AVIRIS multispectral remote-sensing images are conducted.

In this paper, a technique is presented for the fusion of multispectral (MS) and hyperspectral (HS) images to enhance the spatial resolution of the latter. The technique works in the wavelet domain and is based on a Bayesian estimation of the HS image, assuming a joint normal model for the images and an additive noise imaging model for the HS image. In the complete model, an operator is defined, describing the spatial degradation of the HS image. Since this operator is, in general, not exactly known and in order to alleviate the burden of solving the inverse operation (a deconvolution problem), an interpolation is performeda priori. Furthermore, the knowledge of the spatial degradation is restricted to an approximation based on the resolution difference between the images. The technique is compared to its counterpart in the image domain and validated for noisy conditions. Furthermore, its performance is compared to several state-of-the-art pansharpening techniques, in the case where the MS image becomes a panchromatic image, and to MS and HS image fusion techniques from the literature.

This contribution concerns the retrieval of colour texture. The interband correlation structure is considered by modeling the heavy-tailed image wavelet histograms with a multivariate generalized Gaussian. As a similarity measure we propose to use the Rao geodesic distance, which, in contrast to the Kullback-Leibler divergence, exists in a closed form for any fixed value of the shape pa rameter of the distribution. We apply this in several retrieval experiments. The modeling of the interband correlation significantly increases retrieval rates, while the geodesic distance is shown to outperform the Kullback- Leibler divergence. A multivariate Laplace distribution yields better results than a Gaussian, indicating the potential of a model with variable shape parameter together with the geodesic distance.

In this paper, restoration of multispectral images is performed. The presented procedure is based on an Expectation-Maximization algorithm, which applies iteratively a deconvolution and a denoising step. The deconvolution step is a Landweber iteration step, while in the denoising step wavelet shrinkage is performed. The restoration is improved by using a multispectral approach instead of a bandwise one. To account for interband correlations, a multispectral probability density model for the wavelet coefficients is chosen. Furthermore, more, an auxiliary coregistered noise-free image of the same scene is used to improve the restoration. Experiments on a Landsat multispectral remote sensing image are conducted.

This work evaluates a classification method, including spatial information, for dune vegetation along the Belgian coastline. The used method is a recursive supervised segmentation algorithm based on a tree-structured Markov Random Field. This technique describes a K-ary field as a sequence of binary Markov Random Fields, each of which is represented by a node in the tree. The obtained classification results were compared to results with the same data set, for a purely spectral classification and a spectral classification, followed by spatial smoothing.

In this work, a technique is presented for the fusion of multi-spectral (MS) and hyperspectral (HS) images to enhance the spatial resolution of the latter. The technique works in the wavelet domain, and is based on a Bayesian estimation of the HS image, assuming a joint normal model for the images, and an additive noise imaging model for the HS image. An appropriate estimation strategy is also proposed. The technique is compared to its counterpart in the spatial domain, and validated for noisy conditions. Further, its performance is compared to several state-of-the-art pansharpening techniques, in the case where the MS image becomes a panchromatic image, and to some MS and HS image fusion techniques from the literature.

In this paper, a Bayesian wavelet-based denoising procedure for multicomponent images is proposed. A denoising procedure is constructed that (1) fully accounts for the multicomponent image covariances, (2) makes use of Gaussian scale mixtures as prior models that approximate the marginal distributions of the wavelet coefficients well, and (3) makes use of a noise-free image as extra prior information. It is shown that such prior information is available with specific multicomponent image data of, e.g., remote sensing and biomedical imaging. Experiments are conducted in these two domains, in both simulated and real noisy conditions.

In this paper, a Bayesian wavelet denoising procedure for multicomponent images is proposed. The procedure makes use of a noise-free single component image as prior information. The prior model for the wavelet coefficient marginals is a Gaussian scale mixture (GSM) model. Experiments on color images and multispectral remote sensing images are conducted to validate the procedure.

The color of the sea is determined by the contents of the water, especially the concentrations of suspended particulate matter (SPM), phytoplankton pigments such as chlorophyll (CHL) and colored dissolved organic matter (CDOM). Reversely, optical sensors that measure the water-leaving reflectance spectra allow us to calculate the desired concentration products. In this paper, a method is introduced that is valid for both case 1 and 2 waters. To this end, model is fitted to reflectance spectra, using simulated annealing for optimizing the mean square of the reflectance over all spectra.

In hyperspectral remote sensing, sensors acquire reflectance values at many different wavelength bands, to cover a complete spectral interval. These measurements are strongly correlated, and no new information might be added when increasing the spectral resolution. Moreover, the higher number of spectral bands increases the complexity of a classification task. Therefore, feature reduction is a crucial step. An alternative would be to choose the required sensor bands settings a priori. In this letter, we introduce a statistical procedure to provide band settings for a specific classification task. The proposed procedure selects wavelength band settings which optimize the separation between the different spectral classes. The method is applicable as a band reduction technique, but it can as well serve the purpose of data interpretation or be an aid in sensor design. Results on a vegetation classification task show an improvement in classification performance over feature selection and other band selection techniques.

This communication studies the detection of vegetation stress in hyperspectral data. Compared to traditional vegetation stress indices, the proposed approach uses the complete reflectance spectrum and its wavelet representation. The detection strategy is formulated as a classification problem. Experiments are conducted on fruit tree stress detection. The experiments show the superior performance of the proposed strategy and demonstrate its generic nature.

In this paper, a wavelet-based technique for the demosaicking of color filter arrays (CFA) is proposed. Conventional demosaicking techniques perform interpolation of the missing pixels in the YC/sub r/C/sub b/ color space, where emphasis is put on an optimal interpolation of the luminance. We make advantage of this by merging the obtained luminance image with interpolated R1G and B images. The merging is performed in a multiresolution way, using the wavelet transform. This postprocessing technique is demonstrated to outperform traditional demosaicking interpolation techniques, visually as well as quantitatively, using two measures, the PSNR and /spl Delta/E/sub ab/, which is a measure for the average color distance between original and demosaicked images in the CIELAB color space.

This paper exploits the interband correlations of color and multispectral images for wavelet-based denoising. For this, a multispectral extension of the linear minimum mean squared error estimation (LMMSE) is constructed to estimate the signal from the observed wavelet coefficients. The calculation involves the signal autocovariance matrices, which are estimated globally or locally far centered square windows using maximum likelihood and MAP. The method is demonstrated to outperform single-hand denoising on color and 7-band Landsat multispectral images.

This paper studies the potential of airborne hyperspectral imagery for classifying vegetation along the Belgian coastlines. Here, the aim is to build vegetation maps using automatic classification. Besides a general linear multiclass classifier (Linear Discriminant Analysis), several strategies for combining binary classifiers are proposed: one based on a hierarchical decision tree, one based on the Hamming distance between the codewords obtained by binary classifiers and one based on the coupling of posterior probabilities. In addition, a new procedure is proposed for spatial classification smoothing. This procedure takes into account spatial information by letting the decision for classification of a pixel depend on the classification probabilities of neighboring pixels. This is shown to render smoother classification images.

In this paper, a denoising technique for multivalued images exploiting interband correlations is proposed. A redundant wavelet transform is applied and denoising is applied by thresholding wavelet coefficients. Specific functions of the wavelet coefficients are defined that exploit interscale and/or interband correlation of the signal. Three functions are studied: the square of the wavelet coefficients, products of coefficients at adjacent scales, and products of coefficients from different bands. For these functions, the signal and noise probability density functions (pdf) become more separated. The high signal correlation between bands is exploited by summing these products over all bands, in this way separating noise and signal pdfs even more. The noise pdf of the proposed quantities is derived analytically and from this, a wavelet threshold is derived. The technique is demonstrated to outperform single band wavelet thresholding on multispectral remote sensing images and on multimodal MRI images.

In this paper, a new orthogonal wavelet representation of multivalued images is presented. The idea for this representation is based on the concept of maximal gradient of multivalued images. This concept is generalized from gradients toward linear vector operators in the image plane with equal components along rows and columns. Using this generalization, the pyramidal dyadic wavelet transform algorithm using quadrature mirror filters is modified to be applied to multivalued images. This results in a representation of a single image, containing multiscale detail information from all component images involved. This representation leads to multiple applications ranging from multispectral image fusion to color and multivalued image enhancement, denoising and segmentation. In this paper, the representation is applied for fusion of images. More in particular, we introduce a scheme to merge high spatial resolution greylevel images with low spatial resolution multivalued images to improve spatial resolution of the latter while preserving spectral resolution. Two applications are studied: demosaicing of color images and merging of multispectral remote sensing images.

A concept called multiscale structure tensor is introduced, as an extension of the single-scale structure tensor. It is based on a redundant dyadic wavelet transform, and allows for accumulating multiscale gradient information of local regions. In this way, the technique has averaging properties, while preserving edge structures. We show how to make advantage of this for image enhancement and denoising purposes.

In this paper, a new wavelet representation for multivalued images is presented. The idea for this representation is based on the first fundamental form that provides a local measure for the contrast of a multivalued image. In this paper, this concept is extended toward multiscale fundamental forms using the dyadic wavelet transform of Mallat. The multiscale fundamental forms provide a local measure for the contrast of a multivalued image at different scales. The representation allows for a multiscale edge description of multivalued images. A variety of applications is presented, including multispectral image fusion, color image enhancement and multivalued image noise filtering. In an experimental section, the presented techniques are compared to single valued and/or single scale algorithms that were previously described in the literature. The techniques, based on the new representation are demonstrated to outperform the others.

In this paper, a new multispectral image wavelet representation is introduced, based on multiscale fundamental forms. This representation describes gradient information of multispectral images in a multiresolution framework. The representation is in particular extremely suited for the fusion and merging of multispectral images. For fusion as well as for merging, a strategy is described. Experiments are performed on multispectral images. In these experiments, Landsat Thematic Mapper images are fused and merged with panchromatic images. The proposed techniques are compared to wavelet-based techniques described in the literature.

A new anisotropic diffusion noise filtering technique is proposed. The technique has two specific features: it is designed for vector-valued images in general and for color images in particular, and it is a multiresolution technique, using wavelet transforms. The vector-valued feature is based on the first fundamental form, which reflects the edge information of a vector-valued image. The multiresolution feature is based on an extension of the first fundamental form towards multiscale fundamental forms. In an experimental section, the proposed technique is evaluated and compared to the single-valued and single-scale versions.

In this paper, fusion of multispectral images for visualization is aimed at, based on the projection of the scatter-diagrams onto a one-dimensional space. Linear as well as nonlinear projection techniques are used. In contrast with existing mapping techniques which work globally, a local mapping technique is constructed. In this technique, the images are subdivided into blocks, where each block of pixels is visualized through a different map. Then, for each pixel, a locally adapted map is created by weighting the maps of the surrounding blocks using Euclidean distance measure. A linear local mapping, based on local PCA and a nonlinear local mapping, based on Kohonen's SOM map are generated and compared to the global procedures. Experiments are conducted on multispectral LANDSAT imagery.

For high-dimensional data, the appropriate selection of features has a significant effect on the cost and accuracy of an automated classifier. A feature selection technique using genetic algorithms is applied. For classification, hard and fuzzy kNN classifiers are compared. Composite Fuzzy classifier architectures are investigated. Experiments are conducted on AVIRIS data, and the results are evaluated in the paper.

We conjecture that texture can be characterized by the statistics of the wavelet detail coefficients and therefore introduce two feature sets: (1) the wavelet histogram signatures which capture all first order statistics using a model based approach and (2) the wavelet co-occurrence signatures, which reflect the coefficients' second-order statistics. The introduced feature sets outperform the traditionally used energy. Best performance is achieved by combining histogram and co-occurrence signatures.

Deals with the extraction of rotation-invariant texture features from multiscale image decomposition. We argue that the often used separable filtering schemes are very impractical for rotation-invariant feature extraction. Therefore we propose a scheme based on non-separable isotropic wavelet frames. The performance of the features is evaluated in a classification experiment.

This paper describes a genetic differential competitive learning algorithm, which is proposed to prevent fixation to the local minima and improve the unsupervised training results for the classification of remotely sensed data. The differential competitive learning (DCL) combines competitive and differential-Hebbian learning and represents a neural version of adaptive delta modulation. This learning law uses the neural signal velocity as a local unsupervised reinforcement mechanism. The Jeffries-Matusita (J-M) distance, which is a measure of statistical separability of pairs of the 'trained' clusters, is used for the evaluation of the proposed algorithm. The Landsat Thematic Mapper (TM) data will be used for simulation to show the effectiveness of the algorithm.

The problem of parameter estimation from Rician distributed data (e.g., magnitude magnetic resonance images) is addressed. The properties of conventional estimation methods are discussed and compared to maximum-likelihood (ML) estimation which is known to yield optimal results asymptotically. In contrast to previously proposed methods, ML estimation is demonstrated to be unbiased for high signal-to-noise ratio (SNR) and to yield physical relevant results for low SNR.

A competitive learning scheme for color image quantization is elaborated, in which the dithering process for eliminating contouring effects, instead of performed a posteriori, is imbedded in the quantization process. Quantization is performed by clustering in color space. The dithering process is a simple error diffusion which diffuses the quantization error made by one pixel to its local neighborhood. For small color palettes, this is demonstrated to improve the visual quality of quantized images.

An important issue in the analysis of two-dimensional electrophoresis images is the detection and quantification of protein spots. In this paper we describe a new robust technique to segment and model the different spots present in the gels. For the segmentation a watershed technique is applied. For the quantification of the spots, a new spot model is constructed, based on diffusion principles. Besides the advantage of having a physical interpretation, the model is demonstrated to be superior to the commonly used Gaussian models.

In this paper the problem of local optimality of color image quantization procedures is discussed. The well-known and frequently used C-means clustering algorithm (CMA) is applied to the problem, and its dependence on initial conditions is studied. A hybrid approach, combining CMA with a genetic algorithm is constructed, and it is shown that this approach is insensitive to its initial conditions. Results compare the performance of the genetic approach with CMA on three different types of initial conditions: random initial conditions and two popular color image quantization algorithms: the median-cut algorithm and the variance-based algorithm. In all cases the genetic approach outperforms CMA.

In this paper optimal image quantization algorithms and their dependence on initial conditions are studied. For gray-level images using four different types of initial conditions, the behaviour of the well-known optimal Lloyd-Max quantization (LMQ) algorithm is studied and compared to a fuzzy version (FLMQ). A generic quantization algorithm is developed which is a hybrid approach combining a genetic algorithm with optimal quantization. It is shown that the latter technique is almost insensitive to initial conditions and performs better than the former two. For color images the technique is shown to lead to visual improvement of the image quality.

This paper describes a novel speech signal classification scheme based on spectrograms which are subjected to wavelet transform: a procedure which yields specific information regarding time and frequency variation of the signal. Feature vectors are extracted and classified using LVQ networks. The output of the network is interpreted as a fuzzy membership coefficient. This scheme is applied to the classification of voice dysphonia.