In recent years, due to the rapid development of deep learning, autoencoders (AEs) has become a popular technique for hyperspectral unmixing. However, most of the AE-based unmixing networks only use one network to learn the features, resulting in the imbalance of the endmembers and abundances. Furthermore, the spectral fidelity of the loss function is usually measured in a linear manner by omitting the non-linearity in spectral mixing. Therefore, we propose a two-stream autoencoder blind unmixing network combined with the non-linear Hapke unmixing model. Specifically, we construct a network for solving endmembers then transfer the estimated weight matrices to another designed abundance network. Notably, we introduce a new loss function consisting of three parts: network error estimated using spectral angle distance, reconstruction error based on the non-linear Hapke model, and TV regularization of the endmembers. Experimental results show that our method has certain advantages in endmember extraction and abundance estimation compared to other excellent unmixing methods.

Over the past few decades, researchers have proposed various hyperspectral unmixing (HU) methods. Among these methods, deep learning (DL) has emerged as a promising approach for HU, providing new opportunities for advancement. However, accurately quantifying the presence of spectral variability factors within a mixture remains a challenging task. Therefore, numerous literatures have concerned the HU with spectral variability, in which the variation spectra are generated through the network. However, there is a lack of the connection between the network and spectral variability, so they fail to provide physically meaningful interpretability of spectral variability. To this end, we use physics-driven model to represent spectral variability and introduce it to the two-stream autoencoder unmixing network, resulting in the improved endmember and abundance estimations. Specifically, the endmember extraction network learn spectral variability parameters associated the dispersion model to generate the variations of spectra, which enhancing physical interpretability of endmember variability. In addition, the abundance estimation autoencoder network, tied to the endmember extraction network by shared weights, estimates abundances using the reconstructed hyperspectral image. Compared with the state-of-the-art HU approaches on three real hyperspectral image datasets, our method outperforms these techniques with improved unmixing accuracy, especially on endmember estimation.

For unmixing of sequences of hyperspectral images (SHS), spectral variability is an important factor to be considered. However, most existing unmixing methods tend to model the endmember and its variability in spatial domain rather than transform domain. In fact, the intrinsic and invariant features of the spectral curve can be effectively represented by wavelet transform. Therefore, this paper proposes to perform SHS unmixing in the wavelet domain by combing the Bayesian method. Firstly, the assumption of abundance being invariability in both the spatial and wavelet domains is made, then the formulation of unmixing in the wavelet domain using Perturbed Linear Mixing Model (PLMM) is presented. Secondly, based on the Bayesian framework, the likelihood and prior are both given, in which the parameter priors are divided into two parts: low and high frequency wavelet coefficients. Moreover, by considering the sparsity of the high-frequency wavelet coefficients of endmembers, a non-informative prior with zero-mean is designed. Meanwhile, for the coefficients of endmember variability, Gaussian distributions are utilized to represent the steady fluctuation along the temporal dimension. Finally, using the maximum a posterior (MAP) rule, a hierarchical spectral variability unmixing model in wavelet domain is built and solved by the Markov chain Monte Carlo (MCMC) sampling algorithm. Numerical experiments show that the proposed method generates more accurate estimates for endmembers and their variation.

Hyperspectral unmixing is an important task in hyperspectral applications. Its essence is to estimate the spectra ( endmembers ) and corresponding proportion (abundances) of pure substances. In this paper, we propose a new hyperspectral unmixing method with autoencoder network in wavelet domain. Based on the sparsity of wavelet coefficients, the high frequency parts are truncated to provide more reliable spectral similarity. After that, the batch normalization and ReLU function are followed to construct the hidden layer. In terms of loss function, the$l_{2}$and$l_{1}$norm are added to low and high frequency coefficients to ensure the energy fidelity and enhance the sparsity, respectively. Moreover, the hidden layer is characterized by$l_{1/2}$norm to model the sparse prior of abundance, and SAD is used to enhance the spectral similarity. A large number of experiments show that the proposed method is superior to the most advanced methods.

A hyperspectral image sequence can be obtained at different time in the same region from a hyperspectral sensor. The environmental change usually leads to variation in endmember reflectance, which has an important influence on unmixing process. In this article, a Bayesian unmixing model considering spectral variability for hyperspectral sequence is proposed, in which composite prior distributions of abundance and endmember variability are developed. The abundance priors consider the continuity of abundance in the temporal and spatial domains, simultaneously. Specifically, in the spatial domain, a data-adaptive variance of the abundance prior distribution is put forward based on local spatial difference. Moreover, the priors of endmember variability in temporal continuity and spectral smoothness are also exploited. Finally, a joint posterior distribution is obtained by the likelihood function and the parameter prior distributions, which can be calculated by the Markov chain Monte Carlo (MCMC) algorithm. Experiments on synthetic and real data sets demonstrate the effectiveness of the proposed approach in terms of abundance, endmember, and its variability estimation accuracy.

Synthesizing high resolution (HR) hyperspectral image (HSI) by merging a low resolution (LR) HSI with corresponding HR multispectral image (MSI) has become a promising HSI super-resolution scheme. Most existing HSI-MSI fusion methods are effective to some extent, while several challenges remain. First, the spectral response of a given material exhibits considerable variability due to different acquisition time and conditions, however variations in spectral signatures are often neglected. Second, a majority of off-the-shelf methods require predefined degradation operators, which can be unavailable in practice. To tackle above issues, we introduce a novel fusion approach with Bayesian framework. Specifically, we regard the up-sampled LR-HSI as the low frequency component of the underlying HR-HSI. We characterize the texture features of high and low frequency components respectively, which can enlarge modeling capacity and bypass the absence of degradation operators. Furthermore, we depict the relative smoothness of reflectance spectra with Gaussian Process. Extensive experiments on synthesized and real datasets illustrate the superiority of the proposed strategy in terms of fusion performance and robustness to spectral variability.

Deep Convolutional Neural Networks (CNN) have drawn great attention in image super-resolution (SR). Recently, visual attention mechanism, which exploits both of the feature importance and contextual cues, has been introduced to image SR and proves to be effective to improve CNN-based SR performance. In this paper, we make a thorough investigation on the attention mechanisms in a SR model and shed light on how simple and effective improvements on these ideas improve the state-of-the-arts. We further propose a unified approach called “multi-grained attention networks (MGAN)” which fully exploits the advantages of multi-scale and attention mechanisms in SR tasks. In our method, the importance of each neuron is computed according to its surrounding regions in a multi-grained fashion and then is used to adaptively re-scale the feature responses. More importantly, the “channel attention” and “spatial attention” strategies in previous methods can be essentially considered as two special cases of our method. We also introduce multi-scale dense connections to extract the image features at multiple scales and capture the features of different layers through dense skip connections. Ablation studies on benchmark datasets demonstrate the effectiveness of our method. In comparison with other state-of-the-art SR methods, our method shows the superiority in terms of both accuracy and model size.

Classification is one of the major research fields in hyperspectral imagery. Due to the fact that neighboring pixels are more likely to share the same label, it is practical to use spatial information in hyperspectral image to achieve higher accuracy. On the other hand, however, spatial information also leads to higher computational complexity. This paper proposes an efficient implementation of a spatial-spectral kernel sparse representation for hyperspectral image classification base on the multi-GPU platform. The proposed implementation takes advantage of the capability of compute-unified device architecture (CUDA), such as shared memory, streams and peer-to-peer (P2P) transfer of data. In addition, an improvement of performance can be achieved by calculation reorganization and bandwidth usage optimization. Experimental results demonstrate that the proposed method achieves an up to 56.81X speedup in computation time while guaranteeing the classification accuracy.

The environmental change caused by time interval usually leads to the disturbance of endmember reflectance curve, which has an important influence on multi-temporal hyperspectral unmixing process. In this paper, a Bayesian unmixing model considering the spectral variability is proposed, in which a mixed prior distribution of the abundance is constructed. Different form the existing methods, the continuity of the abundance in time and spatial domain is considered simultaneously. In order to describe the smoothness of the abundance, a data-adaptive variance of probability distribution is designed based on the spatial local difference. Then combined with the priors of endmembers and spectral variability, the joint posterior distribution is set up and Markov chain Monte-Carlo(MCMC) algorithm is developed for posterior computation. Experiments on simulated and real datasets demonstrate the effectiveness of the proposed algorithm in terms of abundance estimation and endmember estimation.

As a low-rank tensor modeling, tensor tubal rank has been received more attention in hyperspectral image (HSI) recovery. However, the tubal rank is often approximated by tensor nuclear norm, which leads to modeling bias. To achieve an unbiased approximation and improve the model robustness, in this paper, a non-convex relaxation based HSI low-rank recovery model is proposed. And the model is solved efficiently by alternating direction method of multipliers (ADMM) optimization method. Two HSI datasets are employed to exhibit the superiority of the proposed model over the nuclear norm penalization method in terms of the accuracy and robustness.

Anomaly detection has become a hot topic in hyperspectral image (HSI) processing. Both spatial and spectral features have been proven to be very important for accurate and efficient hyperspectral anomaly detection. The traditional HIS anomaly detection algorithms usually reshape HSI to a matrix, which destroy spatial or spectral structure. In this paper, we propose a novel method of hyperspectral anomaly detection based on LOW RANK AND SPARSE TENSOR DECOMPOSITION (LRASTD). Taking into consideration that HSI data can be essentially regarded as a three-order tensor. HSI is modeled as a background tensor and a sparse anomalies tensor. A tensor nuclear norm is employed to constrain the core tensor, which be designed to characterize the low dimensional structure of the core tensor. Furthermore, a novel sparse tenor norm is proposed to constrain the anomaly targets. Experiments on both simulated and real hyperspectral data sets demonstrate the efficiency and effectiveness of the proposed method.

A new pan-sharpening method based multispectral (MS) image intensity modulation is proposed. Firstly, according to the spectral correlation between MS image and panchromatic (PAN) image, a new spectral-enhanced PAN (SPAN) image is obtained. Then, SPAN image is utilized to calculate the spectral and spatial difference coefficients. Finally, the fused high resolution MS (HRMS) image is generated by modulating the intensity of the upsampled MS image. The experimental results demonstrate that the proposed method performs well both on spatial details preservation and spectral distortion reduction.

Change Detection in Remote sensing image is, in essence, to detect the changes of ground features with regard to time from remote sensing perspective. It is usually realized by analyzing and processing multi-temporal high resolution images. Change Detection based on fully connected conditional random field not only improves the detection accuracy of remote sensing image, but also achieves better robustness. However, with the growth of high-resolution data volumes, this algorithm consumes a huge amount of time and computational resources, and therefore needs to be improved accordingly. Spark is an open-source distributed general- purpose cluster-computing framework. It has powerful memory computing and efficient task scheduling capabilities for complex iterative calculations. Based on Spark, this paper proposes a distributed and parallel method of change detection in remote sensing image based on Fully Connected Conditional Random Field that analyzes the data input form, and proposes a multi-temporal image reading strategy on cloud platforms. This method decomposes the algorithm flow, and performs distributed parallel processing on each stage and makes full use of the processing advantages of data locality to implement a reasonable intermediate data storage. Experimental results demonstrate that this parallel method achieves a promising speedup with high scalability, while guaranteeing remarkable detection accuracy.

Hyperspectral image (HSI) is often contaminated by mixed noise, which severely affects the visual quality and subsequent applications of the data. In this paper, HSI restoration based on low-rank recovery with a local neighborhood weighted spectral-spatial total variation (TV) model is proposed, which focuses on the preservation of spatial structure and spectral fidelity. The low-rank matrix model is adopted to exploit the spectral and spatial correlation information, and the l1-norm is used as a prior to remove the sparse noise. Furthermore, a local spatial neighborhood weighted spectral-spatial TV is utilized to jointly model the spectral-spatial prior information; specifically, the spectral and spatial differences are both considered in the TV term, and the weight is computed by considering the local neighborhood information in the spatial domain. Alternating direction method of multipliers optimization procedure is extended to solve the presented model. Experimental results demonstrate that the proposed method can remove the mixed noise, enhance the structural information simultaneously, and offer the best performance compared with several state-of-the-art HSI restoration methods.

Pan-sharpening is a common image-fusion method. To improve the quality of fused images, a multilevel deep learning Pan-sharpening method is proposed in this paper. In the training phase, we introduce Coupled Sparse Denoising Autoencorder (CSDA) to reconstruct high-Resolution (HR) multispectral (MS) image from low-Resolution (LR) MS image and HR Panchromatic (Pan) image. CSDA has four networks including LM-HP network, HR-MS network, feature mapping network and fine-tuning network. The hidden features in LM-HP network and HR-MS network as well as the mapping function between the two features are learned through joint optimization. In LM-HP and HR-MS networks, the hidden features of image patch pairs are extracted by the sparse autoencoder. A sparse denoising autoencoder is used to build the nonlinear mapping between the extracted features. In the testing phase, the LR-MS and HR-Pan images patches are fed to the CSDA network to reconstruct the fused HR-MS image. The experimental results show that the proposed method is better than the traditional pans-sharpening methods.

Most traditional compressive sensing (CS) reconstruction methods suffer from the intensive computation caused by iterations. This paper aims at presenting a non-iterative algorithm to reconstruct hyperspectral images (HSI) from patch-based compressively sensed measurements. Our method contains two residual convolutional neural networks. One is reconstruction network for compressive sensing reconstruction and the other is deblocking network for removing the blocky effect, which is caused by patch-based sampling. The reconstruction network can efficiently reconstruct all the bands of HSI jointly, thus the spectral correlation is well preserved. In addition, the deblock performance is enhanced by combining more patches into a larger patch in the deblocking network. Experimental results verify that our method outperforms the state-of-the-art compressive sensing reconstruction methods with patch-based CS measurement.

Low-rank representation has been widely used as a powerful tool in hyperspectral image (HSI) recovery. The existing studies involving low-rank problems are commonly under the nuclear norm penalization. However, nuclear norm minimization tends to over-shrink the components of rank, which leads to modeling bias. In this paper, a new nonconvex penalty is introduced to obtain an unbiased low-rank approximation. In Addition, local spatial neighborhood weighted spectral-spatial total variation (TV) regularization is introduced to preserve spatial structural information. And sparse l1-norm is used as a constraint to sparse noise. Finally, a novel HSI non-convex low-rank relaxation restoration model is proposed. A number of experiments show that the proposed method can effectively remove the mixed-noise, and result in an unbiased estimate with better robustness.

A single sensor camera can capture scenes by means of color filter array. Each pixel samples only one of the three primary colors. Color demosaicking (CDM) is a process of reconstruction a full color image from this sensor data. In this paper, we propose a novel CDM scheme based on learned simultaneous sparse coding over nonlocal tensor representation. First, similar 2D patches are grouped to form a three-order tensor, that is, 3D array. Then, three sub-dictionaries, which characterize the coherent structures that appear in each dimension of the grouped tensor, are learned jointly by using Tucker decomposition. The consequent coefficient tensor is imposed by the grouped-block-sparsity constraint, which forces the similar patches to share the same atoms of the dictionaries in their sparse decomposition. Experimental results demonstrate the effectiveness both in the average CPSNR and visual quality.

This paper presents a novel method for anomaly detection in hyperspectral image(HSI) based on low-rank representation. In the observed HSI, the anomalies can be separated from the background. Since each pixel in the background can be approximately represented by a background dictionary, and the representation coefficients of the background pixels are correlative, a low-rank representation model is used to model the background part. Besides, to gain robust representation coefficient, the sum-to-one constraint is added. The advantage of the proposed low-rank representation sum-to-one (LRRSTO) method is that it makes use of the global correlation of the background and strength the robustness of the representation. Experiments results have been conducted using both simulated and real data sets. These experiments indicated that our algorithm achieves very promising performance.

Colour image enhancement is an important preprocessing phase of many image analysis tasks such as image segmentation, pattern recognition and so on. This study presents a new local brightness adaptive variational model using Wasserstein distance for colour image enhancement. Under the perceptually inspired variational framework, the proposed energy functional consists of an improved contrast energy term and a Wasserstein dispersion energy term. To better adjust image dynamic range, the authors propose a local brightness adaptive contrast energy term using the average brightness of image local patch as the local brightness indicator. To restore image true colours, a Wasserstein distance‐based dispersion energy term is used to measure the statistical similarity between the original image and the enhanced image. The proposed energy functional is minimised by using a gradient descent algorithm. Two objective measures are used to quantitatively measure the enhancement quality. Experimental results demonstrate the efficiency of the proposed model for removing colour cast and haze, enhancing contrast, recovering details and equalising low key images.

A deep neural network (DNN)-based new pansharpening method for the remote sensing image fusion problem is proposed in this letter. Research on representation learning suggests that the DNN can effectively model complex relationships between variables via the composition of several levels of nonlinearity. Inspired by this observation, a modified sparse denoising autoencoder (MSDA) algorithm is proposed to train the relationship between high-resolution (HR) and low-resolution (LR) image patches, which can be represented by the DNN. The HR/LR image patches only sample from the HR/LR panchromatic (PAN) images at hand, respectively, without requiring other training images. By connecting a series of MSDAs, we obtain a stacked MSDA (S-MSDA), which can effectively pretrain the DNN. Moreover, in order to better train the DNN, the entire DNN is again trained by a back-propagation algorithm after pretraining. Finally, assuming that the relationship between HR/LR multispectral (MS) image patches is the same as that between HR/LR PAN image patches, the HR MS image will be reconstructed from the observed LR MS image using the trained DNN. Comparative experimental results with several quality assessment indexes show that the proposed method outperforms other pan-sharpening methods in terms of visual perception and numerical measures.

This paper proposes a new hyperspectral images superresolution (HSI-SR) method based on compressive sensing (CS) theory, spatial sparsity and spectral similarity prior. First, according to sparsity and incoherence of CS theory, we propose a new dictionary learning method, ensuring that the learned dictionary not only has less dimensionality to speed up the sparse decomposition, but also satisfies sparsity well. Then, we introduce the spatial sparsity and spectral similarity regularizations into HSI-SR model, which can recover the spatial information effectively and preserve the spectral information well. The experimental results show the proposed method outperforms other well-known methods in terms of both objective measurements and visual evaluation.

Kernel regression has been shown to be a powerful image denoising technique. In this paper, a three-dimensional (3-D) kernel regression hyperspectral image (HSI) denoising mechanism is proposed. The main contributions of this paper can be summarized as follows: Three orientation vectors and the corresponding coefficients are presented, which are adaptive for each pixel based on the innovation of 2-D structure tensor. An adaptive-driven 3-D tensor matrix is proposed for kernel regression, in which the spatial geometric structure and spectrum continuity are both considered. The proposed adaptive kernel regression is applied to HSI denoising. Both stimulated and real data experiments indicate that the proposed method can work well in detail preservation and noise removal.

In this paper, we propose a variational Bayesian method for Retinex to simulate and interpret how the human visual system perceives color. To construct a hierarchical Bayesian model, we use the Gibbs distributions as prior distributions for the reflectance and the illumination, and the gamma distributions for the model parameters. By assuming that the reflection function is piecewise continuous and illumination function is spatially smooth, we define the energy functions in the Gibbs distributions as a total variation function and a smooth function for the reflectance and the illumination, respectively. We then apply the variational Bayes approximation to obtain the approximation of the posterior distribution of unknowns so that the unknown images and hyperparameters are estimated simultaneously. Experimental results demonstrate the efficiency of the proposed method for providing competitive performance without additional information about the unknown parameters, and when prior information is added the proposed method outperforms the non-Bayesian-based Retinex methods we compared.