Although light-weighted explainable deep learning techniques are critical for operational hyperspectral image (HSI) classification, it is very challenging to achieve these techniques due to difficulties to deal with the spatial-spectral complexity and coupling effect in HSI. Leveraging the excellent feature learning capability of the attention mechanism, this paper presents a spatial-spectral dual transformer (SSDT) network that decomposes the conventional spatial-spectral transformer operation into a spatial transformer and a spectral transformer, which not only reduce the model complexity, but also allows the use of self-attention to explain feature relevance. The proposed approach is tested on some benchmark HSI scenes and the results demonstrate that the proposed dual transformer network not only achieves new state-of-the-art performance due to its excellent feature extraction capability, but also enables the analysis and visualization of feature importance and decision making process.

Algorithms designed for ice–water classification of synthetic aperture radar (SAR) sea ice imagery produce only binary (ice and water) output typically using manually labeled samples for assessment. This is limiting because only a small subset of labeled samples are used, which, given the nonstationary nature of the ice and water classes, will likely not reflect the full scene. To address this, we implement a binary ice–water classification in a more informative manner considering the uncertainty associated with each pixel in the scene. To accomplish this, we have implemented a Bayesian convolutional neural network (CNN) with variational inference to produce both aleatoric (data-based) and epistemic (model-based) uncertainty. This valuable information provides feedback as to regions that have pixels more likely to be misclassified and provides improved scene interpretation. Testing was performed on a set of 21 RADARSAT-2 dual-polarization SAR scenes covering a region in the Beaufort Sea captured regularly from April to December. The model is validated by demonstrating: 1) a positive correlation between misclassification rate and model uncertainty and 2) a higher uncertainty during the melt and freeze-up transition periods, which are more challenging to classify. By incorporating the iterative region growing with semantics (IRGS) segmentation algorithm and an uncertainty value-based thresholding algorithm, the Bayesian CNN classification outputs are improved significantly via both numerical analysis and visual inspection.

Unsupervised subpixel mapping (SPM) of hyperspectral image (HSI) is a challenging task due to the difficulties to integrate different prior information and model constraints into a coherent framework. This paper presents a Bayesian neural network for unsupervised HSI SPM, which has the following characteristics. First, the deep image prior (DIP) achieved by a fully convolutional neural network (FCNN) is used to model the spatial correlation efficiently and adaptively in the subpixel label domain. Second, a discrete spectral mixture model (DSMM) is designed to leverage the forward model for enhanced SPM. Third, an auto-encoder architecture is designed to integrate the FCNN and the DSMM to allow efficient unsupervised representational learning using both data and knowledge. Fourth, an expectation-maximization approach is designed to solve the resulting maximum a posteriori problem, where a purified means approach extracts endmembers, and the gradient descent approach updates FCNN parameters for subpixel label estimation. Comparative experiments on both real and simulated HSIs demonstrate that the proposed method outperforms other state-of-the-art methods in terms of both numerical accuracies and visual subpixel mapping results.

Spectral unmixing (SU) plays a fundamental role in hyperspectral image (HSI) processing. Effective SU relies on the accurate and efficient characterization of the noise effect, the endmembers, and the spatial correlation effect in abundances, as well as efficient optimization techniques to estimate these effects. To address these issues, this article presents a Bayesian fully convolutional hyperspectral unmixing network (BCUN) with the following key characteristics. First, a fully convolutional neural network (FCNN)-based deep image prior (DIP) is designed for enhanced characterization and estimation of the spatial context information in abundance maps, leading to more efficient and accurate abundance modeling than the traditional nonnegative least squares (NNLS) approaches. Second, a multivariate Gaussian distribution with an anisotropic covariance matrix is designed to characterize the conditional distribution of the spectral observations, leading to a novel Mahalanobis distance-based loss for FCNN training that is better capable of addressing the noise heterogeneous effect in HSI than the Euclidean distance-based mean squared error (MSE) loss in traditional deep neural networks. Third, the designed conditional distribution of spectral observations also enables the incorporation of the spectral mixture model (SMM) into the FCNN training process for effectively leveraging the knowledge in the forward spectral model. Fourth, the endmembers are modeled and estimated by a “purified means” approach that is capable of better characterizing endmembers. Finally, the above key components are coherently integrated into a Bayesian framework, and the resulting maximuma posteriori(MAP) problem is solved by a designed expectation–maximization (EM) algorithm. Experimental results on both simulated and real HSIs demonstrate that the proposed BCUN approach outperforms the other classical and state-of-the-art methods on both endmember estimation and abundance estimation.

Bidirectional reflectance distribution function (BRDF) models are used to correct surface bidirectional effects and estimate land surface albedo. Many operational BRDF/albedo algorithms adopt a Roujean linear kernel-driven BRDF (RLKB) model because of its simple form and good performance in fitting multidirectional surface reflectance values. However, this model does not explicitly consider topographic effects, resulting in errors when applied over rugged terrain. To address this issue, we proposed a hybrid algorithm suitable for both flat and rugged terrain, called topographical kernel-driven (Topo-KD). First, we constructed a linear kernel-driven BRDF model considering terrain (LKB_T) which describes the topographic effects with a mountain radiative transfer (MRT) model. Then, the Topo-KD algorithm adaptively selects the most suitable model (RLKB or LKB_T) according to the terrain conditions and fitting residuals. The performances of Topo-KD and RLKB using the RossThick–LiSparseReciprocal (RTLSR) kernel are compared using simulated data sets and moderate-resolution imaging spectroradiometer (MODIS) observations. The results show that the BRDF of the pixel is affected by topography. But the RTLSR model does not specifically account for it, resulting in larger biases over rugged terrain than the Topo-KD algorithm in both the red and near-infrared (NIR) bands. The experiment using MODIS data sets demonstrates that the Topo-KD algorithm reduces fitting residuals in the red and NIR bands by 21.5% and 27.4% compared with the RTLSR model. These results indicate that the Topo-KD algorithm can be a better choice for retrieving land surface parameters and describing the radiative transfer process in mountainous areas.

Although accurate training and initialization information is difficult to acquire, unsupervised hyperspectral subpixel mapping (SPM) without relying on this predefined information is an insufficiently addressed research issue. This letter presents a novel Bayesian approach for unsupervised SPM of hyperspectral imagery (HSI) based on the Markov random field (MRF) and a band-weighted discrete spectral mixture model (BDSMM), with the following key characteristics. First, this is an unsupervised approach that allows adjustment of abundance and endmember information adaptively for less relying on algorithm initialization. Second, this approach consists of the BDSMM for accommodating the noise heterogeneity and the hidden label field of subpixels in HSI. The BDSMM also integrates SPM into the spectral mixture analysis and allows enhanced SPM by fully exploring the endmember-abundance patterns in HSI. Third, the MRF and BDSMM are integrated into a Bayesian framework to use both the spatial and spectral information efficiently, and an expectation-maximization (EM) approach is designed to solve the model by iteratively estimating the endmembers and the label field. Experiments on both simulated and real HSI demonstrate that the proposed algorithm can yield better performance than traditional methods.

Unsupervised classification of hyperspectral imagery (HSI) relies on a data generative model, based on which the labels of pixels and the model parameters are iteratively estimated. Traditionally, the generative model is based on the Gaussian mixture model (GMM) that describes the data generation process from a statistical perspective. However, considering the fact that a semantic class is always dominated by a particular endmember, classifying the spectral pixels based on the associated endmember-abundance pattern as described by the spectral mixture model (SMM) is more meaningful from a physical perspective. In this paper, we explore the potential of spectral mixture model for assisting unsupervised classification of HSI based on a recently proposed K-P-Means unmixing algorithm. Moreover, we investigate modeling the spatial information using Markov random field in this new context. We incorporate SMM and MRF into the Bayesiam framework and solve it via the maximum a posterior (MAP) approach. The results on both simulated and real hyperspectral images demonstrate that this new approach can effectively exploit the spatial-spectral information of HSI for improved unsupervised classification of HSI.

Efficient super-resolution of hyperspectral images (HSI) relies on the representational model (RM) that is capable of capturing the spatial and spectral correlation in hyperspectral images. In this paper, the spectral information in hyperspectral images is explained by linear spectral mixture model (LSMM), which expressed the observed pixels as a linear combination of endmembers, and the spatial information is captured by a spatial auto-regression model. The two component is combined in the maximum likelihood estimation (MLE) framework and solved by the expectation and maximization (EM) algorithm. Experiments on both simulated and real hyperspectral images demonstrate that the proposed method is not only capable of providing an accurate and effective super-resolution reconstruction of the image, but also capable of resisting the influence of noise.

Efficient denoising of hyperspectral imagery (HSI) relies on an representational model that is capable of capturing the spatial and spectral correlation in HSI. Recently, an intrinsic representation (IR) approach based on the linear spectral mixture model (LSMM) was proposed for unsupervised feature extraction. The IR model constitutes a sound representational model due to its ability to account for the physical data generation process of HSI, the spatial correlation effect, and the noise variance heterogeneity effect. In this paper, we explore the potential of IR for the denoising of HSI. A noisy pixel in HSI is expressed as a nonnegative linear combination of several endmembers, plus some Gaussian noise with heterogeneous noise variances. In order to perform denoising, the IR approach is used to adaptively estimate both the endmembers and the nonnegative coefficients (i.e., the abundances), which are finally used to reconstruct the clean image. The experiments on both simulated and real hyperspectral images demonstrate that the IR approach not only can resist the influence of noise, but also can preserve the image details.