Typically, neural networks are trained using supervised learning (SL) and evaluated on unseen data. This type of training relies on a substantial amount of data, including clean images. However, in the case of hyperspectral images (HSIs), acquiring a large number of images along with clean versions can be challenging and expensive. This study proposes a two-stage learning strategy to train the model for HSI data with previously unseen noise patterns. The first stage involves supervised learning to train the model on noisy and clean data pairs. The second stage incorporates self-supervised calibration using only noisy data to adapt the model to specific noise patterns. For the latter, to estimate the middle spectral band, we leverage the information from its neighboring band as a target. To ensure the network learns meaningful relationships rather than merely copying the input, we strategically create a blind spot by excluding the target band from the input data. Therefore, our self-supervised learning technique is named as Blind Band Self-Supervised (BBSS) Learning. Our approach has been shown to improve the accuracy of the model for noisy HSIs, even when the network did not previously encounter the specific noise patterns in SL.

Objects exhibit changing spectral characteristics based on weather conditions, and the dynamic spectral behavior of materials under varying weather conditions poses challenges in detecting them in hyperspectral images (HSIs). In this paper, we propose a generative adversarial network (GAN) based scene transfer (ST) method to reduce the negative effects of changing weather conditions on the detection performance of long wave infrared (LWIR) HSIs. The proposed method initially converts the test scene at an arbitrary air temperature to the reference scene at a reference air temperature with the GAN-based ST method. Then, it performs the detection on the transferred scene. When the proposed ST-based method is used, detection results improve even for the HSIs captured at night, and the overall area under the curve (AUC) scores increase by about 8%. Hence, our proposed ST method emerges as a promising solution to enhance the reliability of thermal LWIR HSI analysis, particularly in the presence of different weather conditions between the capturing time of the test and reference HSIs.

This paper investigates the performance of hyperspectral target detection methods for target rediscovery on temperature profiles extracted from longwave infrared (LWIR) hyperspectral images (HSI). The targets in the experiments are selected as the three military vehicles in the captured scenes at different times. These targets are placed in the scene with different angles, at 0, 90, 135 degrees, respectively. The scope of the performed experiments includes the investigation and comparison of target detection performances, (i) with respect to sampling period in a day, (ii) time and temperature difference between reference and test days (iii) time window in a day. We utilized adaptive coherence estimator (ACE) target detection method on temperature profiles extracted over HSI. The experiments indicated that the sampling period during the day should be less than 1 hour for a robust target detection. Secondly, the target detection performance decreases as the distance between the days increases, but can be increased dramatically if similar weather temperatures are observed despite a long duration between the data recordings. Third, it is observed that the time zones with intense temperature changes such as sunrise and sunset are suitable for target detection.

Sparse unmixing (SU) aims to express the observed image signatures as a linear combination of pure spectra knowna prioriand has become a very popular technique with promising results in analyzing hyperspectral images (HSIs) over the past ten years. In SU, utilizing the spatial–contextual information allows for more realistic abundance estimation. To make full use of the spatial–spectral information, in this letter, we propose a pointwise mutual information (PMI)-based graph Laplacian (GL) regularization for SU. Specifically, we construct the affinity matrices via PMI by modeling the association between neighboring image features through a statistical framework and then we use them in the GL regularizer. We also adopt a double reweighted$\ell _{1}$norm minimization scheme to promote the sparsity of fractional abundances. Experimental results on simulated and real datasets prove the effectiveness of the proposed method and its superiority over competing algorithms in the literature.

Semantic segmentation is an emerging field in the computer vision community where one can segment and label an object all at once, by considering the effects of the neighbouring pixels. In this study, the authors propose a new semantic segmentation model that fuses hyperspectral images with light detection and ranging (LiDAR) data in the three‐dimensional space defined by Universal Transverse Mercator (UTM) coordinates and solves the task using a fully‐connected conditional random field (CRF). First, the authors’ pairwise energy in the CRF model takes into account the UTM coordinates of the data; and performs fusion in the real world coordinates. Second, as opposed to the commonly used Markov random fields (MRFs) which consider only the nearby pixels; the fully‐connected CRF considers all the pixels in an image to be connected. In doing so, they show that these long‐term interactions significantly enhance the results when compared to traditional MRF models. Third, they propose an adaptive scaling scheme to decide the weights of LiDAR and hyperspectral sensors in shadowy or sunny regions. Experimental results on the Houston dataset indicate the effectiveness of their method in comparison to the several MRF based approaches as well as other competing methods.

In this study, we propose a novel single image Bayesian super-resolution (SR) algorithm where the hyperspectral image (HSI) is the only source of information. The main contribution of the proposed approach is to convert the ill-posed SR reconstruction (SRR) problem in the spectral domain to a quadratic optimization problem in the abundance map domain. In order to do so, Markov Random Field (MRF) based energy minimization approach is proposed and proved that the solution is quadratic. The proposed approach consists of five main steps. First, the number of endmembers in the scene is determined using virtual dimensionality. Second, the endmembers and their low resolution abundance maps are computed using simplex identification via the splitted augmented Lagrangian (SISAL) and fully constrained least squares (FCLS) algorithms. Third, high resolution (HR) abundance maps are obtained using our proposed maximum a posteriori (MAP) based energy function. This energy function is minimized subject to smoothness, unity and boundary constraints. Fourth, the HR abundance maps are further enhanced with texture preserving methods. Finally, HR HSI is reconstructed using the extracted endmembers and the enhanced abundance maps. The proposed method is tested on three real HSI datasets; namely the Cave, Harvard and Hyperspectral Remote Sensing Scenes (HRSS) and compared to state-of-the-art alternative methods using peak signal to noise ratio, structural similarity, spectral angle mapper and relative dimensionless global error in synthesis metrics. It is shown that the proposed method outperforms the state of the art methods in terms of quality while preserving the spectral consistency.

Semantic segmentation is an emerging field in the computer vision community where one can segment and label an object all at once. In this paper, we propose a semantic segmentation algorithm that takes into account both the hyperspectral images and the LiDAR data. In our segmentation framework, we propose a new energy function that is composed of two terms: a unary energy term and a pairwise energy term. The unary energy term provides the segmentation maps for the hyperspectral data as well as for the LiDAR data which is explained with Fisher Vectors. The pairwise spatial term uses both the UTM coordinates as well as the LiDAR data. Finally, the system is solved with graph-cuts. We report the effect of the parameters in energy minimization and show that the best results are achieved with an SVM-MRF classifier among the several classifiers.

Super-resolution Reconstruction (SRR) is technique to increase the spatial resolution of images. It is especially useful for hyperspectral images (HSI), which have good spectral resolution but low spatial resolution. In this study, we propose an improvement to our previous work and present a novel MAP-MRF (maximum a posteriori-Markov random Fields) based approach for the SRR of HSI. The key point of our approach is to find the abundance maps of an HSI and perform SRR on the abundance maps using MRF based energy minimization, without needing any other additional source of information. In order to do so, first, PCA is used to determine the endmembers. Second, SISAL and fully constraint least squares (FCLS) are used to estimate the abundance maps. Third, in order to find the high resolution abundance maps, the ill-posed inverse SRR problem for abundances is regularized with a MAP-MRF based approach. The MAP-MRF formulation is restricted with the constraints which are specific to the abundances. Using the non-linear programming (NLP) techniques, the convex MAP formulation is minimized and High Resolution (HR) abundance maps are obtained. Then, these maps are used to construct the HR HSI. This improved SRR method is verified on real data sets, and quantitative performance comparison is achieved using PSNR, SSIM and PSNR metrics. Our results indicate that this improved method gives very close results to the original high resolution images, keeps the spectral consistency, and performs better than the compared algorithms.

In many applications data classification may be hindered by the existence of multiple contexts that produce an input sample. To alleviate the problems associated with multiple contexts, context‐based classification is a process that uses different classifiers depending on a measure of the context. Context‐based classifiers offer the promise of increasing performance by allowing classifiers to become experts at classifying input samples of certain types, rather than trying to force single classifiers to perform well on all possible inputs. This study introduces a novel mixture of experts (ME) model, the mixture of hidden Markov model experts, for context‐based classification of samples that are variable length sequences; and derives the update equations for a single probabilistic model that to learn the experts and a gate that connects the experts. The model has a similar high‐level structure to the ME model but has the novelty that the gates and the experts are HMMs and the input data are sequences. Experimental results are presented on three datasets including one for landmine detection. Detailed analysis of the model is provided; which, over multiple runs and cross‐validation experiments, show superior results over the compared algorithms.

An extension to the sparsity promoting iterated constrained endmember (SPICE) algorithm, named as SPICEE, has been presented. In ICE and SPICE, endmembers are estimated using a pseudoinverse method, which may generate endmembers that are not physically possible when representing normalized reflectance spectra. Although this problem can be alleviated by increasing the regularization, too much regularization leads to finding erroneous endmembers. To solve these problems, in this letter, a quadratic optimization solution is proposed that constrains the endmembers to have values between zero and one. The results on three data sets indicate that when regularization is large enough, SPICE and SPICEE generate similar answers; and when regularization is small to none, SPICEE stays more robust. In doing so, besides generating realistic endmembers, SPICEE helps in decreasing the effort necessary for fine-tuning the regularization parameter.

A novel multiple-instance hidden Markov model (MI-HMM) is introduced for classification of time-series data, and its training is developed using stochastic expectation maximization. The MI-HMM provides a single statistical form to learn the parameters of an HMM in a multiple-instance learning framework without introducing any additional parameters. The efficacy of the model is shown both on synthetic data and on a real landmine data set. Experiments on both the synthetic data and the landmine data set show that an MI-HMM can achieve statistically significant performance gains when compared with the best existing HMM for the landmine detection problem, eliminate the ad hoc approaches in training set selection, and introduce a principled way to work with ambiguous time-series data.

This paper introduces a novel mixture of experts model, the Mixture of Hidden Markov Model Experts (MHMME). This model is designed to perform context-based classification of samples that are variable length sequences. The contexts are determined by the gates and the classifiers are determined by the experts. The gates and the experts are learned simultaneously using a single probabilistic model. Experimental results on landmine dataset show that MHMME significantly outperforms the HMM-based and ME-based models.

In this paper, we provide a comprehensive survey of the mixture of experts (ME). We discuss the fundamental models for regression and classification and also their training with the expectation-maximization algorithm. We follow the discussion with improvements to the ME model and focus particularly on the mixtures of Gaussian process experts. We provide a review of the literature for other training methods, such as the alternative localized ME training, and cover the variational learning of ME in detail. In addition, we describe the model selection literature which encompasses finding the optimum number of experts, as well as the depth of the tree. We present the advances in ME in the classification area and present some issues concerning the classification model. We list the statistical properties of ME, discuss how the model has been modified over the years, compare ME to some popular algorithms, and list several applications. We conclude our survey with future directions and provide a list of publicly available datasets and a list of publicly available software that implement ME. Finally, we provide examples for regression and classification. We believe that the study described in this paper will provide quick access to the relevant literature for researchers and practitioners who would like to improve or use ME, and that it will stimulate further studies in ME.

In this paper, we (1) provide a complete framework for classification using Variational Mixture of Experts (VME); (2) derive the variational lower bound; and (3) apply the method to landmine, or simply mine, detection and compare the results to the Mixtures of Experts trained with Expectation Maximization (EMME). VME has previously been used for regression and Waterhouse explained how to apply VME to classification (which we will call as VMEC). However, the steps to train the model were not made clear since the equations were applicable to vector valued parameters as opposed to matrices for each expert. Also, a variational lower bound was not provided. The variational lower bound provides an excellent stopping criterion that resists over-training. We demonstrate the efficacy of the method on real-world mine classification; in which, training robust mine classification algorithms is difficult because of the small number of samples per class. In our experiments VMEC consistently improved performance over EMME.

A variety of algorithms are presented and employed in a hierarchical fashion to discriminate both Anti-Tank (AT) and Anti-Personnel (AP) landmines using data collected from Wideband Electro-Magnetic Induction (WEMI) and Ground Penetrating Radar (GPR) sensors mounted on a robotic platform. The two new algorithms for WEMI are based on the In-phase vs. Quadrature plot (the Argand diagram) of the complex measurement obtained at a single spatial location. The Angle Prototype Match method uses the sequence of angles as a feature vector. Prototypes are constructed from these feature vectors and used to assign mine confidence to a test sample. The Angle Model Based KNN method uses a two parameter model; where the parameters are fit to the In-phase and Quadrature data. For the GPR data, the Linear Prediction Processing and Spectral Features are calculated. All four features from WEMI and GPR are used in a Hierarchical Mixture of Experts model to increase the landmine detection rate. The EM algorithm is used to estimate the parameters of the hierarchical mixture. Instead of a two way mine/non-mine decision, the HME structure is trained to make a five way decision which aids in the detection of the low metal anti personnel mines.