Convolutional neural networks (CNNs) have been shown to be a powerful tool for image classification. Recently, they have been adopted into the remote sensing community with applications in material classification from hyperspectral images. However, CNNs are time-consuming to train and often require large amounts of labeled training data. The widespread use of CNNs in the image processing and computer vision communities has been facilitated by the networks that have already been trained on large amounts of data. These pretrained networks can be used to initialize networks for new tasks. This transfer of knowledge makes it far less time-consuming to train a new classifier and reduces the need for a large labeled data set. This concept of transfer learning has not yet been fully explored by those using CNNs to train material classifiers from hyperspectral data. This paper provides an insight into training hyperspectral CNN classifiers by transferring knowledge from well labeled data sets to data sets that are less well labeled. It is shown that these CNNs can transfer between completely different domains and sensing platforms, and still improve classification performance. The application of this work is in the training of material classifiers of data acquired from field-based platforms, by transferring knowledge from publicly accessible airborne data sets. Factors, such as training set size, CNN architectures, and the impact of filter width and wavelength interval, are studied.

This paper proposes the Relit Spectral Angle-Stacked Autoencoder, a novel unsupervised feature learning approach for mapping pixel reflectances to illumination invariant encodings. This work extends the Spectral Angle-Stacked Autoencoder so that it can learn a shadow-invariant mapping. The method is inspired by a deep learning technique, Denoising Autoencoders, with the incorporation of a physics-based model for illumination such that the algorithm learns a shadow invariant mapping without the need for any labelled training data, additional sensors, a priori knowledge of the scene or the assumption of Planckian illumination. The method is evaluated using datasets captured from several different cameras, with experiments to demonstrate the illumination invariance of the features and how they can be used practically to improve the performance of high-level perception algorithms that operate on images acquired outdoors.

The illumination conditions of a scene create intra-class variability in outdoor visual data, degrading the performance of high-level algorithms. Using only the image, and with hyper-spectral data as a case study, this paper proposes a deep learning approach to learn illumination invariant features from the data in an unsupervised manner. The proposed approach incorporates a similarity measure, the Spectral Angle, that is relatively insensitive to brightness into the cost function of a Stacked Auto-Encoder so that an illumination invariant mapping is learned from the input data to the hidden layer. Experiments using synthetic and real imagery show that this novel feature learning approach produces a more illumination invariant representation of the data, improving the results of a high-level algorithm (clustering) under such conditions.

Some spectral unmixing methods incorporate endmember variability within endmember classes. It is, however, uncertain whether these methods work well when endmember spectra do not completely describe the variability that exists within endmember classes. This paper proposes a novel spectral unmixing method, Spectral Unmixing within a multi-task Gaussian Process framework (SUGP), which is more resistant to problems caused by the use of a small number of endmember spectra. SUGP models the latent function between spectra and abundances in a training set and predicts abundances from a given pixel spectrum. SUGP is different from existing methods in that it incorporates all spectra within each endmember class to estimate abundances within a probabilistic framework. Using simulated data, SUGP was compared with existing linear unmixing methods and was found to be superior in determining the number of endmember classes within each pixel and in estimating abundances. It was also more effective in cases where a small number of spectra within endmember classes were specified and was more resistant to the effects of spectral noise. Methods were applied to the hyperspectral imagery of a mine wall and to imagery acquired over Cuprite, Nevada. Abundance maps generated by SUGP were consistent with the validated reference maps. SUGP opens up possibilities for estimating accurate abundances under conditions where endmember variability is present and where endmember spectra incompletely describe the true variability of each endmember class.

Spectral variability, unrelated to the purity of endmembers, can change the geometry of the dataspace and affect conventional methods used to identify endmembers. Several methods have been developed to identify and extract endmember bundles representing the spectral variability within each endmember class. These methods, however, operate on the geometry of the dataspace. In addition, they commonly use k-means clustering that requires a priori the number of endmember classes present in a scene and may fail to group endmember spectra representing spectral variability within each class. This paper introduces a novel approach, spectral curve-based endmember extraction (SCEE), which allows for the extraction and clustering of multiple spectra representing spectral variability within endmember classes. The significant differences between SCEE and conventional methods are: i) SCEE is based on the shape of a spectral curve, not the geometry of the data simplex; and ii) SCEE extracts multiple endmember bundle candidates representing a particular class, without a priori knowledge of the number of endmember classes in a scene. Once multiple endmember bundle candidates are identified, they are automatically grouped by sequential pairwise clustering in order to determine the final number of endmember classes. The performance of SCEE is compared with that of other state-of-the-art endmember bundle extraction methods using simulated data and hyperspectral imagery of a mine pit and Cuprite. Results showed that multiple endmember bundles identified by SCEE gave better matches with spectral variability of reference spectra than those by other methods and were better able to encompass the range of variability within each class.

Incorporating endmember variability and spatial information into spectral unmixing analyses is important for producing accurate abundance estimates. However, most methods do not incorporate endmember variability with spatial regularization. This paper proposes a novel 2-step unmixing approach, which incorporates endmember variability and spatial information. In step 1, a probability distribution representing abundances is estimated by spectral unmixing within a multi-task Gaussian process framework (SUGP). In step 2, spatial information is incorporated into the probability distribution derived by SUGP through an a priori distribution derived from a Markov random field (MRF). The proposed method (SUGP-MRF) is different to the existing unmixing methods because it incorporates endmember variability and spatial information at separate steps in the analysis and automatically estimates parameters controlling the balance between the data fit and spatial smoothness. The performance of SUGP-MRF is compared with the existing unmixing methods using synthetic imagery with precisely known abundances and real hyperspectral imagery of rock samples. Results show that SUGP-MRF outperforms the existing methods and improves the accuracy of abundance estimates by incorporating spatial information.

Many economically important minerals have absorption features in the short-wave infrared (SWIR; 2000-2500 nm). Sensors which measure this part of the spectrum cannot detect the wavelength minimum of a feature at '900 nm (F900), indicative of ferric iron mineralogy. A method based on Gaussian processes (GPs) was developed and compared with multiple linear regression (MLR) to estimate the wavelength position of F900 from SWIR data (1002-1355 nm). SWIR data with different signal-to-noise ratios were acquired from crushed rock samples by a nonimaging spectrometer and an imaging spectrometer. GP estimates of wavelength position were converted to the proportion of goethite using coefficients from a regression of the proportion of goethite determined from X-ray diffraction (XRD) on wavelength position measured directly from spectra. GP-estimated wavelength positions were within the 2-nm and '4-nm root-mean-square error of measurements made directly from spectra for nonimaging and imaging spectrometer data, respectively. Proportions of goethite derived from these estimates were respectively within 4% and 6% of the values measured by XRD. MLR performed poorly compared to GPs when applied to data with no added noise and failed when applied to data with added noise or to imaging spectrometer data. These findings indicate that the wavelength position of F900-an indicator of ferric iron mineralogy-can be estimated from data acquired at SWIR wavelengths (1002- 1355 nm). This opens up possibilities for using a single (SWIR) sensor to acquire information on ferric iron mineralogy (using F900) and other minerals with diagnostic absorptions between 1000 and 2500 nm.

A novel spectral unmixing technique is presented which addresses the problem of spectral variability within each endmember class and determines endmember types present in each pixel. The proposed unmixing method is a multi-task framework, based on Multi-task Gaussian Process (MTGP). The Unmixing within a MTGP framework (UMTGP) is different to conventional unmixing approaches in that it assumes that spectral variation exists within each endmember class. Using synthetic and real data, the fractional abundances estimated by the UMTGP are compared with conventional methods such as Fully Constrained Least Squares (FCLS) and Multiple Endmember Spectral Mixture Analysis (MESMA). Hyperspectral data acquired from field-based platforms are used for evaluation because intra-class spectral variability is commonly large in these datasets. The results show that the UMTGP outperforms FCLS in terms of estimating fractional abundance and provides better estimates than MESMA, especially when a small number of endmember spectra for each class are available.

Several environmental and sensor effects make the determination of the wavelength position of absorption features in the visible near infrared (VNIR) (400-1200 nm) from hyperspectral imagery more difficult than from nonimaging spectrometers. To evaluate this, we focus on the ferric iron crystal field absorption, located at about 900 nm (F900), because it is impacted by both environmental and sensor effects. The consistency with which the wavelength position of F900 can be determined from imagery acquired in laboratory and field settings is evaluated under artificial and natural illumination, respectively. The wavelength position of F900, determined from laboratory imagery, is also evaluated as an indicator of the proportion of goethite in mixtures of crushed rock. Results are compared with those from a high-resolution field spectrometer. Images describing the wavelength position of F900 showed large amounts of spatial variability and contained an artifact-a consistent shift in the wavelength position of F900 to longer wavelengths. These effects were greatly reduced or removed when wavelength position was determined from a polynomial fit to the data, enabling wavelength position to be used to map hematite and goethite in samples of ore and on a vertical surface (a mine face). The wavelength position of F900 from a polynomial fit was strongly positively correlated with the proportion of goethite (R2=0.97). Taken together, these findings indicate that the wavelength position of absorption features from VNIR imagery should be determined from a polynomial (or equivalent) fit to the original data and not from the original data themselves.

This paper presents an approach to automatically register hyperspectral images with lidar point clouds using a combination of SIFT and SURF feature descriptors. The aim is to generate 3D terrain maps of the environment combining spectral and geometrical information. The datasets are acquired from field-based platforms which, due to the lack of georeferencing, cannot be simply fused and require a registration processing step. In addition, some applications, such as in mining, cannot rely on reliable GPS signal. The proposed method is validated using experimental data acquired from vertical mine walls.

There is a strong push within the mining sector to develop and adopt automation technology, including autonomous vehicles such as excavators, trucks and drills. However, for autonomous systems to operate effectively in this domain, new perception capabilities are required to build rich models of a mine. A key element of this is an ability to sense and model the sub-surface geological structure as well as the more traditional robotic models, which typically estimate terrain and obstacles. This paper presents a new automated geological perception system to support autonomous mining. It uses hyperspectral imaging sensors and a supervised learning algorithm to detect and classify geological structures, and ultimately build a rich model of the operating environment. The presented algorithm uses Gaussian Processes (GPs) and an Observation Angle Dependent (OAD) covariance function. Further, the resulting geological model can be improved by fusing data from two hyperspectral scanners which measure different regions of the spectrum. The approach is demonstrated using data from an operational iron-ore mine. Fusion of classification results from the two sensors shows better agreement with ground truth mapping done in the field, compared to results from individual sensors.

Hyperspectral data acquired from field-based platforms present new challenges for their analysis, particularly for complex vertical surfaces exposed to large changes in the geometry and intensity of illumination. The use of hyperspectral data to map rock types on a vertical mine face is demonstrated, with a view to providing real-time information for automated mining applications. The performance of two classification techniques, namely, spectral angle mapper (SAM) and support vector machines (SVMs), is compared rigorously using a spectral library acquired under various conditions of illumination. SAM and SVM are then applied to a mine face, and results are compared with geological boundaries mapped in the field. Effects of changing conditions of illumination, including shadow, were investigated by applying SAM and SVM to imagery acquired at different times of the day. As expected, classification of the spectral libraries showed that, on average, SVM gave superior results for SAM, although SAM performed better where spectra were acquired under conditions of shadow. In contrast, when applied to hypserspectral imagery of a mine face, SVM did not perform as well as SAM. Shadow, through its impact upon spectral curve shape and albedo, had a profound impact on classification using SAM and SVM.

In this paper we use a machine learning algorithm based on Gaussian Processes (GPs) and the Observation Angle Dependent (OAD) covariance function to classify hyper spectral imagery for the first time. This paper demonstrates the potential of the GP-OAD method for use in autonomous mining to identify and map geology and mineralogy on a vertical mine face. We discuss the importance of independent training data (i.e. a spectral library) to map any mine face without a priori knowledge. We compare an independent spectral library to other libraries, based on image data, and evaluate their relative performances to distinguish ore bearing zones from waste. Results show that the algorithm yields high accuracies (90%) and F-scores (77%), the best results are achieved when libraries are combined. We also demonstrate mapping of geology using imagery under different conditions of illumination (e.g. shade).

Feature selection is an important step in hyperspectral analysis using machine learning for many applications, in particular to avoid the curse of dimensionality when there is limited available ground truth. This paper presents an approach to select hyperspectral bands using boosting. Boosting decision trees is an efficient and accurate classification technique that has been applied successfully to process hyperspectral data. The learned structure of the trees can provide insight about which bands are more relevant for the classification. We develop a method that takes into account the improvement obtained by each split of the tree ensemble and calculates a relative importance measure of the input features. The method was evaluated using hyperspectral data of rock samples from an iron ore mine in Australia. We show that by retaining only the most relevant features it is possible to reduce the computational load while retaining classification performance.

A new method is presented which combines a deterministic analytical method and a probabilistic measure to classify rock types on the basis of their hyperspectral curve shape. This method is a supervised learning algorithm using Gaussian Processes (GPs) and the Observation Angle Dependent (OAD) covariance function. The OAD covariance function makes use of the properties of the Spectral Angle Mapper (SAM) which is used frequently for classifying hyperspectral data. Results show that it is possible to identify and classify rocks in an `One vs. One' and an `One vs. All' approach using the entire spectral curve (0.35-2.5 μm). The results show an average classification accuracy of 98% and an F-score of 92% for the new method in an `One vs. All' approach. Slightly higher classification accuracy and F-measure for the new method can be achieved for the `One vs. One' binary approach. This paper extends the ideas of the deterministic SAM method to a probabilistic framework and enables data fusion with similar and disparate kinds of sensors. This paper demonstrates a superior classification performance of the new probabilistic method over the classical SAM.

This paper investigates the performance of machine learning methods for classifying rock types from hyperspectral data. The main objective is to test the impact on classification error rate of calibrating the model's output into class probability estimates. The base classifiers included in this study are: boosted decision trees, support vector machines and logistic regression. The standard algorithm for some of these methods provides a non-probabilistic, hard decision as output. For those methods, posterior class probability estimates were approximated by fitting a sigmoid function to the classifier predictions. To perform multi-class classification, a one-versus-all approach was used. The different methods were compared using hyperspectral data acquired from ore-bearing rocks under different environmental conditions. The calibration of class probabilities improved the overall performance for almost all algorithms tested; an improvement of over 10% was observed in some cases.