There is a strong push within the mining sector to develop and adopt automation technology, including autonomous vehicles such as excavators, trucks and drills. However, for autonomous systems to operate effectively in this domain, new perception capabilities are required to build rich models of a mine. A key element of this is an ability to sense and model the sub-surface geological structure as well as the more traditional robotic models, which typically estimate terrain and obstacles. This paper presents a new automated geological perception system to support autonomous mining. It uses hyperspectral imaging sensors and a supervised learning algorithm to detect and classify geological structures, and ultimately build a rich model of the operating environment. The presented algorithm uses Gaussian Processes (GPs) and an Observation Angle Dependent (OAD) covariance function. Further, the resulting geological model can be improved by fusing data from two hyperspectral scanners which measure different regions of the spectrum. The approach is demonstrated using data from an operational iron-ore mine. Fusion of classification results from the two sensors shows better agreement with ground truth mapping done in the field, compared to results from individual sensors.

Automated rock recognition is a key step for building a fully autonomous mine. When characterizing rock types from drill performance data, the main challenge is that there is not an obvious one-to-one correspondence between the two. In this paper, a hybrid rock recognition approach is proposed which combines Gaussian Process (GP) regression with clustering. Drill performance data is also known as Measurement While Drilling (MWD) data and a rock hardness measure - Adjusted Penetration Rate (APR) is extracted using the raw data in discrete drill holes. GP regression is then applied to create a more dense APR distribution, followed by clustering which produces discrete class labels. No initial labelling is needed. Comparisons are made with alternative measures of rock hardness from MWD data as well as state-of-the-art GP classification. Experimental results from an actual mine site show the effectiveness of our proposed approach.

This work is motivated by the need to develop new perception and modeling capabilities to support a fully autonomous, remotely operated mine. The application differs from most existing robotics research in that it requires a detailed world model of the sub-surface geological structure. This in-ground geological information is then used to drive many of the planning and control decisions made on a mine site. This paper formulates a method for automatically detecting in-ground geological boundaries using geophysical logging sensors and a supervised learning algorithm. The algorithm uses Gaussian Processes (GPs) and a single length scale squared exponential covariance function. The approach is demonstrated on data from a producing iron-ore mine in Australia. Our results show that two separate distinctive geological boundaries can be automatically identified with an accuracy of over 99 percent. The alternative approach to automatic detection involves manual examination of these data.

Obtaining a comprehensive model of large and complex terrain typically entails the use of both multiple sensory modalities and multiple data sets. This paper demonstrates the use of dependent Gaussian processes for data fusion in the context of large scale terrain modeling. Specifically, this paper derives and demonstrates the use of a non-stationary kernel (Neural Network) in this context. Experiments performed on multiple large scale (spanning about 5 sq km) 3D terrain data sets obtained from multiple sensory modalities (GPS surveys and laser scans) demonstrate the approach to data fusion and provide a preliminary demonstration of the superior modeling capability of Gaussian processes based on this kernel.

Autonomous operation of blast hole drill rigs requires monitoring of drilling parameters known as "Measurement While Drilling" (MWD) data. From these data, rock properties can be inferred. A supervised classification scheme is usually used to map MWD data inputs to rock type outputs given some labeled training data. However, the geology has no definite ground truth that can allow a reliable labeling of the training data, nor is there a clear input-output pair connection between the MWD data and the rock types. In this paper, an adaptive unsupervised approach is proposed to estimate the rock types in a data driven way by minimizing the entropy gradient of the characterizing measure "Optimized Adjusted Penetration Rate" (OAPR). Neither data labeling nor fixed model parameters are required because of the data driven nature of the algorithm. Experimental results illustrate the effectiveness of our solution.

In this paper we use a machine learning algorithm based on Gaussian Processes (GPs) and the Observation Angle Dependent (OAD) covariance function to classify hyper spectral imagery for the first time. This paper demonstrates the potential of the GP-OAD method for use in autonomous mining to identify and map geology and mineralogy on a vertical mine face. We discuss the importance of independent training data (i.e. a spectral library) to map any mine face without a priori knowledge. We compare an independent spectral library to other libraries, based on image data, and evaluate their relative performances to distinguish ore bearing zones from waste. Results show that the algorithm yields high accuracies (90%) and F-scores (77%), the best results are achieved when libraries are combined. We also demonstrate mapping of geology using imagery under different conditions of illumination (e.g. shade).

This paper develops an information (inverse-covariance) based method for efficient fusion and distributed estimation of large scale terrain. The output resembles a standard triangulated irregular network (TIN) terrain representation. However the proposed method uses distributed information fusion to estimate the elevations of the mesh vertices. This terrain mapping system is intended to use multiple scanning vehicles for online monitoring of the terrain for automated mining operations or other multi-vehicle field robotics systems. The method is based on a pre-specified regular finite-element mesh to define the set of estimated state variables. The method maintains a joint Gaussian distribution of the mesh vertices' elevations, in the information (inverse-covariance) form. The mesh elevations are estimated jointly given the irregular terrain observations, together with smoothness terms. The smoothness terms enable interpolation into unobserved regions as well as reducing noise. In the information form, the observations and smoothness terms are additive and the information matrix remains sparse in a fixed pattern, enabling constant-memory fusion of observations, efficient distribution among multiple sensing platforms and efficient solving for the estimates and uncertainty. Results show the reduction in data size for the fused observations compared to the raw observations, whilst still obtaining large scale high quality terrain maps. This paper focuses on a hierarchical distributed system in which each node estimates a subset of its parent's region, with the top-level node estimating a terrain map of the whole area. This paper compares two methods for the distributed communication from parent to child: An exact but expensive method, and an approximate fast method. Results compare the communication cost and resulting level of estimation approximation, showing that the marginalised information is expensive and the approximation is acceptable without it. This paper is applied to the estimation of large scale surface terrain from a distributed network of multiple sensors, such as 3D laser scanners, for automated terrain mapping for large scale mining applications.

This paper introduces a simple yet powerful data transformation strategy for kernel machines. Instead of adapting the parameters of the kernel function w.r.t. the given data (as in conventional methods), we adjust both the kernel hyper-parameters and the given data itself. Using this approach, the input data is transformed to be more representative of the assumptions encoded in the kernel function. A novel complex mapping is proposed to nonlinearly adjust the data. Optimization of the data transformation parameters is performed in two different manners. Firstly, the complex data mapping parameters and kernel hyper-parameters are selected separately, with the former guided by frequency metrics and the latter under the Bayesian framework. Next, the complex data mapping parameters and kernel hyper-parameters are optimized simultaneously in a Bayesian formulation by creating a new category of "integrated kernel" with the complex data mapping embedded. Experiments using Gaussian Process learning have shown that both methods improve the learning accuracy in either classification or regression tasks, with the complex mapping embedded kernel approach outperforming the separate complex mapping one.

Trajectory generation and control of large equipment in open field environments involves systematically and robustly operating in uncertain and dynamic terrain. This paper presents an integrated motion planning and control system for tracked vehicles. Flexible path-end adjustments and adaptive look-ahead are introduced to a state lattice planning approach with waypoint control. For a given processing horizon, this increases search coverage and reduces planning error. This tramming approach has been successfully fielded on a 98-ton autonomous blast hole drill rig used in iron ore mining in Western Australia. The system has undergone extensive testing and is now integrated into a production environment. This work is a key element in a larger program aimed at developing a fully autonomous, remotely operated mine.

This paper investigates the Centralized Multi-vehicle Simultaneous Localization and Mapping (CMSLAM) problem in the context of the nonlinear observability. Theory is first developed for the nonlinear observability of CMSLAM using the relatively simple unicycle vehicle model, which gives rise to a CMSLAM problem in control affine form. Conditions required for nonlinear observability of CMSLAM when estimating 1. One landmark and 2. More than one landmark are detailed. The theory developed is then extended for more practical car-like vehicle models. CMSLAM simulations and experiments are demonstrated showing the effects of nonlinear observability.

This paper presents a novel approach to data fusion for stochastic processes that model spatial data. It addresses the problem of data fusion in the context of large scale terrain modeling for a mobile robot. Building a model of large scale and complex terrain that can adequately handle uncertainty and incompleteness in a statistically sound manner is a very challenging problem. To obtain a comprehensive model of such terrain, typically, multiple sensory modalities as well as multiple data sets are required. This work uses Gaussian processes to model large scale terrain. The model naturally provides a multi-resolution representation of space, incorporates and handles uncertainties appropriately and copes with incompleteness of sensory information. Gaussian process regression techniques are applied to estimate and interpolate (to fill gaps in unknown areas) elevation information across the field. In this work, the GP modeling approach is extended to fuse multiple, multi-modal data sets to obtain a best estimate of the elevation given the individual data sets. The individual data sets are treated as different noisy samples of the same underlying terrain. Experiments performed on sparse GPS based survey data and dense laser scanner data taken at mine-sites are reported.

A crucial component of an autonomous mine is the ability to infer rock types from mechanical measurements of a drill rig. The major difficulty lies in that there is not a clear one to one correspondence between the mechanical measurements and the rock type due to the mechanical noise as well as the variety of the rock geology. This paper proposes a novel wavelet feature space projection approach to robustly classify rock types from drilling data with Gaussian Process classification. Instead of applying Gaussian Process classifier directly to the given measurement pieces, a group of wavelet features are extracted from the neighboring region of a specific data point. Gaussian Process classification is then carried out on the new extracted wavelet features. By putting neighboring data points into consideration rather than dealing with each data point individually, the underlying pattern can be better captured and more robust to noise and data variations. Experimental results on synthetic data as well as varied real world drilling data have shown the effectiveness of our approach.

A new method is presented which combines a deterministic analytical method and a probabilistic measure to classify rock types on the basis of their hyperspectral curve shape. This method is a supervised learning algorithm using Gaussian Processes (GPs) and the Observation Angle Dependent (OAD) covariance function. The OAD covariance function makes use of the properties of the Spectral Angle Mapper (SAM) which is used frequently for classifying hyperspectral data. Results show that it is possible to identify and classify rocks in an `One vs. One' and an `One vs. All' approach using the entire spectral curve (0.35-2.5 μm). The results show an average classification accuracy of 98% and an F-score of 92% for the new method in an `One vs. All' approach. Slightly higher classification accuracy and F-measure for the new method can be achieved for the `One vs. One' binary approach. This paper extends the ideas of the deterministic SAM method to a probabilistic framework and enables data fusion with similar and disparate kinds of sensors. This paper demonstrates a superior classification performance of the new probabilistic method over the classical SAM.

The theory of stochastic observability is vital in describing the performance of Simultaneous Localization and Mapping (SLAM) as a nonlinear stochastic state estimation problem quantifying effects of random noise on its observability. We show that the eigen space corresponding to the stochastically unobservable states of the state error covariance matrix of the SLAM problem initialized with unknown initial conditions are in the null space of the information matrix associated with observations of the SLAM problem. We establish by using theoretical analysis and examples that the stochastically unobservable directions of the SLAM state space can be changed by modifying the observation model of the SLAM problem. We then use simulations and experiments to show that stochastically observable directions of state space and their degree of stochastic observability can be modified as required in a particular application (such as surveying, mapping and surveillance) by changing the vehicle path with respect to the landmarks in the environment by selecting which landmarks to observe and by modifying the observation model.

Terrain modeling remains a challenging yet key component for the deployment of ground robots to the field. The difficulty arrives from the variability of terrain shapes, sparseness of the data, and high degree uncertainty often encountered in large, unstructured environments. This paper presents significant advances to data fusion for stochastic processes modeling spatial data, demonstrated in large-scale terrain modeling tasks. We explore dependent Gaussian processes to provide a multi-resolution representation of space and associated uncertainties, while integrating sensors from different modalities. Experiments performed on multiple multi-modal datasets (3D laser scans and GPS) demonstrate the approach for terrains of about 5 km2.

This paper addresses the problem of large scale terrain modeling for a mobile robot. Building a model of large scale terrain data that can adequately handle uncertainty and incompleteness in a statistically sound way is a very challenging problem. This work proposes the use of Gaussian processes as models of large scale terrain. The proposed model naturally provides a multi-resolution representation of space, incorporates and handles uncertainties aptly and copes with incompleteness of sensory information. Gaussian process regression techniques are applied to estimate and interpolate (to fill gaps in unknown areas) elevation information across the field. The estimates obtained are the best linear unbiased estimates for the data under consideration. A single non-stationary (neural network) Gaussian process is shown to be powerful enough to model large and complex terrain, handling issues relating to discontinuous data effectively. A local approximation methodology based on KD-trees is also proposed in order to ensure local smoothness and yet preserve the characteristic features of rich and complex terrain data. The use of the local approximation technique based on KD-trees further addresses concerns relating to the scalability of the proposed approach for large data sets. Experiments performed on sparse GPS based survey data as well as dense laser scanner data taken at different mine-sites are reported in support of these claims.

The theory of nonlinear observability is an important tool available for the assessment of highly nonlinear estimation problems such as simultaneous localization and mapping (SLAM). It is shown that all the estimated landmarks must be observed and at least two a priori known landmarks be observed for the nonlinear observability of single vehicle SLAM when estimating any number of unknown landmark locations. The relationship between the information form of SLAM and the nonlinear observability is established. It is shown that when the nonlinear observability conditions are satisfied the single vehicle SLAM problem can in theory be initialized with infinitely large initial uncertainties. Simulations and experiments are also provided to substantiate the theoretical results.

This paper presents the system architecture of a real time multi-UAV simulator (RMUS). The simulator has been implemented as both a testing and validation mechanism for the real demonstration of multiple UAVs conducting both decentralised data fusion and control. These mechanisms include the off-line simulation of complex scenarios, hardware-in-the-loop tests, validation of real test results, and online mission control system demonstrations. The paper also present CommLibX, a novel communication framework for the system which allows simulation modules to communicate over single or multiple virtual channels. This unique communication system is then easily ported onto the real hardware allowing for maximum reuse of software and integrity.

This paper describes a series of trials that were done at an underground mine in New South Wales, Australia. Experimental results are presented from the data obtained during the field trials and suitable sensor suites for an autonomous mining vehicle navigation system are evaluated.