Deploying autonomous systems in safety critical settings necessitates methods to verify their safety properties. This is challenging because real-world systems may be subject to disturbances that affect their performance, but are unknown a priori. This work develops a safety-verification strategy wherein data is collected online and incorporated into a reachability analysis approach to check in real-time that the system avoids dangerous regions of the state space. Specifically, we employ an optimization-based moving horizon estimator (MHE) to characterize the disturbance affecting the system, which is incorporated into an online reachability calculation. Reachable sets are calculated using a computational graph analysis tool to predict the possible future states of the system and verify that they satisfy safety constraints. We include theoretical arguments proving our approach generates reachable sets that bound the future states of the system, as well as numerical results demonstrating how it can be used for safety verification. Finally, we present results from hardware experiments demonstrating our approach’s ability to perform online reachability calculations for an unmanned surface vehicle subject to currents and actuator failures.

Deep neural networks provide deep extracted features for image classification. As a high dimension data, hyperspectral image (HSI) feature extraction is unlike an RGB image whose feature representation could not be simply generated in the spatial domain. To take full advantage of HSI, a dual-channel convolutional neural network (CNN) is applied, 1D convolution for the spectral domain and 2D convolution for spatial domain. For pixel-wise classification of HSI, in our network model, one-dimensional customized DenseNet is for extracting the hierarchical spectral features and another customized DenseNet is applied to extract the hierarchical spatial-related feature. Furthermore, we experimentally tuned the several widen factors and dense-net growth rates to evaluate the impact of hyper-parameter. To compare our proposed method with HSI classification methods, we test other three DNNs based method in two real-world HSI dataset. The result demonstrated our approach outperformed the state-of-art method.

The framework consisting of a pixel-wise classification followed by a Markov random field has been very successful for spatial-spectral hyperspectral classification. While training such frameworks, the classifier and the Markov random field are generally tuned greedily one after another. However, better results could be obtained by tuning both of the components simultaneously with the objective of producing the best result at the end. This paper investigates the joint optimization of the hyperparameters of the classifier and the random field using Bayesian optimization. Experimental evaluation on the model comprising of a support vector machine classifier and a grid-structured Markov random field is provided. The results of the experiments, conducted on two independent datasets, suggest that the jointly tuned models can provide better accuracy.

Automatic recognition for complex scenes from aerial images and other sensor data (e.g. LiDAR) has become an active topic in the remote sensing community. In this paper, we proposed a novel framework that utilizes higher-order CRFs (HCRFs) to capture the spatial contextual information for the RGB aerial images along with their co-registered LiDAR data (DSMs). Our proposed HCRFs framework exploits the spatial contextual information in two levels. The first level encourages harmonic label co-existence within one segment, which can be generated by an unsupervised superpixel algorithm. The second level takes into account the local object co-occurrence among adjacent segments. We then show that how to apply the move making graph cuts algorithm to perform efficient inference for our proposed CRFs framework. Based on the experiments on a challenging data set, our proposed higher-order CRF framework generated state-of-the-art semantic segmentation results.

Remotely extracting information about the biochemical properties of the materials in an environment from airborne- or satellite-based hyperspectral sensor has a variety of applications in forestry, agriculture, mining, environmental monitoring and space exploration. In this paper, we propose a new non-stationary covariance function, called exponential spectral angle mapper (ESAM) for predicting the biochemistry of vegetation from hyperspectral imagery using Gaussian processes. The proposed covariance function is based on the angle between the spectra, which is known to be a better measure of similarity for hyperspectral data due to its robustness to illumination variations. We demonstrate the efficacy of the proposed method with experiments on a real-world hy-perspectral dataset.

Classification plays a significant role in analyzing remotely sensed imagery. In order to obtain an optimized classier, following aspects are rather challenging: 1) complexity in dealing with the overwhelming amount of data information from an advanced high resolution hyperspectral imaging sensor; 2) difficulty in leveraging spectral and spatial information across the sensed wavelengths; 3) struggles in obtaining adequate dataset as in the same modalities with labeled ground truth in the training process. Therefore, we propose a novel classification approach to tackle these issues by utilizing probabilistic graphical model on super-pixel segmentation. This method is capable of compacting hyperspectral information efficiently which decreases computing complexity. Moreover, the employment of probabilistic graphical models that weighs the strong dependency in spatial and spectral neighbors improves accuracy. One of the most successful probabilistic graphical models is Conditional Random Fields (CRFs). Conventional methods utilize all spectral bands and assign the corresponding raw intensity values into the feature functions in CRFs and build the grid graph. These methods, however, require significant computational efforts and yield an ambiguous summary from the data. To mitigate these problems, we cooperate a non-linear kernel based classier to provide the meaningful probability features for CRFs and learn the non-grid graph from super pixel segmentation.

Object detection in high resolution remote sensing images is a crucial yet challenging problem for many applications. With the development of satellite and sensor technologies, remote sensing images attain very high spatial resolution, giving rise to the employment of many computer vision algorithms. Therefore, the object detection is usually formalized as a supervised classification task. In this paper, we propose to apply the Gaussian process (GP) classification algorithm for our detection problem. Among different classifiers, the GP classifier is a Bayesian classification method that is able to make estimations in a probabilistic way. To demonstrate the performance of the proposed approach, we experiment the proposed framework with different feature extraction schemes and classification methods. We carry out a cross-validation experiment over an image dataset that consists of objects and non-objects to train an object detector, and apply the trained detector in an unobserved image scene to search for the objects of interest. Our results show that the GP classifier is competitive to support vector machines (SVM), which is considered state-of-the-art.

Vehicle detection and recognition from aerial imagery provides useful information for local vehicle volume estimation and traffic monitoring. In this paper, we propose a method that accurately detects vehicles in urban environment using a probabilistic classification method followed by a refinement based on object segments. Both classification and segmentation methods make use of coregistered aerial RGB images and airborne LiDAR data. Pixel-wise vehicle probability estimation is achieved using Gaussian process (GP) classification and object segments are obtained by applying a gradient based segmentation algorithm (GSEG). The vehicle is then detected by refining the initial probability estimation with the following constraints: car size, statistical significance and 3D surface shape. Experimental results show our method achieves 90.8% precision and 93.7% recall, which outperforms the ones that only use size constraints.

Attempting to understand and characterize trends in the stock market has been the goal of numerous market analysts, but these patterns are often difficult to detect until after they have been firmly established. Recently, attempts have been made by both large companies and individual investors to utilize intelligent analysis and trading algorithms to identify potential trends before they occur in the market environment, effectively predicting future stock values and outlooks. In this paper, three different classification algorithms will be compared for the purposes of maximizing capital while minimizing risk to the investor. The main contribution of this work is a demonstrated improvement over other prediction methods using machine learning; the results show that tuning support vector machine parameters with particle swarm optimization leads to highly accurate (approximately 95%) and robust stock forecasting for historical datasets.

Several environmental and sensor effects make the determination of the wavelength position of absorption features in the visible near infrared (VNIR) (400-1200 nm) from hyperspectral imagery more difficult than from nonimaging spectrometers. To evaluate this, we focus on the ferric iron crystal field absorption, located at about 900 nm (F900), because it is impacted by both environmental and sensor effects. The consistency with which the wavelength position of F900 can be determined from imagery acquired in laboratory and field settings is evaluated under artificial and natural illumination, respectively. The wavelength position of F900, determined from laboratory imagery, is also evaluated as an indicator of the proportion of goethite in mixtures of crushed rock. Results are compared with those from a high-resolution field spectrometer. Images describing the wavelength position of F900 showed large amounts of spatial variability and contained an artifact-a consistent shift in the wavelength position of F900 to longer wavelengths. These effects were greatly reduced or removed when wavelength position was determined from a polynomial fit to the data, enabling wavelength position to be used to map hematite and goethite in samples of ore and on a vertical surface (a mine face). The wavelength position of F900 from a polynomial fit was strongly positively correlated with the proportion of goethite (R2=0.97). Taken together, these findings indicate that the wavelength position of absorption features from VNIR imagery should be determined from a polynomial (or equivalent) fit to the original data and not from the original data themselves.

This paper presents an approach to automatically register hyperspectral images with lidar point clouds using a combination of SIFT and SURF feature descriptors. The aim is to generate 3D terrain maps of the environment combining spectral and geometrical information. The datasets are acquired from field-based platforms which, due to the lack of georeferencing, cannot be simply fused and require a registration processing step. In addition, some applications, such as in mining, cannot rely on reliable GPS signal. The proposed method is validated using experimental data acquired from vertical mine walls.

Robust manipulation with tractability in unstructured environments is a prominent hurdle in robotics. Learning algorithms to control robotic arms have introduced elegant solutions to the complexities faced in such systems. A novel method of Reinforcement Learning (RL), Gaussian Process Dynamic Programming (GPDP), yields promissing results for closed-loop control of a low-cost manipulator however research surrounding most RL techniques lack breadth of comparable experiments into the viability of particular learning techniques on equivalent environments. We introduce several model-based learning agents as mechanisms to control a noisy, low-cost robotic system. The agents were tested in a simulated domain for learning closed-loop policies of a simple task with no prior information. Then, the fidelity of the simulations is confirmed by application of GPDP to a physical system.

Automated rock recognition is a key step for building a fully autonomous mine. When characterizing rock types from drill performance data, the main challenge is that there is not an obvious one-to-one correspondence between the two. In this paper, a hybrid rock recognition approach is proposed which combines Gaussian Process (GP) regression with clustering. Drill performance data is also known as Measurement While Drilling (MWD) data and a rock hardness measure - Adjusted Penetration Rate (APR) is extracted using the raw data in discrete drill holes. GP regression is then applied to create a more dense APR distribution, followed by clustering which produces discrete class labels. No initial labelling is needed. Comparisons are made with alternative measures of rock hardness from MWD data as well as state-of-the-art GP classification. Experimental results from an actual mine site show the effectiveness of our proposed approach.

Hyperspectral data acquired from field-based platforms present new challenges for their analysis, particularly for complex vertical surfaces exposed to large changes in the geometry and intensity of illumination. The use of hyperspectral data to map rock types on a vertical mine face is demonstrated, with a view to providing real-time information for automated mining applications. The performance of two classification techniques, namely, spectral angle mapper (SAM) and support vector machines (SVMs), is compared rigorously using a spectral library acquired under various conditions of illumination. SAM and SVM are then applied to a mine face, and results are compared with geological boundaries mapped in the field. Effects of changing conditions of illumination, including shadow, were investigated by applying SAM and SVM to imagery acquired at different times of the day. As expected, classification of the spectral libraries showed that, on average, SVM gave superior results for SAM, although SAM performed better where spectra were acquired under conditions of shadow. In contrast, when applied to hypserspectral imagery of a mine face, SVM did not perform as well as SAM. Shadow, through its impact upon spectral curve shape and albedo, had a profound impact on classification using SAM and SVM.

Hyperspectral image data has great potential to identify and classify the chemical composition of materials remotely. Factors limiting the use of hyperspectral sensors in practical land-based applications, such as robotics and mining, are the complexity and cost of data acquisition, and the processing time required for the subsequent analysis. This is mainly due to the high dimensional and high volume nature of hyperspectral image data. In this paper, we propose to combine a feature selection method, based on particle swarm optimization (PSO), with a kernel method, support vector machines (SVM), to reduce the dimensionality of hyperspectral data for classification. We evaluate several different kernels, including some optimized for hyperspectral analysis. In particular, a recent kernel called observation angle dependent (OAD) kernel, originally designed for Gaussian Process regression, was extended for SVM classification. The SVM with the optimized kernel was then applied to induce the feature selection of a binary version of PSO. We validate the method using hyperspectral data sets acquired of rock samples from Western Australia. The empirical results demonstrate that our method is able to efficiently reduce the number of features while keeping, or even improving, the performance of the SVM classifier.

Feature selection is an important step in hyperspectral analysis using machine learning for many applications, in particular to avoid the curse of dimensionality when there is limited available ground truth. This paper presents an approach to select hyperspectral bands using boosting. Boosting decision trees is an efficient and accurate classification technique that has been applied successfully to process hyperspectral data. The learned structure of the trees can provide insight about which bands are more relevant for the classification. We develop a method that takes into account the improvement obtained by each split of the tree ensemble and calculates a relative importance measure of the input features. The method was evaluated using hyperspectral data of rock samples from an iron ore mine in Australia. We show that by retaining only the most relevant features it is possible to reduce the computational load while retaining classification performance.

A crucial component of an autonomous mine is the ability to infer rock types from mechanical measurements of a drill rig. The major difficulty lies in that there is not a clear one to one correspondence between the mechanical measurements and the rock type due to the mechanical noise as well as the variety of the rock geology. This paper proposes a novel wavelet feature space projection approach to robustly classify rock types from drilling data with Gaussian Process classification. Instead of applying Gaussian Process classifier directly to the given measurement pieces, a group of wavelet features are extracted from the neighboring region of a specific data point. Gaussian Process classification is then carried out on the new extracted wavelet features. By putting neighboring data points into consideration rather than dealing with each data point individually, the underlying pattern can be better captured and more robust to noise and data variations. Experimental results on synthetic data as well as varied real world drilling data have shown the effectiveness of our approach.

This paper investigates the performance of machine learning methods for classifying rock types from hyperspectral data. The main objective is to test the impact on classification error rate of calibrating the model's output into class probability estimates. The base classifiers included in this study are: boosted decision trees, support vector machines and logistic regression. The standard algorithm for some of these methods provides a non-probabilistic, hard decision as output. For those methods, posterior class probability estimates were approximated by fitting a sigmoid function to the classifier predictions. To perform multi-class classification, a one-versus-all approach was used. The different methods were compared using hyperspectral data acquired from ore-bearing rocks under different environmental conditions. The calibration of class probabilities improved the overall performance for almost all algorithms tested; an improvement of over 10% was observed in some cases.

This paper presents a ground based system for mapping the geology and the geometry of the environment remotely. The main objective of this work is to develop a framework for a mobile robotic platform that can build 3D geological maps. We investigate classification and registration algorithms that can work without any manual intervention. The system capabilities are demonstrated with data acquired from a working mine environment. Geological maps are built by applying classification techniques to hyperspectral images of the rocks' surface. The result from the classification is then fused with laser images to form the 3D geological models of the environment.

Measurement-while-drilling (MWD) data recorded from drill rigs can provide a valuable estimation of the type and strength of the rocks being drilled. Typical MWD sensors include bit pressure, rotation pressure, pull-down pressure, pull-down rate, and head speed. This letter presents an empirical comparison of the statistical performance, ease of implementation, and computational efficiency associated with three machine-learning techniques. A recently proposed method, boosting, is compared with two well-established methods, neural networks and fuzzy logic, used as benchmarks. MWD data were acquired from blast holes at an iron ore mine in Western Australia. The boreholes intersected a number of rock types including shale, iron ore, and banded iron formation. Boosting and neural networks presented the best performance overall. However, from the viewpoint of implementation simplicity and computational load, boosting outperformed the other two methods.

Analysis of drill performance data provides a powerful method for estimating subsurface geology. While there have been studies relating such measurement-while-drilling (MWD) parameters to rock properties, none of them has attempted to model context, that is, to associate local measurements with measurements obtained in neighbouring regions. This paper proposes a novel approach to infer geology from drill measurements by incorporating spatial relationships through a Conditional Random Field (CRF) framework. A boosting algorithm is used as a local classifier mapping drill measurements to corresponding geological categories. The CRF then uses this local information in conjunction with neighbouring measurements to jointly reason about their categories. Model parameters are learned from training data by maximizing the pseudo-likelihood. The probability distribution of classified borehole sections is calculated using belief propagation. We present experimental results of applying the method to MWD data collected from a semi-autonomous drill rig at an iron ore mine in Western Australia.

This paper investigates the application of artificial neural networks for classifying grass species from hyperspectral image data. High-resolution spatial and spectral data of localized fields were collected using a hyperspectral sensor mounted on the tip of a crane. The hyperspectral datasets are processed using normalization and second derivative in order to reduce the effect of variations in the intensity level of reflectance and to improve the classification accuracy and generalization performance of the neural network-based model. An experimental comparison of the pre-processing methods shows that the best classification accuracy is obtained by the second derivative transformed dataset. Normalization, and a combination of both methods, did not improve accuracy of the neural network models of our experimental datasets more than simple raw reflectance.

In this paper, a feature selection algorithm based on particle swarm optimization for processing remotely ac quired hyperspectral data is presented. Since particle swarm optimization was originally developed to search only continuous spaces, it could not deal with the problem of spectral band selection directly. We propose a method utilizing two swarms of particles in order to optimize simultaneously a desired performance criterion and the number of selected features. The candidate feature sets were evaluated on a regression problem using artificial neural networks to construct nonlinear models of chemical concentration of glucose in soybean crops. Experimen tal results attesting the viability of the method utilizing real world hyperspectral data are presented. The particle swarm optimization-based approach presented superior performance in comparison with a conventional feature extraction method.

Feature selection is necessary to reduce the dimensionality of spectral image data. Particle swarm optimization was originally developed to search only continuous spaces and, although many applications on discrete spaces had been proposed, it could not tackle the problem of feature selection directly. We developed a formulation utilizing two particles swarms in order to optimize a desired performance criterion and the number of selected features, simultaneously. Candidate feature sets were evaluated on a regression problem modeled using neural networks, which were trained to construct models of chemical concentration of glucose in soybeans. We present experimental results utilizing real-world spectral image data to attest the viability of the method. The particle swarms approach presented superior performance for linear modeling of chemical contents when compared to a conventional feature extraction method.

In this paper, we investigate a hyperspectral imagery data processing method to predict the sweetness and amino acid content of green vegetal soybean crops. Regression models based on neural networks were developed in order to calculate the level of sucrose, glucose, and nitrogen concentration, which can be related to sweetness and amino acid concentration of vegetables. We demonstrate the method using hyperspectral data of wavelengths ranging from the visible to the near infrared acquired from an experimental field of green vegetal soybeans. A performance analysis is reported comparing regression models built using datasets pre-processed using the first and second derivative analysis. The second derivative transformed dataset presented the best performance overall. Glucose could be predicted with greater accuracy.