Despite the continual advances in Advanced Driver Assistance Systems (ADAS) and the development of high-level autonomous vehicles (AV), there is a consensus that for the short to medium term, there is a requirement for a human supervisor to handle the edge cases that inevitably arise. Given this requirement, the state of the autonomous vehicle operator (referred to as the safety driver) must be monitored to ensure their contribution to the vehicle's safe operation. This paper introduces a dual-source approach integrating data from an infrared camera facing the safety driver and vehicle perception systems to produce a metric for safety driver alertness to promote and ensure safe operator behaviour. The infrared camera detects the safety driver’s head, enabling the calculation of head orientation, which is relevant as the head typically moves according to the individual's focus of attention. By incorporating environmental data from the perception system, it becomes possible to determine whether the safety driver observes objects in the surroundings. Experiments were conducted using data collected in Sydney, Australia, simulating AV operations in an urban environment. Our results demonstrate that the proposed system effectively determines a metric for the attention levels of the safety driver, enabling interventions such as warnings or reducing autonomous functionality as appropriate. The results indicate reduced awareness on subsequent laps during the study, demonstrating the "automation complacency" phenomenon. This comprehensive solution shows promise in contributing to ADAS and AVs’ overall safety and efficiency in a real-world setting.

Every autonomous driving dataset has a different configuration of sensors, originating from distinct geographic regions and covering various scenarios. As a result, 3D detectors tend to overfit the datasets they are trained on. This causes a drastic decrease in accuracy when the detectors are trained on one dataset and tested on another. We observe that lidar scan pattern differences form a large component of this reduction in performance. We address this in our approach, SEE-VCN, by designing a novel viewer-centred surface completion network (VCN) to complete the surfaces of objects of interest within an unsupervised domain adaptation framework, SEE [1]. With SEE-VCN, we obtain a unified representation of objects across datasets, allowing the network to focus on learning geometry, rather than overfitting on scan patterns. By adopting a domain-invariant representation, SEE-VCN can be classed as a multi-target domain adaptation approach where no annotations or re-training is required to obtain 3D detections for new scan patterns. Through extensive experiments, we show that our approach outperforms previous domain adaptation methods in multiple domain adaptation settings. Our code and data are available at https://github.com/darrenjkt/SEE-VCN.

Autonomous vehicle navigation in shared pedestrian environments requires the ability to predict future crowd motion as well as understand human behaviour. However, most existing methods predict pedestrian future motion without considering potential collisions within the crowd. Furthermore, most current predictive models are tested on datasets that assume full observability of the crowd by relying on a top-down view, which does not reflect the real-world use case of autonomous vehicles due to the inherent limitations of on-board sensors such as visual occlusion. Inspired by prior works, we propose a pedestrian motion prediction model trained via contrastive learning, improving prediction accuracy as well as forecasting collision-free trajectories. Additionally, we propose a method for implementing a predictor using a multi-pedestrian probabilistic tracker, which fuses multiple on-board sensors to track pedestrians in 3D space. Through comprehensive experiments on both aerial view and driving datasets collected in a real-world urban environment, we show that our proposed method improves on state of art methods with better prediction accuracy and more socially acceptable prediction trajectories.

An automated vehicle operating in an urban environment must be able to perceive and recognise objects and obstacles in a three-dimensional world for navigation and path planning. In order to plan and execute accurate and sophisticated driving maneuvers, a high-level contextual understanding of the surroundings is essential. Due to the recent progress in image processing, it is now possible to obtain high definition semantic information in 2D from monocular cameras, though cameras cannot reliably provide the high accuracy 3D information provided by lasers. The fusion of these two sensor modalities can overcome the shortcomings of each individual sensor, though there are a number of important challenges that need to be addressed in a probabilistic manner. In this paper we address the common, yet challenging, LIDAR/camera/semantic fusion problems which are seldom approached in a wholly probabilistic manner. Our approach is capable of using a multi-sensor platform to build a three-dimensional semantic voxelized map that considers the uncertainty of all of the processes involved. We present a probabilistic pipeline that incorporates uncertainty from the sensor readings (cameras, LIDAR, IMU and wheel encoders), compensation for the motion of the vehicle, and heuristic label probabilities for the semantic images depicted inFig. 1. We also present a novel and efficient viewpoint validation algorithm to check for occlusions within the camera frame. A probabilistic projection is performed from the camera images to the LIDAR point cloud. Each labelled LIDAR scan then feeds into an octree map-building algorithm that updates the class probabilities of the map voxels every time a new observation is available. We validate our approach using a set of qualitative and quantitative experiments using the USyd Campus Dataset.1These tests demonstrate the usefulness of a probabilistic sensor fusion approach by evaluating the performance of the perception system in a typical autonomous vehicle application.

For autonomous vehicles to operate persistently in a typical urban environment, it is essential to have high accuracy position information. This requires a mapping and localisation system that can adapt to changes over time. A localisation approach based on a single-survey map will not be suitable for long-term operation as it does not incorporate variations in the environment. In this paper, we present new algorithms to maintain a featured-based map. A map maintenance pipeline is proposed that can continuously update a map with the most relevant features taking advantage of the changes in the surroundings. Our pipeline detects and removes transient features based on their geometrical relationships with the vehicle’s pose. Newly identified features became part of a new feature map and are assessed by the pipeline as candidates for the localisation map. By purging out-of-date features and adding newly detected features, we continually update the prior map to more accurately represent the most recent environment. We have validated our approach using the USyd Campus Dataset, which includes more than 18 months of data. The results presented demonstrate that our maintenance pipeline produces a resilient map which can provide sustained localisation performance over time.

The rise of autonomous systems in cities, such as automated vehicles (AVs), requires new approaches for prototyping and evaluating how people interact with those systems through context-based user interfaces, such as external human-machine interfaces (eHMIs). In this paper, we present a comparative study of three prototype representations (real-world VR, computer-generated VR, real-world video) of an eHMI in a mixed-methods study with 42 participants. Quantitative results show that while the real-world VR representation results in higher sense of presence, no significant differences in user experience and trust towards the AV itself were found. However, interview data shows that participants focused on different experiential and perceptual aspects in each of the prototype representations. These differences are linked to spatial awareness and perceived realism of the AV behaviour and its context, affecting in turn how participants assess trust and the eHMI. The paper offers guidelines for prototyping and evaluating context-based interfaces through simulations.

Autonomous vehicle navigation in shared pedestrian environments requires the ability to predict future crowd motion both accurately and with minimal delay. Understanding the uncertainty of the prediction is also crucial. Most existing approaches however can only estimate uncertainty through repeated sampling of generative models. Additionally, most current predictive models are trained on datasets that assume complete observability of the crowd using an aerial view. These are generally not representative of real-world usage from a vehicle perspective, and can lead to the underestimation of uncertainty bounds when the on-board sensors are occluded. Inspired by prior work in motion prediction using spatio-temporal graphs, we propose a novel Graph Convolutional Neural Network (GCNN)-based approach, Attentional-GCNN, which aggregates information of implicit interaction between pedestrians in a crowd by assigning attention weight in edges of the graph. Our model can either output a probabilistic distribution or faster deterministic prediction, demonstrating applicability to autonomous vehicle use cases where either speed or accuracy with uncertainty bounds are required. To further improve the training of predictive models, we propose an automatically labelled pedestrian dataset collected from an intelligent vehicle platform representative of real-world use. Through experiments on a number of datasets, we show our proposed method achieves an improvement over the state of the art by 10% on Average Displacement Error (ADE) and 12% on Final Displacement Error (FDE) with fast inference speeds.

Self-driving cars are becoming slowly but surely the future of transport. Nonetheless, in order to achieve fully automatic operation, several challenges are still needed to be tackled. One of the main goals that is currently being pursued is a very accurate scene understanding and object detection. In this regard, the most accurate object detectors are image-based. However, these methods yield critical flaws that make them prone to error in some specific scenarios. For instance, actual objects would be detected in depictions of such objects. The urban environment is strewn with these cases. Namely, in billboards and advertisements. However, most of the self-driving cars feature a lidar that provides 3D perception. This sensor could help to disambiguate the cases mentioned before.In this paper, we combine the accuracy of 2D deep learning object detectors with a 3D Convolutional Neural Network (3D CNN) for rejecting false positives on pedestrian detection. First, the object detector provides all the detected pedestrians in the scene, and then the 3D CNN is in charge of rejecting or verify the detections. Our proposal is tested on two well-known publicly available datasets and provides up to 84% accuracy.

One of the fundamental challenges in the design of perception systems for autonomous vehicles is validating the performance of each algorithm under a comprehensive variety of operating conditions. In the case of vision-based semantic segmentation, there are known issues when encountering new scenarios that are sufficiently different to the training data. In addition, even small variations in environmental conditions, such as illumination and precipitation, can affect the classification performance of the segmentation model. Given the reliance on visual information, these effects often translate into poor semantic pixel classification which can potentially lead to catastrophic consequences when driving autonomously. This paper presents a novel method for analyzing the robustness of semantic segmentation models and provides a number of metrics to evaluate the classification performance over a variety of environmental conditions. The process incorporates an additional sensor (lidar) to automate the process and improve the system integrity, eliminating the need for labor-intensive hand labeling of validation data. The experimental results are presented based on multiple datasets collected at different times of the year with different environmental conditions. We extract the ”Road” class using the lidar to demonstrate the concepts, but this could be extended to other classes with different feature detection algorithms. These results show that the semantic segmentation performance varies depending on the weather, camera parameters, and existence of shadows. The results also demonstrate how the metrics can be used to compare and validate the performance after making improvements to a model, and compare the performance of different networks.

Understanding the intentions of drivers at intersections is a critical component for autonomous vehicles. Urban intersections that do not have traffic signals are a common epicenter of highly variable vehicle movement and interactions. We present a method for predicting driver intent at urban intersections through multi-modal trajectory prediction with uncertainty. Our method is based on recurrent neural networks combined with a mixture density network output layer. To consolidate the multi-modal nature of the output probability distribution, we introduce a clustering algorithm that extracts the set of possible paths that exist in the prediction output and ranks them according to probability. To verify the method's performance and generalizability, we present a real-world dataset that consists of over 23 000 vehicles traversing five different intersections, collected using a vehicle-mounted lidar-based tracking system. An array of metrics is used to demonstrate the performance of the model against several baselines.

Combining multiple sensors for advanced perception is a crucial requirement for autonomous vehicle navigation. Heterogeneous sensors are used to obtain rich information about the surrounding environment. The combination of the camera and lidar sensors enables precise range information that can be projected onto the visual image data. This gives a high level understanding of the scene which can be used to enable context based algorithms such as collision avoidance and navigation. The main challenge when combining these sensors is aligning the data into a common domain. This can be difficult due to the errors in the intrinsic calibration of the camera, extrinsic calibration between the camera and the lidar and errors resulting from the motion of the platform. In this paper, we examine the algorithms required to provide motion correction for scanning lidar sensors. The error resulting from the projection of the lidar measurements into a consistent odometry frame is not possible to remove entirely, and as such it is essential to incorporate the uncertainty of this projection when combining the two different sensor frames. This work proposes a novel framework for the prediction of the uncertainty of lidar measurements (in 3D) projected in to the image frame (in 2D) for moving platforms. The proposed approach fuses the uncertainty of the motion correction with uncertainty resulting from errors in the extrinsic and intrinsic calibration. By incorporating the main components of the projection error, the uncertainty of the estimation process is better represented. Experimental results for our motion correction algorithm and the proposed extended uncertainty model are demonstrated using real-world data collected on an electric vehicle equipped with wide-angle cameras covering a 180-degree field of view and a 16-beam scanning lidar.

To operate in an urban environment, an automated vehicle must be capable of accurately estimating its position within a global map reference frame. This is necessary for optimal path planning and safe navigation. To accomplish this over an extended period of time, the global map requires long term maintenance. This includes the addition of newly observable features and the removal of transient features belonging to dynamic objects. The latter is especially important for the long-term use of the map as matching against a map with features that no longer exist can result in incorrect data associations, and consequently erroneous localisation. This paper addresses the problem of removing features from the map that correspond to objects that are no longer observable/present in the environment. This is achieved by assigning a single score which depends on the geometric distribution and characteristics when the features are re-detected (or not) on different occasions. Our approach not only eliminates ephemeral features, but can also be used as a reduction algorithm for highly dense maps. We tested our approach using half a year of weekly drives over the same 500 metre section of road in an urban environment. The results presented demonstrate the validity of the long term approach to map maintenance.

Mobile vehicles operating in urban navigation applications can achieve high integrity localisation with high accuracy by using maps of the surroundings. To accomplish this, the map should always have an accurate representation of the environment. Thus, it is necessary to detect and remove the map components that no longer exist in the current environment. This maintains the map compactness and dependability while simplifying the data association problem. This paper addresses the problem of deletion of transient map components by taking advantage of the geometric connection between the map and agent poses in order to establish and update the visibility of each feature. Once the map is created an initial visibility vector is associated with every map element and updated over time. The visibility of a map element which no longer exists is reduced and ultimately removed from the map. We demonstrate our approach in a 2D feature-based map composed of poles and corners extracted from information provided by a Iidar sensor. The experimental results show the map update using a seven-month data set collected in the University of Sydney campus.

This work focuses on a novel probabilistic approach for extended vehicle tracking, where multiple spatially distributed measurements can originate from the target, and kinematic state and geometry variables are estimated jointly. Prominent shape features extracted from raw measurement points contain spatial uncertainties due to noise in sensor measurements, the feature extraction process, approximation error of shape hypothesis, partial vision occlusion, to name a few. This work proposes a novel tracking paradigm that respects the variant spatial measurement model subject to changes in target pose and sensor viewpoint. This is achieved through probabilistic projection of the spatial measurement points to the predicted measurement sources on the visible side(s) of the target shape. The spatial uncertainties in the shape features are probabilistically modelled and incorporated in the unscented Kalman filter based estimation. The proposed approach is validated with field experiment results using cameras and a laser range scanner.

Robustness and safety are crucial properties for the real-world application of autonomous vehicles. One of the most critical components of any autonomous system is localisation. During the last 20 years there has been significant progress in this area with the introduction of very efficient algorithms for mapping, localisation and SLAM. Many of these algorithms present impressive demonstrations for a particular domain, but fail to operate reliably with changes to the operating environment. The aspect of robustness has not received enough attention and localisation systems for self-driving vehicle applications are seldom evaluated for their robustness. In this paper we propose novel metrics to effectively quantify localisation robustness with or without an accurate ground truth. The experimental results present a comprehensive analysis of the application of these metrics against a number of well known localisation strategies.

Vision systems are widely used in autonomous vehicle systems due to the rich information that camera sensors provide of the surrounding environment. This paper presents an automatic algorithm to obtain the drivable path of a vehicle operating in urban roads with or without clear lane markings. The developed system projects trajectories obtained during human operation of the vehicle and utilizes these to generate automatic labels for training a semantic based path prediction model. The system segments an urban scenario into 13 categories including vehicles, pedestrian, undrivable road, other categories relevant to urban roads, and a new class for a path proposal. The drivable path information is essential particularly in unstructured scenarios, and is critical for an intelligent vehicle system to make sound driving decisions. The path proposal category is a car-width drivable lane estimated to be safe to drive for the vehicle under consideration. The data collection, model training and inference process requires only images from a monocular camera and odometry from a low-cost IMU combined with a wheel encoder. The algorithm has been successfully demonstrated on the Sydney University campus, which is a challenging environment without clear road markings. The algorithm was demonstrated to run in real-time, proving its applicability for intelligent vehicles.

To navigate through urban roads, an automated vehicle must be able to perceive and recognize objects in a three-dimensional environment. A high level contextual understanding of the surroundings is necessary to execute accurate driving maneuvers. This paper presents a novel approach to build three dimensional semantic octree maps from lidar scans and the output of a convolutional neural network (CNN) to obtain the labels of the environment. We present a heuristic method to associate uncertainties to the labels from the images based on a combination of the labels themselves, score maps retrieved by the CNN and the raw images. These uncertainties and the camera-lidar calibration parameters for multiple cameras are considered in the projection of the labels and their uncertainties into the point cloud. Every labeled lidar scan works as an input to an octree map building algorithm that calculates and updates the label probabilities of the voxels in the map. This paper also presents a qualitative and quantitative evaluation of accuracy, analyzing projection in single lidar scans and complete maps built with our probabilistic octree framework.

The estimation and prediction of pedestrian motion is of fundamental importance in ITS applications. Most existing solutions have utilized a particular type of sensor for perception such as cameras (stereo, monocular, infrared) or other modalities such as a laser range finder or radar. The advent of wearable devices with inertial sensors have led to the development of systems capable of the robust inference of pedestrian intention. Unfortunately, these devices do not have communications capabilities to broadcast this information to all vehicles in proximity, and also this strategy requires functioning devices on all pedestrians to work. This paper presents a robust perception method that is able to extract dynamic pedestrian information with accuracy comparable to that of typical gyroscopes and accelerometers installed in wearable devices. Experimental results are presented to demonstrate the potential for obtaining very comprehensive dynamic information from limbs representing the skeleton of a pedestrian. This work also demonstrates the accuracy of vision based systems by comparing these results to the rotation and acceleration measured directly on the pedestrian using a wearable device. The contributions of this paper demonstrate that it is possible to significantly improve both the detection and estimation of pedestrian intention by incorporating dynamic information obtained from vision sensors.

Advanced Driver Assistance Systems have been shown to greatly improve road safety. However, existing systems are typically reactive with an inability to understand complex traffic scenarios. We present a method to predict driver intention as the vehicle enters an intersection using a Long Short Term Memory (LSTM) based Recurrent Neural Network (RNN). The model is learnt using the position, heading and velocity fused from GPS, IMU and odometry data collected by the ego-vehicle. In this paper we focus on determining the earliest possible moment in which we can classify the driver's intention at an intersection. We consider the outcome of this work an essential component for all levels of road vehicle automation.

Innovation in intelligent transportation systems relies on analysis of high-quality data. In this paper, we describe the design principles behind our data management infrastructure. The principles we adopt place an emphasis on flexibility and maintainability. This is achieved by breaking up code into a modular design that can be run on many independent processes. Message passing over a publish-subscribe network enables interprocess communication and promotes data-driven execution. By following these principles, rapid prototyping and experimentation with new sensing modalities and algorithms are possible. The communication library underpinning our proposed architecture is compared against several popular communication libraries. Features designed into the system make it decentralized, robust to failure, and amenable to scaling across multiple machines with minimal configuration. Code written using the proposed architecture is compact, transparent, and easy to maintain. Experimentation shows that our proposed architecture offers a high performance when compared against alternative communication libraries.

Intelligent transportation systems are able to collect large volumes of high-resolution data. The amount of data collected by these systems can quickly overwhelm the ability of human analysts to draw meaningful conclusions from the data, particularly in large-scale multivehicle field trials. As advanced driver assistance systems develop, they will also be required to form a rich and high-level understanding of the world from the data they receive, including the behavior of the driver. These applications motivate the need for unsupervised tools capable of forming a high-level summary of low-level driving data. This paper presents an unsupervised method for converting naturalistic driving data into high-level behaviors. The proposed method works in two steps. In the first step, inertial data are automatically decomposed into linear segments. In the second step, the segments are assigned to high-level driving behaviors. The proposed method is computationally efficient and completely unsupervised and requires minimal preprocessing. Although the method is unsupervised, the clusters produced exhibit high-level patterns that can easily be associated with driving behaviors such as braking, turning, accelerating, and coasting. The effectiveness of the proposed algorithms is demonstrated in an offline application where the objective is to summarize inertial data into driving behaviors. The method is also demonstrated in an online application where the aim is to infer the current driving behavior using only inertial data. Both experiments were conducted using driving data collected in natural driving conditions.

This paper presents a novel nonparametric approach toward delayed-state filtering for cooperative tracking. Standard parametric cooperative localization/tracking approaches are generally aimed at problems that can be easily parameterized and/or are limited to incorporate only real-time measurements. This paper provides a nonparametric yet computationally tractable alternative that is suitable for tracking cases where real-time observations are not always possible, e.g., in a sparse mesh network. The proposed delayed-state cooperative particle filter features forward filtering and backward smoothing to incorporate measurements that are received with time delays. A record of historical marginal states is kept for each mobile node within a sliding time window, instead of the high-dimensional joint state. Essentially, it replaces the importance sampling in traditional particle filters by a Gibbs sampler, which is a Markov chain Monte Carlo method, to fuse all available egocentric and internode relative observations into the global position estimate, thus alleviating the high-dimensionality problems in cooperative tracking. The performance of the proposed approach is evaluated in a multiagent simulation, and experimental results from a large-scale multivehicle industrial operation clearly demonstrate that the proposed approach effectively facilitates the tracking of mobile nodes without position awareness, through the use of relative range, negative detection, and time-delayed measurements.

This paper presents a new nonparametric approach for cooperative localisation and tracking by fusing relative range measurements between mobile network equipped nodes. Standard approaches based on parametric methods are known to be limited to problems that contain Gaussian properties. This paper overcomes this limitation by proposing a novel particle filter based cooperative tracking approach that is suitable for mobile ad-hoc networks (MANETs). The filter maintains the marginal state of every node instead of a joint state of the group, and updates the estimates using a Gibbs sampler, which is known as a Markov chain Monte Carlo (MCMC) method. The performance of the proposed algorithm is demonstrated by examining 16 mobile nodes moving randomly while sharing information with neighbouring nodes in a MANET. The results show that the proposed approach facilitates the tracking of mobile nodes that do not have egocentric position information available. It also outperforms the EKF in systems with non-Gaussian properties.

Vehicle-to-vehicle (V2V) communication systems allow vehicles to share state information with one another to improve safety and efficiency of transportation networks. One of the key applications of such a system is in the prediction and avoidance of collisions between vehicles. If a method to do this is to succeed it must be robust to measurement uncertainty. The method should also be general enough that it does not rely on constraints on vehicle motion for the accuracy of its predictions. It should work for all interactions between vehicles and not just a select subset. This paper presents a method for collision probability calculation that addresses these problems.

The tracking of vehicles over large areas with limited position observations is of significant importance in many industrial applications. This paper presents algorithms for long-term vehicle motion estimation based on a vehicle motion model that incorporates the properties of the working environment and information collected by other mobile agents and fixed infrastructure collection points. The prediction algorithm provides long-term estimates of vehicle positions using speed and timing profiles built for a particular environment and considering the probability of a vehicle stopping. A limited number of data collection points distributed around the field are used to update the estimates, with negative information (no communication) also used to improve the prediction. This paper introduces the concept of observation harvesting, a process in which peer-to-peer communication between vehicles allows egocentric position updates to be relayed among vehicles and finally conveyed to the collection point for an improved position estimate. Positive and negative communication information is incorporated into the fusion stage, and a particle filter is used to incorporate the delayed observations harvested from vehicles in the field to improve the position estimates. The contributions of this work enable the optimization of fleet scheduling using discrete observations. Experimental results from a typical large-scale mining operation are presented to validate the algorithms.

This paper presents a statistical method for detecting anomalous driving behavior by analyzing cross-sectional profiles of the road. A profile captures the way vehicles normally traverse the road; a statistical hypothesis test determines whether the observed behavior is anomalous. Experimental results on genuine data collected by a fleet of vehicles demonstrates the potential of this new method.

This paper presents a robust, non-linear smoothing algorithm and develops the theory behind it. This algorithm is extremely robust to outliers and missing data and handles state-dependent noise. Implementing it is straightforward as it consists mainly of two sub-routines: (a) the Rauch-Tung-Striebel recursions, or Kalman smoother; and (b) a backtracking line search strategy. The computational load grows linearly with the number of data because the algorithm preserves the underlying structure of the problem. Global convergence to a local optimum is guaranteed, under mild assumptions.

Frequency-modulated continuous waveform (FMCW) microwave and millimetre-wave radar is an attractive sensor for intelligent transport systems due to its reliable all-weather performance. This paper discusses issues involved in the design of FMCW radar mapping systems for use in collision avoidance in large vehicles operating in dynamic environments. The performance characteristics of radar are examined before an analysis is made of traditional grid-based and feature-based mapping approaches, both conceptually and in terms of implementation. The probability hypothesis density (PHD) filter is discussed as a potentially superior approach for radar mapping in dynamic environments.

This paper presents results of the deployment of vehicle safety systems developed by the Intelligent Vehicles and Safety Systems Group at the Australian Centre for Field Robotics. The technology was deployed within an active mine site in Australia undergoing standard open-cut mining operations. Data were collected from the vehicles and analysed in order to assess the overall safety and performance of the vehicle operations within the mine. Metrics calculated include distributions of vehicle paths along stretches of road for driving line analysis, and statistics around vehicle events such as overspeed and proximity to other vehicles.

An increasing number of intelligent transportation applications require robust and reliable wireless communication. To achieve the required quality of service it is necessary to implement redundancy in the critical path which includes the radio software and hardware. In a real-world application there are many things that can cause the communication between two vehicles to degrade or stop completely. This paper describes a novel technique for detecting degradation or failure of communication links by comparing the performance of the radios to a probabilistic model built using data collected in the field. The results show that this techinique can successfully detect when there is partial or complete failure to communicate due to damage to the external components such as antennas, connectors and cables.

Vehicle position tracking and prediction over large areas is of significant importance in many industrial applications, such as mining operations. In a small area, this can easily be achieved by providing vehicles with a constant communication link to a control center and having the vehicles broadcast their position. The problem dramatically changes when vehicles operate within a large environment of potentially hundreds of square kilometers and in difficult terrain. This paper presents algorithms for long-term vehicle motion prediction and tracking based on a multiple-model approach. It incorporates a probabilistic vehicle model that includes the structure of the environment. The prediction algorithm evaluates the vehicle position using acceleration, speed, and timing profiles built for the particular environment and considers the probability that the vehicle will stop. A limited number of data collection points distributed around the field are used to update the vehicle position estimate when in communication range, and prediction is used at points in between. A particle filter is used to estimate the vehicle position using both positive and negative information (whether communication is possible) in the fusion stage. The algorithms presented are validated with experimental results using data collected from a large-scale mining operation.

This paper addresses the issue of creating a sensor model for a new short-range 24GHz close proximity detection (CPD) radar. The CPD radar is designed to provide improved situational awareness to the driver of a large vehicle. It is able to detect light vehicles and other targets at ranges from 2.2m to 45m within an arc of 160° azimuth, but radar measurements contain data from noise and clutter which must be filtered out. Dynamic thresholds such as constant false-alarm rate (CFAR) processors [19] do not work well with short measurement vectors or targets that occupy multiple measurement bins. In this paper, a new method is used where measurements of environmental noise, clutter and targets are used to calculate the false alarm and target detection probabilities for each bin and develop a fixed detection threshold for each bin. This filter is used to construct a sensor model which maps measurement power to probability of bin occupancy, which is then used to generate an occupancy grid map of the environment from CPD radar measurements.

This paper addresses the issue of improving situational awareness for large vehicles operating in all weather conditions. A new short-range 24GHz close proximity detection (CPD) microwave radar technology for providing situational awareness to the driver of large vehicles is presented. A series of experiments are performed in which the radar is used to detect a variety of static targets. The CPD radar is able to detect light vehicles at ranges from 2.2m to 17.6m within a horizontal arc of 160° with superior angular resolution to many existing low-cost radars. From this data, key characteristics of the radar's performance and the radar cross-section of light vehicles at 24GHz are calculated. The data gathered allows the design of future tests and the development of a model of the radar that allows a better understanding of its performance characteristics and can be used for occupancy grid mapping.

Human drivers are endowed with an inborn ability to put themselves in the position of other drivers and reason about their behavior and intended actions. State-of-the-art driving-assistance systems, on the other hand, are generally limited to physical models and ad hoc safety rules. In order to drive safely amongst humans, autonomous vehicles need to develop an understanding of the situation in the form of a high-level description of the state of traffic participants. This paper presents a probabilistic model to estimate the state of vehicles by considering interactions between drivers immersed in traffic. The model is defined within a probabilistic filtering framework; estimation and prediction are carried out with statistical inference techniques. Memory requirements increase linearly with the number of vehicles, and thus, it is possible to scale the model to complex scenarios involving many participants. The approach is validated using real-world data collected by a group of interacting ground vehicles.

We introduce a novel approach for processing sequential data in the presence of outliers. The outlier-robust Kalman filter we propose is a discrete-time model for sequential data corrupted with non-Gaussian and heavy-tailed noise. We present efficient filtering and smoothing algorithms which are straightforward modifications of the standard Kalman filter Rauch-Tung-Striebel recursions and yet are much more robust to outliers and anomalous observations. Additionally, we present an algorithm for learning all of the parameters of our outlier-robust Kalman filter in a completely unsupervised manner. The potential of our approach is borne out in experiments with synthetic and real data.

This paper presents a probabilistic framework for road geometry estimation using a millimetre wave radar. It aims at estimating the geometry of roads without assuming any particular infrastructure such as lane marks. It provides also the vehicle location with respect to the edges of the road. This system employs a radar sensor in view of its robustness to weather conditions such as fog, dust, rain and snow. The proposed approach is robust to noisy measurements since the radar target locations are modelled as Gaussian distributions. These observations are integrated into a Kalman Particle filter to estimate the posterior distribution of the parameters that best describe the geometry of the road. Experimental results using data acquired on a highway road are presented. The effectiveness of the proposed approach is demonstrated by a qualitative analysis of the results.

Human drivers are endowed with an inborn ability to put themselves in the position of other drivers and reason about their behaviors and intended actions. State-of-the-art driving assistance systems, on the other hand, are generally limited to physical models and ad-hoc safety rules. In order to drive safely amongst humans, autonomous vehicles require a high-level description of the state of traffic participants. This paper presents a probabilistic model for estimating and predicting the behavior of drivers immersed in traffic. The model is defined within a stochastic filtering framework and estimation and prediction are carried out with statistical inference techniques. The approach is validated with real data from a fleet of mining vehicles.

Over the last few years, electronic vehicle guidance systems have become increasingly more popular. However, despite their ubiquity, performance will always be subject to availability of detailed digital road maps. Most current digital maps are still inadequate for advanced applications in unstructured environments. Lack of up-to-date information and insufficient refinement of the road geometry are among the most important shortcomings. The massive use of inexpensive Global Positioning System (GPS) receivers, combined with the rapidly increasing availability of wireless communication infrastructure, suggests that large amounts of data combining both modalities will be available in the near future. The approach presented here draws on machine-learning techniques and processes logs of position traces to consistently build a detailed and fine-grained representation of the road network by extracting the principal paths followed by the vehicles. Although this work addresses the road-building problem in dynamic environments such as open-pit mines, it is also applicable to urban environments. New contributions include a fully unsupervised segmentation method for sampling roads and inferring the network topology, which is a general technique for extracting detailed information about road splits, merges, and intersections, as well as a robust algorithm that articulates these two. Experimental results with data from large mining operations are presented to validate the new algorithm.

Over the last ten years, electronic vehicle guidance systems have become increasingly popular. However, their performance is subject to the availability and accuracy of digital road maps. Most current digital maps are still inadequate for advanced applications in unstructured environments. Lack of detailed up-to-date information and insufficient accuracy and refinement of the road geometry are among the most important shortcomings. The massive use of inexpensive GPS receivers, combined with the rapidly increasing availability of wireless communication infrastructure, suggests that large volumes of data combining both modalities will be available in a near future. The approach presented here draws on machine learning techniques to process logs of position traces to consistently build a detailed and accurate representation of the road network and, more importantly, extract the actual paths followed by vehicles. Experimental results with data from large mining operations are presented to validate the algorithm.

This paper addresses the problem of safety in mining applications. It presents new metrics that can be used to determine dangerous situations during mine operation in real time. It also presents a fast and robust algorithm for extracting significant places from information logged by a state-of-the-art collision avoidance system. Determining significant places provides valuable context information in a variety of applications such as map building, vehicle tracking and user assistance. In our case, we are interested in obtaining context information as a preliminary step towards improving mining safety. The algorithm presented here is validated with experimental data obtained from a fleet of haulage vehicles operating in various open pit mines.

In mining operations it is advantageous to be able to predict the future movements of nearby vehicles. For autonomous mining, this can be used for localised, short term path planning and risk assessment. For semi-autonomous or non-autonomous mining, this can be used for collision avoidance, situational awareness and risk assessment of maneuvers between a human operated vehicle, and another vehicle (operated by a human or otherwise). This paper introduces a probabilistic approach to predicting vehicle movements, in particular, the time until two vehicle paths intersect. Results are shown using real data collected from the operation of two separate fleets of vehicles.

This work introduces a self-supervised, multi-sensor architecture that performs automatic moving obstacles classification. Our approach presents a hierarchical scheme that relies on the ldquostabilityrdquo of a subset of features given by a sensor to perform an initial robust classification based on unsupervised techniques. The obtained results are used as labels to train a set of supervised classifiers, which can be then combined to improve the final classification accuracy. The proposed architecture is general and can be instantiated in a variety of ways, using different sensors and classifiers. The applicability and validity of the proposed architecture is evaluated for a particular realization based on range and visual information that achieves 83% accuracy without using manually labeled data. Experimental results also demonstrate how accuracy can be maintained through self-training capabilities when working conditions change.

This paper presents a motion detection scheme using laser scanners mounted on a mobile vehicle. We propose a stable, yet simple motion detection scheme that can be used and improved with tracking and classification procedures. The salient contribution of the developed architecture is twofold. It proposes a spatio-temporal correspondence procedure based on a scan registration algorithm. The detection is cast as a probability decision problem that accounts for sensor noise and achieves robust classification. Probabilistic occlusion checking is finally performed to improve robustness. Experimental results show the performance of the proposed architecture under different settings in urban environments.

Mining operations generally involve a large number of expensive vehicles, and for the efficient management of these vehicles it is very beneficial to know their location at all times. The current procedure for vehicle localisation in mines is to provide the mine with complete wireless network coverage to facilitate the broadcasting of vehicle positions. This paper examines an alternative method of localisation that does not require the expense of a radio network with full mine coverage. Two different non-parametric filter approaches are presented to estimate the location of the vehicles. A comparison of the two filters is also presented with experimental results using data collected in two operational mines.

This paper presents an analysis of FastSLAM - a Rao-Blackwellised particle filter formulation of simultaneous localisation and mapping. It shows that the algorithm degenerates with time, regardless of the number of particles used or the density of landmarks within the environment, and would always produce optimistic estimates of uncertainty in the long-term. In essence, FastSLAM behaves like a non-optimal local search algorithm; in the short-term it may produce consistent uncertainty estimates but, in the long-term, it is unable to adequately explore the state-space to be a reasonable Bayesian estimator. However, the number of particles and landmarks does affect the accuracy of the estimated mean and, given sufficient particles, FastSLAM can produce good non-stochastic estimates in practice. FastSLAM also has several practical advantages, particularly with regard to data association, and would probably work well in combination with other versions of stochastic SLAM, such as EKF-based SLAM

This paper presents an analysis of the extended Kalman filter formulation of simultaneous localisation and mapping (EKF-SLAM). We show that the algorithm produces very optimistic estimates once the "true" uncertainty in vehicle heading exceeds a limit. This failure is subtle and cannot, in general, be detected without ground-truth, although a very inconsistent filter may exhibit observable symptoms, such as disproportionately large jumps in the vehicle pose update. Conventional solutions - adding stabilising noise, using an iterated EKF or unscented filter, etc., - do not improve the situation. However, if "small" heading uncertainty is maintained, EKF-SLAM exhibits consistent behaviour over an extended time-period. Although the uncertainty estimate slowly becomes optimistic, inconsistency can be mitigated indefinitely by applying tactics such as batch updates or stabilising noise. The manageable degradation of small heading variance SLAM indicates the efficacy of submap methods for large-scale maps

Although full 3D navigation and mapping is recognized as one of the most important challenges for autonomous navigation, the lack of robust sensors, providing 3D information in real time, has burdened the progress in this direction. This paper presents our ongoing work towards the deployment of an integrated sensing system for 3D mapping in outdoor environments. We first describe a 3D data acquisition architecture based on a standard 2D laser. Techniques for registering scans using a scan matching procedure and for estimating the errors are then introduced. We finally present results showing the performance of the proposed architecture in real outdoor environments by means of the integration of the 3D scans with dead reckoning and inertial measurement unit (IMU) information

This paper describes the derivation and experimental validation of a novel sensor model for radio frequency sensors to be used for localisation purposes. A comprehensive description of the modelling aspects is presented. Outdoor range only tracking results using the newly derived model are fully described with a comparison to the free space model commonly used for radio frequency applications. Although this approach is applicable for generic localisation purposes in indoor and outdoor environments, it is of fundamental importance when applied to proximity detection involving large machines. In particular in environments such as mining, stevedoring and construction there is a need to detect the presence of personnel in close proximity to machines. A close proximity system that makes use of the newly derived model is presented working with a 100 tons mining haul truck

This work presents a new hybrid metric map representation (HYMM) that combines feature maps with other dense metric sensory information. The global feature map is partitioned into a set of connected local triangular regions (LTRs), which provide a reference for a detailed multi-dimensional description of the environment. The HYMM framework permits the combination of efficient feature-based SLAM algorithms for localisation with, for example, occupancy grid (OG) maps. This fusion of feature and grid maps has several complementary properties; for example, grid maps can assist data association and can facilitate the extraction and incorporation of new landmarks as they become identified from multiple vantage points. The representation presented here will allow the robot to perform DenseSLAM. DenseSLAM is the process of performing SLAM whilst obtaining a dense environment representation.

This paper proposes a new algorithm named "SIGMA" to address the problem of simultaneous information and global motion analysis for a car working in unstructured outdoor environments. The map of the environment is made by a Simultaneous Localization and Mapping (SLAM) algorithm that uses an Hybrid Metric Map (HYMM) structure for mapping. The path planning approach presents a global solution maximizing overall information gain of the map. The cost function used considers the present and future uncertainty in the map and vehicle and is based on the variation of the covariance matrix trace. Eigenvalue concepts are utilized to determine overall information change from the information matrix properties. An information graph is constructed followed by a search to find the optimal information-based rough path. Results are presented to demonstrate performance of the algorithm.

The ability to simultaneously localise a robot and accurately map its surroundings is considered by many to be a key prerequisite of truly autonomous robots. This paper presents a real-world implementation of FastSLAM, an algorithm that recursively estimates the full posterior distribution of both robot pose and landmark locations. In particular, we present an extension to FastSLAM that addresses the data association problem using a nearest neighbor technique. Building on this, we also present a novel multiple hypotheses tracking implementation (MHT) to handle uncertainty in the data association. Finally an extension to the multi-robot case is introduced. Our algorithm has been run successfully using a number of data sets obtained in outdoor environments. Experimental results are presented that demonstrate the performance of the algorithms when compared with standard Kalman filter-based approaches.

This paper addresses the problem of on-line path following for a car working in unstructured outdoor environments. The partially known map of the environment is updated and expanded in real time by a Simultaneous Localization and Mapping (SLAM) algorithm. This information is used to implement global path planning. A cost graph is initially constructed followed by a search to find the near-optimal path considering uncertainty in both vehicle location and map. Selected points in the global path are connected by continuous-curvature paths. An improved feedback linearization technique is presented to guide the car along the defined path. Experimental results are presented to demonstrate the performance of the algorithms.

This paper presents new algorithms to implement simultaneous localization and mapping in environments with very large numbers of features. The algorithms present an efficient solution to the full update required by the compressed extended Kalman filter algorithm. It makes use of the relative landmark representation to develop very close to optimal decorrelation solutions. With this approach, the memory and computational requirements are reduced from /spl sim/O(N/sup 2/) to /spl sim/O(N/sup */N/sub a/), N and N/sub a/ proportional to the number of features in the map and features close to the vehicle, respectively. Experimental results are presented to verify the operation of the system when working in large outdoor environments.

Addresses the problem of implementing simultaneous localisation and map building (SLAM) in very large outdoor environments. A method is presented to reduce the computational requirement from /spl sim/O(N/sup 2/) to /spl sim/O(N), N being the states used to represent all the landmarks and vehicle pose. With this implementation the memory requirements are also reduced to /spl sim/O(N). This algorithm presents an efficient solution to the full update required by the compressed extended Kalman filter algorithm. Experimental results are also presented.

This paper address the problem of navigating in very large outdoor unstructured environments. It presents solutions to the problem of closing large loops in simultaneous localization and map building applications. A hybrid architecture is presented that make use of the extended Kalman filter to perform SLAM in an efficient form and a Monte Carlo type filter to resolve the data association problem present when closing large loops. The proposed algorithm incorporates integrity to the standard SLAM algorithms by allowing multimode distribution to be handled in real time. Experimental results in outdoor environments are also presented.

This paper presents a new method for improving the accuracy of inertial measurement units (IMUs) mounted on land vehicles. The algorithm exploits nonholonomic constraints that govern the motion of a vehicle on a surface to obtain velocity observation measurements which aid in the estimation of the alignment of the IMU as well as the forward velocity of the vehicle. It is shown that this can be achieved without any external sensing provided that certain observability conditions are met. A theoretical analysis is provided together with a comparison of experimental results between a nonlinear implementation of the algorithm and an IMU/GPS navigation system. This comparison demonstrates the effectiveness of the algorithm. The real time implementation is also addressed through a multiple observation inertial aiding algorithm based on the information filter. The results show that the use of these constraints and vehicle speed guarantees the observability of the velocity and the attitude of the inertial unit, and hence bounds the errors associated with these states. The strategies proposed provides a tighter navigation loop which can sustain outages of GPS for a greater amount of time as compared to when the inertial unit is used with standard integration algorithms.

Addresses real-time implementation of the simultaneous localization and map-building (SLAM) algorithm. It presents optimal algorithms that consider the special form of the matrices and a new compressed filler that can significantly reduce the computation requirements when working in local areas or with high frequency external sensors. It is shown that by extending the standard Kalman filter models the information gained in a local area can be maintained with a cost /spl sim/O(N/sub a//sup 2/), where N/sub a/ is the number of landmarks in the local area, and then transferred to the overall map in only one iteration at full SLAM computational cost. Additional simplifications are also presented that are very close to optimal when an appropriate map representation is used. Finally the algorithms are validated with experimental results obtained with a standard vehicle running in a completely unstructured outdoor environment.

Data association is the process of relating features observed in the environment to features viewed previously or to features in a map. This paper presents a graph theoretic method that is applicable to data association problems where the features are observed via a batch process. Batch observations detect a set of features simultaneously or with sufficiently small temporal difference that, with motion compensation, the features can be represented with precise relative coordinates. This data association method is described in the context of two possible navigation applications: metric map building with simultaneous localisation, and topological map based localisation. Experimental results are presented using an indoor mobile robot with a 2D scanning laser sensor. Given two scans from different unknown locations, the features common to both scans are mapped to each other and the relative change in pose (position and orientation) of the vehicle between the two scans is obtained.

This paper presents the design of a high accuracy outdoor navigation system based on standard dead reckoning sensors and laser range and bearing information. The data validation problem is addressed using laser intensity information. Beacon design aspect and location of landmarks are also discussed in relation to desired accuracy and required area of operation. The results are important for simultaneous localization and map building applications since the feature extraction and validation are resolved at the sensor level using laser intensity. This facilitates the use of additional natural landmarks to improve the accuracy of the localization algorithm. Experimental results in outdoor environments are also presented.

Describes a real time, on-the-fly, roll and pitch alignment algorithm for inertial measurement units (IMUs) mounted on land vehicles. Unlike conventional strategies, the alignment is achieved without external observations. This is achieved by exploiting the nonholonomic constraints that govern the motion of a vehicle on a surface to obtain the roll and pitch of the IMU. The position of the vehicle along its path is still unobservable and the algorithm is effective only when there is sufficient excitation in all degrees of freedom. However, with the proposed alignment algorithm, the IMU is able to provide sufficiently accurate position information for substantially longer periods of time compared with conventional methods. Experimental results are provided along with a comparison of an IMU/GPS navigation loop showing the effectiveness of the algorithm.

This work presents a general nonlinear psi-angle approach that does not require coarse alignment. In this psi-angle model, the three misalignment angles are assumed all large. Three states are used to describe three psi-angles rather than the four used in previous works. The approach is identical to the standard small error methods when the process errors diminish to small angles. The position and velocity error models are also presented. Standard extended Kalman filter techniques are used to solve the nonlinear data fusion problem. Experimental results of in-flight inertial navigation systems (INS) alignment and calibration are presented considering total uncertainty in azimuth orientation using a low cost inertial measurement unit aided with a differential global positioning system.

This paper describes a series of trials that were done at an underground mine in New South Wales, Australia. Experimental results are presented from the data obtained during the field trials and suitable sensor suites for an autonomous mining vehicle navigation system are evaluated.

Describes the theoretical development and experimental evaluation of a navigation system for an autonomous load, haul, and dump truck based on the results obtained during extensive in-situ field trials. The particular contributions of the theoretical work are in designing the navigation system to be able to cope with vehicle slip in rough uneven terrain using information from inertial sensors, odometry, and a bearing only laser. Results are presented using data obtained during the field trials.

This paper describes the development and implementation of a high integrity navigation system, based on the combined use of the Global Positioning System (GPS) and an inertial measurement unit (IMU), for autonomous land vehicle applications. The paper focuses on the issue of achieving the integrity required of the navigation loop for use in autonomous systems. The paper highlights the detection of possible faults both before and during the fusion process in order to enhance the integrity of the navigation loop. The implementation of this fault detection methodology considers both low frequency faults in the IMU caused by bias in the sensor readings and the misalignment of the unit, and high frequency faults from the GPS receiver caused by multipath errors. The implementation, based on a low-cost, strapdown IMU, aided by either standard or carrier phase GPS technologies, is described. Results of the fusion process are presented.

Presents an efficient method of multi-sensor estimation for the navigation of vehicles. A decentralised architecture is used for the fusion of information obtained from several asynchronous measurements. The issue of the synchronisation of the information, which is critical in the proposed method, is addressed. The information form of the Kalman filter (information filter) is used as the main algorithm for position estimation. The method is implemented as part of the navigation system for an autonomous land vehicle. The navigation system includes two independent loops which communicate through an assimilation unit. One loop incorporates inertial and GPS information and the other uses millimetre wave radar and encoder measurements to obtain local estimates. The information obtained from each measurement is then broadcast to the other loops after being synchronised. This information is used in an assimilation stage to achieve more accurate estimates. The assimilation takes place periodically, where the assimilation frequency can be selected. The performance of the navigation method is examined by comparing the resulting position estimates to those of independent navigation loops.

This paper provides an analysis of Kalman filter based systems with respect to fault detection. By using frequency domain techniques, a metric is developed that describes the detectability of a fault by showing how a fault is transmitted to the filter innovations (if at all). Through experiment, it is shown that redundancy must be employed for guaranteed detection of faults, and that unlike sensors should be used. Further, it is shown that modelling errors can be treated within the same framework as "hard" actuator or sensor faults.

The objective of this paper is to introduce and investigate the issue of integrity in an INS/GPS navigation loop for autonomous land vehicle applications. The paper briefly outlines the standard fusion algorithm for the INS/GPS loop, while the focus is on the detection of possible faults both before and during the fusion process. The concept of fault detection focuses on the low frequency faults of the INS, caused by bias and drift, and the high frequency faults of the GPS unit caused by multipath errors and changes in satellite geometry.

We examine the problem of constructing and maintaining a map of an autonomous vehicle's environment for the purpose of navigation, using evidential reasoning. The inherent uncertainty in the origin of measurements of sensors demands a probabilistic approach to processing, or fusing, the new sensory information to build an accurate map. In the paper, the map is based on a two-dimensional (2-D) occupancy grid. The sensor readings are fused into the map using the Dempster-Shafer inference rule. This evidential approach with its multivalued hypotheses allows quantitative analysis of the quality of the data. The map building system is experimentally evaluated using sonar data from real environments.

Position information obtained from standard GPS receivers is known to be corrupted with coloured (time-correlated) noise. To make effective use of GPS information in a navigation system it is essential to model this coloured noise and to incorporate additional sensing to de-correlate and eliminate its effect. In this paper frequency domain techniques are employed to generate a model for GPS noise sources. This model shows clearly what type and combination of additional sensor information is necessary to de-correlate GPS errors and to make best use of position information in navigation tasks. The frequency-domain methodology proposed has wider application in the design of sensor suites for high-performance navigation systems. Experimental results are presented demonstrating the method in fusing standard GPS latitude and longitude information with information from a velocity sensor.

This paper describes the theoretical development and experimental evaluation of a guidance system for an autonomous load, haul and dump truck (LHD) for use in underground mining. The particular contributions of this paper are in designing the navigation system to be able to cope with vehicle slip in rough uneven terrain using information from an inertial navigation system (INS) and a bearing only laser. Results are presented using data obtained during field trials.

This paper presents the results of an experimental program for evaluating sensors and sensing technologies in an underground mining applications. The objective of the experiments is to infer what combinations of sensors will provide reliable navigation systems for autonomous vehicles operating in a harsh underground environment. Results from a wide range of sensors are presented and analysed. A conclusion as to the best combination of sensors is drawn.

Examines the problem of constructing and maintaining a map of of an autonomous vehicle's environment for the purpose of navigation, using evidential reasoning. The inherent uncertainty in the origin of measurements of sensors demands a probabilistic approach to processing, or fusing, the new sensory information to build an accurate map. In this paper, the map is based on a two-dimensional occupancy grid. The sensor readings are 'fused' into the map using the Dempster-Shafer inference rule. This evidential approach with its multi-valued hypotheses allows quantitative analysis of the quality of the data. The map building system is experimentally evaluated using sonar data from real environments.