Mobile robots and autonomous vehicles are often required to function in environments where critical position estimates from sensors such as GPS become uncertain or unreliable. Single image visual place recognition (VPR) provides an alternative for localization but often requires techniques such as sequence matching to improve robustness, which incurs additional computation and latency costs. Even then, the sequence length required to localize at an acceptable performance level varies widely; and simply setting overly long fixed sequence lengths creates unnecessary latency, computational overhead, and can even degrade performance. In these scenarios it is often more desirable to meet or exceed a set target performance at minimal expense. In this paper we present an approach which uses a calibration set of data to fit a model that modulates sequence length for VPR as needed to exceed a target localization performance. We make use of a coarse position prior, which could be provided by any other localization system, and capture the variation in appearance across this region. We use the correlation between appearance variation and sequence length to curate VPR features and fit a Multi-Layer Perceptron (MLP) for selecting the optimal length. We demonstrate that this method is effective at modulating sequence length to maximize the number of sections in a dataset which meet or exceed a target performance whilst minimizing the median length used. We show applicability across several datasets and reveal key phenomena like generalization capabilities, the benefits of curating features and the utility of non-state-of-the-art feature extractors with nuanced properties.

Navigating natural environments with deformable terrain is a difficult challenge in robotics. Understanding the interaction dynamics between robots and such terrain is an important first step in enabling them to explore these environments. Terramechanics models are largely developed and tested on wheeled and tracked platforms, but with the advent of readily available lightweight legged robots, developing an understanding of how robot feet interact with the terrain becomes increasingly important. Works on estimating terramechanical properties of deformable sands and soils use an underlying assumption that translation of the robot foot along the surface of the terrain is due to internal shear deformation of the soil. We show that for lightweight legged robots, this is not the case. Shear forces acting on the foot of a robot during a stride are not accurately predicted by the widely-used Janosi-Hanamoto formula. We propose a new model in which two forces acting on the foot dominate the foot-terrain interaction - gross sliding friction and bulldozing resistance - and propose a model of how these forces act on the foot. We test this model on multiple soil types with different foot materials. Experimental data, collected on a testbench equipped with actuators and sensors identical to those deployed on a robot in the field, is used to validate our proposed model.

Autonomous navigation in dense vegetation remains an open challenge and is an area of major interest for the research community. In this paper we propose a novel traversability estimation method, the Forest Traversability Map, that gives autonomous ground vehicles the ability to navigate in harsh forests or densely vegetated environments. The method estimates travers ability in unstructured environments dominated by vegetation, void of any dominant human structures, gravel or dirt roads, with higher accuracy than the state of the art: we demonstrate an improvement of over 20% F1 score (from 0.71 to 0.91) on challenging real-world data. Our method is based on 3D voxel representation and introduces a robust colour fusion method to overcome occlusion and frequent changes of lighting conditions in these environments. We also introduce and fuse multi-return lidar measurements into our probabilistic map representation in a recursive manner. Finally, we include information of neighboring voxels to increase our ability to assess the terrain travers ability correctly. These measures improve the state-of-the-art results and allow for effective traversability estimation in very challenging, densely vegetated environments.

Multi-legged robots are effective at traversing rough terrain. However, terrains that include collapsible footholds (i.e. regions that can collapse when stepped on) remain a significant challenge, especially since such situations can be extremely difficult to anticipate using only exteroceptive sensing. State-of-the-art methods typically use various stabilisation techniques to regain balance and counter changing footholds. However, these methods are likely to fail if safe footholds are sparse and spread out or if the robot does not respond quickly enough after a foothold collapse. This paper presents a novel method for multi-legged robots to probe and test the terrain for collapses using its legs while walking. The proposed method improves on existing terrain probing approaches, and integrates the probing action into a walking cycle. A follow-the-leader strategy with a suitable gait and stance is presented and implemented on a hexapod robot. The proposed method is experimentally validated, demonstrating the robot can safely traverse terrain containing collapsible footholds.

A key capability for autonomous underground mining vehicles is real-time accurate localisation. While significant progress has been made, currently deployed systems have several limitations ranging from dependence on costly additional infrastructure to failure of both visual and range-sensor-based techniques in highly aliased or visually challenging environments. In our previous work, we presented a lightweight coarse vision-based localisation system that could map and then localise to within a few metres in an underground mining environment. However, this level of precision is insufficient for providing a cheaper, more reliable vision-based automation alternative to current range sensor-based systems. Here we present a new precision localisation system dubbed “LookUP”, which learns a neural-network-based pixel sampling strategy for estimating homographies based on ceiling-facing cameras without requiring any manual labelling. This new system runs in real time on limited computation resource and is demonstrated on two different underground mine sites, achieving real time performance at ~5 frames per second and a much improved average localisation error of ~1.2 metre.

This work presents Semi-Supervised SLAM - a method for developing a map suitable for coarse localization within an underground environment with minimal human intervention, with system characteristics driven by real-world requirements of major mining companies. This work leverages existing information common within a mining environment - namely a surveyed mine map - which is used to sparsely ground map locations within the mine environment, increasing map accuracy and allowing localization within a global frame. Map creation utilizes a low cost camera sensor and minimal user information to produce a map which can be used for single camera localization within a mining environment. We evaluate the localization capabilities of the proposed approach in depth by performing data collection on operational underground mining vehicles within an active underground mine and by simulating occlusions common to the environment such as dust and water. The proposed system is capable of producing maps which have an average localization error 2.5 times smaller than the next best performing method ORB-SLAM2, comparable localization performance to a state-of-the-art deep learning approach (which is not a feasible solution due to both compute and training requirements) and is robust to simulated environmental obscurants.

Minimally invasive arthroscopic surgery is a very challenging procedure that requires the manipulation of instruments in limited intraarticular space using distorted and sometimes uninformative images. Localizing the arthroscope reliably and at all times w.r.t. surrounding tissue is of fundamental importance to prevent unintended injury to patients. However, even highly-trained surgeons can struggle to localize the arthro-scope using poor image feedback. In this paper, we propose and demonstrate for the first time a visual Simultaneous Localisation and Mapping (SLAM) system, termed ArthroSLAM, capable of robustly and reliably localizing an arthroscope inside a human knee joint. The proposed system fuses the information obtained from the arthroscope, an external camera mounted on an arthroscope holder, and the odometry of a robotic arm manipulating the scope, in an Extended Kalman Filter framework. Also for the first time, we implement five alternative strategies for localization and compare them to our method in a realistic setup with a human cadaver knee joint. ArthroSLAM is shown to outperform the alternative strategies under various challenging conditions, localizing reliably and at all times with a mean Relative Pose Error of up to 1.4mm and 0.7°. Additional experiments conducted with degraded odometry data also validate the robustness of the method. An initial evaluation of the sparse map of a knee section computed by our method exhibits good morphological agreement. All results suggest that ArthroSLAM is a viable component for the robotic orthopedic surgical assistant of the future.

A critical requirement for safe autonomous navigation of a planetary rover is the ability to accurately estimate the traversability of the terrain. This work considers the problem of predicting the attitude and configuration angles of the platform from terrain representations that are often incomplete due to occlusions and sensor limitations. Using Gaussian Processes (GP) and exteroceptive data as training input, we can provide a continuous and complete representation of terrain traversability, with uncertainty in the output estimates. In this paper, we propose a novel method that focuses on exploiting the explicit correlation in vehicle attitude and configuration during operation by learning a kernel function from vehicle experience to perform GP regression. We provide an extensive experimental validation of the proposed method on a planetary rover. We show significant improvement in the accuracy of our estimation compared with results obtained using standard kernels (Squared Exponential and Neural Network), and compared to traversability estimation made over terrain models built using state-of-the-art GP techniques.

Operating in vegetated environments is a major challenge for autonomous robots. Obstacle detection based only on geometric features causes the robot to consider foliage, for example, small grass tussocks that could be easily driven through, as obstacles. Classifying vegetation does not solve this problem since there might be an obstacle hidden behind the vegetation. In addition, dense vegetation typically needs to be considered as an obstacle. This paper addresses this problem by augmenting probabilistic traversability map constructed from laser data with ultra-wideband radar measurements. An adaptive detection threshold and a probabilistic sensor model are developed to convert the radar data to occupancy probabilities. The resulting map captures the fine resolution of the laser map but clears areas from the traversability map that are induced by obstacle-free foliage. Experimental results validate that this method is able to improve the accuracy of traversability maps in vegetated environments.

It is well recognized that many scientifically interesting sites on Mars are located in rough terrains. Therefore, to enable safe autonomous operation of a planetary rover during exploration, the ability to accurately estimate terrain traversability is critical. In particular, this estimate needs to account for terrain deformation, which significantly affects the vehicle attitude and configuration. This paper presents an approach to estimate vehicle configuration, as a measure of traversability, in deformable terrain by learning the correlation between exteroceptive and proprioceptive information in experiments. We first perform traversability estimation with rigid terrain assumptions, then correlate the output with experienced vehicle configuration and terrain deformation using a multi-task Gaussian Process (GP) framework. Experimental validation of the proposed approach was performed on a prototype planetary rover and the vehicle attitude and configuration estimate was compared with state-of-the-art techniques. We demonstrate the ability of the approach to accurately estimate traversability with uncertainty in deformable terrain.

Motion planning for planetary rovers must consider control uncertainty in order to maintain the safety of the platform during navigation. Modelling such control uncertainty is difficult due to the complex interaction between the platform and its environment. In this paper, we propose a motion planning approach whereby the outcome of control actions is learned from experience and represented statistically using a Gaussian process regression model. This model is used to construct a control policy for navigation to a goal region in a terrain map built using an on-board RGB-D camera. The terrain includes flat ground, small rocks, and non-traversable rocks. We report the results of 200 simulated and 35 experimental trials that validate the approach and demonstrate the value of considering control uncertainty in maintaining platform safety.

This paper proposes an approach to obtain a localisation that is robust to smoke by exploiting multiple sensing modalities: visual and infrared (IR) cameras. This localisation is based on a state-of-the-art visual SLAM algorithm. First, we show that a reasonably accurate localisation can be obtained in the presence of smoke by using only an IR camera, a sensor that is hardly affected by smoke, contrary to a visual camera (operating in the visible spectrum). Second, we demonstrate that improved results can be obtained by combining the information from the two sensor modalities (visual and IR cameras). Third, we show that by detecting the impact of smoke on the visual images using a data quality metric, we can anticipate and mitigate the degradation in performance of the localisation by discarding the most affected data. The experimental validation presents multiple trajectories estimated by the various methods considered, all thoroughly compared to an accurate dGPS/INS reference.

This work aims to promote integrity in autonomous perceptual systems, with a focus on outdoor unmanned ground vehicles equipped with a camera and a 2D laser range finder. A method to check for inconsistencies between the data provided by these two heterogeneous sensors is proposed and discussed. First, uncertainties in the estimated transformation between the laser and camera frames are evaluated and propagated up to the projection of the laser points onto the image. Then, for each pair of laser scan-camera image acquired, the information at corners of the laser scan is compared with the content of the image, resulting in a likelihood of correspondence. The result of this process is then used to validate segments of the laser scan that are found to be consistent with the image, while inconsistent segments are rejected. Experimental results illustrate how this technique can improve the reliability of perception in challenging environmental conditions, such as in the presence of airborne dust.

This work aims to promote reliability and integrity in autonomous perceptual systems, with a focus on outdoor unmanned ground vehicle (UGV) autonomy. For this purpose, a comprehensive UGV system, comprising many different exteroceptive and proprioceptive sensors has been built. The first contribution of this work is a large, accurately calibrated and synchronised, multi-modal data-set, gathered in controlled environmental conditions, including the presence of dust, smoke and rain. The data have then been used to analyse the effects of such challenging conditions on perception and to identify common perceptual failures. The second contribution is a presentation of methods for mitigating these failures to promote perceptual integrity in adverse environmental conditions.

This paper presents an approach to autonomously monitor the behavior of a robot endowed with several navigation and locomotion modes, adapted to the terrain to traverse. The mode selection process is done in two steps: the best suited mode is firstly selected on the basis of initial information or a qualitative map built on-line by the robot. Then, the motions of the robot are monitored by various processes that update mode transition probabilities in a Markov system. The paper focuses on this latter selection process: the overall approach is depicted, and preliminary experimental results are presented.

This article presents an approach to improve and monitor the behavior of a skid-steering rover on rough terrains. An adaptive locomotion control generates speeds references to avoid slipping situations. An enhanced odometry provides a better estimation of the distance travelled. A probabilistic classification procedure provides an evaluation of the locomotion efficiency on-line, with a detection of locomotion faults. Results obtained with a Marsokhod rover are presented throughout the paper.