Teleoperated driving (ToD) is increasingly considered as a fallback solution for autonomous driving. Up to now, ToD requires a highly reliable mobile network capable of transmitting multiple video streams with low latency. Recently, significant advancements have been made in vehicular sensors and perception algorithms, which have huge implications for the challenges faced in ToD. We envisage that a real-time digital twin that tracks the remote vehicle's environment will play a crucial role in reducing the required communication band-width and providing a more convenient teleoperator interface, ultimately enhancing safety of ToD in crowded environments. Furthermore, it would allow various degrees of cooperation between automated driving functionalities and human teleoperators. In this paper, the concept of digital twin for ToD is outlined and a proof of concept is implemented using a real-world vehicle simulator and a teleoperator hardware setup. A significant reduction in required bandwidth is reported by transmitting less video data and reconstructing the scene from the digital twin.

Odometry forms an important component of many manned and autonomous systems. In the rail industry in particular, having precise and robust odometry is crucial for the correct operation of the Automatic Train Protection systems that ensure the safety of high-speed trains in operation around the world. Two problems commonly encountered in such odometry systems are miscalibration of the wheel encoders and slippage of the wheels under acceleration and braking, resulting in incorrect velocity estimates. This paper introduces an odometry system that addresses these problems. It comprises of an Extended Kalman Filter that tracks the calibration of the wheel encoders as state variables, and a measurement pre-processing stage called Sensor Consensus Analysis (SCA) that scales the uncertainty of a measurement based on how consistent it is with the measurements from the other sensors. SCA uses the statistical z-test to determine when an individual measurement is inconsistent with the other measurements, and scales the uncertainty until the z-test passes. This system is demonstrated on data from German Intercity-Express highspeed trains and it is shown to successfully deal with errors due to miscalibration and wheel slip.

Autonomous Underwater Vehicles (AUVs) gather large volumes of visual imagery, which can help monitor marine ecosystems and plan future surveys. One key task in marine ecology is benthic habitat mapping, the classification of large regions of the ocean floor into broad habitat categories. Since visual data only covers a small fraction of the ocean floor, traditional habitat mapping is performed using shipborne acoustic multi-beam data, with visual data as ground truth. However, given the high resolution and rich textural cues in visual data, an ideal approach should explicitly utilise visual features in the classification process. To this end, we propose a multimodal model which utilises visual data and shipborne multi-beam bathymetry to perform both classification and sampling tasks. Our algorithm learns the relationship between both modalities, but is also effective when visual data is missing. Our results suggest that by performing multimodal learning, classification performance is improved in scenarios where visual data is unavailable, such as the habitat mapping scenario. We also demonstrate empirically that the model is able to perform generative tasks, producing plausible samples from the underlying data-generating distribution.

A method for detecting changes in the environment using only vision sensors is presented. We demonstrate that optical flow can be used to detect these changes at key locations in outdoor scenarios in difficult and varying lighting conditions. These key locations are used as nodes in a topological mapping and localization framework. To close the loop we employ a bag-of-words methodology. We show that bag-of-words methods can be used in real-time on a standard computer to detect loop closures in sparse topological maps. Experimental results from field trials using our quad-rotor UAV demonstrate the capability of the proposed scene change detection method.

A method for calculating visual odometry for ground vehicles with car-like kinematic motion constraints similar to Ackerman's steering model is presented. By taking advantage of this non-holonomic driving constraint we show a simple and practical solution to the odometry calculation by clever placement of a single camera. The method has been implemented successfully on a large industrial forklift and a Toyota Prado SUV. Results from our industrial test site is presented demonstrating the applicability of this method as a replacement for wheel encoder-based odometry for these vehicles.

A coverage algorithm is an algorithm that deploys a strategy as to how to cover all points in terms of a given area using some set of sensors. In the past decades a lot of research has gone into development of coverage algorithms. Initially, the focus was coverage of structured and semi-structured indoor areas, but with time and development of better sensors and introduction of GPS, the focus has turned to outdoor coverage. Due to the unstructured nature of an outdoor environment, covering an outdoor area with all its obstacles and simultaneously performing reliable localization is a difficult task. In this paper, two path planning algorithms suitable for solving outdoor coverage tasks are introduced. The algorithms take into account the kinematic constraints of an under-actuated car-like vehicle, minimize trajectory curvatures, and dynamically avoid detected obstacles in the vicinity, all in real-time. We demonstrate the performance of the coverage algorithm in the field by achieving 95% coverage using an autonomous tractor mower without the aid of any absolute localization system or constraints on the physical boundaries of the area.