The successful deployment of deep learning-based techniques for autonomous systems is highly dependent on the data availability for the respective system in its deployment environment. Especially for unstructured outdoor environments, very few datasets exist for even fewer robotic platforms and scenarios. In an earlier work, we presented the German Outdoor and Offroad Dataset (GOOSE) framework along with 10000 multimodal frames from an offroad vehicle to enhance the perception capabilities in unstructured environments. In this work, we address the generalizability of the GOOSE framework. To accomplish this, we open-source the GOOSE-Ex dataset, which contains additional 5000 labeled multimodal frames from various completely different environments, recorded on a robotic excavator and a quadruped platform. We perform a comprehensive analysis of the semantic segmentation performance on different platforms and sensor modalities in unseen environments. In addition, we demonstrate how the combined datasets can be utilized for different downstream applications or competitions such as offroad navigation, object manipulation or scene completion. The dataset, its platform documentation and pre-trained state-of-the-art models for offroad perception will be made available on https://goose-dataset.de/.

An increasing number of datasets sharing similar domains for semantic segmentation have been published over the past few years. But despite the growing amount of overall data, it is still difficult to train bigger and better models due to inconsistency in taxonomy and/or labeling policies of different datasets. To this end, we propose a knowledge distillation approach that also serves as a label space unification method for semantic segmentation. In short, a teacher model is trained on a source dataset with a given taxonomy, then used to pseudo-label additional data for which ground truth labels of a related label space exist. By mapping the related taxonomies to the source taxonomy, we create constraints within which the model can predict pseudo-labels. Using the improved pseudo-labels we train student models that consistently outperform their teachers in two challenging domains, namely urban and off-road driving. Our ground truth-corrected pseudo-labels span over 12 and 7 public datasets with 388.230 and 18.558 images for the urban and off-road domains, respectively, creating the largest compound datasets for autonomous driving to date.

For autonomous driving applications, knowledge of the ego position, orientation, and velocity is a necessary prerequisite for recognizing landmarks and moving targets. We use radar sensors for the determination of these quantities in a radar odometry system. Radar odometry uses the advantage of a direct measurement of the radial speed using radar sensors. Radar sensors are less susceptible to bad weather and lighting conditions than camera and lidar sensors. In addition, radar data is not susceptible to wheel slippage or blocked wheels compared with wheel speed measurements. However, radar data is still susceptible to clutter. In order to achieve a combination of good precision under optimal conditions and good precision under adverse weather conditions, we fuse measurements from radar sensors, wheel speed sensors and the gyrometer. We do not simply combine these measurements according to assumed covariances. Instead, we check the plausibility of the measurements based on their likelihood. Subsequently, we weight the results of the sensor combinations accordingly. The decision about sensor weighting is carried out in a principled, probabilistic manner and adaptively with regard to environmental influences. We validate our approach using real data. Our approach is more precise under adverse conditions than using wheel speed sensors and gyrometers alone. On the other hand, it is more precise under good conditions than using only radar measurements.

The potential for deploying autonomous systems can be significantly increased by improving the perception and interpretation of the environment. However, the development of deep learning-based techniques for autonomous systems in unstructured outdoor environments poses challenges due to limited data availability for training and testing. To address this gap, we present the German Outdoor and Offroad Dataset (GOOSE), a comprehensive dataset specifically designed for unstructured outdoor environments. The GOOSE dataset incorporates 10000 labeled pairs of images and point clouds, which are utilized to train a range of state-of-the-art segmentation models on both image and point cloud data. We open source the dataset, along with an ontology for unstructured terrain, as well as dataset standards and guidelines. This initiative aims to establish a common framework, enabling the seamless inclusion of existing datasets and a fast way to enhance the perception capabilities of various robots operating in unstructured environments. This framework also makes it possible to query data for specific weather conditions or sensor setups from a database in future. The dataset, pre-trained models for offroad perception, and additional documentation can be found at https://goose-dataset.de/.

This paper investigates the impact of a corrupted map on the quality of GNSS-free localization. Using a High Definition (HD) map, we gradually remove and shift features in space. From originally 200 features per kilometer, we randomly discard up to 90% of our map and introduce perturbations of up to +/-0.9m in each direction to our HD map features. Our evaluation is based on a GNSS-free LiDAR-SLAM, which utilizes panoptic segmentation to observe geometric primitives. In the back-end, a graph optimization is performed to estimate the vehicle’s and the landmarks’ positions. Further, we also assess the impact of conducting pure localization instead of Simultaneous Localization and Mapping (SLAM). The effects are evaluated using Mean Absolute Error for accuracy evaluation. For stability assessment, we calculate two percentiles: one for deviations of 0.3m or less and the second for deviations of 2m or less. We conduct real-life experiments with our testing vehicle which is equipped with a reference system based on RTK-GNSS. We use a commercial, standardized HD map of our suburban campus. Our findings aim to provide insights into the tradeoffs between mapping effort, map maintenance and accurate positioning. We demonstrate that a simultaneous mapping of the environment enhances the stability of localization.

The random hypersurface model is well-suited to describe extended target contours. Its applicability is limited only by the mild assumption that the target contour has to be star convex. Gaussian processes provide a sound way to estimate the contour functions, and the ability to model the contour uncertainty in a detailed way at different contour points. However, the association of measurements to target contour points is not optimal in current implementations using Gaussian Processes and the random hypersurface model. In this work, we provide an improved approach compared to the standard approach. The standard approach projects measurements radially onto the predicted contour. Our approach provides expected measurements matching the physical reality of the measurement process more closely. In addition, we perform the association of the whole batch of measurements to the expected contour measurements at once. Compared to a sequential association of individual measurements, this leads to a better association decision.

Central low-level sensor data fusion approaches are getting more popular in advanced driver assistant systems. They allow for the resolution of ambiguities in the retrieval of environmental information on the basis of a large, raw data pool. Hereby, one emerging challenge is the unification of sensor data of different formats and sensor types. A popular intermediate layer of data is given by spatial occupancy grids. The conversion of a discrete list of radar detections, which is a commonly utilized measurement format, is problematic due to the sparse spatial resolution. This work addresses this conversion by interpolating the data spatially using generic sensor model knowledge. Traditional approaches derive occupancy evidence in the vicinity of a detection. In addition, we analyze spatial and kinematic properties derived from Doppler measurements, compute likelihoods that multiple detections are caused by the same object and deduce the space between them accordingly. The incorporation of sensor parameters allows full-and short-range radars to be used generically. In addition, we outline the deduction of free space evidence. The elaborated models and algorithms are evaluated on realworld datasets and discussed w.r.t. their applicability in a subsequent Dempster-Shafer-based sensor data fusion approach.

The radar sensor is an important component in autonomous driving applications. Compared with other sensor types like LiDAR or camera, the radar sensor comes with best weather robustness and ease of integration. Its exclusive capability to measure electromagnetic reflectivity and Doppler-derived radial speeds plays a major role in environment perception applications. However, the processing chain of the radar is more complex and often results in unintuitive measurement effects. In this paper, we explain the technical background of high-resolution radar detections. We give a probabilistic architecture modeling Doppler and Micro-Doppler measurements and their influence on the Monopulse-based azimuth angle, the peak detection and the resulting measurement data. Examples accompany the description of the modeling steps.

For the purpose of extended object tracking in multiple hypothesis tracking algorithms such as the Gaussian mixture probability hypothesis density filter (GMPHD), we develop an approach for the combination of different contour estimates. The developed approach works for tracking algorithms that represent target shapes using contour functions to describe the target shape as the distance of the contour to a reference point over the angle. In a heterogeneous multiple sensor setup, the individual sensors' measurements lead to different extent estimates due to their individual measurement principles. Thus a straight forward use of the extended object state in the traditional merging algorithm either results in unexpected shapes, or tracks cannot be merged due to the differing shape of the objects. Our merging procedure explicitly takes the extent estimates into account by using a merging function. The choice of the merging function provides the means to reach objectives such as a conservative or a generous extent estimate. We evaluate the approach using simulated multisensor data in a GMPHD filter. Compared to the traditional merging method, our approach results in better shape estimates.

We present a novel approach to trajectory generation that enables an autonomous vehicle to accurately follow a lead vehicle tracked by on-board sensors. In contrast to other approaches, we ignore the structure of the environment (e.g., lane markings), focusing purely on following the path driven by the vehicle ahead. Based on the leader's velocity, its distance to the ego vehicle and its recorded path, a continuous-curvature trajectory is generated using Sequential Quadratic Programming. As the optimization process takes the ego vehicle's kinematic and dynamic constraints into account, the resulting trajectory is guaranteed to be feasible and safe. The algorithm was tested extensively during practical experiments with an actual autonomous car. Our tests were conducted in both on- and off-road environments, with speeds ranging from 1 m/s to 15 m/s. Ground truth data shows that the system achieves a high degree of accuracy even in difficult scenarios.

The Munich Cognitive Autonomous Robot Car 3rdGeneration (MuCAR-3) has won several international achievements in the past. Recently, the system's control architecture (meaning the interplay between perception, planning and control) was overhauled. Our goals were to simplify the interaction between modules as well as to meet higher requirements for both smoothness and precision. The decoupling of modules helps with tackling more challenging scenarios and facilitates the development of each module. Since state machines struggle with scalability, its interactions with other modules were minimized. We now use a generalized planning layer rather than so-called maneuvers. This paper aims at showcasing the difference between our previous and current architecture. We focus on the improvements that were achieved even for very simple scenarios - in this case off-road platooning. Using the same control algorithms, we achieve both improvements in smoothness and precision, two classically orthogonal goals. Tests were conducted in simulation and verified with MuCAR-3 on our test site.

In recent years vehicles have been equipped with more and more sensors for environment perception. Among these sensors are cameras, RADAR, single-layer and multi-layer LiDAR. One key challenge for the fusion of these sensors is sensor calibration. In this paper we present a novel extrinsic calibration algorithm based on sensor odometry. Given the time-synchronized delta poses of two sensors our technique recursively estimates the relative pose between these sensors. The method is generic in that it can be used to estimate complete 6DOF poses, given the sensors provide a 6DOF odometry, as well as 3DOF poses (planar offset and yaw angle) for sensors providing a 3DOF odometry, like a single-beam LiDAR. We show that the proposed method is robust against motion degeneracy and present results on both simulated and real world data using an inertial navigation system (INS) and a stereo camera system.

This paper presents a robust method for vehicle tracking with a monocular camera. A previously published model-based tracking method uses a particle filter which needs an initial vehicle hypothesis both at system start and in case of a tracking loss. We present a template-based solution using different features to estimate a 3D vehicle pose roughly but fast. Combining model- and template-based object tracking keeps the advantages of each algorithm: Precise estimation of the 3D vehicle pose and velocity combined with a fast (re-) initialization approach. The improved tracking system was evaluated while driving autonomously in urban and unstructured environments. The results show that poorly visible vehicles can be tracked during different weather conditions in real-time.

Autonomous vehicles promise numerous improvements to vehicular traffic: an increase in both highway capacity and traffic flow because of faster response times, less fuel consumption and pollution thanks to more foresighted driving, and hopefully fewer accidents thanks to collision avoidance systems. In addition, drivers can save time for more useful activities. In order for these vehicles to safely operate in everyday traffic or in harsh off-road environments, a multitude of problems in perception, navigation, and control have to be solved. This paper gives an overview of the most current trends in autonomous vehicles, highlighting the concepts common to most successful systems as well as their differences. It concludes with an outlook into the promising future of autonomous vehicles.

In this paper we describe a novel approach to model-based monocular vehicle tracking out of a moving vehicle using active vision. The designed algorithm can cope with cluttered color images, complex lighting conditions as well as partial occlusion of the leading vehicle and is able to detect and track a vehicle even within unstructured offroad environments. Thanks to the used 3D model which describes the characteristic vehicle geometry and appearance in terms of vertexes, edges and colored surfaces, no special visual markers are required. The knowledge of vehicle's geometry and appearance gained from the model are used within a particle filter to estimate the 6DoF position relative to the ego vehicle, thereby fusing edge as well as color information. We successfully use the proposed algorithm for pure vision based autonomous offroad convoy driving.

Three-dimensional ultrasound can be an effective imaging modality for image-guided interventions since it enables visualization of both the instruments and the tissue. For robotic applications, its realtime frame rates create the potential for image-based instrument tracking and servoing. These capabilities can enable improved instrument visualization, compensation for tissue motion as well as surgical task automation. Continuum robots, whose shape comprises a smooth curve along their length, are well suited for minimally invasive procedures. Existing techniques for ultrasound tracking, however, are limited to straight, laparoscopic-type instruments and thus are not applicable to continuum robot tracking. Toward the goal of developing tracking algorithms for continuum robots, this paper presents a method for detecting a robot comprised of a single constant curvature in a 3D ultrasound volume. Computational efficiency is achieved by decomposing the six-dimensional circle estimation problem into two sequential three-dimensional estimation problems. Simulation and experiment are used to evaluate the proposed method.

This paper describes a method for an object related reactive offset maneuver increasing the number of action alternatives for an autonomous ground vehicle. The method improves a variety of time-critical maneuvers as for example merging into moving traffic, changing lanes, passing other traffic participants as well as emergency obstacle avoidance without having to replan a path using high-level planning methods. It is especially targeted at autonomous driving on country roads. As oncoming traffic increases the relative speed between participants, a quick response to changes of the traffic situation are critically important. Moreover the method does not require constant vehicle velocities throughout a lane change maneuver as most existing approaches do. Simulations show improvements of the behaviour capabilities of an autonomous vehicle using the presented lateral reactive offset maneuvers.

Autonomous navigation in unstructured environments like forest or country roads with dynamic objects remains a challenging task, particularly with respect to the perception of the environment using multiple different sensors. The problem has been addressed from both, the computer vision community as well as from researchers working with laser range finding technology, like the Velodyne HDL-64. Since cameras and LIDAR sensors complement one another in terms of color and depth perception, the fusion of both sensors is reasonable in order to provide color images with depth and reflectance information as well as 3D LIDAR point clouds with color information. In this paper we propose a method for sensor synchronization, especially designed for dynamic scenes, a low-level fusion of the data of both sensors and we provide a solution for the occlusion problem that arises in conjunction with different viewpoints of the fusioned sensors.

This paper describes a LIDAR-based perception system for ground robot mobility, consisting of 3D object detection, classification and tracking. The presented system was demonstrated on-board our autonomous ground vehicle MuCAR-3, enabling it to safely navigate in urban traffic-like scenarios as well as in off-road convoy scenarios. The efficiency of our approach stems from the unique combination of 2D and 3D data processing techniques. Whereas fast segmentation of point clouds into objects is done in a 2¿D occupancy grid, classifying the objects is done on raw 3D point clouds. For fast object feature extraction, we advocate the use of statistics of local point cloud properties, captured by histograms over point features. In contrast to most existing work on 3D point cloud classification, where real-time operation is often impossible, this combination allows our system to perform in real-time at 0.1s frame-rate.