In this paper, we present a method to build precise local terrain maps for an autonomous vehicle from vision and LiDAR that surpass most existing maps used in the field, both in the details they represent and in efficiency of construction. The high level of detail is obtained by spatio-temporal fusion of data from multiple, complementary sensors in a grid map. The map not only consists of obstacle probabilities, but contains different features of the environment: elevation, color, infrared reflectivity, terrain slopes and surface roughness. Still, an efficient way to manage the map's memory allows us to build the maps online on-board our autonomous vehicle. As we demonstrate by describing some of its applications, the maps can serve as a unified representation to solve further perception and navigation problems without having to resort to individual sensor data again. The proposed terrain maps have proven their robustness and precision in many real-world scenarios, leading to award-winning performances at international robotics competitions.

In this paper, we introduce a new geometric 3D feature combined with a clustering approach. Besides 3D data provided by a LiDAR point cloud, reflectivity information is used to further enhance the descriptivity of the feature. The proposed feature can be extracted and compared in real-time. Similar parts of an object, such as features belonging to an automobile headlight, are automatically clustered in an object model without explicit specification. Additionally, we provide a method for autonomous vehicles to automatically learn the shapes of observed moving objects and use them for real-time classification. The resulting object models consisting of the extracted feature clusters are interpretable by humans.

Precise perception of a vehicle's surrounding is crucial for safe autonomous driving. It requires a high sensor resolution and a large field of view. Active perception, i.e. the redirection of a sensor's focus of attention, is an approach to provide both. With active perception, however, the selection of an appropriate sensor orientation becomes necessary. This paper presents a method for determining the sensor orientation in urban traffic scenarios based on three criteria: the importance of traffic participants w.r.t. the current situation, the available information about traffic participants while considering alternative sensor orientations as well as sensor coverage of the vehicle's relevant surrounding area.

Recently, the introduction of dense, long-range 3D sensors has facilitated tracking of arbitrary objects. Especially in the context of autonomous driving, other traffic participants driving the streets usually stay well-segmented from each other. In contrast, pedestrians or bicyclists do not always stay on the road and they often get close to static structure of the environment, e.g. traffic lights or signs, bushes, parking cars etc. These objects are not as easy to segment, often resulting in an under-segmentation of the scene and wrong tracking results. This paper addresses the problem of tracking moving objects that are hard to segment from their static surroundings by utilizing top-down knowledge about the geometry of existing tracks during segmentation. This includes methods for discerning static from moving objects to reduce the rate of false positive tracks as well as a classification of tracks into pedestrian, bicyclist, motor bike, passenger car, van and truck classes by considering an objects appearance and motion history. The proposed tracking system is experimentally validated in challenging real-world inner-city traffic scenes.

Autonomous vehicles promise numerous improvements to vehicular traffic: an increase in both highway capacity and traffic flow because of faster response times, less fuel consumption and pollution thanks to more foresighted driving, and hopefully fewer accidents thanks to collision avoidance systems. In addition, drivers can save time for more useful activities. In order for these vehicles to safely operate in everyday traffic or in harsh off-road environments, a multitude of problems in perception, navigation, and control have to be solved. This paper gives an overview of the most current trends in autonomous vehicles, highlighting the concepts common to most successful systems as well as their differences. It concludes with an outlook into the promising future of autonomous vehicles.

Three-dimensional ultrasound can be an effective imaging modality for image-guided interventions since it enables visualization of both the instruments and the tissue. For robotic applications, its realtime frame rates create the potential for image-based instrument tracking and servoing. These capabilities can enable improved instrument visualization, compensation for tissue motion as well as surgical task automation. Continuum robots, whose shape comprises a smooth curve along their length, are well suited for minimally invasive procedures. Existing techniques for ultrasound tracking, however, are limited to straight, laparoscopic-type instruments and thus are not applicable to continuum robot tracking. Toward the goal of developing tracking algorithms for continuum robots, this paper presents a method for detecting a robot comprised of a single constant curvature in a 3D ultrasound volume. Computational efficiency is achieved by decomposing the six-dimensional circle estimation problem into two sequential three-dimensional estimation problems. Simulation and experiment are used to evaluate the proposed method.

This paper describes a fast method for segmentation of large-size long-range 3D point clouds that especially lends itself for later classification of objects. Our approach is targeted at high-speed autonomous ground robot mobility, so real-time performance of the segmentation method plays a critical role. This is especially true as segmentation is considered only a necessary preliminary for the more important task of object classification that is itself computationally very demanding. Efficiency is achieved in our approach by splitting the segmentation problem into two simpler subproblems of lower complexity: local ground plane estimation followed by fast 2D connected components labeling. The method's performance is evaluated on real data acquired in different outdoor scenes, and the results are compared to those of existing methods. We show that our method requires less runtime while at the same time yielding segmentation results that are better suited for later classification of the identified objects.

Autonomous navigation in unstructured environments like forest or country roads with dynamic objects remains a challenging task, particularly with respect to the perception of the environment using multiple different sensors. The problem has been addressed from both, the computer vision community as well as from researchers working with laser range finding technology, like the Velodyne HDL-64. Since cameras and LIDAR sensors complement one another in terms of color and depth perception, the fusion of both sensors is reasonable in order to provide color images with depth and reflectance information as well as 3D LIDAR point clouds with color information. In this paper we propose a method for sensor synchronization, especially designed for dynamic scenes, a low-level fusion of the data of both sensors and we provide a solution for the occlusion problem that arises in conjunction with different viewpoints of the fusioned sensors.

This paper describes a LIDAR-based perception system for ground robot mobility, consisting of 3D object detection, classification and tracking. The presented system was demonstrated on-board our autonomous ground vehicle MuCAR-3, enabling it to safely navigate in urban traffic-like scenarios as well as in off-road convoy scenarios. The efficiency of our approach stems from the unique combination of 2D and 3D data processing techniques. Whereas fast segmentation of point clouds into objects is done in a 2¿D occupancy grid, classifying the objects is done on raw 3D point clouds. For fast object feature extraction, we advocate the use of statistics of local point cloud properties, captured by histograms over point features. In contrast to most existing work on 3D point cloud classification, where real-time operation is often impossible, this combination allows our system to perform in real-time at 0.1s frame-rate.