Localizing a camera in a given map is essential for vision-based navigation. In contrast to common methods for visual localization that use maps acquired with cameras, we propose a novel approach, which tracks the pose of monocular camera with respect to a given 3D LiDAR map. We employ a visual odometry system based on local bundle adjustment to reconstruct a sparse set of 3D points from image features. These points are continuously matched against the map to track the camera pose in an online fashion. Our approach to visual localization has several advantages. Since it only relies on matching geometry, it is robust to changes in the photometric appearance of the environment. Utilizing panoramic LiDAR maps additionally provides viewpoint invariance. Yet low-cost and lightweight camera sensors are used for tracking. We present real-world experiments demonstrating that our method accurately estimates the 6-DoF camera pose over long trajectories and under varying conditions.

Reliable detection and avoidance of obstacles is a crucial prerequisite for autonomously navigating robots as both guarantee safety and mobility. To ensure safe mobility, the obstacle detection needs to run online, thereby taking limited resources of autonomous systems into account. At the same time, robust obstacle detection is highly important. Here, a too conservative approach might restrict the mobility of the robot, while a more reckless one might harm the robot or the environment it is operating in. In this paper, we present a terrain-adaptive approach to obstacle detection that relies on 3D-Lidar data and combines computationally cheap and fast geometric features, like step height and steepness, which are updated with the frequency of the lidar sensor, with semantic terrain information, which is updated with at lower frequency. We provide experiments in which we evaluate our approach on a real robot on an autonomous run over several kilometers containing different terrain types. The experiments demonstrate that our approach is suitable for autonomous systems that have to navigate reliable on different terrain types including concrete, dirt roads and grass.

Networks of laser range finders are a popular tool for monitoring large cluttered areas and to track people. Whenever multiple scanners are used for this purpose, one major problem is how to determine the relative positions of all the scanners. In this paper, we present a novel approach to calibrate a network of multiple planar laser range finders scanning horizontally. To robustly deal with the potentially restricted overlap between the fields of view, our approach only requires a dynamic object, e.g., a person, moving through the observed area. We employ a RANSAC-like algorithm to find the correspondences between the measurements of the different laser range finders. Based on these correspondences we formulate a graph-based optimization problem to determine the maximum likelihood extrinsic parameters of the sensor network. Furthermore, we present a method to evaluate the consistency of the resulting calibration based on visibility constraints. Experiments on real and simulated data show that the proposed approach yields better results than techniques that only perform pairwise calibration.

To determine the pose of a vehicle is a fundamental problem in mobile robotics. Most approaches relate the current sensor observations to a map generated with previously acquired data of the same system or by another system with a similar sensor setup. Unfortunately, previously acquired data is not always available. In outdoor settings, GPS is a very useful tool to determine a global estimate of the vehicles pose. Unfortunately, GPS tends to be unreliable in situations in which a clear view to the sky is restricted. Yet, one can make use of publicly available map material as prior information. In this paper, we describe an approach to localize a robot equipped with a 3D range scanner with respect to a road network created from OpenStreetMap data. To successfully localize a mobile robot we propose a road classification scheme for 3D range data together with a novel sensor model, which relates the classification results to a road network. Compared to other approaches, our system does not require the robot to actually travel on the road network. We evaluate our approach in extensive experiments on simulated and real data and compare favorably to two state-of-the-art methods on those data.

Laser range scanners are commonly used in mobile robotics to enable a robot to sense the spatial configuration of its environment. In addition to the range measurements, most scanners provide remission values, representing the intensity of the returned light pulse. These values add a visual component to the measurement and can be used to improve reasoning on the data. Unfortunately, a remission value does not directly tell us how bright a measured surface is in the infrared spectrum. Rather, it varies with respect to the incidence angle and the range at which it was measured. In addition, multiple scanners typically do not agree upon the values of a certain surface. In this paper, we present a calibration method for remission values of multiple laser scanners considering dependencies in range, incidence angle of the measured surface, and the respective scanner unit. Our system learns the calibration parameters based on a set of registered point clouds. It uses a graph optimization scheme to minimize the error between different measurements, so that all involved scanners yield consistent reflection values, independent of the perspective from which the corresponding surface is observed.

The ability to safely navigate is a crucial prerequisite for truly autonomous systems. A robot has to distinguish obstacles from traversable ground. Failing on this task can cause great damage or restrict the robots movement unnecessarily. Due to the security relevance of this problem, great effort is typically spent to design models for individual robots and sensors, and the complexity of such models is correlated to the complexity of the environment and the capabilities of the robot. We present a semi supervised learning approach, where the robot learns its traversability capabilities from a human operating it. From this partially and only positive labeled training data, our approach infers a model for the traversability analysis, thereby requiring very little manual effort for the human. In practical experiments we show that our method can be used for robots that need to reliably navigate on dirt roads as well as for robots that have very restricted traversability capabilities.

Accurate localization in indoor environments is widely regarded as a key opener for various location-based services. Despite tremendous advancements in the development of innovative sensor concepts, the most effective and accurate solutions to this problem make use of a map computed from sensory data. In this paper, we present an efficient approach to localize an RGB-D smartphone or tablet that only makes use of a two-dimensional outline of the environment as a map as it is typically available from architectural drawings. Our technique employs a particle filter to estimate the 6DoF pose. We propose a sensor model that robustly estimates the likelihood of measurements and accommodates the disagreements between floor plans and real world data. In extensive experiments, we demonstrate that our approach is able to globally localize a user in a given 2D floor plan using a Google Tango device and to accurately track the user in such an environment.

Over the past years, there has been a tremendous progress in the area of robot navigation. Most of the systems developed thus far, however, are restricted to indoor scenarios, non-urban outdoor environments, or road usage with cars. Urban areas introduce numerous challenges to autonomous mobile robots as they are highly complex and in addition to that dynamic. In this paper, we present a navigation system for pedestrian-like autonomous navigation with mobile robots in city environments. We describe different components including a SLAM system for dealing with huge maps of city centers, a planning approach for inferring feasible paths taking also into account the traversability and type of terrain, and a method for accurate localization in dynamic environments. The navigation system has been implemented and tested in several large-scale field tests in which the robot Obelix managed to autonomously navigate from our university campus over a 3.3 km long route to the city center of Freiburg.

In this paper we address the topic of feature extraction in 3D point cloud data for object recognition and pose identification. We present a novel interest keypoint extraction method that operates on range images generated from arbitrary 3D point clouds, which explicitly considers the borders of the objects identified by transitions from foreground to background. We furthermore present a feature descriptor that takes the same information into account. We have implemented our approach and present rigorous experiments in which we analyze the individual components with respect to their repeatability and matching capabilities and evaluate the usefulness for point feature based object detection methods.

Place recognition, i.e., the ability to recognize previously seen parts of the environment, is one of the fundamental tasks in mobile robotics. The wide range of applications of place recognition includes localization (determine the initial pose), SLAM (detect loop closures), and change detection in dynamic environments. In the past, only relatively little work has been carried out to attack this problem using 3D range data and the majority of approaches focuses on detecting similar structures without estimating relative poses. In this paper, we present an algorithm based on 3D range data that is able to reliably detect previously seen parts of the environment and at the same time calculates an accurate transformation between the corresponding scan-pairs. Our system uses the estimated transformation to evaluate a candidate and in this way to more robustly reject false positives for place recognition. We present an extensive set of experiments using publicly available datasets in which we compare our system to other state-of-the-art approaches.

The problem of place recognition appears in different mobile robot navigation problems including localization, SLAM, or change detection in dynamic environments. Whereas this problem has been studied intensively in the context of robot vision, relatively few approaches are available for three-dimensional range data. In this paper, we present a novel and robust method for place recognition based on range images. Our algorithm matches a given 3D scan against a database using point features and scores potential transformations by comparing significant points in the scans. A further advantage of our approach is that the features allow for a computation of the relative transformations between scans which is relevant for registration processes. Our approach has been implemented and tested on different 3D data sets obtained outdoors. In several experiments we demonstrate the advantages of our approach also in comparison to existing techniques.

In this paper we describe a novel algorithm for constructing a compact representation of 3D laser range data. Our approach extracts an alphabet of local scans from the scene. The words of this alphabet are used to replace recurrent local 3D structures, which leads to a substantial compression of the entire point cloud. We optimize our model in terms of complexity and accuracy by minimizing the Bayesian information criterion (BIC). Experimental evaluations on large real-world data show that our method allows robots to accurately reconstruct environments with as few as 70 words.

We present an algorithm for learning 3D object models from partial object observations. The input to our algorithm is a sequence of 3D laser range scans. Models learned from the objects are represented as point clouds. Our approach can deal with partial views and it can robustly learn accurate models from complex scenes. It is based on an iterative matching procedure which attempts to recursively merge similar models. The alignment between models is determined using a novel scan registration procedure based on range images. The decision about which models to merge is performed by spectral clustering of a similarity matrix whose entries represent the consistency between different models.

In this paper, we address the problem of creating an objective benchmark for comparing SLAM approaches. We propose a framework for analyzing the results of SLAM approaches based on a metric for measuring the error of the corrected trajectory. The metric uses only relative relations between poses and does not rely on a global reference frame. The idea is related to graph-based SLAM approaches in the sense that it considers the energy needed to deform the trajectory estimated by a SLAM approach to the ground truth trajectory. Our method enables us to compare SLAM approaches that use different estimation techniques or different sensor modalities since all computations are made based on the corrected trajectory of the robot. We provide sets of relative relations needed to compute our metric for an extensive set of datasets frequently used in the SLAM community. The relations have been obtained by manually matching laser-range observations. We believe that our benchmarking framework allows the user an easy analysis and objective comparisons between different SLAM approaches.

A mobile robot that accomplishes high level tasks needs to be able to classify the objects in the environment and to determine their location. In this paper, we address the problem of online object detection in 3D laser range data. The object classes are represented by 3D point-clouds that can be obtained from a set of range scans. Our method relies on the extraction of point features from range images that are computed from the point-clouds. Compared to techniques that directly operate on a full 3D representation of the environment, our approach requires less computation time while retaining the robustness of full 3D matching. Experiments demonstrate that the proposed approach is even able to deal with partially occluded scenes and to fulfill the runtime requirements of online applications.

The ability to learn a map of the environment is important for numerous types of robotic vehicles. In this paper, we address the problem of learning a visual map of the ground using flying vehicles. We assume that the vehicles are equipped with one or two low-cost downlooking cameras in combination with an attitude sensor. Our approach is able to construct a visual map that can later on be used for navigation. Key advantages of our approach are that it is comparably easy to implement, can robustly deal with noisy camera images, and can operate either with a monocular camera or a stereo camera system. Our technique uses visual features and estimates the correspondences between features using a variant of the progressive sample consensus (PROSAC) algorithm. This allows our approach to extract spatial constraints between camera poses that can then be used to address the simultaneous localization and mapping (SLAM) problem by applying graph methods. Furthermore, we address the problem of efficiently identifying loop closures. We performed several experiments with flying vehicles that demonstrate that our method is able to construct maps of large outdoor and indoor environments.

In this paper, we address the problem of learning 3D maps of the environment using a cheap sensor setup which consists of two standard web cams and a low cost inertial measurement unit. This setup is designed for lightweight or flying robots. Our technique uses visual features extracted from the web cams and estimates the 3D location of the landmarks via stereo vision. Feature correspondences are estimated using a variant of the PROSAC algorithm. Our mapping technique constructs a graph of spatial constraints and applies an efficient gradient descent-based optimization approach to estimate the most likely map of the environment. Our approach has been evaluated in comparably large outdoor and indoor environments. We furthermore present experiments in which our technique is applied to build a map with a blimp.