Semantic segmentation of images is an important issue for intelligent vehicles and mobile robotics because it offers basic information which can be used for complex reasoning and safe navigation. Different solutions have been proposed for this problem along the last two decades, where recent deep neural networks approaches have shown very promising results in the context of urban navigation. One of the main problems when comparing different semantic segmentation solutions is how to select an appropriate metric to evaluate their accuracy. On the one hand, classic metrics do not measure properly the accuracy on the object contours, which is important in urban driving to differentiate road from sidewalk for instance. On the other hand, contour-based metrics [1] disregard the information far from class contours. This paper explores the problem multi-modal image segmentation, and presents a new metric to leverage global and contour accuracy in a simple formulation. This metric is validated with the evaluation of several semantic segmentation solutions that exploit RGB-D images to rank these solutions taking into account the quality of the segmented contours. We also present a comparative analysis of several commonly used metrics together with a statistical analysis of their correlation.

Image and point cloud registration methods compute the relative pose between two images. Commonly used registration algorithms are iterative and rely on the assumption that the motion between the images is small. In this work, we propose a fast pose estimation technique to compute a rough estimate of large motions between depth images, which can be used as initialization to dense registration methods. The main idea is to explore the properties given by planar surfaces with co-visibility and their normals from two distinct viewpoints. We present, in two decoupled stages, the rotation and then the translation estimation, both based on the normal vectors orientation and on the depth. These two stages are efficiently computed by using low resolution depth images and without any feature extraction/matching. We also analyze the limitations and observabilty of this approach, and its relationship to ICP point-to-plane. Notably, if the rotation is observable, at least five degrees of freedom can be estimated in the worst case. To demonstrate the effectiveness of the method, we evaluate the initialization technique in a set of challenging scenarios, comprising simulated spherical images from the Sponza Atrium model benchmark and real spherical indoor sequences.

The integration of several 2D laser rangefinders in a vehicle is a common resource employed for 3D mapping, obstacle detection and navigation. The extrinsic calibration between such sensors (i.e. finding their relative poses) is required to exploit effectively the sensor measurements and to perform data fusion. The approaches found in the literature to obtain such calibration either rely on the approximated parameters from the rig construction or propose ad-hoc solutions for specific LRF rigs. In this paper we present a novel solution for the extrinsic calibration of a set of at least three laser scanners from the information provided by the sensor measurements. This method only requires the lasers to observe a common planar surface from different orientations, thus there is no need of any specific calibration pattern. This calibration technique can be used with almost any geometric sensor configuration (except for sensors scanning parallel planes), and constitutes a versatile solution that is accurate, fast and easy to apply. This approach is validated with both simulated and real data.

Robots are often equipped with 2D laser-rangefinders (LRFs) and cameras since they complement well to each other. In order to correctly combine measurements from both sensors, it is required to know their relative pose, that is, to solve their extrinsic calibration. In this paper we present a new approach to such problem which relies on the observations of orthogonal trihedrons which are profusely found as corners in human-made scenarios. Thus, the method does not require any specific pattern, which turns the calibration process fast and simpler to perform. The estimated relative pose has proven to be also very precise since it uses two different types of constraints, line-to-plane and point-to-plane, as a result of a richer configuration than previous proposals that relies on plane or V-shaped patterns. Our approach is validated with synthetic and real experiments, showing better performance than the state-of-art methods.

This paper presents a methodology to combine information from a sequence of RGB-D spherical views acquired by a home-made multi-stereo device in order to improve the computed depth images both in terms of accuracy and completeness. This methodology is embedded in a larger visual mapping framework aiming to produce accurate and dense topometric urban maps. Our method is based on two main filtering stages. Firstly, we perform a segmentation process considering both geometric and photometric image constraints, followed by a regularization step (spatial-integration). We then proceed to a fusion stage where the geometric information is further refined by considering the depth images of nearby frames (temporal integration). This methodology can be applied to other projective models, such as perspective stereo images. Our approach is evaluated within the frameworks of image registration, localization and mapping, demonstrating higher accuracy and larger convergence domains over different datasets.

The representation of the world upon planar patches has proven to be simple, robust and useful for a variety of robotic tasks, including SLAM, autonomous navigation, or scene recognition. In this work we investigate how to incorporate color information into such representation to improve the matching of planar patches while maintaining the model compactness, which is essential for real-time applications. We propose a descriptor based on the dominant color of the patch, which is defined as the center of the biggest cluster in the patch histogram. In the paper, different color spaces and methods for extracting the dominant color are analyzed. We compare this descriptor with a recent proposal (saturated hue based histogram) and provide some conclusions on the trade-off between their descriptiveness and compactness. Finally, we present experimental results showing how our color descriptor can be exploited to increase the efficiency of both: plane-based place recognition and planar patch categorization.

The integration of several range cameras in a mobile platform is useful for applications in mobile robotics and autonomous vehicles that require a large field of view. This situation is increasingly interesting with the advent of low cost range cameras like those developed by Primesense. Calibrating such combination of sensors for any geometric configuration is a problem that has been recently solved through visual odometry (VO) and SLAM. However, this kind of solution is laborious to apply, requiring robust SLAM or VO in controlled environments. In this paper we propose a new uncomplicated technique for extrinsic calibration of range cameras that relies on finding and matching planes. The method that we present serves to calibrate two or more range cameras in an arbitrary configuration, requiring only to observe one plane from different viewpoints. The conditions to solve the problem are studied, and several practical examples are presented covering different geometric configurations, including an omnidirectional RGB-D sensor composed of 8 range cameras. The quality of this calibration is evaluated with several experiments that demonstrate an improvement of accuracy over design parameters, while providing a versatile solution that is extremely fast and easy to apply.

In the last very few years, monocular SLAM approaches based on bundle adjustment are achieving amazing results in terms of accuracy, computational efficiency, and density of the map. When such solutions are applied on large scenarios it is crucial for the system scalability to maintain a map representation that permits efficient map optimization and augmentation. In order to cope with such large maps, we present an on-the-fly partitioning technique which allows abstraction from the metric map to operate more efficiently. The result is a metric-topological arrangement where the areas with highly-connected observations are grouped in submaps weakly interconnected to each other. This is accomplished by progressively cutting a graph representation of the map, where the nodes are keyframes and the arcs between them represent their shared observations. The experimental results indicate that the proposed approach improves the efficiency of monocular SLAM and provides a metric-topological world representation suitable for other robotic tasks.

This paper presents a new method for recognizing places in indoor environments based on the extraction of planar regions from range data provided by a hand-held RGB-D sensor. We propose to build a plane-based map (PbMap) consisting of a set of 3D planar patches described by simple geometric features (normal vector, centroid, area, etc.). This world representation is organized as a graph where the nodes represent the planar patches and the edges connect planes that are close by. This map structure permits to efficiently select subgraphs representing the local neighborhood of observed planes, that will be compared with other subgraphs corresponding to local neighborhoods of planes acquired previously. To find a candidate match between two subgraphs we employ an interpretation tree that permits working with partially observed and missing planes. The candidates from the interpretation tree are further checked out by a rigid registration test, which also gives us the relative pose between the matched places. The experimental results indicate that the proposed approach is an efficient way to solve this problem, working satisfactorily even when there are substantial changes in the scene (lifelong maps).