Cameras and 2D laser scanners, in combination, are able to provide low-cost, light-weight and accurate solutions, which make their fusion well-suited for many robot navigation tasks. However, correct data fusion depends on precise calibration of the rigid body transform between the sensors. In this paper we present the first framework that makes use of Convolutional Neural Networks (CNNs) for odometry estimation fusing 2D laser scanners and mono-cameras. The use of CNNs provides the tools to not only extract the features from the two sensors, but also to fuse and match them without needing a calibration between the sensors. We transform the odometry estimation into an ordinal classification problem in order to find accurate rotation and translation values between consecutive frames. Results on a real road dataset show that the fusion network runs in real-time and is able to improve the odometry estimation of a single sensor alone by learning how to fuse two different types of data information.

The use of 2D laser scanners is attractive for the autonomous driving industry because of its accuracy, light-weight and low-cost. However, since only a 2D slice of the surrounding environment is detected at each scan, it is a challenge to execute important tasks such as the localization of the vehicle. In this paper we present a novel framework that explores the use of deep Recurrent Convolutional Neural Networks (RCNN) for odometry estimation using only 2D laser scanners. The application of RCNNs provides the tools to not only extract the features of the laser scanner data using Convolutional Neural Networks (CNNs), but in addition it models the possible connections among consecutive scans using the Long Short-Term Memory (LSTM) Recurrent Neural Network. Results on a real road dataset show that the method can run in real-time without using GPU acceleration and have competitive performance compared to other methods, being an interesting approach that could complement traditional localization systems.

Automation driving techniques have seen tremendous progresses these last years, particularly due to a better perception of the environment. In order to provide safe yet not too conservative driving in complex urban environment, data fusion should not only consider redundant sensing to characterize the surrounding obstacles, but also be able to describe the uncertainties and errors beyond presence/absence (be it binary or probabilistic). This paper introduces an enriched representation of the world, more precisely of the potential existence of obstacles through an evidential grid map. A method to create this representation from 2 very different sensors, laser scanner and stereo camera, is presented along with algorithms for data fusion and temporal updates. This work allows a better handling of the dynamic aspects of the urban environment and a proper management of errors in order to create a more reliable map. We use the evidential framework based on the Dempster-Shafer theory to model the environment perception by the sensors. A new combination operator is proposed to merge the different sensor grids considering their distinct uncertainties. In addition, we introduce a new long-life layer with high level states that allows the maintenance of a global map of the entire vehicle's trajectory and distinguish between static and dynamic obstacles. Results on a real road dataset show that the environment mapping data can be improved by adding relevant information that could be missed without the proposed approach.

The self-localization of a vehicle is still an ongoing and challenging task for the autonomous driving development. At the same time, the correct understanding of the vehicle's surroundings and the creation of a map with the traversed trajectory is essential for driving in complex urban scenarios. This paper proposes a solution to create an enriched global map of the environment while localizing the vehicle within it. We use the Evidential framework based on the Dempster-Shafer theory to create a map able to distinguish between static and dynamic obstacles and to keep the information from the entire traversed path. Additionally, we propose a method that at each current map creation estimates the vehicle's position by a grid matching algorithm based on image registration techniques. The method transforms the grid maps into images using the evidential data to extract relevant information to perform a better localization. The proposed method was tested on the KITTI dataset and shows that the solution is able to reduce the drift and improve the localization compared to other methods using similar configurations.

Image road detection in unstructured environments is a crucial and challenging problem in the application of mobile robots and autonomous vehicles. In this paper, we present an effective and computationally efficient solution to segment the road region for structured and unstructured roads. We propose a new method that incorporates two different approaches: road detection based on the vanishing point and image segmentation using a seeded region growing (SRG) algorithm. First, a fast vanishing point detection algorithm is applied and used to find an estimation of the road boundaries. Subsequently, we segment the road area region executing a SRG algorithm based on the vanishing point and the road boundaries found previously. Evaluation of our method over different images datasets demonstrates that it is effective in challenging conditions such as dirt and curved roads.

Grasping objects in confined environments, such as shelves, fridges, or drawers, is challenging due to the difficulty of avoiding gripper and arm collisions with the surfaces surrounding the object. In this paper we explore the use of a tool to reconfigure objects in such environments so as to make them graspable. The proposed tool has a simple form that allows it to be used in confined environments and a high friction tool tip that enables not only pushing objects but also pulling them. Our approach involves learning predictive models of pre-defined object-directed tool actions from experience. For each action, we train a multi-modal regressor that maps the initial state of an object to changes in that state, such that future states of the object can be estimated. These allow the robot to choose a sequence of tool actions that yield graspable configurations. We demonstrate that our approach enables a PR2 robot to grasp five different objects from different, initially ungraspable, configurations on a shelf.