One of the most challenging tasks in the development of path planners for intelligent vehicles is the design of the cost function that models the desired behavior of the vehicle. While this task has been traditionally accomplished by hand-tuning the model parameters, recent approaches propose to learn the model automatically from demonstrated driving data using Inverse Reinforcement Learning (IRL). To determine if the model has correctly captured the demonstrated behavior, most IRL methods require obtaining a policy by solving the forward control problem repetitively. Calculating the full policy is a costly task in continuous or large domains and thus often approximated by finding a single trajectory using traditional path-planning techniques. In this work, we propose to find such a trajectory using a conformal spatiotemporal state lattice, which offers two main advantages. First, by conforming the lattice to the environment, the search is focused only on feasible motions for the robot, saving computational power. And second, by considering time as part of the state, the trajectory is optimized with respect to the motion of the dynamic obstacles in the scene. As a consequence, the resulting trajectory can be used for the model assessment. We show how the proposed IRL framework can successfully handle highly dynamic environments by modeling the highway tactical driving task from demonstrated driving data gathered with an instrumented vehicle.

In an autonomous vehicle setting, we propose a method for the estimation of a semantic grid, i.e. a bird's eye grid centered on the car's position and aligned with its driving direction, which contains high-level semantic information about the environment and its actors. Each grid cell contains a semantic label with divers classes, as for instance {Road, Vegetation, Building, Pedestrian, Car...}. We propose a hybrid approach, which combines the advantages of two different methodologies: we use Deep Learning to perform semantic segmentation on monocular RGB images with supervised learning from labeled groundtruth data. We combine these segmentations with occupancy grids calculated from LIDAR data using a generative Bayesian particle filter. The fusion itself is carried out with a deep neural network, which learns to integrate geometric information from the LIDAR with semantic information from the RGB data. We tested our method on two datasets, namely the KITTI dataset, which is publicly available and widely used, and our own dataset obtained with our own platform, equipped with a LIDAR and various sensors. We largely outperform baselines which calculate the semantic grid either from the RGB image alone or from LIDAR output alone, showing the interest of this hybrid approach.

Robust perception is the cornerstone of safe and environmentally-aware autonomous navigation systems. Autonomous robots are expected to recognise the objects in their surroundings under a wide range of challenging environmental conditions. This problem has been tackled by combining multiple sensor modalities that have complementary characteristics. This paper proposes an approach to multi-sensor-based robotic perception that leverages the rich and dense appearance information provided by camera sensors, and the range data provided by active sensors independently of how dense their measurements are. We introduce a framework we call XDvision where colour images are augmented with dense depth information obtained from sparser sensors such as lidars. We demonstrate the utility of our framework by comparing the performance of a standard CNN-based image classifier fed with image data only with the performance of a two-layer multimodal CNN trained using our augmented representation.

This paper presents a motion detection approach based on a combination of dense optical flow and 3D stereo reconstruction. Our motion detection is not based on predefined templates, providing a generic framework suitable for a broad range of applications such as situation awareness. The approach estimates the likelihood of pixels motion from the fusion of dense optical flow and dense depth information estimated from a stereo camera. Temporal consistency is incorporated by tracking moving objects across consecutive images. The proposed algorithm is validated with publicly available datasets. The consistent results across different scenarios demonstrate the robustness of our framework, presenting an average detection rate of 92%.

Learning motion patterns in dynamic environments is a key component of any context-aware robotic system, and probabilistic mixture models provide a sound framework for mining these patterns. This paper presents an approach for learning motion models from trajectories provided by the tracking system of a moving platform. We present a learning approach in which a Linear Dynamical System (LDS) is augmented with a discrete hidden variable that has a number of states equal to the number of behaviours in the environment. As a result, a mixture of linear dynamical systems (MLDSs) capable of explaining several motion behaviours is developed. The model is learned by means of the Expectation Maximization (EM) algorithm.