Long-term situation prediction plays a crucial role for intelligent vehicles. A major challenge still to overcome is the prediction of complex downtown scenarios with multiple road users, e.g., pedestrians, bikes, and motor vehicles, interacting with each other. This contribution tackles this challenge by combining a Bayesian filtering technique for environment representation, and machine learning as long-term predictor. More specifically, a dynamic occupancy grid map is utilized as input to a deep convolutional neural network. This yields the advantage of using spatially distributed velocity estimates from a single time step for prediction, rather than a raw data sequence, alleviating common problems dealing with input time series of multiple sensors. Furthermore, convolutional neural networks have the inherent characteristic of using context information, enabling the implicit modeling of road user interaction. Pixel-wise balancing is applied in the loss function counteracting the extreme imbalance between static and dynamic cells. One of the major advantages is the unsupervised learning character due to fully automatic label generation. The presented algorithm is trained and evaluated on multiple hours of recorded sensor data and compared to Monte-Carlo simulation. Experiments show the ability to model complex interactions.

We tackle the problem of object detection and pose estimation in a shared space downtown environment. For perception multiple laser scanners with $360^{o}$ coverage were fused in a dynamic occupancy grid map (DOGMa). A single-stage deep convolutional neural network is trained to provide object hypotheses comprising of shape, position, orientation and an existence score from a single input DOGMa. Furthermore, an algorithm for offline object extraction was developed to automatically label several hours of training data. The algorithm is based on a two-pass trajectory extraction, forward and backward in time. Typical for engineered algorithms, the automatic label generation suffers from misdetections, which makes hard negative mining impractical. Therefore, we propose a loss function counteracting the high imbalance between mostly static background and extremely rare dynamic grid cells. Experiments indicate, that the trained network has good generalization capabilities since it detects objects occasionally lost by the label algorithm. Evaluation reaches an average precision (AP) of 75.9%.

A major challenge in autonomous driving is the prediction of complex downtown scenarios with mutiple road users. This contribution tackles this challenge by combining a Bayesian filtering technique for environment representation and machine learning as long-term predictor. Therefore, a dynamic occupancy grid map representing the static and dynamic environment around the ego-vehicle is utilized as input to a deep convolutional neural network. This yields the advantage of using data from a single timestamp for prediction, rather than an entire time series. Furthermore, convolutional neural networks have the inherent characteristic of using context information, enabling the implicit modeling of road user interaction. One of the major advantages is the unsupervised learning character due to fully automatic label generation. The presented algorithm is trained and evaluated on multiple hours of recorded sensor data containing multiple road users, e.g., pedestrians, bikes and vehicles.

The correct handling of complex traffic-light-controlled intersections is still a challenge for automated vehicles. While a number of image-based approaches tackle close-range recognitions, an early traffic light detection at high distances is of great importance in the area of energy-efficient driving. For this reason, a traffic light detection system consisting of multiple on-board cameras is presented in this work, enabling the detection of traffic lights even from a distance of more than 200m. Furthermore, the presented system is based on tracking techniques using a Labeled Multi-Bernoulli filter in combination with the fusion of classifications based on the Dempster-Shafer theory of evidence. The system was tested on a real world data set collected in Germany and an increase in performance was demonstrated by a multi-camera approach.

The project “Autonomous Driving” at Ulm University aims at advancing highly-automated driving with close-to-market sensors while ensuring easy exchangeability of the particular components. In this contribution, the experimental vehicle that was realized during the project is presented along with its software modules. To achieve the mentioned goals, a sophisticated fusion approach for robust environment perception is essential. Apart from the necessary motion planning algorithms, this paper thus focuses on the sensor-independent fusion scheme. It allows for an efficient sensor replacement and realizes redundancy by using probabilistic and generic interfaces. Redundancy is ensured by utilizing multiple sensors of different types in crucial modules like grid mapping, localization and tracking. Furthermore, the combination of the module outputs to a consistent environment model is achieved by employing their probabilistic representation. The performance of the vehicle is discussed using the experience from numerous autonomous driving tests on public roads.