One of the most popular approaches to multi-target tracking is tracking-by-detection. Current min-cost flow algorithms which solve the data association problem optimally have three main drawbacks: they are computationally expensive, they assume that the whole video is given as a batch, and they scale badly in memory and computation with the length of the video sequence. In this paper, we address each of these issues, resulting in a computationally and memory-bounded solution. First, we introduce a dynamic version of the successive shortest-path algorithm which solves the data association problem optimally while reusing computation, resulting in faster inference than standard solvers. Second, we address the optimal solution to the data association problem when dealing with an incoming stream of data (i.e., online setting). Finally, we present our main contribution which is an approximate online solution with bounded memory and computation which is capable of handling videos of arbitrary length while performing tracking in real time. We demonstrate the effectiveness of our algorithms on the KITTI and PETS2009 benchmarks and show state-of-the-art performance, while being significantly faster than existing solvers.

Today, visual recognition systems are still rarely employed in robotics applications. Perhaps one of the main reasons for this is the lack of demanding benchmarks that mimic such scenarios. In this paper, we take advantage of our autonomous driving platform to develop novel challenging benchmarks for the tasks of stereo, optical flow, visual odometry/SLAM and 3D object detection. Our recording platform is equipped with four high resolution video cameras, a Velodyne laser scanner and a state-of-the-art localization system. Our benchmarks comprise 389 stereo and optical flow image pairs, stereo visual odometry sequences of 39.2 km length, and more than 200k 3D object annotations captured in cluttered scenarios (up to 15 cars and 30 pedestrians are visible per image). Results from state-of-the-art algorithms reveal that methods ranking high on established datasets such as Middlebury perform below average when being moved outside the laboratory to the real world. Our goal is to reduce this bias by providing challenging benchmarks with novel difficulties to the computer vision community. Our benchmarks are available online at: www.cvlibs.net/datasets/kitti.

Modern driver assistance systems such as collision avoidance or intersection assistance need reliable information on the current environment. Extracting such information from camera-based systems is a complex and challenging task for inner city traffic scenarios. This paper presents an approach for object detection utilizing sparse scene flow. For consecutive stereo images taken from a moving vehicle, corresponding interest points are extracted. Thus, for every interest point, disparity and optical flow values are known and consequently, scene flow can be calculated. Adjacent interest points describing a similar scene flow are considered to belong to one rigid object. The proposed method does not rely on object classes and allows for a robust detection of dynamic objects in traffic scenes. Leading vehicles are continuously detected for several frames. Oncoming objects are detected within five frames after their appearance.

Environment perception and scene understanding is an important issue for modern driver assistance systems. However, adverse weather situations and disadvantageous illumination conditions like cast shadows have a negative effect on the proper operation of these systems. In this paper, we propose a novel approach for cast shadow recognition in monoscopic color images. In a first step, shadow edge candidates are extracted evaluating binarized channels in the color-opponent and perceptually uniform CIE L*a*b* space. False detections are rejected in a second verification step, using SVM classification and a combination of meaningful color features. We introduce a non-parametric representation for complex shadow edge geometries that enables utilizing shadow edge information for improving downstream vision-based driver assistance systems. A quantitative evaluation of the classification performance as well as results on multiple real-world traffic scenes show a reliable cast shadow recognition with only a few false detections.

In this paper we propose a novel framework for road reflectivity classification in cluttered traffic scenarios by measuring the bidirectional reflectance distribution function of road surfaces from inside a moving vehicle. The predominant restrictions in our application are a strongly limited field of observations and a weakly defined illumination environment. To overcome these problems, we estimate the parameters of an extended Oren-Nayar model that considers the diffuse and specular behavior of real-world surfaces and extrapolate the surface reflectivity measurements to unobservable angle combinations. Model ambiguities are decreased by utilizing standardized as well as customized reflection characteristics. In contrast to existing approaches that require special measurement setups, our approach can be implemented in vision-based driver assistance systems using radiometrically uncalibrated gray value cameras and GPS information. The effectiveness of our approach is demonstrated by a successful classification of the road surface reflectance of expressway scenes with low error rates.