Autonomous driving is a topic in computer vision which has captured a great deal of attention in recent years. One key problem is the detection and state analysis of traffic lights. Even over time, very few datasets for research in this topic have been published and they vary widely in quantity and in quality. To address the complexity of traffic light recognition, we introduce the DriveU**driveU is a joint innovation center of the Daimler AG and the University of Ulm Traffic Light Dataset (DTLD), a large-scale dataset consisting of more than 230,000 annotations. All annotations are hand-labeled according to strict rules and show a high quality. Recordings were made in eleven different cities during different weather conditions. Our dataset exceeds previous traffic light datasets in size, variance, annotation quality and amount of additional sensor data. We prove the extent of our dataset by an extensive comparison with existing traffic light datasets. Miscellaneous dataset criteria are compared, illustrated and statistically analyzed. In the process, metrics to express the quality and variance of datasets are developed and verified. The dataset can be downloaded from http://traffic-light-data.de.

Sliding window approaches have been widely used for object recognition tasks in recent years [19], [4], [5], [18]. They guarantee an investigation of the entire input image for the object to be detected and allow a localization of that object. Despite the current trend towards deep neural networks, sliding window methods are still used in combination with convolutional neural networks [22]. The risk of overlooking an object is clearly reduced compared to alternative detection approaches which detect objects based on shape, edges or color. Nevertheless, the sliding window technique strongly increases the computational effort as the classifier has to verify a large number of object candidates. This paper proposes a sliding window approach which also uses depth information from a stereo camera. This leads to a greatly decreased number of object candidates without significantly reducing the detection accuracy. A theoretical investigation of the conventional sliding window approach is presented first. Other publications to date only mentioned rough estimations of the computational cost. A mathematical derivation clarifies the number of object candidates with respect to parameters such as image and object size. Subsequently, the proposed disparity sliding window approach is presented in detail. The approach is evaluated on pedestrian detection with annotations and images from the KITTI [10] object detection benchmark. Furthermore, a comparison with two state-of-the-art methods is made. Code is available in C++ and Python https://github.com/julimueller/disparity-sliding-window.

This paper introduces three methods to improve traffic light recognition by using the stereo camera color image in tandem with the disparity image. The first method is object candidate filtering by analyzing the disparity values inside an object candidate. The second method applies the relative positioning filter. Using the depth measurement obtained from the disparity image as well as the intrinsic and extrinsic calibration, a three dimensional distance from an object to the vehicle can be calculated. Based on known real world traffic light locations, the filter is able to suppress thirty to seventy percent of false positives while only decreasing the detection rate by one percent. This result shows the huge potential for range filtering in traffic light recognition in general. The third method is the hypothesis size enhancement when re-projecting a hypothesis into the image. This process is enabled by a real world traffic light model in conjunction with the depth measurement. It is shown that all true positive hypotheses will benefit from this technique, resulting in a massively better overlap with the ground truth labels. When evaluated frame wise, re-projection can improve detection rate by up to fifteen percent. This work primarily enhances traffic light hypotheses obtained by a baseline detector and thus requires a hypothesis disparity value. As a further contribution this paper presents different methods for determining a hypothesis-wide disparity and evaluates the differences in quality and quantity.