When developing autonomous or automated driving functions, having a sound understanding of the environment is critical. The earlier and the more detailed the environment is recognized, the more precise the vehicle can plan future actions. One major component in the field of autonomous driving, therefore, is visual perception. Cameras deliver consecutive images of the world, and computer vision algorithms attempt to extract the relevant information, for example, detecting other road users such as vehicles. However, current automated driving functions share the limitation of relying on the object to be directly visible to be detected. On the other side, humans intuitively use other visual effects to draw assumptions about objects that are not yet directly visible. Such effects can be shadows during the day or light reflections during the night. Recent work has already shown the potential of using this information when driving at night on highways. Highways at night thereby are easy ground: The traffic volume is considerably low, and the surrounding environment is mainly dark (e. g., there are no street lamps causing other light sources or reflections). This paper takes the research on this topic a step further and brings light to the question to which extent this effect of detecting oncoming vehicles at night can also be used in cities. We present a comprehensive analysis of human behavior when detecting other road users at night in urban areas while driving. The data was gathered throughout a test group study in two medium-sized German cities under various weather and traffic conditions. We prove the importance of light reflections when driving through urban areas at night and provide a solid human benchmark to compare future perception algorithms’ performance.

In current object detection, algorithms require the object to be directly visible in order to be detected. As humans, however, we intuitively use visual cues caused by the respective object to already make assumptions about its appearance. In the context of driving, such cues can be shadows during the day and often light reflections at night. In this paper, we study the problem of how to map this intuitive human behavior to computer vision algorithms to detect oncoming vehicles at night just from the light reflections they cause by their headlights. For that, we present an extensive open-source dataset containing 59 746 annotated grayscale images out of 346 different scenes in a rural environment at night. In these images, all oncoming vehicles, their corresponding light objects (e. g., headlamps), and their respective light reflections (e. g., light reflections on guardrails) are labeled. In this context, we discuss the characteristics of the dataset and the challenges in objectively describing visual cues such as light reflections. We provide different metrics for different ways to approach the task and report the results we achieved using state-of-the-art and custom object detection models as a first benchmark. With that, we want to bring attention to a new and so far neglected field in computer vision research, encourage more researchers to tackle the problem, and thereby further close the gap between human performance and computer vision systems.

Abstract Neural networks are state-of-the-art classification approaches but are generally difficult to interpret. This issue can be partly alleviated by constructing a precise decision process within the neural network. In this work, a network architecture, denoted as Classification-By-Components network (CBC), is proposed. It is restricted to follow an intuitive reasoning based decision process inspired by Biederman's recognition-by-components theory from cognitive psychology. The network is trained to learn and detect generic components that characterize objects. In parallel, a class-wise reasoning strategy based on these components is learned to solve the classification problem. In contrast to other work on reasoning, we propose three different types of reasoning: positive, negative, and indefinite. These three types together form a probability space to provide a probabilistic classifier. The decomposition of objects into generic components combined with the probabilistic reasoning provides by design a clear interpretation of the classification decision process. The evaluation of the approach on MNIST shows that CBCs are viable classifiers. Additionally, we demonstrate that the inherent interpretability offers a profound understanding of the classification behavior such that we can explain the success of an adversarial attack. The method's scalability is successfully tested using the ImageNet dataset.