We have recently proposed a speech-based on- demand intersection assistant which helps the driver to handle urban intersections by informing him of the traffic situation on the right hand side and recommending suitable gaps in traffic. In a previous user study, conducted in a simulator, we could show that the system is in general well accepted and preferred by drivers compared to driving without assistance or with only visual support. In this paper, we report on an implementation of this system and its evaluation in real urban traffic. We use LIDAR sensors for the perception of the traffic environment. A scene analyzer estimates the gaps between the vehicles in real time. The result of this analysis is provided to a dialog manager, which uses it to inform the driver of approaching vehicles and suitable gaps. While approaching the intersection, the driver can activate the system via a wake-up-word and control it with subsequent speech commands. The design of the data analyzer and dialog manager is based on evaluations at real intersections. The resulting system can provide suitable support to the driver in a wide range of traffic situations.

Advanced Driver Assistance Systems (ADAS) are becoming more and more popular. Many of these systems though are limited to specific scenes and often detect risky situations very late so they can only mitigate accidents. These effects are mainly caused by the use of simple physical prediction methods, e.g. to estimate the time-to-contact with another vehicle. In this paper we show an ADAS that extends the functionality of physical collision warnings by additionally estimating potential risks based on implicit predictions. As an example we demonstrate the use of vehicle orientation information for classifying situations. Through this, we can in particular assess the risk of static cars, for which physical prediction does not apply, but which can nevertheless easily cause an accident if they start moving into our driving corridor. The proposed system is evaluated online in a test car and is shown to reliably detect classical risky situations as well as those involving static cars.

Man-made real-world environments are dominated by planar surfaces many of which constitute behavior-relevant entities. Thus, the ability to perceive planar surfaces is vital for any embodied system operating in such environments, be it human or robotic. In this paper, we present an architecture for detection and estimation of planar surfaces in the scene from calibrated stereo images. They are represented in a behavior-oriented way, focusing on geometrical properties that are relevant for enabling basic interaction between a robot and the planar surfaces it perceives. Ego-motion of the robot is compensated for by transforming the representations into a global coordinate system using the kinematics of the robot. Our architecture is able to detect and estimate arbitrary planar surfaces, regardless of their visual appearance, their geometrical properties other than planarity and their being static or arbitrarily moving. The latter is achieved by processing each frame independently of the others. Stable representations are obtained by establishing spatio-temporal coherence between the single-frame representations of subsequent frames. Based on a RANSAC approach to plane fitting, our method is robust to unreliable 3D data such as obtained by local stereo correlation, for example. In our experiments using the Honda humanoid robot ASIMO, we show that our method is able to provide a robot in real-time with representations of planar surfaces in its environment that are sufficiently accurate for basic interaction.

Based on inspirations from infant development we present a system which learns associations between acoustic labels and visual representations in interaction with its tutor. The system is integrated with a humanoid robot. Except for a few trigger phrases to start learning all acoustical representations are learned online and in interaction. Similar, for the visual domain the clusters are not predefined and fully learned online. In contrast to other interactive systems the interaction with the acoustic environment is solely based on the two microphones mounted on the robots head. In this paper we give an overview on all key elements of the system and focus on the challenges arising from the headset-free learning of speech labels. In particular we present a mechanism for auditory attention integrating bottom-up and top-down information for the segmentation of the acoustic stream. The performance of the system is evaluated based on offline tests of individual parts of the system and an analysis of the online behavior.

We describe a system for visual interaction developed for humanoid robots. It enables the robot to interact with its environment using a smooth whole body motion control driven by stabilized visual targets. Targets are defined as visually extracted "proto-objects" and behavior-relevant object hypotheses and are stabilized by means of a short-term sensory memory. Selection mechanisms are used to switch between behavior alternatives for searching or tracking objects as well as different whole body motion strategies for reaching. The decision between different motion strategies like reaching with right or left hand or with and without walking is made based on internal predictions that use copies of the whole-body control algorithm. The results show robust object tracking and a smooth interaction behavior that includes a large variety of whole-body postures.