Social interaction is the new frontier in contemporary robotics: we want to build robots that blend with ease into our daily social environments, following their norms and rules. The cognitive skill that bootstraps social awareness in humans is known as "intention reading" and it allows us to interpret other agents' actions and assign them meaning. Given its centrality for humans, it is likely that intention reading will foster the development of robotic social understanding. In this paper, we present an artificial cognitive architecture for intention reading in human-robot interaction (HRI) that makes use of social cues to disambiguate goals. This is accomplished by performing a low-level action encoding paired with a high-level probabilistic goal inference. We introduce a new clustering algorithm that has been developed to differentiate multi-sensory human social cues by performing several levels of clustering on different feature-spaces, paired with a Bayesian network that infers the underlying intention. The model has been validated through an interactive HRI experiment involving a joint manipulation game performed by a human and a robotic arm in a toy block scenario. The results show that the artificial agent was capable of reading the intention of its partner and cooperate in mutual interaction, thus validating the novel methodology and the use of social cues to disambiguate goals, other than demonstrating the advantages of intention reading in social HRI.

One of the key challenges in realizing a robot that is capable of completing a variety of manipulation tasks in the real world is the need to utilize sufficiently compact and rich world models. If the assumed prediction model does not match real observations, planning systems are unable to perform properly. We propose a system that corrects the models based on information collected from the robot's sensors. We encode prior experiences in a neural network to generate possible parameters of the models for a physics engine from real observations. An online POMDP solver is used to plan actions to complete the task while progressively validating and improving the models. We perform experiments in simulations and on a real robot. The results show that this approach appropriately clarifies observed environments, can handle dynamics with discontinuities, and with increasing domain complexity achieves a better success rate than baseline methods.

The detection of road area in the surroundings of the ego-vehicle is a key issue for modern ADAS. Camera-based direct detection systems are able to reliably accomplish this task only within a limited spatial range or in simple environments, due to hardware limitations and unfavorable situations, like shadows or occlusions. In complex environments, like inner city, traffic is a real issue, since the mere presence of other cars can significantly restrict the field of view of the ego-vehicle. In order to extend the spatial range of road detection, indirect detection systems are a viable resource. They can complement state-of-the-art direct detection systems and help motion control systems to plan smooth and stable trajectories. In this paper we propose a probabilistic grid-based approach based on the interpretation of the motion of other vehicles in the scene. The approach uses the position and velocity of those vehicles in order to infer the presence and location of occluded road area. We will show that this approach can complement an already established feature-based detection system, taking advantage of those situations that are the most challenging for the latter. Evaluations on real-world scenes show that the union between this approach and direct road detection significantly extends the spatial range of detection, thus is able to provide a motion control system a longer horizon for planning trajectories.

Detecting the road terrain ahead of the ego-vehicle is an important issue for modern driver assistance systems. In particular, vehicle motion planning in inner city environment requires the detection of road terrain up to 3 seconds in advance. State-of-the-art visual road terrain detection systems have a hard time fulfilling this task, due to their limited range and the presence of occlusions (other vehicles, buildings, etc.), which are expected to occur often in complex scenarios. However, those systems provide significant information where the conditions are favorable (proximity to the ego-vehicle, no occlusions). Therefore, a complementary approach is needed to enhance already existing and established detection systems. In this paper we propose a probabilistic grid-based approach based on the observation and interpretation of other vehicles' behavior in the scene. It exploits their movements in order to infer the presence and location of occluded road surface. We will show that this approach presents various advantages over current visual road terrain detection systems, especially in those situations that are the most challenging for them. We will illustrate how our approach is designed to work in concert also with other available resources, e.g. offline road maps. Qualitative results on real-world scenes taken from the KITTI benchmark[11] demonstrate that the fusion of this method with visual road terrain detection can potentially extend our time horizon well over the 3 seconds mentioned above. Finally, we will show how our approach is planned to develop into a semantically enriched representation of the road, including road properties such as availability, lanes and directions.

Advanced Driver Assistance Systems (ADAS) are becoming more and more popular. Many of these systems though are limited to specific scenes and often detect risky situations very late so they can only mitigate accidents. These effects are mainly caused by the use of simple physical prediction methods, e.g. to estimate the time-to-contact with another vehicle. In this paper we show an ADAS that extends the functionality of physical collision warnings by additionally estimating potential risks based on implicit predictions. As an example we demonstrate the use of vehicle orientation information for classifying situations. Through this, we can in particular assess the risk of static cars, for which physical prediction does not apply, but which can nevertheless easily cause an accident if they start moving into our driving corridor. The proposed system is evaluated online in a test car and is shown to reliably detect classical risky situations as well as those involving static cars.

This paper presents a visual-servoing method for compensating motion of soft tissue structures using 4D ultra-sound. The motion of soft tissue structures caused by physiological and external motion makes it difficult to investigate them for diagnostic and therapeutic purposes. The main goal is to track non-rigidly moving soft tissue structures and compensate the motion in order to keep a lesion on its target position during a treatment. We define a 3D non-rigid motion model by extending the Thin-Plate Spline (TPS) algorithm. The motion parameters are estimated with intensity-value changes of a points set in a tracking soft tissue structure. Finally, the global rigid motion is compensated with a 6-DOF robot according to the motion parameters of the tracking structure. Simulation experiments are performed with recorded 3D US images of in-vivo soft tissue structures and validate the effectiveness of the non-rigid motion tracking method. Robotic experiments demonstrated the success of our method with a deformable phantom.

The detection of objects in the surrounding environment is a key functionality for every Advanced Driver Assistance System (ADAS). Otherwise, it is impossible to realize any kind of advanced assistance functionality as e.g. a Forward Collision Warning. Besides static obstacles that limit the drivable area, dynamic objects can even be more dangerous due to their ego-movement. Therefore, a detection and also motion estimation of moving objects in the surrounding is essential for every ADAS. In this contribution we present a novel way to detect moving objects only with 3D-measurements of the current surrounding environment that we called 3D-Warping. But we are not only able to detect but also to provide the direction and speed of moving objects in world coordinates. The method is generic and can be applied on any kind of depth sensor, as e.g. a stereo-camera, time-of-flight, and 3D-laser scanner.

We investigate incremental word learning in a Hidden Markov Model (HMM) framework suitable for human-robot interaction.In interactive learning, the tutoring time is a crucial factor.Hence our goal is to use as few training samples as possible while maintaining a good performance level.To adapt the states of the HMMs, different large-margin discriminative training strategies for increasing the separability of the classes are proposed.We also present a novel estimation of the variance floor when a very low number of training data is used.Finally our approach is successfully evaluated on isolated digits taken from the TIDIGITS database.

Currently available traffic sign recognition systems typically focus on a single class of traffic sign and therefore, the algorithms are optimized to find only this specific class. To this end, a number of approaches for real time capable classification of mostly circular signs exist. Nevertheless, to simultaneously recognize a number of classes a different way has to be taken. This paper presents a real-time capable approach, which uses a two-tiered process independent of the diameter of the sign to cope with all distances. The first stage is our attention system, parameterized to find a number of different types of sign classes. The output of our attention system is a region of interest with a potential traffic sign candidate. The second stage is an array of weak classifiers similar to the idea of Viola and Jones, computing a probability value for each of the sign classes. As application area we focus on inner city and therefore, evaluate on the most important traffic sign classes of Stop and Give Way. Nevertheless, the approach can also detect Warning signs and is easily extensible to additional sign classes. The evaluation results show the reliability and mark it as first step towards an overall traffic sign recognition.

State-of-the-art advanced driver assistance systems (ADAS) typically focus on single tasks and therefore, have clearly defined functionalities. Although said ADAS functions (e.g. lane departure warning) show good performance, they lack the general ability to extract spatial relations of the environment. These spatial relations are required for scene analysis on a higher layer of abstraction, providing a new quality of scene understanding, e.g. for inner-city crash prevention when trying to detect a Stop sign violation in a complex situation. Otherwise, it will be difficult for an ADAS to deal with complex scenes and situations in a generic way. This contribution presents the novel task dependent generation of spatial representations, allowing task specific extraction of knowledge from the environment based on our biologically motivated ADAS. Additionally, the hierarchy of the approach provides advantages when dealing with heterogeneous processing modules, a large number of tasks and additional new input cues. First results show the reliability of the approach.

Newly emerging, highly complex Advanced Driver Assistance Systems (ADAS) fuse the output of various system modules (e.g., lane detection, object classification). Such knowledge fusion is realized in order to gain additional information of the environment allowing for complex system tasks as path planning, the active search for specific objects and task-specific analysis of the environment. As part of our previous work, we realized a highly generic type of such ADAS using biological principles. The present contribution offers a novel approach for the detection of curbstones and elevated roadsides in inner-city that relies on biological principles taking inspiration from the human neural signal processing. The gathered results can be fused to an ADAS in order to improve the quality of various other system percepts and allow additional system tasks.

We present a framework for estimating formant trajectories. Its focus is to achieve high robustness in noisy environments. Our approach combines a preprocessing based on functional principles of the human auditory system and a probabilistic tracking scheme. For enhancing the formant structure in spectrograms we use a Gammatone filterbank, a spectral preemphasis, as well as a spectral filtering using difference-of-Gaussians (DoG) operators. Finally, a contrast enhancement mimicking a competition between filter responses is applied. The probabilistic tracking scheme adopts the mixture modeling technique for estimating the joint distribution of formants. In conjunction with an algorithm for adaptive frequency range segmentation as well as Bayesian smoothing an efficient framework for estimating formant trajectories is derived. Comprehensive evaluations of our method on the VTR-formant database emphasize its high precision and robustness. We obtained superior performance compared to existing approaches for clean as well as echoic noisy speech. Finally, an implementation of the framework within the scope of an online system using instantaneous feature-based resynthesis demonstrates its applicability to real-world scenarios.

Many everyday human skills can be framed in terms of performing some task subject to constraints imposed by the environment. Constraints are usually unobservable and frequently change between contexts. In this paper, we present a novel approach for learning (unconstrained) control policies from movement data, where observations come from movements under different constraints. As a key ingredient, we introduce a small but highly effective modification to the standard risk functional, allowing us to make a meaningful comparison between the estimated policy and constrained observations. We demonstrate our approach on systems of varying complexity, including kinematic data from the ASIMO humanoid robot with 27 degrees of freedom.

Recent advances in the field of humanoid robotics increase the complexity of the tasks that such robots can perform. This makes it increasingly difficult and inconvenient to program these tasks manually. Furthermore, humanoid robots, in contrast to industrial robots, should in the distant future behave within a social environment. Therefore, it must be possible to extend the robot's abilities in an easy and natural way. To address these requirements, this work investigates the topic of imitation learning of motor skills. The focus lies on providing a humanoid robot with the ability to learn new bi-manual tasks through the observation of object trajectories. For this, an imitation learning framework is presented, which allows the robot to learn the important elements of an observed movement task by application of probabilistic encoding with Gaussian Mixture Models. The learned information is used to initialize an attractor-based movement generation algorithm that optimizes the reproduced movement towards the fulfillment of additional criteria, such as collision avoidance. Experiments performed with the humanoid robot ASIMO show that the proposed system is suitable for transferring information from a human demonstrator to the robot. These results provide a good starting point for more complex and interactive learning tasks.

Man-made real-world environments are dominated by planar surfaces many of which constitute behavior-relevant entities. Thus, the ability to perceive planar surfaces is vital for any embodied system operating in such environments, be it human or robotic. In this paper, we present an architecture for detection and estimation of planar surfaces in the scene from calibrated stereo images. They are represented in a behavior-oriented way, focusing on geometrical properties that are relevant for enabling basic interaction between a robot and the planar surfaces it perceives. Ego-motion of the robot is compensated for by transforming the representations into a global coordinate system using the kinematics of the robot. Our architecture is able to detect and estimate arbitrary planar surfaces, regardless of their visual appearance, their geometrical properties other than planarity and their being static or arbitrarily moving. The latter is achieved by processing each frame independently of the others. Stable representations are obtained by establishing spatio-temporal coherence between the single-frame representations of subsequent frames. Based on a RANSAC approach to plane fitting, our method is robust to unreliable 3D data such as obtained by local stereo correlation, for example. In our experiments using the Honda humanoid robot ASIMO, we show that our method is able to provide a robot in real-time with representations of planar surfaces in its environment that are sufficiently accurate for basic interaction.

Abstract — In this paper we propose a system architecture that extends the current state-of-the-art in computational visual attention by incorporating the biological concept of ventral attention. According to recent findings regarding the neurobiological foundations of attention, there exist two separate but interacting attention systems in the human brain: the dorsal attention system and the ventral attention system. As opposed to the well-known computational concepts of bottomup and top-down saliency, which both correspond to the dorsal attention system, the ventral attention system is sensitive to behavior-relevant stimuli that are unexpected (i.e. not top-down salient), independent of their perceptual saliency (bottom-up saliency). This results in a dynamic interplay between topdown saliency, bottom-up saliency and ventral attention in the proposed system architecture, enabling the system to redirect its focus of attention to important stimuli while being absorbed in a task, even if their perceptual saliency is low. Our technical system instance implementing the proposed architecture integrates several state-of-the-art methods in a coherent system and concentrates on unexpected motion as a first technical account of ventral attention. In our experiments, we demonstrate that the ventral attention enables our system to detect and reorient to important situations in real-world traffic environments that are relevant for the behavior of driving. I.

Many everyday human skills can be framed in terms of performing some task subject to constraints imposed by the environment. Constraints are usually unobservable and frequently change between contexts. In this paper, we present a novel approach for learning (unconstrained) control policies from movement data, where observations are recorded under different constraint settings. Our approach seamlessly integrates unconstrained and constrained observations by performing hybrid optimisation of two risk functionals. The first is a novel risk functional that makes a meaningful comparison between the estimated policy and constrained observations. The second is the standard risk, used to reduce the expected error under impoverished sets of constraints. We demonstrate our approach on systems of varying complexity, and illustrate its utility for transfer learning of a car washing task from human motion capture data.

Previous work [1] shows that the movement representation in task spaces offers many advantages for learning object-related and goal-directed movement tasks through imitation. It allows to reduce the dimensionality of the data that is learned and simplifies the correspondence problem that results from different kinematic structures of teacher and robot. Further, the task space representation provides a first generalization, for example wrt. differing absolute positions, if bi-manual movements are represented in relation to each other. Although task spaces are widely used, even if they are not mentioned explicitly, they are mostly defined a priori. This work is a step towards an automatic selection of task spaces. Observed movements are mapped into a pool of possibly even conflicting task spaces and we present methods that analyze this task space pool in order to acquire task space descriptors that match the observation best. As statistical measures cannot explain importance for all kinds of movements, the presented selection scheme incorporates additional criteria such as an attention-based measure. Further, we introduce methods that make a significant step from purely statistically-driven task space selection towards model-based movement analysis using a simulation of a complex human model. Effort and discomfort of the human teacher is being analyzed and used as a hint for important task elements. All methods are validated with real-world data, gathered using color tracking with a stereo vision system and a VICON motion capturing system.

In this article we present a new framework for auditory processing that combines feature extraction and grouping processes to form what we call audio proto objects. These proto objects combine an arbitrary number of audio features in a compact representation that allows a more precise sound localization and also better interfacing to behavior-control in robotics. We compare our standard sound localization system with the new approach in several scenarios to demonstrate the potential of the new approach.

Reactive control and planning are complementary methods in robot motion control. The advantage of planning is the ability to find difficult solutions, optimize trajectories globally and not getting stuck in local minima but at higher computational cost. On the other hand, reactive control can handle dynamic or uncertain environments at low computational cost, but may get stuck in local minima.

This paper presents a decentralized planning method for generating dynamic whole body motions of multilink robots including humanoids. First, a robotic system will be modeled as a general multi-body dynamical system. The planning problem of a multi-body system will then be formulated as a constraint resolution problem. The problem will be solved by means of an extended Gauss-Seidel method, which is capable of handling multiple constraint groups with different priorities. The method will be demonstrated in whole-body motion generation tasks of a humanoid, both in numerical simulations and in experiments using a real humanoid robot.

Based on inspirations from infant development we present a system which learns associations between acoustic labels and visual representations in interaction with its tutor. The system is integrated with a humanoid robot. Except for a few trigger phrases to start learning all acoustical representations are learned online and in interaction. Similar, for the visual domain the clusters are not predefined and fully learned online. In contrast to other interactive systems the interaction with the acoustic environment is solely based on the two microphones mounted on the robots head. In this paper we give an overview on all key elements of the system and focus on the challenges arising from the headset-free learning of speech labels. In particular we present a mechanism for auditory attention integrating bottom-up and top-down information for the segmentation of the acoustic stream. The performance of the system is evaluated based on offline tests of individual parts of the system and an analysis of the online behavior.

Understanding the bootstrapping process of speech representation in infants is one key issue towards systems which may provide humanlike speech recognition abilities some day. Until now, almost all current speech recognition systems have failed to integrate learning into the recognition process. Here we propose a system for unsupervised word-clustering, which is able to recognize and learn the structure of speech online in a unified framework. To do so we've extended HMM-based filler-free keyword spotting with acoustic model acquisition. To evaluate and control the dynamics of the combined acquisition-recognition process we propose measures for model activity, model correlation and speech coverage.

Previously we presented an auditory-inspired feed-forward architecture which achieves good performance in noisy conditions on a segmented word recognition task. In this paper we propose to use a modified version of this hierarchical model to generate features for standard hidden Markov models. To obtain these features we firstly compute the spectrograms using a Gammatone filterbank. A filtering over the channels permits to enhance the formant frequencies which are afterwards detected using Gabor-like receptive fields. Then the responses of the receptive fields are combined to complex features which span the whole frequency range and extend over three different time windows. The features have been evaluated on a single digit recognition task. The results show that their combination with MFCCs or RASTA features yields improved recognition scores in noise.

Many advanced motion control strategies developed in robotics use captured human motion data as valuable source of examples to simplify the process of programming or learning complex robot motions. Direct and online control of robots from observed human motion has several inherent challenges. The most important may be the representation of the large number of mechanical degrees of freedom involved in the execution of movement tasks. Attempting to map all such degrees of freedom from a human to a humanoid is a formidable task from an instrumentation and sensing point of view. More importantly, such an approach is incompatible with mechanisms in the central nervous system which are believed to organize or simplify the control of these degrees of freedom during motion execution and motor learning phase. Rather than specifying the desired motion of every degree of freedom for the purpose of motion control, it is important to describe motion by low dimensional motion primitives that are defined in Cartesian (or task) space. In this paper, we formulate the human to humanoid retargeting problem as a task space control problem. The control objective is to track desired task descriptors while satisfying constraints such as joint limits, velocity limits, collision avoidance, and balance. The retargeting algorithm generates the joint space trajectories that are commanded to the robot. We present experimental and simulation results of the retargeting control algorithm on the Honda humanoid robot ASIMO.

We recently introduced a computationally efficient framework for tracking formants which combines a biologically inspired preprocessing for enhancing formants in spectrograms with a probabilistic framework for estimating formant trajectories. In contrast to previously published approaches our tracking scheme relies on the joint distribution of formants rather than using independent tracking instances for each formant separately. Therewith more precise formant estimates could be obtained. In this paper we will briefly review our algorithm and extend it by using more sophisticated models of the formants underlying dynamics. Furthermore, we will dwell on the robustness of our method for speech degraded by various types of noise. A comprehensive evaluation on a large publicly available database containing hand-labeled formant trajectories shows significant performance improvements in both clean and noisy speech compared to state of the art approaches.

Speech recognition robust against interfering noise remains a difficult task. We previously presented a set of spectrotemporal speech features which we termed Hierarchical Spectro-Temporal (HIST) features showing improved robustness, especially when combined with RASTA-PLP. They are inspired by the receptive fields found in the mammalian auditory cortex and are organized in two hierarchical levels. A set of filters learned via ICA captures local variations and constitutes the first layer of the hierarchy. In the second layer these local variations are combined to form larger receptive fields learned via Non Negative Sparse Coding. In this paper we introduce a non-linear smoothing along the time axis of the spectrograms at the input to the hierarchy and, additionally, a more thorough performance analysis on an isolated and a continuous digit recognition task. The results show that the combination of HIST and RASTA-PLP features yields improved recognition scores in noise.

Transferring motion from a human demonstrator to a humanoid robot is an important step toward developing robots that are easily programmable and that can replicate or learn from observed human motion. The so called motion retargeting problem has been well studied and several off-line solutions exist based on optimization approaches that rely on pre-recorded human motion data collected from a marker-based motion capture system. From the perspective of human robot interaction, there is a growing interest in online and marker-less motion transfer. Such requirements have placed stringent demands on retargeting algorithms and limited the potential use of off-line and pre-recorded methods. To address these limitations, we present an online task space control theoretic retargeting formulation to generate robot joint motions that adhere to the robot’s joint limit constraints, self-collision constraints, and balance constraints. The inputs to the proposed method include low dimensional normalized human motion descriptors, detected and tracked using a vision based feature detection and tracking algorithm. The proposed vision algorithm does not rely on markers placed on anatomical landmarks, nor does it require special instrumentation or calibration. The current implementation requires a depth image sequence, which is collected from a single time of flight imaging device. We present online experimental results of the entire pipeline on the Honda humanoid robot - ASIMO.

This paper presents an integrative approach to solve the coupled problem of reaching and grasping an object in a cluttered environment with a humanoid robot. While finding an optimal grasp is often treated independently from reaching to the object, in most situations it depends on how the robot can reach a pregrasp pose while avoiding obstacles. We tackle this problem by introducing the concept of task maps which represent the manifold of feasible grasps for an object. Rather than defining a single end-effector goal position, a task map defines a goal hyper volume in the task space. We show how to efficiently learn such maps using the rapidly exploring random tree algorithm. Further, we generalise a previously developed motion optimisation scheme, based on a sequential attractor representation of motion, to cope with such task maps. The optimisation procedure incorporates the robotpsilas redundant whole body controller and uses analytic gradients to jointly optimise the motion costs (including criteria such as collision and joint limit avoidance, energy efficiency, etc.) and the choice of the grasp on the manifold of valid grasps. This leads to a preference of grasps which are easy to reach. The approach is demonstrated in two reach-grasp simulation scenarios with the humanoid robot ASIMO.

We present a system for online extraction of the fundamental frequency and the first four formant frequencies from a speech signal. In order to evaluate the performance of the extraction a resynthesis of the speech signal is performed. The resynthesis is based on the extracted frequencies and the energy of the input signal at the formant locations. The extraction of the fundamental frequency and the formants is robust against room echoes and interfering noise. In order to improve the robustness against background noise a noise reduction was implemented. Tests in three rooms of different size at varying distances to the system (up to 8 m yielding an SNR of approx. 0 dB) were performed.

It is a common assumption that with just two microphones only the azimuth angle of a sound source can be estimated and that a third, orthogonal microphone (or set of microphones) is necessary to estimate the elevation of the source. Recently, using specially designed ears and analyzing spectral cues several researchers managed to estimate sound source elevation with a binaural system. In this work, we show that with two bionic ears both azimuth and elevation angle can be determined using both binaural (e.g. IID and ITD) and spectral cues. This ability can also be used to disambiguate signals coming from the front or back. We present a detailed analysis of both azimuth and elevation localization performance for binaural and spectral cues in comparison. We demonstrate that with a small extension of a standard binaural system a basic elevation estimation capacity can be gained.

For the interaction of a mobile robot with a dynamic environment, the estimation of object motion is desired while the robot is walking and/or turning its head. In this paper, we describe a system which manages this task by combining depth from a stereo camera and computation of the camera movement from robot kinematics in order to stabilize the camera images. Moving objects are detected by applying optical flow to the stabilized images followed by a filtering method, which incorporates both prior knowledge about the accuracy of the measurement and the uncertainties of the measurement process itself. The efficiency of this system is demonstrated in a dynamic real-world scenario with a walking humanoid robot.

We propose a method for the joint estimation of formant trajectories from spectrograms. Formants are enhanced in the spectrograms obtained from the application of a Gammatone filterbank via a smoothing along the frequency axis. In contrast to previously published approaches, the used tracking algorithm relies on the joint distribution of formants rather than using independent tracker instances. More precisely, Bayesian mixture filtering in conjunction with adaptive frequency range segmentation as well as Bayesian smoothing are used. The algorithm was evaluated on a publicly available database containing hand-labeled formant tracks. Experimental results show a significant performance improvement compared to a state of the art approach.

We describe a system for visual interaction developed for humanoid robots. It enables the robot to interact with its environment using a smooth whole body motion control driven by stabilized visual targets. Targets are defined as visually extracted "proto-objects" and behavior-relevant object hypotheses and are stabilized by means of a short-term sensory memory. Selection mechanisms are used to switch between behavior alternatives for searching or tracking objects as well as different whole body motion strategies for reaching. The decision between different motion strategies like reaching with right or left hand or with and without walking is made based on internal predictions that use copies of the whole-body control algorithm. The results show robust object tracking and a smooth interaction behavior that includes a large variety of whole-body postures.

In this paper we propose an algorithm for the robust extraction of pitch combining both temporal (rate) and pattern matching (place) techniques. Following a transformation into the spec-tral domain via the application of a Gammatone filter bank the rate information is extracted in each band via the zero crossing distances in that band. Next a comb filter with teeth at the har-monics of the current fundamental frequency hypothesis is set up, reflecting the pattern matching aspect. The signals below the teeth of the comb filter are analyzed upon consistency. This yields an allocation pattern for the filter. The current allocation pattern is compared to prototypical ones allowing the suppres-sion of side peaks at harmonics and sub-harmonics of the true fundamental. A comparison to a state of the art autocorrela-tion based algorithm is performed showing significantly better results for our algorithm. Index Terms: fundamental frequency, pitch, zero crossings, rate model, place model

We present a system for an online-adaptation of auditory-motor maps that doesn't require a special set-up or dedicated robot movements and can therefore work during the normal operation of the robot. Our approach is based purely on auditory cues and motor position feedback for estimating the correct sound source position. The system can learn the correct auditory-motor map within 1-2 hours, starting from a random initialization, in a room with an active radio as the main sound source.

We propose a self collision avoidance system that superposes trajectories in order not only to protect the robot's hardware but also to enable continuous motions. The system runs in real-time so that the robot can work in an uncertain environment. It is based on virtual forces between close segments of the robot. The avoidance movements are blended with a whole body motion control in order to change the priority between target reaching and collision avoidance. The blending is performed autonomously without the necessity of external switching. Our method works both while the robot is standing and walking. Reaching motions from the front to the side of the body without the arm colliding with the body are possible. Even if the target is inside the body, the arm stops at the closest point to the target outside of the body. Our method can be used for other applications: We apply it to realizing a "body schema" and for "occlusion avoidance."

Real-world robotic environments are highly structured. The scalability of planning and reasoning methods to cope with complex problems in such environments crucially depends on exploiting this structure. We propose a new approach to planning in robotics based on probabilistic inference. The method uses structured Dynamic Bayesian Networks to represent the scenario and efficient inference techniques (loopy belief propagation) to solve planning problems. In principle, any kind of factored or hierarchical state representations can be accounted for. We demonstrate the approach on reaching tasks under collision avoidance constraints with a humanoid upper body.

We present a biologically motivated architecture for object recognition that is capable of online learning of several objects based on interaction with a human teacher. The system combines biological principles such as appearance-based representation in topographical feature detection hierarchies and context-driven transfer between different levels of object memory. Training can be performed in an unconstrained environment by presenting objects in front of a stereo camera system and labeling them by speech input. The learning is fully online and thus avoids an artificial separation of the interaction into training and test phases. We demonstrate the performance on a challenging ensemble of 50 objects.

We propose a new algorithm for voiced/unvoiced classification of speech on a phoneme or sample level. The algorithm is inspired by auditory based approaches and combines two cues. One cue is based on the energy distribution of the signal and the other on the harmonicity. In order to extract the harmonicity of the signal we calculate a histogram of the zero crossings of the filter channels after applying a Gammatone filterbank to the signal. A measure similar to the variance of the zero crossings yields the harmonicity cue. The performance of the algorithm was measured on several minutes of read and spontaneous speech with various speakers. An algorithm proposed by Mustafa et al. [1] served as benchmark. The results show that our algorithm performs significantly better as well on read as on spontaneous speech and seems in particular be better able to to cope with different speaking styles. Index Terms: speech analysis, voiced/unvoiced detection, speaking style, zero crossings.

We present a new way of modelling the Precedence Effect to enable the robust measurement of localization cues (ITD and IID) in echoic environments. Based on this we developed a localization system which is inspired by the auditory system of mammals. It uses a Gammatone filter bank for preprocessing and extracts the ITD cue via zero crossings (IID calculation is straight forward). The mapping between the cue values and the different angles is learned offline which facilitates the adaptation to different head geometries. The performance of the system is demonstrated by localization results for two simultaneous speakers and the mixture of a speaker, music, and fan noise in a normal meeting room. A real-time demonstrator of the system is presented in [1]. Index Terms: sound source localization, binaural, precedence effect, reverberant, echoic.

In the field of intelligent systems, research and design approaches vary from predefined architectures to self-organizing systems. Regardless of the architectural approach, such systems may grow in size and complexity to levels where the capacities of people are strongly challenged. Such systems are commonly researched, designed and developed following several methods and with the help of a variety of software tools. In this paper we want to describe our research and development environment. It is composed of a set of tools that support our research and enable us to develop large scale intelligent systems used in our robots and in our test platforms. The main parts of our research and development environment are: the component models BBCM (brain bytes component model) and BBDM (brain bytes data model), the middleware RTBOS (real-time brain operating system), the monitoring system CMBOS (control-monitor brain operating system) and the design environment DTBOS (design tool for brain operating system). We will compare our research and development environment with others available on the market or still in research phase and we will describe some of our experiments

This paper presents a whole body motion algorithm and shows some steps towards its feasibility in complex scenarios. We employ the framework of Liegeois, (1977) which solves the redundant inverse kinematics problem on velocity level. To make the controller suitable for a variety of different applications, task descriptors for the relative effector positions as well as a one-and two-dimensional attitude representation are proposed. The inverse kinematics are extended by allowing for "displacement intervals" which are formulated in task space. The proposed control scheme guarantees that the effector motion lies within the specified interval. However, the motion inside the interval is determined by optimization criteria, which can effectively be utilized to generate a more flexible and robust motion. We discuss an example and show simulation and experimental results on the humanoid robot ASIMO

We propose a new approach for binaural sound source localization in real world environments implementing a new model of the precedence effect. This enables the robust measurement of the localization cue values (ITD, UD and IED) in echoic environments. The system is inspired by the auditory system of mammals. It uses a Gammatone filter bank for preprocessing and extracts the ITD and IED cues via zero crossings (UD calculation is straight forward). The mapping between the cue values and the different angles is learned offline which facilitates the adaptation to different head geometries. The performance of the system is demonstrated by localization results for two simultaneous speakers and the mixture of a speaker, music, and fan noise in a normal meeting room. A real time demonstrator of the system is presented in T. Rodemann, et al. (2006)

We present a sound localization system that operates in real-time, calculates three binaural cues (IED, UD, and ITD) and integrates them in a biologically inspired fashion to a combined localization estimation. Position information is furthermore integrated over frequency channels and time. The localization system controls a head motor to fovealize on and track the dominant sound source. Due to an integrated noise-reduction module the system shows robust localization capabilities even in noisy conditions. Real-time performance is gained by multi-threaded parallel operation across different machines using a timestamp-based synchronization scheme to compensate for processing delays

Pattern-based learning processes are usually analyzed by means of probability density functions of the weights or moments thereof. During the derivation of these equations, some averaging has to be performed. We show that the manner of averaging is crucial for the results of the analysis. We do this by comparing two types of analysis (Langevin type and discrete-time moments) for one learning system.

The success of learning as well as the learning speed of an artificial neural network (ANN) strongly depends on the initial weights. If problem or domain specific knowledge exists, it can be transferred to the ANN by means of a special choice of the initial weights. In this paper, we focus on the choice of a set of initial weights, well suited to fast and robust learning of all particular problems out of a class of related problems. Our evolutionary approach particularly takes the learning algorithm into consideration in the design of the initial weights. The superior properties of the initial weights resulting from this algorithm are corroborated using a class defined by solving a differential equation with variable boundary conditions.

Artificial neural networks have evolved from their biologically inspired roots to a well established means to solve a broad spectrum of engineering problems. Their embedding into modern statistics has provided the necessary theoretical foundation for challenging engineering tasks, such as advanced real time image and signal processing. These are exemplary demonstrations for the applicability of this approach to complex information processing. However, the large number of applications must not obscure the fact that there are some major unsolved problems concerning neural networks. There are still no satisfactorily constructive ways to determine the optimal structure (elements as well as organization) or the learning and evaluation dynamics. Ongoing research addresses these problems. In addition to pursuing this direction, one can ask what other lessons we can learn from biology concerning complex information processing. Our goal is to sketch a possible pathway from neural networks to more comprehensive neural strategies.