In comparison to industrial settings with structured environments, the operation of autonomous robots in unstructured and uncertain environments is more challenging. This video presents a generic control and system architecture ARCADE, applicable for real-time robot control in complex task situations. Several methods to cope with uncertainties are demonstrated with the example task of changing tires on a car. Approaches of object detection (applied to car, tires, and humans), robust real-time control of robot arms under perception uncertainty, and human-friendly haptic interaction are detailed. The video shows two robots jointly performing the task of mounting a mock-up tire to a real car using the proposed methods, realizing robust performance in an uncertain environment.

This work presents three imaging radar sensor approaches to estimate road courses, needed by intelligent vehicle systems such as active cruise control or collision avoidance. Two of the approaches use gridmap data. A gridmap integrates each measurement in a chronological order. The third approach analyzes moving objects ahead of the ego vehicle. One approach has been published previously, the other two are new. A range estimation is necessary on country roads and completes each approach. All approaches are evaluated using a huge dataset of country roads. The driven trajectory is taken as ground truth for the evaluation. The advantages and disadvantages are determined for each approach. The results show the new approach based on gridmap data performs up to 78% better than the known one. The other new approach using moving objects as input information yields estimations which are about three times more accurate than the ones from the known approach.

This work presents three different quality functions for road course estimation using an imaging radar sensor. The quality functions work on gridmap data. A gridmap integrates each measurement in chronological order. Range estimation is found out to be necessary on country roads and a solution is introduced. All quality functions are evaluated using a huge set of data consisting of highways and country roads. The driven trajectory is taken as ground truth for the evaluation. The results show that on highways the quality functions perform nearly equal. However on country roads there are differences. The huge error reduction achieved by the range estimation is pointed out. At the end the quality function performing best is determined.

In this paper, the impact of facial expressions on HRI is explored. To determine their influence on empathy of a human towards a robot and perceived subjective performance, an experimental setup is created, in which participants engage in a dialog with the robot head EDDIE. The web-based gaming application “Akinator” serves as a backbone for the dialog structure. In this game, the robot tries to guess a thought-of person chosen by the human by asking various questions about the person. In our experimental evaluation, the robot reacts in various ways to the human's facial expressions, either ignoring them, mirroring them, or displaying its own facial expression based on a psychological model for social awareness. In which way this robot behavior influences human perception of the interaction is investigated by a questionnaire. Our results support the hypothesis that the robot behavior during interaction heavily influences the extent of empathy by a human towards a robot and perceived subjective task-performance, with the adaptive modes clearly leading compared to the non-adaptive mode.

Every day human communication relies on a large number of different communication mechanisms like spoken language, facial expressions, body pose and gestures, allowing humans to pass large amounts of information in short time. In contrast, traditional human-machine communication is often unintuitive and requires specifically trained personal. In this paper, we present a real-time capable framework that recognizes traditional visual human communication signals in order to establish a more intuitive human-machine interaction. Humans rely on the interaction partner’s face for identification, which helps them to adapt to the interaction partner and utilize context information. Head gestures (head nodding and head shaking) are a convenient way to show agreement or disagreement. Facial expressions give evidence about the interaction partners’ emotional state and hand gestures are a fast way of passing simple commands. The recognition of all interaction queues is performed in parallel, enabled by a shared memory implementation.

In everyday live head gestures such as head shaking or nodding and hand gestures like pointing gestures form important aspects of human-human interaction. Therefore, recent research considers integrating these intuitive communication cues into technical systems for improving and easing human-computer interaction. In this paper we present a vision-based system to recognize head gestures (nodding, shaking, neutral) and dynamic hand gestures (hand moving right/left/up/down, fist moving right/left) in real-time. The gestural input delivers a communication modality for a human-robot interaction scenario situated in an assistive household environment. The use of fast low-level image-feature extraction methods contributes to the real-time capability of the system and advanced classification approaches relying on Graphical Models provide high robustness. Graphical Models offer the possibility to group the input features in several sub-nodes resulting in a better classification than obtained via a traditional Hidden Markov Model classification. The applied grouping can regard interdependencies owing to, either physical constraints (like for the head gestures), or interrelations between shape and motion (like for the hand gestures).

We introduce a facial mimicry system, which combines facial expression analysis and synthesis on a robot, utilizing the facial action coding system. The activation of action units on a user's face is automatically extracted from a video stream and mapped to the robot, thus mirroring the facial expression. As a novel approach, a user study quantifies the congruence of the initial human facial expression with the robotic facial expression. The evaluation shows that the robotic facial expression is perceived to be close to the human facial expression, from which it is derived. This is a fundamental aspect for a mimicry system, providing a basis for future research on empathy and emotional closed loop control.

We present a fully automated three-dimensional model-based, real-time capable approach to distinguish six universal facial expressions from visual images of human faces. The face model is fitted to the images of a publicly available data base. From the model parameters two sets of features are computed, person-specific and non-person-specific to estimate the facial expression visible in the current image sequence. We integrate the complex, state-of-the-art Candide-3 face model, which is specifically appropriate for facial expression recognition and is also publicly available, which further increases the comparability of our approach. Our experimental evaluation presents recognition rates of the six basic facial expressions defined by Ekman et al.

In this paper, we present a vehicle safety application based on data gathered by a laser scanner and two short-range radars that recognize unavoidable collisions with stationary objects before they take place to trigger restraint systems. Two different software modules that perform the processing of raw data and deliver a description of the vehicle's environment are compared. A comprehensive experimental evaluation based on relevant crash and noncrash scenarios is presented.

Computers have been widely deployed to our daily lives, but human-computer interaction still lacks intuition. Researchers intend to resolve these shortcomings by augmenting traditional systems with human-like interaction capabilities. Knowledge about human emotion, behavior, and intention is necessary to construct convenient interaction mechanisms. Today, dedicated hardware often infers the emotional state from human body measures. Similar to humans interpreting facial expressions, our approach acquires video information using standard hardware that does not interfere with people to accomplish this task. It exploits model-based techniques that accurately localize facial features, seamlessly track them through image sequences, and finally interpret the visible information. We make use of state-of-the-art techniques and specifically adapt most of the components involved to this scenario, which provides high accuracy and real-time capability. We base our experimental evaluation on publicly available databases and compare its results to related approaches. Our proof-of-concept demonstrates the feasibility of our approach and shows promising for integration into various applications.

Our system runs at 10 fps on a 2.0 GHz processor and an image resolution of 640times480 pixels. High quality objective functions that are learned from annotated example images ensure both an accurate and fast computation of the model parameters. Our demonstrator for facial expression estimation has been presented at several events with political audience and on TV. However, the approach of robust face models fitting, forms the basis of various more applications such as gaze detection or gender estimation. The drawback of our approach is that the data base from which the objective function is learned needs to cover all aspects of face properties. If, for instance, the database did not contain images of bearded men the objective function will fail when confronted with such an image. Furthermore, the data base has to be manually annotated. Although no expert knowledge is required, this task requires a considerable amount of time. An online fitting demonstration is available.

Due to their use of information contained in texture, active appearance models (AAM) generally outperform active shape models (ASM) in terms of fitting accuracy. Although many extensions and improvements over the original AAM have been proposed, on of the main drawbacks of AAMs remains its dependence on good initial model parameters to achieve accurate fitting results. In this paper, we determine the initial model parameters for AAM fitting with ASM fitting, and use machine learning techniques to improve the scope and accuracy of ASM fitting. Combining the precision of AAM fitting with the large radius of convergence of learned ASM fitting improves the results by an order of magnitude, as our empirical evaluation on a database of publicly available benchmark images demonstrates.

Efficient and accurate localization of the components of human faces, such as skin, lips, eyes, and brows, provides benefit to various real-world applications. However, high intra-class and small inter-class variations in color prevent simple but quick pixel classifiers from yielding robust results. In contrast, more elaborate classifiers consider shape or region features but they do not achieve real-time performance. In this paper, we show that it definitely is possible to robustly determine the facial components and achieve far more than real-time performance. We also use quick pixel-level classifiers and provide them with a set of pixel features that are adapted to the image characteristics beforehand. We do not manually select the pixel features and specify the calculation rules. In contrast, our idea is to provide a multitude of features and let the machine learning algorithm decide which of them are important. The evaluation draws a comparison to fixed approaches that do not adapt the computation of the features to the image content in any way. The obtained accuracy is precise enough to be used for real-world applications such as for model-based interpretation of human faces.

This paper introduces the assistive kitchen as a comprehensive demonstration and challenge scenario for technical cognitive systems. We describe its hardware and software infrastructure. Within the assistive kitchen application, we select particular domain activities as research subjects and identify the cognitive capabilities needed for perceiving, interpreting, analyzing, and executing these activities as research foci. We conclude by outlining open research issues that need to be solved to realize the scenarios successfully.

Model-based techniques have proven to be successful in interpreting the large amount of information contained in images. Associated fitting algorithms search for the global optimum of an objective function, which should correspond to the best model fit in a given image. Although fitting algorithms have been the subject of intensive research and evaluation, the objective function is usually designed ad hoc, based on implicit and domain-dependent knowledge. In this article, we address the root of the problem by learning more robust objective functions. First, we formulate a set of desirable properties for objective functions and give a concrete example function that has these properties. Then, we propose a novel approach that learns an objective function from training data generated by manual image annotations and this ideal objective function. In this approach, critical decisions such as feature selection are automated, and the remaining manual steps hardly require domain-dependent knowledge. Furthermore, an extensive empirical evaluation demonstrates that the obtained objective functions yield more robustness. Learned objective functions enable fitting algorithms to determine the best model fit more accurately than with designed objective functions.

Great interest is recently shown in behavior modeling, especially in public surveillance tasks. In general it is agreed upon the benefits of use of several input cues as audio and video. Yet, synchronization and fusion of these information sources remains the main challenge. We therefore show results for a feature space combination, which allows for overall feature space optimization. Audio and video features are thereby firstly derived as low-level-descriptors. Synchronization and feature combination is achieved by multivariate time-series analysis. Test-runs on a database of aggressive, cheerful, intoxicated, nervous, neutral, and tired behavior in an airplane situation show a significant improvement over each single modality.

Model-based image interpretation extracts high-level information from images using a priori knowledge about the object of interest. The computational challenge in model fitting is to determine the model parameters that best match a given image, which corresponds to finding the global optimum of the objective function. When it comes to the robustness and accuracy of fitting models to specific images, humans still outperform state- of-the-art model fitting systems. Therefore, we propose a method in which non-experts can guide the process of designing model fitting algorithms. In particular, this paper demonstrates how to obtain robust objective functions for face model fitting applications, by learning their calculation rules from example images annotated by humans. We evaluate the obtained function using a publicly available image database and compare it to a recent state-of-the-art approach in terms of accuracy.

Model-based methods in computer vision have proven to be a good approach for compressing the large amount of information in images. Fitting algorithms search for those parameters of the model that optimise the objective function given a certain image. Although fitting algorithms have been the subject of intensive research and evaluation, the objective function is usually designed ad hoc and heuristically with much implicit domain-dependent knowledge. This paper formulates a set of requirements that robust objective functions should satisfy. Furthermore, we propose a novel approach that learns the objective function from training images that have been annotated with the preferred model parameters. The requirements are automatically enforced during the learning phase, which yields generally applicable objective functions. We compare the performance of our approach to other approaches. For this purpose, we propose a set of indicators that evaluate how well an objective function meets the stated requirements. 1

With the services that autonomous robots are to provide becoming more demanding, the states that the robots have to estimate become more complex. In this paper, we develop and analyze a probabilistic, vision-based state estimation method for individual autonomous robots. This method enables a team of mobile robots to estimate their joint positions in a known environment and track the positions of autonomously moving objects. The state estimators of different robots cooperate to increase the accuracy and reliability of the estimation process. This cooperation between the robots enables them to track temporarily occluded objects and to faster recover their position after they have lost track of it. The method is empirically validated based on experiments with a team of physical robots.

A new real time 3D-data and color acquisition system for the mid-distance range (0.5-3 m) employing color-encoded structured light is presented. The system is integrated using low-cost components and allows the combination of 2D and 3D image processing algorithms since it provides a 2D-color image of the scene in addition to the range data. Its design is focused on enabling the system to operate reliably in real-world scenarios, i.e. in uncontrolled environments and with arbitrary scenes. To that end novel approaches for encoding and recognizing the projected light are used which make the system practically independent of intrinsic object colors and minimize the influence of the ambient light conditions. Experimental results obtained for the first applications of the system, 3D face and gesture recognition tasks, are presented.