This paper reports a failed attempt to predict deviations of human arm movements from task elicited by perception of simultaneous robot arm movements (motor interference) using subjective ratings of anthropomorphism. This motor interference effect is known to result from unconscious imitation of movements of a human-like counterpart due to mirror-neuron activation. A study is conducted, with test participants and a robot conducting vertical and horizontal arm movements in front of each other in all combinations, and additionally with and without a robot head being present, in a between-subjects design. Perceived anthropomorphism is acquired using the evaluated tests HRIES and Godspeed. A correlation analysis on the dependency of motor interference on anthropomorphism did not reveal significant results of the constructs targeting human-like appearance, intelligence or movement behavior, which were expected to represent relevant predictor parameters. Further, scatter plots do not show a clear pattern. Only the likability Godspeed sub-scale is significant, which however may not be related to the targeted effect. Additionally, the ‘consciousness’ item within the ‘anthropomorphism sub-scale of Godspeed is significant, however, with unclear effect. So, no evidence could be found, that subjective assessments of anthropomorphism with existing tests may be suitable for prediction of motor interference and further research is required.

In this paper, results from a video-based study on the influence of prior information given to users and explanations situationally given by the vehicle itself on trust and perceived intelligence are presented using a simulated autonomous vehicle in an ambiguous driving situation. A 2x2 between-subjects design is chosen with two independent variables ‘prior information’ (extended/short) and ‘explanations’ (yes/no) with users pseudo-randomly assigned to one of the four conditions. Significant results from 189 test persons reveal, that trust depends on how the capabilities of the intelligent vehicle are explained a priori and not on situational explanations, while perceived intelligence is influenced by both variables. Additional interactions of prior information and user gender is noted with respect to perceived intelligence. As one side effect, it is found, that male users felt significantly more safe than female users with also higher ratings of intention to use the vehicle independently of given information and explanations. Another side effect is that situational explanations lead to better ratings of subjective performance, while also here a significant interaction of gender and prior information is noted. Thus, contrary to expectations, a dominant role of continuous situational explanations (Explainable AI) of the intelligent vehicle for increasing trust is not confirmed and the extent of given prior information seems the deciding factor for initial trust building, which is an important aspect for the introduction of new intelligent technology into society. This is remarkable as at the same time perceived intelligence seems to be dependent on both variables. So it appears, that a vehicle being able to explain its actions may well appear more intelligent, but not necessarily appear more trustworthy.

This paper presents results from a study on the perception of robot attributes by human observers in their peripheral field of view depending on types of eye-movements of the latter. A between-subjects design is used, where a picture of a robot head is presented on a screen in the peripheral field of view of the participants with two conditions of eye-movements: static and pursuit. The two conditions are realized by a dot to be tracked by the participants, which is placed either in the center of the screen or moving linearly between left and right screen margins. As a subjective measure for appearance attributes, anthropomorphism is used using the multi-component Human-Robot Interaction Evaluation Scale (HRIES). Significant differences are revealed in the sociability and agency sub-scales of HRIES with respect to eye-movement behaviors. The results show, that humans perceive robot appearance attributes differently depending on whether or not pursuit eye-movements are conducted supporting the main hypothesis. Further, for pursuit movements, the scale center is rated for these sub-scales, which may indicate, that the particular robot characteristic is perceived less distinctively. No significant interactions are found for animacy and disturbance, which may be due to less applicability of these sub-scales to static images. The findings may have an impact on close interaction between humans and robots and potential anthropomorphism-related influences on task accomplishment to be considered in interaction design.

This paper is a first step towards the investigation of civil courage in human-robot interaction (HRI). The main research question is if human users would help a robot being bullied by other humans. Previous work showed that pro-social behavior can be induced in human users towards a robot pro-actively asking for their help in order to accomplish a specific task by applying mechanisms of social bonding. In contrast, this paper investigates unsolicited helpful behavior towards a robot being bullied by a third person subsequent to an interaction task. To this end, social bonding in terms of small talk including explicit emotional adaptation to induce a feeling of similarity is applied to a human-robot dialog scenario in a user study. As an interaction context, a cooperative object classification task is chosen, where a robot reads objects from a list needed by the robot to fulfill another task later. To induce bullying behavior, the list is took away from the robot by a disturbing third person after the completed interaction. The two experimental conditions of the study differ in whether or not social bonding is applied prior to the interaction. According to previous work, results showed increased ratings for social presence and anthropomorphism, as well as increased unsolicited helpfulness of the participants in the social bonding condition. Surprisingly, unsolicited help occurred only verbally and directed towards the robot and none of the human users took action against the bullying third person. It is discussed, that this may be due to social-psychological side-effects caused by the passive presence of the human experimental examiner and that additional channels of emotional adaptation by the robot may be required.

Results from a study on human gaze behavior during a synchronous human-robot arm movement task with incongruent motions and robot appearance variation are presented. Significant interactions of human gaze behavior and robot appearance (presence or absence of a robot head) towards deviations of human hand movements from task direction are found. Depending on the level of humanoid embodiment, different gaze strategies lead to different deviations from task motion, which may be explained with different allocation of attention in pursuit and static point fixation behaviors. Besides a better understanding of gaze dependent impact of robot appearance on interaction, a practical implication is the potential for deriving respective situational uncertainty models of human motion in order to improve safety and efficiency of close human-robot interaction.

This paper investigates the progression of human hand trajectory variabilities during a pick-and-place task. A user study is conducted and human hand positions are tracked optically. Standard deviations of human hand positions over all trajectories within a trial are computed point-wise orthogonally to the direct path between start and goal positions. Statistical tests reveal a decrease of standard deviations from hand start to goal positions. Moreover, stronger variations of standard deviations are noted in during the center part of the trajectories. Contrary to expectations, a longitudinal study design does not reveal learning effects in terms of reduction of trajectory variabilities. The results suggest, that uncertainties of human hand positions increase with the distance to a goal location and could constitute a larger risk for collisions within a cooperative human-robot pick-and-place scenario, e.g. assembly.

Emotional adaptation increases pro-social behavior of humans towards robotic interaction partners. Social cues are an important factor in this context. This work investigates, if emotional adaptation still works under absence of human-like facial Action Units. A human-robot dialog scenario is chosen using NAO pretending to work for a supermarket and involving humans providing object names to the robot for training purposes. In a user study, two conditions are implemented with or without explicit emotional adaptation of NAO to the human user in a between-subjects design. Evaluations of user experience and acceptance are conducted based on evaluated measures of human-robot interaction (HRI). The results of the user study reveal a significant increase of helpfulness (number of named objects), anthropomorphism, and empathy in the explicit emotional adaptation condition even without social cues of facial Action Units, but only in case of prior robot contact of the test persons. Otherwise, an opposite effect is found. These findings suggest, that reduction of these social cues can be overcome by robot experience prior to the interaction task, e.g. realizable by an additional bonding phase, confirming the importance of such from previous work. Additionally, an interaction with academic background of the participants is found.

This work investigates the impact of simply keeping eye-contact on users' perceptions of the humanoid robot NAO during the carry and handover phases of a fetch- and-handover scenario. Two conditions of NAO looking either towards its planned path or towards the human test subject's face are implemented in a within-subjects design. Evaluations of user experience and acceptance are conducted based on evaluated measures of human-robot interaction. The results of the user study reveal a significant increase of anthropomorphism, animacy, and social presence in the eye-contact condition even without an intelligent gaze control implementation. Additionally, significant interactions with the professional background of the test participants are found.

Motion control is certainly a challenging task when faced with high speed and high precision control requirements. After the theoretical design, the motion controller should be evaluated by simulation, then refined on the real hardware, under the influence of unmodeled parameters, such as friction, update latency etc. For this purpose, we propose a hardware-in-the-loop (HIL) development and real-time testing framework based on RTAI Linux and Matlab/Simulink. The MATLAB backend of RTAI-Lab is extended to support the external mode of Simulink, such that the internal data structure of the controller running on target boards, is directly exposed to Simulink on local or distributed development workstation. Furthermore, the parameters of the controller can be changed online using the existing tools of Simulink, or through a newly developed web interface. Based on our experience, the changed workflow introduced by the HIL simulation environment is more efficient than the two phase development pattern we have used before, and the real-time testing makes the debugging of the motion control algorithms much easier.

The stability of feature matching is a fundamental problem for many robotic tasks, such as visual servoing and navigation. This paper presents a new feature extractor which is able to improve the robustness of feature matching under large scale change. The new extractor consists of a fast and scalable Laplacian of Gaussian (LoG) approximator based on blocky Mexican hat wavelet, and an optimized sampling distribution for the features in the multi-resolution scale space. The sampling distribution is a critical factor to boosting the matching rate, however it was not discussed in depth by the studies in recent years. For evaluation, the new algorithm is compared with SIFT and SURF, it demonstrates a significant matching rate improvement.

Most MAP inference algorithms for CRFs optimize an energy function knowing all the potentials. In this paper, we focus on CRFs where the computational cost of instantiating the potentials is orders of magnitude higher than MAP inference. This is often the case in semantic image segmentation, where most potentials are instantiated by slow classifiers fed with costly features. We introduce Active MAP inference 1) to on-the-fly select a subset of potentials to be instantiated in the energy function, leaving the rest of the parameters of the potentials unknown, and 2) to estimate the MAP labeling from such incomplete energy function. Results for semantic segmentation benchmarks, namely PASCAL VOC 2010 and MSRC-21, show that Active MAP inference achieves similar levels of accuracy but with major efficiency gains.

In this paper, we present an object recognition and pose estimation framework consisting of a novel global object descriptor, so called Viewpoint oriented Color-Shape Histogram (VCSH), which combines object's color and shape information. During the phase of object modeling and feature extraction, the whole object's color point cloud model is built by registration from multi-view color point clouds. VCSH is trained using partial-view object color point clouds generated from different synthetic viewpoints. During the recognition phase, the object is identified and the closest viewpoint is extracted using the built feature database and object's features from real scene. The estimated closest viewpoint provides a good initialization for object pose estimation optimization using the iterative closest point strategy. Finally, objects in real scene are recognized and their accurate poses are retrieved. A set of experiments is realized where our proposed approach is proven to outperform other existing methods by guaranteeing highly accurate object recognition, fast and accurate pose estimation as well as exhibiting the capability of dealing with environmental illumination changes.

This paper presents an approach to combine automatic semantic place labeling of robot-generated maps with reasoning on human route descriptions. Enabling robots to understand human route descriptions can simplify HRI situations in household or industrial settings. However, solving this problem requires handling the ambiguity present in route descriptions and the possible unreliability of the semantic perception capabilities of the robot. We address this problem by absorbing these uncertainties in a probability distribution measuring the likelihood of the different interpretations (paths) of a given route description and selecting its MAP solution. The approach is evaluated on a dataset of route descriptions transcribed into a suitable representation using standard information retrieval metrics. These performance measurements indicate that the method can correctly interpret route descriptions even in challenging environments.

Body movements communicate affective expressions and, in recent years, computational models have been developed to recognize affective expressions from body movements or to generate movements for virtual agents or robots which convey affective expressions. This survey summarizes the state of the art on automatic recognition and generation of such movements. For both automatic recognition and generation, important aspects such as the movements analyzed, the affective state representation used, and the use of notation systems is discussed. The survey concludes with an outline of open problems and directions for future work.

The performance of vision-based control is usually limited by the low sampling rate of the visual feedback. We address Ping-Pong robotics as a widely studied example which requires high-speed vision for highly dynamic motion control. In order to detect a flying ball accurately and robustly, a multi-threshold segmentation algorithm is applied in a stereo-vision running at 150Hz. Based on the estimated 3D ball positions, a novel two-phase trajectory prediction is exploited to determine the hitting position. Benefiting from the high-speed visual feedback, the hitting position and thus the motion planning of the manipulator are updated iteratively with decreasing error. Experiments are conducted on a 7 degrees of freedom humanoid robot arm. A successful Ping-Pong playing between the robot arm and human is achieved with a high successful rate of 88%.

This video presents a motion planning method enabling an autonomous mobile robot to approach a person to initiate a conversation proactively. The proposed concept implements social aspects to give motions a more human-like appearance. This is intended since experiments discussed in literature have shown that it is easier for people to predict and read the purpose of human-like movements. User study results show that the readability of the approach behavior is improved by including parameters such as path execution speed, distance between goal pose and person, position and orientation in front of the person, and trajectory shape. The video depicts the results of an implemented human approach planner for dynamic environments. It plans a trajectory towards a person applying the named set of social constraints. Simulations and real world experiments show how the approaching behavior smoothly brings the robot close to a person for interaction.

This paper describes a new methodological approach and robot system to trigger more prosocial human reactions towards a robot by transferring social-psychological principles from human-human interaction to human-robot interaction (HRI). The main idea is to trigger increased helpfulness by proactively creating similarity through dynamic emotional adaption of the robot to the mood of the human. This is achieved in an explicit and implicit way: Explicitly, by a similarity-statement of the robot of being in the same mood as the user, and implicitly by controlling the affective parameters of facial and verbal expressions of a robot head in an interaction scenario such that the current values of the human mood in the dimensions of pleasure, arousal, and dominance (PAD) are matched. In a first step, this is accomplished by an initial self-assessment by the human participant to be extended by automatic emotion recognition modules in a later stage. The effectiveness of the approach is confirmed by significant experimental results.

In comparison to industrial settings with structured environments, the operation of autonomous robots in unstructured and uncertain environments is more challenging. This video presents a generic control and system architecture ARCADE, applicable for real-time robot control in complex task situations. Several methods to cope with uncertainties are demonstrated with the example task of changing tires on a car. Approaches of object detection (applied to car, tires, and humans), robust real-time control of robot arms under perception uncertainty, and human-friendly haptic interaction are detailed. The video shows two robots jointly performing the task of mounting a mock-up tire to a real car using the proposed methods, realizing robust performance in an uncertain environment.

Visual perception capabilities are still highly unreliable in unconstrained settings, and solutions might not be accurate in all regions of an image. Awareness of the uncertainty of perception is a fundamental requirement for proper high level decision making in a robotic system. Yet, the uncertainty measure is often sacrificed to account for dependencies between object/region classifiers. This is the case of Conditional Random Fields (CRFs), the success of which stems from their ability to infer the most likely world configuration, but they do not directly allow to estimate the uncertainty of the solution. In this paper, we consider the setting of assigning semantic labels to the pixels of an image sequence. Instead of using a CRF, we employ a Perturb-and-MAP Random Field, a recently introduced probabilistic model that allows performing fast approximate sampling from its probability density function. This allows to effectively compute the uncertainty of the solution, indicating the reliability of the most likely labeling in each region of the image. We report results on the CamVid dataset, a standard benchmark for semantic labeling of urban image sequences. In our experiments, we show the benefits of exploiting the uncertainty by putting more computational effort on the regions of the image that are less reliable, and use more efficient techniques for other regions, showing little decrease of performance.

A novel concept for visual configuration estimation of a continuum robot is presented, using solely an array of miniature cameras mounted to the surface of the robot as only input sensors. A two-step algorithm is implemented, consisting of a configuration estimation refinement step and a full Bundle Adjustment step, where 3D coordinates and robot configuration parameters are estimated optimally and simultaneously. Real-time capability is provided by fast feature matching procedures and a special solver for the sparse Bundle Adjustment problem. Furthermore, stochastically optimal estimates are guaranteed as the robot parameters are directly estimated and a detour via an individual camera pose estimation is spared. Finally, experimental results are presented to confirm the performance speed and accuracy of the proposed approach.

In this paper, a vision-guided autonomous quadrotor in an air-ground multi-robot system has been proposed. This quadrotor is equipped with a monocular camera, IMUs and a flight computer, which enables autonomous flights. Two complementary pose/motion estimation methods, respectively marker-based and optical-flow-based, are developed by considering different altitudes in a flight. To achieve smooth take-off, stable tracking and safe landing with respect to a moving ground robot and desired trajectories, appropriate controllers are designed. Additionally, data synchronization and time delay compensation are applied to improve the system performance. Real-time experiments are conducted in both indoor and outdoor environments.

In order to speed up image processing in visual servoing, the distributed computational power across networks and appropriate data transmission mechanisms are of particular interest. In this paper, a high sampling rate of visual feedback is achieved by distributed computation on a cloud image processing platform. For target tracking with a networked visual servo control system, a switching control law considering the varying feedback delay caused by image processing and data transmission is applied to improve the control performance. A sending rate scheduling strategy aiming at saving the network load is proposed based on the tracking error. Experiments on a 7 degree-of-freedom (DoF) manipulator are carried out to validate the proposed approach. The proposed approach shows a similar control performance as a system without sending rate scheduling, however, beneficially with largely reduced network load.

This paper implements a real-time hand gesture recognition algorithm based on the inexpensive Kinect sensor. The use of a depth sensor allows for complex 3D gestures where the system is robust to disturbing objects or persons in the background. A Haarlet-based hand gesture recognition system is implemented to detect hand gestures in any orientation, and more in particular pointing gestures while extracting the 3D pointing direction. The system is integrated on an interactive robot (based on ROS), allowing for real-time hand gesture interaction with the robot. Pointing gestures are translated into goals for the robot, telling him where to go. A demo scenario is presented where the robot looks for persons to interact with, asks for directions, and then detects a 3D pointing direction. The robot then explores his vicinity in the given direction and looks for a new person to interact with.

In this paper, the impact of facial expressions on HRI is explored. To determine their influence on empathy of a human towards a robot and perceived subjective performance, an experimental setup is created, in which participants engage in a dialog with the robot head EDDIE. The web-based gaming application “Akinator” serves as a backbone for the dialog structure. In this game, the robot tries to guess a thought-of person chosen by the human by asking various questions about the person. In our experimental evaluation, the robot reacts in various ways to the human's facial expressions, either ignoring them, mirroring them, or displaying its own facial expression based on a psychological model for social awareness. In which way this robot behavior influences human perception of the interaction is investigated by a questionnaire. Our results support the hypothesis that the robot behavior during interaction heavily influences the extent of empathy by a human towards a robot and perceived subjective task-performance, with the adaptive modes clearly leading compared to the non-adaptive mode.

Abstract — In this paper, an approach is presented that allows a robot to navigate in an urban environment by following natural language route instructions. In this situation, neither maps nor GPS information are available to the robot thus it has to rely solely on the human-given route description and the observations from its sensors. An architecture for solving problems such as navigation on the sidewalk, street direction inference, and environment labeling that arise in this situation is presented. Our initial experiments indicate that the proposed methods enable a robot to safely navigate in urban environments by following abstract route descriptions and reach previously unknown points in a city. I.

Every day human communication relies on a large number of different communication mechanisms like spoken language, facial expressions, body pose and gestures, allowing humans to pass large amounts of information in short time. In contrast, traditional human-machine communication is often unintuitive and requires specifically trained personal. In this paper, we present a real-time capable framework that recognizes traditional visual human communication signals in order to establish a more intuitive human-machine interaction. Humans rely on the interaction partner’s face for identification, which helps them to adapt to the interaction partner and utilize context information. Head gestures (head nodding and head shaking) are a convenient way to show agreement or disagreement. Facial expressions give evidence about the interaction partners’ emotional state and hand gestures are a fast way of passing simple commands. The recognition of all interaction queues is performed in parallel, enabled by a shared memory implementation.

This paper reports about a human-robot interaction field trial conducted with the autonomous mobile robot ACE (Autonomous City Explorer) in a public place, where the ACE robot needs the support of human passers-by to find its way to a target location. Since the robot does not possess any prior map knowledge or GPS support, it has to acquire missing information through interaction with humans. The robot thus has to initiate communication by asking for the way, and retrieves information from passers-by showing the way by gestures (pointing) and marking goal positions on a still image on the touch screen of the robot. The aims of the field trial where threefold: (1) Investigating the aptitude of the navigation architecture, (2) Evaluating the intuitiveness of the interaction concept for the passers-by, (3) Assessing people's willingness to support the ACE robot in its task, i.e. assessing the social acceptability. The field trial demonstrates that the architecture enables successful autonomous path finding without any prior map knowledge just by route directions given by passers-by. An additional street survey and observational data moreover attests the intuitiveness of the interaction paradigm and the high acceptability of the ACE robot in the public place.

In this paper a new approach for robot control using position based visual servoing (PBVS) with an omnidirectional multi-camera system is presented. PBVS requires the explicit calculation of the position and orientation of the robot tool. Given only images without depth information, either additional geometric properties of the observed scene or stereo correspondences have to be provided for pose estimation. In this paper we use triangulation of images given by many cameras pointing in various directions. We confirm this approach with some simulational results.

In this paper, a networked visual servo control system with distributed computation is proposed to overcome the low sampling rate problem in vision-based control systems. A real-time image data transmission protocol based on Realtime Transport Protocol (RTP) is developed. The captured images are sent to different processing nodes connected over a communication network and processed in parallel. Thus, a high sampling rate of the visual feedback is achieved under a cloud image processing architecture. The varying image processing delay caused by the varying number of extracted features and the random transmission delay are modeled as a random process with Bernoulli distribution. By using the input-delay approach, the resulted networked visual servo control system is reformulated into a stochastic continuous-time system with time-varying delay. Experiments on two 1-DoF linear motor modules are carried out to validate the proposed approach. A visual servo control system without parallel distributed computation is implemented for comparison. The experimental results demonstrate significant performance improvement by the proposed approach.

In this paper, a novel switching controller is proposed for a networked visual servo control system with varying feedback delay due to image processing and data transmission. The varying image processing delay caused by the varying number of extracted features for pose estimation due to different view angles, illumination conditions and noise, is modeled by its occurrence probability. The time delay due to transmission over the communication network is also modeled as random process. By using a sampled-data system approach and an input-delay approach, the linearized visual servo control system is reformulated into a stochastic continuous-time system with time-varying delay. A novel stability condition and associated switching controller are derived based on the occurrence probabilities of delays. Experiments on a 1-DoF linear module equipped with a camera are conducted to validate the proposed approach. A non-switching controller approach is implemented for comparison. The experimental results demonstrate significant performance improvement of the proposed control approach.

The stability and performance of visual servo control systems strongly depend on the delays caused by image processing. In order to accelerate the visual feedback, the distributed computational power across networks and appropriate data transmission mechanism are of particular interest. In this paper, a novel distributed computation with data scheduling is proposed for networked visual servo control systems (NVSCSs) aiming at improving the control performance. A realtime transport protocol is developed for image data transmission. For a NVSCS which is modeled as a continuous-time system with computation, transmission and holding delays, a switching control law is applied. A probabilistic sampling scheduler is derived such that the control performance and the network load caused by image data transmission are balanced. Experiments on two 1-DoF linear modules equipped with a camera are conducted to validate the proposed approach. A visual servo system without data scheduling is implemented for comparison. The experimental results demonstrate a comparable control performance of the proposed approach with an advantage of reduced network load.

Integration of emotions enhances naturalness of human-robot interaction (HRI). This requires that the robot is equipped with hardware to express emotions. Believability and recognition of expressions is increased if the same emotional state is expressed in all modalities which the robot is capable of. Within this aspect, our work analyzes if a walking robot can express emotions in the way it walks and if these expressions are recognizable. The emotive gait patterns are derived from human characteristics for emotive gait. The parameters step length, height and time for a single step vary depending on the emotion. Mapping these changes to the kinematics of the robot and exaggeration of the walking styles leads to distinguishable expressions for the dimensions pleasure, arousal and dominance. Experimental results on a hexapod show that differences in arousal are best expressed and thus recognized. Comparing an animation of the hexapod with the real robot indicates that the robot is perceived as slightly more pleasant and active. This study shows that by changing its walking style the hexapod expresses emotions, in particular differences in arousal, which can be used to increase expressiveness in HRI.

We introduce a facial mimicry system, which combines facial expression analysis and synthesis on a robot, utilizing the facial action coding system. The activation of action units on a user's face is automatically extracted from a video stream and mapped to the robot, thus mirroring the facial expression. As a novel approach, a user study quantifies the congruence of the initial human facial expression with the robotic facial expression. The evaluation shows that the robotic facial expression is perceived to be close to the human facial expression, from which it is derived. This is a fundamental aspect for a mimicry system, providing a basis for future research on empathy and emotional closed loop control.

In this paper, autonomous switching between two basic attention selection mechanisms, i.e., top-down and bottom-up, is proposed. This approach fills a gap in object search using conventional top-down biased bottom-up attention selection, which fails, if a group of objects is searched whose appearances cannot be uniquely described by low-level features used in bottom-up computational models. Three internal robot states, such as observing, operating, and exploring, are included to determine the visual selection behavior. A vision-guided mobile robot equipped with an active stereo camera is used to demonstrate our strategy and evaluate the performance experimentally. This approach facilitates adaptations of visual behavior to different internal robot states and benefits further development toward cognitive visual perception in the robotics domain.

To provide a means for recognition of affect from a distance, this paper analyzes the capability of gait to reveal a person's affective state. We address interindividual versus person-dependent recognition, recognition based on discrete affective states versus recognition based on affective dimensions, and efficient feature extraction with respect to affect. Principal component analysis (PCA), kernel PCA, linear discriminant analysis, and general discriminant analysis are compared to either reduce temporal information in gait or extract relevant features for classification. Although expression of affect in gait is covered by the primary task of locomotion, person-dependent recognition of motion capture data reaches 95% accuracy based on the observation of a single stride. In particular, different levels of arousal and dominance are suitable for being recognized in gait. It is concluded that gait can be used as an additional modality for the recognition of affect. Application scenarios include monitoring in high-security areas, human-robot interaction, and cognitive home environments.

This video presents the Autonomous City Explorer (ACE) project. Its goal was to create a robot capable of navigating unknown urban environments without the use of GPS data or prior map knowledge. The robot had to find its way solely by interacting with pedestrians and building a topological representation of its surroundings. This video outlines the necessary ingredients for successful low-level navigation on sidewalks, information retrieval from pedestrians as well as the construction of a semantic representation of an urban environment. A system architecture for outdoor localization, traversability assessment, path planning, behavior selection and topological abstraction in urban environments is presented.

In the autonomous city explorer (ACE) project a mobile robot is developed, which is capable of finding its way to a given destination in an unknown urban environment. An exemplary mission is to find the way from our institute to the Marienplatz, a public place in the center of Munich, without any prior knowledge or GPS information. Inspired by the behavior of humans in unknown environments, ACE must find its way by asking pedestrians. The route is about 1.5 kilometers far and includes heavily traveled roads and crowded public places. In order to navigate safely in an unknown urban environment, some challenges arise for the vision system. Robust human detection, tracking and the estimation of human body poses is essential for natural interaction with pedestrians. Furthermore, the robot needs to be able to detect sidewalk and crossroads. A visual odometry system is used to support the conventional navigation. Outdoor experiments were conducted twice successfully. After about 5 hours and interacting with 25 and 38 persons respectively, ACE arrived the Marienplatz. This paper describes both, an architecture of the vision system used for ACE and the algorithms used to deal with the described challenges.

In this paper, an insect-inspired motion detector (Reichardt-model) is applied to visual servo control to ensure the stability of the system with high gain and time delay in its feedback. A Reichardt-based control scheme is compared with a conventional visual servoing approach. As a consequence of the specific velocity dependence of the Reichardt-model, the stability margin of the visual servo control is increased and high overall gains, thus, better performance are achievable. The response of the Reichardt-model in the experiment and the control performance of velocity control approach with the Reichardt-model in the closed loop are investigated. The velocity control model is tested on a 1-DOF linear motor module with different feedback gain and different time delay in the loop. The results of simulation and realtime experiments demonstrate the stabilizing character of the Reichardt-based approach.

In this paper a novel implementation of the saliency map model on a multi-GPU platform using CUDA technology is presented. The saliency map model is a well-known computational model for bottom-up attention selection and serves as a basis of many attention control strategies of cognitive vision systems. A real-time implementation is the prerequisite of an application of bottom-up attention on mobile robots and vehicles. Parallel computation on graphics processing unit (GPU) provides an excellent solution for this kind of compute-intensive image processing. Running on 1 to 4 NVIDIA GeForce 8800 (GTX) graphics cards a frame rate of 313 fps at resolution of 640 times 480 is achieved, which is approximately 8.5 times faster than the standard implementations on CPUs. The implementation is also evaluated using a high-speed camera at 200 Hz. Using two GPUs only 2 ms extra computational time for the saliency map generation in addition to the camera capture time is required for images of 640 times 480 pixels.

A biologically inspired foveated attention system in an object detection scenario is proposed. Thereby, a high-performance active multi-focal camera system imitates visual behaviors such as scan, saccade and fixation. Bottom-up attention uses wide-angle stereo data to select a sequence of fixation points in the peripheral field of view. Successive saccade and fixation of high foveal resolution using a telephoto camera enables high accurate object recognition. Once an object is recognized as target object, the bottom-up attention model is adapted to the current environment, using the top-down information extracted from this target object. The bottom-up attention model and the object recognition algorithm based on SIFT are implemented using CUDA technology on Graphics Processing Units (GPUs), which highly accelerates image processing. In the experimental evaluation, all the target objects were detected in different backgrounds. Evident improvements in accuracy, flexibility and efficiency are achieved.

In this paper an autonomous switching between two basic attention selection mechanisms, top-down and bottom-up, is proposed, substituting manual switching. This approach fills the gab in object search using conventional top-down biased bottom-up attention selection: the latter one fails, if a group of objects is searched whose appearances can not be uniquely described by low-level features used in bottom-up computation models. Two internal robot states, observing and operating, are included to determine the visual selection behavior. A vision guided mobile robot, equipped with an active stereo camera, is used to demonstrate our strategy and evaluate the performance experimentally.

The goal of the autonomous city explorer (ACE) is to navigate autonomously, efficiently and safely in an unpredictable and unstructured urban environment. To achieve this aim, an accurate localization is one of the preconditions. Due to the characteristics of our navigation environment, an elaborated visual odometry system is proposed to estimate the current position and orientation of the ACE platform. The existing algorithms of optical flow computation are experimentally evaluated and compared. The method based on pyramidal Lucas-Kanade algorithm with high-speed performance is selected. Based on the optical flow in 2D images, the camera ego-motion is estimated using image Jacobian matrix and least squares method. The kinematic model is set up to map the camera ego-motion to the robot motion. To eliminate systematic errors, a novel system calibration approach is proposed. Finally the odometry system is evaluated in experiments.

Emotional expressions are considered to be important for robotic and virtual agents to improve nonverbal communication in human-machine-interaction. In this paper we focus on a subset of emotional expressions, namely the smile and it's variations. The proposed concept for generating artificial smile sequences is based on the system-theoretic psychological model of smiling, which is based on the Zurich Model of Social Motivation. The model and seven different types of smiles are introduced and it is presented how to integrate this model in a virtual agent. The evaluation of the generated facial expressions shows that the seven types of smiles are distinguishable from each other and can be classified according to given categories.

This work proposes a Piecewise Linear (PL) system to model transitions of affect. Parameters of the model are identified based on a psychological experiment. The PL system describes affective reactions of humans to an external affective stimulus depending on the previous affective state. Results of the statistical analysis support that the previous affective state influences significantly the current affective state. Evaluation of the model shows that it is suitable as mathematical representation for the development of affect over time under the influence of an external stimulus. A following system-theoretic analysis of the model reveals that the PL system shows complex dynamic characteristics. It suggests that internal affective fluctuations exist.

In the project Autonomous City Explorer, an interactive robot is designed to find its way to a given destination in unknown urban environments by interacting with pedestrians. Considering applications in a human dominated environment, the robot can be sent to a destination by tracking a landmark selected by users and described by 2D image features. To achieve a natural landmark selection from the user perspective and an accurate feature tracking for a safe robot navigation, the robot preselects visual features and presents the users only the image regions providing higher tracking accuracy. Furthermore, a multi-focal camera system is used to extend the sensing range. SIFT, Harris corner and optical flow used for tracking and self-localization are compared and applied to different visual sensors. A coordination strategy is realized, in which the camera with wide field of view is used for robot orientation control and the high-resolution camera is applied for robot forward motion control. The performance is experimentally evaluated.

Knowledge about the environment is essential for humanoid and mobile robots to move and act safely. The most intuitive way to perceive information about the environment is through the vision system. However, the accuracy provided by stereo vision is insufficient for many tasks. A more accurate representation is created by a laser range-finder, which delivers no color information. This paper describes a novel approach to merge data obtained by stereo vision and laser range-finders by using a genetic ICP algorithm, which is able to register noisy point clouds with different resolutions and a small overlap. Furthermore, it describes an easy-to-use and robust method to calibrate the extrinsic parameters of two or more laser range-finders.

Inspired by the expectation-based perception of humans, a surprise-driven active vision system is proposed. This vision system not only considers spatial saliency of objects in the environment, but also investigates temporal novelty in the neighborhood. Surprise is defined as the difference of the saliency probability distributions of two consecutive input images, which is measured using Kullback-Leibler divergence. The high-speed gaze shift capability of the camera platform and the parallel computation with the aid of GPUs enable a real-time tracking of the surprising event.

Goal-directed guidance of gaze control based on coordinated task and stimulus parameters is essential for steering a mobile cognitive system efficiently and autonomously through the real world. This paper focuses on coordination mechanisms of top-down and bottom-up attentional allocation, with particular consideration of the current local environment. The top-down attention selection in the task-space and the bottom-up attention selection in the image-space is evaluated and combined using information theory. An information-based scene context classification considering scene dynamics is proposed to bias attention selection, which is the main contribution of this paper. Experiments are conducted to evaluate the performance.

In this paper, an array of biologically inspired elementary motion detectors (EMDs) is implemented on an FPGA (Field Programmable Gate Array) platform. The well-known Reichardt-type EMD, modeling the insect’s visual signal processing system, is very sensitive to motion direction and has low computational cost. A modified structure of EMD is used to detect local optical flow. Six templates of receptive fields, according to the fly’s vision system, are designed for simple ego-motion estimation. The results of several typical experiments demonstrate local detection of optical flow and simple motion estimation under specific backgrounds. The performance of the real-time implementation is sufficient to deal with a video frame rate of 350 fps at 256 x 256 pixels resolution. The execution of the motion detection algorithm and the resulting time delay is only 0.25 μs. This hardware is suited for obstacle detection, motion estimation and UAV/MAV attitude control.

Estimating the human body pose is of great interest for many tasks, such as human robot interaction, people tracking and surveillance. During the recent years, several approaches have been presented, which still have weaknesses regarding occlusions or complex scenes. In this paper, we present a novel algorithm for human body pose estimation using any three-dimensional representation of the environment, like stereo vision. The presented algorithm is able to leave out body parts and is therefore able to deal with occluded body parts. In a first step, possible humans need to be detected, e.g. by using a skin color filter. A disparity map containing depth information is computed using a stereo matching algorithm. It leads to a three-dimensional representation of the scene. Starting with the detected skin parts, our algorithm segments this point cloud into smaller clusters. The possible matches are then verified, and the body pose is estimated using a kinematic human model with 28 degrees of freedom. As our algorithm is capable of dealing with arbitrary three-dimensional representations, it can easily be adapted to use a three-dimensional laser range finder instead of a stereo camera system.

Several variations of methodological approaches are used to study the social acceptance in human-robot interaction. Due to the introduction of robots in the home, working practice and usage typically informing the design of new forms of technology are missing. Studying social acceptance in human-robot interaction thus needs new methodological concepts. We propose a so called breaching experiment with additional ethnographic observation to close this gap. To investigate the methodological concept we have been conducting a field trial on a public place. We gathered feedback using questionnaires, in order to estimate whether this method can be beneficially to evaluate social acceptance. We could show that breaching experiments can be a useful method to investigate social acceptance in the field.

Contrary to common emotion recognition techniques by face or speech analysis, physiological data are involuntary and continuously available. Thus, they allow for emotion detection even in situations without spoken words or in case of non-extreme emotions, which are more likely to occur in human-robot interaction (HRI). In this paper, we describe the results of an experiment investigating non-extreme emotional states relevant for HRI scenarios (over- and underchallenge). Those states occurred naturally during the course of a LEGO construction task by manipulating working speed. Data collected from 28 subjects were analyzed and the results of different types of discriminant analysis and nearest neighbour methods were compared. Based on two physiological modalities (HR, SCR), correct classification rates of up to 76% for seven features and 74% for only two features were achieved. Overchallenge could be discriminated very well from the other two conditions (96.4 - 85.7%), whereas underchallenge is often confused with the intermediate condition with normal working speed.

Common works on emotion expressing robots are theoretically based on a dimensional (continuous) model of emotions. Nevertheless, performance tests, which are used to evaluate the emotion expressing robots, are based on categorical (discrete) models of emotions. In this paper the use of dimensional-based tests is suggested, e.g. semantic differential approaches like the pleasure-arousal-dominance-model. By deriving the test from the theory on which the design of an object is based, the validity of the test raises significantly. Major benefits are explicit guidelines for design improvement and the possible integration of arbitrary actuated expressive features for which no common framework as, for example, the facial action coding system (FACS) exists. For illustration purposes, a comparative evaluation study of the robot EDDIE is conducted: one test is based on a categorical model and one test is based on a dimensional model of emotion. A third study based on a dimensional model demonstrates the evaluation of the influence of animal like features on the perceived emotion state.

In this paper, a control scheme that combines trajectory planning and gaze direction control for robotic exploration is presented. The objective is to calculate the gaze direction and simultaneously plan the trajectory of the robot over a given time horizon, so that localization and map estimation errors are minimized while the unknown environment is explored. Most existing approaches perform a greedy optimization for the trajectory generation only over the next time step and usually neglect the limited field of view of visual sensors and consequently the need for gaze direction control. In the proposed approach an information-based objective function is used, in order to perform multiple step planning of robot motion, which quantifies a trade-off between localization, map accuracy and exploration. Relative entropy is used as an information metric for the gaze direction control. The result is an intelligent exploring mobile robot, which produces an accurate model of the environment and can cope with very uncertain robot models and sensor measurements

As robots are gradually leaving highly structured factory environments and moving into human populated environments, they need to possess more complex cognitive abilities. Not only do they have to operate efficiently and safely in natural populated environments, but also be able to achieve higher levels of cooperation and interaction with humans. The Autonomous City Explorer (ACE) project envisions to create a robot that will autonomously navigate in an unstructured urban environment and find its way through interaction with humans. To achieve this, research results from the fields of autonomous navigation, path planning, environment modeling, and human-robot interaction are combined. In this paper a novel hardware platform is introduced, a system overview is given, the research foci of ACE are highlighted, approaches to the occurring challenges are proposed and analyzed, and finally some first results are presented.

A novel multi-focal four camera vision system is presented. The dynamic performance reaches velocities and accelerations exceeding far beyond human capabilities. Extreme wide-angle aperture angles to narrow foveated are provided simultaneously and are independently controllable. The vision system is designed in the context of the new development of the successor model of the humanoid robot Johnnie targeting at real-world wide distance range applications. This paper presents the design, kinematic and perception models, and mechatronical integration. The dynamic performance is evaluated in experiments. A novel multi-camera view direction stabilization strategy is presented and experimentally validated

A novel camera view direction stabilization strategy based on a multi-camera vision system is proposed. Relative motion information of an observed target with respect to the camera system is acquired by at least two cameras: a tracking and a stabilizing camera. The stabilizing camera provides a velocity contribution to the tracking camera motion controlled by visual servoing. Thereby, a significant reduction of the tracking error is achieved. The proposed strategy is evaluated in simulations and experiments based on a novel multi-focal vision system designed for the humanoid LOLA

This paper focuses on the development of EDDIE, a flexible low-cost emotion-display with 23 degrees of freedom. Actuators are assigned to particular action units of the facial action coding system (FACS). Emotion states represented by the circumplex model of affect are mapped to individual action units. Thereby, continuous, dynamic, and realistic emotion state transitions are achieved. EDDIE is largely developed and manufactured in a rapid-prototyping process. Miniature off-the-shelf mechatronics components are used providing high functionality at low-cost. Evaluations conducted in a user-study show that emotions can be recognized very well. Further experiments show that additional features adapted from animals have significant but small influence on the display of the human emotion 'disgust'

EDDIE, a novel mechatronical emotion-display designed for dynamic non-verbal human-robot interaction is presented. A special feature are dynamic and realistic emotional state transitions. Therefore, the emotional state-space based on the circumplex model of affect is directly mapped to joint space. The display is largely developed and manufactured in a rapid-prototyping process. Only miniature off-the-shelf mechatronic components are used providing high functionality at low cost.

This paper focuses on the development of EDDIE, a flexible low-cost emotion-display with 23 degrees of freedom. Actuators are assigned to particular action units of the facial action coding system (FACS). Emotion states represented by the circumplex model of affect are mapped to individual action units. Thereby, continuous, dynamic, and realistic emotion state transitions are achieved. EDDIE is largely developed and manufactured in a rapid-prototyping process. Miniature off-the-shelf mechatronics are used providing high functionality while extremely low-cost. Evaluations are conducted based on a user-study

Visual servoing has become an important and powerful tool in robot control. However, the operating range is restricted by the properties of the vision sensor. Thus, in a wide range visual servoing workspace the utilization of one sensor with fixed properties is not advisable. In this work the impact of focal length and operating distance on visual servoing performance in presence of noise and quantization is investigated. The impact of sensor selection for different distances with respect to control performance is revealed. A multifocal visual servoing strategy is presented.