Geometric Deep Learning is a modern approach to deep learning that focuses on the assessment of the structure and inherent symmetry of the data. How to best represent Electrocardiograms (ECG) data is unclear. The single-lead ECG can be interpreted as a shift-invariant one-dimensional grid as common for time series. However, the leads' spatial relationships in multi-lead ECGs are non-Euclidean due to the spread of electrical activation along the heart tissue. In the domain of Electroencephalograms (EEG), Graph Neural Networkss are being adopted to represent channel interactions. This method is transferred to ECGs as well, calling for a systematic literature review to compare approaches and identify leading directions and research gaps. Consequently, we conducted this review about Geometric Deep Learning-based approaches to ECG data. However, since all but two approaches used Graph Neural Networkss, we focus more heavily on them. Our results suggest missing diversity in applications for 12-lead ECG. It is mainly used for the recognition of arrhythmias, where single- or two-lead approaches already showed superb performance. It is unclear whether multi-lead spatial modeling would significantly improve classification for this use case. While a variety of approaches from EEG analysis have swept over to ECG, there is no consensus on topology, feature set or network architecture. In consequence, we propose two directions of future work. A more application-driven branch to compare existing approaches and expand on their explainability, and a more fundamental branch to focus on theoretical insights and experiment with simulation data to integrate physiological and anatomical information.

This research addresses the question, which characteristics a cognitive architecture must have to leverage the benefits of natural language in Co-constructive Task Learning (CCTL). To provide context, we first discuss Interactive Task Learning (ITL), the mechanisms of the human memory system, and the significance of natural language and multi-modality. Next, we examine the current state of cognitive architectures, analyzing their capabilities to inform a concept of CCTL grounded in multiple sources. We then integrate insights from various research domains to develop a unified framework. Finally, we conclude by identifying the remaining challenges and requirements necessary to achieve CCTL in Human-Robot Interaction (HRI).

Current XAI research shows that explanations of AI need to be tailored to the individual explainee. We investigate whether XAI explanations can be successfully adapted to humans, when based on an appropriate static partner model representing relevant features. More specifically, we analyze the effects of static socio-demographic and individual factors on the advice-taking of different explanation strategies in a human-agent-interaction scenario. Results showed significant effects of the participant’s risk value, the mathematical self-assessment and the distance and direction of the advice to the first selection on advice-taking. Leveraging these results for an adaptation scheme, we train a classifier to predict a suitable explanation strategy based on all static features. We compare this classifier to results from a classifier working on dynamic features. Our results show that dynamic factors are as important as the static ones. Using static and dynamic factors increased the classifier’s accuracy, but the model showed overfitting and no generalization. Dividing the dataset by nationality yielded better generalization performance, indicating that nationality has an effect on predicting advice-taking. In addition, we propose an adapted measurement for advice-taking that considers the adaption beyond the given advice in advice-taking.

Virtual reality is a powerful tool for industrial applications. The article at hand addresses designers of industrial virtual environments. It summarizes key aspects to design immersive and coherent virtual environments. Furthermore, relevant influencing factors for a high quality virtual environment and tools to quantify this quality are presented. So far, a methodology to design, evaluate, and transfer knowledge from virtual environments into reality has been missing and is of high value for industrial applications. The proposed methodical approach includes the steps application analysis, technology selection and integration, design of virtual environment, evaluation of simulator quality, as well as discussion of the real-world validity. The method is shown on the example of a virtual human-robot working cell used to analyze the human perception of robot behavior during mutual assembly processes. The quality of the virtual environment is evaluated to be adequate for those purposes and the transfer of knowledge gained in virtuality on a corresponding real-world application is discussed. To the best of our knowing a system like the presented one, including full-body tracking, finger tracking, a virtual avatar and a head-mounted display has not been used for industrial use cases and human-robot cooperation before.

Intelligent Interactive Systems work well in well defined contexts. Therefore, current research focuses on developing specialised systems for specific tasks. However, real life situations violate these restrictions. Even with a specialised task focus Intelligent Interactive Systems still need to be able to react in a meaningful way in real world situations. In this paper, we propose a set of context factors to determine context and a coarse dependency structure for predicting behaviour. We argue that with such a limited set of context factors it will be possible to model context-specific behaviours for different contexts. Also, it can serve as a basis to determine the complexity of a situation.

In this paper we present an interaction model for incremental information presentation for situated human-agent assistive systems, which support a user in daily activities such as packing a bag or fetching ingredients for a cake or a menu. In a smart home interaction scenario, we provide a first realization as a proof of concept for our approach.

This article discusses the relevance of the motion behavior and adaptation of a collaborative robot for human-robot cooperation. Two experiments on cooperative assembly are shown. First, a human-human experiment with defined test conditions evaluates the aspects of distance, nearest body part, and predictability as significant. Second, a human-robot experiment shows that fixed trajectories and conservative dynamic parameters lead to a quick gain of confidence of the participants. Besides, the data shows that a realistic use case with complex tasks is key to evaluate the impact of motion parameters.

Interactive storytelling is a social situation that places a number of demands on a system when realized by an artificial agent. It can be used as a means of teaching or entertainment by a social robot or agent. To be successful, the storytelling has to be interesting and responsive. An agent needs to be aware of the user's state and coordinate the course of the story with the user input. Thus, this is an attractive scenario for the examination of HAI topics by user studies. We implemented an interactive storytelling system using the anthropomorphic robot head Flobi that presents the story with multimodal output and, at the same time, is responsive to multimodal input. The system is designed to serve as a basis for future experiments.

Many studies emphasize the importance of infant-directed speech: stronger articulated, higher-quality speech helps infants to better distinguish different speech sounds. This effect has been widely investigated in terms of the infant's perceptual capabilities, but few studies examined whether infant-directed speech has an effect on articulatory learning. In earlier studies, we developed a model that learns articulatory control for a 3D vocal tract model via goal babbling. Exploration is organized in the space of outcomes. This so called goal space is generated from a set of ambient speech sounds. Similarly to how speech from the environment shapes infant's speech perception, the data from which the goal space is learned shapes the later learning process: it determines which sounds the model is able to discriminate, and thus, which sounds it can eventually learn to produce. We investigate how speech sound quality in early learning affects the model's capability to learn new vowel sounds. The model is trained either on hyperarticulated (tense) or on hypoarticulated (lax) vowels. Then we retrain the model with vowels from the other set. Results show that new vowels can be acquired although they were not included in early learning. There is, however, an effect of learning order, showing that models first trained on the stronger articulated tense vowels easier accommodate to new vowel sounds later on.

In this paper we present results of an exploratory experiment investigating the effects of a contingently self-interrupting vs non-self-interrupting virtual agent who transmits information to a human interaction partner. In the experimental condition self-interruptions of the agent were triggered by an external event whereas in the control group the agent did not react to this event. We measured the effect of the agent's self-interruptions on human attention, memory performance and subjective ratings. In this paper we discuss the results with respect to the design of incremental human-agent dialogue modeling.

In this paper, we present our humanoid robot "Meka", participating in a multi party human robot dialogue scenario. Active arbitration of the robot's attention based on multi-modal stimuli is utilised to observe persons which are outside of the robots field of view. We investigate the impact of this attention management and addressee recognition on the robot's capability to distinguish utterances directed at it from communication between humans. Based on the results of a user study, we show that mutual gaze at the end of an utterance, as a means of yielding a turn, is a substantial cue for addressee recognition. Verification of a speaker through the detection of lip movements can be used to further increase precision. Furthermore, we show that even a rather simplistic fusion of gaze and lip movement cues allows a considerable enhancement in addressee estimation, and can be altered to adapt to the requirements of a particular scenario.

The first workshop on embodied interaction with smart environments aims to bring together the very active community of multi-modal interaction research and the rapidly evolving field of smart home technologies. Besides addressing the software architecture of such very complex systems, it puts an emphasis on questions regarding an intuitive interaction with the environment. Thereby, especially the role of agency leads to interesting challenges in the light of user interactions. We therefore encourage a lively discussion on the design and concepts of social robots and virtual avatars as well as innovative ambient devices and their implementation into smart environments.

Semantic grammars can be applied both as a language model for a speech recognizer and for semantic parsing, e.g. in order to map the output of a speech recognizer into formal meaning representations. Semantic speech recognition grammars are, however, typically created manually or learned in a supervised fashion, requiring extensive manual effort in both cases. Aiming to reduce this effort, in this paper we investigate the induction of semantic speech recognition grammars under weak supervision. We present empirical results, indicating that the induced grammars support semantic parsing of speech with a rather low loss in performance when compared to parsing of input without recognition errors. Further, we show improved parsing performance compared to applying n-gram models as language models and demonstrate how our semantic speech recognition grammars can be enhanced by weights based on occurrence frequencies, yielding an improvement in parsing performance over applying unweighted grammars.

Based on different psychological models of emotion, we argue that an intrapersonal account of emotion is not sufficient. Rather, we need interpersonal accounts of emotion that go beyond the assumption that a communicative agent simply displays her internal affective state and takes situational aspects into account. In this paper a computational model for the application of natural emotions in HRI is presented. Having a robot which is able to mimic emotional expressions and also use emotions in a strategic manner will enhance its emotional and social competence.

No abstract available.

Robots are not human. They might in some cases have a similar appearance but different behavioral and cognitive strengths and limitations. In this sense, an interaction with a robot is asymmetric. When interacting with a robot one is unsure what behavior to expect as the appearance does not necessarily make the abilities of the robot transparent. In human-human interaction, we can also find asymmetric interactions to occur. For example, in an interaction with a child, adults have to adapt to the learner's capabilities and understanding. Similarly, in interactions with special populations such as persons with autistic spectrum disorders (ASD), asymmetry occurs as specific information seems to be processed differently.

The Visual Focus of Attention (what or whom a person is looking at) or VFOA is a fundamental cue in non-verbal communication and plays an important role when designing effective human-machine interaction systems. However, recognizing the VFOA of an interacting person is difficult for a robot, since due to low resolution imaging, eye gaze estimation is not possible. Rather, head pose cue is used as a substitute for gaze, but leads to ambiguities in its interpretation as VFOA indicator. In this paper, we investigate the use of the robot conversational state, which the robot is aware of, as contextual information to improve VFOA recognition from head pose. We propose a dynamic Bayesian model that accounts for the robot state (speaking status, person he addresses, reference to objects) along with a dynamic head-to-gaze mapping function. Experiments on a publicly available human-robot interaction dataset, where a humanoid robot plays the role of an art guide and quiz master, shows that using such conversational context is effective in improving VFOA.

We show how incompressibility, a well-described property of some prosodic timing effects, can be accounted for in an optimization-based model of speech timing.Preliminary results of a corpus study are presented, replicating and generalizing previous findings on incompressibility as a function of increasing speaking rate.We then introduce the architecture of our model and present results of simulation experiments that reproduce the results of the corpus analysis.Results suggest that incompressibility can be interpreted as a consequence of tradeoffs between competing requirements of production efficiency and communicative efficacy.

Social robots are often applied in recreational contexts to improve the experience of using technical systems, but they are also increasingly used for therapeutic purposes. In this study, we compared how patients with Autism Spectrum Disorder (ASD) interact with a social robot and a human actor. We examined the gaze behavior of nine ASD patients and 15 matched controls using a mobile eye-tracker. Participants performed a task in which they were required to follow the gaze of a robot or human actor. Our results show that ASD patients preferentially maintain eye contact during interaction with the social robot as compared to the human actor.

Human learning strongly depends on the ability to structure the actions of teachers in order to identify relevant parts. We propose that this is also true for learning in robots. Therefore, we apply a method for multimodal action segmentation called Acoustic Packaging to a corpus of pairs of users teaching object names to a robot. Going beyond previous use cases, we analyze how the structure of human actions changes if the robot is learning quickly or slowly. Our results reveal differences between action structuring in the conditions such as longer utterances and more motion when the robot learns slowly. We also evaluate how the partners in the pair influence each other's action structuring. The results show a strong correlation between the participants in the pairs, even more so in the trials where the robot is learning slowly. We conclude that the action structuring based on Acoustic Packaging allows robots to differentiate how well the interaction with multiple users is going and is, thus, a vehicle for feedback generation.

In several research areas, e.g. in the field of human-robot interaction, ratings or questionnaires are applied using offline and online methods. An argument for the use of online methods is the efficiency. By using the Internet, data can be collected much faster than in an offline experiment and the administration effort is very low. The goal of our study was to find out, if there is a difference in accuracy between an online and an offline rating task of human and robot emotional facial expressions. Results indicate, that emotional expressions are best recognized in humans (versus robots) and in the offline (versus online) condition. Furthermore, the influence of the emotional category on the accuracy rate varies between conditions. Therefore, we discuss environmental factors of online experiments that are difficult to control as main reasons for these results. We conclude that online rating studies should always be combined with more reliable offline evaluations.

We present the design process, realization and evaluation of a robot system for nteractive object learning. The system-oriented evaluation, in particular, addresses an open problem for the evaluation of systems, where overall user satisfaction depends not only on the performance of the parts, but also on their combination, and on user behavior. Based on the PARADISE method known from spoken dialog systems, we have defined and applied internal and external metrics for fine-grained and largely automatable identification of such relationships. Through evaluation with n=28 subjects, indicator functions explaining up to 55% of variation in several satisfaction metrics were found. Furthermore, we demonstrate that the system's interaction style reduces the need for instruction and successfully recovers partial failures.

Regarding the future usage of social robots in workplace scenarios, we addressed the question of potential mere robotic presence effects on human performance. Applying the experimental social facilitation paradigm in social robotics, we compared task performance of 106 participants on easy and complex cognitive and motoric tasks across three presence groups (alone vs. human present vs. robot present). Results revealed significant evidence for the predicted social facilitation effects for both human and robotic presence compared to an alone condition. Implications of these findings are discussed with regard to the consideration of the interaction of robotic presence and task difficulty in modeling robotic assistance systems.

This paper presents a system that uses various simple auditory and visual features to achieve human-robot dialog scene analysis. Our scene analysis system is able to learn how many speakers are in the scenario, where the speakers are and who is currently speaking. Speakers are unknown in advance. A visual short-term-memory (STM) helps to memorize persons, even if they disappear from the camera's field of view for a while due to movements of persons or the robot head. In comparison to our previous work, we apply more visual features such as height, color and texture features of different upper body parts, to improve the scene representation performance. We show that our system is able to assign words to corresponding speakers. A speaker is recognized again when he leaves and enters the scene, or changes his position even with a newly appearing person.

The paper investigates the effects of a robot's “on-line” feedback during a tutoring situation with a human tutor. Analysis is based on a study conducted with an iCub robot that autonomously generates its feedback (gaze, pointing gesture) based on the system's perception of the tutor's actions using the idea of reciprocity of actions. Sequential micro-analysis of two opposite cases reveals how the robot's behavior (responsive vs. non-responsive) pro-actively shapes the tutor's conduct and thus co-produces the way in which it is being tutored. A dialogic and a monologic tutoring style are distinguished. The first 20 seconds of an encounter are found to shape the user's perception and expectations of the system's competences and lead to a relatively stable tutoring style even if the robot's reactivity and appropriateness of feedback changes.

Human interaction is highly intuitive: we infer reactions of our opponents mainly from what we have learned in years of experience and often assume that other people have the same knowledge about certain situations, abilities, and expectations as we do. In human-robot interaction (HRI) we cannot take for granted that this is equally true since HRI is asymmetrical. In other words, robots have different abilities, knowledge, and expectations than humans. They need to react appropriately to human expectations and behaviour. With this respect, scientific advances have been made to date for applications in entertainment and service robotics that largely depend on intuitive interaction. However, HRI today is often still unnatural, slow, and unsatisfactory for the human interlocutor. Both the sensorimotor interaction with environment and interlocutor, and the social aspects of the interaction still need to be researched and improved. Therefore, this full-day workshop aims to bring together researchers from different scientific fields to discuss these crosscutting issues and to exchange views on what are the preconditions and principles of intuitive interaction.

Robots interacting with humans need to understand actions and make use of language in social interactions. Research on infant development has shown that language helps the learner to structure visual observations of action. This acoustic information typically in the form of narration overlaps with action sequences and provides infants with a bottom-up guide to find structure within them. This concept has been introduced as acoustic packaging by Hirsh-Pasek and Golinkoff. We developed and integrated a prominence detection module in our acoustic packaging system to detect semantically relevant information linguistically\nhighlighted by the tutor. Evaluation results on speech data from adult-infant interactions show a significant agreement with human raters. Furthermore a first approach based on acoustic packages which uses the prominence detection results to generate acoustic feedback is presented.\n\nIndex Terms: prominence, multimodal action segmentation,\nhuman robot interaction, feedback

Domestic tasks such as grasping or navigation for robotic systems can be supported by vision. However, the environment provides a vast amount of visual information and concentrating on the information related to the task being undertaken is an important job. Active vision is an approach that provides such a filtering mechanism by using camera movements to bring relevant information into the focus of attention. However timing of gaze shifts (i.e. when to look where) is crucial for cognitive tasks to proceed simultaneously (multitasking). We developed a framework that learns task dependent management of gaze control. We adopted a systems approach where individual visual processes were formalised as modules such as a colour saliency module or object recognition module. Modules may generate motor commands for gaze shifts to acquire visual information relevant to their operation. The system learns how to use its modules (i.e. when to give motor control access to which module) for a task in a reward-based concept. The framework was used in a reaching-while-interacting scenario using the humanoid iCub in a simulation environment.

In this paper, we present a first step towards a typology of relevant signals and cues in human-robot interaction (HRI). In human as well as in animal communication systems, signals and cues play an important role for senders and receivers of such signs. In our typology, we systematically distinguish between a robot's signals and cues which are either designed to be human-like or artificial to create meaningful information. Subsequently, developers and designers should be aware of which signs affect a user's judgements on social robots. For this reason, we first review several signals and cues that have already been successfully used in HRI with regard to our typology. Second, we discuss crucial human-like and artificial cues which have so far not been considered in the design of social robots - although they are highly likely to affect a user's judgement of social robots.

In this article, we propose a bottom-up saliency model which works on capturing the contrast between random pixels in an image. The model is explained on the basis of the stimulus bias between two given stimuli (pixel intensity values) in an image and has a minimal set of tunable parameters. The methodology does not require any training bases or priors. We followed an established experimental setting and obtained state-of-the-art-results for salient region detection on the MSR dataset. Further experiments demonstrate that our method is robust to noise and has, in comparison to six other state-of-the-art models, a consistent performance in terms of recall, precision and F-measure.

A robot's head is important both for directional sensors and, in human-directed robotics, as the single most visible interaction interface. However, designing a robot's head faces contradicting requirements when integrating powerful sensing with social expression. Furher, reactions of the general public show that current head designs often cause negative user reactions and distract from the functional capabilities. Therefore, this contribution presents a novel anthropomorphic robot head called "Flobi", which combines state-of-the-art sensing functionality with an exterior that elicits a sympathetic emotional response. It can display primary and secondary emotions in a human-like way, to enable intuitive human-robot-interaction. To facilitate further research on facial appearance, the exterior is fully modular and replaceable. While current state-of-the-art still requires trade-offs when integrating sensing and social expression, Flobi has been designed to enable service robotic applications, with high-resolution, wide-angle stereo vision, gyroscope motion compensation and stereo audio. For ease of integration, the head is self-contained, including 18 actuators, sensors and control boards, all in a human-head sized package.

We investigate incremental word learning in a Hidden Markov Model (HMM) framework suitable for human-robot interaction.In interactive learning, the tutoring time is a crucial factor.Hence our goal is to use as few training samples as possible while maintaining a good performance level.To adapt the states of the HMMs, different large-margin discriminative training strategies for increasing the separability of the classes are proposed.We also present a novel estimation of the variance floor when a very low number of training data is used.Finally our approach is successfully evaluated on isolated digits taken from the TIDIGITS database.

This paper introduces the industrial design of the social robot `Flobi'. In total, three key concepts influenced the industrial design: First, the robot head of Flobi appears as a cartoon-like character and has a `hole-free' design without any visible conjunctions. Second, Flobi has dynamic features to display not only primary emotions, but also shame, a typical secondary emotion. Third, the structural design implements exchangeable modular parts. Through modular design, the underlying hardware is quickly accessible and the visual features of the robot (e.g., hairstyle, facial features) can be altered easily. A first study demonstrated the successful implementation of Flobi's dynamic features, whereas a second study demonstrates that the exchangeable hair modules influence gender-schematic perceptions of the robot.

Research on social robots is mainly comprised of research into algorithmic problems in order to expand a robot's capabilities to improve communication with human beings. Also, a large body of research concentrates on the appearance, i.e. aesthetic form of social robots. However, only little reference to their definition is made. In this paper we argue that form, function, and context have to be taken systematically into account in order to develop a model to help us understand social robots. Therefore, we address the questions: What is a social robot, what are the interdisciplinary research aspects of social robotics, and how are these different aspects interlinked? In order to present a comprehensive and concise overview of the various aspects we present a framework for a definition towards social robots.

If robots are to succeed in novel tasks, they must be able to learn from humans. To improve such human-robot interaction, a system is presented that provides dialog structure and engages the human in an exploratory teaching scenario. Thereby, we specifically target untrained users, who are supported by mixed-initiative interaction using verbal and non-verbal modalities. We present the principles of dialog structuring based on an object learning and manipulation scenario. System development is following an interactive evaluation approach and we will present both an extensible, event-based interaction architecture to realize mixed-initiative and evaluation results based on a video-study of the system. We show that users benefit from the provided dialog structure to result in predictable and successful human-robot interaction.

In scenarios that require a close collaboration and knowledge transfer between inexperienced users and robots, the ldquolearning by interactingrdquo paradigm goes hand in hand with appropriate representations and learning methods. In this paper we discuss a mixed initiative strategy for robotic learning by interacting with a user in a joint map acquisition process. We propose the integration of an environment representation approach into our interactive learning framework. The environment representation and mapping system supports both user driven and data driven strategies for the acquisition of spatial information, so that a mixed initiative strategy for the learning process is realised. We evaluate our system with test runs according to the scenario of a guided tour, extending the area of operation from structured laboratory environment to less predictable domestic settings.

This paper investigates the influence of visual appearance of social robots on judgments about their potential applications. 183 participants rated the appropriateness of thirteen categories of applications for twelve social robots in an online study. The ratings were based on videos displaying the appearance of the robot combined with basic information about the robots' general functions. The results confirmed the hypothesis that the visual appearance of robots is a significant predictor for the estimation of applications in the eye of the beholder. Furthermore, the ratings showed an attractiveness bias: robots being judged as more attractive by the users also received more positive evaluations (i.e., ldquolikingrdquo).

Theory of Mind (ToM) is not only a key capability for cognitive development but also for successful social interaction. In order for a robot to interact successfully with a human both interaction partners need to have an adequate representation of the other's actions. In this paper we address the question of how a robot's actions are perceived and represented in a human subject interacting with the robot and how this perception is influenced by the appearance of the robot. We present the preliminary results of an fMRI-study in which participants had to play a version of the classical Prisoners' Dilemma Game (PDG) against four opponents: a human partner (HP), an anthropomorphic robot (AR), a functional robot (FR), and a computer (CP). The PDG scenario enables to implicitly measure mentalizing or Theory of Mind (ToM) abilities, a technique commonly applied in functional imaging. As the responses of each game partner were randomized unknowingly to the participants, the attribution of intention or will to an opponent (i.e. HP, AR, FR or CP) was based purely on differences in the perception of shape and embodiment.

This paper investigates the influence of feedback provided by an autonomous robot (BIRON) on users' discursive behavior. A user study is described during which users show objects to the robot. The results of the experiment indicate, that the robot's verbal feedback utterances cause the humans to adapt their own way of speaking. The changes in users' verbal behavior are due to their beliefs about the robots knowledge and abilities. In this paper they are identified and grouped. Moreover, the data implies variations in user behavior regarding gestures. Unlike speech, the robot was not able to give feedback with gestures. Due to the lack of feedback, users did not seem to have a consistent mental representation of the robot's abilities to recognize gestures. As a result, changes between different gestures are interpreted to be unconscious variations accompanying speech.

Anthropomorphism is one of the keys to understand the expectations people have about social robots. In this paper we address the question of how a robotpsilas actions are perceived and represented in a human subject interacting with the robot and how this perception is influenced only by the appearance of the robot. We present results of an interaction-study in which participants had to play a version of the classical Prisonerspsila Dilemma Game (PDG) against four opponents: a human partner (HP), an anthropomorphic robot (AR), a functional robot (FR), and a computer (CP). As the responses of each game partner were randomized unknowingly to the participants, the attribution of intention or will to an opponent (i.e. HP, AR, FR or CP) was based purely on differences in the perception of shape and embodiment. We hypothesize that the degree of human-likeness of the game partner will modulate what the people attribute to the opponents - the more human like the robot looks the more people attribute human-like qualities to the robot.

Human-robot interaction (HRI) research is here presented into social robots that have to be able to interact with inexperienced users. In the design of these robots many research findings of human-human interaction and human-computer interaction are adopted but the direct applicability of these theories is limited because a robot is different from both humans and computers. Therefore, new methods have to be developed in HRI in order to build robots that are suitable for inexperienced users. In this paper we present a video study we conducted employing our robot BIRON (Bielefeld robot companion) which is designed for use in domestic environments. Subjects watched the system during the interaction with a human and rated two different robot behaviours (extrovert and introvert). The behaviours differed regarding verbal output and person following of the robot. Aiming to improve human-robot interaction, participantspsila ratings of the behaviours were evaluated and compared.

Human-robot collaboration requires both communicative and decision making skills of a robot. To enable flexible coordination and turn-taking between human users and a robot in joint tasks, the robot's dialog and decision making mechanism have to be synchronized in a meaningful way. In this paper, we propose a integration framework to combine the dialog and the decision making processes. With this framework, we investigate various task negotiation situations for a social robot in a fetch-and-carry scenario. For the technical realization of the framework, the interface specification between the dialog and the decision making systems is also presented. Further, we discuss several challenging issues identified in our integration effort that should be adddressed in the future.

The paper introduces an online user study on applications for social robots with 127 participants. The potential users proposed 570 application scenarios based on the appearance and functionality of four robots presented (AIBO, BARTHOC, BIRON, iCat). The items were grouped into 13 categories which are interpreted and discussed by means of four dimensions: public vs. private use, intensity of interaction, complexity of interaction model, and functional vs. human-like appearance. The interpretation lead to three classes of applications for social robots according to the degree of social interaction: (1) Specialized Applications where the robot has to perform clearly defined tasks which are delegated by a user, (2) Public Applications which are directed to the communication with many users, and (3) Individual Applications with the need of a highly elaborated social model to maintain a variety of situations with few people.

An important goal for research on service robots is the cooperation of a human and a robot as team. A service robot in a domestic environment needs to build a representation of its future workspace that corresponds to the human user's understanding of these surroundings. But it also needs to apply this model about the "where" and "what" in its current interaction to allow communication about objects and places in a human-adequate way. In this paper we present the integration of a hierarchical robotic mapping system into an interactive framework controlled by a dialog system. The goal is to use interactively acquired environment models to implement a robot with interaction aware behaviors. A major contribution of this work is a three-level hierarchy of spatial representation affecting three different communication dimensions. This hierarchy is consequently applied in the design of the grounding-based dialog, laser-based topological mapping, and an objects attention system. We demonstrate the benefits of this integration for learning and tour guiding in a human- comprehensible interaction between a robot and its user in a home-tour scenario. The enhanced interaction capabilities are crucial for developing a new generation of robots that will be accepted not only as service robots but also as robot companions.

Topics in situated and task oriented communication depend heavily on the given, often changing environment, making the detection of predetermined topics in many cases useless. Detection of non-predefined topics can enhance Human-Robot-Interaction (HRI) in a variety of ways, though. In this paper we propose a way to dynamically determine topics during Human-Robot-Communication using well established techniques such as Latent Semantic Analysis (LSA). The procedure is based on multimodal cues, supporting the view that topics are not simply a property of spoken or written language, but of multimodal situated communication. An online version of the topic detection system has been developed and is currently being tested on our mobile robot BIRON. To demonstrate the feasibility of our approach, we present the results of an evaluation of our system on the BITT corpus. Index Terms: topic tracking, multimodal dialogue, human robot interaction.

Natural language user interfaces for robots with cognitive capabilities should be designed to reduce the occurrence of miscommunication in order to be perceived as providing a smooth and intuitive interaction to its users. This paper describes how miscommunication analysis is integrated in the design process. Observations from 12 user sessions revealed that users misunderstand the robot's functionality; and that feedback sometimes is ill-timed with respect to the situation. We provide a set of design implications to prevent errors from occurring, to influence or adapt to users' behavior

An ambitious goal in modern robotic science is to build mobile robots that are able to interact as companions in real world environments. Especially for caretaking of elderly people a system robustly working at private homes is essential, requiring a very natural and human oriented way of communication. Since home environments are usually very individual a first task for a newly acquired robot is to get familiar with its new environment. This paper gives a short overview on how we integrated a vision based localization using the advantages of a very modular architecture and extending a spoken dialog system for online labeling and interaction about different locations. We present results from the integrated system working in a real, fully furnished home environment where it was able to learn the names of different rooms. This system enables us to perform real user studies in future without the need to fall back to Wizard-of-Oz experiments. Ongoing work aims at enabling the robot to take initiative by asking for unknown locations. A future extension is the ability to generalize over features of known rooms to make predictions when encountering unknown rooms

One ambitious goal in current robotics research is to build robots that can interact with humans in an intuitive way and can do so outside the lab in real world situations and environments such as private homes or public places. Toy robots like, e.g., Sony's AIBO are already being sold successfully for entertainment purposes, but they usually lack sophisticated human-like interaction capabilities preventing non-expert users to instruct them for useful tasks. We have developed a robot that is capable of processing multi-modal instructions and can, therefore, be instructed interactively in a social situation. This paper gives an overview of the components of the system and their integration. The system performance is described in detail based on observations from human-robot interactions and processing times to identify critical system components and further research directions. Finally, we report on first human-robot interactions with our robot BIRON (Bielefeld Robot companiON) being situated in a real home environment. These interactions demonstrate that our robot is able to interact with users in a real home environment and can thus serve as a basis for comprehensive user studies focussing on embodied interaction for social learning

In order to model complex human robot interaction researchers not only have to consider different tasks but also to handle the complex interplay of different modules of one single robot system. In our context we constructed a robot assistant integrated in a home or office environment. We allow for a fairly natural communication style, which means that the users communicate using speech but are also allowed to use gestures and moreover to use contextual scene knowledge. Against this background, this paper presents a robust speech understanding component for situated human-robot communication. It serves as interface between speech recognition and dialog management. To increase robustness of speech processing it rates the speech recognition output by means of semantic coherence. Even if the recognized word-stream is not grammatically correct the speech understanding component provides semantic interpretations in context of multi-modal input for dialog management. For the understanding process, we designed special semantic concepts grounded to the domain of situated communication. They also provide additional information about the dialog act. A processing mechanism uses these concept units to generate the most likely semantic interpretation of the utterances

In domestic social robot systems the dialog system is often the main user interface. The verbal behavior of such a robot, therefore, plays crucial role in human-robot interaction. Comparative user studies on various verbal behaviors of a robot can effectively contribute to human-robot interaction research. In this paper we present a dialog system that can be easily configured to demonstrate different verbal, initiative-taking behaviors for a robot and, thus, can be used as a platform for such comparative user studies. The pilot study we conducted does not only provide strong evidence for this suitability, but also reveals benefits of comparative studies on a real robot in general

Creating robots with extendable social skills and interaction capabilities that suffice their operation in the real world with naive users is a very challenging task. In this paper we present a new approach using topic tracking on multi-modal dialogue to provide a mobile robot with a higher level situation awareness in human-robot interaction. The robot is no longer operating in laboratory surroundings, but in its own real world flat. We describe how our topic tracking approach is implemented in this integrated system, operating on verbal speech input. Different modalities like data from video cameras and laser scans are used as additional cues to a semantic understanding and grouping of user utterances into different topics. Both the amount of topics and the according topic names are created dynamically. Evaluating an offline speech corpus demonstrates the suitability of our approach. It is now possible to ask "BIRON, what's the topic?", making the interaction more social

In research on human-robot interaction the interest is currently shifting from uni-modal dialog systems to multi-modal interaction schemes. We present a system for human-style interaction with a robot that is integrated on our mobile robot BIRON. To model the dialog we adopt an extended grounding concept with a mechanism to handle multi-modal in- and output where object references are resolved by the interaction with an object attention system (OAS). The OAS integrates multiple input from, e.g., the object and gesture recognition systems and provides the information for a common representation. This representation can be accessed by both modules and combines symbolic verbal attributes with sensor-based features. We argue that such a representation is necessary to achieve a robust and efficient information processing.

This paper presents the humanoid robot BARTHOC who has been developed to study human-robot interaction (HRI). The main focus of BARTHOC's design was to realize the expression and behavior of the robot to be as human-like as possible. This allows to apply the platform to manifold research and demonstration areas. With his human-like look and mimic possibilities, he differs from other platforms like ASIMO or QRIO, and enables experiments even close to Mori's 'uncanny valley'. The paper describes details of the mechanical and electrical design of BARTHOC together with its PC control interface. Through its humanoid appearance, it can imitate human behavior with its soft- and hardware. Currently, several components for HRI on a mobile robot platform are being ported to BARTHOC. Starting with these components, the robot's human-like appearance enables us to study embodied interaction and to explore theories of human intelligence.

A challenging domain for dialog systems is their use for the communication with robotic assistants. In contrast to the classical use of spoken language for information retrieval, on a mobile robot multi-modal dialogs and the dynamic interaction of the robot system with its environment have to be considered. \nIn this paper we will present the dialog system developed for BIRON — the Bielefeld Robot Companion. The system is able to handle multi-modal dialogs by augmenting semantic interpretation structures derived from speech with hypotheses for additional modalities as e.g. speech-accompanying gestures. The architecture of the system is modular with the dialog manager being the central component. In order to be aware of the dynamic behavior of the robot itself, the possible states of the robot control system are integrated into the dialog model.\nFor flexible use and easy configuration the communication between the individual modules as well as the declarative specification of the dialog model are encoded in XML.\nWe will present example interactions with BIRON from the ’ scenario defined within the COGNIRON project.

We describe a method to model pronunciation variation for ASR in a data-driven way, namely by use of automatically derived acoustic subword units. The inventory of units is designed so as to produce maximal separable pronunciation variants of words while at the same time only the most important variants for the particular application are trained. In doing so, the optimal number of variants per word is determined iteratively. All this is accomplished (almost) fully automatically by use of a state splitting algorithm and a variant distance measure. Compared to a baseline system using triphones as subword units and with minimal pronunciation variants, this method achieved a relative improvement of the word error rate by 10%.

Recent interest in the automatic processing of meetings is motivated by a desire to summarize, browse, and retrieve important information from lengthy archives of spoken data. One of the most useful capabilities such a technology could provide is a way for users to locate “hot spots” or regions in which participants are highly involved in the discussion (e.g. heated arguments, points of excitement, etc.). We ask two questions about hot spots in meetings in the ICSI Meeting Recorder corpus. First, we ask whether involvement can be judged reliably by human listeners. Results show that despite the subjective nature of the task, raters show significant agreement in distinguishing involved from non-involved utterances. Second, we ask whether there is a relationship between human judgments of involvement and automatically extracted prosodic features of the associated regions. Results show that there are significant differences in both F0 and energy between involved and non-involved utterances. These findings suggest that humans do agree to some extent on the judgment of hot spots, and that acoustic-only cues could be used for automatic detection of hot spots in natural meetings.

For the modelling of speech rate variation in speech recognition many approaches have been suggested. However, the training of speech-rate dependent models has by far received most of the attention. In order to investigate problematic aspects related with the classification of the speech data which represents one of the major problems of these approaches extensive experiments were carried out on a German corpus of read speech. The results indicate that while the kind of the model-driven speech-rate measure is only of minor importance a data-driven classification of the speech data significantly improves the performance of rate-dependent models. Further results suggest a detailed modelling of speech rate based on more general models. This means that it might be possible to model speech rate adaptation by means of a transformation based on a continuous measure.

Changes in speech rate severely affect the performance of continuous speech recognition systems. In order to better understand the underlying effects of speech rate changes an analysis was carried out on the influence of duration on the spectral properties of vowels in a large German corpus of spontaneous speech. The results show a strong centralisation effect of the vowel formant frequencies due to shorter duration while the formant movements are only slightly affected. The data suggest that the movement velocity is not changed in vowels with a limited duration. As the means of the on- and offset frequencies also remain stable only the middle part of the vowels are affected by the centralisation effect. These results are discussed in the light of the modelling of varying speech rate in automatic speech recognition systems. 1.