The feedback of a robot is a powerful means to establish smooth human-robot interaction (HRI). We report on a user study to assess the applicability of a screen in a human-robot game-playing scenario. The screen was deployed to compensate for expressive shortcomings of a social robot due to its mechanical limitations of non-movable facial features. The participants played Rock-Paper-Scissors with the robot for which we programmed the right hand to show gestures. Half of the participants received facial expressions via a screen during the game, whereas the other half did not get screen feedback. We annotated the video-recorded interactions and collected questionnaire and interview data to assess the applicability of the screen. With our data we could show noteworthy results: First, the robot which provided facial expressions was rated more intelligent. Second, the participants who had interacted with the robot showing facial expressions, rated the task as more attractive than participants who did not receive facial expressions from the robot. The fact that we could not detect a negative influence of the screen, underlines the applicability of a screen in an HRI game-playing scenario to display facial expressions and contribute to an enhanced interaction.

It is the aim of this work to research how short-comings of a social robot due to its expressive limitations may be overcome by multimodal feedback. An experiment is proposed in which a robot that cannot produce facial expressions plays a game of rock, paper, scissors with people. A screen which is built-in the torso of the robot is used to compensate for these limitations in expressiveness and provide the participant with facial expressions during the game. To assess the impact of the screen on the user's rating of the robot and the interaction as such, there will be a control condition in which the screen will stay turned off. With this experiment we would like to show that sophisticated feedback setup can contribute to make a playful interaction between a human and a robot even more enjoyable.

In this paper, a novel vision system is proposed to estimate attention of people from rich visual clues for social robot to perform natural interactions with multiple participants in public environments. The vision detection and recognition modules include multi-person detection and tracking, upper-body pose recognition, face and gaze detection, lip motion analysis for speaking recognition, and facial expression recognition. A computational approach is proposed to generate a quantitative estimation of human attention. The vision system is implemented on a robotic receptionist "EVE" and encouraging results have been obtained.

This paper proposes an efficient system which integrates multiple vision models for robust multiperson detection and tracking for mobile service and social robots in public environments. The core technique is a novel maximum likelihood (ML)-based algorithm which combines the multimodel detections in mean-shift tracking. First, a likelihood probability which integrates detections and similarity to local appearance is defined. Then, an expectation-maximization (EM)-like mean-shift algorithm is derived under the ML framework. In each iteration, the E-step estimates the associations to the detections, and the M-step locates the new position according to the ML criterion. To be robust to the complex crowded scenarios for multiperson tracking, an improved sequential strategy to perform the mean-shift tracking is proposed. Under this strategy, human objects are tracked sequentially according to their priority order. To balance the efficiency and robustness for real-time performance, at each stage, the first two objects from the list of the priority order are tested, and the one with the higher score is selected. The proposed method has been successfully implemented on real-world service and social robots. The vision system integrates stereo-based and histograms-of-oriented-gradients-based human detections, occlusion reasoning, and sequential mean-shift tracking. Various examples to show the advantages and robustness of the proposed system for multiperson tracking from mobile robots are presented. Quantitative evaluations on the performance of multiperson tracking are also performed. Experimental results indicate that significant improvements have been achieved by using the proposed method.

This video shows a service robot named FusionBot autonomously serving coffees to visitors on their request, which occurred during two days-long experiment in TechFest 2008 event. The coffee serving task involves taking coffee order from a visitor, identifying a cup and smart coffee machine, moving towards the coffee machine, communicating with the coffee machine and fetching the coffee cup to the visitor.

It is believe that developing a robot without considering users feedback on appearance, functions and behavior will cause a share in the failure of research and evaluation of human robot interaction. This paper researched and solicited feedback from targeted users on their expectations and views of a robot, which will be used as a receptionist in a Singapore research centre. Based on design inputs from BMW Group DesignworksUSA, an experiment with 36 participants was conducted. The study covered areas such as perceptions of a robot, concerns of developing a robotic receptionist, expectations for a robotic receptionist, and preference in terms of overall appearance and functionalities of the intended robotic receptionist. The approach to the robotic development was based on a design methodology that took into consideration elements that would improve human robot interaction. The robotic receptionist was designed with a strong emphasis on the robot's appearance, behavior and functions that would impact the end users. The intent was to create a pleasurable experience for the end users during their encounter with the robot. This study finally led to conceptual designs based on response and selection from the participants. The robotic receptionist design had been selected and the first prototype, Olivia 2.0 has been developed for further human robot interaction studies.

In this paper we present a multimodal interface that employs speech recognition and eye gaze tracking technology for use in data entry tasks. The aim of this work is to compare the usability of this multimodal system against other data entry methods (handwriting, mouse and keyboard and speech only) when carrying out the data entry task of filling a form. Discussions regarding the relationships between efficiency, effectiveness, ergonomic quality, hedonic quality, naturalness, familiarity and users preference are presented. The experimental results show that the majority of the users prefer using the proposed eye and speech system compared to the other form-filling methods even though such a method is neither the fastest nor the most accurate.

In the work carried out earlier [1][2], it was found that an eye gaze and speech enabled interface was the most preferred form of data entry method when compared to other methods such as mouse and keyboard, handwriting and speech only. It was also found that several non-native United Kingdom (UK) English speaking speakers did not prefer the eye gaze and speech system due to the low success rate caused by the inaccuracy of the speech recognition component. Hence in order to increase the usability of the eye gaze and speech data entry system for these users, error recovery methods are required. In this paper we present three different multimodal interfaces that employ the use of speech recognition and eye gaze tracking within a virtual keypad style interface to allow for the use of error recovery (re-speak with keypad, spelling with keypad and re-speak and spelling with keypad). Experiments show that through the use of this virtual keypad interface, an accuracy gain of 10.92% during first attempt and 6.20% during re-speak by non-native speakers in ambiguous fields (initials, surnames, city and alphabets) can be achieved [3]. The aim of this work is to investigate whether the usability of the eye gaze and speech system can be improved through one of these three multimodal blended multimodal error recovery methods.