Robots are increasingly present in human spaces, such as for conducting deliveries in hospitals, interacting with visitors at museums, and stocking items in warehouses. To ensure the seamless integration of robots into these spaces, a new role in human-robot interaction is emerging—the robot wrangler, namely an individual who is responsible for setting up, overseeing, and troubleshooting the robot. To understand the needs of this stakeholder, we conducted a scoping review that uncovered a typology of robot wrangling across the research literature, and discovered that wrangling is an umbrella term that collapses a highly complex and heterogeneous space of activities, often rendering this labor difficult to characterize and support. To further clarify and understand robot wrangling, we then reflected on our own firsthand and imagined experiences as robot wranglers within our own respective domains. Guided by the scoping review and our reflections, we devise a series of design implications for supporting wranglers directly as individuals and as members of a wider service ecology.

We propose a proactive teleoperation support system to help high-power individuals exercise power appropriately during hierarchical interactions. In our study, seniors (high-power individuals) remotely operated an avatar robot as operators, while juniors (low-power individuals) exercised as exercisers. To ground the design, we conducted an observational study of hierarchical interactions, identifying three recurring challenges: perceived loss of power, escalation of negativity, and abandonment of power. Based on these findings, we proposed interaction policies and integrated them into a teleoperation system that provides operators with guidance for the appropriate use of power, aiming to prevent conflict and foster positive communication. We evaluated this system in a 2-hour study with 17 senior–junior pairs, comparing it to a baseline teleoperation system where operators controlled the avatar robot directly without such guidance. Results showed that our system significantly improved operator satisfaction and reduced workload. Exercisers reported greater enjoyment and acceptance, with similar exercise counts in both conditions. Interviews revealed that the system broadened operators’ communication strategies and fostered more positive, supportive environments for low-power participants.

We integrated state-of-the-art vision–language models (VLMs) into a hat shop robot, enabling four visually grounded human shop-worker behaviors: 1) offering personalized greetings based on customers' visual aspects, 2) recognizing their shopping activities and delivering timely encouragement to try hats, 3) offering feedback on hat fit, and 4) responding to customer inquiries that contain deictic references to hats. We strategically utilized both local and cloud-based VLMs to develop the robot’s visual perception, leveraging their complementary capabilities. We report key insights from our development process, including our experience collaborating with a domain expert on robot interaction development. Results from an 8-day field trial at a hat shop show that the robot’s advanced visual perception capabilities made a positive impression on both customers and the shop manager. Customers were excited by the robot’s visually grounded remarks, and the manager adopted a complementary working style that leveraged these new capabilities.

Bias in algorithmic systems is well documented, but the growing deployment of social robots in workplaces, homes, and public spaces poses a distinct challenge: their physical embodiment and social presence make bias visible in appearance, tangible in interaction, and consequential in everyday inclusion. This paper proposes a framework for understanding robotic bias across three dimensions - stereotypes, constraints, and treatments. The framework clarifies how different forms of bias emerge through embodiment and interaction, and highlights why they call for different strategies of intervention. In doing so, it provides a foundation for more systematic recognition and mitigation of bias in human-robot interaction.

We proposed a system for generating relational gestures that convey semantic relations such as similarity and difference between two objects. To understand how humans naturally express such relations, we conducted an observational study with experienced shopkeepers as they frequently compare objects using both speech and gestures. Through analysis of their interactions, we identified four common types of object relations and extracted representative gesture patterns for each. For example, similarity was often conveyed through synchronized hand movements bringing both hands closer together, accompanied by alternating gaze between two objects. Based on these findings, we developed a gesture generation system in which one large language model (LLM) infers the intended object relation from utterance text, and another LLM adapts gestures from a co-speech gesture system that aligns them with speech, integrating relational cues without disrupting this alignment. These modified gestures were automatically mapped onto a dual-arm robot. We evaluated the system through two user studies. In the first study, 20 participants were asked to identify object relations from 24 relational gestures performed by the robot without accompanying speech. They correctly identified the intended relations with an average accuracy of 89.8% across all relation types. In the second study, another 20 participants compared two robot conditions in a within-subjects design: one with relational gestures and one without. Results showed that the robot using relational gestures was perceived as more competent, sociable, and animate compared to the robot without them.

We introduce a novel immersive social teleoperation interface to perform social interaction intuitively and semi-automatic locomotion simultaneously. Teleoperated robots in complex, human-centric social environments present a significant challenge on simultaneously managing intricate navigation while engaging in natural social interaction. This dual task imposes a high cognitive load, hindering the fluidity and quality of social interaction. As existing systems typically prioritize either task-oriented teleoperation control or non-moving social interaction, failing to integrate dynamic locomotion with social engagement effectively. Our system combines a head-mounted display with a wide-angle, 270-degree video feed to support extensive situation awareness and a strong sense of presence to overcome these limitations by fostering an immersive and socially aware experience. Our interface performs low-level navigation of the robot by pointing at a place to go and selecting a person to follow. The operator can focus on high-level goals (social interactions). We evaluated our interface through a rigorous field experiment, using a testbed (remote guide) scenario developed through iterative pilot studies in a real-world shopping mall. Our findings demonstrate that the operator's task performance improves statistically significantly. We report other findings and discuss limitations and future improvements. In short, our novel interface, integrating immersive visualization with autonomous navigation, enables operators to achieve a more intuitive and engaging social interaction in dynamic remote social environments.

This study investigated the differences between human and robot gaze in influencing preference formation, and examined the role of Theory of Mind (ToM) abilities in this process. Human eye gaze is one of the most important sources of information for social interaction and research has demonstrated its effectiveness in influencing people's preference. With increasing technological development, we will interact with robots that can exhibit gaze behavior and influence people's preference. It is unclear whether there are any differences between humans and robots in this process. The present study aimed to analyze the role of the gaze of a robot and a human in influencing the ascription of a preference to the gazer and the participants' preference. Furthermore, we examined the role of ToM abilities in preference formation. The results showed that the gaze has a greater effect on the gazer preference compared to participants' preference regardless of the agent (human or robot). In addition, ToM abilities predict both gazer and individual preferences in the robot's condition only even though different socio-cognitive mechanisms are involved. The study suggests that adults are cognitively able to process the gaze of a robot similar to a human, recognizing the underlying mental state. However, only for the robot, different cognitive mechanisms are involved in the gazer (i.e., perspective taking) and participants' preference formation (i.e., advanced ToM).

This study investigates how different forms of operator embodiment (avatar robots, telepresence robots, and direct in-person interaction) affect people’s willingness and ability to approach and initiate social interactions with strangers in public spaces. Participants were tasked with distributing information to pedestrians in a shopping mall across the three embodiment conditions. We examined behavioral performance (e.g., frequency of approaches, hesitation time), self-reported anxiety, fear of talking to strangers, and post-interaction reflections. Results showed that participants using robot-mediated embodiment (avatar and telepresence robots) exhibited greater social engagement and reported less hesitation time, lower state anxiety, and reduced fear of talking to strangers compared to the in-person condition. Notably, the avatar robot condition was associated with a greater reduction in fear of talking to strangers than the telepresence condition. Our interview results suggest that the reduction may be attributed to the anonymity provided by the avatar, which decreases self-consciousness and social pressure. Additionally, many participants noted that pedestrians showed heightened interest in the robots, which made initiating interactions easier. Telepresence robots also offered psychological relief by removing the need for physical presence, although some participants were more self-aware due to their visible face and voice. While most participants reported feeling nervous in face-to-face interactions, some appreciated the physical freedom and focused delivery enabled by in-person engagement. These findings suggest that teleoperated robots can mitigate psychological barriers in socially demanding tasks, and that embodiment plays a role in shaping operator experience and social outcomes.

We define authoritative presence as letting people experience authority through human-made technology in sensory or non-sensory ways. Our goal is to design a robot that creates the impression of possessing capabilities worthy of respect as a source of authority, thereby enhancing compliance without attributing that authority to external sources, such as a specific person or organization. We hypothesized that strategies commonly used in the Wizard-of-Oz method could help manipulate authoritative presence as it strives to ensure that the robot is not perceived as having additional abilities beyond those introduced by the manipulations. Wizards typically need to maintain the robot’s functionality and abilities at an appropriate level to minimize unwanted influence on participants’ perceptions and interactions. By interviewing HRI researchers who have wizarded, we summarized their usual strategies and implemented the opposite behaviors in a robot to investigate if this would contribute to authoritative presence. Based on the findings, we designed four behaviors that include (1) let the robot have an open-ended conversation with people, (2) randomize the robot’s reaction delay timing, (3) let the robot move with inconsistent velocity, and (4) let the robot perceive people’s status without looking at them. To evaluate the impact of these behaviors, we conducted a video-based online experiment with 942 participants, using a between-subjects design. The experiment aimed to determine whether the behaviors conveying authoritative presence would make people perceive the robot as having more authority and increase their likelihood of complying with its requests. A mediation analysis indicated that despite a decrease in perceived authority, the imply authoritative presence condition had a positive effect on participant compliance. Our study formally introduces the concept of authoritative presence, providing a proof-of-concept for how robots can create authoritative presence through specific behaviors. This work lays the groundwork for future research on authority and robotics.

Irony can have remarkable positive effects on human behavior. However, those effects depend on the appropriate use of irony. Existing theoretical frameworks in psychology and sociology discuss the use of irony, including its effects on emotional states and behaviors. However, these frameworks often rely on subjective intuition, rendering them impractical for robotic systems. Additionally, our investigation of prior research in human-robot interaction (HRI) reveals a lack of systematic, context-sensitive frameworks for irony usage. This gap limits the ability of robots to dynamically evaluate the effectiveness of irony and adapt its use based on user states and interaction contexts. This study introduces a novel operational framework for irony usage in HRI, addressing these gaps through dynamic, real-time guidelines that enable robots to effectively and adaptively deploy irony. Given the lack of direct support from existing theoretical frameworks, our research adopts an exploratory approach. Specifically, we conducted semi-structured interviews to investigate successful irony in interpersonal interactions that can positively impact humans, evaluating it from the perspective of those who received irony. In the investigation, 25 irony receivers were interviewed, and the essential factors in successful irony usage were extracted from those interviews. The interview results indicated that successful irony is typically used when the receiver is in an inefficient state but in a good mood and energetic. Based on these findings, guidelines for the successful use of irony by a robotic system were established. In the proposed system, the robot acts as an exercise partner, encouraging users to adhere to their ideal plans. The robot uses irony on users when they do not make sufficient efforts, but only if they are in a good mental state and energetic. In an experiment involving 22 participants, the proposed system outperformed a baseline system, where the robot used irony whenever participants exhibited inefficiency. Participants using the proposed system exercised for 59 minutes on average, compared to 48 minutes with the baseline system. Moreover, they felt that the robot with the proposed robot-behavior model better understood their states, and they also felt more enjoyment.

To investigate the use of different pointing forms in service scenarios, we collected the ShopPoint dataset, a skeleton-based dataset of pointing gestures from customer–shopkeeper interactions in a camera shop scenario. Thirteen participants took part in the data collection, including 3 shopkeepers with real-world customer service experience and 10 customers. We recorded 61 one-to-one role-played interactions. Coders annotated pointing gestures from videos of these interactions, emphasizing pointing arm forms (straight-arm, bent-arm, and hand-only pointing) and hand forms (index-finger and open-hand pointing). This annotation process resulted in 2,959 pointing gestures. We conducted statistical analysis on the annotated data. The analysis revealed that bent-arm pointing was used more frequently than other arm forms. Straight-arm pointing was used more for far targets than for close targets, and hand-only was used more for close targets. Shopkeepers used bent-arm pointing more frequently than customers when referring to far targets. To evaluate the recognition of these pointing gestures, we tested several existing Skeleton-based Action Recognition (SAR) methods on the dataset. The highest accuracy was achieved at 72.51% by using transfer learning (i.e., pretraining and fine-tuning). This evaluation indicates that though transfer learning aids performance, recognizing pointing with diverse forms remains challenging.

We propose a method to automatically assess group openness by observing interactions within socially occupied spaces. Openness refers to a group’s predisposition to allow external parties to join a social encounter. Robots need the capability of assessing openness to ensure social comfort and improve first impressions, saving time and resources by avoiding unwelcomed interactions. We extract social signals from gestures, posture, space and constellations using skeleton data from depth sensors. We collected 88 unique around 5-minute group interactions involving 82 Japanese participants across two scenarios: one with an external focus of attention (a poster illustration on the wall, Poster scenario) and another without one (standing conversation, No Poster scenario). We trained a linear classifier optimised with a Stochastic Gradient Descent on raw data, as well as aggregated bodily motion and spatial cues with a full and a half temporal overlap by implementing a sliding window temporal analysis across observation windows, demonstrating its capability to assess group openness accurately. Our approach was evaluated across temporal analysis, scenario comparison and group sizes, reaching 79% peak prediction accuracy and an F1 score of 0.8201 for the No Poster scenario and 78% with a 0.8147 F1 score for the Poster scenario, outperforming a human baseline of 50% and 70%, respectively. Feature importance scores suggest groups display different cues for openness in each scenario. Important cues for the No Poster scenario included tightness, group size, body crunching, right arm extension and mouth covering. For the Poster scenario, relevant cues were arm extension, distance to the wall, gap between members and tightness. These findings contribute to understanding group openness prior to engagement, improving robots’ social intelligence in successfully approaching human groups.

Social robots are gradually being integrated into diverse social spheres, assuming multifaceted roles such as store clerks. Many of the interactive tasks entrusted to these robots are intricate, requiring more than simply responding to requests, and thus are not easily preprogrammed. One such task is sales negotiation. Empowering robots with negotiation skills is challenging due to the nuanced, context-dependent nature of effective tactics, which typically vary across different stores. To tackle this, we explore the potential of reinforcement learning without demonstrations for robots to grasp complex social interactions and adapt to specific environments, using sales negotiation as a test case. Drawing insights from interviews with experienced sales clerks and observations of negotiation role-plays, we identified essential verbal and nonverbal features. Coupled with a fast-learning reinforcement method, we evaluated the system through interactions with human participants in two distinct store settings. Results show that the robot was able to acquire negotiation strategies tailored to each environment.

Data-driven social imitation learning is a minimally supervised approach to generating robot behaviors for human–robot interaction (HRI). However, like any learning-based approach, there are inherent challenges and potential errors. Existing strategies for error prevention in HRI predominantly rely on handcrafted methods and predefined design guidelines, with limited focus on learning-based approaches. In this article, we present an autonomous, online continual learning strategy for data-driven HRI that uses error detections to facilitate error prevention. We leverage error detections as negative training signals, which dynamically update the data-driven robot action model parameters, guiding it away from repeating errors in future interactions. This approach modifies the standard (offline) data-driven HRI training objective to continually refine the robot’s performance in an online manner. We use simulated HRI data to show that our error prevention method can effectively reduce the average rate of robot errors in HRI by up to 6.68%. Furthermore, even with a limited-precision error detector, our method can achieve a 4.68% error rate reduction, paving the way towards a completely autonomous error management framework for data-driven HRI.

We explored whether and how individuals attribute authority to a robot. Online participants ( N = 362 Japanese adults) received short videos showing a social interaction between a group of three human actors and a lone agent. Crucially, in the videos, the group interacted with a robot or a fourth human actor. The group bowed deeply for the lone agent. After each video, participants were asked to say to what extent the lone agent can be described as a source of authority ( authority attribution ) and to what extent the group would comply with an order issued by the lone agent ( obedience expectation ). We found that participants perceived the robot as in charge and expected the group to comply with its orders, although participants attributed less authority to the robot than to the human actor and expected the group to obey the human actor more than the robot. As such, we conclude that bowing as a social cue for authority attribution applies to interactions between robots and humans. These findings can have important applications for designing authority- or leader-like robot figures.

In public spaces, robots often need to stop suddenly while navigating among pedestrians, such as when they need to make a quick turn or investigate something detected by their sensors. Poorly signaled stops can cause disruptions, forcing pedestrians to make an abrupt avoidance or even lead to risky collisions. To address this issue, we propose a human-inspired stopping model based on the observation of real people's stops. Interestingly, we found that people also often fail to stop without bothering those behind them, causing disruptions. In contrast, we found that stops that do not bother people following closely involve natural signals such as gradual deceleration, head motion, and body orientation changes. These cues help others anticipate upcoming changes, like an upcoming stop. Based on these observations, we implemented our model on a humanoid robot and conducted a field study in a shopping mall. The results show that incorporating these human-like signals leads to smoother interactions with pedestrians following behind, as compared to simple stopping using in standard robot navigation.

We developed a robot that dynamically asks questions after listening to a lecture in university group classes to enhance students' learning. Based on interview results with ten professors, we designed our robot to let professors choose when a robot asks questions, ask various kinds of questions by letting professors choose question types, and ask questions dynamically without sharing them with professors in advance. The prepared question types include such as clarification questions, which are fact-based questions about a lecture, and thought-provoking questions, which activate students' critical thinking. We expanded them into eight sub-categories of questions that the most common types that are likely to emerge in classrooms. Leveraging large language models (LLMs), based on lecture transcripts, we developed a robot system that generates questions in the eight sub-categories and yields 0.93 classification accuracy toward human coding results. We also conducted a case study with our developed robot in four university lectures. In these lectures that involved our robot, the professors often asked reasoning questions and criticism questions, and the students showed engagement with the robot-professor dialogues. Interviews with 20 students revealed that the robot's questions contributed to the classes by helping the students reflect on the lectures and gain new perspectives. The professors also recognized some benefits of the robot, perceiving its presence as a question facilitator, a mood maker, a communication tool as well as a motivator to prepare more diligently for their own lectures.

This study investigated the integration of smell detection, storage, and reproduction capabilities in robot avatars, examining how these functions impact interactions between a remote operator and a co-present partner. We developed a robot avatar system equipped with olfactory technology, using a combination of actual hardware and Wizard-of-Oz techniques. In an experimental setting, participants engaged in a task that involved spontaneous discussions about smells. Our findings show that the inclusion of olfactory functionality not only affected the dynamics of interaction but also enhanced the perceived quality of the experience for both the operator and the partner.

When verbal communication is limited, robots passing objects to humans without providing additional information (e.g., temperature, weight) can result in potential poor handovers and disappointing user experiences. To address this issue, we introduced a method for conveying object properties through robot manipulation. We began by proposing four criteria for selecting properties from two widely recognized sets: one focusing on the semantic features of objects and the other on tactile sensations. These properties were clustered into eight physical categories: hot, cold, heavy, light, slippery, sticky, fragile, and smelly. Professional actors were then recruited to demonstrate these properties through object manipulation, from which we extracted a set of fundamental yet expressive manipulation behaviors, i.e., key elements, that help people recognize these properties. These elements were implemented on a dual-arm robot, followed by an evaluation of their utility through participant feedback. To generate time-constrained sequences of elements, we developed a property-based motion planner that balances time and utility in conveying object properties. Results from a within-subjects study involving 20 participants showed that individuals could accurately interpret the properties conveyed by robot object manipulation, validating the effectiveness of the proposed approach.

Recent progress in robot autonomy and safety has significantly improved human-robot interactions, enabling robots to work alongside humans on various tasks. However, complex assembly tasks still present significant challenges due to inherent task variability and the need for precise operations. This work explores deploying robots in an assistive role for such tasks, where the robot assists by fetching parts while the skilled worker provides high-level guidance and performs the assembly. We introduce GEAR, a gaze-enabled system designed to enhance human-robot collaboration by allowing robots to respond to the user’s gaze. We evaluate GEAR against a touch-based interface where users interact with the robot through a touchscreen. The experimental study involved 30 participants working on two distinct assembly scenarios of varying complexity. Results demonstrated that GEAR enabled participants to accomplish the assembly with reduced physical demand and effort compared to the touchscreen interface, especially for complex tasks, maintaining great performance, and receiving objects effectively. Participants also reported enhanced user experience while performing assembly tasks. Project page:sites.google.com/view/gear-hri

The study aims to investigate the effect of cultural background between Italy and Japan on 5-year-old children’s moral judgements and emotion attributions towards human and robot. The children watched videos drawn from classic ‘happy victimizer’ stories, in which the transgressors were either a child or a robot, violating rules against stealing or not sharing. We assessed the children’s attribution of emotions both to the transgressors and to themselves as victimisers (i.e. first-person perspective), as well as the moral judgement of the violations. The results showed that children from both cultures do not significantly discriminate between human and robot in their moral judgement and emotion attributions. Concerning moral emotions, Italian children tend to attribute fewer negative emotions to the transgressor than Japanese children, especially in the not sharing scenario. Furthermore, adopting a first-person perspective to evaluate moral transgressions reduces cultural differences in emotion attributions. The study highlights how culture, rather than the transgressor’s agency (human or robot), influences early moral reasoning.

This study explores the potential benefits of robots having the capability to anticipate people’s mental states in an exercise context. We designed 80 utterances for a robot with associated gestures that exhibit a range of emotional characteristics and then performed a 23-person data collection to investigate the effects of these robot behaviors on human mental states during exercise. The results of cluster analysis revealed that (1) utterances with similar meanings had the same effect and (2) the effects of a certain cluster on different people depend on their emotional state. On the basis of these findings, we proposed a robotic system that anticipates the effect of utterances on the individual’s future mental state, thereby choosing utterances that can positively impact the individual. This system incorporates three main features: (1) associating the relevant events detected by sensors with a user’s emotional state; (2) anticipating the effects of robot behavior on the user’s future mental state to choose the next behavior that maximizes the anticipated gain; and (3) determining appropriate times to provide coaching feedback, using predefined rules in the motion module for timing decisions. To evaluate the proposed system’s overall performance comprehensively, we compare robots equipped with the system’s unique features to those lacking these features. We design the baseline condition that lacks these unique features, opting for periodic random selection of utterances for interaction based on the current context. We conducted a 21-person experiment to evaluate the system’s performance. We found that participants perceived the robot to have a good understanding of their mental states and that they enjoyed the exercises more and put in more effort due to the robot’s encouragement.

Service robots in public spaces can sometimes become victims of ‘abuse’, manifesting as persistent blocking of the robot’s path, physical violence such as pushing or pulling, or abusive language. This kind of abuse can be a serious obstacle to the deployment of robots. We studied the possibility of mitigating obstructive behaviour towards the robot, which is known to happen in the early stages of robot abuse, by having the robot express anger-like emotions. We identified and implemented three distinct anger behaviours: furious, quiet and scolding anger. Through an online video survey, we confirmed that all three designed robot behaviours were perceived as expressions of anger. Finally, we ran an in-lab experiment with child participants to study the influence of the robot’s expressions of anger on children’s obstructive behaviours. The results show that ‘furious anger’ was effective in reducing the frequency of obstructions, whereas the other two anger types were not.

Avatar robots allow a teleoperator to interact with the people and environment of a remote place. Malicious operators can use this technology to perpetrate malicious or low-moral actions. In this study, we used hazard identification workshops to identify low-moral actions that are possible through the locomotor movement, cameras, and microphones of an avatar robot. We conducted three workshops, each with four potential future users of avatars, to brainstorm possible low-moral actions. As avatars are not yet widespread, we gave participants experience with this technology by having them control both a simulated avatar and a real avatar as a malicious anonymous operator in a variety of situations. They also experienced sharing space with an avatar controlled by a malicious anonymous operator. We categorized the ideas generated from the workshops using affinity diagram analysis and identified four major categories: violate privacy and security, inhibit, annoy, and destroy or hurt. We also identified subcategories for each. In the second half of this study, we discuss all low-moral action subcategories in terms of their detection, mitigation, and prevention by studying literature from autonomous, social, teleoperated, and telepresence robots as well as other fields where relevant.

This study discusses the moral concern exhibited toward the robots that are being introduced into society. Although much debate has focused on whether robots should be granted moral status, some of us do show concern for them as moral patients. We conducted three surveys in creating and validating a new scale called Moral Concern for Robot Scale (MCRS), designed to measure such concern. In Study 1, we created an initial version of MCRS, confirmed its content validity by experts, and explored its factor structure with 500 participants. We verified the three-factor structure derived in Study 1 by a confirmatory factor analysis in Study 2 (n = 500). Study 2 also confirmed construct validity by authenticating correlations with other measures that were predicted to be theoretically relevant. Study 3 (n = 500) confirmed predictive validity using experimental vignette methodology. Specifically, MCRS positively influenced the willingness to engage in prosocial behavior toward a robot and better predicted it than other relevant scales. From the overall results, MCRS is demonstrated to be a valid and reliable instrument.

Research has shown that data-driven robots deployed in social settings are likely to unconsciously perpetuate systemic social biases. Despite this, robots can also be deployed to promote fair behaviour in humans. These phenomena have led to the development of two broad sub-disciplines in HRI concerning ‘fairness’: a data-centric approach to ensuring robots operate fairly and a human-centric approach which aims to use robots as interventions to promote fairness in society. To date, these two fields have developed independently, thus it is unknown how data-driven robots can be used to suppress biases in human-human interactions. In this paper, we present a conceptual framework and hypothetical example of how robots might deploy data-driven fairness interventions, to actively suppress social biases in human-human interactions.

We investigate how gender stereotypes in authority influence the perceptions and behavior of avatar robots operators and their interlocutors. Gender stereotypes, which typically place men in more authoritative positions than women, are present in not only inter-human but also human-robot interaction. As avatar robots become more integrated into our lives and serve for diverse usages, they may be utilized in positions where they require authority. We study how avatar robot gender and operator gender affect expressions and perception of gender stereotypes in a customer service scenario with 41 pairs of participants. Operators controlled binary gendered avatar robots one at a time, acting as shopkeepers that had to assert authority over customers behaving improperly. The operators perceived their authority to be higher with male avatar robots compared to female ones, regardless of operator gender. We did not detect an effect on customer's perception of the shopkeeper's authority. While less than half of operators and customers perceived authority for reasons related to traditional gender stereotypes, others observed behaviors that did not align with stereotypes. Avatar embodiment may also help operators assert authority safely due to being physically hidden from the customers.

We studied how robots could stop people from repeatedly obstructing them by using reactions that people commonly use. From 35 hours of observation of people in a shopping mall, we identified one commonly used reaction, which we named suggestive avoidance. It consists of making a quick movement to the side while rotating the body and gaze toward the obstructing person, in a way that seems to imply that they were bothered by the obstruction. We modeled the human suggestive avoidance behavior, implemented it on a robot, and tested it both in a lab experiment and a field study. The results from the lab study confirmed that people perceive a robot using suggestive avoidance as being more bothered, as well as more human-like. The field study showed that when a robot uses suggestive avoidance people are less likely to bother it again.

We developed an autonomous hatshop robot for encouraging customers to try on hats by providing comments that appropriately fit their actions, and in such a way also indirectly exerting social pressure. To enable it to offer such a service smoothly in a real shop, we developed a large system (around 150k lines of code with 23 ROS packages) integrated with various technologies, like people tracking, shopping activity recognition and navigation. The robot needed to move in narrow corridors, detect customers, and recognise their shopping activities. We employed an iterative development process, repeating trial-and-error integration with the robot in the actual shop, while also collecting real-world data during field-testing. This process enabled us to improve our shopping activity recognition system by collecting real-world data, and to adapt our software modules to the target shop environment. We report the lessons learnt during our system development process. The results of our 11-day field trial show that our robot was able to provide its services reasonably well. Many customers expressed a positive impression of the robot and its services.

We developed a motion control algorithm for a social mobile robot to intuitively convey its intent via social cues to pass through aisles and avoid misunderstanding in passing interactions with people, which frequently occur when a robot navigates in narrow shared environments. Inspired by observations of human behavior, the proposed algorithm estimates the extent to which a person understands the robot's intent on the basis of the person's reactions to the oncoming robot and provides the robot with corresponding motion strategies for effective passing interactions. We implemented the proposed algorithm onto an omni-directional humanoid robot and conducted a field study over six days in a store with 75 cm wide narrow aisles. The resulting behaviors of 50 customers demonstrated that our proposed method provided people with a clearer understanding of the robot's intent in passing interactions, and thus the robot had more opportunity (73.1%) to pass through aisles compared to 16.7% if the robot moved and then waited for people to make space.

In retail settings, a robot's one-handed manipulation of objects can come across as thoughtless and impolite, thus creating a negative customer experience. To solve this problem, we first observed how human shopkeepers interact with customers, specifically focusing on their hand movements during object manipulation. From the observation and analysis of shopkeepers' hand movements, we identified an essential element of their idle hand movements: "support" provided by the idle hand as the primary hand manipulates an object. Based on this observation, we proposed a model that coordinates the movements of a robot's idle hand with its primary task-engaged hand, emphasizing its supportive behaviors. In a within-subjects study, 20 participants interacted with robot shopkeepers under different conditions to assess the impact of incorporating support behavior with the idle hand. The results show that the proposed model significantly outperforms a baseline in terms of politeness and competence, suggesting enhanced object-based interactions between the robot shopkeepers and customers.

Data-driven social imitation learning is a minimally-supervised approach to generating robot behaviors for human-robot interaction (HRI). However, this type of learning-based approach is error-prone. Existing error detection methods for HRI rely on data labeling, rendering them inappropriate for the data-driven paradigm. We present a zero-shot error detection strategy that requires no labeled data. We use human interaction data to learn models of normal human behavior, then use these models to extract features that help discriminate abnormal human reactions to robot errors. In this feature space, we frame error detection as a novelty detection task, utilizing human interaction data to learn a model of non-erroneous interactions in an unsupervised fashion. Then, we apply the fitted novelty detector to HRI data to identify erroneous robot behavior. We show that our method obtains an average precision of 0.497 on errors, outperforming unsupervised baselines and supervised approaches with limited training data.

Avatar robots are telepresence robots that allow people to project themselves physically in distant places. Teleoperators use such avatars to have remote social interactions. Mobility adds to their interactive capabilities. However, it also presents challenges in social spaces; for instance, teleoperators can mistakenly move their avatars in a socially inappropriate manner. Since these robots have yet to become ubiquitous, there is little real-world data to objectively analyze the presence and frequency of such undesirable movement. Consequently, the different ways such movements manifest are unknown, and their occurrence cannot be detected. To address this problem, we conducted a laboratory experiment with hired participants in a replica artwork exhibition so that we could gather realistic human-avatar interaction data using sensors. Our data analysis helped us identify three types of inadvertently inappropriate movement behavior. We present a definition of such behavior based on these three inappropriate movement types and a labeled dataset containing more than 400 interactions, 18% of which were inappropriate. We also present detection models for each inappropriate movement type trained using our dataset.

Telepresence technology creates the opportunity for people that were traditionally left out of the workforce to work remotely. In the service industry, a pool of novice remote workers could teleoperate robots to perform short work stints to fill in the gaps left by the dwindling workforce. A hurdle is that consistently talking appropriately and politely imposes a severe mental burden on such novice operators and the quality of the service may suffer. In this study, we propose a teleoperation support system that lets novice remote workers talk freely without considering appropriateness and politeness while maintaining the quality of the service. The proposed system exploits intent recognition to transform casual utterances into predefined appropriate and polite utterances. We conducted a within-subject user study where 23 participants played the role of novice remote operators controlling a guardsman robot in charge of monitoring customers’ behaviors. We measured the workload with and without using the proposed support system using NASA task load index questionnaires. The workload was significantly lower ( p < .001) when using the proposed support system ( M = 46.07, SD = 14.36) than when not using it ( M = 62.74, SD = 12.70). The effect size was large (Cohen’s d = 1.23).

We developed a security guard robot that is specifically designed to manage queues of people and conducted a field trial at an actual public event to assess its effectiveness. However, the acceptance of robot instructions or admonishments poses challenges in real-world applications. Our primary objective was to achieve an effective and socially acceptable queue-management solution. To accomplish this, we took inspiration from human security guards whose role has already been well received in society. Our robot, whose design embodied the image of a professional security guard, focused on three key aspects: duties, professional behavior, and appearance. To ensure its competence, we interviewed professional security guards to deepen our understanding of the responsibilities associated with queue management. Based on their insights, we incorporated features of ushering, admonishing, announcing, and question answering into the robot’s functionality. We also prioritized the modeling of professional ushering behavior. During a 10-day field trial at a children’s amusement event, we interviewed both the visitors who interacted with the robot and the event staff. The results revealed that visitors generally complied with its ushering and admonishments, indicating a positive reception. Both visitors and event staff expressed an overall favorable impression of the robot and its queue-management services. These findings suggest that our proposed security guard robot shows great promise as a solution for effective crowd handling in public spaces.

During the previous field study with a robot and its interaction with mall visitors, we observed a surprising event during which a leaflet-distributing robot was abused, although it was subsequently helped by one of its previous abusers. After analyzing 72.25 hours of video data, we identified 47 cases where a robot dropped a leaflet and classified them according to following three criteria: 1) interaction between the potential helper or others with the robot before it dropped the leaflet, 2) the nature of the interaction (abused or not), and 3) whether it was helped. Using the Trajectory Equifinality Model (TEM), we analyzed 19 cases where the robot was helped. We identified the following interaction process that started with individuals who paid attention to the robot, whether they had abusive or non-abusive interactions with it, whether they noticed its failure, and finally whether they helped it. The presence of others encouraged the person to focus on the robot, and the interactions with it led to helping, regardless whether the interaction was abusive. The absence of others when the robot dropped the leaflet encouraged helping. The findings of this study will motivate interaction designs for social robots that can leverage human help.

We considered the robot’s use of smells as a nonverbal way of interacting with people. In particular, we were interested in active smell delivery, which involves delivering scents to specific locations and at precise moments. Since smell is still largely unexplored as an interaction modality for human-robot interactions, many aspects are unknown. To address this gap, through exploratory studies we identified the important factors of mounting positions as well as timing and movement coordination, which need to be considered for the successful integration of active smell delivery on robots.

To personalize the robot guide experience, the robot needs to detect a person's indifference and adjust its explanation toward the person's interest in topics. However, detecting the person's indifference is challenging in a museum, as we cannot use a bulky wearable or facial expression recognition due to unexpected light condition or standing position. We propose to observe people's behaviors and movements on detecting people's indifference. To prove its feasibility, we invited 11 participants to our in-lab museum-like environment. Our robot explains exhibits while videorecording the interaction. Then, we asked participants to watch the recordings and report when they felt bored or indifferent to the explanation. We labelled their movement and matched them to their report so that we know which behaviors and movements hint the person's indifference. We used the decision tree and random forest methods to understand the common pattern when people are indifferent during the explanation in a museum scenario. From our observation experiment, we found that if the listener nods their heads many times or looks at the exhibit for a long time, they are likely interested in the topic, fewer overall movements or looking elsewhere hint that the listener may be indifferent, and if the explanation goes longer than three minutes, the listener would be likely bored.

In this paper, we propose a system that supports operators who provide services to customers using teleoperated robots. We observed that unprofessional or lazy operators of teleoperated robots are a risk for businesses as they are likely to speak in ways that are inappropriate for customer services. The proposed system lets competent operators talk freely to customers and thus provide high quality service. For subpar operators, the proposed system filters inappropriate utterances to improve the service they provide. We conducted a user study with 21 participants to compare the proposed support system to a baseline system where operators talk freely to customers. For subpar operators, the quality of the service is significantly higher in terms of perceived politeness and reported customer satisfaction when using the proposed support system compared to when using the baseline system. For competent operators, we found no significant differences in the quality of the service between the two systems.

We developed an admonishing service for a shopworker robot and conducted a field trial to investigate the impressions of a shop’s staff and customers. Applying the admonishing service in a real-world robot is difficult due to the high risk of rejection by society. We wanted to achieve an acceptable admonishing service while simultaneously avoiding the impression of a forceful request from a machine. We proposed a harmonized design that provided friendly and admonishing services. First, we interviewed a shop’s staff to learn their strategies for both friendly and admonishing services. From our evaluation of the interview results, we derived three design principles: friendly impressions, zero erroneous admonishments, and polite requests. Based on the design principles, we implemented our harmonized design on a social robot that guides customers to product locations and admonishes those who are not wearing face masks. We conducted a 13-day field trial in a retail shop and interviewed the customers and shopworkers to learn their impressions of our robot. The results of the field trial imply that our harmonized design approach is successful. Both the customers and the shop staff had overall positive impressions of the robot, its admonishing and friendly services, and expressed an intention to use it in the future. Furthermore, we studied the robot’s autonomous service-providing capability in the field and conducted an evaluation with hired participants to deepen our study of the robot’s mask recognition capability.

Service and social robot in public scenarios will face various tasks in future applications, such as guiding people or admonishing them to provide assistance or convey social norms. Robots in public spaces might also incorporate roles of authority figures who might admonish people (e.g., security or guard robots). However, recent investigations showed that people ignore the admonishment of robots. Thus, in this work, we are looking at the reasons why people might ignore robots based on the Cognitive Dissonance Theory (CDT). We present the results of two consecutive field observations where a robot admonishes participants (i.e., pedestrians in a shopping mall) and requests them to stop using a smartphone while walking, which is considered an unmoral behavior. In the first field observation, we approached 160 participants over four days and conducted semi-structured interviews with 19 of them. Approximately half of the people ignored the robot, and half of them followed the instructions. Our interview results show that people who ignore the robot indeed use trivialization as a cognitive dissonance reduction strategy to justify ignoring the robot. Based on our analysis of the results, we developed a counter-trivialization strategy that anticipates this dissonance reduction strategy. We admonished 167 participants in our second field observation over four days, and our results show that significantly fewer people ignore the instructions of the robot when the robot uses a counter-trivialization strategy.

Omni-directional robots have gradually been popular for social interactions with people in human environments. The characteristics of omni-directional bases allow the robots to change their body orientation freely while moving straight. However, human spectators show dislike when observing robots behave unnaturally. In this paper, we observed how humans naturally move to goals and then developed a motion planning algorithm for omni-directional robots to resemble human movements in a time-efficient manner. Instead of treating the translation and rotation of a robot separately, the proposed motion planner couples the two motions with constraints inspired from the observation of human behaviors. We implemented the proposed method onto an omni-directional robot and conducted navigation experiments in a shop with shelves and narrow corridors at width of 90cm. Results from a within-participants study of 300 human spectators validated that the proposed human-inspired motion planner provided people with more natural and predictable feelings compared to the common rotate-while-move or rotate-then-move strategies.

Can robots influence the moral behavior of humans by simply doing their job? Robots have been considered as replacement for humans in repetitive jobs in public spaces, but what could that mean for the behavior of the surrounding people is not known. In this work we were interested to see how people change their behavior when they observe either a robot or a person do a morally-laden task. In particular, we studied the influence of seeing a robot or a human picking up and discarding garbage on the observer's willingness to litter or to pick up garbage. The study was done as a video-based survey. Results show that while observing a person clean up does make people less keen to litter, this effect is not present when people watch a robot doing the same action. Moreover, people appear to feel less guilty about littering if they observed a robot doing the cleaning up than in the case when they watched a human cleaner.

Most social robot behaviors in human-robot interaction are designed to be polite, but there is little research about how or when a robot could be impolite, and if that may ever be beneficial. We explore the potential benefits and tradeoffs of different politeness levels for human-robot interaction in an exercise context. We designed impolite and polite phrases for a robot exercise trainer and conducted a 24-person experiment where people squat in front of the robot as it uses (im)polite phrases to encourage them. We found participants exercised harder and felt competitive with the impolite robot, while the polite robot was found to be friendly, but sometimes uncompelling and disingenuous. Our work provides evidence that human-robot interaction should continue to aim for more nuanced and complex models of communication.

We studied the programming and debugging processes of an autonomous mobile social robot with a focus on the programmers. This process is time-consuming in a populated environment where a mobile social robot is designed to interact with real pedestrians. From our observations, we identified two types of time-wasting behaviors among programmers: cherry-picking and a shortage of coverage in their testing. We developed a new tool, a test generator framework, to help avoid these testing time-wasters. This framework generates new testing scenarios to be used in a simulator by blending a user-prepared test with pre-stored pedestrian patterns. Finally, we conducted a user study to verify the effects of our test generator. The results showed that our test generator significantly reduced the programming and debugging time needed for autonomous mobile social robots.

Data-driven imitation learning enables service robots to learn social interaction behaviors, but these systems cannot adapt after training to changes in the environment, such as changing products in a store. To solve this, a novel learning system that uses neural attention and approximate string matching to copy information from a product information database to its output is proposed. A camera shop interaction dataset was simulated for training/testing. The proposed system was found to outperform a baseline and a previous state of the art in an offline, human-judged evaluation.

The service industry is facing an increase in the number of malicious customers (customers with unreasonable complaints). Employees have reported that handling unreasonable complaints is particularly stressful. Considering the recent push for workplace automation, we should have robots handling this task in place of humans. We propose a robot behavioral model designed for handling unreasonable complaints. The robot with this model has to "please the customer'' without proposing a settlement. From a large survey of Japanese workers conducted by labor unions and an interview survey of experienced workers we conducted, we identified the conventional complaint handling flow as 1) listen to the complaint, 2) confirm the content of the complaint, 3) apologize, 4) give an explanation, and 5) conclude. The proposed behavioral model is a variation of this flow that takes into account the "state of mind'' of the customer. In particular, the robot with this model does not leave the first step and keeps asking questions until the customer is "ready to listen''. We conducted a user study, using a Wizard-of-Oz approach, to compare the proposed behavioral model to a baseline one implementing the conventional flow. We replicated in our laboratory the situation of a customer in a mobile phone shop. The proposed behavioral model was significantly better at making the customers believe that the robot listened to them and tried to help.

We envision a future where service robots autonomously learn how to interact with humans directly from human-human interaction data, without any manual intervention. In this paper, we present a data-driven pipeline that: (1) takes in low-level data of a human shopkeeper interacting with multiple customers (28 hours of collected data); (2) autonomously extracts high-level actions from that data; and (3) learns -- without manual intervention -- how a robotic shopkeeper should respond to customers' actions online. Our proposed system for learning the interaction logic uses neural networks to first learn which customer actions are important to respond to and then learn how the shopkeeper should respond to those important customer actions. We present a novel technique for learning which customer actions are important by first learning the hidden causal relationship between customer and shopkeeper actions. In an offline evaluation, we show that our proposed technique significantly outperforms state-of-the-art baselines, in both which customer actions are important and how to respond to them.

Interacting physically with robots and sharing environment with them leads to situations where humans and robots have to cross each other in narrow corridors. In these cases, the robot has to make space for the human to pass. From observation of human-human crossing behaviours, we isolated two main factors in this avoiding behaviour: body rotation and sliding motion. We implemented a robot controller able to vary these factors and explored how this variation impacted on people's perception. Results from a within-participants study involving 23 participants show that people prefer a robot rotating its body when crossing them. Additionally, a sliding motion is rated as being warmer. These results show the importance of social avoidance when interacting with humans.

We reveal the process of children engaging in such serious abuse as kicking and punching robots. In study 1, we established a process model of robot abuse and used a qualitative analysis method specialized for time-series data: the Trajectory Equifinality Model (TEM). With the TEM method, we analyzed interactions from nine children who committed serious robot abuse from which we developed a multi-stage model: the abuse escalation model. The model has four stages: approach, mild abuse, physical abuse, and escalation. For each stage, we identified social guides (SGs), which are influencing events that fuel the stage. In study 2, we conducted a quantitative analysis to examine the effect of these SGs. We analyzed 12 hours of data that included 522 children who visited the observed area nearby the robot, coded their behaviors, and statistically tested whether the presence of each SG promoted the stage. Our analysis confirmed the correlations of four SGs and children's behaviors: the presence of other children related a new child to approach the robot (SG1); mild abuse by another child related a child to do mild abuse (SG2); physical abuse by another child related a child to conduct physical abuse (SG3); and encouragement from others related a child to escalate the abuse (SG5).

In this paper we present a lab study on robot abuse by children. 61 Japanese children of ages 7-9 interacted individually with Robovie, a social robot, in a context that promoted children's free disruptive behaviors towards the robot. We compared the robot's use of an adaptation of a parental discipline strategy, the so-called love-withdrawal technique, to a similar set of robot behaviors that lacked any specific strategy (neutral condition). The main insight we gained was that perhaps we should better not focus on general robot behaviors to try to fit all children, but rather, we should adapt the robot behaviors to children's individual differences. For instance, we found that the love-withdrawal-based strategy was significantly more effective in children of age 8-9 than on children of 7.

We modeled a robot's approaching behavior for giving admonishment. We started by analyzing human behaviors. We conducted a data collection in which a guard approached others in two ways: 1) for admonishment, and 2) for a friendly purpose. We analyzed the difference between the admonishing approach and the friendly approach. The approaching trajectories in the two approaching types are similar; nevertheless, there are two subtle differences. First, the admonishing approach is slightly faster (1.3 m/sec) than the friendly approach (1.1 m/sec). Second, at the end of the approach, there is a `shortcut' in the trajectory. We implemented this model of the admonishing approach into a robot. Finally, we conducted a field experiment to verify the effectiveness of the model. A robot is used to admonish people who were using a smartphone while walking. The result shows that significantly more people yield to admonishment from a robot using the proposed method than from a robot using the friendly approach method.

We developed a self-report measurement, Moral Concern for Robots Scale (MCRS), which measures whether people believe that a robot has moral standing, deserves moral care, and merits protection. The results of an online survey ( \pmbN = 200) confirmed the concurrent validity and predictive validity of the scale in the sense that the scale scores are successfully used to predict people's intentions for prosocial behaviors.

We are facilitating software development for mobile social robots that operate `in the wild': that is, in real daily environments such as shopping malls. One fundamental difficulty in development is testing unfinished programs on a real robot and in the presence of people, because this process can take a significantly time-consuming. To ease the software development, we developed a simulator that allows the simulation of interactions among people, and interactions between people and the robot. With a user study, we analyzed how people's working process changed and evaluated the amount of time they saved in software development with the simulator.

A series of studies were conducted as an exploratory survey to examine the possible roles of a robot as a partner in healthcare. The results show that Japanese people are willing to use a humanoid robot as an exercise partner in a variety of usage scenarios. In particular, a walking robot was found to be useful for socially anxious individuals, in that it can serve as a safe partner that is not evaluates them.

This paper presents a robot behavioral model designed to support children during self-study. In particular, we want to investigate how a robot could increase the time children keep concentration. The behavioral model was developed by observing children during self-study and by collecting information from experienced tutors through interviews. After observing the children, we decided to consider three states corresponding to different levels of concentration. The child can be smoothly performing the task (“learning” state), encountering some difficulties (“stuck” state) or distracted (“distracted” state). The behavioral model was designed to increase the time spent concentrating on the task by implementing adequate behaviors for each of these three states. These behaviors were designed using the advices collected during the interview survey of the experienced tutors. A self-study system based on the proposed behavior model was implemented. In this system, a small robot sits on the table and encourages the child during self-study. An operator is in charge of determining the state of the child (Wizard of Oz) and the behavioral model triggers the appropriate behaviors for the different states. To demonstrate the effectiveness of the proposed behavioral model, a user study was conducted: 22 children were asked to solve problems alone and to solve problems with the robot. The children spent significantly (p = 0.024) more time in the “learning” state when studying with the robot.

This paper presents a new method to allow robots to accompany a person or a group of people imitating pedestrians behavior. Two-people groups usually walk in a side-by-side formation and three-people groups walk in a V-formation so that they can see each other. For this reason, the proposed method combines a Side-by-side and V-formation pedestrian model with the Anticipative Kinodynamic Planner (AKP). Combining these methods, the robot is able to do an anticipatory accompaniment of groups of humans, as well as to avoid static and dynamic obstacles in advance, while keeping the prescribed formations. The proposed framework allows also a dynamical re-positioning of the robot, if the physical position of the partners change in the group formation. Furthermore, people have a randomness factor that the robot has to manage, for that reason, the system was adapted to deal with changes in people's velocity, orientation and occlusions. Finally, the method has been validated using synthetic experiments and real-life experiments with our Tibi robot. In addition, a user study has been realized to reveal the social acceptability of the method.

Many recent studies have shown that behaviors and interaction logic for social robots can be learned automatically from natural examples of human-human interaction by machine learning algorithms, with minimal input from human designers [1--4]. In this work, we exceed the capabilities of the previous approaches by giving the robot memory . In earlier work, the robot's actions were decided based only on a narrow temporal window of the current interaction context. However, human behaviors often depend on more temporally distant events in the interaction history. Thus, we raise the question of whether (and how) an automated behavior learning system can learn a memory representation of interaction history within a simulated camera shop scenario. An analysis of the types of memory-setting and memory-dependent actions that occur in the camera shop scenario is presented. Then, to create more examples of such actions for evaluating a shopkeeper robot behavior learning system, an interaction dataset is simulated. A Gated Recurrent Unit (GRU) neural network architecture is applied in the behavior learning system, which learns a memory representation for performing memory-dependent actions. In an offline evaluation, the GRU system significantly outperformed a without-memory baseline system at generating appropriate memory-dependent actions. Finally, an analysis of the GRU architecture's memory representation is presented.

Learning from human interaction data is a promising approach for developing robot interaction logic, but behaviors learned only from offline data simply represent the most frequent interaction patterns in the training data, without any adaptation for individual differences. We developed a robot that incorporates both data-driven and interactive learning. Our robot first learns high-level dialog and spatial behavior patterns from offline examples of human--human interaction. Then, during live interactions, it chooses among appropriate actions according to its curiosity about the customer's expected behavior, continually updating its predictive model to learn and adapt to each individual. In a user study, we found that participants thought the curious robot was significantly more humanlike with respect to repetitiveness and diversity of behavior, more interesting, and better overall in comparison to a non-curious robot.

People engage in social coordination without explicitly communicating when they are conflicting over spatial resources, e.g., a shop clerk who yields to customers the best place to view products. In this study, we proposed a method that achieves such social coordination with a robot. Our idea is that the social coordination between two agents can be represented as utility-maximizing behavior for joint utility rather than just by a single agent utility. That is, given that each agent's reasonable behavior can be represented as utility-maximizing behavior for single agent utility, we model each agent's plans for himself as well as for the partner agent. Moreover, superiority relationships exist in this joint-utility computation. Since each agent knows such superiority relationships, social coordination can be modeled as utility-yielding behavior based on informed superiority. We specifically focus on looking-together situations for which we developed a utility model. With simulations, we investigate whether the above joint-utility-based modeling successfully reproduces social coordination in looking-together situations. We conducted an experiment in a situation where a tele-operated robot and a customer together look at products in a shop environment. Our experimental results show that our proposed method enables the robot to socially coordinate spatial resources, yielding significantly more thoughtful, less-self-centered, and appropriate impressions than the alternate robot.

We aim to develop a walking partner robot with the capability to select small-talk topics that are associative to visual scenes. We first collected video sequences from five different locations and prepared a dataset about small-talk topics associated to visual scenes. Then we developed a technique to associate the visual scenes with the small-talk topics. We converted visual scenes into lists of words using an off-the-shelf vision library and formed a topic space with a Latent Dirichlet Allocation (LDA) method in which a list of words is transformed to a topic vector. Finally, the system selects the most similar utterance in the topic vectors. We tested our developed technique with a dataset, which successfully selected 72% appropriate utterances, and conducted a user study outdoors where participants took a walk with a small robot on their shoulder and engaged in small talk. We confirmed that the participants more highly perceived the robot with our developed technique because it selected appropriate utterances than a robot that randomly selected utterances. Further, they also felt that the former type of robot is a better walking partner.

In this work, we were interested in creating a robot service for offering help to people who appear to be in need for guidance. In order to achieve that, we first developed a technique to detect pedestrians who walk in an atypical way (e.g., people who do not know their way). In our approach, a motion model of typical pedestrians developed in our previous work was used, and a novel predictability feature was defined that quantifies how well can a person's future position be predicted using that model. The classification method based on this feature gave accurate results and outperformed alternative methods. Using this detection method, we created a robot service for offering guidance to people who were classified as atypical. Experiments done in a shopping mall have shown that the robot was successful in choosing the people to approach, and the reactions from users in the interviews were very positive.

This study examined response tendencies in 4-year-old Japanese children (N = 45) to yes-no questions asked by a communicative, or a non-communicative robot. The children watched a video of a robot that was either responsive (communicative condition), or unresponsive (non-communicative condition) to human actions. Then, all the children watched a video of the same robot asking yes-no questions pertaining to familiar and unfamiliar objects. The children in both conditions exhibited a nay-saying bias to questions about unfamiliar objects, with children in the non-communicative condition tending to show a stronger nay-saying bias than children in the communicative condition. Children's response tendencies towards questions asked by humans and other agents are discussed.

Interaction Composer, a visual programming environment designed to enable programmers and non-programmers to collaboratively design social human-robot interactions in the form of state-based flows, has been in use at our laboratory for eight years. The system architecture and the design principles behind the framework have been presented in other work, but in this paper we take a case-study approach, examining several actual examples of the use of this toolkit over an eight-year period. We examine the structure and content of interaction flows, identify common design patterns, and discuss elements of the framework which have proven valuable, features which did not solve their intended purposes, and ways that future systems might better address these issues. It is hoped that the insights gained from this study will contribute to the development of more effective and more usable tools and frameworks for interaction design.

Is it possible to design robots of the future so that they can enhance people's creative endeavors? Forty-eight young adults were asked to produce their most creative ideas in a small Zen rock garden in a laboratory setting. Participants were randomly assigned to one of two conditions. In one condition, the robot Robovie (through a Wizard of Oz interface) encouraged participants to generate creative ideas (e.g., “Can you think of another way to do that?”), and pulled relevant images and video clip from the web for each participant to look at which could help spur the participant into more creative expressions. In a second condition, participants engaged in the same Zen rock garden task with the same core information, but through the modality of a self-paced PowerPoint presentation. Results showed that participants engaged in the creativity task longer and provided almost twice the number of creative expressions in the robot condition compared to the PowerPoint condition. Discussion focuses on a vision of social robotics coupled with advances in natural language processing to enhance the human creative mind and spirit.

We conducted a field study to investigate the social acceptance of social robots by stores, particularly for attracting passersby, which today's robots can autonomously perform. From interviews with ten store managers, we identified two main reasons they want to employ such social robots in their stores: robots offer cheap labor and provide unique value that humans cannot. They believe that robots are good at attracting the attention of visitors without causing or receiving stress. We also conducted three case studies in which we observed how store managers employed social robots in their stores. Each store manager requested different designs in the preparation phase. After deployment, we found that the managers were generally satisfied with the services autonomously offered by the robots, which successfully encouraged people to stop. For two out of three stores the robots successfully encouraged visitors to visit. The store managers were satisfied with the results and expressed a desire to use the robots again.

Many researchers have started to explore natural interaction scenarios for children. No matter if these children are normally developing or have special needs, evaluating Child-Robot Interaction (CRI) is a challenge. To find methods that work well and provide reliable data is difficult, for example because commonly used methods such as questionnaires do not work well particularly with younger children. Previous research has shown that children need support in expressing how they feel about technology. Given this, researchers often choose time-consuming behavioral measures from observations to evaluate CRI. However, these are not necessarily comparable between studies and robots.

Recent studies in human-robot interaction (HRI) have investigated ways to harness the power of the crowd for the purpose of creating robot interaction logic through games and teleoperation interfaces. Sensor networks capable of observing human-human interactions in the real world provide a potentially valuable and scalable source of interaction data that can be used for designing robot behavior. To that end, we present here a fully automated method for reproducing observed real-world social interactions with a robot. The proposed method includes techniques for characterizing the speech and locomotion observed in training interactions, using clustering to identify typical behavior elements and identifying spatial formations using established HRI proxemics models. Behavior logic is learned based on discretized actions captured from the sensor data stream, using a naïve Bayesian classifier. Finally, we propose techniques for reproducing robot speech and locomotion behaviors in a robust way, despite the natural variation of human behaviors and the large amount of sensor noise present in speech recognition. We show our technique in use, training a robot to play the role of a shop clerk in a simple camera shop scenario, and we demonstrate through a comparison experiment that our techniques successfully enabled the generation of socially appropriate speech and locomotion behavior. Notably, the performance of our technique in terms of correct behavior selection was found to be higher than the success rate of speech recognition, indicating its robustness to sensor noise.

Human infants can consider humans as collaborative agents, but little is known about whether they also recognize robots as collaborative agents. This study investigated that how infants understand robot agents while they watched collaborative interactions between a human and a robot. We presented a novel visual habituation paradigm in which a human and a robot performed a collaborative activity to 13-month-old infants. Our findings suggested that 13-month-olds can appreciate robots as collaborative partners. We interpreted infants' expectancy violation responses to actions of the robot as facilitating their understanding about nonhuman agents as social partner.

Social robots working in public space often stimulate children's curiosity. However, sometimes children also show abusive behavior toward robots. In our case studies, we observed in many cases that children persistently obstruct the robot's activity. Some actually abused the robot by saying bad things, and at times even kicking or punching the robot. We developed a statistical model of occurrence of children's abuse. Using this model together with a simulator of pedestrian behavior, we enabled the robot to predict the possibility of an abuse situation and escape before it happens. We demonstrated that with the model the robot successfully lowered the occurrence of abuse in a real shopping mall.

Will people keep the secret of a socially compelling robot who shares, in confidence, a "personal" (robot) failing? Toward answering this question, 81 adults participated in a 20-minute interaction with (a) a humanoid robot (Robovie) interacting in a highly social way as a lab tour guide, and (b) with a human being interacting in the same highly social way. As a baseline comparison, participants also interacted with (c) a humanoid robot (Robovie) interacting in a more rudimentary social way. In each condition, the tour guide asks for the secret keeping behavior. Results showed that the majority of the participants (59%) kept the secret of the highly social robot, and did not tell the experimenter when asked directly, with the robot present. This percentage did not differ statistically from the percentage who kept the human's secret (67%). It did differ statistically when the robot engaged in the more rudimentary social interaction (11%). These results suggest that as humanoid robots become increasingly social in their interaction, that people will form increasingly intimate and trusting psychological relationships with them. Discussion focuses on design principles (how to engender psychological intimacy in human-robot interaction) and norms (whether it is even desirable to do so, and if so in what contexts).

When should service staff initiate interaction with a visitor? Neither simply-proactive (e.g. talk to everyone in a sight) nor passive (e.g. wait until being talked to) strategies are desired. This paper reports our modeling of polite approaching behavior. In a shopping mall, there are service staff members who politely approach visitors who need help. Our analysis revealed that staff members are sensitive to "intentions" of nearby visitors. That is, when a visitor intends to talk to a staff member and starts to approach, the staff member also walks a few steps toward the visitors in advance to being talked. Further, even when not being approached, staff members exhibit "availability" behavior in the case that a visitor's intention seems uncertain. We modeled these behaviors that are adaptive to pedestrians' intentions, occurred prior to initiation of conversation. The model was implemented into a robot and tested in a real shopping mall. The experiment confirmed that the proposed method is less intrusive to pedestrians, and that our robot successfully initiated interaction with pedestrians.

We developed an information-providing robot for a shopping mall. It moves around and hands over flyers about the mall and when people are interested or approach the robot to obtain some information, it starts the information-providing service. During the information-providing service, it provides directions and recommendations of shops in the mall. The robot is developed to be fully autonomous, but due to low performance in automatic speech recognition, this function is overtaken by a human operator. In our field trial, the robot worked for 53.4 hours (over 11 days) without any major failure (83.5% of time autonomously operated). We conducted interviews with visitors, and revealed that the majority of them found the information from the robot useful, and wanted to use it again.

Knowledge about space usage from variables such as density and walking speed could support a variety of service applications. However, there is not much knowledge on how the usage of space changes during extended periods of time and what affects the changes. We have installed a person tracking system in a large area of a shopping center and collected pedestrian data over a year. In this paper, we analyze the collected data to find the changes in pedestrian density and speed, percentage of children, and pedestrian trajectories. The changes from day to day, as well as during the day are examined, and a number of factors that affect them are identified. This is in turn used in the prediction of the state of the space using a Gaussian process model.

Social robots working among pedestrians can attract crowds of people around them and consequently become bothersome entities causing congestion in narrow spaces. This in turn can affect the comfort of pedestrians who wish to pass through. To address this problem, our idea is to endow the robot with three capabilities: anticipating pedestrian crowding around the robot, understanding pedestrians' walking comfort, and planning to avoid congestions in advance. Combining several elementary pedestrian behavior models, the robot is able to simulate hypothetical situations where it navigates between pedestrians and anticipate the degree to which this would affect the pedestrians' walking comfort. During planning, the robot determines the best next navigation step based on the results of simulations. We tested the developed system in a real shopping mall and confirmed that it successfully reduces the robot's influence on pedestrian walking comfort due to congestions.

This study investigates whether a social robot which interacts with children via quiz-style conversations increases their understanding of science classes. We installed a social robot in an elementary school science classroom where children could freely interact with it during their breaks. The robot asks children questions related to their latest science classes to support their understanding of the classes. During interaction, the robot says children's name and distribute its gaze among the group of children by using a face recognition system and a human tracking system. Still, speech recognition is difficult in the noisy elementary school environment; therefore the operator takes over this function during interactions. In this study our result did not show significant effects of the robot for helping children's understanding, but we found several interesting interaction scenes which shows that robot had a certain effect on specific children.

An online survey for 1,200 Japanese participants from 20's to 50's was conducted to clarify what type of robot and under what situation humans expect rapport with, and explore human factors influential into these expectations. The survey was based on hypothetical situation method consisting of three situations: a vacuum robot, a pet-type robot, and a robot instructor. The results suggested that; 1) expectations of rapport with robots were dependent on types of robots and application contexts, and were directly not affected by experiences of robots; 2) these expectations were influenced by negative attitudes toward social influences of and emotional interaction with robots, and emotional sensitivity for others; 3) expectations of rapport with robots were influenced by different psychological factors dependent on robot types and application contexts.

The unique social presence of robots can be leveraged in learning situations to reduce student evaluation anxiety, while still providing instructional guidance on multiple levels of communication. Furthermore, social role of the instructor can also impact the prevalence of evaluation apprehension. In this study, we examine how human and robot social role affects help-seeking behaviors and learning outcomes in a one-on-one tutoring setting. Our results show that help-seeking is a moderator of the significant relationship between condition and learning, with the "human teacher" condition resulting in significantly less learning (and marginally less help-seeking) than the "human assistant" and both robot conditions.

This paper shows how humor can be used as an interaction pattern to help establish sociality in human-robot interaction. Drawing illustratively from our published research on people interacting with ATR's humanoid robot Robovie, we highlight four forms of humor that we successfully implemented: wit and the ice-breaker, corny joke, subtle humor, and dry humor and self-deprecation.

We offer three illustrative examples from one of our recent studies in HRI to suggest that it's possible for people to engage in mutual deliberation with a social robot. Each example illustrates discourse and argument, but ends through different means: Accepting the Robot as Arbitrator; Delegitimizing the Robot; and Agreeing to Disagree.

Walking side by side is a common situation when we go from one place to another with another person while talking. Our previous study reported a basic mechanism for side-by-side walking, but in the previous model it was crucial that each agent knew where he or she was going, i.e. the route to the destination. However, we have recognized the need to model the situation where one of the agents does not know the destination. The extended model presented in this work has two states: leader-follower state and collaborative state. Depending on whether the follower agent has obtained a reliable estimate of the route to follow, the walking agents transition between the two states. The model is calibrated with trajectories acquired from pairs of people walking side by side, and then it is tested in a human-robot interaction scenario. The results demonstrate that the new extended model achieves better side-by-side performance than a standard method without knowledge of the subgoal.

This paper models the concept of the "territory" of shops. First, we interviewed three shopkeepers and found that they perceived the space near their shop as their territory and that they interpreted some types of behaviors as invasive. Second, we confirmed that potential visitors share this notion of territory. We also confirmed that the size of the territory depends on the characteristics of a shop's facade. While there is little territory in front of walls, there is more territory in front of shelves and entrances. Our robot traversed around two real shopping malls that included 50 shops and took 3-D scans of their environment shapes. Each shop's facade was analyzed and the shop territory was computed. The computation results match people's perception. The recognition rate accuracy reached 93.5% for the territory areas. User evaluations in a virtual shop environment confirmed that a robot with a territory model behaves better than one without it.

This research builds on the UbiComp vision of systems that do not do things for people but engage people in their computational environment so that people can do things for themselves better. In this investigation, we sought to make good on a proof-of-concept where people interact with a social robot whereby the robot helps people to be more humanly creative. Twenty seven participants interacted with ATR's humanoid robot Robovie (through a WoZ interface) in a creativity task. Results supported our proof of concept insofar as 100% of the participants generated creative ideas, and 63% incorporated the robot's ideas into their own ideas for their creative output. Of the participants who had the highest creativity scores, 83% incorporated the robot's ideas into their own. Discussion focuses on next steps toward building the Natural Language Processing system, and integrating the system into a more extensive networked UbiComp environment.

Developing interactive behaviors for social robots presents a number of challenges. It is difficult to interpret the meaning of the details of people's behavior, particularly non-verbal behavior like body positioning, but yet a social robot needs to be contingent to such subtle behaviors. It needs to generate utterances and non-verbal behavior with good timing and coordination. The rules for such behavior are often based on implicit knowledge and thus difficult for a designer to describe or program explicitly. We propose to teach such behaviors to a robot with a learning-by-demonstration approach, using recorded human-human interaction data to identify both the behaviors the robot should perform and the social cues it should respond to. In this study, we present a fully unsupervised approach that uses abstraction and clustering to identify behavior elements and joint interaction states, which are used in a variable-length Markov model predictor to generate socially-appropriate behavior commands for a robot. The proposed technique provides encouraging results despite high amounts of sensor noise, especially in speech recognition. We demonstrate our system with a robot in a shopping scenario.

This paper presents a computational model for side-by-side walking within human-robot interaction (HRI). In this work we address the importance of future motion utility (motion anticipation) of two walking partners. Previous studies only considered a robot moving alongside a person without collisions and with simple velocity-based predictions. In contrast, our proposed model includes two major considerations. First, it considers the current goal, modeling side-by-side walking as a process of moving toward a goal while maintaining a relative position with the partner. Second, it takes the partner's utility into consideration; it models side-by-side walking as a phenomenon where two agents maximize mutual utilities rather than only considering a single agent utility. The model is constructed based in a set of trajectories from pairs of people recorded in side-by-side walking; then, parameters of the model were calibrated for a mobile robot and tested in an autonomous robot walking side-by-side with participants. Finally, two evaluations were performed. The first evaluation shows that the proposed model considering mutual utilities performs better than a single utility method and a method that keeps distance from the walking partner. In the second evaluation the proposed method was used for a robot deployed in a shopping mall environment where it demonstrated to be effective.

The development of humanlike service robots which interact socially raises a new question: How can we make good interaction content for such robots? Domain experts specializing in the target service have the knowledge for making such content. Yet, while they can easily engage in good face-to-face interactions, we found it difficult for them to prepare conversational content for a robot in written form. Instead, we propose involving experts as teleoperators in an iterative development process in which the expert develops content, teleoperates a robot using that content, and then revises the content based on that interaction. We propose a software system and design guidelines to enable such an iterative design process. To validate these solutions, we conducted a comparison experiment in the field, with elderly volunteer guides teleoperating a robot at a tourist information center in Nara, Japan. The results showed that our system and guidelines enabled domain experts with no robotics background to create better interaction content and conduct better interactions than domain experts without our system.

A method for tracking the position, orientation, and height of persons in large public environments is presented. Such a piece of information is known to be useful both for understanding their actions, as well as for applications such as human-robot interaction. We use multiple 3-D range sensors, which are mounted above human height to have less occlusion between persons. A computationally simple-tracking method is proposed that works on single sensor data and combines multiple sensors so that large areas can be covered with a minimum number of sensors. Moreover, it can work with different sensor types and is robust to the imperfect sensor measurements; therefore, it is possible to combine currently available 3-D range sensor solutions to achieve tracking in wide public spaces. The method was implemented in a shopping center environment, and it was shown that good tracking performance can be achieved.

Human-Robot Interaction (HRI) systems are often realized through a complex integration of relevant state-of-the-art algorithms, low-level software and robotic hardware. As such, advances in techniques and approaches for integrating components into a robotic system are essential to advance the field. To develop such techniques and systems requires one to address real-world challenges, particularly those arising from the complex nature of human-robot interaction embedded in a dynamic social and/or task environment. This special issue focuses on the concept of "systems", highlighting the importance of building actual HRI systems to validate and drive the science and technology in the field.

When robots serve in urban areas such as shopping malls, they will often be required to approach people in order to initiate service. This paper presents a technique for human-robot interaction that enables a robot to approach people who are passing through an environment. For successful approach, our proposed planner first searches for a target person at public distance zones anticipating his/her future position and behavior. It chooses a person who does not seem busy and can be reached from a frontal direction. Once the robot successfully approaches the person within the social distance zone, it identifies the person's reaction and provides a timely response by coordinating its body orientation. The system was tested in a shopping mall and compared with a simple approaching method. The result demonstrates a significant improvement in approaching performance; the simple method was only 35.1% successful, whereas the proposed technique showed a success rate of 55.9%.

The teleoperation of mobile social robots requires operators to understand facial gestures and other nonverbal communication from a person interacting with the robot. It is also critical for the operator to comprehend the surrounding environment in order to facilitate both navigation and human-robot interaction. Allowing the operator to control the robot's gaze direction can help the operator observe a person's nonverbal communication; however, manually actuating a gaze increases the operator's workload and conflicts with the use of the robot's camera for navigation. To address these problems, the authors developed a teleoperation system that combines automatic control of the robot's gaze and a 3-D graphical representation of the surrounding environment, such as location of items and configuration of a shop. A study where a robot plays the role of a shopkeeper was conducted to validate the effectiveness of the proposed gaze-control technique and control interface. It was demonstrated that the combination of automatic gaze control and representations of spatial relationships improved the quality of the robot's interaction with the customer.

This study provides an in-depth analysis and practical solution to the problem of designing and implementing a human-robot team for simple conversational interactions. Models for operation timing, customer satisfaction and customer-robot interaction are presented, based on which a simulation tool is developed to estimate fan-out and robot team performance. Techniques for managing interaction flow and operator task assignment are introduced. In simulation, the effectiveness of different techniques and factors related to team performance are studied. A case study on deploying multiple robots in a shopping mall is then presented to demonstrate the usefulness of our study in helping the design and implementation of social robots in real-world settings.

Robots will increasingly take on roles in our social lives where they can cause humans harm. When robots do so, will people hold robots morally accountable? To investigate this question, 40 undergraduate students individually engaged in a 15-minute interaction with ATR's humanoid robot, Robovie. The interaction culminated in a situation where Robovie incorrectly assessed the participant's performance in a game, and prevented the participant from winning a $20 prize. Each participant was then interviewed in a 50-minute session. Results showed that all of the participants engaged socially with Robovie, and many of them conceptualized Robovie as having mental/emotional and social attributes. Sixty-five percent of the participants attributed some level of moral accountability to Robovie. Statistically, participants held Robovie less accountable than they would a human, but more accountable than they would a vending machine. Results are discussed in terms of the New Ontological Category Hypothesis and robotic warfare.

We used a social robot as a teaching assistant in a class for children's collaborative learning. In the class, a group of 6th graders learned together using Lego Mindstorms. The class consisted of seven lessons with Robovie, a social robot, followed by one lesson to test their individual achievement. Robovie managed the class and explained how to use Lego Mindstorms. In addition to such basic management behaviors for the class, we prepared social behaviors for building relationships with the children and encouraging them. The result shows that the social behavior encouraged children to work more in the first two lessons, but did not affect them in later lessons. On the other hand, social behavior contributed to building relationships and attaining better social acceptance.

We aim to develop a shopping companion robot that can share experience with users. In this study, we focused on the shared memory acquired when a robot walks together with a user. We developed a computational model of memory recall of visited locations in a shopping mall. The model was developed with data collection from 30 participants. We found that shop size, color intensity of facade, relative visibility, and time elapsed are the influencing features for recall. The model was used in a scenario of a shopping companion robot. The robot, Robovie, autonomously follows a user while inferring the user's memory recall of shops in the visited route. When the user asks the location of other shops, Robovie replied with destination description, referring to the known locations inferred with the model of the user's memory recall. With this scenario, we verified the effectiveness of the developed computational model of memory recall. The evaluation experiment revealed that the model outputs shops that the participants are likely to recall, and makes the directions given easier to understand.

This paper presents a computational model for side-by-side walking for human-robot interaction (HRI). In this work we address the importance of future motion utility (motion anticipation) of the two walking partners.

To investigate the effects of robots on human self-assessment under the more general contexts of learning, this research focused on the evaluative feedback by robots. A psychological experiment was conducted by using a human-sized humanoid robot. The results found that the evaluative context tended to increase the subjects' negative responses regarding disclosure of their skills to the robot and the robot evaluating the skills, and the internal locus of control and the fear of negative evaluation positively influenced the subjects' responses to disclosing their skills to the robot. This paper discusses the implications on the use of robots in self-monitoring for healthcare and education.

During conversations, speakers employ a number of verbal and nonverbal mechanisms to establish who participates in the conversation, when, and in what capacity. Gaze cues and mechanisms are particularly instrumental in establishing the participant roles of interlocutors, managing speaker turns, and signaling discourse structure. If humanlike robots are to have fluent conversations with people, they will need to use these gaze mechanisms effectively. The current work investigates people's use of key conversational gaze mechanisms, how they might be designed for and implemented in humanlike robots, and whether these signals effectively shape human-robot conversations. We focus particularly on whether humanlike gaze mechanisms might help robots signal different participant roles, manage turn-exchanges, and shape how interlocutors perceive the robot and the conversation. The evaluation of these mechanisms involved 36 trials of three-party human-robot conversations. In these trials, the robot used gaze mechanisms to signal to its conversational partners their roles either of two addressees, an addressee and a bystander, or an addressee and a nonparticipant. Results showed that participants conformed to these intended roles 97% of the time. Their conversational roles affected their rapport with the robot, feelings of groupness with their conversational partners, and attention to the task.

Teleoperation of multiple robots by a single operator has been studied extensively for applications such as search and navigation; however, this concept has never been applied to the field of social, conversational robots. In this paper, we explore the unique challenges posed by the remote operation of multiple social robots, where an operator must perform auditory multitasking to assist multiple interactions at once. It describes the general system requirements in four areas: social human-robot interaction (HRI) design, autonomy design, multirobot coordination, and teleoperation interface design. Based on this design framework, we have developed a system in which a single operator can simultaneously control four robots in conversational interactions with users. Key elements of our implementation include a control architecture enabling the scripting of conditional behavior flows for social interaction, a graphical interface enabling an operator to control one robot at a time while monitoring several others in the background, and a technique called “proactive timing control,” which is an automated method for smoothly interleaving the demands of multiple robots for the operator's attention. We also present metrics for describing and predicting robot performance, and we show experimental results demonstrating the effectiveness of our system through simulations and a laboratory experiment based on real-world interactions.

Awareness of time is particularly important for teleoperation of conversational robots, both for controlling the robot and for estimating interaction success, because people have a low tolerance for long pauses in conversation. Findings have shown that people engaged in high-workload tasks tend to underestimate the passage of time. This study confirms that this problem exists for operators controlling a conversational robot, and it investigates mechanisms for improving temporal awareness and task performance while minimizing workload. In a laboratory experiment, two approaches to helping an operator perform various information input tasks were compared: first, assisting temporal awareness by using a clock display, and second, using autonomy to assist one of the operator's time-dependent tasks. Results revealed that assisting the task itself, even without the clock, improved not only task performance but also the operator's temporal awareness. However, results regarding the effect of the clock were ambiguous: it increased workload in general and did not help temporal awareness overall, but it did improve temporal awareness for text entry tasks in particular. As text entry is an important task for teleoperation of social robots, we further investigated the problem of improving temporal awareness during text entry tasks. As the first experiment suggested the effectiveness of a clock, we further validated that the clock is specifically useful to improve temporal awareness. These results showed that the clock did not increase workload for text entry tasks; however, for touch-typing operators, the results suggested that showing a clock after the end of an interaction, rather than continuously throughout the task, could lower the operator's perceived workload.

Assistive robots can be perceived in two main ways: tools or partners. In past research, assistive robots that offer physical assistance for the elderly are often designed in the context of a tool metaphor. This paper investigates the effect of two design considerations for assistive robots in a partner metaphor: conversation and robot-type. The former factor is concerned with whether robots should converse with people even if the conversation is not germane for completing the task. The latter factor is concerned with whether people prefer a communication/function oriented design for assistive robots. To test these design considerations, we selected a shopping assistance situation where a robot carries a shopping basket for elderly people, which is one typical scenario used for assistive robots. A field experiment was conducted in a real supermarket in Japan where 24 elderly participants shopped with robots. The experimental results revealed that they prefer a conversational humanoid as a shopping assistant partner.

This paper discusses converging evidence to support the hypothesis that personified robots and other embodied personified computational systems may represent a new ontological category, where ontology refers to basic categories of being, and ways of distinguishing them.

No abstract available.

This paper reports an ethnographic study on the use of a conversational robot. We placed a robot for 3.5 months in an elderly care center. Assuming a real deployment scenario, the robot was managed by a single non-programmer person during the field trial, who teleoperated the robot and updated the contents. The robot was designed to engage in daily greetings and chatting with elderly people. Through the ethnographic approach, we clarified how the elderly people interacted with this conversational robot, how the deployment process adopted to introduce the robot was designed, and how the organization's personnel involved themselves in this deployment.

Environment attributes are perceived or remembered differently according to the perspective used. In this study, two different perspectives, a survey perspective and a route perspective, are explained and discussed. This paper proposes an approach for modeling human environments from a route perspective, which is the perspective used when a human navigates through the environment. The process for route perspective semi-autonomous data extraction and modeling by a mobile robot equipped with a laser sensor and a camera is detailed. Finally, as an example of a route perspective application, a route direction robot was developed and tested in a real mall environment. Experimental results show the advantages of the proposed route perspective model compared with a survey perspective approach. Moreover, the route model is comparable to the performance of an expert person giving route guidance in the mall.

This study explores the nature of the multi-robot control problem for social robots. It begins by modeling the overall structure of a human-robot team for social interactions, and implements it for specific applications to dialog-based interactions. Operator activity during control of a social robot is studied. Customer satisfaction is proposed as an important metric for evaluating the performance of a human-robot team for social interactions with customers. Based on the modeling, fan-out of a social robot team can be calculated, and the performance of the team is estimated by simulation. A field trial was conducted in a shopping mall to demonstrate a successful deployment of social robots for a real-world application with ensured performance prior to installation using our modeling and simulation approach.

In daily conversation, we sometimes observe a deictic interaction scene that refers to a region in a space, such as saying "please put it over there" with pointing. How can such an interaction be possible with a robot? Is it enough to simulate people's behaviors, such as utterance and pointing? Instead, we highlight the importance of simulating human cognition. In the first part of our study, we empirically demonstrate the importance of simulating human cognition of regions when a robot engages in a deictic interaction by referring to a region in a space. The experiments indicate that a robot with simulated cognition of regions improves efficiency of its deictic interaction. In the second part, we present a method for a robot to computationally simulate cognition of regions.

In recent work, "interaction patterns" have been proposed as a means to characterize essential features of human-robot interaction. A problem arises, however, in knowing whether the interaction patterns generated are valid. The same problem arises when researchers in HRI propose other broad conceptualizations that seek to structure social interaction. In this paper, we address this general problem by distinguishing three ways of establishing the validity of interaction patterns. The first form of validity seeks to establish whether the conclusions about interaction patterns are warranted from the data. The second seeks to establish whether the interaction patterns account for the data. And the third seeks to provide sound reasons for the labels of the patterns themselves. Often these three forms of validity are confused in discussions about conceptual categories in HRI.

This conceptual paper broaches possibilities and limits of establishing psychological intimacy in HRI.

Tour guidance is a common task of social robots. Such a robot must be able to encourage the participation of people who are not directly interacting with it. We are particularly interested in encouraging people to overhear its interaction with others, since it has often been observed that even people who hesitate to interact with a robot are willing to observe its activity. To encourage such participation as bystanders, we developed a robot that walks backwards based on observations of human tour guides. Our developed system uses a robust human tracking system that enables a robot to guide people by walking forward/backward and allows us to scrutinize people's behavior after the experiment. We conducted a field experiment to compare the ratios of overhearing in "walking forward" and "walking backward." The experimental results revealed that in fact people do more often overhear the robot's conversation in the "walking backward" condition.

People like to play, and robotic technology offers the opportunity to interact with artifacts in new ways. Robots co-existing with humans in domestic and public environments are expected to behave as companions, also engaging in playful interaction. If a robot is small, we foresee that people will want to be able to pick it up and express their intentions playfully by hugging, shaking and moving it around in various ways. Such robots will need to recognize these gestures--which we call “full-body gestures” because they affect the robot's full body. Inertial sensors inside the robot could be used to detect these gestures, in order to avoid having to rely on external sensors in the environment. However, it is not obvious which gestures typically occur during play, and which of these can be reliably detected. We therefore investigate full-body gesture recognition using Sponge Robot, a small humanoid robot equipped with inertial sensors and designed for playful human-robot interaction.

Due to the anticipated future, extensive use of robots, human beings will probably share common spaces with them. The relationships between robots and humans will be conducted at close distances. Predicting people's future positions helps robots understand human behavior and react safely and naturally. In this paper, we propose a method for predicting people's positions in crossing behaviors, i.e. different trajectories people follow when they are crossing each other. We conducted a field experiment to gather various crossing behaviors of pedestrians in a shopping mall environment and analyzed them by focusing on “hot areas” spaces where people modify their trajectories for crossing. We clustered typical crossing behaviors in hot areas and modeled them using Hidden Markov Models for predictions. Our algorithm more accurately predicts the future positions of pedestrians by considering moving direction and speed.

This paper presents a development framework for social robots with which developers can easily prepare communicative behaviors based on tag-based sentences. Previous literature in human-robot interaction has revealed various useful non-verbal behaviors. But for developers, integrating such a large amount of non-verbal behaviors each time they build a social robot is not realistic. The more repertory of non-verbal behaviors acquire, the larger is the burden faced by developers to implement the non-verbal behaviors. Our software, however, only requires developers to prepare sentences with simple markup language for controlling explicit non-verbal behaviors and utterances. The backend system analyzes the tag input and adds implicit non-verbal behaviors accordingly. With the help of previous literature, the system is equipped with various communicative behaviors, so that the robot autonomously adjusts its gazes, gestures, and standing points. The system's effectiveness is demonstrated with examples on robot interaction where the system receives only simple sentences with scripting language to produce complex and communicative robot behaviors.

We present a study on people's acceptance of robots performing tasks in a city. Three different entities (a human, a human in the costume of a mascot, and a robot) performed tasks in three different scenarios: endlessly guiding, responding to irrational complaints, and picking an accidentally discarded key out of a trash. All of these tasks involve entities interacting with visitors in situations that are troublesome, i.e. dull, stressful, and dirty, respectively. For this paper, 24 participants watched 9 videos (three tasks performed by three entities) and evaluated the appropriateness of each entity for the task and the human-likeness of the entity. The results indicate that people prefer to have a robot rather than a human perform these troublesome tasks, even though these tasks require much interaction with people. In addition, the comparison with a human in costume enables us to further speculate that people's expectations of an entity deserving human rights rather than being human-like in appearance or cognitive capability would be a reason for their judgments regarding appropriateness.

This paper reports our development of a communication robot for use in a shopping mall to provide shopping information, offer route guidance, and build rapport. In the development, the major difficulties included sensing human behaviors, conversation in a noisy daily environment, and the needs of unexpected miscellaneous knowledge in the conversation. We chose a network-robot system approach, where a single robot's poor sensing capability and knowledge are supplemented by ubiquitous sensors and a human operator. The developed robot system detects a person with floor sensors to initiate interaction, identifies individuals with radio-frequency identification (RFID) tags, gives shopping information while chatting, and provides route guidance with deictic gestures. The robot was partially teleoperated to avoid the difficulty of speech recognition as well as to furnish a new kind of knowledge that only humans can flexibly provide. The information supplied by a human operator was later used to increase the robot's autonomy. For 25 days in a shopping mall, we conducted a field trial and gathered 2642 interactions. A total of 235 participants signed up to use RFID tags and, later, provided questionnaire responses. The questionnaire results are promising in terms of the visitors' perceived acceptability as well as the encouragement of their shopping activities. The results of the teleoperation analysis revealed that the amount of teleoperation gradually decreased, which is also promising.

In this paper, we report a model that allows a robot to appropriately control its position as it presents information to a user. This capability is indispensable, since in the future, many robots will function in daily situations such as shopkeepers presenting products to customers or museum guides presenting information to visitors. Psychology research suggests that people adjust their positions to establish a joint view toward a target object. Similarly, when a robot presents an object, it should stand at an appropriate position that considers the positions of both the listener and the object to optimize the listener's field of view and establish a joint view. We observed human-human interaction situations, where people presented objects, and developed a model for an information-presenting robot to appropriately adjust its position. Our model consists of four constraints to establish O-space: 1) proximity to listener; 2) proximity to object; 3) listener's field of view; and 4) presenter's field of view. We also experimentally evaluate the effectiveness of our model.

Simultaneous teleoperation of mobile, social robots presents unique challenges, combining the real-time demands of conversation with the prioritized scheduling of navigational tasks. We have developed a system in which a single operator can effectively control four mobile robots performing both conversation and navigation. We compare the teleoperation requirements for mobile, social robots with those of traditional robot systems, and we identify metrics for evaluating task difficulty and operator performance for teleoperation of mobile social robots. As a proof of concept, we present an integrated priority model combining real-time conversational demands and non-real-time navigational demands for operator attention, and in a pioneering study, we apply the model and metrics in a demonstration of our multi-robot system through real-world field trials in a shopping arcade.

To explore possible robot tasks in daily life, we developed a guide robot for a shopping mall and conducted a field trial with it. The robot was designed to interact naturally with customers and to affectively provide shopping information. It was also designed to repeatedly interact with people to build a rapport; since a shopping mall is a place people repeatedly visit, it provides the chance to explicitly design a robot for multiple interactions. For this capability, we used RFID tags for person identification. The robot was semi-autonomous, partially controlled by a human operator, to cope with the difficulty of speech recognition in a real environment and to handle unexpected situations.

During conversations, speakers establish their and others' participant roles (who participates in the conversation and in what capacity)--or "footing" as termed by Goffman-using gaze cues. In this paper, we study how a robot can establish the participant roles of its conversational partners using these cues. We designed a set of gaze behaviors for Robovie to signal three kinds of participant roles: addressee, bystander, and overhearer. We evaluated our design in a controlled laboratory experiment with 72 subjects in 36 trials. In three conditions, the robot signaled to two subjects, only by means of gaze, the roles of (1) two addressees, (2) an addressee and a bystander, or (3) an addressee and an overhearer. Behavioral measures showed that subjects' participation behavior conformed to the roles that the robot communicated to them. In subjective evaluations, significant differences were observed in feelings of groupness between addressees and others and liking between overhearers and others. Participation in the conversation did not affect task performance-measured by recall of information presented by the robot-but affected subjects' ratings of how much they attended to the task.

Human communication involves a number of nonverbal cues that are seemingly unintentional, unconscious, and automatic-both in their production and perception-and convey rich information on the emotional state and intentions of an individual. One family of such cues is called "nonverbal leakage." In this paper, we explore whether people can read nonverbal leakage cues-particularly gaze cues-in humanlike robots and make inferences on robots' intentions, and whether the physical design of the robot affects these inferences. We designed a gaze cue for Geminoid-a highly humanlike android-and Robovie-a robot with stylized, abstract humanlike features-that allowed the robots to "leak" information on what they might have in mind. In a controlled laboratory experiment, we asked participants to play a game of guessing with either of the robots and evaluated how the gaze cue affected participants' task performance. We found that the gaze cue did, in fact, lead to better performance, from which we infer that the cue led to attributions of mental states and intentionality. Our results have implications for robot design, particularly for designing expression of intentionality, and for our understanding of how people respond to human social cues when they are enacted by robots.

No abstract available.

Providing route directions is a complicated interaction. Utterances are combined with gestures and pronounced with appropriate timing. This study proposes a model for a robot that generates route directions by integrating three important crucial elements: utterances, gestures, and timing. Two research questions must be answered in this modeling process. First, is it useful to let robot perform gesture even though the information conveyed by the gesture is given by utterance as well? Second, is it useful to implement the timing at which humans speaks? Many previous studies about the natural behavior of computers and robots have learned from human speakers, such as gestures and speech timing. However, our approach is different from such previous studies. We emphasized the listener's perspective. Gestures were designed based on the usefulness, although we were influenced by the basic structure of human gestures. Timing was not based on how humans speak, but modeled from how they listen. The experimental result demonstrated the effectiveness of our approach, not only for task efficiency but also for perceived naturalness.

This paper proposes a model of approach behavior with which a robot can initiate conversation with people who are walking. We developed the model by learning from the failures in a simplistic approach behavior used in a real shopping mall. Sometimes people were unaware of the robot's presence, even when it spoke to them. Sometimes, people were not sure whether the robot was really trying to start a conversation, and they did not start talking with it even though they displayed interest. To prevent such failures, our model includes the following functions: predicting the walking behavior of people, choosing a target person, planning its approaching path, and nonverbally indicating its intention to initiate a conversation. The approach model was implemented and used in a real shopping mall. The field trial demonstrated that our model significantly improves the robot's performance in initiating conversations.

In this paper, we present our current research on developing a model of robot behavior that leads to feelings of being together using the robot's body position and orientation. Creating feelings of"being together"will be an essential skill for robots that live with humans and adapt to daily human activities such as walking together or establishing joint attention to information in the environment. We observe people's proxemic behavior in joint attention situations and develop a model of behavior for robots to detect a partner's attention shift and appropriately adjust its body position and orientation in establishing joint attention with the partner. We experimentally evaluate the effectiveness of our model, and our results demonstrate the model's effectiveness.

Social interactions unfold over time, at multiple time scales, and can be observed through multiple sensory modalities. In this paper, we propose a machine learning framework for selecting and combining low-level sensory features from different modalities to produce high-level characterizations of human-robot social interactions in real-time.

Robust localization of robots and reliable tracking of people are both critical requirements for the deployment of service robots in real-world environments. In crowded public spaces, occlusions can impede localization using on-board sensors. At the same time, teams of service robots working together need to share the locations of people and other robots on the same global coordinate system in order to provide services efficiently. To solve this problem, our approach is to use an infrastructure of sensors embedded in the environment to provide an inertial reference frame and wide-area coverage. Based on a people-tracking system we have previously established which uses laser range finders to track people's trajectories, we have developed a technique to localize a team of service robots on a shared global coordinate system. Each robot's odometry data is associated with the observed trajectory of an entity detected by the laser tracking system, and Kalman filters are used to correct rotational offsets between the robots' individual coordinate systems and the global reference frame. We present our data association and pose correction algorithms and show results demonstrating the performance of our system in a shopping arcade.

This paper reports the challenges of developing multiple social robots that operate in a shopping mall. We developed a networked robot system that coordinates multiple social robots and sensors to provide efficient service to customers. It directs the tasks of robots based on their positions and people's walking behavior, manages the paths of robots, and coordinates the conversation-performance between two robots. Laser range finders were distributed in the environment to estimate people's positions. The system estimates such human walking behaviors as ¿stopping¿ or ¿idle walking¿ to direct robots to provide appropriate tasks to appropriate people. Each robot interacts with people to provide recommendation information and route information about shops. The system sometimes simultaneously uses two robots to lead people from one place to another. The field trial, which was conducted in a shopping mall where four robots interacted with 414 people, revealed the effectiveness of the network robot system for guiding people around a shopping mall as well as increasing their interest.

The Uncanny Valley hypothesis has been widely used in the areas of computer graphics and human-robot interaction to motivate research and to explain the negative impressions that participants report after exposure to highly realistic characters or robots. Despite its frequent use, empirical proof for the hypothesis remains scarce. This study empirically tested two predictions of the hypothesis: a) highly realistic robots are liked less than real humans and b) the highly realistic robot's movement decreases its likeability. The results do not support these hypotheses and hence expose a considerable weakness in the Uncanny Valley hypothesis. Anthropomorphism and likeability may be multi-dimensional constructs that cannot be projected into a two-dimensional space. We speculate that the hypothesis' popularity may stem from the explanatory escape route it offers to the developers of characters and robots. In any case, the Uncanny Valley hypothesis should no longer be used to hold back the development of highly realistic androids.

The first robotic copies of real humans have become available. They enable their users to be physically present in multiple locations at the same time. This study investigates what influence the embodiment of an agent has on its persuasiveness and its perceived personality. Is a robotic copy as persuasive as its human counterpart? Does it have the same personality? We performed an experiment in which the embodiment of the agent was the independent variable and the persuasiveness and perceived personality were the dependent measurement. The persuasive agent advertised a Bluetooth headset. The results show that an android is found to be as persuasive as a real human or a video recording of a real human. The personality of the participant had a considerable influence on the measurements. Participants that were more open to new experiences rated the persuasive agent lower on agreeableness and extroversion. They were also more willing to spend money on the advertised product.

For a robot providing services to people in a public space such as a shopping mall, it is important to distinguish potential customers, such as window shoppers, from other people, such as busy commuters. In this paper, we present a series of abstraction techniques for people's trajectories and a service framework for using these techniques in a social robot, which enables a designer to make the robot proactively approach customers by only providing information about target local behavior. We placed a ubiquitous sensor network consisting of six laser range finders in a shopping arcade. The system tracks people's positions as well as their local behaviors, such as fast walking, idle walking, wandering, or stopping. We accumulated people's trajectories for a week, applying a clustering technique to the accumulated trajectories to extract information about the use of space and people's typical global behaviors. This information enables the robot to target its services to people who are walking idly or stopping. The robot anticipates both the areas in which people are likely to perform these behaviors as well as the probable local behaviors of individuals a few seconds in the future. In a field experiment, we demonstrate that this service framework enables the robot to serve people efficiently.

Teleoperation of multiple robots has been studied extensively for applications such as robot navigation; however, this concept has never been applied to the field of social robots. To explore the unique challenges posed by the remote operation of multiple social robots, we have implemented a system in which a single operator simultaneously controls up to four robots, all engaging in communication interactions with users. We present a user inter-face designed for operating a single robot while monitoring several others in the background, then we propose methods for characterizing task difficulty and introduce a technique for improving multiple-robot performance by reducing the number of conflicts between robots demanding the operator's attention. Finally, we demonstrate the success of our system in laboratory trials based on real-world interactions.

We propose that Christopher Alexander's idea of design patterns can benefit the emerging field of HRI. We first discuss four features of design patterns that appear particularly useful. For example, a pattern should be specified abstractly enough such that many different instantiations of the pattern can be uniquely realized in the solution to specific problems in context. Then, after describing our method for generating patterns, we offer and describe eight possible design patterns for sociality in human robot interaction: initial introduction, didactic communication, in motion together, personal interests and history, recovering from mistakes, reciprocal turn-taking in game context, physical intimacy, and claiming unfair treatment or wrongful harms. We also discuss the issue of validation of design patterns. If a design pattern program proves successful, it will provide HRI researchers with basic knowledge about human robot interaction, and save time through the reuse of patterns to achieve high levels of sociality.

This paper reports an initial field trial with a prototype of a semiautonomous communication robot at a train station. We developed an operator-requesting mechanism to achieve semiautonomous operation for a communication robot functioning in real environments. The operator-requesting mechanism autonomously detects situations that the robot cannot handle by itself; a human operator helps by assuming control of the robot.

This paper reports a study about system response time (SRT) in communication robots that utilize human-like social features, such as anthropomorphic appearance and conversation in natural language. Our research purpose established a design guideline for SRT in communication robots. The first experiment observed user preferences toward different SRTs in interaction with a robot. In other existing user interfaces, faster response is usually preferred. In contrast, our experimental result indicated that user preference for SRT in a communication robot is highest at one second, and user preference ratings level off at two seconds.

This paper describes a model for a robot to appropriately control its position when it presents information to a user. This capability is indispensable since in the future many robots will be functioning in daily situations as shopkeepers presenting products to customers or museum guides presenting information to visitors. Psychology research suggests that people adjust their positions to establish a joint view toward a target object. Similarly, when a robot presents an object, it should stand at an appropriate position that considers the positions of both the listener and the object to optimize the listener's field of view and to establish a joint view. We observed human-human interaction situations where people presented objects and developed a model for an information-presenting robot to appropriately adjust its position. Our model consists of four constraints for establishing O-space: (1) proximity to listener, (2) proximity to object, (3) listener's field of view, and (4) presenter's field of view. We also present an experimental evaluation of the effectiveness of our model.

For a robot providing services to people in a public space such as a train station or a shopping mall, it is important to distinguish potential customers, such as window-shoppers, from other people, such as busy commuters. In this paper, we present a series of techniques for anticipating people's behavior in a public space, mainly based on the analysis of accumulated trajectories, and we demonstrate the use of these techniques in a social robot. We placed a ubiquitous sensor network consisting of six laser range finders in a shopping arcade. The system tracks people's positions as well as their local behaviors such as fast walking, idle walking, or stopping. We accumulated people's trajectories for a week, applying a clustering technique to the accumulated trajectories to extract information about the use of space and people's typical global behaviors. This information enables the robot to target its services to people who are walking idly or stopping. The robot anticipates both the areas in which people are likely to perform these behaviors, and also the probable local behaviors of individuals a few seconds in the future. In a field experiment we demonstrate that this system enables the robot to serve people efficiently.

For a person to feel comfortable when approaching a robot it is necessary for the robot to behave in an expected way. Peoplepsilas behavior around a robot not being aware of them were observed during a preliminary experiment. Based on those observations people were classified into four groups depending on their interest in the robot. People were tracked with a laser range finder based system, and their positions, directions and velocities were estimated. A second classification based on that information was made and the relation between the two classifications were mapped. Different actions were created for the robot to be able to react naturally to different human behaviors. In this paper we evaluate three different robot behaviors with respect to how natural they appear. One behavior that actively tries to engage people, one that passively indicates that people have been noticed and a third that makes random gestures. During an experiment test subjects were instructed to act according to the groups from the classification based on interest, and the robotpsilas performance with regard to naturalness was evaluated. Both first and third person evaluation made clear that the active and passive behavior were considered equally natural, while a robot randomly making gestures was considered much less natural.

This paper proposes a new server platform for supporting various robotic services that interact with people. Every robot serving people in public spaces need to know: where the person is, what the person is doing, and what is happening at the place. The proposed platform is the first integrated system that can robustly measure, recognize and provide information on human activities and spatial nature in the environment, in a standard way. In the shopping mall experiment, the system is able to track and capture activities of more than 20 people simultaneously in real time. We also describe the result of robotic service experiment utilizing the power of this platform.

This paper reports a method in which a communicative robot simultaneously interacts with two persons and identifies them with passive-type RFID and floor sensors. The difficulty emanates from the association of these inputs. A passive-type RFID reader provides very accurate person identification. However, the robot cannot specify who actually used the RFID if surrounded by many people. A floor sensor provides robust tracking of people’s positions, but it does not identify any person by itself. Thus, combining these sensors into one device that provides robust tracking of position with accurate person identification would be very beneficial. Toward this problem, in our key idea, the robot keeps multiple hypotheses about the interpretation of sensor information and gradually improves their accuracy through interaction with the people. The multimodal inputs from sensors are integrated with a Bayesian network that allows us to exploit people’s behavior patterns as parameters to estimate the accuracy of our hypotheses. Our method also considers the vagueness of current hypotheses when choosing appropriate interactive behaviors. When the robot has reservations about the situation, it prefers non-conclusive behavior and tries to solve the ambiguity by performing verification behaviors. Once the robot is certain, it prefers conclusive behavior. The developed system was tested in a real daily environment, a shopping center, where we gathered interaction data between the robot and multiple people as teaching data for the Bayesian network. The experimental results revealed that the system successfully identified 79.2% of the visitors. In particular, the hypothesis-based system for behavior control improved the successful rate by 16.7%.

The application range of communication robots could be widely expanded by the use of automatic speech recognition (ASR) systems with improved robustness for noise and for speakers of different ages. In past researches, several modules have been proposed and evaluated for improving the robustness of ASR systems in noisy environments. However, this performance might be degraded when applied to robots, due to problems caused by distant speech and the robot's own noise. In this paper, we implemented the individual modules in a humanoid robot, and evaluated the ASR performance in a real-world noisy environment for adults' and children's speech. The performance of each module was verified by adding different levels of real environment noise recorded in a cafeteria. Experimental results indicated that our ASR system could achieve over 80% word accuracy in 70-dBA noise. Further evaluation of adult speech recorded in a real noisy environment resulted in 73% word accuracy.

Identifying the extent to which the appearance of a humanoid robot affects human behavior toward it is important. We compared participant impressions of and behaviors toward two real humanoid robots in simple human-robot interactions. These two robots, which have different appearances but are controlled to perform the same recorded utterances and motions, are adjusted by a motion-capturing system. We conducted an experiment with 48 human participants who individually interacted with the two robots and also with a human for reference. The results revealed that different appearances did not affect participant verbal behaviors, but they did affect such nonverbal behaviors as distance and delay of response. These differences are explained by two factors: impressions and attributions.

Human beings subconsciously adapt their behaviors to a communication partner in order to make interactions run smoothly. In human-robot interactions, not only the human but also the robot is expected to adapt to its partner. Thus, to facilitate human-robot interactions, a robot should be able to read subconscious comfort and discomfort signals from humans and adjust its behavior accordingly, just like a human would. However, most previous research works expected the human to consciously give feedback, which might interfere with the aim of interaction. We propose an adaptation mechanism based on reinforcement learning that reads subconscious body signals from a human partner, and uses this information to adjust interaction distances, gaze meeting, and motion speed and timing in human-robot interactions. The mechanism uses gazing at the robot's face and human movement distance as subconscious body signals that indicate a human's comfort and discomfort. A pilot study with a humanoid robot that has ten interaction behaviors has been conducted. The study result of 12 subjects suggests that the proposed mechanism enables autonomous adaptation to individual preferences. Also, detailed discussion and conclusions are presented.

When people interact with communication robots in daily life, their attitudes and emotions toward the robots affect their behavior. From the perspective of robotics design, we need to investigate the influences of these attitudes and emotions on human-robot interaction. This paper reports our empirical study on the relationships between people's attitudes and emotions, and their behavior toward a robot. In particular, we focused on negative attitudes, anxiety, and communication avoidance behavior, which have important implications for robotics design. For this purpose, we used two psychological scales that we had developed: negative attitudes toward robots scale (NARS) and robot anxiety scale (RAS). In the experiment, subjects and a humanoid robot are engaged in simple interactions including scenes of meeting, greeting, self-disclosure, and physical contact. Experimental results indicated that there is a relationship between negative attitudes and emotions, and communication avoidance behavior. A gender effect was also suggested.

The Human Robot Interaction 2007 conference hosted a video session, in which movies of interesting, important, illustrative, or humorous HRI research moments are shown. This paper summarizes the abstracts of the presented videos. Robots and humans do not always behave as expected and the results can be entertaining and even enlightening - therefore instances of failures have also been considered in the video session. Besides the importance of the lessons learned and the novelty of the situation, the videos have also an entertaining value.

This paper reports a method that uses humanoid robots as a communication medium. There are many interactive robots under development, but due to their limited perception, their interactivity is still far poorer than that of humans. Our approach in this paper is to limit robots' purpose to a non-interactive medium and to look for a way to attract people's interest in the information that robots convey. We propose using robots as a passive-social medium, in which multiple robots converse with each other. We conducted a field experiment at a train station for eight days to investigate the effects of a passive-social medium.

In this research, we realize human telepresence by developing a remote-controlled android system called Geminoid HI-1. Experimental results confirm that participants felt stronger presence of the operator when he talked through the android than when he appeared on a video monitor in a video conference system. In addition, participants talked with the robot naturally and evaluated its human likeness as equal to a man on a video monitor. At this paper's conclusion, we will discuss a remote-control system for telepresence that uses a human-like android robot as a new telecommunication medium.

This paper describes a group attention control (GAC) system that enables a communication robot to simultaneously interact with many people. GAC is based on controlling social situations and indicating explicit control to unify all purposes of attention. We implemented a semi-autonomous GAC system into a communication robot that guides visitors to exhibits in a science museum and engages in free-play interactions with them. The GAC system's effectiveness was demonstrated in a two-week experiment in the museum. We believe these results will allow us to develope interactive humanoid robots that can interact effectively with groups of people.

We have taken steps towards developing a method that enables an interactive humanoid robot to adapt its speed to a walking human that it is moving together with. This is difficult because the human is simultaneously adapting to the robot. From a case study in human-human walking interaction we established a hypothesis about how to read a human's speed preference based on a relationship between humans' walking speed and their relative position in the direction of walking. We conducted two experiments to verify this hypothesis: one with two humans walking together, and one with a human subject walking with a humanoid robot, Robovie-IV. For 11 out of 15 subjects who walked with the robot, the results were consistent with the speed-position relationship of the hypothesis. We also conducted a preferred speed estimation experiment for six of the subjects. All of them were satisfied with one or more of the speeds that our algorithm estimated and four of them answered one of the speeds as the best one if the algorithm was allowed to give three options. In the paper, we also discuss the difficulties and possibilities that we learned from this preliminary trial.

This paper reports a study that estimated visitor positions, visiting patterns, and inter-human relationships at a science museum using information from RFID readers. In the science museum, we exhibited humanoid robots. Visitors were invited to wear RFID tags to interact with the robots. Visitor behavior was simultaneously observed using 20 RFID readers, distributed throughout the entire floor, that roughly measured the distances of nearby tags. We integrated the outputs from all RFID readers to estimate visitor trajectories that were used to analyze three perspectives: space, visiting patterns, and relationships. Regarding space, we identified crowded and uncrowded areas. We found several typical visiting patterns, such as visited at every exhibit and directly going to robot area. We also identified atypical visiting behavior. Regarding relationships, for example, we estimated 68% coverage of group-member relationships with 91% reliability.

A simple view of deictic communication only includes the indication process and recognition process: a person points at an object and says something about it such as “look at this,” and then the other person recognizes the pointing gesture and pays attention to the indicated object. However, this simple view lacks three important processes: attention synchronization, context focus, and believability establishment. We refer to these three processes as “facilitation processes” and implement them in a humanoid robot with a motion capturing system. An experiment with 30 subjects revealed that the facilitation processes make deictic communication natural.

Recently, remote-controlled communication robots have been studied. Our question is whether the person interacting with them feels that he/she is interacting with the robot itself or the human behind it. How do such beliefs affect interaction? We conducted an experiment to study the effect of a human presence behind an interactive humanoid robot. We used an interactive robot system based on a motion-capturing system where participants can bodily interact with the robot. They were deceptively informed that they would either be interacting with a robot controlled by a program or a human; in fact, in both conditions the robot was autonomous. Experimental results indicated that two-thirds of the participants felt that they were interacting with the robot itself. Their enjoyment was unaffected by the knowledge of whether the robot was controlled by a program or a human, although their impression of robot intelligence indicated that they distinguished between these conditions. On the contrary, since the remaining one-third of the participants felt they were interacting with a human behind the robot, they were affected by the knowledge: when they were told they were interacting with a teleoperated robot, they enjoyed it less; in contrast, their enjoyment increased when they were told they were interacting with an autonomous robot.

The uncanny valley theory proposed by Mori in 1970 has been a hot topic in human robot interaction research, in particular since the development of increasingly human-like androids and computer graphics. In this paper we describe an empirical study that attempts to plot Mori's hypothesized curve. In addition, the influence of framing on the users' perception of the stimuli was investigated. Framing had no significant influence on the measurements. The pictures of robots and humans were rated independently of whether the participants knew a particular picture showed a robot or human. Anthropomorphism had a significant influence on the measurements, but not even pictures of real humans were rated as likeable as the pictures of humanoids or toy robots. As a result we suggest the existence of an uncanny cliff model as an alternative to the uncanny valley model. However, this study focused on the perception of pictures of robots and the results, including the suggested model, may be different for the perception of movies of moving robots or the perception of standing right in front of a moving robot.

Over the past several years, many humanoid robots have been developed, and they can typically make sophisticated human-like expressions with their head and arms. We believe that humanoid robots will be suitable for communicating with people. The human-like bodies of humanoid robots enable humans to intuitively understand their gestures and cause people to unconsciously behave as if they were communicating with humans. In the future, we believe that these robots will behave as peer-partners to support daily human activities based on advanced communication capabilities. In order to develop such a "communication robot," we are studying bodily interaction between humans and the robot in a laboratory environment, where we found the importance of cooperative behaviors. Moreover, the developed robots were tested in field trials, which are important for studying potential real-world applications of communication robots, still a largely unexplored field.

Humanoids are the most advanced robots and have been expected to act in various fields including education. Thus, it is important to investigate in different cultures what people actually assume when they encounter the word "humanoid robots," from not only a psychological perspective but also an engineering one, focusing on such aspects as design and market of robotics. For this aim, a cross-cultural research instrument, the robot assumptions questionnaire (RAQ) was administered to university students in Japan, Korea, and the USA. As a result, it was found that the Japanese students more strongly assume autonomy, social relationships, and emotional capacity of humanoid robots than the Korean and USA students, and there are more detailed cultural differences of assumptions about humanoids related to daily-life fields, in particular, pedagogical fields. Moreover, it was found that the USA students have more ambivalent images of humanoids than the Japanese students, and the Korean students have more careful attitudes toward humanoid robots than the Japanese students. In addition, the paper discusses engineering implications of the research results.

We believe that lifelikeness is important to make it possible for a communication robot to interact naturally with people. In this research, the relationship between contingency and complexity in a lifelike robot was investigated. We have developed a robot control system to control experimentally contingency and complexity in interaction by combining a humanoid robot with a motion-capturing system. Two independent experiments were conducted with different levels of interaction complexity: simple interaction and complex interaction. Experiments in the simple interaction situation indicated that subjects felt that the robot behaved autonomously when contingency was low, and there was no significant relationship between contingency and a lifelike impression. On the other hand, experiments in the complex interaction situation showed that the robot gave the subjects the impression of being more autonomous and lifelike when contingency was high.

Interactive robots participating in our daily lives should have the fundamental ability to socially communicate with humans. In this paper, we propose a mechanism for two social communication abilities: forming long-term relationships and estimating friendly relationships among people. The mechanism for long-term relationships is based on three principles of behavior design. The robot we developed, Robovie, is able to interact with children in the same way as children do. Moreover, the mechanism is designed for long-term interaction along the following three design principles: (1) it calls children by name using radio frequency identification tags; (2) it adapts its interactive behaviors for each child based on a pseudo development mechanism; and (3) it confides its personal matters to the children who have interacted with the robot for an extended period of time. Regarding the estimation of friendly relationships, the robot assumes that people who spontaneously behave as a group together are friends. Then, by identifying each person in the interacting group around the robot, it estimates the relationships between them. We conducted a two-month field trial at an elementary school. An interactive humanoid robot, Robovie, was placed in a classroom at the school. The results of the field trial revealed that the robot successfully continued interacting with many children for two months, and seemed to have established friendly relationships with them. In addition, it demonstrated reasonable performance in identifying friendships among children. We believe that these results demonstrate the potential of current interactive robots to establish social relationships with humans in our daily lives.

The development of interactive robots is a complicated process, involving a plethora of psychological, technical, and contextual influences. To design a robot capable of operating ldquointelligentlyrdquo in everyday situations, one needs a profound understanding of human-robot interaction (HRI). We propose an approach based on integral analysis of multimodal data to pursue this understanding and support interdisciplinary research and development in the field of robotics. To adopt this approach, a software tool named interaction debugger was developed that features user-friendly navigation, browsing, searching, viewing, and annotation of data; it enables fine-grained inspection of the HRI. In four case studies, we demonstrated how our analysis approach aids the development process of interactive robots.

Along with the development of interactive robots, controlled experiments and field trials are regularly conducted to stage human-robot interaction. Experience in this field has shown that analyzing human-robot interaction for evaluation purposes fosters the development of improved systems and the generation of new knowledge. In this paper, we present the interaction debugging approach, which is based on the collection and analysis of data from robots and their environment. Considering the multimodality of robotic technology, often only audio and video are insufficient for detailed analysis of human-robot interaction. Therefore, in our analysis we integrate multimodal information using audio, video, sensory data, and intermediate variables. An important aspect of the interaction debugging approach is using a tool called Interaction Debugger to analyze data. By supporting user-friendly data presentation, annotation and navigation, Interaction Debugger enables fine-grained inspection of human-robot interaction. The main goal of this paper is to address how an integral approach to the analysis of human-robot interaction can be adopted. This is demonstrated by three case studies.

This paper investigates human behavior around an interactive robot at a science museum. To develop a communication robot that works in daily environments, it is important to investigate the available information from a robot about people's behavior. Such information will enable the robot to predict people's behavior so that the robot can optimize its interactive behavior. We analyzed visitor behavior toward a simple interactive robot exhibited at a science museum in relation to information from sound level and range sensors. We discovered factors that influence the way people approach, maintain distance, and interact both physically and verbally with the robot. This enabled us to extract meaningful information from the sensory information and apply it to communication robots.

This paper reports on a field trial with interactive humanoid robots at a science museum where visitors are supposed to study and develop an interest in science. In the trial, each visitor wore an RFID tag while looking around the museum's exhibits. Information obtained from the RFID tags was used to direct the robots' interaction with the visitors. The robots autonomously interacted with visitors via gestures and utterances resembling the free play of children [1]. In addition, they performed exhibit-guiding by moving around several exhibits and explaining the exhibits based on sensor information. The robots were highly evaluated by visitors during the two-month trial. Moreover, we conducted an experiment in the field trial to compare the detailed effects of exhibit-guiding and free-play interaction under three operating conditions. This revealed that the combination of the free-play interaction and exhibit-guiding positively affected visitors' experiences at the science museum.

The purpose of our research is to develop lifelike behavior in a communication robot, which is expected to potentially make human-robot interaction more natural. Our earlier research demonstrated the importance of a robot's contingency for lifelikeness [1]. On the other hand, perfect contingency seems to give us a non-lifelike impression. In order to explore the appropriate contingency for communication robots, we developed a robot system that allows us to adjust its contingency to an interacting person in a simple mimic interaction. As a result of an experiment, we identified the relationships between the degree of contingency and the subjective impressions of lifelikeness, autonomy, and preference. However, the experimental result also seems to suggest the importance of the complexity of interaction for investigating the appropriate contingency of communication robots.

This paper presents a three-layer model for generation and recognition of attention-drawing behavior. The model enables a robot to recognize people's attention-drawing behavior as well as to perform attention-drawing behavior to people. It consists of three layers: the PSM (pointing space model), the RTM (reference term model), and the OPM (object property model). The PSM associates the pointing gesture with a reference term, the RTM associates positional relationships with a reference term, and the OPM associates other supplemental verbal cues with a reference term. We implemented the model in a humanoid robot, Robovie, and verified its effectiveness through an experiment

In this paper, we move toward offering psychological benchmarks by which to measure success in building increasingly human-like robots. By psychological benchmarks we mean categories of interaction that capture conceptually fundamental aspects of human life, specified abstractly enough so as to resist their identity as a mere psychological instrument, but capable of being translated into testable empirical propositions. Six possible benchmarks are considered: autonomy, imitation, intrinsic moral value, moral accountability, privacy, and reciprocity. Finally, we discuss how getting the right group of benchmarks in human-robot interaction will, in future years, help inform on the foundational question of what constitutes essential features of being human

This paper introduces a "teleoperated communication robot" whose unique point is that its language component is performed by an operator to avoid the automatic recognition difficulty of spoken language. On the other hand, conceptually, its nonverbal behavior will be autonomously controlled. One important point is to identify whether ordinary people will accept this type of communication robot. Therefore, we conducted a preliminary field trial at a shopping center, which revealed positive perspectives as well as technical problems to be solved

Coding can contribute to robot design by suggesting behavioral benchmarks. These, however, depend on the level of analysis. In illustration, semi-formalized rules are used to investigate child-robot encounters. By using behavior-level codes, we extract information about how children use the robot. This leads to findings about longitudinal changes in how children evaluate its behaviors. Children, we find, use the robot as a social mediator - to prompt synchronized social events. By focusing on a behavioral level, coding can benefit designers of robots, software and sensors

To study the short-term and long-term influence of communication robots in daily life applications, it is necessary to develop psychological scales for measuring the mental states of users of robots and analyzing related social trends. In particular, to explore the more internal factors related to communication robots, it is necessary to focus on anxiety. This paper reports the results of developing the Robot Anxiety Scale (RAS) for measuring the anxiety that prevents individuals from interaction with robots having functions of communication in daily life. In particular, we focus on communication in a human-robot dyad

When describing a physical object, we indicate which object by pointing and using reference terms, such as ‘this’ and ‘that’, to inform the listener quickly of an indicated object's location. Therefore, this research proposes using a three-layer attention-drawing model for humanoid robots that incorporates such gestures and verbal cues. The proposed three-layer model consists of three sub-models: the Reference Term Model (RTM); the Limit Distance Model (LDM); and the Object Property Model (OPM). The RTM selects an appropriate reference term for distance, based on a quantitative analysis of human behaviour. The LDM decides whether to use a property of the object, such as colour, as an additional term for distinguishing the object from its neighbours. The OPM determines which property should be used for this additional reference. Based on this concept, an attention-drawing system was developed for a communication robot named ‘Robovie’, and its effectiveness was tested.

We propose a method for promoting human-robot interaction based on emotion recognition with particular focus on tension emotion. There are two types of emotions expressed in a short time. One is autonomic emotion caused by a stimulus, such as joy and fear. The other is self-reported emotion, such as tension, that is relatively independent of a single stimulus. In our preliminary experiment, we observed that tension emotion (self-reported emotion) obstructs the expression of autonomic emotion, which has demerits on speech recognition and interaction. Our method is based on detection and moderation of tension emotion. If a robot detects tension emotion, it tries to ease it so that a person will interact with it more comfortably and express autonomic emotions. It also retrieves nuances from expressed emotions for supplementing insufficient speech recognition, which will also promote interaction.

It is important to identify how much the appearance of a humanoid robot affects human behaviors toward it. We compared participants' impressions of and behaviors toward two real humanoid robots in simple human-robot interaction. These two robots have different appearances but are controlled to perform the same recorded utterances and motions, which are adjusted by using a motion capturing system. We conducted an experiment where 48 human participants participated. In the experiment, participants interacted with the two robots one by one and also with a human as a reference. As a result, we found that the different appearances did not affect the participants' verbal behaviors but did affect their non-verbal behaviors such as distance and delay of response. These differences are explained by two factors, impressions and attributions.

In this paper, we propose an adaptation mechanism for robot behaviors to make robot-human interactions run more smoothly. We propose such a mechanism based on reinforcement learning, which reads minute body signals from a human partner, and uses this information to adjust interaction distances, gaze meeting, and motion speed and timing in human-robot interaction. We show that this enables autonomous adaptation to individual preferences by an experiment with twelve subjects.

When we talk about objects in an environment, we indicate to a listener which object is currently under consideration by using pointing gesture and such reference terms as "this" and "that". Such reference terms play an important role in human interaction by quickly informing the listener of an indicated object's location. In this research, we propose a three-layered draw-attention model for humanoid robots with gestures and verbal cues. Our proposed three-layered model consists of three sub models: reference term model (RTM), limit distance model (LDM) and object property model (OPM). RTM decides an appropriate reference term using functions constructed by an analysis of human behavior. LDM decides whether to use the object's property with a reference term. OPM decides the appropriate property for indicating the object by comparing object properties with each other. We developed an attention drawing system in a communication robot named "Robovie" based on the three layered model. We confirmed its effectiveness through the experiments.

One major design goal in human-robot interaction is that the robots behave in an intelligent manner, preferably in a similar way as humans. This constraint must also be taken into consideration when the navigation system for the platform is developed. However, research in human-robot interaction is often restricted to other components of the system including gestures, manipulation, and speech. On the other hand, research for mobile robot navigation focuses primarily on the task of reaching a certain goal point in an environment. We believe that these two problems can not be treated separately for a personal robot that coexists with humans in the same surrounding. Persons move constantly while they are interacting with each other. Hence, also a robot should do that, which poses constraints on the navigation system. This type of navigation is the focus of this paper. Methods have been developed for a robot to join a group of people engaged in a conversation. Preliminary results show that the platform's moving patterns are very similar to the ones of the persons. Moreover, this dynamic interaction has been judged naturally by the test subjects, which greatly increases the perceived intelligence of the robot.

Previous research works in robotics and cognitive science have reported that humans utilize embodied cooperative behaviors in communication, such as nodding in response to another's it utterance and looking at a certain object in a certain direction as others look or point at. We have developed a humanoid robot that utilizes such an embodied cooperative behavior for natural communication in a route guidance situation. It obtains numerical data on a human's body movement via a motion capturing system and then autonomously selects appropriate cooperative embodiment units from 18 implemented units. Each unit realizes a certain cooperative embodiment behaviors such as eye-contact by using the motion capturing system as well. As a result of a subject experiment, we have verified the effectiveness of the embodied cooperative behaviors of the robot for reliable and sympathetic communication. Moreover, we analyzed how the auditory expression and the embodiment contributed to the effect.

This paper reports our novel approach to developing a social robot. The developed robot is able to identify relationships among humans from their physical behaviors as it establishes long-term relationships with humans. This interactive humanoid robot attracts humans to interact with it and, as a result, induces them to perform their group behaviors in front of it. The robot recognizes friendly relationships among humans by simultaneously identifying each person in the interacting group. We conducted a two-month experiment in an elementary school. As a result, the robot successfully continued friendly interaction with many children for the two months, and demonstrated reasonable performance in identifying friendships among children. We believe this ability to maintain friendly relationships with humans and to identify human relationships is essential to behaving socially.

An interactive exhibition guide robot has been developed, that is integrated with ubiquitous sensors. The robot utilizes its human-like body properties to perform human-like gestures such as eye-contact and pointing. The ubiquitous sensors are embedded in an exhibition room or attached to people in the room, and they provide information not only for the robot's interactive guide but also for the development of the robot. The results of a two-days exhibition experiment show that the robot attracted visitors and successfully guided them in the environment. Moreover, the obtained interaction statistics, including video from the ubiquitous sensors, prove to be useful for analyzing the interaction between the robot and visitors, and this implies that the use of such interaction data is a promising approach for developing interactive robots.

This paper reports progress in the development of a humanoid robot designed for real-time face-to-face interaction with humans. An essential component for face-to-face interaction with humans is being able to find faces, tracking them, and smoothly moving back and forth between gazing to a face and other objects of interest. In this paper we propose a system that integrates peripheral vision and foveal vision in a principled manner using particle filters. The developed system generates hypotheses about face position by using peripheral vision and verifies them by integrating peripheral vision and foveal vision. Even though a face may not be present in foveal vision, while the robot is gazing at another object, it keeps plausible hypotheses about the location of the human face with peripheral vision and restarts the face-following by verifying the hypothesis later. Moreover, this fundamental function provides more rich sensory information for human-robot interaction, such as a human's facial expression and lip motions during utterances.

We report the development and evaluation of a new interactive humanoid robot that communicates with humans and is designed to participate in human society as a partner. A human-like body will provide an abundance of nonverbal information and enable us to smoothly communicate with the robot. To achieve this, we developed a humanoid robot that autonomously interacts with humans by speaking and gesturing. Interaction achieved through a large number of interactive behaviors, which are developed by using a visualizing tool for understanding the developed complex system. Each interactive behavior is designed by using knowledge obtained through cognitive experiments and implemented by using situated recognition. The robot is used as a testbed for studying embodied communication. Our strategy is to analyze human-robot interaction in terms of body movements using a motion-capturing system that allows us to measure the body movements in detail. We performed experiments to compare the body movements with subjective evaluation based on a psychological method. The results reveal the importance of well-coordinated behaviors as well as the performance of the developed interactive behaviors and suggest a new analytical approach to human-robot interaction.

This paper reports a trial of immersing interactive humanoid robots into a real human society, where they are charged with the communication task of foreign language education. For this purpose, we developed interactive humanoid robots equipped with wireless person identification technology for facilitating interaction with multiple persons. We believe that the ability to identify persons allows more meaningful social relationships between robots and humans to develop. This paper discusses the fundamental mechanisms of the multi-person identification ability, and how the robots established and sustained social relationships among children in an elementary school.

In this paper, we report about a new interaction-oriented robot, which communicates with humans and will participate in human society as our partner. For realizing such a robot, we have started a new collaborative work between cognitive science and robotics. In the way of robotics, we have developed a humanoid robot named "Robovie" that has enough physical expression ability. On the other hand, through cognitive experiments, we obtained important ideas about the robot's body property. To incorporate these ideas, we have developed software architecture and implemented autonomous interactive behaviors to the robot. Further, we have evaluated the robot's performance of the interactive behaviors through psychological experiments. The experiments reveal how humans recognize the robot.

There is a strong correlation between the number of appropriate behaviors an interactive robot can produce and its perceived intelligence. We propose a robot architecture for implementing a large number of behaviors and a visualizing tool for understanding the developed complex system. Behaviors are designed by using knowledge obtained through cognitive experiments and implemented by using situated recognition. By representing relationships between behaviors, episode rules help to guide the robot in communicating with people in a consistent manner. We have implemented over 100 behaviors and 800 episode rules in a humanoid robot. As a result, the robot could entice people to relate to it interpersonally. An Episode Editor is a tool to support the development of episode rules and to visualize the complex relationships among the behaviors. We consider the visualization is to be necessary for the constructive approach.

For realizing a robot working in human society, interaction with humans is the key issue. We have developed a robot that interacts with humans based on visual recognition. This robot has two vision systems: an omnidirectional vision system for acquiring necessary visual information and a binocular stereo vision system. The binocular vision system indicates what the robot is looking at and is not used for locomotion. Gaze control plays an important role in human-robot interaction. This paper reports how the robot's gaze influences subjects' impressions of the robot. With a statistically significant number of robot observers, we employed psychological methods, the semantic differential method (SD), and factor analysis. This careful psychological analysis of robot impressions is the first trial in robotics. Through the experiment, we have found that the impressions mainly consist of four factors: familiarity, enjoyment, activity, and performance. The computer skills of subjects affect their impressions of the robot.