We report a mixed-methods field experiment of a conversational service robot deployed under everyday staffing discretion in a live bedding store. Over 12 days we alternated three conditions--Baseline (no robot), Robot-only, and Robot+Fixture--and video-annotated the service funnel from passersby to purchase. An explanatory sequential design then used six post-experiment staff interviews to interpret the quantitative patterns.

Recent research has focused on developing non-task-oriented dialogue agents with high anthropomorphism, aiming to sustain users’ dialogue motivation over prolonged periods. Previous studies have introduced chatbots that deliberately incorporate short response delays to enhance anthropomorphism and user satisfaction. However, in the context of long-term casual conversations, the behavior of these chatbots, which reply within a few seconds, may lead to a perception of the chatbot as always on standby, lacking its own schedule or personal life, thus appearing less humanlike. To address this issue, we propose a non-real-time chatbot that interacts with users while introducing deliberate response delays on a larger time scale, ranging from a few minutes to several hours. A five-day dialogue experiment confirmed that the non-real-time chatbot was perceived as more humanlike and encouraged users to engage in extensive dialogue. These insights are invaluable for designing chatbots that can maintain casual conversations over prolonged periods.

This study investigated the differences between human and robot gaze in influencing preference formation, and examined the role of Theory of Mind (ToM) abilities in this process. Human eye gaze is one of the most important sources of information for social interaction and research has demonstrated its effectiveness in influencing people's preference. With increasing technological development, we will interact with robots that can exhibit gaze behavior and influence people's preference. It is unclear whether there are any differences between humans and robots in this process. The present study aimed to analyze the role of the gaze of a robot and a human in influencing the ascription of a preference to the gazer and the participants' preference. Furthermore, we examined the role of ToM abilities in preference formation. The results showed that the gaze has a greater effect on the gazer preference compared to participants' preference regardless of the agent (human or robot). In addition, ToM abilities predict both gazer and individual preferences in the robot's condition only even though different socio-cognitive mechanisms are involved. The study suggests that adults are cognitively able to process the gaze of a robot similar to a human, recognizing the underlying mental state. However, only for the robot, different cognitive mechanisms are involved in the gazer (i.e., perspective taking) and participants' preference formation (i.e., advanced ToM).

Although most people interact with social robots in daily environments in a friendly manner, some do not. They sometimes destroy, insult, bully, or abuse them. Uncovering such antisocial attitudes and behaviors is a painful but critical research topic in the process of deploying social robots in real society. In this study, we describe the robot harassment we observed in a three-year exhibition with an android robot. During the exhibition, visitors talked freely with the android robot, which autonomously handled the conversation topics. Many people enjoyed chatting with it in a friendly manner, although a few behaved rudely toward it. A coder labeled 26/971 scenes as “harassment” interactions and organized them into four subcategories: unwelcome flirting, rude attitudes, unsolicited comments about appearance, and inappropriate touching. We introduce these details and discuss possible research topics necessary to reduce such harassment behaviors.

Depression is a widespread mental health issue affecting diverse age groups, with notable prevalence among college students and the elderly. However, existing datasets and detection methods primarily focus on young adults, neglecting the broader age spectrum and individual differences that influence depression manifestation. Current approaches often establish a direct mapping between multimodal data and depression indicators, failing to capture the complexity and diversity of depression across individuals. This challenge includes two tracks based on age-specific subsets: Track 1 uses the MPDD-Elderly dataset for detecting depression in older adults, and Track 2 uses the MPDD-Young dataset for detecting depression in younger participants. The Multimodal Personality-aware Depression Detection (MPDD) Challenge aims to address this gap by incorporating multimodal data alongside individual difference factors. We provide a baseline model that fuses audio and video modalities with individual difference information to detect depression manifestations in diverse populations. This challenge aims to promote the development of more personalized and accurate de pression detection methods, advancing mental health research and fostering inclusive detection systems. More details are available on the official challenge website: https://hacilab.github.io/MPDDChallenge.github.io.

The study aims to investigate the effect of cultural background between Italy and Japan on 5-year-old children’s moral judgements and emotion attributions towards human and robot. The children watched videos drawn from classic ‘happy victimizer’ stories, in which the transgressors were either a child or a robot, violating rules against stealing or not sharing. We assessed the children’s attribution of emotions both to the transgressors and to themselves as victimisers (i.e. first-person perspective), as well as the moral judgement of the violations. The results showed that children from both cultures do not significantly discriminate between human and robot in their moral judgement and emotion attributions. Concerning moral emotions, Italian children tend to attribute fewer negative emotions to the transgressor than Japanese children, especially in the not sharing scenario. Furthermore, adopting a first-person perspective to evaluate moral transgressions reduces cultural differences in emotion attributions. The study highlights how culture, rather than the transgressor’s agency (human or robot), influences early moral reasoning.

Understanding pedestrian behavior and predicting their intentions near roads are crucial for enhancing road safety and traffic efficiency. In intelligent transportation systems, especially where computational and energy resources are limited, real-time inference is vital. To address this, our study focuses on pedestrian skeletal information, which offers lower dimensionality and reduces computational demands. We introduce EffIntentGCN, a lightweight graph convolutional network designed to analyze pedestrian poses using a part-based graph and an adaptive graph, emphasizing features crucial for intention prediction and enhancing the learning of local and global interactions. An expansion layer added before each convolution block increases feature dimensionality for a broader representation, without excessive parameter increase. We further incorporate spatial-temporal joint attention, focusing on crucial joints in dynamic skeletal sequences. The model also integrates second-order information, representing joint connections, to provide complementary information. To balance model complexity with computational efficiency, essential for applications in resource-constrained environments like autonomous vehicles, we employ depthwise separable convolution and low-rank approximation techniques, aimed to reduce parameter count and maintain computational efficiency. The model is evaluated on the JAAD dataset, demonstrating its effectiveness in accurately predicting pedestrian crossing intentions while optimizing computational resources.

As the loneliness problem grows more severe among older people living in nursing homes, the use of baby-like robots as companions is becoming an option to relieve this problem. However, it remains unclear how baby-like behavior affects older people. This study investigates the effect of the emotional expression of baby-like robots on older people to realize a companion robot that relieves their loneliness. Through two experiments, we investigated older adults’ impressions of one or two robots that express different emotions. They rated their impressions of the robots by watching videos of one baby-like robot that vocalized different emotional sounds (crying, laughing, or babbling) in Experiment I and two baby-like robots that vocalized different combinations of emotional sounds in Experiment II. We found that the crying robot reduced their willingness and enjoyment to interact with the robot and suppressed the emotional effects of the other robot. Our findings can help improve the well-being of older adults, providing insight into how to support older adults using baby-like robots.

With the age of diversification, it’s important to nurture the qualities of children to enable them to create their future society. To achieve this, it’s crucial for children to broaden and deepen their views through dialogue with others. In the field of human-agent interaction, introducing robots into educational settings has shown positive educational effects. In this study, we developed an autonomous dialogue robot that delved into the children’s ideas and prompted them to think about aspects they had not yet considered, to broaden and deepen their viewpoints. We introduced this developed robot into a 6th-grade science class and conducted an experiment. The results based on subjective evaluations by students through questionnaires and objective evaluations of the students’ reaction papers by third parties, suggest that discussions with the robot can broaden and deepen the viewpoints of students. This study highlights the potential of robots to support the important educational goal.

This study explored a tele-operation notification interface that supports a multitasking operator independently perceiving remote information and shifting their tasks. We proposed a noise-reducing auditory notification method, which artificially replicated the cocktail party effect by decreasing the volume of the background noise and increasing the volume of visitor footsteps. Experimental results show that our proposed method provided an effective interface for operators to independently notice human presence.

Effective teleoperated avatar communication requires expressing social behaviors. Gaze behavior is one of the crucial social behaviors and includes reflexive reactions to the avatar’s surroundings and intentional responses to the operator’s speech and actions. Teleoperated avatars must have their gaze behavior controlled according to situational changes in both the avatar’s and operator’s contexts. However, it is not clear how to adjust the avatar’s gaze in response to changes in both situations. In this paper, we collect a dataset of gazing positions that the avatar is instructed to face, taking into account both avatar and operator situations, and annotation labels that represent both situations in detail. We then exploratorily analyze the ratio of gazing positions per situation through dynamic area-of-interest (AOI) analysis. Our analysis provides insights into determining the gaze behavior of teleoperated avatars.

Facial mimicry is crucial for human-like robots in human-robot interaction. The challenge is that the high diversity of facial expressions proposes difficulties in programming a robotic face to mimic human facial expressions using traditional methods. In this paper, we present a data-driven method to retarget human facial expressions to robotic faces without human effort. Our data collection is fully automatic, where only a robotic face and Apple ARKit are involved to sample actuator commands and record the resulting facial blendshape values. We trained a neural network that predicts blendshape values from commands, which is then used as a surrogate model to optimize command values to resemble given facial expressions. Experiments show that the proposed method has achieved lower error in terms of facial blendshape values than baselines. Moreover, the response time can be reduced to 0.2 seconds via TCP/IP through WiFi, offering great potential for real-time application. Our method is a novel framework for retargeting facial expressions to robotic faces, which can be incorporated into various human-robot interaction systems.

The use of dialogue agents with a strategy that focuses beyond a static depth level and gradually deepens the conversation over time can nurture relationships with users. Thus, this study aimed to develop a dialogue agent capable of gradually deepening conversations over time and consequently examine its effectiveness. First, we constructed a topic depth estimation model capable of estimating the depth of any given topic. Subsequently, we developed a dialogue agent designed to gradually deepen conversation topics as interactions progress, tailored for operations on users’ smartphones. To evaluate the effectiveness of the developed agent, we conducted a 10-day dialogue experiment with 26 participants. The results indicated that the proposed agent elicited deep self-disclosure from users without significantly increasing psychological reluctance compared to the control condition. Moreover, the participants perceived a deeper relationship with the agent and were less likely to consider it as a non-human entity. These findings suggest that the proposed dialogue agent was capable of effectively engaging in deeper conversations over time, thereby enhancing human–agent interactions.

Human robot interaction studies have investigated how various non-verbal expressions of robots can enhance feelings of familiarity and trust in users’ and establish good relationships between users and robots. In the field of art, this approach has been taken one step further by creating robot artworks and demonstrations through which people feel as if they are experiencing heart-to-heart communication with a robot. Although robotic behavior that conveys such a feeling is useful for human coexistence and providing a sense of security and trust, no engineering methodology can yet achieve such behavior. This study attempts to explain what kind of robot behavior is connected to such feelings by analyzing demonstrations of such a robot based on information theory. We found that the intensity of these feelings can be partially explained by transfer entropy between humans and the robot’s body movements. We expect this research to clarify how to design a robot’s behavior so that it can provide heart-to-heart communication with people as well as how to construct a robot that can build solid relationships with people in daily life.

Conversational recommendation systems are crucial for making recommendations agreeable to the user. To reach an agreeable recommendation, this study proposes a dialog strategy that represents a reasonable order of items and elicits the current estimation by the wording of utterances based on the subjective preference estimation. We developed two dialog functions for the topic and word choice based on the history of preference estimations. The human impression of a robot’s diligence, understanding capability, and satisfaction were evaluated through a conversation with a virtual robot using a crowdsourcing platform. We compared six conditions that differed based on two topics and three wording patterns. The experimental results indicated that the main effect of the wording patterns, whereas one of the topic choices was not found to be significant. Further analysis showed that accurate estimation improves the robot’s impression when demonstrating its diligence.

While it is crucial for human-like avatars to perform co-speech gestures, existing approaches struggle to generate natural and realistic movements. In the present study, a novel transformer-based denoising diffusion model is proposed to generate co-speech gestures. Moreover, we introduce a practical sampling trick for diffusion models to maintain the continuity between the generated motion segments while improving the within-segment motion likelihood and naturalness. Our model can be used for online generation since it generates gestures for a short segment of speech, e.g., 2 s. We evaluate our model on two large-scale speech-gesture datasets with finger movements using objective measurements and a user study, showing that our model outperforms all other baselines. Our user study is based on the Metahuman platform in the Unreal Engine, a popular tool for creating human-like avatars and motions.

With the development of automatic speech recognition and text-to-speech technology, high-quality voice conversion can be achieved by extracting source content information and target speaker information to reconstruct waveforms. However, current methods still require improvement in terms of inference speed. In this study, we propose a lightweight VITS-based voice conversion model that uses the HuBERT-Soft model to extract content information features. Unlike the original VITS model, we use the inverse short-time Fourier transform to replace the most computationally expensive part. Through subjective and objective experiments on synthesized speech, the proposed model is capable of natural speech generation and it is very efficient at inference time. Experimental results show that our model can generate samples at over 5000 KHz on the 3090 GPU and over 250 KHz on the i9-10900K CPU, achieving faster speed in comparison to baseline methods using the same hardware configuration.

Voice conversion systems have made significant advancements in terms of naturalness and similarity in common voice conversion tasks. However, their performance in more complex tasks such as cross-lingual voice conversion and expressive voice conversion remains imperfect. In this study, we propose a novel approach that combines a joint training speaker encoder and content features extracted from the cross-lingual speech recognition model Whisper to achieve high-quality cross-lingual voice conversion. Additionally, we introduce a speaker consistency loss to the joint encoder, which improves the similarity between the converted speech and the reference speech. To further explore the capabilities of the joint speaker encoder, we use the Phonetic posteriorgram as the content feature, which enables the model to effectively reproduce both the speaker characteristics and the emotional aspects of the reference speech.

Mobile robots are increasingly being utilized as guides in various settings. Ensuring efficient and hospitable interactions between robots and individuals is crucial for their successful implementation. However, there is a lack of understanding regarding the specific robot behaviors that can effectively guide individuals to their destinations while providing a sense of hospitality. In this study, we investigate two robot behaviors: “vertical oscillation motion” and “looking back motion,” with the objective of identifying behaviors that can optimize efficiency and hospitality in a guide robot. The vertical oscillation motion replicates the natural up-and-down movement of the human body during walking, while the looking back motion establishes eye contact and conveys a sense of hospitality. To evaluate the effectiveness of these behaviors, we conducted experiments with actual visitors at a shopping mall. Participants were exposed to the robot's behaviors and their responses were analyzed through questionnaires and interviews. The results revealed that the looking back motion successfully conveyed the robot's hospitality, while the vertical oscillation motion instilled a sense of urgency in individuals. However, it was found that only the looking back motion effectively motivated individuals to promptly commence walking towards the robot. These findings highlight the significance of incorporating specific robot behaviors to enhance both efficiency and hospitality in guiding individuals to their destinations. By understanding the impact of these behaviors, we can develop guide robots that accommodate a larger number of individuals while increasing their satisfaction. Future research endeavors should focus on evaluating the effectiveness of these behaviors in real-world environments, ensuring their practical applicability.

With recent technological developments, such as machine learning, data collection has become increasingly important. However, due to privacy issues, there is an ethical problem in collecting data on everyday activities. In this study, we aimed to develop a mechanism for acquiring consent interactively by using a robot and, as a first step, focus on a dialogue strategy in which users feel safe to provide their data. Specifically, we investigated the effects of reminder and rationale provided by the robot in situations where consent is obtained. We conducted two experiments in which users conversed with a virtual robot agent in a crowd setting. We found that the robot reminder improved the degree of understanding regarding the use of data. In addition, the permission rate was improved by robot notifications. However, no effect of rationale was observed. These results contribute to the discussion on the ethical aspects of data collection by robots.

This research attempts to develop a conversational robot that motivates users to interact with it for long-term non-task-oriented dialogue. In human-human interactions, the exchange of subjective opinions is important, but in human-robot interactions, it has been reported that the users’ willingness to talk decreases when they cannot attribute subjective opinions to robots. Accordingly, this study investigates what factors are involved in the attribution of subjective opinions to a conversational android robot. The experimental results identified three factors of the participants’ recognition involved in attributing subjective opinions to a conversational android: their own abilities, the android’s five senses, and their own emotionality.

Service robots have increasingly been investigated in retailing. Previous studies mainly focused on the effectiveness of recommendation with regard to a single robot, and whether and how the use of two robots combined can achieve better performance remain unclear. In this study, we address this by exploring the combination power of two service robots for product recommendation in a bakery. We placed one robot inside the store for product recommendation and the other robot outside to promote the inside robot. Particularly, we are interested in the effects of the outside robot on the inside robot's performance in product recommendation. Our results indicate that using the outside robot to promote the inside robot achieved more purchases over using the inside robot alone. Particularly, we discovered that the outside robot increased the attention of customers toward the inside robot; hence, more customers checked and purchased the products. Based on the findings, we discuss the important points for the effective use of service robots.

The prediction of dialogue acts (DA) labels on utterance-level in conversations can be treated as a sequence labeling problem, which requires context- and speaker-aware semantic comprehension, especially for Japanese. In this study, we pro-posed a hierarchical attention with the graph neural network (HAG) to consider the contextual interconnections as well as the semantics carried by the sentence itself. Concretely, the model use long-short term memory networks (LSTMs) to perform a context-aware encoding within a dialogue window. Then, we construct the context graph by aggregating the neighboring utterances. Subsequently, a speaker feature transformation is executed with a graph attention network (GAT) to calculate the interconnections, while a context-level feature selection is performed with a gated graph convolutional network (GatedGCN) to select the salient utterances that contribute to the DA classification. Finally, we merge the representations of different levels and conduct a classification with two dense layers. We evaluate the proposed model on Japanese dialogue act dataset (JPS-DA). The experimental results show that our method outperforms the baselines.

Identifying intentions is a critical task for comprehending the actions of others, anticipating their future behavior, and making informed decisions. However, it is challenging to recognize intentions due to the uncertainty of future human activities and the complex influence factors. In this work, we explore the method of recognizing intentions alluded under human behaviors in the real world, aiming to boost intelligent systems' ability to recognize potential intentions and understand human behaviors. We collect data containing real-world human behaviors before using a hand dispenser and a temperature scanner at the building entrance. These data are processed and labeled into intention categories. A questionnaire is conducted to survey the human ability in inferring the intentions of others. Skeleton data and image features are extracted inspired by the answer to the questionnaire. For skeleton-based intention recognition, we propose a spatial-temporal graph convolutional network that performs graph convolutions on both part-based graphs and adaptive graphs, which achieves the best performance compared with baseline models in the same task. A deep-learning-based method using multimodal features is proposed to automatically infer intentions, which is demonstrated to accurately predict intentions based on past behaviors in the experiment, significantly outperforming humans.

Speaker individual bias may cause emotion-related features to form clusters with irregular borders (non-Gaussian distributions), making the model sensitive to local irregularities of pattern distributions, resulting in the model over-fit of the in-domain dataset. This problem may cause a decrease in the validation scores in cross-domain (i.e., speaker-independent, channel-variant) implementation. To mitigate this problem, in this paper, we propose an adversarial training-based classifier to regularize the distribution of latent representations to further smooth the boundaries among different categories. In the regularization phase, the representations are mapped into Gaussian distributions in an unsupervised manner to improve the discriminative ability of the latent representations. A single Gaussian distribution is used for mapping the latent representations in our previous study. In this presented work, we adopt a mixture of isolated Gaussian distributions. Moreover, multi-instance learning was adopted by dividing speech into a bag of segments to capture the most salient part of presenting an emotion. The model was evaluated on the IEMOCAP and MELD datasets with in-corpus speaker-independent sittings. In addition, we investigated the accuracy of cross-corpus sittings in simulating speaker-independent and channel-variants. In the experiment, the proposed model was compared not only with baseline models but also with different configurations of our model. The results show that the proposed model is competitive with respect to the baseline, as demonstrated both by in-corpus and cross-corpus validation.

Emotional voice conversion (VC) aims to convert a neutral voice to an emotional one while retaining the linguistic information and speaker identity. We note that the decoupling of emotional features from other speech information (such as content, speaker identity, etc.) is the key to achieving promising performance. Some recent attempts of speech representation decoupling on the neutral speech cannot work well on the emotional speech, due to the more complex entanglement of acoustic properties in the latter. To address this problem, here we propose a novel Source-Filter-based Emotional VC model (SFEVC) to achieve proper filtering of speaker-independent emotion cues from both the timbre and pitch features. Our SFEVC model consists of multi-channel encoders, emotion separate encoders, pre-trained speaker-dependent encoders, and the corresponding decoder. Note that all encoder modules adopt a designed information bottleneck auto-encoder. Additionally, to further improve the conversion quality for various emotions, a novel training strategy based on the 2D Valence-Arousal (VA) space is proposed. Experimental results show that the proposed SFEVC along with a VA training strategy outperforms all baselines and achieves the state-of-the-art performance in speaker-independent emotional VC with nonparallel data.

Research on self-recommending product agents is under way. Compared with product promotion by humanoid robot agents, self-recommending agents can call a passing cus-omer's attention to product. As the customer's interest is focused on the product, the self-recommending agent can then give instructions, such as “pick me up.” It has been found that customers tend to follow such instructions, and touching the product is known to effectively support sales promotion. From two experiments in a convenience store, a self-recommending agent in this study was successful in attracting customer interest, which included handling the products. However, we also found that, after being picked up, the products' monologue resulted in customers leaving the product behind on the shelf. Herein, we examine the reasons why.

Robot-mediated interaction may be one of the best approaches to overcome the implementation-related limitations of face-to-face interaction between children and older adults. However, there has been little research about development or demonstration of teleoperated social robot systems that could be implemented in nurseries. We report a preliminary experiment wherein older adults greet nursery pupils on their way to and from school using a teleoperated social robot. The results suggest that using teleoperated robots in dialogue-based tasks motivate the children to interact with the robot, and it increases the probability of learning from the tasks. Further, the older adult teleoperators enjoyed the task and wanted to continue the same. In addition, the teleoperators could inherit the close relationships built up by the previous teleoperator even if the teleoperator was replaced. This study provides a starting point for further research on technology-mediated interactions between children and older adults.

In this study, we explore the effective form of social presence for a tele-operated robot to provide customer service. Particularly, we address the question on if and how a tele-operated robot displays the presence of an operator, and what effects it would have on people's perception and behavior toward it. We launched a tele-operated robot in a supermarket and had it deliver recipe flyers. We adjusted the robot's social presence by showing or not showing the photo of an operator's face (F) and using or not using voice conversion (V), leading to three forms of presence: Wizard of Oz (F: no, V: yes), costume (F: yes, V: yes), and telepresence (F: yes, V: no), which indicated the operator's presence from a low to a high level. We determined that the customers behaved significantly different when they faced the tele-operated robot in different forms. Our robot that exhibited a moderate presence of the operator (costume form) achieved the overall best performance. Based on these findings, we discuss both the strengths and weaknesses of the three forms of presence for a tele-operated robot and recommend the appropriate form for various applications.

Customer service with teleoperated robots is suscep-tible to the same problems related to stress as is emotional labor in general, such as for in-person customer service representatives. In this study, we aimed to reduce that stress by constructing a buddy-like rapport between the robot, which is the target of teleoperation, and its operator. For this purpose, we designed an empathetic interaction between the robot and the operator and conducted a customer service experiment to verify its effectiveness. Our results demonstrate that the proposed interaction can build rapport between the robot and the operator and the operator can feel more reassured. Although the effect on stress could not be isolated directly from the data, detailed analyses of the response to the questionnaires indicated that the proposed interaction may be useful to relieve stress.

In this paper, we report on a field study in which we employed two service robots in a bakery store as a sales promotion. Previous studies have explored public applications of service robots public such as shopping malls. However, more evidence is needed that service robots can contribute to sales in real stores. Moreover, the behaviors of customers and service robots in the context of sales promotions have not been examined well. Hence, the types of robot behavior that can be considered effective and the customers' responses to these robots remain unclear. To address these issues, we installed two tele-operated service robots in a bakery store for nearly 2 weeks, one at the entrance as a greeter and the other one inside the store to recommend products. The results show a dramatic increase in sales during the days when the robots were applied. Furthermore, we annotated the video recordings of both the robots' and customers' behavior. We found that although the robot placed at the entrance successfully attracted the interest of the passersby, no apparent increase in the number of customers visiting the store was observed. However, we confirmed that the recommendations of the robot operating inside the store did have a positive impact. We discuss our findings in detail and provide both theoretical and practical recommendations for future research and applications.

As a type of body language, gestures can largely affect the impressions of human-like robots perceived by users. Recent data-driven approaches to the generation of co-speech gestures have successfully promoted the naturalness of produced gestures. These approaches also possess greater generalizability to work under various contexts than rule-based methods. However, most have no direct control over the human impressions of robots. The main obstacle is that creating a dataset that covers various impression labels is not trivial. In this study, based on previous findings in cognitive science on robot impressions, we present a heuristic method to control them without manual labeling, and demonstrate its effectiveness on a virtual agent and partially on a humanoid robot through subjective experiments with 50 participants.

Social robotics recommendations have been studied for a long time, and many existing studies have addressed in-store recommendations using robots. However, it has been pointed out that many studies conducted in "in-the-wild" field environments have only focused on the initial stages of customer purchase behavior, such as stopping and engaging in a conversation, and few have been able to induce an interest in the product and even purchase. One of the causes is that the robot itself attracts most of the customer’s attention, making it difficult for customers to be interested in the robot’s recommendations. To solve this problem, this study examines the inclusion of clear and specific instructions to customers in interactions in which the robot recommends services and products. We conducted a field experiment to confirm the effectiveness of such instructive interactions, and found that customers are more likely to be interested in the content recommended by the robot, rather than in the robot itself, through the instructive interaction.

The goal of this study is to develop an autonomous conversational robot that acquires knowledge about human society in the real world. In this study, we define experience data as image data obtained from a camera (visual information) and dialogue data obtained from a microphone (auditory information). From experience data, the robot acquires knowledge about the co-occurrence between humans and items, that is, whether it is usual or not for humans to have the items. Not only does the proposed system acquire knowledge, but also generates utterances based on acquired knowledge. We have implemented the proposed method on an android, a human-like robot. We conducted an experiment in which the android was placed in a real-world environment (shopping mall) and interacted with visitors. The percentage of positive responses to the robot's questions based on the acquired knowledge suggests that this system can acquire knowledge from experience data.

In this study, we describe an improved version of our proposed model to generate gaze movements (eye and head movements) of a dialogue robot in multi-party dialogue situations, and investigated how the impressions change for models created by data of speakers with different personalities. For that purpose, we used a multimodal three-party dialogue data, and first analyzed the distributions of (1) the gaze target (towards dialogue partners or gaze aversion), (2) the gaze duration, and (3) the eyeball direction during gaze aversion. We then generated gaze behaviors in an android robot (Nikola) with the data of two people who were found to have distinctive personalities, and conducted subjective evaluation experiments. Results showed that a significant difference was found in the perceived personalities between the motions generated by the two models.

Robot agents are increasingly used for user services, and society is becoming increasingly familiar with such robots. Most robots are of the humanoid type, which are easy to recognize as interaction partners. By interacting with a user, these agents may be able to generate user interest in a product and successfully sell it. However, according to previous studies, it has been suggested that users who are interested in the movement and appearance of the agent may focus only on these aspects and not listen to the recommendation. Therefore,we hypothesized that by making the product the agent, which we call a Self Recommendation Agent(SRA), the attention of the user would be focused on the product itself. Hence, if a user is interested in the agent, it is the same as paying attention to the product. Therefore, we expect that such an agent will be able to gain more attention from users than conventional agents while providing recommendations. To investigate the effectiveness of this agent, we set up a store in a shopping mall and conducted sales experiments. In this field study, we conducted an experiment to compare sales between a SRA and a robot agent.As a result, the SRA was able to make recommendations to many more users than the robot agent, and the users who received recommendations from the SRA remembered more about the recommended product than those who received recommendations from the robot agent.Based on these results, we confirm the possibility that the SRA is effective for advertising.

In recent years, web conferencing systems have become widely used in non-face-to-face interactions. However, web conferencing systems make difficult to establish eye contact for users and can lead to decrease in non-verbal communication. In this paper, we propose a method that will enhance a user’s feeling of interlocutor’s attention, even when the line of sight does not actually match, by displaying a thought bubble for the interlocutor filled with the user’s in-camera image. Because robots have the same gaze problem as humans we thought it is important to know that this web conference system can represent the attention of a simple moving robot without using complex gaze mechanisms. So, we conducted a within-subject design experiment using a robot whose gaze and movements can be controlled and a participant as interlocutors in a web conference with and without the thought bubble. The experimental results indicate that our proposed method increased the sense of being looked at.

In a multi-person face-to-face dialogue, people naturally gaze towards others or avert their gazes, according to their dialogue roles and mental states. The goal of this research is to develop a robot/agent that can generate human-like eye movements, in order to achieve smoother and more engaged dialogue interactions with multiple users. In this study, we analyze the gaze behaviors in three-party dialogue data, accounting for turn-taking, dialogue roles and gaze aversions during the dialogue interactions. Based on the analysis results, we implemented gaze models on a humanoid robot. Subjective evaluation experiments showed that natural behaviors are achieved by our proposed gaze control system, which accounts for dialogue roles and eyeball movement control.

More and more robots are providing services that involve conversations with people in daily life. Among them, the movement and positioning of the robot are effective factors to improve the robot’s expression, and the distance from the human and the direction of the robot are reported as important parameters. In multi-person dialogues, it is expected that robots can communicate their impressions, emotions, and thoughts to other robots by successfully adjusting their movement and positioning. We investigated the effect of the robot’s movement in a conversation on the subsequent conversation and the expression of the moving robot. It was found that there were intentions and emotions of the robot that could be expressed by moving forward and backward, respectively. The next step of this experiment is to conduct interaction between subjects using a system that incorporates forward and backward movements.

In this study, to mitigate cyberbullying in communication tools, we propose a method in which a chatbot pretending to be a member of a chat group defends the victim by proxy without permission (i.e., hijacking and proxy persuasion). An experiment was conducted to verify the effectiveness of the proposed method, the results of which showed that it increases the defending behavior of a user who was temporarily hijacked by the chatbot. However, compared with the existing method, the proposed method was found to have a problem with the continued use of the system.

Deep learning has dramatically improved the performance of speech recognition and image recognition. However, this does not mean that human intelligence has been reproduced. Nevertheless, it is expected that research and development into robots with human-like intelligence, which interact like humans, and have social relationships and coexist with humans, will continue to develop quickly. The speaker has been working on the research and development of autonomous robots that interact with people in the JST ERATO Ishiguro Symbiotic Human Robot Interaction Project.

In this paper, we propose an adversarial auto-encoder-based classifier, which can regularize the distribution of latent representation to smooth the boundaries among categories. Moreover, we adopt multi-instance learning by dividing speech into a bag of segments to capture the most salient moments for presenting an emotion. The proposed model was trained on the IEMOCAP dataset and evaluated on the in-corpus validation set (IEMOCAP) and the cross-corpus validation set (MELD). The experiment results show that our model outperforms the baseline on in-corpus validation and increases the scores on cross-corpus validation with regularization.

Social robots are used in daily life. One of the applications of social robots is as recommendation systems. Previous research has mainly investigated how persuasive recommendations can be improved by focusing on the non-verbal/verbal behavior of robots. However, to use robots as recommendation systems every day, it is extremely important to examine the persistence of repeated persuasion over a long term, rather than the effect of one-time persuasion. Therefore, the objective of this study was to investigate the persistence of repeated persuasive of robots. For this purpose, robots with three types of behavior (Expert Behavior, Local Behavior, and Growth Behavior) recommended nutrition bars in a situation of daily consumption behavior for two weeks. We could confirm significant differences in the persistent persuasiveness in each behavior. The results suggested that the combination of value co-creation using local information and meta-trust expression had a significant impact on the persistence of the repeated persuasiveness of the robots in the longitudinal period. However, the acceptance of the recommendation robot system decreased due to the increase in the amount of information during recommendation; therefore, a new recommend system to solve this problem is desired.

People with response time (RT) management issues usually have high situational communication apprehension, and fear of negative evaluation (FNE) in face-to-face (FtF) and online interactions. Decreasing the stress related to RT management can reduce situational communication apprehension and FNE, ensuring communication successful. In this study, we propose an interactive response strategy involving a robot avatar in a video conference that can reduce the stress related to RT management in a person and making communication successful. Two types of robotic video conferencing systems (VCSs) were considered: 1) a conventional robotic video conferencing system (VCS) where interactive response of robot avatar does not change concerning variation in RT of the subject and 2) a robotic VCS where interactive response of robot avatar changes concerning variation in RT of the subject. Situational communication apprehension measure (SCAM), FNE, sense of being attending (SoBA), and intention to use (ITU) are used as indexes for noticing the decrease in stress related to RT management of a subject in a communication scenario. A total of 51 subjects (M = 33.20, SD = 7.58 years) participated in the subjective evaluation of the web-based survey. The proposed interactive response of a robot avatar was found to have a significant effect on the stress related to RT management in a person. A significant decrease in expected SCAM and FNE scores of subjects was observed. Furthermore, a significant increase in expected SoBA and ITU scores of subjects was also observed. This study contributes to the literature in terms of the impact of the proposed interactive response of the robotic avatar on the stress related to RT management in a person in the video conference, assessed by SCAM, FNE, SoBA, and ITU indexes.

To explore the interaction design of an autonomous social robot stationed in a hotel room, we conducted a Wizard of Oz study. We developed a teleoperated robotic system that appears to move autonomously through voice-to-synthesis processing. Comparing the evaluation of the latest autonomous case with one of these teleoperated cases, the results show that it is possible to construct a robotic system that is more highly rated in terms of warmth, competence, and enjoyment of conversation. The results also suggest novel forms of the hotel room robot’s interactive services, such as a hotel-life management service and conversation partner service as a role of a listening presence, which draws out and understands with the guests’ talk.

Under the circumstances that social robots are increasingly being developed and studied in service encounters at public spaces, are they introduced into residential environments (i.e., private space)? This study hypothesizes that a personal assistant device in residential environments should wear human-like appearance to engage in service as conversation partner. We implemented the interaction design that provides regular services as the current personal assistant and additional service as conversation partner, and then conducted a field experiment where the participants stayed in the hotel rooms with a smart speaker or a social robot. The results support the hypothesis in that of conversation amount and emotional experience by conversation. The results also suggest the possibility of commercial service, namely conversational advertisement through social robots.

Deep discussions on topics without definite answers are important for society, but they are also challenging to facilitate. Recently, advances in the technology of using robots to facilitate discussions have been made. In this study, we developed a multiple-robot mediated discussion system (m-RMDS) to support discussions by having multiple robots assert their own points and lead a dialogue in a group of human participants. The robots involved the participants in a discussion through asking them for advice. We implemented the m-RMDS in discussions on difficult topics with no clear answers. A within-subject experiment with 16 groups (N=64) was conducted to evaluate the contribution of the m-RMDS. The participants completed a questionnaire about their discussion skills and their self-confidence. Then, they participated in two discussions, one facilitated by the m-RMDS and one that was unfacilitated. They evaluated and compared both experiences across multiple aspects. The participants with low confidence in conducting a discussion evaluated the discussion with m-RMDS as easier to move forward than the discussion without m-RMDS. Furthermore, they reported that they heard more of others' frank opinions during the facilitated discussion than during the unfacilitated one. In addition, regardless of their confidence level, the participants tended to respond that they would like to use the system again. We also review necessary improvements to the system and suggest future applications.

In recent years, a substantial amount of research has been aimed at realizing a social robot that can maintain long-term user interest. One approach is using a dialogue strategy in which the robot makes a remark based on previous dialogues with users. However, privacy problems may occur owing to private information of the user being mentioned. We propose a novel dialogue strategy whereby a robot mentions another robot in the form of gossiping. This dialogue strategy can improve the sense of conversation, which results in increased interest while avoiding the privacy issue. We examined our proposal by conducting a conversation experiment evaluated by subject impressions. The results demonstrated that the proposed method could help the robot to obtain higher evaluations. In particular, the perceived mind was improved in the Likert scale evaluation, whereas the robot empathy and intention to use were improved in the binary comparison evaluation. Our dialogue strategy may contribute to understanding the factors regarding the sense of conversation, thereby adding value to the field of human-robot interaction.

We introduce a minimal design approach to achieve a robot for interactive doll therapy. Our approach aims for positive interactions with older adults with dementia by just expressing the most basic elements of human-like features and relying on the user's imagination to supplement the missing information. Based on this approach, we developed HIRO, a baby-sized robot with abstract body representation and without face. The recorded voice of a real human infant emitted by robots enhance human-like features of the robot and then facilitate emotional interaction between older people and the robot. A field study showed that HIRO was accepted by older adults with dementia and facilitated positive interaction by stimulating their imagination.

The following topics are dealt with: human-robot interaction; mobile robots; learning (artificial intelligence); control engineering computing; humanoid robots; service robots; medical robotics; robot vision; computer aided instruction; motion control.

Tsunehiro Arimoto, Ryuichiro Higashinaka, Kou Tanaka, Takahito Kawanishi, Hiroaki Sugiyama, Hiroshi Sawada, Hiroshi Ishiguro. Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2020.

The purpose of this study was to investigate the effect of robotic mediation in promoting conversation of older adults with dementia for improving behavioral and psychological symptoms of dementia (BPSD). In the longitudinal study, we explored how the BPSD in older adults could be affected by use of a teleoperated android robot named Telenoid that could potentially promote conversation. This study was conducted at a long-term residential care facility, and it was a prospective, 10-week, single arm, exploratory user study. Diminishing the barrier of digital divide, the android robot provided those with dementia with a way to remotely communicate with others. BPSD were assessed using the Neuropsychiatric Inventory Nursing Home Version (NPI-NH), and we investigated changes in the participants over time. According to NPI-NH scores, Telenoid could improve BPSD in older adults with "anxiety," and "appetite and eating change." The results indicate that as a non-pharmacological approach, daily use of the conversation promoting android robot could have a therapeutic effect on the BPSD in older adults.

Magnetic lead screw (MLS) is a feed screw that allows non-contact power transmission. The MLSs have some advantages such as reduction of particle emission and improving positioning accuracy. However, conventional MLSs need a supporting mechanism using linear guides or slide bearings. They deteriorate the advantages of MLSs because of friction. In order to solve this problem, we propose a magnetically levitated lead screw (MLLS) for complete non-contact transmission. The proposed magnetic structure and operating principle are verified through a three-dimensional finite element method. The suspension force, thrust, and torque characteristics are investigated, and the effectiveness of the MLLS are clarified.

We, humans, have innate brain function to recognize humans. Therefore, very humanlike robots, androids, can be ideal information media for human-robot/computer interaction. The speaker, Ishiguro, has developed various types of interactive robots and androids so far. These robots can be used for studying on the technologies and understanding human natures. He has contributed to establish the research area of Human-Robot Interaction with the robots.

Humans use not only verbal information but also non-verbal information in daily communication. Among the non-verbal information, we have proposed methods for automatically generating hand gestures in android robots, with the purpose of generating natural human-like motion. In this study, we investigate the effects of hand gesture models trained/designed for different speakers on the impression of the individuality through android robots. We consider that it is possible to express individuality in the robot, by creating hand motion that are unique to that individual. Three factors were taken into account: the appearance of the robot, the voice, and the hand motion. Subjective evaluation experiments were conducted by comparing motions generated in two android robots, two speaker voices, and two motion types, to evaluate how each modality affects the impression of the speaker individuality. Evaluation results indicated that all these three factors affect the impression of speaker individuality, while different trends were found depending on whether or not the android is copy of an existent person.

The growing number of people living with dementia puts a heavy load on elderly care nurses. The use of humanoid robots as communication partners can help alleviate this load with the goal to assist seniors in maintaining their cognitive skills through conversation. In this paper we present an autonomous robotic dialogue system with the humanoid robot Bonoid that is capable of learning a conversation strategy from interactions with humans. The system was tested and evaluated with first results showing natural and engaging interactions with seniors, and the robotic system acquiring a conversation strategy through the use of reinforcement learning.

This study presents a learning-by-imitation technique that learns social robot interaction behaviors from natural human- human interaction data and requires minimum input from a designer. To solve the problem of responding to ambiguous human actions, a novel topic clustering algorithm based on action cooccurrence frequencies is introduced. The system learns human-readable rules that dictate which action the robot should take, based on the most recent human action and the current estimated topic of conversation. The technique is demonstrated in a scenario where the robot learns to play the role of a travel agent. The proposed technique outperformed several baseline techniques in qualitative and quantitative evaluations. It responded more accurately to ambiguous questions and participants found it was easier to understand, provided more information, and required less effort to interact with.

Many recent studies have shown that behaviors and interaction logic for social robots can be learned automatically from natural examples of human-human interaction by machine learning algorithms, with minimal input from human designers [1--4]. In this work, we exceed the capabilities of the previous approaches by giving the robot memory . In earlier work, the robot's actions were decided based only on a narrow temporal window of the current interaction context. However, human behaviors often depend on more temporally distant events in the interaction history. Thus, we raise the question of whether (and how) an automated behavior learning system can learn a memory representation of interaction history within a simulated camera shop scenario. An analysis of the types of memory-setting and memory-dependent actions that occur in the camera shop scenario is presented. Then, to create more examples of such actions for evaluating a shopkeeper robot behavior learning system, an interaction dataset is simulated. A Gated Recurrent Unit (GRU) neural network architecture is applied in the behavior learning system, which learns a memory representation for performing memory-dependent actions. In an offline evaluation, the GRU system significantly outperformed a without-memory baseline system at generating appropriate memory-dependent actions. Finally, an analysis of the GRU architecture's memory representation is presented.

Learning from human interaction data is a promising approach for developing robot interaction logic, but behaviors learned only from offline data simply represent the most frequent interaction patterns in the training data, without any adaptation for individual differences. We developed a robot that incorporates both data-driven and interactive learning. Our robot first learns high-level dialog and spatial behavior patterns from offline examples of human--human interaction. Then, during live interactions, it chooses among appropriate actions according to its curiosity about the customer's expected behavior, continually updating its predictive model to learn and adapt to each individual. In a user study, we found that participants thought the curious robot was significantly more humanlike with respect to repetitiveness and diversity of behavior, more interesting, and better overall in comparison to a non-curious robot.

Emotional labor in text-based chat environments will be widespread in the near future. It is difficult for novice workers to make smooth conversation with an angry customer in a customer-support work- ing environment, one of the stressful emotional labors. In this pa- per, we propose a method to suppress the customer's anger based on a communication agent introduced into customer-support work in order to facilitate smooth communication. The agent first estab- lishes a positive relationship with the customer, and then the atti- tude changes to be sympathetic to the customer-support worker in order for the customer to follow sympathy and reduce anger. From the experimental evaluation, it is found that there is the possibil- ity of suppressing the participants' anger by this method, but the anger itself is at a higher level than in the common environment without the agent.

While more and more robots have been employed in the service industry, the impact of the human--robot social interaction on a heartwarming interaction service mostly remains an open question. For the purpose of exploring the possibility that a humanoid robot engages in a heartwarming interaction service, we developed herein the first prototype of the service system through a humanoid robot and conducted a field study, where a humanoid robot engages in the service for a customer at a hotel to collect a customer's impression of the service by questionnaire. The results demonstrate a humanoid robot's potential of engaging in a heartwarming interaction service that enhances customer satisfaction of the whole service. An exploratory analysis suggests the differences of the impact in sex and long-term interaction. Finally, we discuss the possibility of another application, namely, effective advertisement through a heartwarming interaction with a humanoid robot.

Although it is perceivable that interpersonal touch affects recipient's impression of touch provider as well as the information relating to the provider alike, its utility in touch mediated through communication devices (virtual interpersonal touch) remains unclear to date. In this article, we report the alleviating effect of virtual interpersonal touch on social judgment. In particular, we show that virtual hug with a remote person modulates the impression of the hearsay information about an absentee. In our experiment, participants rate their impressions as well as note down their recall of information about a third person. We communicate this information through either a speaker or a huggable medium. Our results show that virtual hug reduces the negative inferences in the recalls of information about a target person. Furthermore, they suggest the potential that the mediated communication offers in moderating the spread of negative information in human community via virtual hug.

We report results of the analyses of the effect of communication through a humanoid robot in comparison with in-person, video-chat, and speaker on frontal brain activity of older people during an storytelling experiment. Our results suggest that whereas communicating through a physically embodied medium potentially induces a significantly higher pattern of brain activation with respect to video-chat and speaker, its difference is non-significant in comparison with in-person communication. These results imply that communicating through a humanoid robot induces a pattern of brain activity in older people that is potentially similar to in-person communication. Our findings benefit researchers and practitioners in rehabilitation and elderly care facilities in search of effective means of communication with their patients to increase their involvement in the incremental steps of their treatments. Moreover, they imply the utility of brain information as a promising sensory gateway in characterization of the behavioural responses in human-robot interaction.

Hiroshi Ishiguro gave a lecture to a group of young students. We employed CommU, the desktop social robot, to manage the questions and answers for the talk. We encouraged the students to ask questions anytime. Half of the classroom was told to ask questions by raising their hand while the other half was shown an online messaging system developed for CommU, which allows the audience to post questions, which the robot would directly say. We had a gatekeeper to monitor for invalid sentences. In the middle of the presentation we asked the students to shift roles. The robot used a neural network based estimator of interruptibility to find the best time to speak. We did not expect too many questions, but the audience really embraced using the robot. They posted 44 questions to the presenter through CommU. On the other hand they asked 8 direct questions by raising their hands and standing up. Students thought that they gained more information from the lecturer using the robot than using the conventional method. In this instance we didnt stop the students from asking too many questions, but in a real-world application the gatekeeper will have to play an important role.

Authority and obedience are key regulatory elements in a society. Robots are becoming important part of our world, and are starting to interact in domains in which authority is an important aspect, as healthcare, teaching or law enforcement. Yet, there is little research on how people behave when robots show authority. In particular, although extensive investigations have been carried out on how authority can circumvent people's morality with experiments such as Milgram's or Stanford Prison, almost no research evaluated the effect of robots pushing the limits of people's own morality. This experiment tries to study this aspect by using a robot (geminoid) with the appearance, and thus authority of a famous person, and by pushing the boundaries asking morally controversial requests. The results show that, even though most people hesitate and recognize the requests as socially inappropriate, they obey to robots with authority. This suggests that the authority of the robot can push people to perform tasks usually considered as inappropriate.

We propose Intimate Touch Conversation (ITC) as a new remote communication paradigm in which an individual, who is holding a telepresence humanoid, engages in a conversation-over-distance with a remote partner that is tele-operating the humanoid. We investigate the effect of this new communication paradigm on interpersonal closeness in comparison with in-person and video-chat. Our results suggest that ITC significantly enhances the feeling of interpersonal closeness, as opposed to video-chat and in-person. In addition, they show that ITC allows elderly people to find their conversation more interesting. These results imply that feeling of intimate touch that is evoked by the presence of teleoperated android enables elderly users to establish a closer relationship with their conversational partners over distance, thereby reducing their feeling of loneliness. Our findings benefit researchers and engineers in elderly care facilities in search of effective means of establishing a social relation with their elderly users to reduce their feeling of social isolation and loneliness.

In recent decades and years classroom teaching has been influenced by the appearance of digital devices and the pervasiveness of the internet. In recent years robots started appearing in learning environments. Even though classroom presentations improved considerably, interaction between students and teachers has not improved greatly. We propose a new way of using a desktop robot and an online voting/messaging system to make the classroom experience more interactive. Concretely, in this paper we report on a lecture which was assisted by a robot. The students posted questions and opinions on an online messaging platform. The messages were then sent to the robot to be posed to the teacher, either directly or through a voting procedure. Students found this new way of interaction very efficient and useful for improving their communication with the teacher. At the same time the number of interactions should be controlled so it does not interfere with the teaching process.

This paper addresses the effects of a subtle pause in reactions during human-robot touch interactions. Based on the human scientific literature, people's reaction times to touch stimuli range from 150 to 400 msec. Therefore, we decided to use a subtle pause with a similar length for reactions for more natural human-robot touch interactions. On the other hand, in the human-robot interaction research field, a past study reports that people prefer reactions from a robot in touch interaction that are as quick as possible, i.e., a 0- second reaction time is slightly preferred to 1- or 2- second reaction times. We note that since the resolution of the study's time slices was every second, it remains unknown whether a robot should take a pause of hundreds of milliseconds for a more natural reaction time. To investigate the effects of subtle pauses in touch interaction, we experimentally investigated the effects of reaction time to people's touch with a 200-msec resolution of time slices between 0 second and 1 second: 0 second, 200, 400, 600, and 800 msec. The number of people who preferred the reactions with subtle pauses exceeded the number who preferred the 0- second reactions. However, the questionnaire scores did not show any significant differences because of individual differences, even though the 400-msec pause was slightly preferred to the others from the preference perspective.

To provide a better discussion experience in current argumentative dialogue systems, it is necessary for the user to feel motivated to participate, even if the system already responds appropriately.In this paper, we propose a method that can smoothly introduce argumentative dialogue by inserting an initial discourse, consisting of question-answer pairs concerning personality.The system can induce interest of the users prior to agreement or disagreement during the main discourse.By disclosing their interests, the users will feel familiarity and motivation to further engage in the argumentative dialogue and understand the system's intent.To verify the effectiveness of a questionanswer dialogue inserted before the argument, a subjective experiment was conducted using a text chat interface.The results suggest that inserting the questionanswer dialogue enhances familiarity and naturalness.Notably, the results suggest that women more than men regard the dialogue as more natural and the argument as deepened, following an exchange concerning personality.

The present study describes neuromagnetic Geminoid control system by using single-trial decoding of bilateral hand movements as a new approach to enhance a user's ability to interact with a complex environment through a multidimensional brain-computer interface (BCI). Two healthy participants performed or imagined four types of bilateral hand movements during non-invasive magnetic field measurements to control a human-like robot (Geminoid HI-2) in real-time. By applying a nonlinear support vector machine (SVM) method to classify the four movements regarding magnetoencephalography (MEG) sensors obtained from the sensorimotor area, we found the mean accuracy of a 2-class classification using the amplitudes of neuromagnetic fields to be particularly suitable for real time control applications, with accuracies comparable to those obtained in previous studies involving unilateral hand movement. Moreover, our results demonstrated that decoding bilateral movements in real-time is a promising option to design multidimensional-control based BCI applications.

We present results of a case study on effect of humanoid in comparison with voice-only communication on frontal brain activity of elderly adults. Our results indicate that use of humanoid induces an increase in frontal brain activity. Additionally, these results imply an increase in their salivary Immunoglobulin A (sIgA) Antibody, thereby suggesting physical embodiment as a potential health factor in communication with elderly individuals. Such increases in hormonal as well as frontal brain activity, as observed in healthy condition, suggest the potential that physical embodiment can offer to the solution concept of sustaining the process of cognitive decline associated with aging and its consequential diseases such as Alzheimer.

The behavior of a social robot is designed based on either its emotional state or social situation. In this research, we defined these expressions as the affective and social expressions, and proposed a new emotional expression method to integrate these two expressions. It is known that a human has involuntary facial muscles around the eyes, and voluntary facial muscles around the mouth. Thus, it is considered that facial muscles around the eyes express the affective expressions, while facial muscles around the mouth express the social expressions. In an experiment involving a human-robot conversation, using the proposed method, the results showed a sense of human likeness and sociality. Furthermore, the intimacy evaluation results showed that the social expressions work effectively when the intimacy does not require high social bonding such as the desire for friendship. However, the affective expressions are necessary for evaluating intimacy that requires higher social bonding, such as wanting to live with someone.

This paper hypothesized that a straddle-type vehicle (e.g., a motorcycle) would be a suitable platform for haptic human-machine interactions that elicits affective responses or positive modulations of human emotion. Based on this idea, a new human-machine interface (HMI) for a straddle-type vehicle was proposed for haptically interacting with a rider, together with other visual (design), tactile (texture and heat), and auditory features (sound). We investigated endocrine changes after playing a riding simulator with either new interactive HMI or typical HMI. The results showed, in comparison with the typical HMI, a significant decrease of salivary cortisol level was found after riding the interactive HMI. Salivary testosterone also tended to be reduced after riding the interactive HMI, with significant reduce in salivary DHEA. The results demonstrated that haptic interaction from a vehicle, as we hypothesized, can endocrinologically influence a rider and then may mitigate rider's stress and aggression.

For a safe, natural and effective human-robot social interaction, it is essential to develop a system that allows a robot to demonstrate the perceivable responsive behaviors to complex human behaviors. We introduce the Multimodal Deep Attention Recurrent Q-Network using which the robot exhibits human-like social interaction skills after 14 days of interacting with people in an uncontrolled real world. Each and every day during the 14 days, the system gathered robot interaction experiences with people through a hit-and-trial method and then trained the MDARQN on these experiences using end-to-end reinforcement learning approach. The results of interaction based learning indicate that the robot has learned to respond to complex human behaviors in a perceivable and socially acceptable manner.

We present a minimalist two-hidden-layer neural architecture for emotional state estimation using electroencephalogram (EEG) data. Our model introduces a new meta-parameter, referred to as reinforced gradient coefficient, to overcome the peculiar vanishing gradient behaviour, thereby enabling the network to further explore the gradient landscape. This allows our model to further reduce its deviation from expected prediction to significantly minimize its estimation error. Furthermore, it adopts a weighing step that captures the discrepancy between two consecutive predictions during training. The value of this weighing factor is learned throughout the training phase, given its positive effect on the overall prediction accuracy of the model. We validate our approach through comparative analysis of its performance in contrast with state-of-the-art techniques in the literature, using two well known EEG databases. Our model shows significant improvement on prediction accuracy of emotional states of human subjects, while maintaining a highly simple, minimalist architecture.

We propose a probabilistic model that generates nod motions based on utterance categories estimated from the speech input. The model comprises two main blocks. In the first block, dialogue act-related categories are estimated from the input speech. Considering the correlations between dialogue acts and head motions, the utterances are classified into three categories having distinct nod distributions. Linguistic information extracted from the input speech is fed to a cluster of classifiers which are combined to estimate the utterance categories. In the second block, nod motion parameters are generated based on the categories estimated by the classifiers. The nod motion parameters are represented as probability distribution functions (PDFs) inferred from human motion data. By using speech energy features, the parameters are sampled from the PDFs belonging to the estimated categories. Subjective experiment results indicate that the motions generated by our proposed approach are considered more natural than those of a previous model using fixed nod shapes and hand-labeled utterance categories.

In this study, ease of talking is compared in three conversation forms: face-to-face conversation (direct conversation) and conversation with a robot that is tele-operated from the same room and different room (semi-indirect and indirect conversation, respectively). As a result, it was found that the silent time, which indicates how long the participants make an interlocutor wait until a response is given, in both semi-indirect and indirect conversation exceeds that in direct conversation. Additionally, the silent time in direct conversation after semi-indirect conversation exceeds that after direct conversation. The results indicate that a robot tele-operated from different rooms as well as from the same room can provide partners with time to think of the words uttered by them. Moreover, the results indicate that these effects are sustained in direct conversation following a conversation with a robot that is operated from the same room. These results indicate the potential of constructing a system of conversation via a robot that can be used in easier counseling for clients who experience difficulties in directly communicating with a counselor.

Our research goal was to develop a robot counseling system. It is important for a counselor to promote self-disclosure of clients to reduce their anxiety feelings. However, when a counselor is human, clients sometimes hesitate to disclose intrusive topics due to embarrassment and self-esteem issues. We hypothesized that a robot counselor, on account of its unique kind of agency, could remove mental barriers between the counselor and the client, and promote in-depth self-disclosure about negative topics. In this study, we prepared two robots (an android and a desktop robot) as robot counselors. First, we confirmed that subjects eagerly self-disclosed to these prepared robots from the numbers of spoken words about self-disclosure in preliminary experiment. And next, we conducted the experiment to verify whether it is possible to expose more of subjects' weakness to robots than humans. The experimental result suggested that robots can draw out subjects' self-disclosure about negative topics than the human counselor.

This study extends existing Rubber Hand Illusion (RHI) experiments to employ life-size full-body humanlike android robot to investigate body ownership illusion and the sense of agency. For this study, we designed a novel experimental setting. The android robot moved synchronously with a participant's head and arm motion. Furthermore, a stereoscopic camera was mounted on the android's head and the video stream from the camera was provided to a head mounted display (HMD) that the participant wore. Then, the participant and the robot were facing each other, so that the participant could see his/her own body in the HMD like a reflection in a mirror. During the experiment, the participant was asked to touch his/her own body by the hands of an android robot. The results of our study confirmed that the body ownership process worked with the full body android robot. The results also showed that our system enabled the participants to feel the sense of agency to some extent.

Since a talking robot cannot escape from errors in recognizing user's speech in daily environment, its verbal responses are sometimes felt as incoherent with the context of conversation. This paper presents a solution to this problem that generates a social context where a user is guided to find coherency of the robot's utterances, even though its response is produced according to incorrect recognition of user's speech. We designed a novel turn-taking pattern in which two robots behave according to a pre-scheduled scenario to generate such a social context. Two experiments proved that participants who talked to two robots using that turn-taking pattern felt robot's responses to be more coherent than those who talked to one robot not using it; therefore, our proposed turn-taking pattern generated a social context for user's flexible interpretation of robot's responses. This result implies a potential of a multiple robots approach for improving the quality of human-robot conversation.

This pilot study reports an experiment where we introduced huggable communication media into daytime sleep in co-sleeping situation. The purpose of the experiment was to investigate whether it would improve soothing child users' sleep and how hugging experience with anthropomorphic communication media affects child's anthropomorphic impression on the media in co-sleeping. In the experiment, nursery teachers read two-year-old or five-year-old children to sleep through huggable communication media called Hugvie and asked the children to draw Hugvie before and after the reading to evaluate changes in their impressions of Hugvie. The results show the difference of sleeping behavior with and the impressions on Hugvie between the two classes. Moreover, they also showed the possibility that co-sleeping with a humanlike communication medium induces children to sleep deeply.

This study examined response tendencies in 4-year-old Japanese children (N = 45) to yes-no questions asked by a communicative, or a non-communicative robot. The children watched a video of a robot that was either responsive (communicative condition), or unresponsive (non-communicative condition) to human actions. Then, all the children watched a video of the same robot asking yes-no questions pertaining to familiar and unfamiliar objects. The children in both conditions exhibited a nay-saying bias to questions about unfamiliar objects, with children in the non-communicative condition tending to show a stronger nay-saying bias than children in the communicative condition. Children's response tendencies towards questions asked by humans and other agents are discussed.

This work studies a dialogue strategy aimed at building people's motivation for talking with autonomous conversational robots. Even though spoken dialogue systems continue to develop rapidly, the existing systems are insufficient for continuous use because they fail to motivate users to talk with them. One reason is that users fail to realize that the intentions of the system's utterances are based on its values. Since people recognize the values of others and modify their own values in human-human conversations, we hypothesize that a dialogue strategy that makes users saliently feel the difference of their own values and those of the system will increase motivation for the dialogues. Our experiment, which evaluated human-human dialogues, supported our hypothesis. However, an experiment with human-android dialogues failed to produce identical results, suggesting that people did not attribute values to our android. For a conversational robot, we need additional techniques to convince people to believe a robot speaks based on its own values and opinions.

Interaction Composer, a visual programming environment designed to enable programmers and non-programmers to collaboratively design social human-robot interactions in the form of state-based flows, has been in use at our laboratory for eight years. The system architecture and the design principles behind the framework have been presented in other work, but in this paper we take a case-study approach, examining several actual examples of the use of this toolkit over an eight-year period. We examine the structure and content of interaction flows, identify common design patterns, and discuss elements of the framework which have proven valuable, features which did not solve their intended purposes, and ways that future systems might better address these issues. It is hoped that the insights gained from this study will contribute to the development of more effective and more usable tools and frameworks for interaction design.

Is it possible to design robots of the future so that they can enhance people's creative endeavors? Forty-eight young adults were asked to produce their most creative ideas in a small Zen rock garden in a laboratory setting. Participants were randomly assigned to one of two conditions. In one condition, the robot Robovie (through a Wizard of Oz interface) encouraged participants to generate creative ideas (e.g., “Can you think of another way to do that?”), and pulled relevant images and video clip from the web for each participant to look at which could help spur the participant into more creative expressions. In a second condition, participants engaged in the same Zen rock garden task with the same core information, but through the modality of a self-paced PowerPoint presentation. Results showed that participants engaged in the creativity task longer and provided almost twice the number of creative expressions in the robot condition compared to the PowerPoint condition. Discussion focuses on a vision of social robotics coupled with advances in natural language processing to enhance the human creative mind and spirit.

We conducted a field study to investigate the social acceptance of social robots by stores, particularly for attracting passersby, which today's robots can autonomously perform. From interviews with ten store managers, we identified two main reasons they want to employ such social robots in their stores: robots offer cheap labor and provide unique value that humans cannot. They believe that robots are good at attracting the attention of visitors without causing or receiving stress. We also conducted three case studies in which we observed how store managers employed social robots in their stores. Each store manager requested different designs in the preparation phase. After deployment, we found that the managers were generally satisfied with the services autonomously offered by the robots, which successfully encouraged people to stop. For two out of three stores the robots successfully encouraged visitors to visit. The store managers were satisfied with the results and expressed a desire to use the robots again.

Previous research has made various efforts to produce human-like presence of autonomous social robots. However, such efforts often require costly equipment and complicated mechanisms. In this paper, we propose a new method that makes a user feel as if an autonomous robot is controlled by a remote operator, with virtually no cost. The basic idea is to manipulate people's knowledge about a robot by using priming technique. Through a series of experiments, we discovered that subjects tended to deduce the presence/absence of a remote operator based on their prior experience with that same remote operator. When they interacted with an autonomous robot after interacting with a teleoperated robot (i.e., a remote operator) whose appearance was identical as the autonomous robot, they tended to feel that they were still talking with the remote operator. The physically embodied talking behavior reminded the subjects of the remote operator's presence that was felt at the prior experience. Their deductions of the presence/absence of a remote operator were actually based on their “beliefs” that they had been interacting with a remote operator. Even if they had interacted with an autonomous robot under the guise of a remote operator, they tended to believe that they were interacting with a remote operator even when they subsequently interacted with an autonomous robot.

We are dealing with the problem of generating natural human-like motions during speech in android robots, which have human-like appearances. So far, automatic generation methods have been proposed for lip and head motions of tele-presence robots, based on the speech signal of the tele-operator. In the present study, we aim for extending the speech-driven motion generation methods for laughing speech, since laughter often occurs in natural dialogue interactions and may cause miscommunication if there is mismatch between audio and visual modalities. Based on analysis results of human behaviors during laughing speech, we proposed a motion generation method given the speech signal and the laughing speech intervals. Subjective experiments were conducted using our android robot by generating five different motion types, considering several modalities. Evaluation results show the effectiveness of controlling different parts of the face, head and upper body (eyelid narrowing, lip corner/cheek raising, eye blinking, head motion and upper body motion control).

Locomotion on the rough and variable ground surfaces is crucial for robots to complete various tasks. Recent advancement in numerical computation allow such locomotive robots to manipulate in real environments using a model-based control framework. This approach is successful if the precise model it is obtained. However, it is not always feasible to use the precise model because there are various factors related to both the robot and its surroundings. In this paper, we report the locomotion experiments with 2 types of legged robot that can change its behavior by switching connection patterns among hydraulic cylinders mounted on the robot's legs. An actuator network system (ANS) is used to switch mutual interconnection of cylinders. The switching connection allows the robots to realize not only adaptive locomotion to the environment but also operation of the traveling direction.

The development of an android with convincingly lifelike appearance and behavior has been a long-standing goal in robotics, and recent years have seen great progress in many of the technologies needed to create such androids. However, it is necessary to actually integrate these technologies into a robot system in order to assess the progress that has been made towards this goal and to identify important areas for future work. To this end, we are developing ERICA, an autonomous android system capable of conversational interaction, featuring advanced sensing and speech synthesis technologies, and arguably the most humanlike android built to date. Although the project is ongoing, initial development of the basic android platform has been completed. In this paper we present an overview of the requirements and design of the platform, describe the development process of an interactive application, report on ERICA's first autonomous public demonstration, and discuss the main technical challenges that remain to be addressed in order to create humanlike, autonomous androids.

We developed a method to automatically generate humanlike trunk motions (neck and waist motions) of a conversational android from its speech in real time. It is based on a spring-damper dynamical model to simulate a human's trunk movement involved in speech. Differing from the existing methods based on machine learning, our system can easily modulate the motions generated due to speech patterns since the parameters in the model correspond to muscle stiffness. The experimental result showed that the android motions generated by our model could be perceived as more natural and motivate participants to talk with the android more, compared with simple copying of human motions.

We conducted a pilot experiment to examine the potential of a communication robot for supporting verbal interaction among adolescents with autism spectrum disorders (ASD) and their caregivers. Three teenagers with ASD indirectly conversed with their teacher via a desktop humanoid robot for around 40 min on average per day. In this situation, they showed some positive tendencies such as non-echolalic responses and talk about problems related to human relationships; they had not shown such tendencies before when they directly talked with their teacher in their daily life. During the trials, they also showed changes in their attitudes toward their teachers and friends; this implies that individuals with ASD could use their experience of communication with a robot for communication with a human. A quantitative analysis of the utterances in the trials suggested that reduced utterances from a caregiver might positively influence individuals with ASD toward sharing more information.

The goal of this study is to develop a humanoid robot that can conduct a continuous conversation with people. Spoken dialogue systems have recently been rapidly developed, but the existing systems are insufficient for continuous use because they fail to inspire the user's motivation to talk with them. This is because a user is unable to feel that a robot possesses its own intentions, and thus the robot must acquire its own values to convey its sense of intentionality to users through its spoken communications. This paper focuses on a dialogue strategy aimed at building people's motivation under the assumption that the robot has a values-based dialogue system. People's motivation can be influenced by the intentionality as well as by the affinity of the robot. We hypothesized that there is a good disagreement/agreement ratio in a conversation to efficiently balance people's feelings of intentionality and affinity. The result of a psychological experiment using an android robot partially supported our hypothesis.

In 1, an error in one of the author names has been found subsequent to the publication. “Ishiguro Hiroshi” should be “Hiroshi Ishiguro.” The correct name is now presented above.

The present research examines how a robot's physical anthropomorphism interacts with perceived ability of robots to impact the level of realistic and identity threat that people perceive from robots and how it affects their support for robotics research. Experimental data revealed that participants perceived robots to be significantly more threatening to humans after watching a video of an android that could allegedly outperform humans on various physical and mental tasks relative to a humanoid robot that could do the same. However, when participants were not provided with information about a new generation of robots' ability relative to humans, then no significant differences were found in perceived threat following exposure to either the android or humanoid robots. Similarly, participants also expressed less support for robotics research after seeing an android relative to a humanoid robot outperform humans. However, when provided with no information about robots' ability relative to humans, then participants showed marginally decreased support for robotics research following exposure to the humanoid relative to the android robot. Taken together, these findings suggest that very humanlike robots can not only be perceived as a realistic threat to human jobs, safety, and resources, but can also be seen as a threat to human identity and uniqueness, especially if such robots also outperform humans. We also demonstrate the potential downside of such robots to the public's willingness to support and fund robotics research.

Recent studies in human-robot interaction (HRI) have investigated ways to harness the power of the crowd for the purpose of creating robot interaction logic through games and teleoperation interfaces. Sensor networks capable of observing human-human interactions in the real world provide a potentially valuable and scalable source of interaction data that can be used for designing robot behavior. To that end, we present here a fully automated method for reproducing observed real-world social interactions with a robot. The proposed method includes techniques for characterizing the speech and locomotion observed in training interactions, using clustering to identify typical behavior elements and identifying spatial formations using established HRI proxemics models. Behavior logic is learned based on discretized actions captured from the sensor data stream, using a naïve Bayesian classifier. Finally, we propose techniques for reproducing robot speech and locomotion behaviors in a robust way, despite the natural variation of human behaviors and the large amount of sensor noise present in speech recognition. We show our technique in use, training a robot to play the role of a shop clerk in a simple camera shop scenario, and we demonstrate through a comparison experiment that our techniques successfully enabled the generation of socially appropriate speech and locomotion behavior. Notably, the performance of our technique in terms of correct behavior selection was found to be higher than the success rate of speech recognition, indicating its robustness to sensor noise.

The future role of android robots is discussed. Survey results reveal that people are capable of viewing androids as companions and will accept androids operating in various roles within human society; however, people are uncomfortable with androids operating in roles in which they are able to persuade humans to do something, affect humans' minds, or supervise young children. To examine people's response to an actual android robot within human society, an android capable of operating as "lecturer" was implemented and an evaluative experiment conducted. It was found that androids can evoke feelings of attentiveness that are much stronger than those evoked by existing media. Furthermore, use of the lecturer android did not improve efficiency. It is possible that people expected the android to have human-like abilities because of its human-like appearance. However, regardless of the cause, it is clear from the experiment that it is necessary to program androids with more human-like functions if they are to fill useful roles within human society.

Human infants can consider humans as collaborative agents, but little is known about whether they also recognize robots as collaborative agents. This study investigated that how infants understand robot agents while they watched collaborative interactions between a human and a robot. We presented a novel visual habituation paradigm in which a human and a robot performed a collaborative activity to 13-month-old infants. Our findings suggested that 13-month-olds can appreciate robots as collaborative partners. We interpreted infants' expectancy violation responses to actions of the robot as facilitating their understanding about nonhuman agents as social partner.

Will people keep the secret of a socially compelling robot who shares, in confidence, a "personal" (robot) failing? Toward answering this question, 81 adults participated in a 20-minute interaction with (a) a humanoid robot (Robovie) interacting in a highly social way as a lab tour guide, and (b) with a human being interacting in the same highly social way. As a baseline comparison, participants also interacted with (c) a humanoid robot (Robovie) interacting in a more rudimentary social way. In each condition, the tour guide asks for the secret keeping behavior. Results showed that the majority of the participants (59%) kept the secret of the highly social robot, and did not tell the experimenter when asked directly, with the robot present. This percentage did not differ statistically from the percentage who kept the human's secret (67%). It did differ statistically when the robot engaged in the more rudimentary social interaction (11%). These results suggest that as humanoid robots become increasingly social in their interaction, that people will form increasingly intimate and trusting psychological relationships with them. Discussion focuses on design principles (how to engender psychological intimacy in human-robot interaction) and norms (whether it is even desirable to do so, and if so in what contexts).

When should service staff initiate interaction with a visitor? Neither simply-proactive (e.g. talk to everyone in a sight) nor passive (e.g. wait until being talked to) strategies are desired. This paper reports our modeling of polite approaching behavior. In a shopping mall, there are service staff members who politely approach visitors who need help. Our analysis revealed that staff members are sensitive to "intentions" of nearby visitors. That is, when a visitor intends to talk to a staff member and starts to approach, the staff member also walks a few steps toward the visitors in advance to being talked. Further, even when not being approached, staff members exhibit "availability" behavior in the case that a visitor's intention seems uncertain. We modeled these behaviors that are adaptive to pedestrians' intentions, occurred prior to initiation of conversation. The model was implemented into a robot and tested in a real shopping mall. The experiment confirmed that the proposed method is less intrusive to pedestrians, and that our robot successfully initiated interaction with pedestrians.

For social robots to be successful, they need to be accepted by humans. Human-robot interaction (HRI) researchers are aware of the need to develop the right kinds of robots with appropriate, natural ways for them to interact with humans. However, much of human perception and cognition occurs outside of conscious awareness, and how robotic agents engage these processes is currently unknown. Here, we explored automatic, reflexive social attention, which operates outside of conscious control within a fraction of a second to discover whether and how these processes generalize to agents with varying humanlikeness in their form and motion. Using a social variant of a well-established spatial attention paradigm, we tested whether robotic or human appearance and/or motion influenced an agent's ability to capture or direct implicit social attention. In each trial, either images or videos of agents looking to one side of space (a head turn) were presented to human observers. We measured reaction time to a peripheral target as an index of attentional capture and direction. We found that all agents, regardless of humanlike form or motion, were able to direct spatial attention in the cued direction. However, differences in the form of the agent affected attentional capture, i.e., how quickly the observers could disengage attention from the agent and respond to the target. This effect further interacted with whether the spatial cue (head turn) was presented through static images or videos. Overall whereas reflexive social attention operated in the same manner for human and robot social agents for spatial attentional cueing, robotic appearance, as well as whether the agent was static or moving significantly influenced unconscious attentional capture processes. Overall the studies reveal how unconscious social attentional processes operate when the agent is a human vs. a robot, add novel manipulations to the literature such as the role of visual motion, and provide a link between attention studies in HRI, and decades of research on unconscious social attention in experimental psychology and vision science.

In a tele-operated robot system, the reproduction of auditory scenes, conveying 3D spatial information of sound sources in the remote robot environment, is important for the transmission of remote presence to the tele-operator. We proposed a tele-presence system which is able to reproduce and manipulate the auditory scenes of a remote robot environment, based on the spatial information of human voices around the robot, matched with the operator's head orientation. In the robot side, voice sources are localized and separated by using multiple microphone arrays and human tracking technologies, while in the operator side, the operator's head movement is tracked and used to relocate the spatial positions of the separated sources. Interaction experiments with humans in the robot environment indicated that the proposed system had significantly higher accuracy rates for perceived direction of sounds, and higher subjective scores for sense of presence and listenability, compared to a baseline system using stereo binaural sounds obtained by two microphones located at the humanoid robot's ears. We also proposed three different user interfaces for augmented auditory scene control. Evaluation results indicated higher subjective scores for sense of presence and usability in two of the interfaces (control of voice amplitudes based on virtual robot positioning, and amplification of voices in the frontal direction).

Conventional hybrid actuators can achieve a better force/torque bandwidth than a single principle actuator without losing back-drivability. However, hybrid actuators occupy space at least equal to the sum of the volumes of two or more actuators and multiple transmissions consisting of timing belts, gears, links, tendons, or Bowden-cables. Designs in which space and structural elements of a hybrid actuator are integrated have not been sufficiently studied. This paper presents an original development work on the design of an integrated pneumatic-electromagnetic hybrid linear actuator (iPEHLA). In the integrated design, hybrid components of an air cylinder and a linear motor were arranged around a single shaft, and the pneumatic and electromagnetic actuators shared the same moving parts (i.e., piston and moving part) and internal cylindrical spaces (i.e., cylinder and stator). Consequently, space required was less than the sum of the volumes of the two actuators. To verify the feasibility of this novel integrated design, the iPEHLA was designed and prototyped by modifying parts of a commercially available pneumatic cylinder integrated with electromagnetic components. The performances of the iPEHLA under pneumatic actuation and hybrid actuation were compared with respect to a feedforward friction compensation. The experimental results demonstrated that the novel design maintained its hybrid properties.

We developed a tele-operated robot system where the head motions of the robot are controlled by combining those of the operator with the ones which are automatically generated from the operator's voice. The head motion generation is based on dialogue act functions which are estimated from linguistic and prosodic information extracted from the speech signal. The proposed system was evaluated through an experiment where participants interact with a tele-operated robot. Subjective scores indicated the effectiveness of the proposed head motion generation system, even under limitations in the dialogue act estimation.

Summary Thanks to the maturity of motion tracking technologies, it became easy and inexpensive to use avatars that reflect the user's facial and body motions in real time. There is a possibility that those real‐time avatars can serve as a substitute for videos in distant communication. We conducted two experiments to confirm positive effects of avatars on distant communication. In the first experiment, the subjects watched a conversation partner's video, avatar, photo, or nothing when speaking to the partner. Because the video and avatar delivered the partner's motion while the video and photo delivered the partner's appearance, we were able to observe the effects of the motion and appearance separately. As a result, we found that presenting the motion via a video or an avatar increased the degree of the smoothness of speaking to the partner, but the appearance did not significantly affect such an influence. In the second experiment, we compared the avatar and teleoperated robot to find how a physical embodiment influences the degree of the smoothness of speech. Teleoperated robots resemble avatars in transmitting the user's body motions without disclosing the user's appearance but differ in reflecting these motions onto a robot that has a physical embodiment. We predicted that the robot produces smoother speech because the physical embodiment might make the body motions recognizable. However, the robot decreased the degree of the smoothness of speech against our expectation. The results of these experiments showed that real‐time avatars that present only body motions are enough to smooth distant communication. Copyright © 2014 John Wiley & Sons, Ltd.

Activity recognition, involving a capability to recognize people's behavior and its underlying significance, will play a crucial role in facilitating the integration of interactive robotic artifacts into everyday human environments. In particular, social intelligence in recognizing affectionate behavior will offer value by allowing companion robots to bond meaningfully with interacting persons. The current article addresses the issue of designing an affectionate haptic interaction between a person and a companion robot by exploring how a small humanoid robot can behave to elicit affection while recognizing touches. We report on an experiment conducted to gain insight into how people perceive three fundamental interactive strategies in which a robot is either always highly affectionate, appropriately affectionate, or superficially unaffectionate (emphasizing positivity, contingency, and challenge, respectively). Results provide insight into the structure of affectionate interaction between humans and humanoid robots—underlining the importance of an interaction design expressing sincere liking, stability and variation—and suggest the usefulness of novel modalities such as warmth and cold.

This paper proposes a learning from demonstration system based on a motion feature, called phase transfer sequence. The system aims to synthesize the knowledge on humanoid whole body motions learned during teacher-supported interactions, and apply this knowledge during different physical interactions between a robot and its surroundings. The phase transfer sequence represents the temporal order of the changing points in multiple time sequences. It encodes the dynamical aspects of the sequences so as to absorb the gaps in timing and amplitude derived from interaction changes. The phase transfer sequence was evaluated in reinforcement learning of sitting-up and walking motions conducted by a real humanoid robot and compatible simulator. In both tasks, the robotic motions were less dependent on physical interactions when learned by the proposed feature than by conventional similarity measurements. Phase transfer sequence also enhanced the convergence speed of motion learning. Our proposed feature is original primarily because it absorbs the gaps caused by changes of the originally acquired physical interactions, thereby enhancing the learning speed in subsequent interactions.

It is expected that the social presence of a distant person can be well conveyed in telecommunication using a robot as a proxy of that person. This paper proposes a novel form of robotic telecommunication, which consists not only of a proxy robot but also an additional bystander robot that autonomously responds to the interlocutor in front of them. The second robot is expected to complement the responses of the proxy robot so that the interlocutor feels stronger social telepresence of the distant person. A psychological experiment using two small humanoid robots as proxy and bystander reveals a positive effect on conveying a stronger feeling of social telepresence of the distant person.

This study investigates whether a social robot which interacts with children via quiz-style conversations increases their understanding of science classes. We installed a social robot in an elementary school science classroom where children could freely interact with it during their breaks. The robot asks children questions related to their latest science classes to support their understanding of the classes. During interaction, the robot says children's name and distribute its gaze among the group of children by using a face recognition system and a human tracking system. Still, speech recognition is difficult in the noisy elementary school environment; therefore the operator takes over this function during interactions. In this study our result did not show significant effects of the robot for helping children's understanding, but we found several interesting interaction scenes which shows that robot had a certain effect on specific children.

We propose a huggable communication device called Hugvie that encourages children to concentrate on listening by reducing their stress and strengthening the feeling that the storyteller is close. We observed a group of preschool children who listened to a story and conclude that Hugvie has the potential to facilitate attention on stories. This indicates its usefulness to relieve the educational problem where children show disobedient or restless behavior during class. We discuss Hugvie's effect on learning and memory and potential applications to special-needs children.

This paper shows how humor can be used as an interaction pattern to help establish sociality in human-robot interaction. Drawing illustratively from our published research on people interacting with ATR's humanoid robot Robovie, we highlight four forms of humor that we successfully implemented: wit and the ice-breaker, corny joke, subtle humor, and dry humor and self-deprecation.

We offer three illustrative examples from one of our recent studies in HRI to suggest that it's possible for people to engage in mutual deliberation with a social robot. Each example illustrates discourse and argument, but ends through different means: Accepting the Robot as Arbitrator; Delegitimizing the Robot; and Agreeing to Disagree.

Walking side by side is a common situation when we go from one place to another with another person while talking. Our previous study reported a basic mechanism for side-by-side walking, but in the previous model it was crucial that each agent knew where he or she was going, i.e. the route to the destination. However, we have recognized the need to model the situation where one of the agents does not know the destination. The extended model presented in this work has two states: leader-follower state and collaborative state. Depending on whether the follower agent has obtained a reliable estimate of the route to follow, the walking agents transition between the two states. The model is calibrated with trajectories acquired from pairs of people walking side by side, and then it is tested in a human-robot interaction scenario. The results demonstrate that the new extended model achieves better side-by-side performance than a standard method without knowledge of the subgoal.

In this paper, we discussed an autonomous android robot that is recognized as a social entity by observers in the real world. We conducted field experiments to investigate what kind of function the android needs in order to be recognized as a social and humanlike entity by visitors. In the field experiment, the android made conversation with multiple visitors by using several touch displays. The result showed that the visitors evaluate the android as humanlike even though this interaction differs from normal human-human interaction. Moreover, the experiment suggested that the android has a social influence - advertisement effect on visitors.

This research builds on the UbiComp vision of systems that do not do things for people but engage people in their computational environment so that people can do things for themselves better. In this investigation, we sought to make good on a proof-of-concept where people interact with a social robot whereby the robot helps people to be more humanly creative. Twenty seven participants interacted with ATR's humanoid robot Robovie (through a WoZ interface) in a creativity task. Results supported our proof of concept insofar as 100% of the participants generated creative ideas, and 63% incorporated the robot's ideas into their own ideas for their creative output. Of the participants who had the highest creativity scores, 83% incorporated the robot's ideas into their own. Discussion focuses on next steps toward building the Natural Language Processing system, and integrating the system into a more extensive networked UbiComp environment.

To achieve a realistic task by a recent complicated robot, a practical motion planning method is important. Especially in this decade, sampling-based motion planning methods have become popular thanks to recent high performance computers. In sampling-based motion planning, a graph that covers the state space is constructed based on reachability between node pairs, and the motion is planned using the graph. However, it requires an explicit model of a controlled target. In this research, we propose a motion planning method in which a system model is estimated by using Gaussian process regression. We apply our method to the control of an actual robot. Experimental results show that the control of the robot can be achieved by the proposed motion planning method.

Developing interactive behaviors for social robots presents a number of challenges. It is difficult to interpret the meaning of the details of people's behavior, particularly non-verbal behavior like body positioning, but yet a social robot needs to be contingent to such subtle behaviors. It needs to generate utterances and non-verbal behavior with good timing and coordination. The rules for such behavior are often based on implicit knowledge and thus difficult for a designer to describe or program explicitly. We propose to teach such behaviors to a robot with a learning-by-demonstration approach, using recorded human-human interaction data to identify both the behaviors the robot should perform and the social cues it should respond to. In this study, we present a fully unsupervised approach that uses abstraction and clustering to identify behavior elements and joint interaction states, which are used in a variable-length Markov model predictor to generate socially-appropriate behavior commands for a robot. The proposed technique provides encouraging results despite high amounts of sensor noise, especially in speech recognition. We demonstrate our system with a robot in a shopping scenario.

Telenoid, a teleoperated android, is a medium through which teleoperators can transmit both verbal and nonverbal information to interlocutors. Telenoid promotes conversation with its interlocutors, especially elderly people. Nevertheless, since teleoperators admit that they have difficulty feeling that they are actually teleoperating their robots, they cannot use them effectively to transmit nonverbal information; such nonverbal information is one of Telenoid's biggest merits. In this paper, we propose a training program for teleoperators so that they can understand Telenoid's operation and ways to transmit nonverbal information through it. We investigate the effects of the training on teleoperation and communication, and identify three results. Firstly, our training improves Telenoid's head movements leading to clearer transmission of nonverbal information. Secondly, the effects of the training differ between genders. Females communicated with their interlocutors more smoothly than males after training. Males communicated with their interlocutors more smoothly after simple talking practice. Thirdly, we found that correlations exist not only among freely controlling the robot, regarding the robot as oneself, and tele-presence in the interlocutor's room, but also among freely controlling the robot, tele-presence in the interlocutor's room, and feeling involved when talking with the interlocutor. On the other hand, there are no correlations between the feelings about Telenoid's operation and the head movements.

Due to human sensitivity for humanlike objects, designing human-like behavior for a robot that interacts with humans remains a central issue in the HRI field. This issue is formidable because human-likeness is an important factor for better interaction since it can easily convey negative impressions if the design is not perfect, as in the uncanny valley phenomena. This paper addresses this issue with a novel approach that utilizes implicit know-how for performing on stage and is dedicated to represent human beings. Contemporary colloquial theatre theory (CCTT), which is a method of directing plays, is appropriate for this purpose since its reality-oriented instructions are directly applicable for improving robot behavior. In this paper, we report case studies on performing a play involving both humans and a robot. The play in our study was evaluated in public performances in Japan. We report detailed analysis of comparable short plays on human-robot interaction or human-human-robot interaction. Our analysis implies that utterance and motion timings should be tuned depending on the situation. Future work will use a motion capture system to get more precise data and more useful knowledge.

Small robots are being designed to recognize behaviors through playful interaction. Prior work used data from impoverished sensing devices such as inertial sensors to analyze gestures and attitude in playful interaction through time series analysis. However, the prior work did not focus on individual differences required for person identification. This research hypothesizes that person identification can be achieved by determining individual differences in playful interaction by using inertial sensor data. We propose a method that iteratively narrows down the candidates during interaction to achieve accurate person identification. This method calculates the features using a time series of the inertial sensor data. These features identify a candidate who is playfully interacting with the robot using a decision tree classifier that includes combinations of the current candidates. The system stores the results as a dataset for voting, and the voting results are used to reduce the candidates until the number of candidates is winnowed to one. Evaluation results show that our proposed method identifies persons through playful interactions with 99.1% accuracy.

This paper presents ROSlink, an open source project that aims at easing the integration of legacy systems with ROS (Robot Operating System). Its design principles provide a set of unique features that make it appealing for the interconnection of ROS with systems where ROS itself cannot be installed. First, ROSlink requires very limited changes to the legacy system. The project is self contained, bringing in no dependencies, which may be difficult to satisfy in a legacy system. Furthermore, with ROSlink any data type already in use in the legacy system can be employed for the communication of topics and service requests and responses. ROSlink allows run-time rerouting of the communication between the legacy system and ROS. Moreover, it empowers the legacy code with the ROS name remapping system, without enforcing any constraint on the command line parameters of legacy programs. Finally, by simply using a set of API that closely follow the ROS programming interface, ROSlink simplifies any successive porting of the code to a real ROS system. In this paper, the main design choices of ROSlink are discussed. A list of practical applications and tests where ROSlink was employed, as well as a short discussion on the project's future directions are then given.

We have been studying small sized direct-drive electromagnetic linear actuators and their control method. These actuators were able to emulate the viscoelastic characteristics of the human muscle by PD position control to respond to external forces flexibly. However, their thrust force was not enough in order to apply to legs and shoulders of robots. In this paper, we present a novel small cylindrical linear vernier motor for artificial muscle of humanoid robots. First, the structure and operating principle of the proposed linear vernier motor are described. Then, the thrust and detent force of the motor are shown by 2D finite element analysis and measurement on a prototype. As the result, the validity of the simulation is discussed. Finally, the effect of the position controller with the disturbance observer on reduction of detent force and deviation between target and actual mover position is clarified.

The development of humanlike service robots which interact socially raises a new question: How can we make good interaction content for such robots? Domain experts specializing in the target service have the knowledge for making such content. Yet, while they can easily engage in good face-to-face interactions, we found it difficult for them to prepare conversational content for a robot in written form. Instead, we propose involving experts as teleoperators in an iterative development process in which the expert develops content, teleoperates a robot using that content, and then revises the content based on that interaction. We propose a software system and design guidelines to enable such an iterative design process. To validate these solutions, we conducted a comparison experiment in the field, with elderly volunteer guides teleoperating a robot at a tourist information center in Nara, Japan. The results showed that our system and guidelines enabled domain experts with no robotics background to create better interaction content and conduct better interactions than domain experts without our system.

We introduce a communication medium that encourages users to fall in love with their counterparts. Hugvie, the huggable tele-presence medium, enables users to feel like hugging their counterparts while chatting. In this paper, we report that when a participant talks to his communication partner during their first encounter while hugging Hugvie, he mistakenly feels as if they are establishing a good relationship and that he is being loved rather than just being liked.

Body Ownership Transfer (BOT) is an illusion that we feel external objects as parts of our own body that occurs when teleoperating android robots. In past studies, we have been investigating under what conditions this illusion occurs. However, past studies were only conducted with simple operation tasks such as by only moving the robot's hand. Does this illusion occur under much complex tasks such as having a conversation? What kind of conversation setting is required to invoke this illusion? In this paper, we examined how factors in social interaction affects occurrence of BOT. Participants had conversation using the teleoperated robot under different situations and teleoperation settings. The results revealed that BOT does occur by the act of having a conversation, and that conversation partner's presence and appropriate responses are necessary for enhancement of BOT.

Robot avatar systems give the feeling we share a space with people who are actually at a distant location. Since our cognitive system specializes in recognizing a human, avatars of the distant people can make us strongly feel that we share space with them, provided that their appearance has been designed to sufficiently resemble humans. In this paper, we investigate the minimal requirements of robot avatars for distant people to feel their presence. Toward this aim, we give an overview of the chronological development of Dogu̅, which are human figurines made in ancient Japan. This survey of the Dogu̅ shows that the torso, not the face, was considered the primary element for representing a human. It also suggests that some body parts can be represented in a simple form. Following the development of Dogu̅, we use a conversation task to examine what kind of body representation is necessary to feel a distant person's presence. The experimental results show that the forms for the torso and head are required to enhance this feeling, while other body parts have less impact. Finally, we summarize design considerations for communication avatars.

When robots serve in urban areas such as shopping malls, they will often be required to approach people in order to initiate service. This paper presents a technique for human-robot interaction that enables a robot to approach people who are passing through an environment. For successful approach, our proposed planner first searches for a target person at public distance zones anticipating his/her future position and behavior. It chooses a person who does not seem busy and can be reached from a frontal direction. Once the robot successfully approaches the person within the social distance zone, it identifies the person's reaction and provides a timely response by coordinating its body orientation. The system was tested in a shopping mall and compared with a simple approaching method. The result demonstrates a significant improvement in approaching performance; the simple method was only 35.1% successful, whereas the proposed technique showed a success rate of 55.9%.

This study provides an in-depth analysis and practical solution to the problem of designing and implementing a human-robot team for simple conversational interactions. Models for operation timing, customer satisfaction and customer-robot interaction are presented, based on which a simulation tool is developed to estimate fan-out and robot team performance. Techniques for managing interaction flow and operator task assignment are introduced. In simulation, the effectiveness of different techniques and factors related to team performance are studied. A case study on deploying multiple robots in a shopping mall is then presented to demonstrate the usefulness of our study in helping the design and implementation of social robots in real-world settings.

Robots will increasingly take on roles in our social lives where they can cause humans harm. When robots do so, will people hold robots morally accountable? To investigate this question, 40 undergraduate students individually engaged in a 15-minute interaction with ATR's humanoid robot, Robovie. The interaction culminated in a situation where Robovie incorrectly assessed the participant's performance in a game, and prevented the participant from winning a $20 prize. Each participant was then interviewed in a 50-minute session. Results showed that all of the participants engaged socially with Robovie, and many of them conceptualized Robovie as having mental/emotional and social attributes. Sixty-five percent of the participants attributed some level of moral accountability to Robovie. Statistically, participants held Robovie less accountable than they would a human, but more accountable than they would a vending machine. Results are discussed in terms of the New Ontological Category Hypothesis and robotic warfare.

Head motion occurs naturally and in synchrony with speech during human dialogue communication, and may carry paralinguistic information, such as intentions, attitudes and emotions. Therefore, natural-looking head motion by a robot is important for smooth human-robot interaction. Based on rules inferred from analyses of the relationship between head motion and dialogue acts, this paper proposes a model for generating head tilting and nodding, and evaluates the model using three types of humanoid robot (a very human-like android, "Geminoid F", a typical humanoid robot with less facial degrees of freedom, "Robovie R2", and a robot with a 3-axis rotatable neck and movable lips, "Telenoid R2"). Analysis of subjective scores shows that the proposed model including head tilting and nodding can generate head motion with increased naturalness compared to nodding only or directly mapping people's original motions without gaze information. We also find that an upwards motion of a robot's face can be used by robots which do not have a mouth in order to provide the appearance that utterance is taking place. Finally, we conduct an experiment in which participants act as visitors to an information desk attended by robots. As a consequence, we verify that our generation model performs equally to directly mapping people's original motions with gaze information in terms of perceived naturalness.

This year's human-robot interaction (HRI) conference focuses on "robots in the loop" and how robots are capable of enhancing the experiences of human users in everyday life and work. Telepresence robots allow operators the ability to participate in remote locations through their mobility and live bidirectional audio and video feeds. Using robotic telepresence, students with chronic illnesses are attending their regular classes, physicians are conducting virtual "home visits" for recovering patients, and remote teammates are having conversations beyond the office conference room.

The compliance of muscles with external forces and the structural stability given by biarticular muscles are important features of animals to realize dynamic whole body motions such as running and hopping in various environments. For this reason, we have been studying an electromagnetic linear actuator. This actuator can emulate the behavior of a human muscle such as the spring-damper characteristics by quick control of the output force (i.e. impedance control) and it is expected to be used as an artificial muscle. In this paper, we develop a monopedal robot possessing bi- and mono-articular muscles implemented by linear actuators. Thanks to the biarticular muscle, the bouncing direction of the robot can be controlled by changing the stiffness ellipse at the endpoint (i.e. foot) of the robot. We confirm that the bouncing direction of the robot can be controlled and hopping can be achieved by changing the stiffness ellipse.

In order to facilitate integration into domestic and public environments, companion robots can seek to communicate in a familiar, “socially intelligent” manner, recognizing typical behaviors which people direct toward them. One important type of behavior to recognize is the displaying and seeking of affection, which is fundamentally associated with the modality of touch. This paper identifies how people communicate affection through touching a humanoid robot appearance, and reports on the development of a recognition system exploring the modalities of touch and vision. Results of evaluation indicate the proposed system can recognize people's affectionate behavior in the designated context.

Generating natural motion in robots is important for improving human-robot interaction. We developed a tele-operation system where the lip motion of a remote humanoid robot is automatically controlled from the operator's voice. In the present work, we introduce an improved version of our proposed speech-driven lip motion generation method, where lip height and width degrees are estimated based on vowel formant information. The method requires the calibration of only one parameter for speaker normalization. Lip height control is evaluated in two types of humanoid robots (Telenoid-R2 and Geminoid-F). Subjective evaluation indicated that the proposed audio-based method can generate lip motion with naturalness superior to vision-based and motion capture-based approaches. Partial lip width control was shown to improve lip motion naturalness in Geminoid-F, which also has an actuator for stretching the lip corners. Issues regarding online real-time processing are also discussed.

This research constitutes an initial investigation into key issues which arise in designing a flying humanoid robot (FHR), with a focus on human-robot interaction (HRI). The humanoid form offers an interface for natural communication; flight offers excellent mobility. Combining both will yield companion robots capable of approaching, accompanying, and communicating naturally with humans in difficult environments. Problematic is how such a robot should best fly around humans, and what effect a robot's flight will have on a person in terms of non-verbal communicative cues. To answer these questions, we propose an extension to existing proxemics theory (“z-proxemics”) and predict how typical humanoid flight motions will be perceived (“z-kinesics”). Data obtained from participants watching animated sequences are analyzed to check our predictions. The paper also reports on the building of a flying humanoid robot, which we will use in interactions.

Recent research has focused on such physical communication media as teleoperated robots, which provide a feeling of being with people in remote places. Recent invented media resemble cute animals or imaginary creatures that quickly attract attention. However, such appearances could distort tele-communications because they are different from human beings. This paper studies the effect on the speaker's personality that is transmitted through physical media by regarding appearances as a function that transmits the speaker's information. Although communication media's capability to transmit information reportedly influences conversations in many aspects, the effect of appearances remains unclear. To reveal the effect of appearance, we compared three appearances of communication media: stuffed-bear teleoperated robot, human-like teleoperated robot, and video chat. Our results show that communication media whose appearance greatly differs from that of the speaker distorts the personality perceived by interlocutors. This paper suggests that the design of the appearance of physical communication media needs to be carefully selected.

In this study, we examine the factors with which robots are recognized as social beings. Participants joined sessions of the Ultimatum Game, a procedure commonly used for examining attitudes toward others in the fields of economics and social psychology. Several agents differing in their appearances are tested with speech stimuli that are expected to induce a mentalizing effect toward the agents. As a result, we found that while appearance itself did not show significant difference in the attitudes, the mentalizing stimuli affected the attitudes in different ways depending on robots' appearances. This results showed that such elements as simple conversation with the agents and their appearance are important factors so that robots are treated more humanlike and as social beings.

In recent years, research has increased on very human-like androids, generally investigating the following: (1) how people treat such very human-like androids and (2) whether it is possible to replace such existing communication media as telephones or TV conference systems with androids as a communication medium. We found that androids have advantages over humans in specific contexts. For example, in a collaboration theatrical project between artists and androids, audiences were impressed by the androids that read poetry. We, therefore, experimentally compared androids and humans in a poetry-reciting context by conducting an experiment to illustrate the influence of an android who recited poetry. Participants listened to poetry that was read by three poetry-reciting agents: the android, the human model on which the android was based, and a box. The experiment results showed that the enjoyment of the poetry gained the highest score under the android condition, indicating that the android has an advantage for communicating the meaning of poetry.

We previously investigated body ownership transfer to a teleoperated android body caused by motion synchronization between the robot and its operator. Although visual feedback is the only information provided from the robot, due to body ownership transfer, some operators feel as if they were touched when the robot's body was touched. This illusion can help operators transfer their presence to the robotic body during teleoperation. By enhancing this phenomenon, we can improve our communication interface and the quality of the interaction between operator and interlocutor. In this paper, we examined how the change in the operator's perspective affects the body ownership transfer during teleoperation. Based on past studies on the rubber hand illusion, we hypothesized that the perspective change will suppress the body owner transfer. Our results, however, showed that in any perspective condition, the participants felt the body ownership transfer. This shows that its generation process may differ in the body ownership transfer for teleoperated androids and the rubber hand illusion.

What are the characteristics to make something appear as asocial entity? Is sociality limited to human beings? The following article will deal with the borders of sociality and the characterizations for animating a physical object (here: android robot) to a living being. The transition is attributed during interactive encounters. We will introduce implications of an ethnomethodological analysis to show characteristics of transitions in social attribution towards an android robot, which is treated and perceived gradually shifting from an object to a social entity. These characteristics should a) fill the gap in current anthropological and sociological research, dealing with the limits and characteristics of social entities, and b) contribute to the discussion of specifics in human-android interaction compared to human-human interaction.

In this paper, we propose the “Doppel teleoperation system,” which isolates several physical traits from a speaker, to investigate how personal information is conveyed to others during conversation. An underlying problem on designing personality in social robots is that it remains unclear how humans judge the personalities of conversation partners. With the Doppel system, for each of the communication channels to be transferred, one can choose it in its original form or in the one generated by the system. For example, voice and body motions can be replaced by the Doppel system while preserving the speech content. This allows us to analyze the individual effects of the physical traits of the speaker and the content in the speaker's speech on the identification of personality. This selectivity of personal traits provides a useful approach to investigate which information conveys our personality through conversation. To show the potential of our system, we experimentally tested how much the conversation content conveys the personality of speakers to interlocutors without any of their physical traits. Preliminary results show that although interlocutors have difficulty identifying speakers only using conversational contents, they can recognize their acquaintances when their acquaintances are the speakers. We point out some potential physical traits to convey personality.

Teleoperated androids, which are robots with humanlike appearances, are being produced as new media of human relationships. We explored the potential of humanoid robots and how they affect people in the real world when they are employed to express a telecommunication presence and a sense of ‘being there’. We introduced Telenoid, a teleoperated android, to a residential care facility to see how the elderly with dementia respond to it. Our exploratory study focused on the social aspects that might facilitate communication between the elderly and Telenoid's operator. Telenoid elicited positive images and interactive reactions from the elderly with mild dementia, even from those with severe cognitive impairment. They showed strong attachment to its child-like huggable design and became willing to converse with it. Our result suggests that an affectionate bond may be formed between the elderly and the android to provide the operator with easy communication to elicit responses from senior citizens.

During conversations, speakers employ a number of verbal and nonverbal mechanisms to establish who participates in the conversation, when, and in what capacity. Gaze cues and mechanisms are particularly instrumental in establishing the participant roles of interlocutors, managing speaker turns, and signaling discourse structure. If humanlike robots are to have fluent conversations with people, they will need to use these gaze mechanisms effectively. The current work investigates people's use of key conversational gaze mechanisms, how they might be designed for and implemented in humanlike robots, and whether these signals effectively shape human-robot conversations. We focus particularly on whether humanlike gaze mechanisms might help robots signal different participant roles, manage turn-exchanges, and shape how interlocutors perceive the robot and the conversation. The evaluation of these mechanisms involved 36 trials of three-party human-robot conversations. In these trials, the robot used gaze mechanisms to signal to its conversational partners their roles either of two addressees, an addressee and a bystander, or an addressee and a nonparticipant. Results showed that participants conformed to these intended roles 97% of the time. Their conversational roles affected their rapport with the robot, feelings of groupness with their conversational partners, and attention to the task.

Teleoperation of multiple robots by a single operator has been studied extensively for applications such as search and navigation; however, this concept has never been applied to the field of social, conversational robots. In this paper, we explore the unique challenges posed by the remote operation of multiple social robots, where an operator must perform auditory multitasking to assist multiple interactions at once. It describes the general system requirements in four areas: social human-robot interaction (HRI) design, autonomy design, multirobot coordination, and teleoperation interface design. Based on this design framework, we have developed a system in which a single operator can simultaneously control four robots in conversational interactions with users. Key elements of our implementation include a control architecture enabling the scripting of conditional behavior flows for social interaction, a graphical interface enabling an operator to control one robot at a time while monitoring several others in the background, and a technique called “proactive timing control,” which is an automated method for smoothly interleaving the demands of multiple robots for the operator's attention. We also present metrics for describing and predicting robot performance, and we show experimental results demonstrating the effectiveness of our system through simulations and a laboratory experiment based on real-world interactions.

Awareness of time is particularly important for teleoperation of conversational robots, both for controlling the robot and for estimating interaction success, because people have a low tolerance for long pauses in conversation. Findings have shown that people engaged in high-workload tasks tend to underestimate the passage of time. This study confirms that this problem exists for operators controlling a conversational robot, and it investigates mechanisms for improving temporal awareness and task performance while minimizing workload. In a laboratory experiment, two approaches to helping an operator perform various information input tasks were compared: first, assisting temporal awareness by using a clock display, and second, using autonomy to assist one of the operator's time-dependent tasks. Results revealed that assisting the task itself, even without the clock, improved not only task performance but also the operator's temporal awareness. However, results regarding the effect of the clock were ambiguous: it increased workload in general and did not help temporal awareness overall, but it did improve temporal awareness for text entry tasks in particular. As text entry is an important task for teleoperation of social robots, we further investigated the problem of improving temporal awareness during text entry tasks. As the first experiment suggested the effectiveness of a clock, we further validated that the clock is specifically useful to improve temporal awareness. These results showed that the clock did not increase workload for text entry tasks; however, for touch-typing operators, the results suggested that showing a clock after the end of an interaction, rather than continuously throughout the task, could lower the operator's perceived workload.

This paper deals with a multiagent path-planning problem where several robots track humans to obtain detailed information on human behaviors and characteristics. For this, agents' paths are planned on the basis of the similarity between the predicted positions of humans and the agents' field of view. The long-horizon path planned on the basis of an accurate long-horizon prediction improves the tracking performance. However, it requires heavy computation and is less useful if the prediction is inaccurate. Since the accuracy of the prediction depends on the situation, the prediction term is determined by the similarity between the current and previous predictions. The results of computer simulation showed that our path-planning method works well for trajectories of humans in a dynamic environment by changing the horizon length of the path planning.

This paper shows that the augmentation of a remote person's positional movement enhances social telepresence. There are three possible ways of representing a remote person's movement toward the user in visual communication: a) the remote person's movement toward the remote camera, b) the remote camera's zooming in to enlarge the remote person's picture, and c) a forward movement of the display that is displaying the remote person. We conducted an experiment to see the relationship among these three ways and the effects of a remote camera's zooming and a display's movement on social telepresence. In the experiment, we observed that the remote person's movement lowered the reality of conversations, and the remote camera's zooming lowered the visual quality. However, social telepresence was enhanced when both the person's movement and the camera's zooming occurred simultaneously. We also observed that a 6-centimeter movement of the display enhanced social telepresence, whether the remote person moved or not.

Assistive robots can be perceived in two main ways: tools or partners. In past research, assistive robots that offer physical assistance for the elderly are often designed in the context of a tool metaphor. This paper investigates the effect of two design considerations for assistive robots in a partner metaphor: conversation and robot-type. The former factor is concerned with whether robots should converse with people even if the conversation is not germane for completing the task. The latter factor is concerned with whether people prefer a communication/function oriented design for assistive robots. To test these design considerations, we selected a shopping assistance situation where a robot carries a shopping basket for elderly people, which is one typical scenario used for assistive robots. A field experiment was conducted in a real supermarket in Japan where 24 elderly participants shopped with robots. The experimental results revealed that they prefer a conversational humanoid as a shopping assistant partner.

This paper discusses converging evidence to support the hypothesis that personified robots and other embodied personified computational systems may represent a new ontological category, where ontology refers to basic categories of being, and ways of distinguishing them.

On considering symbiosis of humans and robots, its benefits and risks should be taken into account for persons in weaker positions of the society, in particular, children. On the other hand, several robotics applications have been developed, including education and welfare for children. In this stage, it is important that more researchers from interdisciplinary research fields, including robotics, computer science, psychology, sociology, and pedagogy, share an opportunity to discuss about the potential of "robots with children". This half-day workshop aims at providing with the forum where researchers from these interdisciplinary fields discuss about how symbiosis of robots and children should and can be realized, from the perspectives of engineering, psychology, education, and welfare.

This paper presents the effect of a robot's active touch for improving people's motivation. For services in the education and healthcare fields, a robot might be useful for improving the motivation of performing such repetitive and monotonous tasks as exercising or taking medicine. Previous research demonstrated with a robot the effect of user touch on improving its impressions, but they did not clarify whether a robot's touch, especially an active touch, has enough influence on people's motive. We implemented an active touch behavior and experimentally investigated its effect on motivation. In the experiment, a robot requested participants to perform a monotonous task with a robot's active touch, a passive touch, or no touch. The result of experiment showed that an active touch by a robot increased the number of working actions and the amount of working time for the task. This suggests that a robot's active touch can support people to improve their motivation. We believe that a robot's active touch behavior is useful for such robot's services as education and healthcare.

Since most robots are not easily displayable in real-life scenarios, only a few studies investigate users' behavior towards humanoids or androids in a natural environment. We present an observational field study and data on unscripted interactions between humans and the android robot "Geminoid HI-1". First results show that almost half of the subjects mistook Geminoid HI-1 for a human. Even those who recognized the android as a robot rather showed interest than negative emotions and explored the robots capabilities.

This study explores the nature of the multi-robot control problem for social robots. It begins by modeling the overall structure of a human-robot team for social interactions, and implements it for specific applications to dialog-based interactions. Operator activity during control of a social robot is studied. Customer satisfaction is proposed as an important metric for evaluating the performance of a human-robot team for social interactions with customers. Based on the modeling, fan-out of a social robot team can be calculated, and the performance of the team is estimated by simulation. A field trial was conducted in a shopping mall to demonstrate a successful deployment of social robots for a real-world application with ensured performance prior to installation using our modeling and simulation approach.

This paper describes a self-protective whole-body control method for humanoid robots. A set of postural reactions are used to create whole-body movements. A set of reactions is merged to cope with a general falling down direction, while allowing the upper limbs to contact safely with obstacles. The collision detection is achieved by force sensing. We verified that our method generates the self-protective motion in real time, and reduced the impact energy in multiple situations by simulator. We also verified that our systems works adequately in real-robot.

Pitch extraction is important for communication robots, since pitch may carry information about intention, attitude or emotion expression from the user's speech. However, current pitch extraction methods are not robust enough in real noisy environments. In the present work, we propose pitch extraction methods by combining microphone array and auditory scene analysis technologies, and evaluate pitch extraction of multiple speakers in real noisy environments. Evaluation results show that the proposed ML-PSACF (maximum likelihood adaptive beamformer with peak-pruned summary autocorrelation function) contributes to reduce the effects of interference and noise, leading to improvements of 23%±5% on pitch estimation rates, in comparison to the baseline of not using array processing.

Touch is an important means for communication among humans. Sport instructors or dance teachers often use touch to adjust students' postures in a very intuitive way. Using tactile instructions appears thus to be a very appealing modality for developing humanoid robot motions as well. Spontaneous interpretation of tactile instructions given by users reveals itself to be a complex task for artificial systems. This paper describes a proof of concept system for robot motion creation based on tactile interaction. The system is interesting for two reasons. Firstly, it shows the feasibility of using tactile instructions for motion development. Secondly, it can be used as a tool for studying the way humans intuitively use touch to communicate. This, in turn, will allow the development of better algorithms for predicting the meaning of tactile instructions. Results of a pilot experiment are discussed, and a first set of features of tactile communication, yielded by the analysis of the data collected, is identified.

This paper deals with a multi-agent path planning problem for tracking humans in order to obtain detail information such like human behavior and characteristics. To achieve this, paths of agents are planned based on similarity between the predicted positions of humans and agents' field of views, and the path length in the path planning is determined according to the consistency between the current prediction and the previous prediction of the future human positions. We conducted computer simulations and results showed that our path planning method works well for trajectories of human in a real environment.

At the 2008 ACM/IEEE Conference on Human-Robot Interaction, a provocative panel was held to discuss the complicated ethical issues that abound in the field of human-robot interaction. The panel members and the audience participation made it clear that the HRI community desires - indeed, is in need of - an ongoing discussion on the nature of social responsibility in the field of human-robot interaction. At the 2010 Conference, we will hold a panel on the issues of social responsibility in HRI, focusing on the unique features of robotic interaction that call for responsible action (e.g., value-specific domains such as autonomy, accountability, trust, and/or human dignity; and application areas such as military applications, domestic care, entertainment, and/or communication). As a young and rapidly growing field, we have a responsibility to conduct our research in such a way that it leads to human-robot interaction outcomes that promote rather than hinder the flourishing of humans across society. What does social responsibility within the HRI field look like, and how do we conduct our work while adhering to such an obligation? The panelists will be asked to address this and related questions as a means of continuing an ongoing conversation on social responsibility in human-robot interaction.

Head motion naturally occurs in synchrony with speech and may carry paralinguistic information, such as intention, attitude and emotion, in dialogue communication. With the aim of verifying the relationship between head motion and the dialogue acts carried by speech, analyses were conducted on motion-captured data for several speakers during natural dialogues. The analysis results first confirmed the trends of our previous work, showing that regardless of the speaker, nods frequently occur during speech utterances, not only for expressing dialogue acts such as agreement and affirmation, but also appearing at the last syllable of the phrase, in strong phrase boundaries, especially when the speaker is talking confidently, or expressing interest in the interlocutor's talk. Inter-speaker variability indicated that the frequency of head motion may vary according to the speaker's age or status, while intra-speaker variability indicated that the frequency of head motion also differs depending on the inter-personal relationship with the interlocutor. A simple model for generating nods based on rules inferred from the analysis results was proposed and evaluated in two types of humanoid robots. Subjective scores showed that the proposed model could generate head motions with naturalness comparable to the original motions.

On the current situation where robots having functions of communication with humans begin to appear in daily-life fields, it should be considered how symbiosis of humans and robots can be achieved. Many existing studies have focused on how robots can learn from and adapt for humans. This full-day workshop focuses not only on this classical theme but also on how humans can learn in and adapt for environments where robots are acting. In particular, human learning from and adaptation for robots should be covered by interdisciplinary research fields including robotics, computer science, psychology, sociology, and pedagogy.

In recent work, "interaction patterns" have been proposed as a means to characterize essential features of human-robot interaction. A problem arises, however, in knowing whether the interaction patterns generated are valid. The same problem arises when researchers in HRI propose other broad conceptualizations that seek to structure social interaction. In this paper, we address this general problem by distinguishing three ways of establishing the validity of interaction patterns. The first form of validity seeks to establish whether the conclusions about interaction patterns are warranted from the data. The second seeks to establish whether the interaction patterns account for the data. And the third seeks to provide sound reasons for the labels of the patterns themselves. Often these three forms of validity are confused in discussions about conceptual categories in HRI.

This conceptual paper broaches possibilities and limits of establishing psychological intimacy in HRI.

Tour guidance is a common task of social robots. Such a robot must be able to encourage the participation of people who are not directly interacting with it. We are particularly interested in encouraging people to overhear its interaction with others, since it has often been observed that even people who hesitate to interact with a robot are willing to observe its activity. To encourage such participation as bystanders, we developed a robot that walks backwards based on observations of human tour guides. Our developed system uses a robust human tracking system that enables a robot to guide people by walking forward/backward and allows us to scrutinize people's behavior after the experiment. We conducted a field experiment to compare the ratios of overhearing in "walking forward" and "walking backward." The experimental results revealed that in fact people do more often overhear the robot's conversation in the "walking backward" condition.

To provide personal and location-dependent services in public spaces such as shopping malls, it is important to be able to estimate the positions and identities of people in the environment. Sensors in the environment reliably detect their current positions, but it is difficult to identify people using these sensors. On the other hand, signals from wearable sensors can be used to identify people correctly, but precise position estimation remains problematic. In this paper, we describe a novel method of integrating laser range finders (LRFs) in the environment and wearable inertial sensors. Time sequences of angular velocities estimated from both LRFs and wearable sensors are matched to identify people. Examples of tracking individuals in the environment that confirm the effectiveness of this method are shown.

People like to play, and robotic technology offers the opportunity to interact with artifacts in new ways. Robots co-existing with humans in domestic and public environments are expected to behave as companions, also engaging in playful interaction. If a robot is small, we foresee that people will want to be able to pick it up and express their intentions playfully by hugging, shaking and moving it around in various ways. Such robots will need to recognize these gestures--which we call “full-body gestures” because they affect the robot's full body. Inertial sensors inside the robot could be used to detect these gestures, in order to avoid having to rely on external sensors in the environment. However, it is not obvious which gestures typically occur during play, and which of these can be reliably detected. We therefore investigate full-body gesture recognition using Sponge Robot, a small humanoid robot equipped with inertial sensors and designed for playful human-robot interaction.

Laser range finders are a non-invasive tool which can be used for anonymously tracking the motion of people and robots in real-world environments with high accuracy. Based on a commercial system we have developed, this paper addresses two practical issues of using networks of portable laser range finders in field environments. We first describe a technique for automated calibration of sensor positions and orientations, by using velocity-based matching of observed human trajectories to define constraints between the sensors. We then propose a mechanism for detecting when a sensor has been moved out of alignment, which can be used to alert an operator of the condition and automatically exclude erroneous data from tracking calculations. After describing our techniques for solving these problems, we demonstrate the effectiveness of our calibration and error detection systems in live trials with our real-time system, as well as offline tests based on scan data recorded from field trials.

Due to the anticipated future, extensive use of robots, human beings will probably share common spaces with them. The relationships between robots and humans will be conducted at close distances. Predicting people's future positions helps robots understand human behavior and react safely and naturally. In this paper, we propose a method for predicting people's positions in crossing behaviors, i.e. different trajectories people follow when they are crossing each other. We conducted a field experiment to gather various crossing behaviors of pedestrians in a shopping mall environment and analyzed them by focusing on “hot areas” spaces where people modify their trajectories for crossing. We clustered typical crossing behaviors in hot areas and modeled them using Hidden Markov Models for predictions. Our algorithm more accurately predicts the future positions of pedestrians by considering moving direction and speed.

Utterance interval detection is a bottleneck for the current speech recognition performance in robots embedded in real noisy environments. In the present work, we make use of sound localization technology using a microphone array, not only for localizing, but also for detecting sound intervals of multiple sound sources. In our previous work we have implemented and evaluated sound localization in the 3D-space using the MUSIC (MUltiple SIgnal Classification) method. In the present work, we proposed a method for detecting sound intervals based on the sound directivity information inferred from the dynamics of the MUSIC spectrogram. The proposed method achieved high sound interval detection accuracies and low insertion rates compared with the previous sound localization results.

Biological creatures, some of them are very simple, seem to perform efficient search strategy. Recent researches show that noise in their internal mechanism may have an important role to manage this behavior. This paper focuses on realizing a simple, noise utilizing, mathematical framework that enables a mobile robot to perform random search adaptively and efficiently under changing target density. Our approach is to model and implement bacterial movement based on a recent perspective of noise utilizing mechanism in living beings: biological fluctuation. As a result, the robot will adaptively switch its random search pattern between Levy walk and Brownian walk to increase the search efficiency in a patchy environment where the target density naturally alternates.

Recently, research fields of augmented reality and robot navigation are actively investigated. Estimating a relative posture between an object and a camera is an important task in these fields Visual markers are frequently used to estimate a relative posture between an object and a camera, but the usage of visual markers spoils a scene. In this paper, we propose a novel method for posture estimation by using kernel regressions and high frequency markers that do not spoil a scene. The markers are embedded in an object's texture in the high frequency domain and it is hard for people to notice the markers. We observe changes of spatial frequency of object's texture by using a high-definition camera to estimate the current posture of an object. We show the effectiveness of our method by experimental results with both simulated and real data.

This paper presents a development framework for social robots with which developers can easily prepare communicative behaviors based on tag-based sentences. Previous literature in human-robot interaction has revealed various useful non-verbal behaviors. But for developers, integrating such a large amount of non-verbal behaviors each time they build a social robot is not realistic. The more repertory of non-verbal behaviors acquire, the larger is the burden faced by developers to implement the non-verbal behaviors. Our software, however, only requires developers to prepare sentences with simple markup language for controlling explicit non-verbal behaviors and utterances. The backend system analyzes the tag input and adds implicit non-verbal behaviors accordingly. With the help of previous literature, the system is equipped with various communicative behaviors, so that the robot autonomously adjusts its gazes, gestures, and standing points. The system's effectiveness is demonstrated with examples on robot interaction where the system receives only simple sentences with scripting language to produce complex and communicative robot behaviors.

This paper reports the persuasion effect of a robot's whispering behavior that consists of a whispering gesture and a request made in a small voice. Whispering gestures naturally make close distance and create warmth feelings with subjects, and requests in quiet voices with whispering gestures also create familiar impressions, which are effective factors of persuasion. We believe that such physical behavior as whispering is one persuasion advantage held by real robots over computers. We conducted a between-subjects experiment to investigate the effectiveness of these two factors on persuasion. In the experiment, the robot requests an annoying task of the subjects; writing as many multiplication table equations as possible. As a result, whispering gestures significantly increased the working time and the number of equations. On the other hand, the loudness of the voice in the request had no effect. We believe the results indicate the effectiveness of physical behavior for persuasion in human-robot interaction.

In near future artificial social agents embodied as virtual agents or as robots with humanoid appearance, will be placed in public settings and used as interaction tools. Considering the uncanny-valley-effect or images of robots as threat for humanity, a study about the acceptance and handling of such an interaction tool in the broad public is of great interest. The following study is based on qualitative methods of interaction analysis focusing on tendencies of peoples' ways to control or perceive a teleoperated android robot in an open public space. This field study shows the tendency of the users to ascribe an own identity to the teleoperated android robot Geminoid HI-1, which is independent from the identity of the controlling person. Both sides of the interaction unit were analyzed for 1) verbal cues about identity presentation on the side of the teleoperator, controlling the robot and for 2) verbal cues about identity perception of Geminoid HI-1 from the side of the interlocutor talking to the robot. The study unveils identity-creation, identity-switching, identity-mediation and identity-imitation of the teleoperators' own identity cues and the use of metaphorical language of the interlocutors showing forms to anthropomorphize and mentalize the android robot whilst interaction. Both sides of the interaction unit thus confer an `incorporated identity' towards the android robot Geminoid HI-1 and unveil tendencies to treat the android robot as social agent.

This paper reports our development of a communication robot for use in a shopping mall to provide shopping information, offer route guidance, and build rapport. In the development, the major difficulties included sensing human behaviors, conversation in a noisy daily environment, and the needs of unexpected miscellaneous knowledge in the conversation. We chose a network-robot system approach, where a single robot's poor sensing capability and knowledge are supplemented by ubiquitous sensors and a human operator. The developed robot system detects a person with floor sensors to initiate interaction, identifies individuals with radio-frequency identification (RFID) tags, gives shopping information while chatting, and provides route guidance with deictic gestures. The robot was partially teleoperated to avoid the difficulty of speech recognition as well as to furnish a new kind of knowledge that only humans can flexibly provide. The information supplied by a human operator was later used to increase the robot's autonomy. For 25 days in a shopping mall, we conducted a field trial and gathered 2642 interactions. A total of 235 participants signed up to use RFID tags and, later, provided questionnaire responses. The questionnaire results are promising in terms of the visitors' perceived acceptability as well as the encouragement of their shopping activities. The results of the teleoperation analysis revealed that the amount of teleoperation gradually decreased, which is also promising.

In this paper, we report a model that allows a robot to appropriately control its position as it presents information to a user. This capability is indispensable, since in the future, many robots will function in daily situations such as shopkeepers presenting products to customers or museum guides presenting information to visitors. Psychology research suggests that people adjust their positions to establish a joint view toward a target object. Similarly, when a robot presents an object, it should stand at an appropriate position that considers the positions of both the listener and the object to optimize the listener's field of view and establish a joint view. We observed human-human interaction situations, where people presented objects, and developed a model for an information-presenting robot to appropriately adjust its position. Our model consists of four constraints to establish O-space: 1) proximity to listener; 2) proximity to object; 3) listener's field of view; and 4) presenter's field of view. We also experimentally evaluate the effectiveness of our model.

Simultaneous teleoperation of mobile, social robots presents unique challenges, combining the real-time demands of conversation with the prioritized scheduling of navigational tasks. We have developed a system in which a single operator can effectively control four mobile robots performing both conversation and navigation. We compare the teleoperation requirements for mobile, social robots with those of traditional robot systems, and we identify metrics for evaluating task difficulty and operator performance for teleoperation of mobile social robots. As a proof of concept, we present an integrated priority model combining real-time conversational demands and non-real-time navigational demands for operator attention, and in a pioneering study, we apply the model and metrics in a demonstration of our multi-robot system through real-world field trials in a shopping arcade.

To explore possible robot tasks in daily life, we developed a guide robot for a shopping mall and conducted a field trial with it. The robot was designed to interact naturally with customers and to affectively provide shopping information. It was also designed to repeatedly interact with people to build a rapport; since a shopping mall is a place people repeatedly visit, it provides the chance to explicitly design a robot for multiple interactions. For this capability, we used RFID tags for person identification. The robot was semi-autonomous, partially controlled by a human operator, to cope with the difficulty of speech recognition in a real environment and to handle unexpected situations.

During conversations, speakers establish their and others' participant roles (who participates in the conversation and in what capacity)--or "footing" as termed by Goffman-using gaze cues. In this paper, we study how a robot can establish the participant roles of its conversational partners using these cues. We designed a set of gaze behaviors for Robovie to signal three kinds of participant roles: addressee, bystander, and overhearer. We evaluated our design in a controlled laboratory experiment with 72 subjects in 36 trials. In three conditions, the robot signaled to two subjects, only by means of gaze, the roles of (1) two addressees, (2) an addressee and a bystander, or (3) an addressee and an overhearer. Behavioral measures showed that subjects' participation behavior conformed to the roles that the robot communicated to them. In subjective evaluations, significant differences were observed in feelings of groupness between addressees and others and liking between overhearers and others. Participation in the conversation did not affect task performance-measured by recall of information presented by the robot-but affected subjects' ratings of how much they attended to the task.

Human communication involves a number of nonverbal cues that are seemingly unintentional, unconscious, and automatic-both in their production and perception-and convey rich information on the emotional state and intentions of an individual. One family of such cues is called "nonverbal leakage." In this paper, we explore whether people can read nonverbal leakage cues-particularly gaze cues-in humanlike robots and make inferences on robots' intentions, and whether the physical design of the robot affects these inferences. We designed a gaze cue for Geminoid-a highly humanlike android-and Robovie-a robot with stylized, abstract humanlike features-that allowed the robots to "leak" information on what they might have in mind. In a controlled laboratory experiment, we asked participants to play a game of guessing with either of the robots and evaluated how the gaze cue affected participants' task performance. We found that the gaze cue did, in fact, lead to better performance, from which we infer that the cue led to attributions of mental states and intentionality. Our results have implications for robot design, particularly for designing expression of intentionality, and for our understanding of how people respond to human social cues when they are enacted by robots.

Providing route directions is a complicated interaction. Utterances are combined with gestures and pronounced with appropriate timing. This study proposes a model for a robot that generates route directions by integrating three important crucial elements: utterances, gestures, and timing. Two research questions must be answered in this modeling process. First, is it useful to let robot perform gesture even though the information conveyed by the gesture is given by utterance as well? Second, is it useful to implement the timing at which humans speaks? Many previous studies about the natural behavior of computers and robots have learned from human speakers, such as gestures and speech timing. However, our approach is different from such previous studies. We emphasized the listener's perspective. Gestures were designed based on the usefulness, although we were influenced by the basic structure of human gestures. Timing was not based on how humans speak, but modeled from how they listen. The experimental result demonstrated the effectiveness of our approach, not only for task efficiency but also for perceived naturalness.

This paper proposes a model of approach behavior with which a robot can initiate conversation with people who are walking. We developed the model by learning from the failures in a simplistic approach behavior used in a real shopping mall. Sometimes people were unaware of the robot's presence, even when it spoke to them. Sometimes, people were not sure whether the robot was really trying to start a conversation, and they did not start talking with it even though they displayed interest. To prevent such failures, our model includes the following functions: predicting the walking behavior of people, choosing a target person, planning its approaching path, and nonverbally indicating its intention to initiate a conversation. The approach model was implemented and used in a real shopping mall. The field trial demonstrated that our model significantly improves the robot's performance in initiating conversations.

In this paper, we present our current research on developing a model of robot behavior that leads to feelings of being together using the robot's body position and orientation. Creating feelings of"being together"will be an essential skill for robots that live with humans and adapt to daily human activities such as walking together or establishing joint attention to information in the environment. We observe people's proxemic behavior in joint attention situations and develop a model of behavior for robots to detect a partner's attention shift and appropriately adjust its body position and orientation in establishing joint attention with the partner. We experimentally evaluate the effectiveness of our model, and our results demonstrate the model's effectiveness.

Social interactions unfold over time, at multiple time scales, and can be observed through multiple sensory modalities. In this paper, we propose a machine learning framework for selecting and combining low-level sensory features from different modalities to produce high-level characterizations of human-robot social interactions in real-time.

Robust localization of robots and reliable tracking of people are both critical requirements for the deployment of service robots in real-world environments. In crowded public spaces, occlusions can impede localization using on-board sensors. At the same time, teams of service robots working together need to share the locations of people and other robots on the same global coordinate system in order to provide services efficiently. To solve this problem, our approach is to use an infrastructure of sensors embedded in the environment to provide an inertial reference frame and wide-area coverage. Based on a people-tracking system we have previously established which uses laser range finders to track people's trajectories, we have developed a technique to localize a team of service robots on a shared global coordinate system. Each robot's odometry data is associated with the observed trajectory of an entity detected by the laser tracking system, and Kalman filters are used to correct rotational offsets between the robots' individual coordinate systems and the global reference frame. We present our data association and pose correction algorithms and show results demonstrating the performance of our system in a shopping arcade.

With the goal of improving human-robot speech communication, the localization of multiple sound sources in the 3D-space based on the MUSIC algorithm was implemented and evaluated in a humanoid robot embedded in real noisy environments. The effects of several parameters related to the MUSIC algorithm on sound source localization and real-time performances were evaluated, for recordings in different environments. Real-time processing could be achieved by reducing the frame size to 4 ms, without degrading the sound localization performance. A method was also proposed for determination of the number of sources, which is an important parameter that influences the performance of the MUSIC algorithm. The proposed method achieved localization accuracies and insertion rates comparable with the case where the ideal number of sources is given.

In human-robot interaction, robots often fail to lead humans to intended reactions due to their limited ability to express affective nuances. In this paper, we propose a motion modification method that combines affective nuances with arbitrary motions of humanoid robots to induce humans to intended reactions by expressing affective states. The method is applicable to various humanoid robots that differ in degrees of freedom or appearances, and the affective nuances are parametrically expressed in a two-dimensional model comprised of valence and arousal. The experimental results showed that the desired affective nuances could be expressed by our method, but it also suggested some limitations. We believe that the method will contribute to interactive systems in which robots can communicate with appropriate expressions in various contexts.

Many biologically inspired approaches have been investigated in relation with researches on mobile robot(s) that can effectively locate targets that induce gradient information. Here, we concentrate on realizing an adaptive searching behavior in mobile robot that is simple, yet effective with and without sensing the gradient information. We are interested in two searching behaviors found in biological creatures: bacterial chemotaxis, probably the simplest yet effective gradient sources searching behavior found in living creatures; and Levy walk, specialized random walks with fractal movement trajectories that optimize random search for sparsely and randomly distributed target(s). Our approach is to implement and combine the two searching behaviors based on ¿yuragi¿, a Japanese word for biological fluctuation.

This paper reports the challenges of developing multiple social robots that operate in a shopping mall. We developed a networked robot system that coordinates multiple social robots and sensors to provide efficient service to customers. It directs the tasks of robots based on their positions and people's walking behavior, manages the paths of robots, and coordinates the conversation-performance between two robots. Laser range finders were distributed in the environment to estimate people's positions. The system estimates such human walking behaviors as ¿stopping¿ or ¿idle walking¿ to direct robots to provide appropriate tasks to appropriate people. Each robot interacts with people to provide recommendation information and route information about shops. The system sometimes simultaneously uses two robots to lead people from one place to another. The field trial, which was conducted in a shopping mall where four robots interacted with 414 people, revealed the effectiveness of the network robot system for guiding people around a shopping mall as well as increasing their interest.

Living organisms have various kinds of flexibility and robustness which are realized by ¿yuragi,¿ i.e. biological fluctuations or noises. Bacterial motion is a form of noise-based motion, since bacteria can move towards a higher concentration of some chemical which they prefer even though they have only a limited 1-DOF of flagella for mobility. Bacteria also have only a limited sensory device which cannot detect the spatial gradient of the chemical at a time. The simple strategies that bacteria take to realize chemotaxis are (1) to tumble (or turn) to change orientation randomly with unbundled flagella in various directions being hit by surrounding water molecules, and (2) to change the frequency of tumbling according as the time change of chemical concentration. In this paper, we describe a small and simple, 1-DOF swimming robot developed by mimicking the bacterial motion generation mechanism. The robot only has a single motor and a single sensor (a photo detector), however, by changing its orientation in response to various noises which exist in the environment, and by changing the frequency of turning, the robot can approach its goal. Experimental results indicate that the robot statistically approaches the goal (a light source) in two dimensional space with its 1-DOF actuator, which would be impossible for the robot to achieve without utilizing noises in the environment.

Recently, many humanoid robots have been developed and actively investigated all over the world in order to realize partner robots which can coexist in an environment shared with humans. having good communication skills is essential in order to interact naturally with humans. However, even with state-of-the-art interaction technology, it is still difficult for humans to interact with humanoid robots without conscious effort. In this paper, we use the android robot, which has an appearance which is quite similar to that of a human, as a bystander in human-human communication. The android is not explicitly involved in the conversation, however it makes small reactions to the behavior of the humans, and the psychological effects of its behavior on the human subjects are investigated. Through the experiments, it is shown that if the android mimics the behavior of the subject this can be quite effective in harmonizing the human-human communication.

The Uncanny Valley hypothesis has been widely used in the areas of computer graphics and human-robot interaction to motivate research and to explain the negative impressions that participants report after exposure to highly realistic characters or robots. Despite its frequent use, empirical proof for the hypothesis remains scarce. This study empirically tested two predictions of the hypothesis: a) highly realistic robots are liked less than real humans and b) the highly realistic robot's movement decreases its likeability. The results do not support these hypotheses and hence expose a considerable weakness in the Uncanny Valley hypothesis. Anthropomorphism and likeability may be multi-dimensional constructs that cannot be projected into a two-dimensional space. We speculate that the hypothesis' popularity may stem from the explanatory escape route it offers to the developers of characters and robots. In any case, the Uncanny Valley hypothesis should no longer be used to hold back the development of highly realistic androids.

In order for humans and robots to engage in direct physical interaction several requirements have to be met. Among others, robots need to be able to adapt their behavior in order to facilitate the interaction with a human partner. This can be achieved using machine learning techniques. However, most machine learning scenarios to-date do not address the question of how learning can be achieved for tightly coupled, physical touch interactions between the learning agent and a human partner. This paper presents an example for such human in-the-loop learning scenarios and proposes a computationally cheap learning algorithm for this purpose. The efficiency of this method is evaluated in an experiment, where human care givers help an android robot to stand up.

The first robotic copies of real humans have become available. They enable their users to be physically present in multiple locations at the same time. This study investigates what influence the embodiment of an agent has on its persuasiveness and its perceived personality. Is a robotic copy as persuasive as its human counterpart? Does it have the same personality? We performed an experiment in which the embodiment of the agent was the independent variable and the persuasiveness and perceived personality were the dependent measurement. The persuasive agent advertised a Bluetooth headset. The results show that an android is found to be as persuasive as a real human or a video recording of a real human. The personality of the participant had a considerable influence on the measurements. Participants that were more open to new experiences rated the persuasive agent lower on agreeableness and extroversion. They were also more willing to spend money on the advertised product.

How humans recognize others is a central topic in both psychology and the field of robot communication. In this paper, we address the issue of such social cognition in a triadic situation since it has not yet been well explored in psychological studies although it is not negligible in daily situations. Classical Heider's balance theory has been suggested in that there are balanced states of triadic relationships that are perceived as comfortable by persons who are involved in them. However, whether, or how, such balanced states tend to be formed has not been well explored. In this paper, we investigate what we call 'social balancing effect' of synchronized nonverbal behavior: social relationships can be attracted to a balanced state due to the synchronized behavior between two agents. In this paper, we conduct an experiment in a conversational interaction among two humans and an android and report that observing eye contacts between the robot and a confederate affected the subject's recognition of the android as attracted to a balanced state.

Recently, various robots capable of having a video chat with distant people have become commercially available. This paper shows that movement of these robots enhances distant people's presence that the robot operator feels. We conducted an experiment to compare the degrees of social telepresence produced by fixed, rotatable, movable, and automatically moving cameras. In this experiment we found that forward-backward movement of the camera significantly contributed to social telepresence, while rotation did not. We also found that this effect disappeared when the camera moved automatically. We propose the user-controllable movement of cameras as a fundamental function for video-based communication systems.

Teleoperation of multiple robots has been studied extensively for applications such as robot navigation; however, this concept has never been applied to the field of social robots. To explore the unique challenges posed by the remote operation of multiple social robots, we have implemented a system in which a single operator simultaneously controls up to four robots, all engaging in communication interactions with users. We present a user inter-face designed for operating a single robot while monitoring several others in the background, then we propose methods for characterizing task difficulty and introduce a technique for improving multiple-robot performance by reducing the number of conflicts between robots demanding the operator's attention. Finally, we demonstrate the success of our system in laboratory trials based on real-world interactions.

We propose that Christopher Alexander's idea of design patterns can benefit the emerging field of HRI. We first discuss four features of design patterns that appear particularly useful. For example, a pattern should be specified abstractly enough such that many different instantiations of the pattern can be uniquely realized in the solution to specific problems in context. Then, after describing our method for generating patterns, we offer and describe eight possible design patterns for sociality in human robot interaction: initial introduction, didactic communication, in motion together, personal interests and history, recovering from mistakes, reciprocal turn-taking in game context, physical intimacy, and claiming unfair treatment or wrongful harms. We also discuss the issue of validation of design patterns. If a design pattern program proves successful, it will provide HRI researchers with basic knowledge about human robot interaction, and save time through the reuse of patterns to achieve high levels of sociality.

A natural human-robot communication is supported by a person's interpersonal behavior for a robot. The condition to elicit interpersonal behavior is thought to be related to a mechanism to support natural communication. In the present study, we hypothesize that motion diversity produced independently of a subject's intention contributes to the human-like nature of the motions of an android that closely resembles a human being. In order to verify this hypothesis, we construct a model of motion diversity through the observation of human motion, specifically, a touching motion. Psychological experiments have shown that the presence of motion diversity in android motion influences the impression toward the android.

This paper reports an initial field trial with a prototype of a semiautonomous communication robot at a train station. We developed an operator-requesting mechanism to achieve semiautonomous operation for a communication robot functioning in real environments. The operator-requesting mechanism autonomously detects situations that the robot cannot handle by itself; a human operator helps by assuming control of the robot.

This paper reports a study about system response time (SRT) in communication robots that utilize human-like social features, such as anthropomorphic appearance and conversation in natural language. Our research purpose established a design guideline for SRT in communication robots. The first experiment observed user preferences toward different SRTs in interaction with a robot. In other existing user interfaces, faster response is usually preferred. In contrast, our experimental result indicated that user preference for SRT in a communication robot is highest at one second, and user preference ratings level off at two seconds.

This paper describes a model for a robot to appropriately control its position when it presents information to a user. This capability is indispensable since in the future many robots will be functioning in daily situations as shopkeepers presenting products to customers or museum guides presenting information to visitors. Psychology research suggests that people adjust their positions to establish a joint view toward a target object. Similarly, when a robot presents an object, it should stand at an appropriate position that considers the positions of both the listener and the object to optimize the listener's field of view and to establish a joint view. We observed human-human interaction situations where people presented objects and developed a model for an information-presenting robot to appropriately adjust its position. Our model consists of four constraints for establishing O-space: (1) proximity to listener, (2) proximity to object, (3) listener's field of view, and (4) presenter's field of view. We also present an experimental evaluation of the effectiveness of our model.

A spiral response-cascade hypothesis is proposed to model the mechanism that enables human communication to emerge or be maintained among agents. In this hypothesis, we propose the existence of three cascades each of which indicates intrapersonal or interpersonal mutual facilitation in the formation of someone's feelings about one's communication partners and the exhibition of behaviors in communicating with them, i.e., responding. In this paper, we discuss our examination of an important part of the hypothesis, i.e., what we call an intrapersonal responding cascade, through an experiment where the gaze interactions between a participant and a communication robot were controlled not only by controlling the robot's gaze but also by signaling participants when to shift their gaze. We report that the participants' experiences in responding to the robot enable them to regard the robot as a communicative being, which partially supports the hypothesis of the intrapersonal responding cascade.

For a robot providing services to people in a public space such as a train station or a shopping mall, it is important to distinguish potential customers, such as window-shoppers, from other people, such as busy commuters. In this paper, we present a series of techniques for anticipating people's behavior in a public space, mainly based on the analysis of accumulated trajectories, and we demonstrate the use of these techniques in a social robot. We placed a ubiquitous sensor network consisting of six laser range finders in a shopping arcade. The system tracks people's positions as well as their local behaviors such as fast walking, idle walking, or stopping. We accumulated people's trajectories for a week, applying a clustering technique to the accumulated trajectories to extract information about the use of space and people's typical global behaviors. This information enables the robot to target its services to people who are walking idly or stopping. The robot anticipates both the areas in which people are likely to perform these behaviors, and also the probable local behaviors of individuals a few seconds in the future. In a field experiment we demonstrate that this system enables the robot to serve people efficiently.

In order to realize androids and humanoids that have very humanlike mechanisms and movements, the development of new linear actuators is necessary. In the development of the actuators, we have found a structure of the linear actuator that is more powerful and quick through iterative simulations based on the 3D finite element method and developed the prototype. This paper reports the structure and various simulations of the new linear actuator. We consider this new actuator will be one of the important components of robots, such as androids and humanoids.

For a person to feel comfortable when approaching a robot it is necessary for the robot to behave in an expected way. Peoplepsilas behavior around a robot not being aware of them were observed during a preliminary experiment. Based on those observations people were classified into four groups depending on their interest in the robot. People were tracked with a laser range finder based system, and their positions, directions and velocities were estimated. A second classification based on that information was made and the relation between the two classifications were mapped. Different actions were created for the robot to be able to react naturally to different human behaviors. In this paper we evaluate three different robot behaviors with respect to how natural they appear. One behavior that actively tries to engage people, one that passively indicates that people have been noticed and a third that makes random gestures. During an experiment test subjects were instructed to act according to the groups from the classification based on interest, and the robotpsilas performance with regard to naturalness was evaluated. Both first and third person evaluation made clear that the active and passive behavior were considered equally natural, while a robot randomly making gestures was considered much less natural.

In order to develop a robot that has a human-like presence, the robot must be given a very human-like appearance and behavior, and a sense of perception that enables it to communicate with humans. We have developed an android robot called ldquoRepliee Q2rdquo that closely resembles a human being; however, the sensors mounted on its body are not sufficient to allow human-like communication due to factors such as sensing range and spatial resolution. To overcome this problem, we endowed the environment surrounding the android with perceptive capabilities by embedding a variety of sensors into it. This sensor network provides the android with human-like perception by constantly and extensively monitoring human activities in a less obvious manner. This paper reports on an android system that is integrated with a sensor network system embedded in the environment. A human-android interaction experiment shows that the integrated system provides relatively human-like interaction.

In the near future, robots are expected to play active roles in human communities. After this time arrives, robots will need to be socially accepted by the people in the communities to which they belong. However, it remains unknown what issues must be resolved to make robots socially accepted. In this paper, we point out the three criteria with which people in communities evaluate a robot and three types of relations people have with the robot. We conducted a six-week study in an office to statistically test the relationship between the evaluations and how well a robot is accepted depending on relation types. We probe that by improving behaviors in the criteria the robot will be more accepted by each type of people. Then, the discussion are presented about the most important issues regarding the social acceptance of robots.

Communication skills are necessary for robots to get by in daily life. Previous studies have shown that the chameleon effect, which increases the likeability of a person who mimicks another during a conversation, appears not only with humans but also with CG agents. However, if we use humans, they cannot accurately mimic. In addition, if we use a CG agent, it doesnpsilat have a very humanlike appearance. It remains doubtful whether the CG agent reappears in a human-human communication. Thus, we use an android with a very humanlike appearance and approach the principle of human-human communication. In this paper, we compared mimic and non-mimic conditions when an android conversed with subjects in face-to-face. Likeability toward an android increased when the android mimicked its partner. In addition, we confirmed the possibility of using an android to explore human-human communication. These results contribute to verbal communication for robots in daily life.

Robots that are supposed to work and communicate with humans in our daily life have been developed in recent years. For this kind of robots, sensitive tactile sensors are necessary to achieve haptic interactions. However, influences of its own motion on tactile sensors sometimes become larger than the signal to be detected. This paper proposes a method to reduce such noises. It is enabled by estimating and subtracting the influences from sensor outputs. The influences of robotpsilas motions on tactile sensors are estimated by using the sequence of joint angles of the robot. In our method, a partially linear model is built to estimate the influences on a tactile sensor. The robotpsilas posture space, which is represented by its joint angles, is divided into several subspaces to fit to a linear model. We conducted an experiment with a robot covered with tactile sensors to verify the validity of our method. This paper shows that the robot is able to estimate the influences on a tactile sensor in high accuracy by the model.

This paper reports a method in which a communicative robot simultaneously interacts with two persons and identifies them with passive-type RFID and floor sensors. The difficulty emanates from the association of these inputs. A passive-type RFID reader provides very accurate person identification. However, the robot cannot specify who actually used the RFID if surrounded by many people. A floor sensor provides robust tracking of people’s positions, but it does not identify any person by itself. Thus, combining these sensors into one device that provides robust tracking of position with accurate person identification would be very beneficial. Toward this problem, in our key idea, the robot keeps multiple hypotheses about the interpretation of sensor information and gradually improves their accuracy through interaction with the people. The multimodal inputs from sensors are integrated with a Bayesian network that allows us to exploit people’s behavior patterns as parameters to estimate the accuracy of our hypotheses. Our method also considers the vagueness of current hypotheses when choosing appropriate interactive behaviors. When the robot has reservations about the situation, it prefers non-conclusive behavior and tries to solve the ambiguity by performing verification behaviors. Once the robot is certain, it prefers conclusive behavior. The developed system was tested in a real daily environment, a shopping center, where we gathered interaction data between the robot and multiple people as teaching data for the Bayesian network. The experimental results revealed that the system successfully identified 79.2% of the visitors. In particular, the hypothesis-based system for behavior control improved the successful rate by 16.7%.

Vocal fry (also called creak, creaky voice, and pulse register phonation) is a voice quality that carries important linguistic or paralinguistic information, depending on the language. We propose a set of acoustic measures and a method for automatically detecting vocal fry segments in speech utterances. A glottal pulse-synchronized method is proposed to deal with the very low fundamental frequency properties of vocal fry segments, which cause problems in the classic short-term analysis methods. The proposed acoustic measures characterize power, aperiodicity, and similarity properties of vocal fry signals. The basic idea of the proposed method is to scan for local power peaks in a ldquovery short-termrdquo power contour for obtaining glottal pulse candidates, check for periodicity properties, and evaluate a similarity measure between neighboring glottal pulse candidates for deciding the possibility of being vocal fry pulses. In the periodicity analysis, autocorrelation peak properties are taken into account for avoiding misdetection of periodicity in vocal fry segments. Evaluation of the proposed acoustic measures in the automatic detection resulted in 74% correct detection, with an insertion error rate of 13%.

The application range of communication robots could be widely expanded by the use of automatic speech recognition (ASR) systems with improved robustness for noise and for speakers of different ages. In past researches, several modules have been proposed and evaluated for improving the robustness of ASR systems in noisy environments. However, this performance might be degraded when applied to robots, due to problems caused by distant speech and the robot's own noise. In this paper, we implemented the individual modules in a humanoid robot, and evaluated the ASR performance in a real-world noisy environment for adults' and children's speech. The performance of each module was verified by adding different levels of real environment noise recorded in a cafeteria. Experimental results indicated that our ASR system could achieve over 80% word accuracy in 70-dBA noise. Further evaluation of adult speech recorded in a real noisy environment resulted in 73% word accuracy.

Identifying the extent to which the appearance of a humanoid robot affects human behavior toward it is important. We compared participant impressions of and behaviors toward two real humanoid robots in simple human-robot interactions. These two robots, which have different appearances but are controlled to perform the same recorded utterances and motions, are adjusted by a motion-capturing system. We conducted an experiment with 48 human participants who individually interacted with the two robots and also with a human for reference. The results revealed that different appearances did not affect participant verbal behaviors, but they did affect such nonverbal behaviors as distance and delay of response. These differences are explained by two factors: impressions and attributions.

Human beings subconsciously adapt their behaviors to a communication partner in order to make interactions run smoothly. In human-robot interactions, not only the human but also the robot is expected to adapt to its partner. Thus, to facilitate human-robot interactions, a robot should be able to read subconscious comfort and discomfort signals from humans and adjust its behavior accordingly, just like a human would. However, most previous research works expected the human to consciously give feedback, which might interfere with the aim of interaction. We propose an adaptation mechanism based on reinforcement learning that reads subconscious body signals from a human partner, and uses this information to adjust interaction distances, gaze meeting, and motion speed and timing in human-robot interactions. The mechanism uses gazing at the robot's face and human movement distance as subconscious body signals that indicate a human's comfort and discomfort. A pilot study with a humanoid robot that has ten interaction behaviors has been conducted. The study result of 12 subjects suggests that the proposed mechanism enables autonomous adaptation to individual preferences. Also, detailed discussion and conclusions are presented.

This paper reports a method that uses humanoid robots as a communication medium. There are many interactive robots under development, but due to their limited perception, their interactivity is still far poorer than that of humans. Our approach in this paper is to limit robots' purpose to a non-interactive medium and to look for a way to attract people's interest in the information that robots convey. We propose using robots as a passive-social medium, in which multiple robots converse with each other. We conducted a field experiment at a train station for eight days to investigate the effects of a passive-social medium.

In this research, we realize human telepresence by developing a remote-controlled android system called Geminoid HI-1. Experimental results confirm that participants felt stronger presence of the operator when he talked through the android than when he appeared on a video monitor in a video conference system. In addition, participants talked with the robot naturally and evaluated its human likeness as equal to a man on a video monitor. At this paper's conclusion, we will discuss a remote-control system for telepresence that uses a human-like android robot as a new telecommunication medium.

This paper describes a group attention control (GAC) system that enables a communication robot to simultaneously interact with many people. GAC is based on controlling social situations and indicating explicit control to unify all purposes of attention. We implemented a semi-autonomous GAC system into a communication robot that guides visitors to exhibits in a science museum and engages in free-play interactions with them. The GAC system's effectiveness was demonstrated in a two-week experiment in the museum. We believe these results will allow us to develope interactive humanoid robots that can interact effectively with groups of people.

We have taken steps towards developing a method that enables an interactive humanoid robot to adapt its speed to a walking human that it is moving together with. This is difficult because the human is simultaneously adapting to the robot. From a case study in human-human walking interaction we established a hypothesis about how to read a human's speed preference based on a relationship between humans' walking speed and their relative position in the direction of walking. We conducted two experiments to verify this hypothesis: one with two humans walking together, and one with a human subject walking with a humanoid robot, Robovie-IV. For 11 out of 15 subjects who walked with the robot, the results were consistent with the speed-position relationship of the hypothesis. We also conducted a preferred speed estimation experiment for six of the subjects. All of them were satisfied with one or more of the speeds that our algorithm estimated and four of them answered one of the speeds as the best one if the algorithm was allowed to give three options. In the paper, we also discuss the difficulties and possibilities that we learned from this preliminary trial.

We have developed a new communication robot, Robopal, which is an indoor/outdoor robot for use in human-robot interaction research in the context of daily life. Robopal's intended applications involve leading and/or following a human to a destination. Preliminary experiments have been conducted to study nonverbal cues associated with leading and following behavior, and it has been observed that some behaviors, such as glancing towards the leader or follower, appear to be role-dependent. A system for representing these behaviors with a state transition model is described, based on four kinds of interaction roles: directive, responsive, collaborative, and independent. It is proposed that behavior modeling can be simplified by using this system to represent changes in the roles the robot and human play in an interaction, and by associating appropriate behaviors to each role

This paper reports a study that estimated visitor positions, visiting patterns, and inter-human relationships at a science museum using information from RFID readers. In the science museum, we exhibited humanoid robots. Visitors were invited to wear RFID tags to interact with the robots. Visitor behavior was simultaneously observed using 20 RFID readers, distributed throughout the entire floor, that roughly measured the distances of nearby tags. We integrated the outputs from all RFID readers to estimate visitor trajectories that were used to analyze three perspectives: space, visiting patterns, and relationships. Regarding space, we identified crowded and uncrowded areas. We found several typical visiting patterns, such as visited at every exhibit and directly going to robot area. We also identified atypical visiting behavior. Regarding relationships, for example, we estimated 68% coverage of group-member relationships with 91% reliability.

One of the critical research issues for the future robot is the avoidance of collisions with people while it moves among them. Therefore, people-tracking and motion prediction are important. People are tracked more readily and successfully by using distributed sensor networks than robot's local sensors. Floor sensor networks, in particular, are resistant to changes in lighting conditions and other environmental disturbances. However, the problem with the system is that a person walking is observed as if it were being done in "rabbit hops": The signal is nonlinear and even nonholonomic. We tackled the problem by assuming that human walking is regular in terms of walking rhythms. Then the signal was modeled by an oscillation model based on four walking parameters: cycles, phases, directions, and strides. To adapt the parameters to irregular walking, the model was multi-hypothesized based upon a particle filtering algorithm. Experimental results showed that the multi-hypothesized oscillation models showed more than 80% tracking accuracy for four walking patterns: straight walking, stop-and-go, turn, and winding walking. Further, the models were superior to the nearest neighbor filter with regard to the performance of data association for two persons.

In this paper we present a method for determining body orientation and pose information from laser scanner data using particle filtering with an adaptive modeling algorithm. A parametric human shape model is recursively updated to fit observed data after each resampling step of the particle filter. This updated model is then used in the likelihood estimation step for the following iteration. This method has been implemented and tested by using a network of laser range finders to observe human subjects in a variety of interactions. We present results illustrating that our method can closely track torso and arm movements even with noisy and incomplete sensor data, and we show examples of body language primitives that can be observed from this orientation and positioning information.

To realize accurate tracking of people in the environment, many studies have been proposed using vision sensors, floor sensors, and wearable devices. The problem of using vision sensors is that they do not provide ID information of each people and there are ambiguities when people come across. To solve the problem, we propose to combine acceleration sensors that are attached to the human body. Since the signals from vision sensors and acceleration sensors synchronize when they observe same person who are acting or walking in the environment, these signals are not independent. The correlation between the signals is evaluated based on the canonical correlation analysis. Experimental results are shown to detect gesture and to track people to confirm the effectiveness of the proposed method.

With the aim of automatically generating head motions during speech utterances, analyses are conducted for verifying the relations between head motions and linguistic and paralinguistic information carried by speech utterances. Motion captured data are recorded during natural dialogue, and the rotation angles are estimated from the head marker data. Analysis results showed that nods frequently occur during speech utterances, not only for expressing specific dialog acts such as agreement and affirmation, but also as indicative of syntactic or semantic units, appearing at the last syllable of the phrases, in strong phrase boundaries. Analyses are also conducted on the dependence on linguistic, prosodic and voice quality information of other head motions, like shakes and tilts, and discuss about the potentiality for their use in automatic generation of head motions. The paper also proposes a method for controlling the head actuators of an android based on the rotation angles, and evaluates the mapping from the human head motions.

In order to develop a robot working in daily situations, it is necessary to discover the principles relevant to establishing and maintaining social interaction between human and robot. One important issue is discovering how to naturally animate a robot to maintain social interaction. This study tackles the issue through implementing natural motions in the android which closely resembles those of human beings. This paper proposes a method to implement postures that look human by mapping the three-dimensional positions of a human subject body onto the android.

The distance between robots and users increasingly shrinks and with it the potential for robots to inadvertently harm users increasingly grows. Robots need to detect ongoing haptic interaction with users via malleable skin with flexible tactile sensors covering the whole body surface in order to reduce the risk and to realize enriched interactions. In this paper, we propose a feature space based on cross-correlation between tactile sensors, and we visualized it as a 2-dimensional map which we call Somatosensory Map. We construct a haptic interaction database using scenario based haptic interaction between humans and a robot that we equipped with distributed tactile sensors covering the whole body. In the Somatosensory Map, highly correlated sensors make clusters corresponding to distinctive haptic interactions between the robot and the subject. Each haptic interaction can thus be classified using cross-correlation between tactile sensors. To confirm the validity of our proposal, we apply the K-Nearest Neighbor (KNN) method using the feature space. As a validation test result, despite KNN being one of the simplest methods, over 60% of haptic interaction in which we may expect subject’s touches can be recognized. In conclusion, the cross-correlation includes good features to classify the interaction. As future work, the classifier should be improved to use effective cross-correlation shown as distribution of distinctive clusters in the Somatosensory Map.

A simple view of deictic communication only includes the indication process and recognition process: a person points at an object and says something about it such as “look at this,” and then the other person recognizes the pointing gesture and pays attention to the indicated object. However, this simple view lacks three important processes: attention synchronization, context focus, and believability establishment. We refer to these three processes as “facilitation processes” and implement them in a humanoid robot with a motion capturing system. An experiment with 30 subjects revealed that the facilitation processes make deictic communication natural.

The tactile sensor is required for various robots. In humanoid robots, flexibility of the sensor is an important feature for preventing physical damage and for interacting with the human. Moreover, stretchability of the sensor has advantages that the sensor is nonbreakable and that the sensor can be easily mounted on curved surfaces or deformable parts such as joints. This paper proposes a novel tactile sensor made of flexible and stretchable silicone rubber. A structure of the sensor is similar to the capacitive tactile sensors. However, the proposed sensor utilizes a different principle from existing sensors. The sensor utilizes static electricity and electrostatic induction phenomenon, and can detect some touch conditions. This paper reports the principle and characteristics of the proposed sensor. Experiments show that the sensor output depends on touch area, touch velocity, and material of touch objects. However, the sensor does not depend on touch weight. Moreover, the experiment shows that even if the proposed sensor is stretched, it performs as the tactile sensor.

An important part of natural, and therefore effective, communication is facial motion. The android Repliee Q2 should therefore display realistic facial motion. In computer graphics animation, such motion is created by mapping human motion to the animated character. This paper proposes a method for mapping human facial motion to the android. This is done using a linear model of the android, based on blendshape models used in computer graphics. The model is derived from motion capture of the android and therefore also models the android's physical limitations. The paper shows that the blendshape method can be successfully applied to the android. Also, it is shown that a linear model is sufficient for representing android facial motion, which means control can be very straightforward. Measurements of the produced motion identify the physical limitations of the android and allow identifying the main areas for improvement of the android design.

Recently, remote-controlled communication robots have been studied. Our question is whether the person interacting with them feels that he/she is interacting with the robot itself or the human behind it. How do such beliefs affect interaction? We conducted an experiment to study the effect of a human presence behind an interactive humanoid robot. We used an interactive robot system based on a motion-capturing system where participants can bodily interact with the robot. They were deceptively informed that they would either be interacting with a robot controlled by a program or a human; in fact, in both conditions the robot was autonomous. Experimental results indicated that two-thirds of the participants felt that they were interacting with the robot itself. Their enjoyment was unaffected by the knowledge of whether the robot was controlled by a program or a human, although their impression of robot intelligence indicated that they distinguished between these conditions. On the contrary, since the remaining one-third of the participants felt they were interacting with a human behind the robot, they were affected by the knowledge: when they were told they were interacting with a teleoperated robot, they enjoyed it less; in contrast, their enjoyment increased when they were told they were interacting with an autonomous robot.

The uncanny valley theory proposed by Mori in 1970 has been a hot topic in human robot interaction research, in particular since the development of increasingly human-like androids and computer graphics. In this paper we describe an empirical study that attempts to plot Mori's hypothesized curve. In addition, the influence of framing on the users' perception of the stimuli was investigated. Framing had no significant influence on the measurements. The pictures of robots and humans were rated independently of whether the participants knew a particular picture showed a robot or human. Anthropomorphism had a significant influence on the measurements, but not even pictures of real humans were rated as likeable as the pictures of humanoids or toy robots. As a result we suggest the existence of an uncanny cliff model as an alternative to the uncanny valley model. However, this study focused on the perception of pictures of robots and the results, including the suggested model, may be different for the perception of movies of moving robots or the perception of standing right in front of a moving robot.

The "uncanny valley" must be avoided from the viewpoint of communication robot design, while this is an essential phenomenon for discovering the principles relevant to establishing social interaction between human and robot. Studying the uncanny valley allows us to explore the boundary of human-likeness. We have empirically and experimentally obtained evidence to show the uncanny valley. We have also obtained experimental evidence to suggest that the uncanny valley varies owing to the development of individuals. We refer to this variable uncanny valley as the age-dependent uncanny valley. We assume that the uncanny valley is induced by a lateral inhibition effect, and is referred to herein as the lateral inhibition hypothesis of the uncanny valley;. The present paper presents evidence concerning the uncanny valley and describes the likelihood of the present hypothesis.

We believe that lifelikeness is important to make it possible for a communication robot to interact naturally with people. In this research, the relationship between contingency and complexity in a lifelike robot was investigated. We have developed a robot control system to control experimentally contingency and complexity in interaction by combining a humanoid robot with a motion-capturing system. Two independent experiments were conducted with different levels of interaction complexity: simple interaction and complex interaction. Experiments in the simple interaction situation indicated that subjects felt that the robot behaved autonomously when contingency was low, and there was no significant relationship between contingency and a lifelike impression. On the other hand, experiments in the complex interaction situation showed that the robot gave the subjects the impression of being more autonomous and lifelike when contingency was high.

Interactive robots participating in our daily lives should have the fundamental ability to socially communicate with humans. In this paper, we propose a mechanism for two social communication abilities: forming long-term relationships and estimating friendly relationships among people. The mechanism for long-term relationships is based on three principles of behavior design. The robot we developed, Robovie, is able to interact with children in the same way as children do. Moreover, the mechanism is designed for long-term interaction along the following three design principles: (1) it calls children by name using radio frequency identification tags; (2) it adapts its interactive behaviors for each child based on a pseudo development mechanism; and (3) it confides its personal matters to the children who have interacted with the robot for an extended period of time. Regarding the estimation of friendly relationships, the robot assumes that people who spontaneously behave as a group together are friends. Then, by identifying each person in the interacting group around the robot, it estimates the relationships between them. We conducted a two-month field trial at an elementary school. An interactive humanoid robot, Robovie, was placed in a classroom at the school. The results of the field trial revealed that the robot successfully continued interacting with many children for two months, and seemed to have established friendly relationships with them. In addition, it demonstrated reasonable performance in identifying friendships among children. We believe that these results demonstrate the potential of current interactive robots to establish social relationships with humans in our daily lives.

The development of interactive robots is a complicated process, involving a plethora of psychological, technical, and contextual influences. To design a robot capable of operating ldquointelligentlyrdquo in everyday situations, one needs a profound understanding of human-robot interaction (HRI). We propose an approach based on integral analysis of multimodal data to pursue this understanding and support interdisciplinary research and development in the field of robotics. To adopt this approach, a software tool named interaction debugger was developed that features user-friendly navigation, browsing, searching, viewing, and annotation of data; it enables fine-grained inspection of the HRI. In four case studies, we demonstrated how our analysis approach aids the development process of interactive robots.

Along with the development of interactive robots, controlled experiments and field trials are regularly conducted to stage human-robot interaction. Experience in this field has shown that analyzing human-robot interaction for evaluation purposes fosters the development of improved systems and the generation of new knowledge. In this paper, we present the interaction debugging approach, which is based on the collection and analysis of data from robots and their environment. Considering the multimodality of robotic technology, often only audio and video are insufficient for detailed analysis of human-robot interaction. Therefore, in our analysis we integrate multimodal information using audio, video, sensory data, and intermediate variables. An important aspect of the interaction debugging approach is using a tool called Interaction Debugger to analyze data. By supporting user-friendly data presentation, annotation and navigation, Interaction Debugger enables fine-grained inspection of human-robot interaction. The main goal of this paper is to address how an integral approach to the analysis of human-robot interaction can be adopted. This is demonstrated by three case studies.

This paper investigates human behavior around an interactive robot at a science museum. To develop a communication robot that works in daily environments, it is important to investigate the available information from a robot about people's behavior. Such information will enable the robot to predict people's behavior so that the robot can optimize its interactive behavior. We analyzed visitor behavior toward a simple interactive robot exhibited at a science museum in relation to information from sound level and range sensors. We discovered factors that influence the way people approach, maintain distance, and interact both physically and verbally with the robot. This enabled us to extract meaningful information from the sensory information and apply it to communication robots.

This paper reports on a field trial with interactive humanoid robots at a science museum where visitors are supposed to study and develop an interest in science. In the trial, each visitor wore an RFID tag while looking around the museum's exhibits. Information obtained from the RFID tags was used to direct the robots' interaction with the visitors. The robots autonomously interacted with visitors via gestures and utterances resembling the free play of children [1]. In addition, they performed exhibit-guiding by moving around several exhibits and explaining the exhibits based on sensor information. The robots were highly evaluated by visitors during the two-month trial. Moreover, we conducted an experiment in the field trial to compare the detailed effects of exhibit-guiding and free-play interaction under three operating conditions. This revealed that the combination of the free-play interaction and exhibit-guiding positively affected visitors' experiences at the science museum.

The purpose of our research is to develop lifelike behavior in a communication robot, which is expected to potentially make human-robot interaction more natural. Our earlier research demonstrated the importance of a robot's contingency for lifelikeness [1]. On the other hand, perfect contingency seems to give us a non-lifelike impression. In order to explore the appropriate contingency for communication robots, we developed a robot system that allows us to adjust its contingency to an interacting person in a simple mimic interaction. As a result of an experiment, we identified the relationships between the degree of contingency and the subjective impressions of lifelikeness, autonomy, and preference. However, the experimental result also seems to suggest the importance of the complexity of interaction for investigating the appropriate contingency of communication robots.

This paper presents an analysis on the functions carried by phrase final tones in turn-taking and dialog acts, taking into account linguistic information about the part of speech (particles and auxiliary verbs) attributed to the morphemes at phrase finals. Natural conversational speech data are segmented in inter-pause units, and each utterance unit is arranged according to the phrase final morphemes. Turntaking functions are annotated, and tones of each phrase final are described by acoustic-prosodic features. Analysis results show a relationship between tones and turn-taking functions in most of the morphemes, while no clear relationship is found in some classes of morphemes which are final particles.

Aiming to realize a non-verbal communication between humans and robots, the use of acoustic parameters related with voice quality features, besides classical prosodic features, is proposed and evaluated for automatic extraction of paralinguistic information (intentions, attitudes, and emotions) in dialog speech. Experimental results indicated that prosodic features were effective for detecting groups of paralinguistic information expressing specific functions (such as affirmation, denial, and asking for repetition), accounting for 61% of the global identification rate. Voice quality features were effective for detecting part of the paralinguistic information expressing emotions or attitudes (such as surprise, disgust and admiration), leading to 12 % improvement in the global identification rate

We believe that robots will take an active part in our daily lives in the near future. Once this time arrives, robots will be expected to have the social skills necessary for interacting with people in addition to the ability to carry out their own tasks. We call an interaction that increases familiarity and makes communication smoother a "social interaction." We have recently developed a human-size humanoid, called "Robovie-IV," which will have the ability to interact with people in their daily lives. This paper discusses the design requirements for humanrobot interaction by a communication robot, then introduces an overview of Robovie-IV's hardware and software architectures. There are also preliminary results of a daily experiment that we are currently conducting with Robovie-IV in our office.

This paper describes a method for recognizing the environment and selecting behaviors for a humanoid robot that are appropriate to the environment, based on sensor histories. In order for humanoid robots to move around many kinds of environments, one of the keys is selecting the appropriate behavior. Because features of an environment that deeply influence a robot's motion, such as viscous friction or floor elasticity, are difficult to immediately measure before moving, it is difficult for the robots to plan their behaviors to take these factors into account. Therefore, in our method, a robot first records a long time series of data from sensors attached to its body while performing each behavior in each environment, and it builds decision trees based on this data for each of the behaviors, enabling it to recognize the current environment based on the sensor history. By using the trees, the robot selects the behavior which will be most effective for recognizing the environment, acquires a sensor history while doing the behavior, and selects likely candidates for the environment. By iterating over these steps, the robot can recognize the environment and select an appropriate behavior. To verify the validity of the method, we used a small-size humanoid robot and conducted an experiment to recognize environments in a family's house

This paper presents a three-layer model for generation and recognition of attention-drawing behavior. The model enables a robot to recognize people's attention-drawing behavior as well as to perform attention-drawing behavior to people. It consists of three layers: the PSM (pointing space model), the RTM (reference term model), and the OPM (object property model). The PSM associates the pointing gesture with a reference term, the RTM associates positional relationships with a reference term, and the OPM associates other supplemental verbal cues with a reference term. We implemented the model in a humanoid robot, Robovie, and verified its effectiveness through an experiment

In face-to-face communication, eyes play a central role, for example in directing attention and regulating turn-taking. For this reason, it has been a central topic in several fields of interaction study. Although many psychology findings have encouraged previous work in both human-computer and human-robot interaction studies, so far there have been few explorations on how to move the agent's eye, including when to move it, for communication. Therefore, it is this topic we address in this study. The impression a person forms from an interaction is strongly influenced by the degree to which their partner's gaze direction correlates with their own. In this paper, we propose methods of controlling a robot's gaze responsively to its partner's gaze and confirm the effect of this on the feeling of being looked at, which is considered to be the basis of conveying impressions using gaze in face-to-face interaction experiments. Furthermore, an additional preliminary experiment with an on-screen agent shows the possibility of using blinking behaviour as another modality for responding to a partner

We, humans, anthropomorphize targets of communication. In this sense, humanoids or androids can have ideal interface for humans. This paper focuses on two new fundamental issues in the human interface studies. There are two relationships between robots and humans: one is inter-personal and the other is social. In the inter-personal relationships, the appearance of the robot is a new and important research issues. In the social relationships, a function to recognize human relationships through interaction is needed for robots of the next generation. These two issues explore new possibilities of androids and humanoids. Especially, the appearance problem bridges between science and engineering. The approach from robotics tries to build very humanlike robots based on knowledge from cognitive science. The approach from cognitive science uses the robot for verifying hypotheses for understanding humans. We call this cross-interdisciplinary framework android science.

In this paper, we move toward offering psychological benchmarks by which to measure success in building increasingly human-like robots. By psychological benchmarks we mean categories of interaction that capture conceptually fundamental aspects of human life, specified abstractly enough so as to resist their identity as a mere psychological instrument, but capable of being translated into testable empirical propositions. Six possible benchmarks are considered: autonomy, imitation, intrinsic moral value, moral accountability, privacy, and reciprocity. Finally, we discuss how getting the right group of benchmarks in human-robot interaction will, in future years, help inform on the foundational question of what constitutes essential features of being human

In this paper, we propose natural reflexive behaviors for wheeled inverted pendulum type humanoid robots and show that such behaviors change human impressions of the robot. To achieve human-like communication between humans and robots, the latter need to be recognized by the former as communication partners. We believe that the natural reflexive behaviors of robots play an important role in being human partners. To validate the evidence, we generate four types of reflexive robot behaviors when pushed by someone and conducted preliminary experiments with subjects. We verified that people change their impressions of extroversion, agreeableness, intellect, and neuroticism for the robot by changing the reflexive behaviors

This paper introduces a "teleoperated communication robot" whose unique point is that its language component is performed by an operator to avoid the automatic recognition difficulty of spoken language. On the other hand, conceptually, its nonverbal behavior will be autonomously controlled. One important point is to identify whether ordinary people will accept this type of communication robot. Therefore, we conducted a preliminary field trial at a shopping center, which revealed positive perspectives as well as technical problems to be solved

Coding can contribute to robot design by suggesting behavioral benchmarks. These, however, depend on the level of analysis. In illustration, semi-formalized rules are used to investigate child-robot encounters. By using behavior-level codes, we extract information about how children use the robot. This leads to findings about longitudinal changes in how children evaluate its behaviors. Children, we find, use the robot as a social mediator - to prompt synchronized social events. By focusing on a behavioral level, coding can benefit designers of robots, software and sensors

In face-to-face communication, eyes play a central role, for example, in directing attention and regulating turn-taking. For this reason, the eyes have been a central topic in several fields of interaction study. Although many psychological findings have encouraged previous work in both human-computer and human-robot interaction studies, so far there have been few explorations on how to move the agent's eye, including when to move it, for communication. Therefore, we investigate this topic in this study. The impression a person forms from an interaction is strongly influenced by the degree to which their partner's gaze direction correlates with their own. In this paper, we evaluate two primitive ways of controlling a robot's gaze responsively to its partner's gaze by comparison with two additional ways of non-responsive gaze control in an experiment set in a conversational situation. Consequently, we confirm that robots with different types of responsive gaze control can give subjects different impressions of its characteristics

How can we develop a robot that can naturally interact with people? One approach is to build an android whose appearance and movements are very humanlike. People anthropomorphize objects for communication. Therefore, we can use androids as a testbed for studying human–human and human–machine interactions. This novel research approach is called android science. In android science, the methods of engineering and cognitive science converge. Roboticists attempt to build very humanlike robots based on knowledge from cognitive science, while cognitive scientists employ robots to verify cognitive hypotheses about human beings. This paper proposes the framework of android science, introduces the androids we have developed, and discusses conscious and subconscious human recognition of these androids as a fundamental issue for android science.

When describing a physical object, we indicate which object by pointing and using reference terms, such as ‘this’ and ‘that’, to inform the listener quickly of an indicated object's location. Therefore, this research proposes using a three-layer attention-drawing model for humanoid robots that incorporates such gestures and verbal cues. The proposed three-layer model consists of three sub-models: the Reference Term Model (RTM); the Limit Distance Model (LDM); and the Object Property Model (OPM). The RTM selects an appropriate reference term for distance, based on a quantitative analysis of human behaviour. The LDM decides whether to use a property of the object, such as colour, as an additional term for distinguishing the object from its neighbours. The OPM determines which property should be used for this additional reference. Based on this concept, an attention-drawing system was developed for a communication robot named ‘Robovie’, and its effectiveness was tested.

Many robotics researchers are exploring new possibilities of intelligent robots in our everyday life. Humanoid and androids, which have various modalities, can communicate with humans as new information media. In this talk, we argue how to develop the interactive robots and how to evaluate them as introducing several robots developed in ATR Intelligent Robotics and Communications Laboratories and Department of Adaptive Machine Systems, Osaka University. Especially, we focus on a constructive approach to developing the interactive robots, cognitive studies using the humanoids and androids for evaluating the interactions, and long-term field experiments in an elementary school.The talk consists of two parts. There are two relationships between robots and humans: one is inter-personal and the other is social. In the inter-personal relationships, the appearance of the robot is a new and important research issues. In the social relationships, a function to recognize human relationships through interactions is needed for robots of the next generation. These two issues explore new possibilities of robots.In these issues, the appearance problem bridges between science and engineering. In the development of humanoids, both the appearance and behavior of the robots are significant issues. However, designing the robot's appearance, especially to give it a humanoid one, was always a role of the industrial designer. To tackle the problem of appearance and behavior, two approaches are necessary: one from robotics and the other from cognitive science. The approach from robotics tries to build very humanlike robots based on knowledge from cognitive science. The approach from cognitive science uses the robot for verifying hypotheses for understanding humans. We call this cross-interdisciplinary framework android science (www.androidscience.com).The speaker hopes that attendees catch new waves in robotics and media research and our feature life.

Vocal fry is a voice quality that often appears in relaxed voices indicating low tension, or in pressed voices expressing attitudes/feelings of surprise, admiration and suffering. We propose a set of acoustic measures for automatically detecting vocal fry segments in speech utterances. In order to deal with vocal fry utterances with very low fundamental frequencies, where classic short-term analysis methods become problematic, a glottal pulse synchronized method is proposed. The acoustic measures are based on power, periodicity and similarity properties of vocal fry signals. The basic idea is to scan for power peaks in a “very short-term ” power contour, check for periodicity properties and evaluate a similarity measure between power peaks for deciding the possibility of vocal fry pulses. Sub-harmonic properties are also taken into account in the periodicity analysis. Evaluation of the proposed measures in automatic detection resulted in 73.3 % correct detection, with an insertion error rate of 3.9 %. 1.

We propose a method for promoting human-robot interaction based on emotion recognition with particular focus on tension emotion. There are two types of emotions expressed in a short time. One is autonomic emotion caused by a stimulus, such as joy and fear. The other is self-reported emotion, such as tension, that is relatively independent of a single stimulus. In our preliminary experiment, we observed that tension emotion (self-reported emotion) obstructs the expression of autonomic emotion, which has demerits on speech recognition and interaction. Our method is based on detection and moderation of tension emotion. If a robot detects tension emotion, it tries to ease it so that a person will interact with it more comfortably and express autonomic emotions. It also retrieves nuances from expressed emotions for supplementing insufficient speech recognition, which will also promote interaction.

It is important to identify how much the appearance of a humanoid robot affects human behaviors toward it. We compared participants' impressions of and behaviors toward two real humanoid robots in simple human-robot interaction. These two robots have different appearances but are controlled to perform the same recorded utterances and motions, which are adjusted by using a motion capturing system. We conducted an experiment where 48 human participants participated. In the experiment, participants interacted with the two robots one by one and also with a human as a reference. As a result, we found that the different appearances did not affect the participants' verbal behaviors but did affect their non-verbal behaviors such as distance and delay of response. These differences are explained by two factors, impressions and attributions.

One of the main aims of humanoid robotics is to develop robots that are capable of interacting naturally with people. However, to understand the essence of human interaction, it is crucial to investigate the contribution of behavior and appearance. Our group's research explores these relationships by developing androids that closely resemble human beings in both aspects. If humanlike appearance causes us to evaluate an android's behavior from a human standard, we are more likely to be cognizant of deviations from human norms. Therefore, the android's motions must closely match human performance to avoid looking strange, including such autonomic responses as the shoulder movements involved in breathing. This paper proposes a method to implement motions that look human by mapping their three-dimensional appearance from a human performer to the android and then evaluating the verisimilitude of the visible motions using a motion capture system. This approach has several advantages over current research, which has focused on copying a person's moving joint angles to a robot: (1) in an android robot with many degrees of freedom and kinematics that differs from that of a human being, it is difficult to calculate which joint angles would make the robot's posture appear similar to the human performer; and (2) the motion that we perceive is at the robot's surface, not necessarily at its joints, which are often hidden from view.

In this paper, we propose an adaptation mechanism for robot behaviors to make robot-human interactions run more smoothly. We propose such a mechanism based on reinforcement learning, which reads minute body signals from a human partner, and uses this information to adjust interaction distances, gaze meeting, and motion speed and timing in human-robot interaction. We show that this enables autonomous adaptation to individual preferences by an experiment with twelve subjects.

When we talk about objects in an environment, we indicate to a listener which object is currently under consideration by using pointing gesture and such reference terms as "this" and "that". Such reference terms play an important role in human interaction by quickly informing the listener of an indicated object's location. In this research, we propose a three-layered draw-attention model for humanoid robots with gestures and verbal cues. Our proposed three-layered model consists of three sub models: reference term model (RTM), limit distance model (LDM) and object property model (OPM). RTM decides an appropriate reference term using functions constructed by an analysis of human behavior. LDM decides whether to use the object's property with a reference term. OPM decides the appropriate property for indicating the object by comparing object properties with each other. We developed an attention drawing system in a communication robot named "Robovie" based on the three layered model. We confirmed its effectiveness through the experiments.

One major design goal in human-robot interaction is that the robots behave in an intelligent manner, preferably in a similar way as humans. This constraint must also be taken into consideration when the navigation system for the platform is developed. However, research in human-robot interaction is often restricted to other components of the system including gestures, manipulation, and speech. On the other hand, research for mobile robot navigation focuses primarily on the task of reaching a certain goal point in an environment. We believe that these two problems can not be treated separately for a personal robot that coexists with humans in the same surrounding. Persons move constantly while they are interacting with each other. Hence, also a robot should do that, which poses constraints on the navigation system. This type of navigation is the focus of this paper. Methods have been developed for a robot to join a group of people engaged in a conversation. Preliminary results show that the platform's moving patterns are very similar to the ones of the persons. Moreover, this dynamic interaction has been judged naturally by the test subjects, which greatly increases the perceived intelligence of the robot.

Previous research works in robotics and cognitive science have reported that humans utilize embodied cooperative behaviors in communication, such as nodding in response to another's it utterance and looking at a certain object in a certain direction as others look or point at. We have developed a humanoid robot that utilizes such an embodied cooperative behavior for natural communication in a route guidance situation. It obtains numerical data on a human's body movement via a motion capturing system and then autonomously selects appropriate cooperative embodiment units from 18 implemented units. Each unit realizes a certain cooperative embodiment behaviors such as eye-contact by using the motion capturing system as well. As a result of a subject experiment, we have verified the effectiveness of the embodied cooperative behaviors of the robot for reliable and sympathetic communication. Moreover, we analyzed how the auditory expression and the embodiment contributed to the effect.

This paper reports our novel approach to developing a social robot. The developed robot is able to identify relationships among humans from their physical behaviors as it establishes long-term relationships with humans. This interactive humanoid robot attracts humans to interact with it and, as a result, induces them to perform their group behaviors in front of it. The robot recognizes friendly relationships among humans by simultaneously identifying each person in the interacting group. We conducted a two-month experiment in an elementary school. As a result, the robot successfully continued friendly interaction with many children for the two months, and demonstrated reasonable performance in identifying friendships among children. We believe this ability to maintain friendly relationships with humans and to identify human relationships is essential to behaving socially.

An interactive exhibition guide robot has been developed, that is integrated with ubiquitous sensors. The robot utilizes its human-like body properties to perform human-like gestures such as eye-contact and pointing. The ubiquitous sensors are embedded in an exhibition room or attached to people in the room, and they provide information not only for the robot's interactive guide but also for the development of the robot. The results of a two-days exhibition experiment show that the robot attracted visitors and successfully guided them in the environment. Moreover, the obtained interaction statistics, including video from the ubiquitous sensors, prove to be useful for analyzing the interaction between the robot and visitors, and this implies that the use of such interaction data is a promising approach for developing interactive robots.

This paper explores memory-based approaches to the recognition of human behavior that relies on a database of previously categorized instances of sensory data. To overcome the curse of dimensionality, we examine two related methods that both rely on a hierarchical division of the sensory space using a decision tree. The first approach iteratively applies linear discriminant analysis to divide the sensory space in half in order to construct a binary tree for recognizing behaviors. We have verified the effectiveness of this approach for real-time behavior recognition using infrared sensors distributed in a desk environment and compared its results to those of Quinlan's C4.5. The second approach applies the well-known ID3 algorithm to the construction of a decision tree based on an information criterion. We use it to recognize browsing behavior at a video rental shop. Inferences are derived directly from the binarized pixel data of four wide-view cameras. Both systems offer behavior recognition rates in excess of 90%.

This paper presents an omnidirectional distributed vision system that learns to navigate a robot in an office-like environment without any knowledge about the calibration of the cameras or the robot control law. The system is composed of several omnidirectional vision agents (implemented with an omnidirectional camera and a computer). The first vision agent learns to control the robot with SARSA(/spl lambda/) reinforcement learning, using the LEM strategy to speed-up learning. Once the first vision agent learnt the correct policy, it transfers its knowledge to the other vision agents. The other vision agents might have different intrinsic and extrinsic camera parameters (that are unknown), so a certain amount of re-learning is needed. Reinforcement learning is well suited for this. In this paper, we present the structure of the learning system and the discussion about the optimal values for the learning parameters. During the experimentation the learning phase of the first agent has been carried out, then the knowledge propagation and the re-learning stage of three different agents have been tested. The experimental results demonstrate the feasibility of the approach and the possibility to port the system on the actual robot and cameras.

This paper reports progress in the development of a humanoid robot designed for real-time face-to-face interaction with humans. An essential component for face-to-face interaction with humans is being able to find faces, tracking them, and smoothly moving back and forth between gazing to a face and other objects of interest. In this paper we propose a system that integrates peripheral vision and foveal vision in a principled manner using particle filters. The developed system generates hypotheses about face position by using peripheral vision and verifies them by integrating peripheral vision and foveal vision. Even though a face may not be present in foveal vision, while the robot is gazing at another object, it keeps plausible hypotheses about the location of the human face with peripheral vision and restarts the face-following by verifying the hypothesis later. Moreover, this fundamental function provides more rich sensory information for human-robot interaction, such as a human's facial expression and lip motions during utterances.

We report the development and evaluation of a new interactive humanoid robot that communicates with humans and is designed to participate in human society as a partner. A human-like body will provide an abundance of nonverbal information and enable us to smoothly communicate with the robot. To achieve this, we developed a humanoid robot that autonomously interacts with humans by speaking and gesturing. Interaction achieved through a large number of interactive behaviors, which are developed by using a visualizing tool for understanding the developed complex system. Each interactive behavior is designed by using knowledge obtained through cognitive experiments and implemented by using situated recognition. The robot is used as a testbed for studying embodied communication. Our strategy is to analyze human-robot interaction in terms of body movements using a motion-capturing system that allows us to measure the body movements in detail. We performed experiments to compare the body movements with subjective evaluation based on a psychological method. The results reveal the importance of well-coordinated behaviors as well as the performance of the developed interactive behaviors and suggest a new analytical approach to human-robot interaction.

The research approaches utilizing ubiquitous sensors to support human activities have become of major interest lately. Sensor fusion is one of the fundamental issues to develop such intelligent environments. The sensor fusion in previous works is performed in the task-level layer through individual representations of the sensors. Therefore, it does not provide new information by fusing sensors. This paper proposes another method that fuses sensory signals based on mutual information maximization in the signal-level layer. The fused signal provides us new information that cannot be obtained from individual sensors. As an example, this paper also shows experimental results in an audio-visual fusion task.

In robotics, many path-planning algorithms have been proposed but almost all are evaluated only by their path lengths in PC. However, the evaluation is completely independent of a human ability. Recently, a human intends to live with many robots. For this reason, we should experimentally evaluate many kinds of interactions between a human and robots in our living space, which are supervised by each path-planning algorithm. Unfortunately, it is impossible, expensive and dangerous for us to investigate such interactions in a real world. To overcome such drawbacks, we propose wearable-based system to evaluate a path-planning algorithm by a human being. A human avoids some neighbors of virtual robots by using two eyes (using his visual property) and two legs (using his movable ability). In our wearable system, we can quickly develop smart robots and sensors without considering many irrational technical limitations. Therefore, we can test a novel sensor-based path-planning algorithm in the virtual environment. Using this, their superiorities are checked by visual and movable abilities of a human being.

This paper reports a trial of immersing interactive humanoid robots into a real human society, where they are charged with the communication task of foreign language education. For this purpose, we developed interactive humanoid robots equipped with wireless person identification technology for facilitating interaction with multiple persons. We believe that the ability to identify persons allows more meaningful social relationships between robots and humans to develop. This paper discusses the fundamental mechanisms of the multi-person identification ability, and how the robots established and sustained social relationships among children in an elementary school.

This paper proposes a concept of robots called ubiquitous intelligent robot and introduces our research efforts toward establishing its fundamental technology. A ubiquitous intelligent robot is a kind of embodied intelligence that encompasses the sphere of people's activities, records these activities, and supports people. Under this concept, we have developed an intelligent environment for capturing our experiences and a wearable event recording system for medical care. These developments mainly focus on recording people's activities along with their environments.

In this paper, we report about a new interaction-oriented robot, which communicates with humans and will participate in human society as our partner. For realizing such a robot, we have started a new collaborative work between cognitive science and robotics. In the way of robotics, we have developed a humanoid robot named "Robovie" that has enough physical expression ability. On the other hand, through cognitive experiments, we obtained important ideas about the robot's body property. To incorporate these ideas, we have developed software architecture and implemented autonomous interactive behaviors to the robot. Further, we have evaluated the robot's performance of the interactive behaviors through psychological experiments. The experiments reveal how humans recognize the robot.

There is a strong correlation between the number of appropriate behaviors an interactive robot can produce and its perceived intelligence. We propose a robot architecture for implementing a large number of behaviors and a visualizing tool for understanding the developed complex system. Behaviors are designed by using knowledge obtained through cognitive experiments and implemented by using situated recognition. By representing relationships between behaviors, episode rules help to guide the robot in communicating with people in a consistent manner. We have implemented over 100 behaviors and 800 episode rules in a humanoid robot. As a result, the robot could entice people to relate to it interpersonally. An Episode Editor is a tool to support the development of episode rules and to visualize the complex relationships among the behaviors. We consider the visualization is to be necessary for the constructive approach.

Mobile robots operating in indoor spaces, such as office, can benefit from a two-dimensional (2D) map. The map can be used for planning and executing operations such as navigation between different locations in an office. The map might serve as a visual aid to a security operator. This paper presents a method for constructing a 2D map by fusing local images from a special catadioptric sensor taken from several locations in the environment into a single, consistent image.

For realizing a robot working in human society, interaction with humans is the key issue. We have developed a robot that interacts with humans based on visual recognition. This robot has two vision systems: an omnidirectional vision system for acquiring necessary visual information and a binocular stereo vision system. The binocular vision system indicates what the robot is looking at and is not used for locomotion. Gaze control plays an important role in human-robot interaction. This paper reports how the robot's gaze influences subjects' impressions of the robot. With a statistically significant number of robot observers, we employed psychological methods, the semantic differential method (SD), and factor analysis. This careful psychological analysis of robot impressions is the first trial in robotics. Through the experiment, we have found that the impressions mainly consist of four factors: familiarity, enjoyment, activity, and performance. The computer skills of subjects affect their impressions of the robot.

In robot navigation, one of the important and fundamental issues is to find positions of landmarks or vision sensors located around the robot. This paper proposes a method for reconstructing qualitative positions of multiple vision sensors from qualitative information observed by the vision sensors, i.e., motion directions of moving objects. In order to directly acquire the qualitative positions of points, the method proposed in this paper iterates the following steps: 1) observing motion directions (left or right) of moving objects with the vision sensors, 2) classifying the vision sensors into spatially classified pairs based on the motion directions, 3) acquiring three point constraints, and 4) propagating the constraints. Compared with the previous methods, which reconstruct the environment structure from quantitative measurements and acquire qualitative representations by abstracting it, this paper focuses on how to acquire qualitative positions of landmarks from low-level, simple, and reliable information (that is, "qualitative"). The method has been evaluated with simulations and also verified with observation errors.

We propose a framework towards the integration of information sensors based on the idea that the stimulus perceived through different sensors are spatial-time correlated for a short time period. Applications in robotics need to be able to process information from multiple sensors, for instance, in the case of a visible talking person. How can we relate this kind of information in a simple way, without making use of high level representation? This is the question that we want to address. A new framework based on a correlation measure of low level data information is proposed. This low level correlation measure can be used as an integration data engine to support high level task description. In the paper a coherent approach from sensor level to task level for developing a robot which can handle a large number of sensors and actuators is developed. An example how this approach can be used for a visual-sound integration task is also presented.

In the multi-agent environment, it is important to identify position and orientation of multiple robots for accomplishing a given task in cooperative manner. This paper proposes a method for estimating position and orientation of multiple robots using multiple omnidirectional images based on geometrical constraints. Our method reconstruct not only the relative configuration between robots but also an absolute one using the knowledge of landmarks in the environment. Even if there are some obstacles in the environment, our method can estimate absolute configuration between robots based on the results of self-localization of each robot.

Proposes a robot architecture that enables us to progressively develop a robot. The architecture consisting of situated modules has merits of both the traditional function-based and behavior-based architectures in addition to the merit in the development. We have developed a robot based on the architecture. By reporting the development process, the paper discusses advantages of the proposed architecture.

Describes an infrastructural framework to support interactivity among entities that are not residing in the same physical environment. In such an environment, mobile robots supported by a perceptual information infrastructure gather information and communicate with people in a distant place instead of the users. This is one of the possible and promising applications of robots based on the current technologies of robotics, multimedia and computer networks. The paper describes the overall architecture and main system components. We discuss experimental studies where underlying concepts and algorithms are evaluated.

Development of multiple robot systems which solve complex and dynamic problems in parallel and distributed manners is one of the key issues in robotics research. The multiple robot systems require robust methods to identify robots for collaborative behaviors. This paper proposes a method using omnidirectional vision sensors for the identification between the robots. In addition to the several advantages of the omnidirectional vision sensor as a vision of a mobile robot, the omnidirectional vision sensor brings a significant benefit for realizing collaborative behaviors in multiple robot systems. After discussing on the algorithm, this paper shows several simulation results and real experimental results in a real environment.

Proposes an environmental representation, called a T-Net, a network of the path segments which reflects the structure of the environment. The robot iterates panoramic-sensing to find and estimate the skeletons of local areas, as well as their spatial relationships, through active navigation guided by selected targets. The robot represents the environment by a network of the skeletons which are used as the path segments for planning and navigating to a goal in real-time. Dynamic change often occurs in a real environment and the robot, thereby, detects such changes while navigating to a destination and updates the T-Net with alternative path segments if they are structural changes. Results of experiments using a robot with a real-time omnidirectional vision sensor are given.

The state space of a sensor-based robot in the most previous works has been determined based on human intuitions, however the state space constructed from human viewpoints is not always appropriate for the robot. The robot has a different body, sensors, and tasks, therefore, we consider the robot should have an original internal state space determined based on actions, sensors, and tasks. This paper proposes an approach to construct such a robot oriented state space by statistically analyzing the actions, sensor patterns, and rewards given, as results of task executions. In the state space construction, the robot creates sensor pattern classifiers called empirically obtained perceivers (EOPs) the combinations of which represents internal states of the robot. We have confirmed that the robot can construct original state spaces through its vision sensor and achieve navigation tasks with the obtained state spaces in a complicated simulated world.

This paper describes now an intelligent agent can locate itself from iconic memory of scenes. Omnidirectional images of the environment at a number of reference points are memorized as signs of the local areas. Similarity of the omnidirectional views at different locations in a small area decreases with their distance. The agent can thereby find the current local area by examining the similarities of the current view with those memorized already. Since the omnidirectional view are represented by their Fourier co-efficients, cost for memorizing and matching a large number of the views is inexpensive. When the agent explores a new environment, it wanders around, memorizes the views at different locations and organizes them into the iconic memory so as to reflect the environment geometry, which is effective to guide navigation. Results of experiments in an office cluttered with much furniture and objects indicate the agent can reliably locate itself in it from the memorized views.

This paper presents a dynamic method for improving the resolution in estimating edge positions. Each sharp edge is located precisely at a peak in the dynamic imagery taken by an image sensor which is swivelled with a very fine resolution. Experimental results show that the resolution is more than 10 times better than conventional methods using a single image. Its application to binocular stereo is also given.

This paper presents an environmental representation, called T-Net (Target Network), for robot navigation. This representation navigates a mobile robot to move in real time in an unstructured environment. The robot approximates local areas with straight paths and memorizes the paths with pairs of feature points. The T-Net consisting of the paths represents rigid environmental structure and keeps relations between the local areas. By tracking the feature points, the robot can move along the T-Net and investigate the environment in detail. The authors have applied this idea to their mobile robot which has multiple vision agents, and verified it through simulations and experimentations in a real world.

This paper presents a strategy of a vision-guided mobile robot for making models of an unknown environment by panoramic sensing. Before moving in the environment, the robot looks around and finds an outline structure of the local environment as a reference frame for building the models. Then the robot builds the models, which are qualitative, while moving along the local structure. We have implemented the robot behaviors into a mobile robot with multiple vision agents which simultaneously execute different vision tasks needed for the panoramic sensing.>

This paper presents a new concept, in robot visioii that we call sign pattern. Tlic sign patstern is visual pattern cliaracterizing tlie environnient. and resulting a stereotyped robot action. In other word, the sign pat,terii is a key wliicli activates t,lie robot reactive action. The visioiiguided mobile robot can iiiove on the road by detecting and following the sign patterns. Tlie sign patterns are used in tliree fiiiictioiis of the niobile robot: (1) Following. road boundaries, (2) Controlling t.he caniera posit.ion and (3) Approaching targcts. In this papcr. we describe t.he new concept of the sign pattern and image processing for detecting and tracking the sigii patterns.

Conventional active vision systcnis dircct at, trnti'oii serially to each feature in the environriierit by iiioviiig a iiingle or a couple of cameras as huniaii eye balls. This pitper preseiit,s an alternat$ivc: approach that iriiiltiple agents with t,heir own eyes accomplish indepeiident,ly thcir tasks by keep iiig their attentioii t,o difftmmt targets. Five important, issues, iniaging, stereopsis, shapc recovery. mot,ion control and niotion estimation, of the active vision arc discussed and the advantages of the niult,iplv foci of visual attention are given. A vision system with four ('aiiit:~~ riioviiig iiidependent*ly each othrr is described.

Omnidirectional views of an indoor environment at different locations are integrated into a global map. A single camera swiveling about the vertical axis takes consecutive images and arranges them into a panoramic representation, which provides rich information around the observation point: a precise omnidirectional view of the environment and coarse ranges to objects in it. Using the coarse map, the system autonomously plans consecutive observations at the intersections of lines connecting object points, where the directions of the imaging are estimated easily and precisely. From two panoramic views at the two planned locations, a modified binocular stereo method yields a more precise, but with direction-dependent uncertainties, local map. New observation points are selected to decrease the uncertainty, and another local map is yielded, which is then integrated into a more reliable global representation of the world with the adjacent local maps.>

The authors present two computationally inexpensive methods that rapidly and robustly determine both the translational and the rotational components of robot motion through the use of an active camera. Two cases are considered: the first where the instantaneous direction of motion is known, and the second when the instantaneous direction of motion is unknown. Based on the animate vision paradigm, gaze control is used to allow the robot's camera to fix on items on the environment, and then rapidly move on to other fixation points. The algorithms iteratively seek out fixation points that lie in the forward direction, converging so that the active camera is aligned along the translational direction of motion. At that point, the tracking motion of the camera is equal but opposite in sign to the robot's rotational component of motion. Experiments were carried out both in simulation and in the real world, giving results that are close to the actual motion parameters of the robot.>

A method is proposed for obtaining precise range information by swivelling a conventional TV camera, and the experimental verification of the method is described. To realize the method, the blurring that occurs in the image as a result of aberrations of the TV camera lens is used. The direction to the object is measured with a subpixel-level precision which depends on the angular resolution of the camera controller. The uncertainties of the object locations measured by conventional stereo methods depend on the resolution of the image sensor. It is shown that the method can be used to locate objects with a precision depending on the angular resolution of the camera controller, which is 0 to 20 times better than the conventional imaging method.>

A novel imaging method is presented for acquiring an omni-directional view with range information using a single camera. The difficult correspondence problem is solved by tracking each feature in the image sequence in the same manner as in epipolar plane image analysis. Although the equivalent camera distance is not long, the authors can obtain reliable estimates because of the high resolution of the panoramic views. The authors expect the resolution in locating sharp edges to be very fine, up to the resolution of camera rotation, 0.005 degree. A global map making procedure is also proposed which uses omni-directional views at different locations. Omni-directional binocular stereo is used to acquire a path-centered local map with direction-dependent uncertainty. By combining multiple local maps, it is possible to build a more reliable global map. Also explored is the possibility for using the panoramic stereo for map making by a robot.>

A monocular vision sensor on a mobile robot for finding and tracking a feature point in 3D space while it moves along a route is described. Visual feedback is used to fix the feature point, called the fixation point, at the image center. The precise information of the camera angular velocity is used for estimating 3D structure around the fixation point to build a local map. Mathematical equations for linear and circular robot motions are given. Experimental results show the validity of this method.>