Recent research has focused on developing non-task-oriented dialogue agents with high anthropomorphism, aiming to sustain users’ dialogue motivation over prolonged periods. Previous studies have introduced chatbots that deliberately incorporate short response delays to enhance anthropomorphism and user satisfaction. However, in the context of long-term casual conversations, the behavior of these chatbots, which reply within a few seconds, may lead to a perception of the chatbot as always on standby, lacking its own schedule or personal life, thus appearing less humanlike. To address this issue, we propose a non-real-time chatbot that interacts with users while introducing deliberate response delays on a larger time scale, ranging from a few minutes to several hours. A five-day dialogue experiment confirmed that the non-real-time chatbot was perceived as more humanlike and encouraged users to engage in extensive dialogue. These insights are invaluable for designing chatbots that can maintain casual conversations over prolonged periods.

Complaint sharing is a common communicative strategy in human–human interaction, known to foster intimacy and strengthen social bonds. Building on this insight, we proposed a robot that shares complaints with users to promote intimate relationship. Through an online experiment with 210 participants, we found that moderate complaint sharing fosters intimacy similar to not complaining, while also enhancing conversational engagement. In contrast, aggressive complaint sharing evokes discomfort, is perceived as ethically inappropriate, and fails to build intimacy. These findings highlight the potential and risks of complaint sharing, and provide valuable insights for designing robots that build intimate relationships with users.

Depression is a widespread mental health issue affecting diverse age groups, with notable prevalence among college students and the elderly. However, existing datasets and detection methods primarily focus on young adults, neglecting the broader age spectrum and individual differences that influence depression manifestation. Current approaches often establish a direct mapping between multimodal data and depression indicators, failing to capture the complexity and diversity of depression across individuals. This challenge includes two tracks based on age-specific subsets: Track 1 uses the MPDD-Elderly dataset for detecting depression in older adults, and Track 2 uses the MPDD-Young dataset for detecting depression in younger participants. The Multimodal Personality-aware Depression Detection (MPDD) Challenge aims to address this gap by incorporating multimodal data alongside individual difference factors. We provide a baseline model that fuses audio and video modalities with individual difference information to detect depression manifestations in diverse populations. This challenge aims to promote the development of more personalized and accurate de pression detection methods, advancing mental health research and fostering inclusive detection systems. More details are available on the official challenge website: https://hacilab.github.io/MPDDChallenge.github.io.

Emotional expressions serve as important communicative tools in human negotiations, and prior work has shown that artificial agents can use synthetic expressions to enhance negotiation outcomes and to train negotiation skills. These prior findings have focused on virtual agents and little is known about the effect of expressions when negotiating with physical robots. Therefore, in this study, we compared how participants negotiated with emotionally expressive virtual agents and android robots. Participants$(\mathrm{n}={82})$, as a proposer, played a nonverbal version of a four-issue ultimatum bargaining game with a counterpart who was either a virtual agent or an android robot. Before negotiating, participants observed their counterpart's emotional reactions to potential deals. The results showed that participants were better able to estimate the preferences of virtual counterparts compared with robotic counterpart, and thereby achieve better win-win solutions. We find this effect was mediated by uncanniness: participants found the emotional robot to be uncanny, and this undermined their ability to extract information from the robot's expressions. We discuss theoretical mplications for our understanding of human-robot negotiation and practical implications for the design of effective robot negotiators.

With the age of diversification, it’s important to nurture the qualities of children to enable them to create their future society. To achieve this, it’s crucial for children to broaden and deepen their views through dialogue with others. In the field of human-agent interaction, introducing robots into educational settings has shown positive educational effects. In this study, we developed an autonomous dialogue robot that delved into the children’s ideas and prompted them to think about aspects they had not yet considered, to broaden and deepen their viewpoints. We introduced this developed robot into a 6th-grade science class and conducted an experiment. The results based on subjective evaluations by students through questionnaires and objective evaluations of the students’ reaction papers by third parties, suggest that discussions with the robot can broaden and deepen the viewpoints of students. This study highlights the potential of robots to support the important educational goal.

Effective teleoperated avatar communication requires expressing social behaviors. Gaze behavior is one of the crucial social behaviors and includes reflexive reactions to the avatar’s surroundings and intentional responses to the operator’s speech and actions. Teleoperated avatars must have their gaze behavior controlled according to situational changes in both the avatar’s and operator’s contexts. However, it is not clear how to adjust the avatar’s gaze in response to changes in both situations. In this paper, we collect a dataset of gazing positions that the avatar is instructed to face, taking into account both avatar and operator situations, and annotation labels that represent both situations in detail. We then exploratorily analyze the ratio of gazing positions per situation through dynamic area-of-interest (AOI) analysis. Our analysis provides insights into determining the gaze behavior of teleoperated avatars.

The use of dialogue agents with a strategy that focuses beyond a static depth level and gradually deepens the conversation over time can nurture relationships with users. Thus, this study aimed to develop a dialogue agent capable of gradually deepening conversations over time and consequently examine its effectiveness. First, we constructed a topic depth estimation model capable of estimating the depth of any given topic. Subsequently, we developed a dialogue agent designed to gradually deepen conversation topics as interactions progress, tailored for operations on users’ smartphones. To evaluate the effectiveness of the developed agent, we conducted a 10-day dialogue experiment with 26 participants. The results indicated that the proposed agent elicited deep self-disclosure from users without significantly increasing psychological reluctance compared to the control condition. Moreover, the participants perceived a deeper relationship with the agent and were less likely to consider it as a non-human entity. These findings suggest that the proposed dialogue agent was capable of effectively engaging in deeper conversations over time, thereby enhancing human–agent interactions.

Conversational recommendation systems are crucial for making recommendations agreeable to the user. To reach an agreeable recommendation, this study proposes a dialog strategy that represents a reasonable order of items and elicits the current estimation by the wording of utterances based on the subjective preference estimation. We developed two dialog functions for the topic and word choice based on the history of preference estimations. The human impression of a robot’s diligence, understanding capability, and satisfaction were evaluated through a conversation with a virtual robot using a crowdsourcing platform. We compared six conditions that differed based on two topics and three wording patterns. The experimental results indicated that the main effect of the wording patterns, whereas one of the topic choices was not found to be significant. Further analysis showed that accurate estimation improves the robot’s impression when demonstrating its diligence.

Mobile robots are increasingly being utilized as guides in various settings. Ensuring efficient and hospitable interactions between robots and individuals is crucial for their successful implementation. However, there is a lack of understanding regarding the specific robot behaviors that can effectively guide individuals to their destinations while providing a sense of hospitality. In this study, we investigate two robot behaviors: “vertical oscillation motion” and “looking back motion,” with the objective of identifying behaviors that can optimize efficiency and hospitality in a guide robot. The vertical oscillation motion replicates the natural up-and-down movement of the human body during walking, while the looking back motion establishes eye contact and conveys a sense of hospitality. To evaluate the effectiveness of these behaviors, we conducted experiments with actual visitors at a shopping mall. Participants were exposed to the robot's behaviors and their responses were analyzed through questionnaires and interviews. The results revealed that the looking back motion successfully conveyed the robot's hospitality, while the vertical oscillation motion instilled a sense of urgency in individuals. However, it was found that only the looking back motion effectively motivated individuals to promptly commence walking towards the robot. These findings highlight the significance of incorporating specific robot behaviors to enhance both efficiency and hospitality in guiding individuals to their destinations. By understanding the impact of these behaviors, we can develop guide robots that accommodate a larger number of individuals while increasing their satisfaction. Future research endeavors should focus on evaluating the effectiveness of these behaviors in real-world environments, ensuring their practical applicability.

With recent technological developments, such as machine learning, data collection has become increasingly important. However, due to privacy issues, there is an ethical problem in collecting data on everyday activities. In this study, we aimed to develop a mechanism for acquiring consent interactively by using a robot and, as a first step, focus on a dialogue strategy in which users feel safe to provide their data. Specifically, we investigated the effects of reminder and rationale provided by the robot in situations where consent is obtained. We conducted two experiments in which users conversed with a virtual robot agent in a crowd setting. We found that the robot reminder improved the degree of understanding regarding the use of data. In addition, the permission rate was improved by robot notifications. However, no effect of rationale was observed. These results contribute to the discussion on the ethical aspects of data collection by robots.

Service robots have increasingly been investigated in retailing. Previous studies mainly focused on the effectiveness of recommendation with regard to a single robot, and whether and how the use of two robots combined can achieve better performance remain unclear. In this study, we address this by exploring the combination power of two service robots for product recommendation in a bakery. We placed one robot inside the store for product recommendation and the other robot outside to promote the inside robot. Particularly, we are interested in the effects of the outside robot on the inside robot's performance in product recommendation. Our results indicate that using the outside robot to promote the inside robot achieved more purchases over using the inside robot alone. Particularly, we discovered that the outside robot increased the attention of customers toward the inside robot; hence, more customers checked and purchased the products. Based on the findings, we discuss the important points for the effective use of service robots.

Emotional voice conversion (VC) aims to convert a neutral voice to an emotional one while retaining the linguistic information and speaker identity. We note that the decoupling of emotional features from other speech information (such as content, speaker identity, etc.) is the key to achieving promising performance. Some recent attempts of speech representation decoupling on the neutral speech cannot work well on the emotional speech, due to the more complex entanglement of acoustic properties in the latter. To address this problem, here we propose a novel Source-Filter-based Emotional VC model (SFEVC) to achieve proper filtering of speaker-independent emotion cues from both the timbre and pitch features. Our SFEVC model consists of multi-channel encoders, emotion separate encoders, pre-trained speaker-dependent encoders, and the corresponding decoder. Note that all encoder modules adopt a designed information bottleneck auto-encoder. Additionally, to further improve the conversion quality for various emotions, a novel training strategy based on the 2D Valence-Arousal (VA) space is proposed. Experimental results show that the proposed SFEVC along with a VA training strategy outperforms all baselines and achieves the state-of-the-art performance in speaker-independent emotional VC with nonparallel data.

Research on self-recommending product agents is under way. Compared with product promotion by humanoid robot agents, self-recommending agents can call a passing cus-omer's attention to product. As the customer's interest is focused on the product, the self-recommending agent can then give instructions, such as “pick me up.” It has been found that customers tend to follow such instructions, and touching the product is known to effectively support sales promotion. From two experiments in a convenience store, a self-recommending agent in this study was successful in attracting customer interest, which included handling the products. However, we also found that, after being picked up, the products' monologue resulted in customers leaving the product behind on the shelf. Herein, we examine the reasons why.

In this study, we explore the effective form of social presence for a tele-operated robot to provide customer service. Particularly, we address the question on if and how a tele-operated robot displays the presence of an operator, and what effects it would have on people's perception and behavior toward it. We launched a tele-operated robot in a supermarket and had it deliver recipe flyers. We adjusted the robot's social presence by showing or not showing the photo of an operator's face (F) and using or not using voice conversion (V), leading to three forms of presence: Wizard of Oz (F: no, V: yes), costume (F: yes, V: yes), and telepresence (F: yes, V: no), which indicated the operator's presence from a low to a high level. We determined that the customers behaved significantly different when they faced the tele-operated robot in different forms. Our robot that exhibited a moderate presence of the operator (costume form) achieved the overall best performance. Based on these findings, we discuss both the strengths and weaknesses of the three forms of presence for a tele-operated robot and recommend the appropriate form for various applications.

Customer service with teleoperated robots is suscep-tible to the same problems related to stress as is emotional labor in general, such as for in-person customer service representatives. In this study, we aimed to reduce that stress by constructing a buddy-like rapport between the robot, which is the target of teleoperation, and its operator. For this purpose, we designed an empathetic interaction between the robot and the operator and conducted a customer service experiment to verify its effectiveness. Our results demonstrate that the proposed interaction can build rapport between the robot and the operator and the operator can feel more reassured. Although the effect on stress could not be isolated directly from the data, detailed analyses of the response to the questionnaires indicated that the proposed interaction may be useful to relieve stress.

For a humanoid robot to make eye contact and initiate communication with a person, it is necessary to estimate the person's head position. However, eye contact becomes difficult due to the mechanical delay of the robot when the person is moving. Owing to these issues, it is important to conduct a head-position prediction to mitigate the effect of the delay in the robot motion. Based on the fact that humans turn their heads before changing direction while walking, we hypothesized that the accuracy of three-dimensional (3D) head-position prediction from a first-person view can be improved by considering the head pose. We compared our method with a conventional Kalman filter-based approach, and found our method to be more accurate. The experiment results show that considering the head pose helps improve the accuracy of 3D head-position prediction.

In this paper, we report on a field study in which we employed two service robots in a bakery store as a sales promotion. Previous studies have explored public applications of service robots public such as shopping malls. However, more evidence is needed that service robots can contribute to sales in real stores. Moreover, the behaviors of customers and service robots in the context of sales promotions have not been examined well. Hence, the types of robot behavior that can be considered effective and the customers' responses to these robots remain unclear. To address these issues, we installed two tele-operated service robots in a bakery store for nearly 2 weeks, one at the entrance as a greeter and the other one inside the store to recommend products. The results show a dramatic increase in sales during the days when the robots were applied. Furthermore, we annotated the video recordings of both the robots' and customers' behavior. We found that although the robot placed at the entrance successfully attracted the interest of the passersby, no apparent increase in the number of customers visiting the store was observed. However, we confirmed that the recommendations of the robot operating inside the store did have a positive impact. We discuss our findings in detail and provide both theoretical and practical recommendations for future research and applications.

Social robotics recommendations have been studied for a long time, and many existing studies have addressed in-store recommendations using robots. However, it has been pointed out that many studies conducted in "in-the-wild" field environments have only focused on the initial stages of customer purchase behavior, such as stopping and engaging in a conversation, and few have been able to induce an interest in the product and even purchase. One of the causes is that the robot itself attracts most of the customer’s attention, making it difficult for customers to be interested in the robot’s recommendations. To solve this problem, this study examines the inclusion of clear and specific instructions to customers in interactions in which the robot recommends services and products. We conducted a field experiment to confirm the effectiveness of such instructive interactions, and found that customers are more likely to be interested in the content recommended by the robot, rather than in the robot itself, through the instructive interaction.

Robot agents are increasingly used for user services, and society is becoming increasingly familiar with such robots. Most robots are of the humanoid type, which are easy to recognize as interaction partners. By interacting with a user, these agents may be able to generate user interest in a product and successfully sell it. However, according to previous studies, it has been suggested that users who are interested in the movement and appearance of the agent may focus only on these aspects and not listen to the recommendation. Therefore,we hypothesized that by making the product the agent, which we call a Self Recommendation Agent(SRA), the attention of the user would be focused on the product itself. Hence, if a user is interested in the agent, it is the same as paying attention to the product. Therefore, we expect that such an agent will be able to gain more attention from users than conventional agents while providing recommendations. To investigate the effectiveness of this agent, we set up a store in a shopping mall and conducted sales experiments. In this field study, we conducted an experiment to compare sales between a SRA and a robot agent.As a result, the SRA was able to make recommendations to many more users than the robot agent, and the users who received recommendations from the SRA remembered more about the recommended product than those who received recommendations from the robot agent.Based on these results, we confirm the possibility that the SRA is effective for advertising.

In recent years, web conferencing systems have become widely used in non-face-to-face interactions. However, web conferencing systems make difficult to establish eye contact for users and can lead to decrease in non-verbal communication. In this paper, we propose a method that will enhance a user’s feeling of interlocutor’s attention, even when the line of sight does not actually match, by displaying a thought bubble for the interlocutor filled with the user’s in-camera image. Because robots have the same gaze problem as humans we thought it is important to know that this web conference system can represent the attention of a simple moving robot without using complex gaze mechanisms. So, we conducted a within-subject design experiment using a robot whose gaze and movements can be controlled and a participant as interlocutors in a web conference with and without the thought bubble. The experimental results indicate that our proposed method increased the sense of being looked at.

More and more robots are providing services that involve conversations with people in daily life. Among them, the movement and positioning of the robot are effective factors to improve the robot’s expression, and the distance from the human and the direction of the robot are reported as important parameters. In multi-person dialogues, it is expected that robots can communicate their impressions, emotions, and thoughts to other robots by successfully adjusting their movement and positioning. We investigated the effect of the robot’s movement in a conversation on the subsequent conversation and the expression of the moving robot. It was found that there were intentions and emotions of the robot that could be expressed by moving forward and backward, respectively. The next step of this experiment is to conduct interaction between subjects using a system that incorporates forward and backward movements.

In this study, to mitigate cyberbullying in communication tools, we propose a method in which a chatbot pretending to be a member of a chat group defends the victim by proxy without permission (i.e., hijacking and proxy persuasion). An experiment was conducted to verify the effectiveness of the proposed method, the results of which showed that it increases the defending behavior of a user who was temporarily hijacked by the chatbot. However, compared with the existing method, the proposed method was found to have a problem with the continued use of the system.

Social robots are used in daily life. One of the applications of social robots is as recommendation systems. Previous research has mainly investigated how persuasive recommendations can be improved by focusing on the non-verbal/verbal behavior of robots. However, to use robots as recommendation systems every day, it is extremely important to examine the persistence of repeated persuasion over a long term, rather than the effect of one-time persuasion. Therefore, the objective of this study was to investigate the persistence of repeated persuasive of robots. For this purpose, robots with three types of behavior (Expert Behavior, Local Behavior, and Growth Behavior) recommended nutrition bars in a situation of daily consumption behavior for two weeks. We could confirm significant differences in the persistent persuasiveness in each behavior. The results suggested that the combination of value co-creation using local information and meta-trust expression had a significant impact on the persistence of the repeated persuasiveness of the robots in the longitudinal period. However, the acceptance of the recommendation robot system decreased due to the increase in the amount of information during recommendation; therefore, a new recommend system to solve this problem is desired.

In this study, we aimed to verify whether expanding a robot’s embodiment influenced the ease of inducing behavioral changes in passersby. We conducted a field experiment to call out to passersby for disinfection in a real shop using robots with various embodiments. As a result, the expansion of the robot embodiment attracted extensive attention of passersby, and then induced behavioral changes to the desired action, such as stopping in front of the robot and disinfecting their hands. Furthermore, the results revealed that in order to induce large behavioral changes, not only the presence of the robot’s body but also its movements with strong embodiment were necessary. These results indicated that expanding the robot’s embodiment was an important factor in inducing behavioral changes in passersby.

People with response time (RT) management issues usually have high situational communication apprehension, and fear of negative evaluation (FNE) in face-to-face (FtF) and online interactions. Decreasing the stress related to RT management can reduce situational communication apprehension and FNE, ensuring communication successful. In this study, we propose an interactive response strategy involving a robot avatar in a video conference that can reduce the stress related to RT management in a person and making communication successful. Two types of robotic video conferencing systems (VCSs) were considered: 1) a conventional robotic video conferencing system (VCS) where interactive response of robot avatar does not change concerning variation in RT of the subject and 2) a robotic VCS where interactive response of robot avatar changes concerning variation in RT of the subject. Situational communication apprehension measure (SCAM), FNE, sense of being attending (SoBA), and intention to use (ITU) are used as indexes for noticing the decrease in stress related to RT management of a subject in a communication scenario. A total of 51 subjects (M = 33.20, SD = 7.58 years) participated in the subjective evaluation of the web-based survey. The proposed interactive response of a robot avatar was found to have a significant effect on the stress related to RT management in a person. A significant decrease in expected SCAM and FNE scores of subjects was observed. Furthermore, a significant increase in expected SoBA and ITU scores of subjects was also observed. This study contributes to the literature in terms of the impact of the proposed interactive response of the robotic avatar on the stress related to RT management in a person in the video conference, assessed by SCAM, FNE, SoBA, and ITU indexes.

To explore the interaction design of an autonomous social robot stationed in a hotel room, we conducted a Wizard of Oz study. We developed a teleoperated robotic system that appears to move autonomously through voice-to-synthesis processing. Comparing the evaluation of the latest autonomous case with one of these teleoperated cases, the results show that it is possible to construct a robotic system that is more highly rated in terms of warmth, competence, and enjoyment of conversation. The results also suggest novel forms of the hotel room robot’s interactive services, such as a hotel-life management service and conversation partner service as a role of a listening presence, which draws out and understands with the guests’ talk.

Under the circumstances that social robots are increasingly being developed and studied in service encounters at public spaces, are they introduced into residential environments (i.e., private space)? This study hypothesizes that a personal assistant device in residential environments should wear human-like appearance to engage in service as conversation partner. We implemented the interaction design that provides regular services as the current personal assistant and additional service as conversation partner, and then conducted a field experiment where the participants stayed in the hotel rooms with a smart speaker or a social robot. The results support the hypothesis in that of conversation amount and emotional experience by conversation. The results also suggest the possibility of commercial service, namely conversational advertisement through social robots.

Deep discussions on topics without definite answers are important for society, but they are also challenging to facilitate. Recently, advances in the technology of using robots to facilitate discussions have been made. In this study, we developed a multiple-robot mediated discussion system (m-RMDS) to support discussions by having multiple robots assert their own points and lead a dialogue in a group of human participants. The robots involved the participants in a discussion through asking them for advice. We implemented the m-RMDS in discussions on difficult topics with no clear answers. A within-subject experiment with 16 groups (N=64) was conducted to evaluate the contribution of the m-RMDS. The participants completed a questionnaire about their discussion skills and their self-confidence. Then, they participated in two discussions, one facilitated by the m-RMDS and one that was unfacilitated. They evaluated and compared both experiences across multiple aspects. The participants with low confidence in conducting a discussion evaluated the discussion with m-RMDS as easier to move forward than the discussion without m-RMDS. Furthermore, they reported that they heard more of others' frank opinions during the facilitated discussion than during the unfacilitated one. In addition, regardless of their confidence level, the participants tended to respond that they would like to use the system again. We also review necessary improvements to the system and suggest future applications.

In recent years, a substantial amount of research has been aimed at realizing a social robot that can maintain long-term user interest. One approach is using a dialogue strategy in which the robot makes a remark based on previous dialogues with users. However, privacy problems may occur owing to private information of the user being mentioned. We propose a novel dialogue strategy whereby a robot mentions another robot in the form of gossiping. This dialogue strategy can improve the sense of conversation, which results in increased interest while avoiding the privacy issue. We examined our proposal by conducting a conversation experiment evaluated by subject impressions. The results demonstrated that the proposed method could help the robot to obtain higher evaluations. In particular, the perceived mind was improved in the Likert scale evaluation, whereas the robot empathy and intention to use were improved in the binary comparison evaluation. Our dialogue strategy may contribute to understanding the factors regarding the sense of conversation, thereby adding value to the field of human-robot interaction.

Emotional labor in text-based chat environments will be widespread in the near future. It is difficult for novice workers to make smooth conversation with an angry customer in a customer-support work- ing environment, one of the stressful emotional labors. In this pa- per, we propose a method to suppress the customer's anger based on a communication agent introduced into customer-support work in order to facilitate smooth communication. The agent first estab- lishes a positive relationship with the customer, and then the atti- tude changes to be sympathetic to the customer-support worker in order for the customer to follow sympathy and reduce anger. From the experimental evaluation, it is found that there is the possibil- ity of suppressing the participants' anger by this method, but the anger itself is at a higher level than in the common environment without the agent.

While more and more robots have been employed in the service industry, the impact of the human--robot social interaction on a heartwarming interaction service mostly remains an open question. For the purpose of exploring the possibility that a humanoid robot engages in a heartwarming interaction service, we developed herein the first prototype of the service system through a humanoid robot and conducted a field study, where a humanoid robot engages in the service for a customer at a hotel to collect a customer's impression of the service by questionnaire. The results demonstrate a humanoid robot's potential of engaging in a heartwarming interaction service that enhances customer satisfaction of the whole service. An exploratory analysis suggests the differences of the impact in sex and long-term interaction. Finally, we discuss the possibility of another application, namely, effective advertisement through a heartwarming interaction with a humanoid robot.

Hiroshi Ishiguro gave a lecture to a group of young students. We employed CommU, the desktop social robot, to manage the questions and answers for the talk. We encouraged the students to ask questions anytime. Half of the classroom was told to ask questions by raising their hand while the other half was shown an online messaging system developed for CommU, which allows the audience to post questions, which the robot would directly say. We had a gatekeeper to monitor for invalid sentences. In the middle of the presentation we asked the students to shift roles. The robot used a neural network based estimator of interruptibility to find the best time to speak. We did not expect too many questions, but the audience really embraced using the robot. They posted 44 questions to the presenter through CommU. On the other hand they asked 8 direct questions by raising their hands and standing up. Students thought that they gained more information from the lecturer using the robot than using the conventional method. In this instance we didnt stop the students from asking too many questions, but in a real-world application the gatekeeper will have to play an important role.

Authority and obedience are key regulatory elements in a society. Robots are becoming important part of our world, and are starting to interact in domains in which authority is an important aspect, as healthcare, teaching or law enforcement. Yet, there is little research on how people behave when robots show authority. In particular, although extensive investigations have been carried out on how authority can circumvent people's morality with experiments such as Milgram's or Stanford Prison, almost no research evaluated the effect of robots pushing the limits of people's own morality. This experiment tries to study this aspect by using a robot (geminoid) with the appearance, and thus authority of a famous person, and by pushing the boundaries asking morally controversial requests. The results show that, even though most people hesitate and recognize the requests as socially inappropriate, they obey to robots with authority. This suggests that the authority of the robot can push people to perform tasks usually considered as inappropriate.

In recent decades and years classroom teaching has been influenced by the appearance of digital devices and the pervasiveness of the internet. In recent years robots started appearing in learning environments. Even though classroom presentations improved considerably, interaction between students and teachers has not improved greatly. We propose a new way of using a desktop robot and an online voting/messaging system to make the classroom experience more interactive. Concretely, in this paper we report on a lecture which was assisted by a robot. The students posted questions and opinions on an online messaging platform. The messages were then sent to the robot to be posed to the teacher, either directly or through a voting procedure. Students found this new way of interaction very efficient and useful for improving their communication with the teacher. At the same time the number of interactions should be controlled so it does not interfere with the teaching process.

Research on conversational dialogue systems has attracted attention for achieving dialogue systems that can build social relationships with users. Although such systems are required for continuing long conversations with users to build relationships, they sometimes make sentences that are not related to the dialogue context, causing the dialogue to easily break down. The cause of the difficulty is simple. Unlike task-oriented dialogues, unexpected user utterances whose meaning the system cannot capture are frequently spoken in conversational dialogues, since the dialogue domain is much less restricted and the dialogue goal is less obvious. In this paper, we propose a novel strategy for dialogue systems through which two robots coordinate to create long conversations by avoiding dialogue breakdowns. If one of the two robots accepts user utterances with backchannel, the responsibility of responding to the user utterances is resolved and the other robot can change the dialogue topic, which decreases the risk of generating discontinuous utterances. Even if a dialogue nearly breaks down, such multiple robots can present predefined natural interactions among themselves to repair it. Our experiments show that the inter-robot interaction effectively improves the establishment of conversational dialogue that helps users continue the dialogue.

To provide a better discussion experience in current argumentative dialogue systems, it is necessary for the user to feel motivated to participate, even if the system already responds appropriately.In this paper, we propose a method that can smoothly introduce argumentative dialogue by inserting an initial discourse, consisting of question-answer pairs concerning personality.The system can induce interest of the users prior to agreement or disagreement during the main discourse.By disclosing their interests, the users will feel familiarity and motivation to further engage in the argumentative dialogue and understand the system's intent.To verify the effectiveness of a questionanswer dialogue inserted before the argument, a subjective experiment was conducted using a text chat interface.The results suggest that inserting the questionanswer dialogue enhances familiarity and naturalness.Notably, the results suggest that women more than men regard the dialogue as more natural and the argument as deepened, following an exchange concerning personality.

For a safe, natural and effective human-robot social interaction, it is essential to develop a system that allows a robot to demonstrate the perceivable responsive behaviors to complex human behaviors. We introduce the Multimodal Deep Attention Recurrent Q-Network using which the robot exhibits human-like social interaction skills after 14 days of interacting with people in an uncontrolled real world. Each and every day during the 14 days, the system gathered robot interaction experiences with people through a hit-and-trial method and then trained the MDARQN on these experiences using end-to-end reinforcement learning approach. The results of interaction based learning indicate that the robot has learned to respond to complex human behaviors in a perceivable and socially acceptable manner.

In this study, ease of talking is compared in three conversation forms: face-to-face conversation (direct conversation) and conversation with a robot that is tele-operated from the same room and different room (semi-indirect and indirect conversation, respectively). As a result, it was found that the silent time, which indicates how long the participants make an interlocutor wait until a response is given, in both semi-indirect and indirect conversation exceeds that in direct conversation. Additionally, the silent time in direct conversation after semi-indirect conversation exceeds that after direct conversation. The results indicate that a robot tele-operated from different rooms as well as from the same room can provide partners with time to think of the words uttered by them. Moreover, the results indicate that these effects are sustained in direct conversation following a conversation with a robot that is operated from the same room. These results indicate the potential of constructing a system of conversation via a robot that can be used in easier counseling for clients who experience difficulties in directly communicating with a counselor.

Our research goal was to develop a robot counseling system. It is important for a counselor to promote self-disclosure of clients to reduce their anxiety feelings. However, when a counselor is human, clients sometimes hesitate to disclose intrusive topics due to embarrassment and self-esteem issues. We hypothesized that a robot counselor, on account of its unique kind of agency, could remove mental barriers between the counselor and the client, and promote in-depth self-disclosure about negative topics. In this study, we prepared two robots (an android and a desktop robot) as robot counselors. First, we confirmed that subjects eagerly self-disclosed to these prepared robots from the numbers of spoken words about self-disclosure in preliminary experiment. And next, we conducted the experiment to verify whether it is possible to expose more of subjects' weakness to robots than humans. The experimental result suggested that robots can draw out subjects' self-disclosure about negative topics than the human counselor.

Since a talking robot cannot escape from errors in recognizing user's speech in daily environment, its verbal responses are sometimes felt as incoherent with the context of conversation. This paper presents a solution to this problem that generates a social context where a user is guided to find coherency of the robot's utterances, even though its response is produced according to incorrect recognition of user's speech. We designed a novel turn-taking pattern in which two robots behave according to a pre-scheduled scenario to generate such a social context. Two experiments proved that participants who talked to two robots using that turn-taking pattern felt robot's responses to be more coherent than those who talked to one robot not using it; therefore, our proposed turn-taking pattern generated a social context for user's flexible interpretation of robot's responses. This result implies a potential of a multiple robots approach for improving the quality of human-robot conversation.

We conducted a pilot experiment to examine the potential of a communication robot for supporting verbal interaction among adolescents with autism spectrum disorders (ASD) and their caregivers. Three teenagers with ASD indirectly conversed with their teacher via a desktop humanoid robot for around 40 min on average per day. In this situation, they showed some positive tendencies such as non-echolalic responses and talk about problems related to human relationships; they had not shown such tendencies before when they directly talked with their teacher in their daily life. During the trials, they also showed changes in their attitudes toward their teachers and friends; this implies that individuals with ASD could use their experience of communication with a robot for communication with a human. A quantitative analysis of the utterances in the trials suggested that reduced utterances from a caregiver might positively influence individuals with ASD toward sharing more information.

It is expected that the social presence of a distant person can be well conveyed in telecommunication using a robot as a proxy of that person. This paper proposes a novel form of robotic telecommunication, which consists not only of a proxy robot but also an additional bystander robot that autonomously responds to the interlocutor in front of them. The second robot is expected to complement the responses of the proxy robot so that the interlocutor feels stronger social telepresence of the distant person. A psychological experiment using two small humanoid robots as proxy and bystander reveals a positive effect on conveying a stronger feeling of social telepresence of the distant person.

Due to human sensitivity for humanlike objects, designing human-like behavior for a robot that interacts with humans remains a central issue in the HRI field. This issue is formidable because human-likeness is an important factor for better interaction since it can easily convey negative impressions if the design is not perfect, as in the uncanny valley phenomena. This paper addresses this issue with a novel approach that utilizes implicit know-how for performing on stage and is dedicated to represent human beings. Contemporary colloquial theatre theory (CCTT), which is a method of directing plays, is appropriate for this purpose since its reality-oriented instructions are directly applicable for improving robot behavior. In this paper, we report case studies on performing a play involving both humans and a robot. The play in our study was evaluated in public performances in Japan. We report detailed analysis of comparable short plays on human-robot interaction or human-human-robot interaction. Our analysis implies that utterance and motion timings should be tuned depending on the situation. Future work will use a motion capture system to get more precise data and more useful knowledge.

How humans recognize others is a central topic in both psychology and the field of robot communication. In this paper, we address the issue of such social cognition in a triadic situation since it has not yet been well explored in psychological studies although it is not negligible in daily situations. Classical Heider's balance theory has been suggested in that there are balanced states of triadic relationships that are perceived as comfortable by persons who are involved in them. However, whether, or how, such balanced states tend to be formed has not been well explored. In this paper, we investigate what we call 'social balancing effect' of synchronized nonverbal behavior: social relationships can be attracted to a balanced state due to the synchronized behavior between two agents. In this paper, we conduct an experiment in a conversational interaction among two humans and an android and report that observing eye contacts between the robot and a confederate affected the subject's recognition of the android as attracted to a balanced state.

A spiral response-cascade hypothesis is proposed to model the mechanism that enables human communication to emerge or be maintained among agents. In this hypothesis, we propose the existence of three cascades each of which indicates intrapersonal or interpersonal mutual facilitation in the formation of someone's feelings about one's communication partners and the exhibition of behaviors in communicating with them, i.e., responding. In this paper, we discuss our examination of an important part of the hypothesis, i.e., what we call an intrapersonal responding cascade, through an experiment where the gaze interactions between a participant and a communication robot were controlled not only by controlling the robot's gaze but also by signaling participants when to shift their gaze. We report that the participants' experiences in responding to the robot enable them to regard the robot as a communicative being, which partially supports the hypothesis of the intrapersonal responding cascade.

Due to differences in body structure between robots and humans, it is a formidable task for robots to show behaviors that correspond to human behaviors. As a simple case of this correspondence problem, this paper presents a robot that learns to vocalize vowels through interaction with its caregiver. Inspired by the findings in developmental psychology, we focus on the role of maternal imitation (i.e., imitation of a robot voice by a caregiver), which could play a role in guiding the correspondence of sounds. Furthermore, we suppose that it causes unconscious anchoring in which the imitated voice by the caregiver is approaching to one of his/her own vowels without his/her intension, and thereby works for guiding robot's utterances to be more vowel-like. We propose a method for vowel learning with an imitative caregiver under the assumption that the robot knows the desired categories of caregiver's vowels and the rough estimate of mapping between the region of sounds that the caregiver can generate and the region that the robot can generate. Through experiments with a Japanese imitative caregiver, we show that a robot succeeds in acquiring more vowel-like utterances than would be possible without such a caregiver, even when the robot is provided different mapping functions.

In face-to-face communication, eyes play a central role, for example in directing attention and regulating turn-taking. For this reason, it has been a central topic in several fields of interaction study. Although many psychology findings have encouraged previous work in both human-computer and human-robot interaction studies, so far there have been few explorations on how to move the agent's eye, including when to move it, for communication. Therefore, it is this topic we address in this study. The impression a person forms from an interaction is strongly influenced by the degree to which their partner's gaze direction correlates with their own. In this paper, we propose methods of controlling a robot's gaze responsively to its partner's gaze and confirm the effect of this on the feeling of being looked at, which is considered to be the basis of conveying impressions using gaze in face-to-face interaction experiments. Furthermore, an additional preliminary experiment with an on-screen agent shows the possibility of using blinking behaviour as another modality for responding to a partner

In face-to-face communication, eyes play a central role, for example, in directing attention and regulating turn-taking. For this reason, the eyes have been a central topic in several fields of interaction study. Although many psychological findings have encouraged previous work in both human-computer and human-robot interaction studies, so far there have been few explorations on how to move the agent's eye, including when to move it, for communication. Therefore, we investigate this topic in this study. The impression a person forms from an interaction is strongly influenced by the degree to which their partner's gaze direction correlates with their own. In this paper, we evaluate two primitive ways of controlling a robot's gaze responsively to its partner's gaze by comparison with two additional ways of non-responsive gaze control in an experiment set in a conversational situation. Consequently, we confirm that robots with different types of responsive gaze control can give subjects different impressions of its characteristics

Joint attention is supposed to be a basis of the competence of communication with others. The authors have been attacking the issue how to learn joint attention only through interactions with caregivers, in other words, without external task evaluation from a viewpoint of a constructivist approach towards both establishing a design principle of communicative robots and understanding the developmental process of human communication. This paper presents a method for quick learning of joint attention with unfamiliar persons based on a hybrid architecture that consists of (1) a quick but person-dependent learning module and (2) a slow but person-independent learning module. Experimental results show the effectiveness of the proposed method.

Finding the body in uninterpreted sensory data is one of the fundamental competences to construct the body representation that influences on adaptabilities of the robot to the changes in the environment and the robot body. The invariance of sensations in self-observation seems a promising key information to find the body. However, since each sensory attribute can be invariant only in the observation of a part of the body, the robot should complementarily utilize the invariance of the multiple sensory attributes. In this paper, we propose a method of body-nonbody discrimination by complementarily utilizing multiple sensory attributes based on a conjecture about the distribution of the variance of sensations for each observing posture, where it can be approximated by a mixture of two Gaussian distributions, which are for observing the body and the nonbody, respectively. By estimating the distribution, the robot can automatically find a discrimination hyperplane to judge whether it observes its body in the current observing posture. Simple experiments show the validity of the proposed method.

Without any explicit knowledge, human infants acquire the phonemes of adults who have different articulation parameters. By building a robot that reproduces this developmental process, we expect to model as yet unknown cognitive developmental processes and to discover fundamental design principles for machines capable of vocal communication. This paper proposes a model for acquiring Japanese vowels based on observations of imitation between a mother and an infant. We tested the validity of the proposed model by examining whether a real robot can acquire vowels through interactions with a caregiver.

Inspired by the observation that infants acquire phonemes common to adults without having the capability to articulate, nor having a priori knowledge about the relationship between the sensorimotor system and phonemes, a constructivist approach to building a robot that reproduces a similar developmental process is conducted. Two general issues are addressed: what are the interactive mechanisms involved and what should be the behaviour of the caregiver/teacher? Based on findings in developmental psychology, it is conjectured that: (a) the caregiver's vocalization in response to infants' cooing reinforces the infant's articulation along the caregiver's phonemic categories; and (b) the caregiver's repetition with adult phonemes helps to specify the correspondence between cooing and the caregiver's phonemes as well as determining the acoustic properties of the phonemes. The robot consists of an artificial articulatory system with a five-degrees of freedom mechanical system deforming a silicon vocal tract connected to an artificial larynx, an extractor of formants, and a learning mechanism with self-organizing auditory and articulatory layers. Starting off with random vocalizations, the system uses the caregiver's repetitive utterances to bootstrap its learning. In order to resolve the arbitrariness in determining proper articulations, the torque to deform the tract and its resultant deformation are minimized. The experimental results, discussion and future issues are given.

In our previous work (2000), we have developed a method for visual imitation by recovering the demonstrator's view based on the stereo epipolar constraint. The method is applied to the stationary pair of the stereo cameras, therefore, the visual fields to observe the motions of both the demonstrator and the learner are limited. This paper presents a method to extend our previous work by adopting a pair of rotation invariant stereo cameras that has pan and tilt motions without changing the optical center, therefore, the stereo epipolar equation does not change. The spherical projection is used to represent the constraint. The experimental results obtained are shown.

Existing robotic approaches have focused on the behavior generation assuming the observation of the internal model of the demonstrator, but have not paid any attention on how to build such a model from the learner's perception. This paper presents a computational model of view-based imitation learning without any internal model of the demonstrator. Instead, based on the stereo epipolar constraint, the robot learns to imitate the demonstrator's motion by applying adaptive visual servoing that minimizes the residual between the recovered demonstrator's body parts supposed to be viewed by the demonstrator and the learner's ones in the learner's stereo image planes, and then reproducing the recovered demonstrator's trajectories without any reconstruction of the 3D trajectories. The computer simulation and real experiment are shown and discussion is given.