Although most people interact with social robots in daily environments in a friendly manner, some do not. They sometimes destroy, insult, bully, or abuse them. Uncovering such antisocial attitudes and behaviors is a painful but critical research topic in the process of deploying social robots in real society. In this study, we describe the robot harassment we observed in a three-year exhibition with an android robot. During the exhibition, visitors talked freely with the android robot, which autonomously handled the conversation topics. Many people enjoyed chatting with it in a friendly manner, although a few behaved rudely toward it. A coder labeled 26/971 scenes as “harassment” interactions and organized them into four subcategories: unwelcome flirting, rude attitudes, unsolicited comments about appearance, and inappropriate touching. We introduce these details and discuss possible research topics necessary to reduce such harassment behaviors.

Challenges in multimodal task-oriented dialogue between humans and systems, particularly those involving audio and visual interactions, have not been sufficiently explored or shared, forcing researchers to define improvement directions individually without a clearly shared roadmap. To address these challenges, we organized a competition for multimodal task-oriented dialogue systems and constructed a large competition-based dataset of 1,865 minutes of Japanese task-oriented dialogues. This dataset includes audio and visual interactions between diverse systems and human participants. After analyzing system behaviors identified as problematic by the human participants in questionnaire surveys and notable methods employed by the participating teams, we identified key challenges in multimodal task-oriented dialogue systems and discussed potential directions for overcoming these challenges.

In human-human conversation, interpersonal consideration for the interlocutor is essential, and similar expectations are increasingly placed on dialogue systems. This study examines the behavior of dialogue systems in a specific interpersonal scenario where a user vents frustrations and seeks emotional support from a long-time friend represented by a dialogue system. We conducted a human evaluation and qualitative analysis of 15 dialogue systems under this setting. These systems implemented diverse strategies, such as structuring dialogue into distinct phases, modeling interpersonal relationships, and incorporating cognitive behavioral therapy techniques. Our analysis reveals that these approaches contributed to improved perceived empathy, coherence, and appropriateness, highlighting the importance of design choices in socially sensitive dialogue.

This study explored a tele-operation notification interface that supports a multitasking operator independently perceiving remote information and shifting their tasks. We proposed a noise-reducing auditory notification method, which artificially replicated the cocktail party effect by decreasing the volume of the background noise and increasing the volume of visitor footsteps. Experimental results show that our proposed method provided an effective interface for operators to independently notice human presence.

Human robot interaction studies have investigated how various non-verbal expressions of robots can enhance feelings of familiarity and trust in users’ and establish good relationships between users and robots. In the field of art, this approach has been taken one step further by creating robot artworks and demonstrations through which people feel as if they are experiencing heart-to-heart communication with a robot. Although robotic behavior that conveys such a feeling is useful for human coexistence and providing a sense of security and trust, no engineering methodology can yet achieve such behavior. This study attempts to explain what kind of robot behavior is connected to such feelings by analyzing demonstrations of such a robot based on information theory. We found that the intensity of these feelings can be partially explained by transfer entropy between humans and the robot’s body movements. We expect this research to clarify how to design a robot’s behavior so that it can provide heart-to-heart communication with people as well as how to construct a robot that can build solid relationships with people in daily life.

We are dealing with the problem of generating natural human-like motions during speech in android robots, which have human-like appearances. So far, automatic generation methods have been proposed for lip and head motions of tele-presence robots, based on the speech signal of the tele-operator. In the present study, we aim for extending the speech-driven motion generation methods for laughing speech, since laughter often occurs in natural dialogue interactions and may cause miscommunication if there is mismatch between audio and visual modalities. Based on analysis results of human behaviors during laughing speech, we proposed a motion generation method given the speech signal and the laughing speech intervals. Subjective experiments were conducted using our android robot by generating five different motion types, considering several modalities. Evaluation results show the effectiveness of controlling different parts of the face, head and upper body (eyelid narrowing, lip corner/cheek raising, eye blinking, head motion and upper body motion control).