Jessie Wang, Jiawen Duan, Jian Wang, Kaitao Song, Chunpu Xu, Johnny K. W. Ho, YU Fenggang, Johan F. Hoorn, Wenjie Li. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

This study examines user experience evolution across three repeated interactions with an on-screen NAO robot designed to express artificial empathy through verbal communication and music. The participant numbers across the three interactions were N1 = 139, N2 = 129, and N3 = 121, respectively, with 121 participants completing all sessions. During interaction, the robot gave empathic feedback and/or played music to the participant as a token of empathy. Repeated measures MANCOVA and Structural Equation Modeling revealed that initial bonding tendencies and perceptions of the robot trying to be empathetic faded over time. In its place, a tendency emerged of the robot becoming more personally relevant and remarkably, its design appeared to become more realistic, like a human being. When the robot merely tried empathetic conversation or just played music, participants were disappointed about its capabilities, visible in increased levels of negative valence. Bonding and perceived empathy flourished when the robot played music while talking empathically in chorus, a mutual reinforcement effect. At first, for the loneliest individuals, the mere presence of the robot, rather than its empathic behaviors, was more influential in determining the robot’s relevance to their concerns. These results underscore the importance of a multimodal approach in designing empathic robots.

Design of Artificial Intelligence and robotics habitually assumes that adding more humanlike features improves the user experience, mainly kept in check by suspicion of uncanny effects. Three strands of theorizing are brought together for the first time and empirically put to the test: Media Equation (and in its wake, Computers Are Social Actors), Uncanny Valley theory, and as an extreme of human-likeness assumptions, the Singularity. We measured the user experience of real-life visitors of a number of seminars who were checked in either by Smart Dynamics’ Iwaa, Hanson's Sophia robot, Sophia's on-screen avatar, or a human assistant. Results showed that human-likeness was not in appearance or behavior but in attributed qualities of being alive. Media Equation, Singularity, and Uncanny hypotheses were not confirmed. We discuss the imprecision in theorizing about human-likeness and rather opt for machines that ‘function adequately.’

Young adults undergoing psychological changes are particularly vulnerable. Recent social isolation impedes interpersonal help while stress from family, school, work, and society has brought negative effects on mental health, even in otherwise healthy young adults. Recent research has shown that daily creativity contributes to well-being. To circumvent issues of contamination, we tried a NAO robot guiding a Loving-kindness Meditation (LKM) and Walking Meditation (WM). By improving mental states (i.e. positive valence and state openness), we stimulated creative behavior to reduce negative mood. Participants (N = 142) were healthy individuals, aged between 18 and 34, joining a one-time laboratory experiment. They responded to two rounds of questionnaires, with a 10-minute intervention guided by audio or a NAO robot in between each round. A control group with participants with no treatment (i.e. taking a 10-minute rest) was added for comparison. Both audio-guided LKM and WM successfully evoked state openness, with the former also exerting a positive effect on valence. Valence and state openness were positively correlated, and both were associated with a higher willingness to create. With positive valence, young adults likely perform better on convergent thinking. The result may potentially lead to negative mood reduction. The discussion emphasizes the importance of designing specific characteristics of social robots in accordance with the task's context.

After 20 years of testing a framework for affective user responses to artificial agents and robots, we compiled a full formalization of our findings so to make the agent respond affectively to its user. Silicon Coppélia as we dubbed our system works from the features of the observed other, appraises these in various domains (e.g., ethics and affordances), then compares them to goals and concerns of the agent, to finally reach a response that includes intentions to work with the user as well as a level of being engaged with the user. This ultimately results in an action that adds to or changes the situation both agencies are in. Unlike many other systems, Silicon Coppélia can deal with ambiguous emotions of its user and has ambiguous ‘feelings’ of its own, which makes its decisions quite human-like. In the current paper, we advance a fuzzy-sets approach and show the inner workings of our system through an elaborate example. We present a number of simulation experiments, one of which showed decision behaviors based on biases when agent goals had low priorities. Silicon Coppélia is open to scrutiny and experimentation by way of an open-source implementation in Ptolemy.

We investigated the effects of facial similarity between users and embodied agents under different experimental conditions. Sixty-four undergraduates interacted with two different embodied agents: in one case the agent was designed to look somewhat similar to the user, and in the other case the agent was designed to look dissimilar. We varied between subjects how helpful the agent was for a given task. Results showed that the facial similarity manipulation sometimes affected participants' responses, even though they did not consciously detect the similarity. Specifically, when the agent was helpful, facial similarity increased participantsi ratings of involvement. However, when exposed to unhelpful agents, male participants had negative responses to the similar-looking agent compared to the dissimilar one. These results suggest that using facially similar embodied agents has a potential large downside if that embodied agent is perceived to be unhelpful.

Abstract This paper presents an empirically tested theoretical framework to explain user engagement and end‐user satisfaction with interactive agents. Such a framework is not only important from a scientific point of view; application designers may find a set of dos and don'ts that help them create more satisfying embodied agents in different task domains and social settings. From a multidisciplinary perspective, we have conducted a series of experiments to verify underlying mechanisms in the processes of interacting and engaging with embodied agents in various task domains. Our results show that the most commonly held views are not always tenable; sometimes other factors provide better explanations for liking an embodied agent or end‐user satisfaction. For example, it is not realism but rather affordances and ethics that are key for understanding user responses, and a beautiful design is not always the most preferable. From our results, guidelines for designers and future research are reflected upon. Copyright © 2009 John Wiley & Sons, Ltd.

Human-like characters in the interface may evoke social responses in users, and literature suggests that realism is the most important factor herein. However, the effects of interface characters on the user are not well understood. We developed an integrative framework, called I-PEFiC, to explain ‘persona’ and realism effects on the user. We tested an important part of the model using an experimental design in which 140 middle school students were class-wise shown an informative virtual reality demonstration that incorporated either a realistic or an unrealistic (fantasy) interface character, or no character. Findings show, first, no persona effect on task performance. We discuss how user engagement might be related to persona effects. Second, designed realism of the interface character contributed to user engagement when controlled for various user perceptions. Moreover, perceived aesthetics and task-relevance further influenced user engagement. Third, user engagement and task performance combined better predicted satisfaction than either one of the factors alone. In sum, several appearance- and task-related factors contributed to user engagement and user satisfaction. Thus, realism is not all.

To contribute to HCI investigation and interface design that develops interactive systems for creative solutions, I attempt to formulate a model of the human capability to combine familiar objects or concepts in an unusual way. Important components of the creative process are feature association, combination, abstraction, selection, integration, and adaptation to establish an optimal fit between two or more semantically remote entities. In the act of creating, the goal is to show (a quantity of) similarity where no one saw it before. The function is knowledge acquisition (also emotionally), to find all the available possibilities in a given situation, (showing how) to find new solutions, new ways to get what you want. The effect of a creation may be surprise as a function of the tension between similarity and dissimilarity between objects and/or concepts. Depending on individual tolerance levels, the balance between similarity and dissimilarity may be satisfying or pleasing. Consequences for representations design are discussed.