Animacy is central to HRI, as it is critical for perceptions of intentionality and influences acceptance, interaction quality, and trust. In order to better understand the underlying mechanisms of the sense of animacy and its role in HRI, we conducted an experiment (n=13) using a social robotic object that is ambiguous with respect to its perceived animacy. We examined how brain activity related to a subjective animacy rating of the robot by participants after watching it perform simple gestures. Using functional imaging, we found that the ventral occipitotemporal (VOT) cortex, a region known to distinguish animate from inanimate entities, also responds to robots along this same organizational gradient. In an exploratory analysis, activity in the left VOT was associated with the individual differences in participants’ subjective animacy rating of the robot. This extends established principles of animacy perception to non-humanoid robots and suggests that individual differences in brain responses may shape how robots are experienced. We discuss implications for HRI, including guiding social cue design, informing gesture alignment with robot roles, and raising ethical considerations around attachment and trust.

Abstract In this work we extend speaker‐centric audio‐driven gesture synthesis toward a unified conversational model that jointly captures both speaking and listening behaviors. Existing speaker‐centric models effectively generate gestures aligned with speech but overlook the bidirectional dynamics that characterize natural dialogue. To address this limitation, we propose the Conversational Gesture Model (CGM), a cross‐attention‐based model capable of synthesizing gestures conditioned on interlocutor conversational cues such as gestures, tone, and textual semantics. By leveraging cross‐attention mechanisms, the model fuses interlocutor audio and text features with character gesture encodings, enabling a single system to seamlessly alternate between speaking and listening roles of the same character. Hence, our model enables a single system to act as both speaker and listener, capturing the fluid role shifts and mutual influence inherent in conversation. Experiments demonstrate that this approach preserves the quality of speaker‐driven gestures while significantly improving the realism, coherence, and responsiveness of full conversational interactions.

Abstract While recent advances in co‐speech gesture generation have achieved impressive rhythmic synchronization, synthesizing gestures that are both semantically meaningful and faithful to a speaker's unique non‐verbal style remains an open challenge. Semantic gestures, such as iconic shapes or deictic pointing, are statistically sparse, making them difficult to learn effectively within standard generative models. We present SiGnature, a framework for Stylized and Semantic Gesture generation that reconciles precise semantic control with high‐fidelity style preservation. Unlike prevalent methods that rely on entangled latent representations, SiGnature operates in an explicit joint‐rotation space. This design enables our core contribution, Joint Motion Integration (JMI), a training‐free inference mechanism capable of injecting any external motion sequence, particularly in‐the‐wild semantic gestures, directly into the diffusion process. JMI automatically identifies the specific “active joints” conveying a semantic action and injects them into the generation, while relying on the diffusion backbone to synthesize the remaining body dynamics, including posture and flow, in accordance with the pre‐learned style of the target speaker. This allows for the plug‐and‐play integration of arbitrary motions, including complex semantic gestures, without retraining or introducing the “Frankenstein” artifacts typical of cut‐and‐paste methods. Extensive experiments and perceptual studies demonstrate that SiGnature offers superior semantic motion control while maintaining smooth and natural co‐speech gesture generation and preserving the distinct characteristics of the speaker, thereby outperforming state‐of‐the‐art baselines.

an evaluation of Inceptor in a project focused on resilience training in mental health contexts.Using the automated pipeline allowed us to generate 32 VR episodes in a very short time, spanning more than two hours of carefully designed content.Additionally, we assessed the feasibility of using large language models (LLMs) to autonomously select appropriate animations.The content is now being used for training diverse professional groups in mental health.Encouraged by positive feedback and early evidence of its utility, Inceptor will be released as an open-source project, poised for widespread adoption and community-driven enhancements.

Motivational interviewing (MI) is a counseling approach that aims to increase intrinsic motivation and commitment to change. Despite its effectiveness in various disorders such as addiction, weight loss, and smoking cessation, publicly available annotated MI datasets are scarce, limiting the development and evaluation of MI language generation models. We present MI-TAGS, a new annotated dataset of MI therapy sessions written in English collected from video recordings available on public sources. The dataset includes 242 MI demonstration transcripts annotated with the MI Treatment Integrity (MITI) 4.2 therapist behavioral codes and global scores, and Client Language EAsy Rating (CLEAR) 1.0 tags for client speech. In this paper we describe the process of data collection, transcription, and annotation, and provide an analysis of the new dataset. Additionally, we explore the potential use of the dataset for training language models to perform several MITI classification tasks; our results suggest that models may be able to automatically provide utterance-level annotation as well as global scores, with performance comparable to human annotators.

It is becoming increasingly easier to set up multi-user virtual reality sessions, and these can become viable alternatives to video conference in events such as international conferences. Moreover, it is possible to enhance such events with automated virtual humans, who may participate in the discussion. This paper presents the behind-the-scenes work of a panel session titled "Is virtual reality genuine reality?", which was held during a physical symposium, "XR for the people," in June 2022. The panel featured a virtual Albert Einstein, based on a large language model (LLM), as a panelist, alongside three international experts having a live conference panel discussion. The VR discussion was broadcast live on stage, and a moderator was able to communicate with both the live audience, the virtual world participants, and the virtual agent (Einstein). We provide lessons learned from the implementation and from the live production, and discuss the potential and pitfalls of using LLM-based virtual humans for multi-user VR in live hybrid events.

Virtual agents have been used as virtual patients for medical training, as well as for mental health training. When the training takes place inside VR the experience is more immersive, which allows for illusions of presence: the illusion that you are co-present with the virtual agent in the same space, and the illusion that the virtual agent is a real human. We have developed 'Daniel', a VR framework, based on a semi-automated virtual agent, which can be used for training for increasing resilience and for suicide prevention, and has the potential of being used as an intervention. Here we report on two different studies aimed at evaluating the framework and the psychological protocols involved. In the first study we trained participants from the general population to develop a resilience plan intervention (RPI) with a distressed virtual agent, and in the second study we trained therapists to use the safety plan intervention (SPI) with a suicidal virtual agent. In both cases we compare the VR sessions with role-playing by human actors. We report that all interventions resulted in an increase in participant self-efficacy in helping others, and we also report results on the possible importance of presence and social presence.

Sexual harassment (SH) training is a major public health priority, and available programs show limited efficacy. We present a new application for virtual agents in virtual reality (VR), which has the potential to deliver SH training that imitates real world scenarios and is yet safe. We conducted a proof-of-concept study to examine whether a VR simulation of a job interview during which the interviewer, a semi-automated virtual agent, sexually harasses the interviewee is an effective practice tool for women, and could serve as the basis for developing skills for effective response. Five females (24-25 years old) participated in this VR scenario and were then interviewed about their reactions to the VR scenario and the virtual agent. Four main themes emerged: paralysis and fear, uncertainty how to respond, resurfacing of previous SH experiences, and VR as an effective training tool for preventing SH. Findings suggest that contemporary virtual agents in VR can induce the sense of being harassed, similar to real humans. VR offers a safe exposure for SH and may serve as a learning platform to empower women in considering and practicing more effective responses to future SH experiences.

Interactions with virtual agents may have psychological and behavioral implications, even if the participants know that they are interacting with a virtual entity. As virtual agents are gradually becoming part of human society, it is important to understand the extent to which virtual encounters can affect our daily lives, and whether engaging in a specific behavior with virtual humans affects the way that individuals perceive and asses real humans in their surroundings. We examined the effect that seductive interplays might have on individuals in committed relationships and their way of managing a virtual threat to their relationship. One hundred and thirty heterosexual participants conversed with an opposite-sex virtual human in a virtual reality (VR) setup in either a seductive or neutral way. Shortly after, participants were interviewed by an attractive opposite-sex confederate. Results revealed that participants in the seductive condition felt increased feelings of guilt, and that participants in the seductive condition were more prone to devaluate the sexual and intellectual attractiveness of the confederate than participants in the neutral condition. This study thus demonstrates, for the first time, that flirting with a virtual human may influence real-life attitudes towards real people.

We have developed Brain-Voyant, an efficient general-purpose machine learning tool for real-time functional magnetic resonance imaging classification using whole-brain data, which can be used to explore novel brain-computer interface paradigms or advanced neurofeedback protocols. We have created a convenient and configurable front-end tool that receives fMRI-based multi-voxel raw brain data as input. Our tool processes, analyses, classifies and transfers the classification to an external object such as a virtual avatar or a humanoid robot in real-time. Our tool is focused on minimizing delay time, and to that end, it employs a method that is based on examining in advance the voxels that have been found to be task-relevant in the machine learning model training phase. The tool's code base was designed to be easily extended to support additional feature reduction, normalization and classification algorithms. This tool was used in several published studies using motor execution, motor imagery, and visual category classification in cue-based and free-choice brain-computer interface experiments, with both healthy and amputated subjects. This tool is not limited by number of classes, is not limited to predefined regions of interest, and classifier instances can run in parallel to combine multiple classification tasks in real time. Finally, our tool is able use the slow peaking blood-oxygen-level dependent signal to classify our subjects' intention during the two-second window TR. We release this tool as open-source for non-commercial usage.

Affective computing requires a reliable method to obtain real time information regarding affective state, and one of the promising avenues is via electroencephalography (EEG). We have performed a study intended to test whether a low cost EEG device targeted at consumers can be used to measure extreme emotional valence. One of the most studied frameworks related to the way affect is reflected in EEG is based on frontal hemispheric asymmetry. Our results indicate that a simple replication of the methods derived from this hypothesis might not be sufficient. However, using a data-driven approach based on feature engineering and machine learning, we describe a method that can reliably measure valence with the EPOC device. We discuss our study in the context of the theoretical and empirical background for frontal asymmetry.

We have integrated the Graz brain–computer interface (BCI) system with a highly immersive virtual reality (VR) Cave-like system. This setting allows for a new type of experience, whereby participants can control a virtual world using imagery of movement. However, current BCI systems still have many limitations. In this article we present two experiments exploring the different constraints posed by current BCI systems when used in VR. In the first experiment we let the participants make free choices during the experience and compare their BCI performance with participants using BCI without free choice; this is unlike most previous work in this area, in which participants are requested to obey cues. In the second experiment we allowed participants to control a virtual body with motor imagery. We provide both quantitative and subjective results, regarding both BCI accuracy and the nature of the subjective experience in this new type of setting.