This study investigates how interaction methods and spatial distance affect user performance and experience in virtual reality (VR) tasks. We compared the Go-Go technique with ray-based Laser selection across eight target distances (0.67× to 4× arm length) in a memory task to identify distance ranges that minimize motor and cognitive demands and support working memory. Results with 41 participants revealed a trade-off: Go-Go enhanced embodiment, while Laser yielded higher efficiency and lower workload. This trade-off was distance-dependent: Go-Go performed best at 1.2×-1.4×, and Laser at 1.4×-2.2×. Our results offer practical design guidelines for adaptive, distance-aware VR systems that align performance and subjective experience.

The rise of urbanisation has reduced connection with nature and physical interaction, both crucial for well-being and pro-environmental behavior. Sensing Nature is a multisensory virtual reality installation that aims to energize and nourish the spirit by reimagining the natural world as a playful, immersive experience. Users create a sensory journey by interacting with a haptic tree and exploring real-world fabrics and textures. Each touch triggers a transformation of the virtual tree, blending blooming virtual flowers, nature-inspired spatial sounds, and physical vibrations for a unique immersive experience. This project investigated multisensory interaction in virtual reality’ s impacts on people's feelings and attitude to nature, addressing the lack of direct touch-based haptic interactions in VR by incorporating active and passive haptic feedback. Qualitative studies shown that multisensory interactions in virtual reality induce healing effects, relaxation and shift attitudes towards nature, demonstrating sensing nature's potential application for relaxation and pro- environmental attitudes.

We explored how a Large Language model-powered multilingual virtual agent can reduce foreign language anxiety (FLA) in a VRbased icebreaker scenario.Our agent supports hybrid language interaction-accepting input in either English or the user's native language, while responding in English.Ten participants interacted with the agent using both language modes.Quantitative results showed reduced subjective FLA after the English session, compared to their real-world experience, and increased external focus when using their native language (vs English).Qualitative findings suggested that participants experienced greater anxiety in understanding compared to speech production, while language switching also imposed additional cognitive costs.

We conducted a within-subjects study to examine how realistic faces and cartoon faces on avatars affect communication, task satisfaction, sense of presence, and mood perception in mixed reality meetings. Over the course of two weeks, six groups of co-workers (14 people) held recurring meetings using Microsoft HoloLens2 devices, each person embodying a personal full-body avatar with either a realistic face or cartoon face. Half of the groups started with the realistic face avatar and switched to the cartoon face version halfway through (RC condition), and the other half with the cartoon-face avatar first (CR condition). Results showed that participants in the RC condition may have had higher expectations and more errors in perceiving their colleagues’ moods. Participants in the CR condition reported that the avatars’ appearance mattered less over time and experienced increased comfort and improved identification of their colleagues. Participants rated words, tone of voice, and movement as the most useful cues for perceiving colleagues’ moods, regardless of avatar rendering style. In the RC condition, participants rated gaze as more useful than facial expressions, while in the CR condition, both gaze and facial expressions were rated as the least useful. Results also suggested that participants had more errors when perceiving negative moods in their colleagues, with this trend appearing for most moods, but depending on conditions. Implications of these findings for mixed and virtual reality meetings are discussed. This work contributes to the field of remote collaboration by providing insights from longitudinal data on the impact of avatar appearance on various aspects of work meetings in virtual environments. • Realistic avatars may lead to higher expectations and errors in perceiving moods. • Cartoon avatars may increase the comfort and identification of colleagues over time. • For both avatars, words, voice, and movement were the most useful to perceive moods. • Avatar’s evaluation should be longitudinal as attitudes towards them change over time. • Avatar attitudes depend on context; evaluation should prioritise ecological validity. • Avatars’ identification, communication, and emotional trust may outweigh appearance.

Avatars are edging into mainstream videoconferencing, but evaluation of how avatar animation modalities contribute to work meeting outcomes has been limited. We report a within-group videoconferencing experiment in which 68 employees of a global technology company, in 16 groups, used the same stylized avatars in three modalities (static picture, audio-animation, and webcam-animation) to complete collaborative decision-making tasks. Quantitatively, for meeting outcomes, webcam-animated avatars improved meeting effectiveness over the picture modality and were also reported to be more comfortable and inclusive than both other modalities. In terms of avatar satisfaction, there was a similar preference for webcam animation as compared to both other modalities. Our qualitative analysis shows participants expressing a preference for the holistic motion of webcam animation, and that meaningful movement outweighs realism for meeting outcomes, as evidenced through a systematic overview of ten thematic factors. We discuss implications for research and commercial deployment and conclude that webcam-animated avatars are a plausible alternative to video in work meetings.

Stigma towards individuals with schizophrenia reduces quality of life, creating a barrier to accessing education and employment opportunities. Schizophrenia is one of the most stigmatized mental health conditions, and stigma is prevalent particularly among healthcare professionals. In this study, we investigated whether Virtual Reality (VR) can be incorporated into interventions to reduce stigma. In particular, we compared the effectiveness of three VR conditions based on intergroup contact theory in reducing stigma in form of implicit and explicit attitudes, and behavioral intentions. Through an immersive virtual consultation in a clinical setting, participants (N=60) experienced one of three different conditions: the Doctor's perspective (embodiment in a majority group member during contact), the Patient's perspective (embodiment in a minority group member) and a Third-person perspective (vicarious contact). Results demonstrated an increase of stigma on certain explicit measures (perceived recovery and social restriction) but also an increase of empathy (perspective-taking, empathic concern) across all conditions regardless of perspective. More importantly, participants' viewpoint influenced the desire for social distance differently depending on the perspective: the Third-person observation significantly increased the desire for social distance, Doctor embodiment marginally decreased it, while Patient embodiment showed no significant change. No change was found in the Implicit Association Test. These findings suggest that VR intergroup contact can effectively reduce certain dimensions of stigma toward schizophrenia, but the type of perspective experienced significantly impacts outcomes.

Research on the emotional experience of playing videogames has increased in recent years, yet much of this work is focused on the hedonistic player experience (PX) commonly associated with the nebulous concept of ‘fun’ and positive affect. Researchers are increasingly paying more attention to the eudaimonic PX commonly associated with ‘appreciation’, mixed-affect and reflection. To further investigate eudaimonic PX we interviewed 24 games players about ‘significant or memorable emotional experiences’ from their games playing and used grounded theory to analyse their responses. This led to the construction of the concept of ‘emotional exploration’ which is used to help explain (i) why players would seek out a eudaimonic PX, (ii) how eudaimonic PX is constituted and (iii) how developers can design for a eudaimonic PX. We further make the case for the ‘eudaimonic gameplay experience’ to be realised as different and separate to pre-existing notions of eudaimonic entertainment.

Until recently, Virtual Reality (VR) applications relied on controllers to enable user interaction in virtual environments. With advances in tracking technology, HMDs are now able to track the movements of users’ hands in real-time with significantly greater accuracy, allowing us to interact with the digital world directly with our hands. However, it is not entirely clear how hand tracking affects users’ performance. In this study, we investigate user performance using an in-game analytics-based assessment methodology for a VR memory puzzle task. We conducted a within-subjects experiment with 30 participants in three conditions: 1- Hand-tracking, 2-Controller Without Haptics, and 3- Controller With Haptics. In all our measurements (correct order and pattern, correct pattern only, and trial completion) except for the initial selection time, participants performed best with hand tracking. The use of controllers with haptics did not outperform controllers without haptics in most measures, possibly because other feedback cues compensated for the lack of haptics. This study helps us better understand the three selected interactivity methods when used in VR, as well as the importance of naturalistic experience in interaction design.

People can understand how human interaction unfolds and can pinpoint social attitudes such as showing interest or social engagement with a conversational partner. However, summarising this with a set of rules is difficult, as our judgement is sometimes subtle and subconscious. Hence, it is challenging to program agents or non-player characters (NPCs) to react towards social signals appropriately, which is important for immersive narrative games in Virtual Reality (VR). We present a collaborative work between two game studios (Maze Theory and Dream Reality Interactive) and academia to develop an immersive machine learning (ML) pipeline for detecting social engagement. Here we introduce the motivation and the methodology of the immersive ML pipeline, then we cover the motivation for the industry-academia collaboration, how it progressed, the implications of joined work on the industry and reflective insights on the collaboration. Overall, we highlight the industry-academia collaborative work on an immersive ML pipeline for detecting social engagement. We demonstrate how creatives could use ML and VR to expand their ability to design more engaging commercial games.

Nonverbal communication is an important part of human communication, including head nodding, eye gaze, proximity and body orientation. Recent research has identified specific patterns of head nodding linked to conversation, namely mimicry of head movements at 600 ms delay and fast nodding when listening. In this paper, we implemented these head nodding behaviour rules in virtual humans, and we tested the impact of these behaviours, and whether they lead to increases in trust and liking towards the virtual humans. We use Virtual Reality technology to simulate a face-to-face conversation, as VR provides a high level of immersiveness and social presence, very similar to face-to-face interaction. We then conducted a study with human-subject participants, where the participants took part in conversations with two virtual humans and then rated the virtual character social characteristics, and completed an evaluation of their implicit trust in the virtual human. Results showed more liking for and more trust in the virtual human whose nodding behaviour was driven by realistic behaviour rules. This supports the psychological models of nodding and advances our ability to build realistic virtual humans.

In the current consumer market, Virtual reality experiences are predominantly generated through visual and auditory feedback. Haptics are not yet well established, but are increasingly introduced to enhance the user’s sense of ‘reality’. With haptic (vibrotactile) feedback now part of the built-in mechanism of VR consumer devices, there is an urgent need to understand how different modalities work together to improve the user experience. This paper reports an experiment that explores the contributions made to participants’ sense of presence by haptic and visual feedback in a virtual environment. Participants experienced a virtual ball bouncing on a virtual stick resting across their avatar hands. We found that presence was enhanced when they could both see and feel the ball’s action; with a strong suggestion that haptic feedback alone gave rise to a greater sense of presence than visual alone. Similarly, whilst visual or bimodal feedback enhanced participants’ ability to locate where the ball bounced on the stick, our results suggest that the action itself was more readily discerned haptically than visually.

There are vast potential applications for children's entertainment and education with modern virtual reality (VR) experiences, yet we know very little about how the movement or form of such a virtual body can influence children's feelings of control (agency) or the sensation that they own the virtual body (ownership). In two experiments, we gave a total of 197 children aged 4-14 years a virtual hand which moved synchronously or asynchronously with their own movements and had them interact with a VR environment. We found that movement synchrony influenced feelings of control and ownership at all ages. In Experiment 1 only, participants additionally felt haptic feedback either congruently, delayed or not at all - this did not influence feelings of control or ownership. In Experiment 2 only, participants used either a virtual hand or non-human virtual block. Participants embodied both forms to some degree, provided visuomotor signals were synchronous (as indicated by ownership, agency, and location ratings). Yet, only the hand in the synchronous movement condition was described as feeling like part of the body, rather than like a tool (e.g., a mouse or controller). Collectively, these findings highlight the overall dominance of visuomotor synchrony for children's own-body representation; that children can embody non-human forms to some degree; and that embodiment is also somewhat constrained by prior expectations of body form.

Nonverbal cues have multiple roles in social encounters, with gaze behaviour facilitating interactions and conversational flow. In this work, we explore the conversation dynamics in dyadic settings in a free-flow discussion. Using automatic analysis (rather than manual labelling), we investigate how the gaze behaviour of one person is related to how much the other person changes their gaze (frequency in gaze change) and what their gaze target is (direct or avert gaze). Our results show that when one person is looked at they change their gaze direction with a higher frequency compared to when they are not looked at. They also tend to maintain a direct gaze to the other person when they are not looked at.

As immersive technologies are increasingly being adopted by artists, dancers and developers in their creative work, there is a demand for tools and methods to design compelling ways of embodied interaction within virtual environments. Interactive Machine Learning allows creators to quickly and easily implement movement interaction in their applications by performing examples of movement to train a machine learning model. A key aspect of this training is providing appropriate movement data features for a machine learning model to accurately characterise the movement then recognise it from incoming data. We explore methodologies that aim to support creators’ understanding of movement feature data in relation to machine learning models and ask how these models hold the potential to inform creators’ understanding of their own movement. We propose a 5-day hackathon, bringing together artists, dancers and designers, to explore designing movement interaction and create prototypes using new interactive machine learning tool InteractML.

Interactive Machine Learning offers a method for designing movement interaction that supports creators in implementing even complex movement designs in their immersive applications by simply performing them with their bodies. We introduce a new tool, InteractML, and an accompanying ideation method, which makes movement interaction design faster, adaptable and accessible to creators of varying experience and backgrounds, such as artists, dancers and independent game developers. The tool is specifically tailored to non-experts as creators configure and train machine learning models via a node-based graph and VR interface, requiring minimal programming. We aim to democratise machine learning for movement interaction to be used in the development of a range of creative and immersive applications.

Interaction based on human movement has the potential to become an important new paradigm of human-computer interaction. However, high quality, mainstream movement interaction requires effective tools and techniques to support designers. A promising approach to movement interaction design is Interactive Machine Learning, in which designing is done by physically performing an action. This article brings together many different perspectives on understanding human movement knowledge and movement interaction. This understanding shows that the embodied knowledge involved in movement interaction is very different from the representational knowledge involved in a traditional interface, so a very different approach to design is needed. We apply this knowledge to understand why interactive machine learning is an effective tool for motion interaction designers and to make a number of suggestions for future development of the technique.

Machine learning is one of the most important and successful techniques in contemporary computer science. Although it can be applied to myriad problems of human interest, research in machine learning is often framed in an impersonal way, as merely algorithms being applied to model data. However, this viewpoint hides considerable human work of tuning the algorithms, gathering the data, deciding what should be modeled in the first place, and using the outcomes of machine learning in the real world. Examining machine learning from a human-centered perspective includes explicitly recognizing human work, as well as reframing machine learning workflows based on situated human working practices, and exploring the co-adaptation of humans and intelligent systems. A human-centered understanding of machine learning in human contexts can lead not only to more usable machine learning tools, but to new ways of understanding what machine learning is good for and how to make it more useful. This special issue brings together nine articles that present different ways to frame machine learning in a human context. They represent very different application areas (from medicine to audio) and methodologies (including machine learning methods, human-computer interaction methods, and hybrids), but they all explore the human contexts in which machine learning is used. This introduction summarizes the articles in this issue and draws out some common themes.

We are interested the effect of lip and arm synchronization on body ownership in VR (the illusion that the users own a virtual body). Participants were invited to give a presentation in an HMD, while seeing in a virtual mirror a gender-matched avatar who copied their arm and lip movements in sync and a-sync conditions. We measure participants' reaction with questionnaires administrated verbally after their presentation while immersed in VR. The result suggested an interaction effect of arm and lip, showing reports of higher level of embodiment with the congruent as compared to the incongruent conditions. Further study is needed to confirm if the same interaction effect can be captured with objective measurements.

This paper presents an application of the CASSM (Concept-based Analysis of Surface and Structural Misfits) framework to interactive machine learning for a bodily interaction domain. We developed software to enable end users to design full body interaction games involving interaction with a virtual character. The software used a machine learning algorithm to classify postures as based on examples provided by users. A longitudinal study showed that training the algorithm was straightforward, but that debugging errors was very challenging. A CASSM analysis showed that there were fundamental mismatches between the users concepts and the working of the learning system. This resulted in a new design in which aimed to better align both the learning algorithm and user interface with users' concepts. This work provides and example of how HCI methods can be applied to machine learning in order to improve its usability and provide new insights into its use.

This article presents Gesture Interaction DEsigner (GIDE), an innovative application for gesture recognition. Instead of recognizing gestures only after they have been entirely completed, as happens in classic gesture recognition systems, GIDE exploits the full potential of gestural interaction by tracking gestures continuously and synchronously, allowing users to both control the target application moment to moment and also receive immediate and synchronous feedback about system recognition states. By this means, they quickly learn how to interact with the system in order to develop better performances. Furthermore, rather than learning the predefined gestures of others, GIDE allows users to design their own gestures, making interaction more natural and also allowing the applications to be tailored by users' specific needs. We describe our system that demonstrates these new qualities—that combine to provide fluid gesture interaction design—through evaluations with a range of performers and artists.

With characters in computer games and interactive media increasingly being based on real actors, the individuality of an actor's performance should not only be reflected in the appearance and animation of the character but also in the AI that governs the character's behavior and interactions with the environment. Machine learning methods applied to motion capture data provide a way of doing this. This paper presents a method for learning the parameters of a finite-state machine (FSM) controller. The method learns both the transition probabilities of the FSM and also how to select animations based on the current state.

We report work on adding an improvisational AI actor and 3D emotional animation to an existing edrama program, a system for dramatic improvisation in simple virtual scenarios. The improvisational AI actor has an affect-detection component, aimed at detecting affective aspects of human-controlled characters' textual input. It also makes an appropriate response to stimulate the improvisation based on this affective understanding. A distinctive feature of our work is a focus on the metaphorical ways in which affect is conveyed. Moreover, we have also introduced how the detected affective states activate the animation engine to produce emotional gestures for human-controlled characters. Finally, we report user testing conducted for the AI actor. Our work contributes to the conference themes on affective user interfaces, natural language processing and emotionally believable gesture generation.

This paper presents a virtual character animation system for real time multi-modal interaction, which can handle many different styles of interaction and animation simultaneously. Our system is based on a model of animation that represents different types of animations as instantiations of an abstract function representation. This makes it easy to combine different types of animation and encourages the creation of behavior out of basic building blocks.

Abstract Humans use their bodies in a highly expressive way during conversation, and animated characters that lack this form of non‐verbal expression can seem stiff and unemotional. An important aspect of non‐verbal expression is that people respond to each other's behavior and are highly attuned to picking up this type of response. This is particularly important for the feedback given while listening to some one speak. However, automatically generating this type of behavior is difficult as it is highly complex and subtle. This paper takes a data driven approach to generating interactive social behavior. Listening behavior is motion captured, together with the audio being listened to. These data are used to learn an animation model of the responses of one person to the other. This allows us to create characters that respond in real‐time during a conversation with a real human. Copyright ? 2008 John Wiley & Sons, Ltd.

Abstract This paper describes a simulation of attention behaviour aimed at computer‐animated characters. Attention is the focusing of a person's perception on a particular object. This is useful for computer animation as it determines which objects the character is aware of: information that can be used in the simulation of the character's behaviour in order to automatically animate the character. The simulation of attention also determines where the character is looking and so is used to produce gaze behaviour. Copyright © 2002 John Wiley & Sons, Ltd.