Social Virtual Reality (SVR) enables users to embody avatars and interact with others in shared virtual environments. On platforms such as Rec Room, VRChat, and Meta Horizon Worlds, inappropriate social behaviours are common and can lead to harmful emotional experiences for users. To prevent or mitigate these negative effects, SVR platforms offer a range of actions, referred to as safety tools, that allow users to control the information and interactions they are exposed to. Although prior research has analysed and proposed initial classifications of these tools, a structured, theory-driven approach to their characterization is still lacking. In this paper, we propose a taxonomy of safety tools for SVR platforms inspired by existing literature in social psychology and human-computer interaction. In order to characterise these tools, we focus on the area of social interaction, which occurs between a sender and a receiver, within a social context of the SVR medium. Our approach not only provides a model of social interaction in SVR, but also describes how safety tools function, including the processes of selecting and manipulating them, factors that are critical to their effectiveness. Finally, we apply our framework to characterise existing safety tools, identify their limitations, and present design guidelines for future SVR platforms.

International audience

Avoiding moving obstacles in immersive environments requires adjustments in the walking trajectory and depends on the type of obstacle movement. Previous research studied the impact of speed and direction of motion but not much is known about how predictability of motion impacts human circumvention in terms of distance to the obstacle (proximity). In this paper, we investigate how participants navigate through VR collision avoidance scenarios with obstacles of varying motion characteristics in terms of speed and predictability. We introduce a novel concept of creating unpredictable motion using entropy calculations. We anticipated that higher entropy would increase the proximity distance which we measured with several metrics related to the distance from the obstacle and centre of the scene. We found a significant influence of motion speed and predictability on proximity-related metrics, with participants exhibiting a tendency to maintain larger distances in scenarios where obstacle speed and entropy were higher. We also outline two decision-making strategies for avoidance behaviour and investigate the factors that influence individuals’ selection of one strategy over the other.

Abstract Generating realistic animated virtual humans is a problem that has been extensively studied with many applications in different types of virtual environments. However, the creation process of such realistic animations is challenging, especially because of the number and variety of influencing factors, that should then be identified and evaluated. In this paper, we attempt to provide a clearer understanding of how the multiple factors that have been studied in the literature impact the level of realism of animated virtual humans, by providing a survey of studies assessing their realism. This includes a review of features that have been manipulated to increase the realism of virtual humans, as well as evaluation approaches that have been developed. As the challenges of evaluating animated virtual humans in a way that agrees with human perception are still active research problems, this survey further identifies important open problems and directions for future research.

Nonverbal communication is paramount in daily life, as well as in populated virtual reality (VR) environments. In this paper, we focused on gaze behaviour, which is key to initiate and drive social interactions. Previous work on photographs and on virtual agents showed the importance of gaze, even in the presence of multiple stimuli, by demonstrating the stare-in-the-crowd effect: humans detect faster and observe gazes directed towards them longer than the averted ones. While previous studies focused on static scenarios, which fail in representing the complexity of real-life social interactions, we propose to explore the stare-in-the-crowd effect in dynamic situations. To this end, we designed a within-subject experiment where 21 users navigated a virtual street through an idle or moving crowd of virtual agents. Agents’ gaze was manipulated to display averted, directed, or shifting gaze. We analysed the user’s gaze (fixations, dwell time) and locomotor behaviours (path decisions, proximity to agents) as well as their social anxiety. Results showed that the stare-in-the-crowd effect is preserved when walking through both types of crowd, and that social anxiety decreases gaze interaction time and affects proximity behaviours in case of agents with directed gazes. However, virtual agents’ gaze did not elicit significant changes on users’ locomotion. These findings highlight the importance of considering virtual agents’ gaze when creating VR environments, and open future work perspectives to better understand factors that would strengthen or decrease this effect at gaze and locomotor levels.

Nonverbal cues are paramount in real-world interactions. Among these cues, gaze has received much attention in the literature. In particular, previous work has shown a search asymmetry between directed and averted gaze towards the observer using photographic stimuli, with faster detection and longer fixation towards directed gaze by the observer. This is known as the stare-in-the-crowd effect. In this study, we investigate whether stare-in-the crowd effect is preserved in Virtual Reality (VR). To this end, we designed a within-subject experiment where 30 human users were immersed in a virtual environment in front of an audience of 11 virtual agents following 4 different gaze behaviours. We analysed the user’s gaze behaviour when observing the audience, computing fixations and dwell time. We also collected the users’ social anxiety score using a post-experiment questionnaire to control for some potential influencing factors. Results show that the stare-in-the-crowd effect is preserved in VR, as demonstrated by the significant differences between gaze behaviours, similarly to what was found in previous studies using photographic stimuli. Additionally, we found a negative correlation between dwell time towards directed gazes and users’ social anxiety scores. Such results are encouraging for the development of expressive and reactive virtual humans, which can be animated to express natural interactive behaviour.

In this paper, we investigate the effect of a realism mismatch in the voice and appearance of a photorealistic virtual character in both immersive and screen-mediated virtual contexts. While many studies have investigated voice attributes for robots, not much is known about the effect voice naturalness has on the perception of realistic virtual characters. We conducted the first experiment in Virtual Reality (VR) with over two hundred participants investigating the mismatch between realistic appearance and unrealistic voice on the feeling of presence, and the emotional response of the user to the character expressing a strong negative emotion. We predicted that the mismatched voice would lower social presence and cause users to have a negative emotional reaction and feelings of discomfort towards the character. We found that the concern for the virtual character was indeed altered by the unnatural voice, though interestingly it did not affect social presence. The second experiment was conducted with a view towards heightening the appearance realism of the same character for the same scenarios, with an additional lower level of voice realism employed to strengthen the mismatch of perceptual cues. While voice type did not appear to impact reports of empathic responses towards the character, there was an observed effect of voice realism on reported social presence, which was not detected in the first study. There were also significant results on affinity and voice trait measurements that provide evidence in support of perceptual mismatch theories of the Uncanny Valley.

Abstract The real‐time simulation of human crowds has many applications. In a typical crowd simulation, each person ('agent') in the crowd moves towards a goal while adhering to local constraints. Many algorithms exist for specific local ‘steering’ tasks such as collision avoidance or group behavior. However, these do not easily extend to completely new types of behavior, such as circling around another agent or hiding behind an obstacle. They also tend to focus purely on an agent's velocity without explicitly controlling its orientation. This paper presents a novel sketch‐based method for modelling and simulating many steering behaviors for agents in a crowd. Central to this is the concept of aninteraction field(IF): a vector field that describes the velocities or orientations that agents should use around a given ‘source’ agent or obstacle. An IF can also change dynamically according to parameters, such as the walking speed of the source agent. IFs can be easily combined with other aspects of crowd simulation, such as collision avoidance. Using an implementation of IFs in a real‐time crowd simulation framework, we demonstrate the capabilities of IFs in various scenarios. This includes game‐like scenarios where the crowd responds to a user‐controlled avatar. We also present an interactive tool that computes an IF based on input sketches. This IF editor lets users intuitively and quickly design new types of behavior, without the need for programming extra behavioral rules. We thoroughly evaluate the efficacy of the IF editor through a user study, which demonstrates that our method enables non‐expert users to easily enrich any agent‐based crowd simulation with new agent interactions.

Non-verbal communication body cues are paramount to interact. In this preliminary work, we explore ways to let Intelligent Virtual Agents (IVAs) simulating nonverbal communication capabilities. We propose an approach to control IVAs' reactive behaviour from the analysis of other agents' apparent motions, in a situation of "observed" IVAs that act and "observers" that react. For that, first a viewpoint-driven analysis of the observed agent's motion is done, and then a synthesis of this analysis induces the observers' reaction.

Virtual human crowds are regularly featured in movies and video games. With a large number of virtual characters each behaving in their own way, spectacular scenes can be produced. The more diverse the characters and their behaviors are, the more realistic the virtual crowd is expected to be perceived. Hence, creating virtual crowds is a trade-off between the cost associated with acquiring more diverse assets, namely more virtual characters with their animations, and achieving better realism. In this paper, our focus is on the perceived variety in virtual crowd character motions. We present an experiment exploring whether observers are able to identify virtual crowds including motion clones in the case of large-scale crowds (from 250 to 1000 characters). As it is not possible to acquire individual motions for such numbers of characters, we rely on a state-of-the-art motion variation approach to synthesize unique variations of existing examples for each character in the crowd. Participants then compared pairs of videos, where each character was animated either with a unique motion or using a subset of these motions. Our results show that virtual crowds with more than two motions (one per gender) were perceptually equivalent, regardless of their size. We believe these findings can help create efficient crowd applications, and are an additional step into a broader understanding of the perception of motion variety.

In this paper, we investigate the effect of a realism mismatch in the voice and appearance of a photorealistic virtual character in virtual reality. While many studies have investigated voice attributes for robots, not much is known about the effect voice naturalness has on the perception of realistic virtual characters. We conducted an experiment in Virtual Reality (VR) with over two hundred participants investigating the mismatch between realistic appearance and unrealistic voice on the feeling of presence, and the emotional response of the user to the character expressing a strong negative emotion (sadness, guilt). We predicted that the mismatched voice would lower social presence and cause users to have a negative emotional reaction and feelings of discomfort towards the character. We found that the concern for the virtual character was indeed altered by the unnatural voice, though interestingly it did not affect social presence.

Blendshape facial rigs are used extensively in the industry for facial animation of virtual humans. However, storing and manipulating large numbers of facial meshes (blendshapes) is costly in terms of memory and computation for gaming applications. Blendshape rigs are comprised of sets of semantically-meaningful expressions, which govern how expressive the character will be, often based on Action Units from the Facial Action Coding System (FACS). However, the relative perceptual importance of blendshapes has not yet been investigated. Research in Psychology and Neuroscience has shown that our brains process faces differently than other objects so we postulate that the perception of facial expressions will be feature-dependent rather than based purely on the amount of movement required to make the expression. Therefore, we believe that perception of blendshape visibility will not be reliably predicted by numerical calculations of the difference between the expression and the neutral mesh. In this paper, we explore the noticeability of blendshapes under different activation levels, and present new perceptually-based models to predict perceptual importance of blendshapes. The models predict visibility based on commonly-used geometry and image-based metrics.

Synthesizing motions that look realistic and diverse is a challenging task in animation. Therefore, a few generic walking motions are typically used when creating crowds of walking virtual characters, leading to a lack of variations as motions are not necessarily adapted to each and every virtual character’s characteristics. While some attempts have been made to create variations, it appears necessary to identify the relevant parameters that influence users’ perception of such variations to keep a good trade-off between computational costs and realism. In this paper, we therefore investigate the ability of viewers to identify an invariant parameter of human walking named the Walk Ratio (step length to step frequency ratio), which was shown to be specific to each individual and constant for different speeds, but which has never been used to drive animations of virtual characters. To this end, we captured 2 female and 2 male actors walking at different freely chosen speeds, as well as at different combinations of step frequency and step length. We then performed a perceptual study to identify the Walk Ratio that was perceived as the most natural for each actor when animating a virtual character, and compared it to the Walk Ratio freely chosen by the actor during the motion capture session. We found that Walk Ratios chosen by observers were in the range of Walk Ratios measured in the literature, and that participants perceived differences between the Walk Ratios of animated male and female characters, as evidenced in the biomechanical literature. Our results provide new considerations to drive the animation of walking virtual characters using the Walk Ratio as a parameter, and might provide animators with novel means to control the walking speed of characters through simple parameters while retaining the naturalness of the locomotion.

Blendshape facial rigs are used extensively in the industry for facial animation of virtual humans. However, storing and manipulating large numbers of facial meshes is costly in terms of memory and computation for gaming applications, yet the relative perceptual importance of blendshapes has not yet been investigated. Research in Psychology and Neuroscience has shown that our brains process faces differently than other objects, so we postulate that the perception of facial expressions will be feature-dependent rather than based purely on the amount of movement required to make the expression. In this paper, we explore the noticeability of blendshapes under different activation levels, and present new perceptually based models to predict perceptual importance of blendshapes. The models predict visibility based on commonly-used geometry and image-based metrics.

In human interaction, people will keep different distances from each other depending on their gender. For example, males will stand further away from males and closer to females. Previous studies in virtual reality (VR), where people were interacting with virtual humans, showed a similar result. However, many other variables influence proximity, such as appearance characteristics of the virtual character (e.g., attractiveness). Our study focuses on proximity to virtual walkers, where gender could be recognised from motion only, since previous studies using point-light displays found walking motion is rich in gender cues. In our experiment, a walking wooden mannequin approached the participant embodied in a virtual avatar using the HTC Vive Pro HMD and controllers. The mannequin animation was motion captured from several male and female actors and each motion was displayed individually on the character. Participants used the controller to stop the approaching mannequin when they felt it was uncomfortably close to them. Based on previous work, we hypothesised that proximity will be affected by the gender of the character, but unlike previous research, the gender in our experiment could only be determined from character’s motion. We also expected differences in proximity according to the gender of the participant. We additionally expected some motions to be rated more attractive than others and that attractive motions would reduce the proximity measure. Our results show support for the last two assumptions, but no difference in proximity was found according to the gender of the character’s motion. Our findings have implications for the design of virtual characters in interactive virtual environments.

Lighting has been used to enhance emotion and appeal of characters for centuries, from paintings in the Renaissance to the modern-day digital arts. In VFX and animation studios, lighting is considered as important as modelling, shading, or rigging. Most existing work focuses on either empirical best-practice created by artists of the centuries or on lighting perception with basic shapes. In contrast, our work focuses on the effect of lighting on emotional characters. Our study presents an extensive set of novel perceptual experiments designed to investigate the effects of brightness levels (key light brightness) and the proportion of light intensity illuminating the two sides of a character’s face (key-to-fill ratio). We are particularly interested in the effect of lighting on the recognition of emotion, emotion intensity, and the overall appeal, as these are crucial factors for audience engagement. Our results have implications for artists and developers wishing to increase the appeal and emotional expression of their characters, ranging from cartoon to realistic styles. Our key finding is that lighting can be used to effectively alter the intensity of emotion of a character and that brighter conditions increased appeal across all of our experiments.

Lighting is a commonly used tool to manipulate the appearance of virtual characters in a range of applications. However, there are few studies which systematically examine the effect of lighting changes on complex dynamic stimuli. Our study presents several perceptual experiments, designed to investigate the ability of participants to discriminate lighting levels and the ratio of light intensity projected on the two sides of a cartoon character’s face (key-to-fill ratio) in portrait lighting design. We used a standard psychophysical method for measuring discrimination, typical in low-level perceptual studies but not frequently considered for evaluating complex stimuli. We found that people can easily differentiate lighting intensities, and distinguish between shadow strength and scene brightness under bright conditions but not under dark conditions. We provide a model of the results, and empirically validate the predictions of the model. We discuss the practical implications of our results and how they can be exploited to make the process of portrait lighting for CG cartoon characters more consistent, such as a tool for manipulating shadow while maintaining the level of perceived brightness.

Photorealism of virtual characters and environments is becoming more achievable in Virtual Reality (VR). With this development comes the need for further investigation into the role it plays on people’s responses to characters. Whether or not these improvements make any difference to the perception and response towards the virtual character was the central question of the present study. In order to evaluate this, we designed a within-subjects experiment, where participants were embodied in a high-fidelity virtual body in VR and were observing an animated character, rendered in photorealistic and simplified style. The character displayed a simple interactive behaviour with the participant (eye-gaze) and was designed to express an emotional reaction to induce an empathetic response in participants. Our goal was to evaluate if photorealism alone is enough to increase self-reported and behavioural signs (interpersonal distance or proximity) of social presence, place illusion, and empathetic concern for the character in virtual reality. This was found to be the case for self-reported social presence and place illusion, while empathetic concern depended on the order of condition. behavioural measure proximity was not affected by render style.

In recent years, the quality of real-time rendering has reached new heights—realistic reflections, physically based materials, and photometric lighting are all becoming commonplace in modern game engines and even interactive virtual environments, such as virtual reality (VR). As the strive for realism continues, there is a need to investigate the effect of photorealism on users’ perception, particularly for interactive, emotional scenarios in VR. In this article, we explored three main topics, where we predicted photorealism will make a difference: the illusion of being present with the virtual person and in an environment, altered emotional response toward the character, and a subtler response—comfort of being in close proximity to the character. We present a perceptual experiment, with an interactive expressive virtual character in VR, which was designed to induce particular social responses in people. Our participant pool was large (N = 797) and diverse in terms of demographics. We designed a between-group experiment, where each group saw either the realistic rendering or one of our stylized conditions (simple and sketch style), expressing one of three attitudes: Friendly, Unfriendly, or Sad. While the render style did not particularly effect the level of comfort with the character or increase the illusion of presence with it, our main finding shows that the photorealistic character changed the emotional responses of participants, compared to the stylized versions. We also found a preference for realism in VR, reflected in the affinity and higher place illusion in the scenario, rendered in the realistic render style.

Virtual characters that appear almost photo-realistic have been shown to induce negative responses from viewers in traditional media, such as film and video games. This effect, described as the uncanny valley, is the reason why realism is often avoided when the aim is to create an appealing virtual character. In Virtual Reality, there have been few attempts to investigate this phenomenon and the implications of rendering virtual characters with high levels of realism on user enjoyment. In this paper, we conducted a large-scale experiment on over one thousand members of the public in order to gather information on how virtual characters are perceived in interactive virtual reality games. We were particularly interested in whether different render styles (realistic, cartoon, etc.) would directly influence appeal, or if a character's personality was the most important indicator of appeal. We used a number of perceptual metrics such as subjective ratings, proximity, and attribution bias in order to test our hypothesis. Our main result shows that affinity towards virtual characters is a complex interaction between the character's appearance and personality, and that realism is in fact a positive choice for virtual characters in virtual reality.

In recent years, there has been much research and media attention devoted to investigating virtual reality environments. In this paper, we are investigating if there are differences in how characters are perceived in immersive virtual reality as opposed to more common, screen-based environments. We were particularly interested if the spatial and immersive components play an important part in perception of interactive, game-like settings, where characters can either be controlled (avatars) or observed (agents). We focus on the subjective reports on perceived realism, affinity, co-presence and agency. Since appearance of the character is an important component of affinity, we introduced the changes in render style, ranging in three realism levels, to test if appearance would even further influence the perception in relation to control condition and platform. Furthermore, we adapted a behavioural method (proximity task) as a novel approach to establishing if behavioural changes could be recorded based on the introduced conditions and compared those values with the subjective reports of the participants. The conclusions have an important value to character design specific to platform and character control.

Graphical realism and the naturalness of the voice used are important aspects to consider when designing a virtual agent or character.In this work, we evaluate how synthetic speech impacts people's perceptions of a rendered virtual character.Using a controlled experiment, we focus on the role that speech, in particular voice expressiveness in the form of personality, has on the assessment of voice level and character level perceptions.We found that people rated a real human voice as more expressive, understandable and likeable than the expressive synthetic voice we developed.Contrary to our expectations, we found that the voices did not have a significant impact on the character level judgments; people in the voice conditions did not significantly vary on their ratings of appeal, credibility, humanlikeness and voice matching the character.The implications this has for character design and how this compares with previous work are discussed.

In this paper, we conducted a perceptual experiment to determine if specific personality traits can be portrayed through eye and head movement in the absence of other facial animation cues. We created a collection of eye and head motions captured from three female actors portraying different personalities, while listening to instructional videos. In a between-groups experiment, we tested the perception of personality on a realistic model and a cartoon stylisation in order to determine if stylisation can positively influence the perceived personality or if personality is more easily identified on a realistic face. Our results verify that participants were able to differentiate between personality traits portrayed only through eye gaze, blinks and head movement. The results also show that perception of personality was robust across character realism.

Despite the elusive term "Uncanny Valley", research in the area of appealing virtual humans approaching realism continues. The theory suggests that characters lose appeal when they approach photorealism (e.g., [MacDorman et al. 2009]). Realistic virtual characters are judged harshly, since the human visual system has acquired more expertise with the featural restrictions of other humans than with the restrictions of artificial characters [Seyama and Nagayama 2007]. Stylisation (making the character's appearance abstract) is therefore often used to avoid virtual characters to be perceived as unpleasant. We designed an experiment to test if there is a general affinity towards abstract as oppose to realistic characters.

In this article, we investigate the perception of gender from the motion of virtual humans under different emotional conditions and explore the effect of emotional bias on gender perception (e.g., anger being attributed to males more than females). As motion types can present different levels of physiological cues, we also explore how two types of motion (walking and conversations) are affected by emotional bias. Walking typically displays more physiological cues about gender (e.g., hip sway) and therefore is expected to be less affected by emotional bias. To investigate these effects, we used a corpus of captured facial and body motions from four male and four female actors, performing basic emotions through conversation and walk. We expected that the appearance of the model would also influence gender perception; therefore, we displayed both male and female motions on two virtual models of different sex. Two experiments were then conducted to assess gender judgments from these motions. In both experiments, participants were asked to rate how male or female they considered the motions to be under different emotional states, then classified the emotions to determine how accurately they were portrayed by actors. Overall, both experiments showed that gender ratings were affected by the displayed emotion. However, we found that conversations were influenced by gender stereotypes to a greater extent than walking motions. This was particularly true for anger, which was perceived as male on both male and female motions, and sadness, which was perceived as less male when portrayed by male actors. We also found a slight effect of the model when observing gender on different types of virtual models. These results have implications for the design and animation of virtual humans.

Virtual characters contribute strongly to the entire visuals of 3D animated films. However, designing believable characters remains a challenging task. Artists rely on stylization to increase appeal or expressivity, exaggerating or softening specific features. In this paper we analyze two of the most influential factors that define how a character looks: shape and material. With the help of artists, we design a set of carefully crafted stimuli consisting of different stylization levels for both parameters, and analyze how different combinations affect the perceived realism, appeal, eeriness, and familiarity of the characters. Moreover, we additionally investigate how this affects the perceived intensity of different facial expressions (sadness, anger, happiness, and surprise). Our experiments reveal that shape is the dominant factor when rating realism and expression intensity, while material is the key component for appeal. Furthermore our results show that realism alone is a bad predictor for appeal, eeriness, or attractiveness.

Delivering appealing virtual characters conveying personality is becoming extremely important in the entertainment industry and beyond. A theory called the 'Uncanny Valley' has been used to describe the phenomenon that the appearance of a virtual character can contribute to negative/positive audience reactions to that character [Mori 1970]. Since the style used to render a character strongly changes the appearance, we investigate whether a difference in render style can indirectly influence audience reaction, which we measure based on perception of personality. Based on psychology research, we first scripted original character dialogues in order to convey a range of ten typical personality types. Then, a professional actor was recruited to act out these dialogues, while his face and body motion and audio were recorded. The performances were mapped onto a virtual character rendered in two styles that differ in appearance: an appealing cartoon style and unappealing ill style (Figure 1). In our experiment, participants were asked questions about the character's personality in order for us to test if the difference in render style causes differences in personality perception. Our results found an indirect effect of render style where the cartoon style was rated as having a more agreeable personality than the ill style. This result has implications for developers interested in creating appealing virtual humans, avoiding the 'Uncanny Valley' phenomenon.

In this paper, we investigate the ability of humans to determine the gender of conversing characters, based on facial and body cues for emotion. We used a corpus of simultaneously captured facial and body motions from four male and four female actors. In our Gender Rating task, participants were asked to rate how male or female they considered the motions to be, under different emotional states. In our Emotion Recognition task, participants were asked to classify the emotions, in order to determine how accurately perceived those emotions were. We found that gender perception was affected by emotion, where certain emotions facilitated gender determination while others masked it. We also found that there was no correlation between how accurate an emotion was portrayed and how much gender information was present in that motion. Finally, we found that the model used to display the motion did not affect gender perception of motion but did alter emotion recognition.

Recent advances in rendering and data-driven animation have enabled the creation of compelling characters with impressive levels of realism. While data-driven techniques can produce animations that are extremely faithful to the original motion, many challenging problems remain because of the high complexity of human motion. A better understanding of the factors that make human motion recognizable and appealing would be of great value in industries where creating a variety of appealing virtual characters with realistic motion is required. To investigate these issues, we captured thirty actors walking, jogging and dancing, and applied their motions to the same virtual character (one each for the males and females). We then conducted a series of perceptual experiments to explore the distinctiveness and attractiveness of these human motions, and whether characteristic motion features transfer across an individual's different gaits. Average faces are perceived to be less distinctive but more attractive, so we explored whether this was also true for body motion. We found that dancing motions were most easily recognized and that distinctiveness in one gait does not predict how recognizable the same actor is when performing a different motion. As hypothesized, average motions were always amongst the least distinctive and most attractive. Furthermore, as 50% of participants in the experiment were Caucasian European and 50% were Asian Korean, we found that the latter were as good as or better at recognizing the motions of the Caucasian actors than their European counterparts, in particular for dancing males, whom they also rated more highly for attractiveness.