Teleportation is one of the most common locomotion techniques in virtual reality, yet its perceptual properties remain underexplored. While redirected walking research has shown that users’ movements can be subtly manipulated without detection, similar imperceptible adjustments for teleportation have not been systematically investigated. This study examines the thresholds at which teleportation displacements become noticeable to users. We conducted a repeated-measures experiment in which participants’ selected teleport destinations were altered in both direction (forwards, backwards) and at different ranges (small, large). Detection thresholds for these positional adjustments were estimated using a psychophysical staircase method with a two-alternative forced choice (2AFC) task.Results show that teleport destinations can be shifted without detection, with larger tolerances for backward adjustments and across longer teleport ranges. These findings establish baseline perceptual limits for redirected teleportation and highlight its potential as a design technique. Applications include supporting interpersonal distance management in social VR, guiding players toward objectives in games, and assisting novice users with navigation. By identifying the limits of imperceptible teleportation adjustments, this work extends redirection principles beyond walking to teleportation and opens new opportunities for adaptive and socially aware VR locomotion systems.

Mixed Reality (MR) presents novel opportunities to investigate how individuals perceive themselves and others during shared, augmented experiences within a common physical environment. Previous research has demonstrated that users can embody avatars in MR, temporarily extending their sense of self. However, there has been limited exploration of body-swapping, a condition in which two individuals simultaneously inhabit each other's avatars, and its potential effects on social interaction in immersive environments. To address this gap, we adapted the Joint Simon Task (JST), a wellestablished implicit paradigm, to examine how body-swapping influences the cognitive and perceptual boundaries between self and other. Our results indicate that body-swapping led participants to experience themselves and their partner as functioning like a single, unified system, as in two bodies operating as one agent. This suggests possible cognitive and perceptual changes that go beyond simple collaboration. Our findings have significant implications for the design of MR systems intended to support collaboration, empathy, social learning, and therapeutic interventions through shared embodiment.

Sound auralisation in Immersive Virtual Environments (IVEs) has been shown to promote perceptual effects such as presence and accounting for the technical limitations of immersive technologies. Moreover, it has been found to influence perceived audio-visual quality in multisensory contexts. However, this has been examined with sounds produced by an external source within the environment, rather than by the user, such as their own voice. In this study, we examined how real-time first-person vocal auralisation influences the perceived sense of presence and audio-visual quality across different rendering resolutions and visual representations, including Virtual Reality (VR) and Mixed Reality (MR). Experiment 1 compared three levels of sensory feedback. Visual feedback considered full resolution ($\sim 4.6$megapixels per eye), low resolution ($\sim 1.0$megapixels per eye), and no visuals. Audio feedback included two auralisation conditions using measured and synthesised impulse responses and a condition with no auralisation. Experiment 2 explored different types of visual representation: MR, using the passthrough feature of the Head-Mounted Display (HMD) to render the physical environment, and VR, using a 3D modelled reconstruction of the space. Audio feedback conditions consisted of auralisation with measured impulse responses and no auralisation. Our findings consistently revealed higher ratings of presence and perceived audio-visual quality in conditions where real-time vocal auralisation was present, highlighting the importance of rendering the acoustic features of spaces for first-person vocal interaction in immersive experiences.

The creation of virtual humans increasingly leverages automated synthesis of speech and gestures, enabling expressive, adaptable agents that effectively engage users. However, the independent development of voice and gesture generation technologies, alongside the growing popularity of virtual reality (VR), presents significant questions about the integration of these signals and their ability to convey emotional detail in immersive environments. In this paper, we evaluate the influence of real and synthetic gestures and speech, alongside varying levels of immersion (VR vs. 2D displays) and emotional contexts (positive, neutral, negative) on user perceptions. We investigate how immersion affects the perceived match between gestures and speech and the impact on key aspects of user experience, including emotional and empathetic responses and the sense of co-presence. Our findings indicate that while VR enhances the perception of natural gesture–voice pairings, it does not similarly improve synthetic ones—amplifying the perceptual gap between them. These results highlight the need to reassess gesture appropriateness and refine AI-driven synthesis for immersive environments.

This survey reviews the use of computational topology in hand-drawn animation technology over the past 15 years. We discuss three main subfields of hand-drawn animation technology research: frame deformation, frame feature correspondence, and volumetric modeling of hand-drawn characters. We explore the various topological spaces and operators applied to each subfield, detailing the artistic and computational advantages and limitations of each topological approach. Throughout our discussion, we provide industry knowledge derived from interviews with leading hand-drawn animation professionals to enrich our assessments of topological approaches’ suitability in industry settings.

As social VR grows in popularity, understanding how to optimise interactions becomes increasingly important. Interpersonal distance-the physical space people maintain between each other-is a key aspect of user experience. Previous work in psychology has shown that breaches of personal space cause stress and discomfort. Thus, effectively managing this distance is crucial in social VR, where social interactions are frequent. Teleportation, a commonly used locomotion method in these environments, involves distinct cognitive processes and requires users to rely on their ability to estimate distance. Despite its widespread use, the effect of teleportation on proximity remains unexplored. To investigate this, we measured the interpersonal distance of 70 participants during interactions with embodied conversational agents, comparing teleportation to natural walking. Our findings revealed that participants maintained closer proximity from the agents during teleportation. Female participants kept greater distances from the agents than male participants, and natural walking was associated with higher agency and body ownership, though co-presence remained unchanged. We propose that differences in spatial perception and spatial cognitive load contribute to reduced interpersonal distance with teleportation. These findings emphasise that proximity should be a key consideration when selecting locomotion methods in social VR, highlighting the need for further research on how locomotion impacts spatial perception and social dynamics in virtual environments.

Lighting has long been recognized as a crucial element in enhancing the appeal and emotional depth of characters across different media formats. This study investigates the impact of well-established portrait lighting designs on the perception of realistic digital human characters. We provide insight into how real-world lighting styles can be used to enhance the perceived appeal, trustworthiness, and emotional authenticity of digital human characters while minimizing the sense of eeriness. The findings of this study are expected to contribute to the understanding of how lighting influences the perception of digital characters and inform the development of more engaging virtual experiences in various domains.

Many VR animal embodiment sytsems suffer from poor animation fidelity, typically animating the animal avatars using inverse kinematics. We address this issue, presenting a novel deep-learning method, centred around a shared codebook, for mapping human motion to quadruped motion. Rather than trying to directly bridge the gap from human motion to quadruped motion, a task which has proven difficult, we first use a rule-based retargeter, relying on inverse and forward kinematics, to retarget human motions to an intermediate motion domain in which the motions share the same skeleton as the quadruped. We then use finite scalar quantization to construct a shared latent space, or codebook, between this intermediate domain and the quadruped motion domain. We do this by first pre-defining a finite number of discrete latent codes and then teaching these codes, using unsupervised deep-learning, to represent semantically similar motions in the two domains. We incorporate our real-time human-to-quadruped motion mapping into a VR quadruped embodiment system. The output quadruped animations are natural and realistic, while also preserving the semantics of users’ actions. Moreover, there is a strong synchrony between the input human motions and retargeted quadruped motions, an important factor for inducing a strong sense of VR embodiment.

Taking inspiration from recent developments in visual generative tasks using diffusion models, we propose a method for end-to-end speech-driven video editing using a denoising diffusion model. Given a video of a talking person, and a separate auditory speech recording, the lip and jaw motions are re-synchronised without relying on intermediate structural representations such as facial landmarks or a 3D face model. We show this is possible by conditioning a denoising diffusion model on audio mel spectral features to generate synchronised facial motion. Proof of concept results are demonstrated on both single-speaker and multi-speaker video editing, providing a baseline model on the CREMA-D audiovisual data set. To the best of our knowledge, this is the first work to demonstrate and validate the feasibility of applying end-to-end denoising diffusion models to the task of audio-driven video editing. All code, datasets, and models used as part of this work are made publicly available here: https://danbigioi.github.io/DiffusionVideoEditing/.

Multisensory integration influences emotional perception, as the McGurk effect demonstrates for the communication between humans. Human physiology implicitly links the production of visual features with other modes like the audio channel: Face muscles responsible for a smiling face also stretch the vocal cords that result in a characteristic smiling voice. For artificial agents capable of multimodal expression, this linkage is modeled explicitly. In our studies, we observe the influence of visual and audio channels on the perception of the agents' emotional expression. We created videos of virtual characters and social robots either with matching or mismatching emotional expressions in the audio and visual channels. In two online studies, we measured the agents' perceived valence and arousal. Our results consistently lend support to the ‘emotional McGurk effect' hypothesis, according to which face transmits valence information, and voice transmits arousal. When dealing with dynamic virtual characters, visual information is enough to convey both valence and arousal, and thus audio expressivity need not be congruent. When dealing with robots with fixed facial expressions, however, both visual and audio information need to be present to convey the intended expression.

With the remarkable improvement in technical systems for generating realistic virtual humans, there comes a requirement to quantify the effects that different aspects of realism can have on users. The study outlined here sought to advance research on emotion perception and virtual humans by assessing basic empathic responses to high fidelity emotionally expressive characters. We report findings on participants experiences of cognitive, affective and compassionate empathy, as well as measurements for the uncanny valley at two levels of visual realism. We find that the levels of emotion expressed by virtual humans within our study influenced ratings of empathy and the uncanny valley, with positive and negative valence having significant effects on empathic responses and perceived appeal. We discuss these findings in relation to studies which have measured empathy responses to animated characters, as well as notable differences in uncanny valley measurements.

Embodying a virtual creature or animal in Virtual Reality (VR) is becoming common, and can have numerous beneficial impacts. For instance, it can help actors improve their performance of a computer-generated creature, or it can endow the user with empathy towards threatened animal species. However, users must feel a sense of embodiment towards their virtual representation, commonly achieved by providing congruent sensory feedback. Providing effective visuo-motor feedback in dysmorphic bodies can be challenging due to human-animal morphology differences. Thus, the purpose of this study was to experiment with the inclusion of audio and audio-tactile feedback to begin unveiling their influence towards animal avatar embodiment. Two experiments were conducted to examine the effects of different sensory feedback on participants’ embodiment in a dog avatar in an Immersive Virtual Environment (IVE). The first experiment (n= 24) included audio, tactile, audio-tactile, and baseline conditions. The second experiment (n= 34) involved audio and baseline conditions only.

Virtual reality (VR) has emerged as a promising technology for remote teamwork; it allows individuals to collaborate in real-time via virtual avatars that convey body language and other nonverbal cues not available via traditional communication mechanisms. However, there is not much known about how the characteristics of these avatars affect the social interactions taking place in these environments. To bridge this gap, we investigate how avatar clothing influences group dynamics during team collaboration in VR. Particularly, we observe the difference between team dynamics when groups are dressed formally and casually. We asked groups of participants to report on their personality, complete two decision-making tasks while wearing different clothing, and report on their perceptions of work group inclusion and entitativity afterwards. We found that clothing had no influence on inclusion and entitativity, but personality did.

Emotional expressivity is essential for successful Human-Robot Interaction. However, robots often have different levels of expressivity in their face and voice. Here we ask whether this modality mismatch influences human behaviour and perception of the robot. Participants played a cooperative task with a robot that displayed matched and mismatched smiling expressions in the face and voice. Emotional expressivity did not influence acceptance of robot’s recommendations or subjective evaluations of the robot. However, we found that the robot had overall a higher social influence than a virtual character, and was evaluated more positively.

The portrayed personality of virtual characters and agents is understood to influence how we perceive and engage with digital applications. Understanding how the features of speech and animation drive portrayed personality allows us to intentionally design characters to be more personalized and engaging. In this study, we use performance capture data of unscripted conversations from a variety of actors to explore the perceptual outcomes associated with the modalities of speech and motion. Specifically, we contrast full performance-driven characters to those portrayed by generated gestures and synthesized speech, analysing how the features of each influence portrayed personality according to the Big Five personality traits. We find that processing speech and motion can have mixed effects on such traits, with our results highlighting motion as the dominant modality for portraying extraversion and speech as dominant for communicating agreeableness and emotional stability. Our results can support the Extended Reality (XR) community in development of virtual characters, social agents and 3D User Interface (3DUI) agents portraying a range of targeted personalities.

In this paper, we investigate the effect of a realism mismatch in the voice and appearance of a photorealistic virtual character in both immersive and screen-mediated virtual contexts. While many studies have investigated voice attributes for robots, not much is known about the effect voice naturalness has on the perception of realistic virtual characters. We conducted the first experiment in Virtual Reality (VR) with over two hundred participants investigating the mismatch between realistic appearance and unrealistic voice on the feeling of presence, and the emotional response of the user to the character expressing a strong negative emotion. We predicted that the mismatched voice would lower social presence and cause users to have a negative emotional reaction and feelings of discomfort towards the character. We found that the concern for the virtual character was indeed altered by the unnatural voice, though interestingly it did not affect social presence. The second experiment was conducted with a view towards heightening the appearance realism of the same character for the same scenarios, with an additional lower level of voice realism employed to strengthen the mismatch of perceptual cues. While voice type did not appear to impact reports of empathic responses towards the character, there was an observed effect of voice realism on reported social presence, which was not detected in the first study. There were also significant results on affinity and voice trait measurements that provide evidence in support of perceptual mismatch theories of the Uncanny Valley.

Abstract An abundance of older, as well as recent work exists at the intersection of computer vision and computer graphics on accurate estimation of dynamic facial landmarks with applications in facial animation, emotion recognition, and beyond. However, only a few publications exist that optimize the actual layout of facial landmarks to ensure an optimal trade‐off between compact layouts and detailed capturing. At the same time, we observe that applications like social games prefer simplicity and performance over detail to reduce the computational budget especially on mobile devices. Other common attributes of such applications are predefined low‐dimensional models to animate and a large, diverse user‐base. In contrast to existing methods that focus on creating person‐specific facial landmarks, we suggest to derive application‐specific facial landmarks. We formulate our optimization method on the widely adopted blendshape model. First, a score is defined suitable to compute a characteristic landmark for each blendshape. In a following step, we optimize a global function, which mimics merging of similar landmarks to one. The optimization is solved in less than a second using integer linear programming and guarantees a globally optimal solution to an NP‐hard problem. Our application‐specific approach is faster and fundamentally different to previous, actor‐specific methods. Resulting layouts are more similar to empirical layouts. Compared to empirical landmarks, our layouts require only a fraction of landmarks to achieve the same numerical error when reconstructing the animation from landmarks. The method is compared against previous work and tested on various blendshape models, representing a wide spectrum of applications.

Multisensory integration influences emotional perception, as the McGurk effect demonstrates for the communication between humans. Human physiology implicitly links the production of visual features with other modes like the audio channel: Face muscles responsible for a smiling face also stretch the vocal cords that results in a characteristic smiling voice. For artificial agents capable of multimodal expression, this linkage is modeled explicitly. In our study, we observe the influence of visual and audio channel on the perception of the agent’s emotional state. We created two virtual characters to control for anthropomorphic appearance. We record videos of these agents either with matching or mismatching emotional expression in the audio and visual channel. In an online study we measured the agent’s perceived valence and arousal. Our results show that a matched smiling voice and smiling face increase both dimensions of the Circumplex model of emotions: ratings of valence and arousal grow. When the channels present conflicting information, any type of smiling results in higher arousal rating, but only the visual channel increases the perceived valence. When engineers are constrained in their design choices, we suggest they should give precedence to convey the artificial agent’s emotional state through the visual channel.

Social virtual reality uses motion tracking to place people in virtual environments as animated avatars. Often this tracking only measures the position and orientation of the head and hands, and from this estimates the body pose. Optical hand tracking is an important technology to enable such avatars, but can frequently fail and cause motion errors when the hands are visually obscured. This paper presents three amelioration strategies to handle these errors and demonstrates experimentally that all three are effective in reducing their impact. This setting is also used to explore general issues around study design for motion perception. Different strategies for presenting stimuli and soliciting input are compared. The presence of a simultaneous recall task is shown to reduce but not eliminate sensitivity to motion errors. Finally, it is shown that motion errors are interpreted, at least in part, as a shift in interlocutor personality.

Advancements in real-time rendering technology have continued to develop rapidly over the course of the last decade. Consequently, human likenesses have been represented virtually with increasingly impressive detail. There is evidence that this increased resemblance to real humans has an observable and wide-ranging set of effects on human perception, cognition and action in situations that involve digital characters. Studies that seek to advance the science of synthetic animated people have consistently aimed to measure and quantify changes in perceived emotional content mediated through artificial human likenesses. The present study has been built off of this work. The experiment outlined here was constructed to define and measure responses from human participants towards state-of-the-art photorealistic virtual humans under affected conditions. In particular, we sought evidence for changes in perceptions of human likeness, eeriness and attractiveness that could be observably dependent on conditions of photorealism and character representation.

The main use of Augmented Reality (AR) today for the general public is in applications for smartphones. In particular, social network applications allow the use of many AR filters, modifying users’ environments but also their own image. These AR filters are increasingly and frequently being used and can distort in many ways users’ facial traits. Yet, we still do not know clearly how users perceive their faces augmented by these filters. In this paper, we present a study that aims to evaluate the impact of different filters, modifying several facial features such as the size or position of the eyes, the shape of the face or the orientation of the eyebrows, or adding virtual content such as virtual glasses. These filters are evaluated via a self-evaluation questionnaire, asking the participants about the personality, emotion, appeal and intelligence traits that their distorted face conveys. Our results show relative effects between the different filters in line with previous results regarding the perception of others. However, they also reveal specific effects on self-perception, showing, inter alia, that facial deformation decreases participants’ credence towards their image. The findings of this study covering multiple factors allow us to highlight the impact of face deformation on user perception but also the specificity related to this use in AR, paving the way for new works focusing on the psychological impact of such filters.

Research on creation of virtual humans enables increasing automatization of their behavior, including synthesis of verbal and nonverbal behavior. As the achievable realism of different aspects of agent design evolves asynchronously, it is important to understand if and how divergence in realism between behavioral channels can elicit negative user responses. Specifically, in this work, we investigate the question of whether autonomous virtual agents relying on synthetic text-to-speech voices should portray a corresponding level of realism in the non-verbal channels of motion and visual appearance, or if, alternatively, the best available realism of each channel should be used. In two perceptual studies, we assess how realism of voice, motion, and appearance influence the perceived match of speech and gesture motion, as well as the agent's likability and human-likeness. Our results suggest that maximizing realism of voice and motion is preferable even when this leads to realism mismatches, but for visual appearance, lower realism may be preferable. (A video abstract can be found at https://youtu.be/arfZZ-hxD1Y.)

Creating realistic quadruped animations is challenging. Producing realistic animations using methods such as key-framing is time consuming and requires much artistic expertise. Alternatively, motion capture methods have their own challenges (getting the animal into a studio, attaching motion capture markers, and getting the animal to put on the desired performance) and the resulting animation will still most likely require cleaning up. It would be useful if an animator could provide an initial rough animation and in return be given a corresponding high quality realistic one. To this end, we present a deep-learning approach for the automatic enhancement of quadruped animations. Given an initial animation, possibly lacking the subtle details of true quadruped motion and/or containing small errors, our results show that it is possible for a neural network to learn how to add these subtleties and correct errors to produce an enhanced animation while preserving the semantics and context of the initial animation. Our work also has potential uses in other applications, for example, its ability to be used in real-time means it could form part of a quadruped embodiment system.

In this paper, we investigate the effect of a realism mismatch in the voice and appearance of a photorealistic virtual character in virtual reality. While many studies have investigated voice attributes for robots, not much is known about the effect voice naturalness has on the perception of realistic virtual characters. We conducted an experiment in Virtual Reality (VR) with over two hundred participants investigating the mismatch between realistic appearance and unrealistic voice on the feeling of presence, and the emotional response of the user to the character expressing a strong negative emotion (sadness, guilt). We predicted that the mismatched voice would lower social presence and cause users to have a negative emotional reaction and feelings of discomfort towards the character. We found that the concern for the virtual character was indeed altered by the unnatural voice, though interestingly it did not affect social presence.

Blendshape facial rigs are used extensively in the industry for facial animation of virtual humans. However, storing and manipulating large numbers of facial meshes (blendshapes) is costly in terms of memory and computation for gaming applications. Blendshape rigs are comprised of sets of semantically-meaningful expressions, which govern how expressive the character will be, often based on Action Units from the Facial Action Coding System (FACS). However, the relative perceptual importance of blendshapes has not yet been investigated. Research in Psychology and Neuroscience has shown that our brains process faces differently than other objects so we postulate that the perception of facial expressions will be feature-dependent rather than based purely on the amount of movement required to make the expression. Therefore, we believe that perception of blendshape visibility will not be reliably predicted by numerical calculations of the difference between the expression and the neutral mesh. In this paper, we explore the noticeability of blendshapes under different activation levels, and present new perceptually-based models to predict perceptual importance of blendshapes. The models predict visibility based on commonly-used geometry and image-based metrics.

Abstract Co‐speech gestures are a vital ingredient in making virtual agents more human‐like and engaging. Automatically generated gestures based on speech‐input often lack realistic and defined gesture form. We present a database‐driven approach guaranteeing defined gesture form. We built a large corpus of over 23,000 motion‐captured co‐speech gestures and select individual gestures based on expressive gesture characteristics that can be estimated from speech audio. The expressive parameters are gesture velocity and acceleration, gesture size, arm swivel, and finger extension. Individual, parameter‐matched gestures are then combined into animated sequences. We evaluate our gesture generation system in two perceptual studies. The first study compares our method to the ground truth gestures as well as mismatched gestures. The second study compares our method to five current generative machine learning models. Our method outperformed mismatched gesture selection in the first study and showed competitive performance in the second.

Previous perceptual studies on human faces have shown that specific facial features have consistent effects on perceived personality and appeal, but it remains unclear if and how findings relate to perception of virtual characters. For example, wider human faces have been found to appear more aggressive and dominant, whereas studies on virtual characters have shown opposite trends but have suffered from significant eeriness of exaggerated features. In this study, we use highly realistic virtual faces obtained from 3D scanning, as well as cartoon-rendered counterparts retaining facial proportions. We assess the effects of facial width and eye size on perceptions of appeal, trustworthiness, aggressiveness, dominance, and eeriness. Our manipulations did not affect eeriness, and we find the same perceptual trends previously reported for human faces.

Emotional expressivity is essential for human interactions, informing both perception and decision-making. Here, we examine whether creating an audio-visual emotional channel mismatch influences decision-making in a cooperative task with a virtual character. We created a virtual character that was either congruent in its emotional expression (smiling in the face and voice) or incongruent (smiling in only one channel). People (N = 98) evaluated the character in terms of valence and arousal in an online study; then, visitors in a museum played the “lunar survival task” with the character over three experiments (N = 597, 78, 101, respectively). Exploratory results suggest that multi-modal expressions are perceived, and reacted upon, differently than unimodal expressions, supporting previous theories of audio-visual integration.

The demand for cartoon animation content is on the rise, driven largely by social media, video conferencing and virtual reality. New content creation tools and the availability of open-source game engines allow characters to be animated by even novice creators (e.g., Loomie and Pinscreen for creating characters, Hyprface and MotionX for facial motion capture, etc,). However, lighting of animated characters is a skilled art-form, and there is a lack of guidelines for lighting characters expressing emotion in an appealing manner.

Gesture behavior is a natural part of human conversation. Much work has focused on removing the need for tedious hand-animation to create embodied conversational agents by designing speech-driven gesture generators. However, these generators often work in a black-box manner, assuming a general relationship between input speech and output motion. As their success remains limited, we investigate in more detail how speech may relate to different aspects of gesture motion. We determine a number of parameters characterizing gesture, such as speed and gesture size, and explore their relationship to the speech signal in a two-fold manner. First, we train multiple recurrent networks to predict the gesture parameters from speech to understand how well gesture attributes can be modeled from speech alone. We find that gesture parameters can be partially predicted from speech, and some parameters, such as path length, being predicted more accurately than others, like velocity. Second, we design a perceptual study to assess the importance of each gesture parameter for producing motion that people perceive as appropriate for the speech. Results show that a degradation in any parameter was viewed negatively, but some changes, such as hand shape, are more impactful than others. A video summarization can be found at https://youtu.be/aw6-_5kmLjY.

Blendshape facial rigs are used extensively in the industry for facial animation of virtual humans. However, storing and manipulating large numbers of facial meshes is costly in terms of memory and computation for gaming applications, yet the relative perceptual importance of blendshapes has not yet been investigated. Research in Psychology and Neuroscience has shown that our brains process faces differently than other objects, so we postulate that the perception of facial expressions will be feature-dependent rather than based purely on the amount of movement required to make the expression. In this paper, we explore the noticeability of blendshapes under different activation levels, and present new perceptually based models to predict perceptual importance of blendshapes. The models predict visibility based on commonly-used geometry and image-based metrics.

Abstract To simplify and accelerate the creation of blendshape rigs, using a template rig is a common procedure, especially during the creation of digital doubles. Blendshape transfer methods facilitate copy and paste functionality of the blendshapes from the template model to the digital double. However, for adequate personalization, such methods require a set of scanned training expressions of the original actor. So far, the semantics of the facial expressions to scan have been defined manually. In contrast, we formulate the semantics of the facial expressions as an integer optimization of the blendshape weights. By combining different blendshapes of the template model, our method creates facial expressions that serve as semantic references during scanning. Our method guarantees to compute as‐few‐as‐possible training expressions with minimal overlap of activated blendshapes. If the number of training expressions is limited, blendshapes are selected based on their power to personalize the resulting blendshapes compared to generic blendshape transfer methods.

In human interaction, people will keep different distances from each other depending on their gender. For example, males will stand further away from males and closer to females. Previous studies in virtual reality (VR), where people were interacting with virtual humans, showed a similar result. However, many other variables influence proximity, such as appearance characteristics of the virtual character (e.g., attractiveness). Our study focuses on proximity to virtual walkers, where gender could be recognised from motion only, since previous studies using point-light displays found walking motion is rich in gender cues. In our experiment, a walking wooden mannequin approached the participant embodied in a virtual avatar using the HTC Vive Pro HMD and controllers. The mannequin animation was motion captured from several male and female actors and each motion was displayed individually on the character. Participants used the controller to stop the approaching mannequin when they felt it was uncomfortably close to them. Based on previous work, we hypothesised that proximity will be affected by the gender of the character, but unlike previous research, the gender in our experiment could only be determined from character’s motion. We also expected differences in proximity according to the gender of the participant. We additionally expected some motions to be rated more attractive than others and that attractive motions would reduce the proximity measure. Our results show support for the last two assumptions, but no difference in proximity was found according to the gender of the character’s motion. Our findings have implications for the design of virtual characters in interactive virtual environments.

Lighting has been used to enhance emotion and appeal of characters for centuries, from paintings in the Renaissance to the modern-day digital arts. In VFX and animation studios, lighting is considered as important as modelling, shading, or rigging. Most existing work focuses on either empirical best-practice created by artists of the centuries or on lighting perception with basic shapes. In contrast, our work focuses on the effect of lighting on emotional characters. Our study presents an extensive set of novel perceptual experiments designed to investigate the effects of brightness levels (key light brightness) and the proportion of light intensity illuminating the two sides of a character’s face (key-to-fill ratio). We are particularly interested in the effect of lighting on the recognition of emotion, emotion intensity, and the overall appeal, as these are crucial factors for audience engagement. Our results have implications for artists and developers wishing to increase the appeal and emotional expression of their characters, ranging from cartoon to realistic styles. Our key finding is that lighting can be used to effectively alter the intensity of emotion of a character and that brighter conditions increased appeal across all of our experiments.

Lighting is a commonly used tool to manipulate the appearance of virtual characters in a range of applications. However, there are few studies which systematically examine the effect of lighting changes on complex dynamic stimuli. Our study presents several perceptual experiments, designed to investigate the ability of participants to discriminate lighting levels and the ratio of light intensity projected on the two sides of a cartoon character’s face (key-to-fill ratio) in portrait lighting design. We used a standard psychophysical method for measuring discrimination, typical in low-level perceptual studies but not frequently considered for evaluating complex stimuli. We found that people can easily differentiate lighting intensities, and distinguish between shadow strength and scene brightness under bright conditions but not under dark conditions. We provide a model of the results, and empirically validate the predictions of the model. We discuss the practical implications of our results and how they can be exploited to make the process of portrait lighting for CG cartoon characters more consistent, such as a tool for manipulating shadow while maintaining the level of perceived brightness.

Emotional expressivity can boost trust in human-human and human-machine interaction. As a multimodal phenomenon, previous research argued that a mismatch in the expressive channels provides evidence of joint audio-video emotional processing. However, while previous work studied this from the point of view of emotion recognition and processing, not much is known about what effect a multimodal agent would have on a human-agent interaction task. Also, agent appearance could influence this interaction too. Here we manipulated the agent’s multimodal emotional expression (”smiling face” and ”smiling voice”, or both) and agent type (photorealistic or cartoon-like virtual human) and assessed people’s trust toward this agent. We measured trust using a mixed-methods approach, combining behavioural data from a survival task, questionnaire ratings and qualitative comments. These methods gave different results: while people commented on the importance of emotional expressivity in the agent’s voice, this factor had limited influence on trusting behaviours; while people rated the cartoon-like agent on several traits higher than the photorealistic one, the agent’s style also was not the most influential feature on people’s trusting behaviour. These results highlight the contribution of a mixed-methods approach in human-machine interaction, as both explicit and implicit perception and behaviour will contribute to the success of the interaction.

Applications for conversational virtual agents are on the rise, but producing realistic non-verbal behavior for spoken utterances remains an unsolved problem. We explore the use of a generative adversarial training paradigm to map speech to 3D gesture motion. We define the gesture generation problem as a series of smaller sub-problems, including plausible gesture dynamics, realistic joint configurations, and diverse and smooth motion. Each sub-problem is monitored by separate adversaries. For the problem of enforcing realistic gesture dynamics in our output, we train a classifier to automatically detect gesture phases. We find adversarial training to be superior to the use of a standard regression loss and discuss the benefit of each of our training objectives. We recorded a dataset of over 6 hours of natural, unrehearsed speech with high-quality motion capture, as well as audio and video recording.

Photorealism of virtual characters and environments is becoming more achievable in Virtual Reality (VR). With this development comes the need for further investigation into the role it plays on people’s responses to characters. Whether or not these improvements make any difference to the perception and response towards the virtual character was the central question of the present study. In order to evaluate this, we designed a within-subjects experiment, where participants were embodied in a high-fidelity virtual body in VR and were observing an animated character, rendered in photorealistic and simplified style. The character displayed a simple interactive behaviour with the participant (eye-gaze) and was designed to express an emotional reaction to induce an empathetic response in participants. Our goal was to evaluate if photorealism alone is enough to increase self-reported and behavioural signs (interpersonal distance or proximity) of social presence, place illusion, and empathetic concern for the character in virtual reality. This was found to be the case for self-reported social presence and place illusion, while empathetic concern depended on the order of condition. behavioural measure proximity was not affected by render style.

In recent years, the quality of real-time rendering has reached new heights—realistic reflections, physically based materials, and photometric lighting are all becoming commonplace in modern game engines and even interactive virtual environments, such as virtual reality (VR). As the strive for realism continues, there is a need to investigate the effect of photorealism on users’ perception, particularly for interactive, emotional scenarios in VR. In this article, we explored three main topics, where we predicted photorealism will make a difference: the illusion of being present with the virtual person and in an environment, altered emotional response toward the character, and a subtler response—comfort of being in close proximity to the character. We present a perceptual experiment, with an interactive expressive virtual character in VR, which was designed to induce particular social responses in people. Our participant pool was large (N = 797) and diverse in terms of demographics. We designed a between-group experiment, where each group saw either the realistic rendering or one of our stylized conditions (simple and sketch style), expressing one of three attitudes: Friendly, Unfriendly, or Sad. While the render style did not particularly effect the level of comfort with the character or increase the illusion of presence with it, our main finding shows that the photorealistic character changed the emotional responses of participants, compared to the stylized versions. We also found a preference for realism in VR, reflected in the affinity and higher place illusion in the scenario, rendered in the realistic render style.

Correctly interpreting an interlocutor's emotional expression is paramount to a successful interaction. But what happens when one of the interlocutors is a machine? The facilitation of human-machine communication and cooperation is of growing importance as smartphones, autonomous cars, or social robots increasingly pervade human social spaces. Previous research has shown that emotionally expressive virtual characters generally elicit higher cooperation and trust than 'neutral' ones. Since emotional expressions are multi-modal, and given that virtual characters can be designed to our liking in all their components, would a mismatch in the emotion expressed in the face and voice influence people's cooperation with a virtual character? We developed a game where people had to cooperate with a virtual character in order to survive on the moon. The character's face and voice were designed to either smile or not, resulting in 4 conditions: smiling voice and face, neutral voice and face, smiling voice only (neutral face), smiling face only (neutral voice). The experiment was set up in a museum over the course of several weeks; we report preliminary results from over 500 visitors, showing that people tend to trust the virtual character in the mismatched condition with the smiling face and neutral voice more. This might be because the two channels express different aspects of an emotion, as previously suggested.

The growing use of virtual humans demands generating increasingly realistic behavior for them while minimizing cost and time. Gestures are a key ingredient for realistic and engaging virtual agents and consequently automatized gesture generation has been a popular area of research. So far, good gesture generation has relied on explicit formulation of if-then rules and probabilistic modelling of annotated features. Machine learning approaches have yielded only marginal success, indicating a high complexity of the speech-to-motion learning task. In this work, we explore the use of transfer learning using previous motion modelling research to improve learning outcomes for gesture generation from speech. We use a recurrent network with an encoder-decoder structure that takes in prosodic speech features and generates a short sequence of gesture motion. We pre-train the network with a motion modelling task. We recorded a large multimodal database of conversational speech for the purpose of this work.

Human perceptual studies have shown that facial characteristics affect judgments about the personality of a person. For example, larger facial width has been associated with judgments of aggressiveness, dominance, and untrustworthiness. Previous studies of virtual faces have not been able to reflect the same perceptual rules, but have used characters with unrealistic feature sizes or highly abstract characters. For this study, we created virtual characters with realistic feature dimensions and investigated the effects of facial width and eye size on personality perception. Our results indicate that virtual characters may indeed follow different perceptual rules for facial width, and care must be taken when manipulating eye size. These findings are useful for effective character design for video games, movies, and embodied virtual agents.

Virtual characters that appear almost photo-realistic have been shown to induce negative responses from viewers in traditional media, such as film and video games. This effect, described as the uncanny valley, is the reason why realism is often avoided when the aim is to create an appealing virtual character. In Virtual Reality, there have been few attempts to investigate this phenomenon and the implications of rendering virtual characters with high levels of realism on user enjoyment. In this paper, we conducted a large-scale experiment on over one thousand members of the public in order to gather information on how virtual characters are perceived in interactive virtual reality games. We were particularly interested in whether different render styles (realistic, cartoon, etc.) would directly influence appeal, or if a character's personality was the most important indicator of appeal. We used a number of perceptual metrics such as subjective ratings, proximity, and attribution bias in order to test our hypothesis. Our main result shows that affinity towards virtual characters is a complex interaction between the character's appearance and personality, and that realism is in fact a positive choice for virtual characters in virtual reality.

In recent years, there has been much research and media attention devoted to investigating virtual reality environments. In this paper, we are investigating if there are differences in how characters are perceived in immersive virtual reality as opposed to more common, screen-based environments. We were particularly interested if the spatial and immersive components play an important part in perception of interactive, game-like settings, where characters can either be controlled (avatars) or observed (agents). We focus on the subjective reports on perceived realism, affinity, co-presence and agency. Since appearance of the character is an important component of affinity, we introduced the changes in render style, ranging in three realism levels, to test if appearance would even further influence the perception in relation to control condition and platform. Furthermore, we adapted a behavioural method (proximity task) as a novel approach to establishing if behavioural changes could be recorded based on the introduced conditions and compared those values with the subjective reports of the participants. The conclusions have an important value to character design specific to platform and character control.

Graphical realism and the naturalness of the voice used are important aspects to consider when designing a virtual agent or character.In this work, we evaluate how synthetic speech impacts people's perceptions of a rendered virtual character.Using a controlled experiment, we focus on the role that speech, in particular voice expressiveness in the form of personality, has on the assessment of voice level and character level perceptions.We found that people rated a real human voice as more expressive, understandable and likeable than the expressive synthetic voice we developed.Contrary to our expectations, we found that the voices did not have a significant impact on the character level judgments; people in the voice conditions did not significantly vary on their ratings of appeal, credibility, humanlikeness and voice matching the character.The implications this has for character design and how this compares with previous work are discussed.

With the development of increasingly sophisticated computer graphics, there is a continuous growth of the variety and originality of virtual characters used in movies and games. So far, however, their design has mostly been led by the artist’s preferences, not by perceptual studies. In this article, we explored how effective non-player character design can be used to influence gameplay. In particular, we focused on abstract virtual characters with few facial features. In experiment 1, we sought to find rules for how to use a character’s facial features to elicit the perception of certain personality traits, using prior findings for human face perception as a basis. In experiment 2, we then tested how perceived personality traits of a non-player character could influence a player’s moral decisions in a video game. We found that the appearance of the character interacting with the subject modulated aggressive behavior towards a non-present individual. Our results provide us with a better understanding of the perception of abstract virtual characters, their employment in video games, as well as giving us some insights about the factors underlying aggressive behavior in video games.

Studies in the field of social psychology have shown evidence that the dimensions of human facial features can directly impact the perception of personality of that human. Traits such as aggressiveness, trustworthiness and dominance have been directly correlated with facial features. If the same correlations were true for virtual faces, this could be a valuable design guideline to direct the creation of characters with intended personalities. In particular, this is relevant for extremely abstract characters that have minimal facial features (often seen in video games and movies), and rely heavily on these features for portraying personality. We conducted an exploratory study in order to retrieve insights about the way certain facial features affect the perceived personality, as well as affinity of very abstract virtual faces. We specifically tested the effect of different head shapes, eye shapes and eye sizes. Interestingly, our findings show that the same rules for real human faces do not apply to the perception of abstract faces, and in some cases are the complete reverse. These results provide us with a better understanding of the perception of abstract virtual faces, and a starting point for the creation of guidelines for how to portray personality using minimal facial cues.

The design of lighting in Computer Graphics is directly derived from cinematography, and many digital artists follow the conventional wisdom on how lighting is set up to convey drama, appeal, or emotion. In this paper, we are interested in investigating the most commonly used lighting techniques to more formally determine their effect on our perception of animated virtual characters. Firstly, we commissioned a professional animator to create a sequence of dramatic emotional sentences for a typical CG cartoon character. Then, we rendered that character using a range of lighting directions, intensities, and shading techniques. Participants of our experiment rated the emotion, the intensity of the performance, and the appeal of the character. Our results provide new insights into how animated virtual characters are perceived, when viewed under different lighting conditions.

The aim of this study was to investigate the effect of non-player character (NPC) appearance in games - specifically, how a character's appearance affects a player's performance, and their perception of the game. We ran an experiment where participants played a mobile game on a 9.7" tablet, the goal of which was to kill all of the enemy characters in the game. The visual appearance of the enemy characters varied in the level of aggression in appearance. One crowd had small, green characters with no weapons and minimal armour. The other crowd had red characters that were large and wearing both weapons and armour. Both crowds had the same level of aggression in behaviour, stance and animations, as well as the same intersection-test capsule to ensure the gameplay was balanced and both crowds were equally difficult to kill. As expected, the second crowd was perceived as highly aggressive and less friendly than the first crowd. We found no differences in the enjoyment levels of the game but interestingly, we found that the visual appearance of the crowd had a direct effect on the player performance in combat. In contrast to our hypothesis, players performed worse (i.e., were killed more often) when in combat against the characters with the less-aggressive appearance.

For real-time applications, blendshape animations are usually calculated on the CPU, which are slow to animate, and are therefore generally limited to only the closest level of detail for a small number of characters in a scene. In this paper, we present a GPU based blendshape animation technique. By storing the blendshape model (including animations) on the GPU, we are able to attain significant speed improvements over CPU-based animation. We also find that by using compute shaders to decouple rendering and animation we can improve performance when rendering a crowd animation. Further gains are also made possible by using a smaller subset of blendshape expressions, at the cost of expressiveness. However, the quality impact can be minimised by selecting this subset carefully. We discuss a number of potential metrics to automate this selection.

In this paper, we conducted a perceptual experiment to determine if specific personality traits can be portrayed through eye and head movement in the absence of other facial animation cues. We created a collection of eye and head motions captured from three female actors portraying different personalities, while listening to instructional videos. In a between-groups experiment, we tested the perception of personality on a realistic model and a cartoon stylisation in order to determine if stylisation can positively influence the perceived personality or if personality is more easily identified on a realistic face. Our results verify that participants were able to differentiate between personality traits portrayed only through eye gaze, blinks and head movement. The results also show that perception of personality was robust across character realism.

Despite the elusive term "Uncanny Valley", research in the area of appealing virtual humans approaching realism continues. The theory suggests that characters lose appeal when they approach photorealism (e.g., [MacDorman et al. 2009]). Realistic virtual characters are judged harshly, since the human visual system has acquired more expertise with the featural restrictions of other humans than with the restrictions of artificial characters [Seyama and Nagayama 2007]. Stylisation (making the character's appearance abstract) is therefore often used to avoid virtual characters to be perceived as unpleasant. We designed an experiment to test if there is a general affinity towards abstract as oppose to realistic characters.

Appearance and animation realism of virtual characters in games, movies or other VR applications has been shown to affect audiences levels of acceptance and engagement with these characters. However, when a virtual character is representing us in VR setup, the level of engagement might also depend on the levels of perceived ownership and sense of control (agency) we feel towards this virtual character. In this study, we used advanced face-tracking technology in order to map real-time tracking data of participants' head and eye movements, as well as facial expressions on virtual faces with different appearance realism characteristics (realistic or cartoon-like) and different levels of animation realism (complete or reduced facial movements). Our results suggest that virtual faces are perceived as more appealing when higher levels of animation realism are provided through real-time tracking. Moreover, high-levels of face-ownership and agency can be induced through synchronous mapping of the face tracking on the virtual face. In this study, we provide valuable insights for future games that use face tracking as an input.

Abstract A person's emotions and state of mind are apparent in their face and eyes. As a Latin proverb states: ‘The face is the portrait of the mind; the eyes, its informers’. This presents a significant challenge for Computer Graphics researchers who generate artificial entities that aim to replicate the movement and appearance of the human eye, which is so important in human–human interactions. This review article provides an overview of the efforts made on tackling this demanding task. As with many topics in computer graphics, a cross‐disciplinary approach is required to fully understand the workings of the eye in the transmission of information to the user. We begin with a discussion of the movement of the eyeballs, eyelids and the head from a physiological perspective and how these movements can be modelled, rendered and animated in computer graphics applications. Furthermore, we present recent research from psychology and sociology that seeks to understand higher level behaviours, such as attention and eye gaze, during the expression of emotion or during conversation. We discuss how these findings are synthesized in computer graphics and can be utilized in the domains of Human–Robot Interaction and Human–Computer Interaction for allowing humans to interact with virtual agents and other artificial entities. We conclude with a summary of guidelines for animating the eye and head from the perspective of a character animator.

In this article, we investigate the perception of gender from the motion of virtual humans under different emotional conditions and explore the effect of emotional bias on gender perception (e.g., anger being attributed to males more than females). As motion types can present different levels of physiological cues, we also explore how two types of motion (walking and conversations) are affected by emotional bias. Walking typically displays more physiological cues about gender (e.g., hip sway) and therefore is expected to be less affected by emotional bias. To investigate these effects, we used a corpus of captured facial and body motions from four male and four female actors, performing basic emotions through conversation and walk. We expected that the appearance of the model would also influence gender perception; therefore, we displayed both male and female motions on two virtual models of different sex. Two experiments were then conducted to assess gender judgments from these motions. In both experiments, participants were asked to rate how male or female they considered the motions to be under different emotional states, then classified the emotions to determine how accurately they were portrayed by actors. Overall, both experiments showed that gender ratings were affected by the displayed emotion. However, we found that conversations were influenced by gender stereotypes to a greater extent than walking motions. This was particularly true for anger, which was perceived as male on both male and female motions, and sadness, which was perceived as less male when portrayed by male actors. We also found a slight effect of the model when observing gender on different types of virtual models. These results have implications for the design and animation of virtual humans.

Virtual characters contribute strongly to the entire visuals of 3D animated films. However, designing believable characters remains a challenging task. Artists rely on stylization to increase appeal or expressivity, exaggerating or softening specific features. In this paper we analyze two of the most influential factors that define how a character looks: shape and material. With the help of artists, we design a set of carefully crafted stimuli consisting of different stylization levels for both parameters, and analyze how different combinations affect the perceived realism, appeal, eeriness, and familiarity of the characters. Moreover, we additionally investigate how this affects the perceived intensity of different facial expressions (sadness, anger, happiness, and surprise). Our experiments reveal that shape is the dominant factor when rating realism and expression intensity, while material is the key component for appeal. Furthermore our results show that realism alone is a bad predictor for appeal, eeriness, or attractiveness.

Delivering appealing virtual characters conveying personality is becoming extremely important in the entertainment industry and beyond. A theory called the 'Uncanny Valley' has been used to describe the phenomenon that the appearance of a virtual character can contribute to negative/positive audience reactions to that character [Mori 1970]. Since the style used to render a character strongly changes the appearance, we investigate whether a difference in render style can indirectly influence audience reaction, which we measure based on perception of personality. Based on psychology research, we first scripted original character dialogues in order to convey a range of ten typical personality types. Then, a professional actor was recruited to act out these dialogues, while his face and body motion and audio were recorded. The performances were mapped onto a virtual character rendered in two styles that differ in appearance: an appealing cartoon style and unappealing ill style (Figure 1). In our experiment, participants were asked questions about the character's personality in order for us to test if the difference in render style causes differences in personality perception. Our results found an indirect effect of render style where the cartoon style was rated as having a more agreeable personality than the ill style. This result has implications for developers interested in creating appealing virtual humans, avoiding the 'Uncanny Valley' phenomenon.

Mapping the motion of an actor's face to a virtual model is a difficult but important problem, especially as fully animated characters are becoming more common in games and movies. Many methods have been proposed but most require the source and target to be structurally similar. Optical motion capture markers and blendshape weights are an example of topologically incongruous source and target examples that do not have a simple mapping between one another. In this paper, we created a system capable of determining this mapping through supervised learning of a small training dataset. Radial Basis Function Networks (RBFNs) have been used for retargeting markers to blendshape weights before but to our knowledge Multi-Layer Perceptron Artificial Neural Networks (referred to as ANNs) have not been employed in this way. We hypothesized that ANNs would result in a superior retargeting solution compared to the RBFN, due to their theoretically greater representational power. We implemented a retargeting system using ANNs and RBFNs for comparison. Our results found that both systems produced similar results (figure 1) and in some cases the ANN proved to be more expressive although the ANN was more difficult to work with.

In this paper, we investigate the ability of humans to determine the gender of conversing characters, based on facial and body cues for emotion. We used a corpus of simultaneously captured facial and body motions from four male and four female actors. In our Gender Rating task, participants were asked to rate how male or female they considered the motions to be, under different emotional states. In our Emotion Recognition task, participants were asked to classify the emotions, in order to determine how accurately perceived those emotions were. We found that gender perception was affected by emotion, where certain emotions facilitated gender determination while others masked it. We also found that there was no correlation between how accurate an emotion was portrayed and how much gender information was present in that motion. Finally, we found that the model used to display the motion did not affect gender perception of motion but did alter emotion recognition.

It has been shown that humans are sensitive to the portrayal of emotions for virtual characters. However, previous work in this area has often examined this sensitivity using extreme examples of facial or body animation. Less is known about how attuned people are at recognizing emotions as they are expressed during conversational communication. In order to determine whether body or facial motion is a better indicator for emotional expression for game characters, we conduct a perceptual experiment using synchronized full-body and facial motion-capture data. We find that people can recognize emotions from either modality alone, but combining facial and body motion is preferable in order to create more expressive characters.

Subtle animation details such as finger or facial movements help to bring virtual characters to life and increase their appeal. However, it is not always possible to capture finger animations simultaneously with full-body motion, due to limitations of the setup or tight production schedules. Therefore, hand motions are often either omitted, manually created by animators, or captured during a separate session and spliced with full body animation. In this paper, we investigate the perceived fidelity of hand animations where all the degrees of freedom of the hands are computed from reduced marker sets. In a set of perceptual experiments, we found that finger motions reconstructed with inverse kinematics from a reduced marker set of eight markers per hand are perceived to be very similar to the corresponding motions reconstructed using a full set of twenty markers. We demonstrate how using this reduced set of eight large markers enabled us to capture the finger and full-body motions of two actors performing a range of relatively unconstrained actions using a 13-camera motion capture system. This serves to simplify the capture process and to significantly reduce the time for cleanup, while preserving the natural biological movements of the hands relative to the actions performed.

No abstract available.

With recent advances in real-time graphics technology, more realistic, believable and appealing virtual characters are needed than ever before. Both player-controlled avatars and non-player characters are now starting to interact with the environment, other virtual humans and crowds. However, simulating physical contacts between characters and matching appropriate reactions to specific actions is a highly complex problem, and timing errors, force mismatches and angular distortions are common. To investigate the effect of such anomalies on the perceived realism of two-character interactions, we captured a motion corpus of pushing animations and corresponding reactions and then conducted a series of perceptual experiments. We found that participants could easily distinguish between five different interaction forces, even when only one of the characters was visible. Furthermore, they were sensitive to all three types of anomalous interactions: timing errors of over 150ms were acceptable less than 50% of the time, with early or late reactions being equally perceptible; participants could perceive force mismatches, though over-reactions were more acceptable than under-reactions; finally, angular distortions when a character reacts to a pushing force reduce the acceptability of the interactions, but there is some evidence for a preference of expansion away from the pushing character's body. Our results provide insights to aid in designing motion capture sessions, motion editing strategies and balancing animation budgets.

The realistic depiction of lifelike virtual humans has been the goal of many movie makers in the last decade. Recently, films such as Tron: Legacy and The Curious Case of Benjamin Button have produced highly realistic characters. In the real-time domain, there is also a need to deliver realistic virtual characters, with the increase in popularity of interactive drama video games (such as L.A. Noire™ or Heavy Rain™ ). There have been mixed reactions from audiences to lifelike characters used in movies and games, with some saying that the increased realism highlights subtle imperfections, which can be disturbing. Some developers opt for a stylized rendering (such as cartoon-shading) to avoid a negative reaction [Thompson 2004]. In this paper, we investigate some of the consequences of choosing realistic or stylized rendering in order to provide guidelines for developers for creating appealing virtual characters. We conducted a series of psychophysical experiments to determine whether render style affects how virtual humans are perceived. Motion capture with synchronized eye-tracked data was used throughout to animate custom-made virtual model replicas of the captured actors.

No abstract available.

The Uncanny Valley (UV) has become a standard term for the theory that near-photorealistic virtual humans often appear unintentionally erie or creepy. This UV theory was first hypothesized by robotics professor Masahiro Mori in the 1970's [Mori 1970] but is still taken seriously today by movie and game developers as it can stop audiences feeling emotionally engaged in their stories or games. It has been speculated that this is due to audiences feeling a lack of empathy towards the characters. With the increase in popularity of interactive drama video games (such as L.A. Noire or Heavy Rain), delivering realistic conversing virtual characters has now become very important in the real-time domain. Video game rendering techniques have advanced to a very high quality; however, most games still use linear blend skinning due to the speed of computation. This causes a mismatch between the realism of the appearance and animation, which can result in an uncanny character. Many game developers opt for a stylised rendering (such as cel-shading) to avoid the uncanny effect [Thompson 2004]. In this preliminary work, we begin to study the complex interaction between rendering style and perceived trust, in order to provide guidelines for developers for creating plausible virtual characters.

Understanding the perception of humanoid character motion can provide insights that will enable realism, accuracy, computational cost and data storage space to be optimally balanced. In this sketch we describe a preliminary perceptual evaluation of human motion timewarping, a common editing method for motion capture data. During the experiment, participants were shown pairs of walking motion clips, both timewarped and at their original speed, and asked to identify the real animation. We found a statistically significant difference between speeding up and slowing down, which shows that displaying clips at higher speeds produces obvious artifacts, whereas even significant reductions in speed were perceptually acceptable.

In many scenes with human characters, interacting groups are an important factor for maintaining a sense of realism. However, little is known about what makes these characters appear realistic. In this paper, we investigate human sensitivity to audio mismatches (i.e., when individuals' voices are not matched to their gestures) and visual desynchronization (i.e., when the body motions of the individuals in a group are mis-aligned in time) in virtual human conversers. Using motion capture data from a range of both polite conversations and arguments, we conduct a series of perceptual experiments and determine some factors that contribute to the plausibility of virtual conversing groups. We found that participants are more sensitive to visual desynchronization of body motions, than to mismatches between the characters' gestures and their voices. Furthermore, synthetic conversations can appear sufficiently realistic once there is an appropriate balance between talker and listener roles. This is regardless of body motion desynchronization or mismatched audio.

In this article, we investigate human sensitivity to the coordination and timing of conversational body language for virtual characters. First, we captured the full body motions (excluding faces and hands) of three actors conversing about a range of topics, in either a polite (i.e., one person talking at a time) or debate/argument style. Stimuli were then created by applying the motion-captured conversations from the actors to virtual characters. In a 2AFC experiment, participants viewed paired sequences of synchronized and desynchronized conversations and were asked to guess which was the real one. Detection performance was above chance for both conversation styles but more so for the polite conversations, where desynchronization was more noticeable.

In this paper, our aim is to determine factors that influence the perceived sex of virtual characters. In Experiment 1, four different model types were used: highly realistic male and female models, an androgynous character, and a point light walker. Three different types of motion were applied to all models: motion captured male and female walks, and neutral synthetic walks. We found that both form and motion influence sex perception for these characters: for neutral synthetic motions, form determines perceived sex, whereas natural motion affects the perceived sex of both androgynous and realistic forms. These results indicate that the use of neutral walks is better than creating ambiguity by assigning an incongruent motion. In Experiment 2 we investigated further the influence of body shape and motion on realistic male and female models and found that adding stereotypical indicators of sex to the body shapes influenced sex perception. Also, that exaggerated female body shapes influences sex judgements more than exaggerated male shapes. These results have implications for variety and realism when simulating large crowds of virtual characters.

In order to analyze the emotional content of motions portrayed by different characters, we created real and virtual replicas of an actor exhibiting six basic emotions: sadness, happiness, surprise, fear, anger, and disgust. In addition to the video of the real actor, his actions were applied to five virtual body shapes: a low- and high-resolution virtual counterpart, a cartoon-like character, a wooden mannequin, and a zombie-like character (Figures 1 and 2). In a point light condition, we also tested whether the absence of a body affected the perceived emotion of the movements. Participants were asked to rate the actions based on a list of 41 more complex emotions. We found that the perception of emotional actions is highly robust and to the most part independent of the character's body, so long as form is present. When motion alone is present, emotions were generally perceived as less intense than in the cases where form was present.

Populated virtual environments need to be simulated with as much variety as possible. By identifying the most salient parts of the scene and characters, available resources can be concentrated where they are needed most. In this paper, we investigate which body parts of virtual characters are most looked at in scenes containing duplicate characters or clones . Using an eye-tracking device, we recorded fixations on body parts while participants were asked to indicate whether clones were present or not. We found that the head and upper torso attract the majority of first fixations in a scene and are attended to most. This is true regardless of the orientation, presence or absence of motion, sex, age, size, and clothing style of the character. We developed a selective variation method to exploit this knowledge and perceptually validated our method. We found that selective colour variation is as effective at generating the illusion of variety as full colour variation. We then evaluated the effectiveness of four variation methods that varied only salient parts of the characters. We found that head accessories, top texture and face texture variation are all equally effective at creating variety, whereas facial geometry alterations are less so. Performance implications and guidelines are presented.

When simulating large crowds, it is inevitable that the models and motions of many virtual characters will be cloned. However, the perceptual impact of this trade-off has never been studied. In this paper, we consider the ways in which an impression of variety can be created and the perceptual consequences of certain design choices. In a series of experiments designed to test people's perception of variety in crowds, we found that clones of appearance are far easier to detect than motion clones. Furthermore, we established that cloned models can be masked by color variation, random orientation, and motion. Conversely, the perception of cloned motions remains unaffected by the model on which they are displayed. Other factors that influence the ability to detect clones were examined, such as proximity, model type and characteristic motion. Our results provide novel insights and useful thresholds that will assist in creating more realistic, heterogeneous crowds.

Matrix palette skinning (also known as skeletal subspace deformation) is a very popular real-time animation technique. So far, it has only been applied to the class of quasi-articulated objects, such as moving human or animal figures. In this paper, we demonstrate how to automatically construct skinning approximations of arbitrary precomputed animations, such as those of cloth or elastic materials. In contrast to previous approaches, our method is particularly well suited to input animations without rigid components. Our transformation fitting algorithm finds optimal skinning transformations (in a least-squares sense) and therefore achieves considerably higher accuracy for non-quasi-articulated objects than previous methods. This allows the advantages of skinned animations (e.g., efficient rendering, rest-pose editing and fast collision detection) to be exploited for arbitrary deformations.