The perception of one's own body is subject to systematic distortions and can be influenced by exposure to visual stimuli showing distorted bodies. In Virtual Reality (VR), echoing such body judgment inaccuracies, avatars with strong appearance dissimilarities with respect to users' bodies can be successfully embodied. The present experimental work investigates, in the healthy population, the perception of the own body in immersive and embodied VR, as well as the impact of being co-present with virtual humans on such self-perception. Participants were successively presented with different avatars, corresponding to various upper- and lower-body proportions, and were asked to compare them with their perceived own body morphology. To investigate the influence of co-present virtual humans on this judgment, the task was performed in co-presence with virtual agents corresponding to various body appearances. Results show an overall overestimation of one's leg length and no influence of the co-present agent's appearance. Importantly, the embodiment scores reflect such body morphology judgment inaccuracy, with participants reporting lower levels of embodiment for avatars with very short legs than for avatars with very long legs. Our findings suggest specifics of embodied body judgment methods, likely resulting from the experience of embodying the avatar as compared to visual appreciation only.

This article introduces the Least Increase aversion (LIA) protocol to investigate the relative impact of factors that may trigger cybersickness. The protocol is inspired by the Subjective Matching methodology (SMT) from which it borrows the incremental construction of a richer VR experience, except that the full-blown target experience may cause undesired discomfort. In the first session, the participant briefly encounter all factors at the maximum level. Then in the second session they start with the minimum level of all factors as a Baseline. Subsequently, we expect the participant to minimize their exposure to the most adverse factors. This a pproach ranks the factors from mildest to worst and helps detect individual susceptibility to cybersickness triggers. To validate the applicability of LIA protocol, we further evaluate it with an experiment to identify individual susceptibility to three rotational axes (Yaw, Pitch, and Roll). The findings not only confirm the protocol's capability to accurately discern individual rankings of various factors to cybersickness but also indicate that individual susceptibility is more intricate and multifaceted than initially anticipated.

The sense of embodiment in virtual reality (VR) is commonly understood as the subjective experience that one's physical body is substituted by a virtual counterpart, and is typically achieved when the avatar's body, seen from a first-person view, moves like one's physical body. Embodiment can also be experienced in other circumstances (e.g., in third-person view) or with imprecise or distorted visuo-motor coupling. It was moreover observed, in various cases of small or progressive temporal and spatial manipulations of avatars' movements, that participants may spontaneously follow the movement shown by the avatar. The present work investigates whether, in some specific contexts, participants would follow what their avatar does even when large movement discrepancies occur, thereby extending the scope of understanding of the self-avatar follower effect beyond subtle changes of motion or speed manipulations. We conducted an experimental study in which we introduced uncertainty about which movement to perform at specific times and analyzed participants' movements and subjective feedback after their avatar showed them an incorrect movement. Results show that, when in doubt, participants were influenced by their avatar's movements, leading them to perform that particular error twice more often than normal. Importantly, results of the embodiment score indicate that participants experienced a dissociation with their avatar at those times. Overall, these observations not only demonstrate the possibility of provoking situations in which participants follow the guidance of their avatar for large motor distortions, despite their awareness about the avatar movement disruption and on the possible influence it had on their choice, and, importantly, exemplify how the cognitive mechanism of embodiment is deeply rooted in the necessity of having a body.

In this research paper, we conducted a study to investigate the connection between three objective measures: Electrocardio-gram(EGG), Electrogastrogram (EGG), and Electroencephalogram (EEG), and individuals' susceptibility to cybersickness. Our primary objective was to identify which of these factors plays a central role in causing discomfort when experiencing rotations along three different axes: Roll, Pitch, and Yaw. This study involved 35 participants who were tasked with destroying asteroids using their eye gaze while undergoing passive rotations in four separate sessions. The results, when combined with subjective measurements (specifically, Fast motion sickness questionnaire (FMS) and Simulator sickness questionnaire (SSQ) score), demonstrated that EGG measurements were superior in detecting symptoms associated with nausea. As for ECG measurements, our observations did reveal significant changes in Heart Rate Variability (HRV) parameters. However, we caution against relying solely on ECG as a dependable indicator for assessing the extent of cybersickness. Most notably, EEG signals emerged as a crucial resource for discerning individual differences related to these rotational axes. Our findings were significant not only in the context of periodic activities but also underscored the potential of aperiodic activities in detecting the severity of cybersickness and an individual's susceptibility to rotational triggers.

In Virtual Reality, having a virtual body opens a wide range of possibilities as the participant's avatar can appear to be quite different from oneself for the sake of the targeted application (e.g., for perspective-taking). In addition, the system can partially manipulate the displayed avatar movement through some distortion to make the overall experience more enjoyable and effective (e.g., training, exercising, rehabilitation). Despite its potential, an excessive distortion may become noticeable and break the feeling of being embodied into the avatar. Past researches have shown that individuals have a relatively high tolerance to movement distortions and a great variability of individual sensitivities to distortions. In this article, we propose a method taking advantage of Reinforcement Learning (RL) to efficiently identify the magnitude of the maximum distortion that does not get noticed by an individual (further noted the detection threshold). We show through a controlled experiment with subjects that the RL method finds a more robust detection threshold compared to the adaptive staircase method, i.e., it is more able to prevent subjects from detecting the distortion when its amplitude is equal or below the threshold. Finally, the associated majority voting system makes the RL method able to handle more noise within the forced choices input than adaptive staircase. This last feature is essential for future use with physiological signals as these latter are even more susceptible to noise. It would then allow to calibrate embodiment individually to increase the effectiveness of the proposed interactions.

When placed inside an immersive virtual simulation, subjects will tend to experience the feeling of being ’really there’ and to respond realistically to their environment, forgetting that it is not real. This behaviour is observed when subjects experience a high sense of presence, the sensation of being in a real place and that the scenario being depicted to them is real. Here we present an experiment designed to evaluate the impact of different levels of immersion, and of different blending of virtual and real objects and body representations, on participant’s subjective experience. Presence is evaluated with an innovative method combining the random introduction of breaks-in-presence (BiP) with a rapid decision-making test. Results show that the level of immersion impacts both the Sense of Presence (SoP) and the Sense of Embodiment (SoE), that the BiP has a limited impact on the SoE without breaking it, and that the level of confidence in the decision test correlates with both the SoP and the SoE.

Cybersickness is still an inevitable adverse effect when using VR systems, resulting in different levels of discomfort, and potentially breaking the immersive experience. To date, few studies have focused on exploring VR experience in the supine posture. Recent research indicated that simple adoption of VR game initially designed for a seated posture (by rotating 90 degrees) could lead to more severe cybersickness, even to experienced users. Following the insights from previous literature and the widely-accepted sensory conflict theory, we proposed an assumption that might explain such a phenomenon. The hypothesis is that when the perceived virtual coordinate system contradicts the received real-world coordinate through our vestibular system, the conflict appears, which can lead to a sense of discomfort. Hence, the primary goal of the study is to evaluate whether such conflict has an impact on cybersickness. Furthermore, we explored methods of mitigating this conflict through different game designs so as to improve the experience for the supine posture. The final results show that the design that aligned with the real vertical axis is effective in mitigating cybersickness, especially for games that present an acceleration sensation.

Over the past few years scientific research has opened up to the idea of using digital games for human-based studies. Fields such as Neuroscience, Medical and Affective Computing are currently using games to study human-based phenomena. Even though a vast amount of work exists within the field, rarely is the subject of designing such games ever touched upon. In fact a common problem within the field is that the games themselves are often an afterthought, where certain gameplay limitations are never truly acknowledged and tend to be mostly ignored. Thus, this paper intends to provide some game design guidelines to the most common problems found in literature from work specifically using games for human physiological data-collection purposes. Furthermore, a brief description of the most popular physiological recording methods: Skin Conductance (SC), Heart-Rate Variability (HRV), Electromyogram (EMG), Electroencephalogram (EEG) and Functional Magnetic Resonance Imaging (fMRI); are provided as the game-play “limitations” of using such devices are an important factor to take into consideration in the game design process. As such, the objective of this paper is to provide awareness of specific game design limitations found in literature and analyse them from a game design perspective.

This study attempts to evaluate whether a navigation technique based on scaling the user’s avatar impacts the user’s ability to correctly assess the size of virtual objects in a virtual environment. This study was realized during the CERN Open Days with data from 177 participants over eighteen years old. We were able to observe well-established phenomena such as the effect of inter-pupillary distance (IPD) on perception of scale, as well as original results associated with scaling factor and avatar embodiment. We observed that the user is more prone to overestimate object sizes from the Virtual Environment (VE) when provided with an avatar, while scaling the IPD according to the scale of the user’s avatar contributes to a reduction in the overestimation of object sizes within the VE.

Cybersickness induced through Virtual Reality (VR) applications is still one of its main barriers as it can induce unwanted side-effects in players, significantly hindering the overall experience. Despite the wealth of research available on this topic, it is still an unsolved problem. Although previous studies have explored methods of cybersickness mitigation in addition to correlating physiological factors, there has been little research on the potential correlation of eye behavior and cybersickness. Thanks to advances in eye-tracking technology within HMDs, detecting eye behavior has become a much easier process. This paper explores the differences of pupil position and blink rate in relation to cybersickness intensity. The latter is measured through the standard and a customized version of the Simulator Sickness Questionnaire (SSQ). A total of 34 participant data was collected from two separate playing sessions of a VR maze game, where each session presented a unique control scheme.

Virtual Reality (VR) has the capability of fully immersing users into a wide variety of gaming experiences where the sole focus of the user is on that experience itself. One problem that often occurs when it comes to VR experiences is that of simulation sickness (SS), which is particularly prevalent when the user is forced to play at an abnormal posture due to physical limitations or specific treatment procedures. In this paper we report on a between group study comparing the effect of SS mitigating factors for two postures: seated and lying-down on one's back (known as the supine pose). A 3D recreation of the popular game Pacman (Namco, 1980) was developed specifically for VR. Each subject participated a total of six sessions of six minutes each taking place over the course of two to three weeks with a two to five day minimum and maximum between each session, respectively. At the start and end of each session participants were asked to fill-in the Simulator Sickness Questionnaire (SSQ), allowing to effectively rate the intensity of SS per session. Results show a lack of significant differences when examining participants as a whole regarding the SS decrease in relation with the field of view (FOV) and the rest frame factors. However, when considering the participant experience in VR, it appears that the lying-down pose does trigger simulation sickness symptoms for more proficient participants whereas they do not report such symptoms for the seated posture.

Performing motor tasks in virtual environments is best achieved with motion capture and animation of a 3D character that participants control in real time and perceive as being their avatar in the virtual environment. A strong Sense of Embodiment (SoE) for the virtual body not only relies on the feeling that the virtual body is their own (body ownership), but also that the virtual body moves in the world according to their will and replicates precisely their body movement (sense of agency). Within that frame of mind our specific aim is to demonstrate that the avatar can even be programmed to be better at executing a given task or to perform a movement that is normally difficult or impossible to execute precisely by the user. More specifically, our experimental task consists in asking subjects to follow with the hand a target that is animated using non-biological motion; the unnatural nature of the movement leads to systematic errors by the subjects. The challenge here is to introduce a subtle distortion between the position of the real hand and the position of the virtual hand, so that the virtual hand succeeds in performing the task while still letting subjects believe they are fully in control. Results of two experiments (N=16) show that our implementation of a distortion function, that we name the attraction well, successfully led participants to report being in control of the movement (agency) and being embodied in the avatar (body ownership) even when the distortion was above a threshold that they can detect. Furthermore, a progressive introduction of the distortion (starting without help, and introducing distortion on the go) could even further increase its acceptance.

Creating interactive characters is one of the most challenging tasks of videogame design. In order to facilitate such an endeavor, we introduce a decisional and behavior synthesis architecture integrated in the game engine Unity3D. A distinguishing feature of this architecture is that it embraces embodied cognition principles and uses them as implementation requirements. From these, we derive an architecture, which is based on a novel combination of previously proposed systems, together with some simplifications. We also argue that the architecture proposed has properties-modularity, scalability, and stability-which can be beneficial for its practical industrial adoption, particularly in the context of the recent improvement in machine learning techniques. Artificial intelligence is a quite technical domain, and we believe such a tool can facilitate interactive character creation by creative minds in industrial applications.

Hands deserve particular attention in virtual reality (VR) applications because they represent our primary means for interacting with the environment. Although marker-based motion capture with inverse kinematics works adequately for full body tracking, it is less reliable for small body parts such as hands and fingers which are often occluded when captured optically, thus leading VR professionals to rely on additional systems (e.g. inertial trackers). We present a machine learning pipeline to track hands and fingers using solely a motion capture system based on cameras and active markers. Our finger animation is performed by a predictive model based on neural networks trained on a movements dataset acquired from several subjects with a complementary capture system. We employ a two-stage pipeline, which first resolves occlusions, and then recovers all joint transformations. We show that our method compares favorably to inverse kinematics by inferring automatically the constraints from the data, provides a natural reconstruction of postures, and handles occlusions better than three proposed baselines.

We investigate the self-attribution of distorted pointing movements in immersive virtual reality. Participants had to complete a multidirectional pointing task in which the visual feedback of the tapping finger could be deviated in order to increase or decrease the motor size of a target relative to its visual appearance. This manipulation effectively makes the task easier or harder than the visual feedback suggests. Participants were asked whether the seen movement was equivalent to the movement they performed, and whether they have been successful in the task. We show that participants are often unaware of the movement manipulation, even when it requires higher pointing precision than suggested by the visual feedback. Moreover, subjects tend to self-attribute movements that have been modified to make the task easier more often than movements that have not been distorted. We discuss the implications and applications of our results.

Hands in virtual reality applications represent our primary means for interacting with the environment. Although marker-based motion capture with inverse kinematics (IK) works for body tracking, it is less reliable for fingers often occluded when captured with cameras. Many computer vision and virtual reality applications circumvent the problem by using an additional system (e.g. inertial trackers). We explore an alternative solution that tracks hands and fingers using solely a motion capture system based on cameras and active markers with machine learning techniques. Our animation of fingers is performed by a predictive model based on neural networks, which is trained on a movements dataset acquired from several subjects with a complementary capture system (inertial). The system is as efficient as a traditional IK algorithm, provides a natural reconstruction of postures, and handles occlusions.

This study explores the extent to which individuals embodied in Virtual Reality tend to self-attribute the movements of their avatar. More specifically, we tested subjects performing goal-directed movements and distorted the mapping between user and avatar movements by decreasing or increasing the amplitude of the avatar hand movement required to reach for a target, while maintaining the apparent amplitude – visual distance – fixed. In two experiments, we asked subjects to report whether the movement that they have seen matched the movement that they have performed, or asked them to classify whether a distortion was making the task easier or harder to complete. Our results show that subjects perform poorly in detecting discrepancies when the nature of the distortion is not made explicit and that subjects are biased to self-attributing distorted movements that make the task easier. These findings, in line with previous accounts on the sense of agency, demonstrate the flexibility of avatar embodiment and open new perspectives for the design of guided interactions in Virtual Reality.

With the broad range of motion capture devices available on the market, it is now commonplace to directly control the limb movement of an avatar during immersion in a virtual environment. Here, we study how the subjective experience of embodying a full-body controlled avatar is influenced by motor alteration and self-contact mismatches. Self-contact is in particular a strong source of passive haptic feedback and we assume it to bring a clear benefit in terms of embodiment. For evaluating this hypothesis, we experimentally manipulate self-contacts and the virtual hand displacement relatively to the body. We introduce these body posture transformations to experimentally reproduce the imperfect or incorrect mapping between real and virtual bodies, with the goal of quantifying the limits of acceptance for distorted mapping on the reported body ownership and agency. We first describe how we exploit egocentric coordinate representations to perform a motion capture ensuring that real and virtual hands coincide whenever the real hand is in contact with the body. Then, we present a pilot study that focuses on quantifying our sensitivity to visuo-tactile mismatches. The results are then used to design our main study with two factors, offset (for self-contact) and amplitude (for movement amplification). Our main result shows that subjects' embodiment remains important, even when an artificially amplified movement of the hand was performed, but provided that correct self-contacts are ensured.

The relative location of human body parts often materializes the semantics of on-going actions, intentions and even emotions expressed, or performed, by a human being. However, traditional methods of performance animation fail to correctly and automatically map the semantics of performer postures involving self-body contacts onto characters with different sizes and proportions. Our method proposes an egocentric normalization of the body-part relative distances to preserve the consistency of self contacts for a large variety of human-like target characters. Egocentric coordinates are character independent and encode the whole posture space, i.e., it ensures the continuity of the motion with and without self-contacts. We can transfer classes of complex postures involving multiple interacting limb segments by preserving their spatial order without depending on temporal coherence. The mapping process exploits a low-cost constraint relaxation technique relying on analytic inverse kinematics; thus, we can achieve online performance animation. We demonstrate our approach on a variety of characters and compare it with the state of the art in online retargeting with a user study. Overall, our method performs better than the state of the art, especially when the proportions of the animated character deviate from those of the performer.

This project investigates the impact of a virtual zero gravity experience on the human gravity model. In the planned experiment, subjects are immersed with HMD and full body motion capture in a virtual world exhibiting either normal gravity or the apparent absence of gravity (i.e. body and objects floating in space). The study evaluates changes in the subjects' gravity model by observing changes on motor planning of actions dependent on gravity. Our goal is to demonstrate that a virtual reality exposure can induce some modifications to the humans internal gravity model, analogous to those resulting from real exposure (e.g. parabolic flights), even if users remain under normal gravity condition in reality.

Photorealistic animation is a desirable technique for computer games and movie production. We propose a new method to synthesize plausible videos of human actors with new motions using a single cheap RGB-D camera. A small database is captured in a usual office environment, which happens only once for synthesizing different motions. We propose a marker-less performance capture method using sparse deformation to obtain the geometry and pose of the actor for each time instance in the database. Then, we synthesize an animation video of the actor performing the new motion that is defined by the user. An adaptive model-guided texture synthesis method based on weighted low-rank matrix completion is proposed to be less sensitive to noise and outliers, which enables us to easily create photorealistic animation videos with new motions that are different from the motions in the database. Experimental results on the public data set and our captured data set have verified the effectiveness of the proposed method.

The locomotion pattern is characterized by a translation displacement mostly occurring along the forward frontal body direction, whereas local repositioning with large re-orientations, i.e. stepping, may induce translations both along the frontal and the lateral body directions (holonomy). We consider here a stepping pattern with initial and final null speeds within a radius of 40% of the body height and re-orientation up to 180°. We propose a robust step detection method for such a context and identify a consistent intra-subject behavior in terms of the choice of starting foot and the number of steps.

In this paper we present an intuitive tool suitable for 2D artists using touch-enabled pen tablets. An artist-oriented tool should be easy-to-use, real-time, versatile, and locally refinable. Our approach uses an interactive system for 3D character posing from 2D strokes. We employ a closed-form solution for the 2D strokes to 3D skeleton registration problem. We first construct an intermediate 2D stroke representation by extracting local features using meaningful heuristics. Then, we match 2D stroke segments to 3D bones. Finally, 3D bones are carefully realigned with the matched 2D stroke segments while enforcing important constraints such as bone rigidity and depth. Our technique is real-time and has a linear time complexity. It is versatile, as it works with any type of 2D stroke and 3D skeleton input. Finally, thanks to its coarse-to-fine design, it allows users to perform local refinements and thus keep full control over the final results. We demonstrate that our system is suitable for 2D artists using touch-enabled pen tablets by posing 3D characters with heterogeneous topologies (bipeds, quadrupeds, hands) in real-time.

In this article, we present a parallel prioritized Jacobian-based inverse kinematics algorithm for multithreaded architectures. We solve damped least squares inverse kinematics using a parallel line search by identifying and sampling critical input parameters. Parallel competing execution paths are spawned for each parameter in order to select the optimum that minimizes the error criteria. Our algorithm is highly scalable and can handle complex articulated bodies at interactive frame rates. We show results on complex skeletons consisting of more than 600 degrees of freedom while being controlled using multiple end effectors. We implement the algorithm both on multicore and GPU architectures and demonstrate how the GPU can further exploit fine-grain parallelism not directly available on a multicore processor. Our implementations are 10 to 150 times faster compared to a state-of-the-art serial implementation while providing higher accuracy. We also demonstrate the scalability of the algorithm over multiple scenarios and explore the GPU implementation in detail.

In this paper we evaluate the use of non-planar projections as a means to increase the Field of View (FoV) in embodied Virtual Reality (VR). Our main goal is to bring the virtual body into the user's FoV and to understand how this affects the virtual body/environment relation and quality of interaction. Subjects wore a Head Mounted Display (HMD) and were instructed to perform a selection and docking task while using either Perspective (≈ 106 ° vertical FoV), Hammer or Equirectangular (≈ 180 ° vertical FoV for both) projection. The increased FoV allowed for a shorter search time as well as less head movements. However, quality of interaction was generally inferior, requiring more time to dock, increasing docking error and producing more body/environment collisions. We also assessed cybersickness and the sense of embodiment toward the virtual body through questionnaires, for which the difference between projections seemed to be less pronounced.

In this paper we propose the Middle-Axis-Rotation (MAR) parametrization of human limbs that addresses the ill-conditioned cases of analytical Inverse Kinematics (IK) algorithms. The MAR parametrization is singularity-free in the reach space of the human limbs. Unlike the swivel representation, it does not rely on the projection of an additional fixed vector. In addition, we express the joint limits of each joint of the limb in terms of the redundancy of the new decomposition. In the specific case of the upper limb, we analyse the contribution of the clavicle to produce biomechanically meaningful postures. We illustrate various real-time applications of this approach.

We present Disambiguation canvas, a technique for selection by progressive refinement using a mobile device and consisting of two steps. During the first, the user defines a subset of objects through the orientation sensors of the device and a volume-casting pointing technique. The subsequent step consists of the disambiguation of the desired target among the previously-defined subset of objects, and is accomplished using the mobile device touchscreen. By relying on the touchscreen for the last step the user can disambiguate among hundreds of objects at once, previous progressive refinement techniques do not scale as well as ours. Disambiguation canvas is mainly developed for easy, accurate and fast selection of small objects, or objects inside cluttered virtual environments. User tests show that our technique performs faster than ray-casting for targets with ≈ 0.53° of angular size, and is also much more accurate for all the tested target sizes.

Following limb loss, patients usually continue having sensations on their missing limbs as if they were still present. Significant amount of such sensations are painful and referred as Phantom Limb Pain (PLP). Previous research has shown that providing the patient with the visual feedback of a limb at the place of the missing one in Virtual Reality (VR) can reduce PLP. In this paper we introduce a model to coordinate the arms allowing the exercising of a much broader range of reach tasks for alleviating the PLP more efficiently. Our Two-Arm Coordination Model (TACM) synthesizes the missing limb pose from the instantaneous variations of the intact opposite limb for a given reach task. Moreover, we propose a setup that makes use of a virtual mirror to enhance the full-body awareness of the patient in the virtual space.

ABSTRACT In this paper, we propose a method to achieve effective facial emotional expressivity for embodied conversational agents by considering two types of asymmetry when exploiting the valence–arousal–dominance representation of emotions. Indeed, the asymmetry of facial expressions helps to convey complex emotional feelings such as conflicting and/or hidden emotions due to social conventions. To achieve such a higher degree of facial expression in a generic way, we propose a new model for mapping the valence–arousal–dominance emotion model onto a set of 12 scalar facial part actions built mostly by combining pairs of antagonist action units from the Facial Action Coding System. The proposed linear model can automatically drive a large number of autonomous virtual humans or support the interactive design of complex facial expressions over time. By design, our approach produces symmetric facial expressions, as expected for most of the emotional spectrum. However, more complex ambivalent feelings can be produced when differing emotions are applied on the left and right sides of the face. We conducted an experiment on static images produced by our approach to compare the expressive power of symmetric and asymmetric facial expressions for a set of eight basic and complex emotions. Results confirm both the pertinence of our general mapping for expressing basic emotions and the significant improvement brought by asymmetry for expressing ambivalent feelings. Copyright © 2013 John Wiley & Sons, Ltd.

Despite the large success of games grounded on movement-based interactions the current state of full-body motion capture technologies still prevents the exploitation of precise interactions with complex environments. The first key requirement in the line of work we present here is to ensure a precise spatial correspondence between the user and the Avatar. For that purpose, we build upon our past effort in human postural control with a prioritized inverse kinematics (PIK) framework. One of its key advantages is to ease the dynamic- and priority-based combination of multiple conflicting constraints such as ensuring the balance and reaching a goal. However, its reliance on a linearized approximation of the problem makes it vulnerable to the well-known full extension singularity of the limbs. We address this issue by presenting a new type of 1D analytic constraint that smoothly integrates within the PIK framework under the name of FLEXT constraint (for FLexion-EXTension constraint). We further ease the full-body interaction by combining this new constraint with a recently introduced motion constraint to exploit the data-based synergy of full-body reach motions. The combination of both techniques allows immersive full-body interactions with a small set of active optical marker. Copyright © 2010 John Wiley & Sons, Ltd.

Our goal is to create dancing crowds in cyber worlds and to use this feature to support creative endeavors such as pre-visualization of choreography and actual stage performances. In this paper we present a method of motion planning using dance motion clips. We describe a trial algorithm of collision avoidance using a grid map. We also present methods to create variety in animation for dance choreographies. As a result, we confirmed that most collisions could be avoided by the motion planning method. However, the need for some improvements in creating a conceptual dancing crowd was found.

Due to its intrinsic complexity, full-body postural input has been mostly limited to off-line motion capture and to on-line puppetry of a virtual character with little interaction with its environment (e.g. floor). The motion capture technology is now mature enough to envision the on-line full-body postural control of virtual mannequins involved in precise reaching tasks. We have investigated such tasks for mannequins of differing body heights in comparison to that of the system user. Such broad-range avatar control is relevant for virtual prototyping in various industrial sectors as a single person is responsible for evaluating a virtual prototype for a full range of potential end-users. In the present paper we report on two scaling strategies that can be enforced in such a context of height-differing avatar control. Both scaling strategies have been evaluated in a wide-range reach study both in front of a stationary immersive display and with an HMD. A comparison is also made with a baseline scenario, which exploits a simple rigid shape (i.e. a proxy), to assess the specific influence of controlling a complex articulated avatar.

In this paper we propose the progressive clamping method to better model the kinematic anisotropy of joint limits for virtual mannequins or robots. Like recent approaches our method damps only the joints' variation component heading towards the limits. In addition we propose to dynamically express the corrective joint variation as a highest priority constraint that naturally extends the management of inequality constraints. This process is iterative within linear computing cost of the number of independent joints. We present how our approach is exploited for the major classes of rotation joints from one and up to three degrees of freedom. A comparison with other joint limit avoidance methods is given. We demonstrate the validity of our approach on various experiments targeting on the control of virtual mannequins.

Interactive control of a virtual character through full body movement has a wide range of applications. However, there is a need for systems that accurately reproduce the motion of a performer while accounting for surrounding obstacles. We propose an approach based on a Prioritized Inverse Kinematics constraint solver. Several markers are placed on the user's body. A set of kinematic constraints make the virtual character track these markers. At the same time, we monitor the instantaneous displacements of a set of geometric primitives, called observers, attached to different parts of the virtual character. When an observer enters the influence area of an obstacle, its motion is damped by means of automatically created preventive constraints. The IK solver satisfies both maker and preventive constraints simultaneously, yielding postures of the virtual character that remain close to those of the user, while avoiding collisions with the virtual environment. Our performance measurements show the maturity of the IK technology for real-time full-body interactions.

Abstract This paper explores the issue of interactive low‐dimensional human motion synthesis. We compare the performances of two motion models, i.e. Principal Components Analysis (PCA) or Probabilistic PCA (PPCA), for solving a constrained optimization problem within a low‐dimensional latent space. We use PCA or PPCA as a first step of preprocessing to reduce the dimensionality of the database to make it tractable, and to encapsulate only the essential aspects of a specific motion pattern. Interactive user control is provided by formulating a low‐dimensional optimization framework that uses a Prioritized Inverse Kinematics (PIK) strategy. The key insight of PIK is that the user can adjust a motion by adding constraints with different priorities. We demonstrate the robustness of our approach by synthesizing various styles of golf swing. This movement is challenging in the sense that it is highly coordinated and requires a great precision while moving with high speeds. Hence, any artifact is clearly noticeable in the solution movement. We simultaneously show results comparing local and global motion models regarding synthesis realism and performance. Finally, the quality of the synthesized animations is assessed by comparing our results against a per‐frame PIK technique. Copyright © 2007 John Wiley & Sons, Ltd.

Handling the evolving permanent contact of deformable objects leads to a collision detection problem of high computing cost. Situations in which this type of contact happens are becoming more and more present with the increasing complexity of virtual human models, especially for the emerging medical applications. In this context, we propose a novel collision detection approach to deal with situations in which soft structures are in constant but dynamic contact, which is typical of 3D biological elements. Our method proceeds in two stages: First, in a preprocessing stage, a mesh is chosen under certain conditions as a reference mesh and is spherically sampled. In the collision detection stage, the resulting table is exploited for each vertex of the other mesh to obtain, in constant time, its signed distance to the fixed mesh. The two working hypotheses for this approach to succeed are typical of the deforming anatomical systems we target: First, the two meshes retain a layered configuration with respect to a central point and, second, the fixed mesh tangential deformation is bounded by the spherical sampling resolution. Within this context, the proposed approach can handle large relative displacements, reorientations, and deformations of the mobile mesh. We illustrate our method in comparison with other techniques on a biomechanical model of the human hip joint.

In this paper, we present our spreadsheet framework, which uses a spreadsheet-like interface for exploring biomedical datasets. The principles and advantages of this class of visualization systems are illustrated, and a case study for the analysis of hip joint congruity is presented. Throughout this use case, we see how end users can compare different datasets, apply parallel operations on data, create analysis templates, and how this helps them in the exploration process.

Motion blending is widely accepted as a standard technique in computer animation, allowing the generation of new motions by interpolation and/or transition between motion capture sequences. To ensure smooth and seamless results, an important property has to be taken into account: similar constraints sequences have to be time aligned. But traditional blending approaches let the user choose manually the transition time and duration. In addition, according to the animation context, blending operations should not be performed immediately. They can only occur during a precise period of time, while preserving specific physical properties. We present in this paper an improved blending technique allowing automatic controlled transition between motion patterns whose parameters are not known in advance. This approach ensures coherent movements over the parameter space of the original input motions. To illustrate our approach, we focus on walking and running motions blended with jumps, where animators may vary the jump length and style. The proposed method specifies automatically the support phases of the input motions, and controls on the fly a correct transition time. Moreover the current locomotion type and speed are smoothly adapted given a specific jump type and length.

This paper aims to propose a novel approach to generate new generic human walking patterns using motion-captured data, leading to a real-time engine intended for virtual humans animation. The method applies the PCA (principal component analysis) technique on motion data acquired by an optical system to yield a reduced dimension space where not only interpolation, but also extrapolation are possible, controlled by quantitative speed parameter values. Moreover, with proper normalization and time warping methods, the generic presented engine can produce walking motions with continuously varying human height and speed with real-time reactivity

The present paper describes the integration of a multi-finger haptic device with deformable objects in an interactive environment. Repulsive forces are synthesized and rendered independently for each finger of a user wearing a Cybergrasp force-feedback glove. Deformation and contact models are based on mass-spring systems, and the issue of the user independence is dealt with through a geometric calibration phase. Motivated by the knowledge that human hand plays a very important role in the somatosensory system, we focused on the potential of the Cybergrasp device to improve perception in Virtual Reality worlds. We especially explored whether it is possible to distinguish objects with different elasticities. Results of performance and perception tests are encouraging despite current technical and computational limitations.

This paper addresses evaluation and visualization of stress and strain on soft biological tissues in contact given three-dimensional models of reconstructed organs from magnetic resonance images (MRI), we use an anatomy-based kinematical model combined with a soft tissues model to represent their shape and behavior. Then, we compute resulting distribution of stress and strain on deforming surface when motion is simulated. The computed stress and strain are then effectively visualized using an interactive animation framework. Experimental results are illustrated in the case of the hip joint cartilage.

Abstract Representing motions as linear sums of principal components has become a widely accepted animation technique. While powerful, the simplest version of this approach is not particularly well suited to modeling the specific style of an individual whose motion had not yet been recorded when building the database: it would take an expert to adjust the PCA weights to obtain a motion style that is indistinguishable from his. Consequently, when realism is required, the current practice is to perform a full motion capture session each time a new person must be considered. In this paper, we extend the PCA approach so that this requirement can be drastically reduced: for whole classes of cyclic and noncyclic motions such as walking, running or jumping, it is enough to observe the newcomer moving only once at a particular speed or jumping a particular distance using either an optical motion capture system or a simple pair of synchronized video cameras. This one observation is used to compute a set of principal component weights that best approximates the motion and to extrapolate in real‐time realistic animations of the same person walking or running at different speeds, and jumping a different distance.

We present a system to analyze the reachability of the human body. The inverse kinematics technique is employed to find which regions of space are reachable using a certain reach strategy. This information is stored in a data structure called volume approximation tree (VATree). This tree has proven itself to be an appropriate data structure for two reasons: it provides an efficient representation of the reachable volumes and it reduces the number of inverse kinematics simulations necessary for its construction. Once the VATrees are constructed for the different reach strategies, that information can be used to determine in real time which strategy is most suitable for a given reach task.

Human reaching in a 3D environment is an interesting matter of research due to its application to workplace or vehicle-interior design. We introduce a 3D environment where a virtual human performs reaching tasks over 3D objects in the world. This environment also provides tools to generate and visualize reachable volumes. Reachable spaces are approximated using adjacent box-shaped voxels. We define several strategies in order to model different types of reaching, and we employ our system to construct and analyze reachable spaces for these strategies. In general, different strategies do have reachable spaces that share a common region of intersection. Therefore, goals exists that can be reached using two or more strategies. For those goals, a high-level layer is responsible for selecting the most appropriate given a certain reaching task. As a practical application, this paper presents a comparison of two usual strategies to model standing and seated reaching. The generated reach spaces show that, for each of them, a strategy is clearly more adequate than the other.

Optical motion capture provides an impressive ability to replicate gestures. However, even with a highly professional system there are many instances where crucial markers are occluded or when the algorithm confuses the trajectory of one marker with that of another. This requires much editing work on the part of the animator before the virtual characters are ready for their screen debuts. In this paper, we present an approach to increasing the robustness of a motion capture system by using a sophisticated anatomic human model. It includes a precise description of the skeleton's mobility and an approximated envelope. It allows us to accurately predict the 3-D location and visibility of markers, thus significantly increasing the robustness of marker tracking and assignment, and drastically reducing-or even eliminating-the need for human intervention during the 3D reconstruction process.

We show how the mass properties of an articulated body, such as its center of mass and its moments of inertia, may be manipulated like any conventional end effector with well-known iterative, numerical techniques. Within our interactive simulation testbed software, we illustrate this tool for the geometrical design of rigid bodies with prescribed mass properties, and for the manipulation of human figures in computer animation.

In this paper, we present an integration framework for heterogeneous motion generators. The objective is to outline issues that are currently easily solved in professional post‐processing systems used in film and game production but which cannot be transposed as is to real‐time systems with autonomous agents. We summarise our approach for articulated agent‐modelling and their animation by combining heterogeneous motion generators, such as real‐time motion capturing, key‐framing, inverse kinematics, procedural walking. We propose an agent/action‐oriented framework. Activity properties such as action simultaneity and motion blending, spatial coherence, motion‐flow update schemes, agent attachments, and location corrections, are the main topics handled by our generic animation framework. Numerous examples throughout the paper illustrate our approach and outline encountered problems and solutions or open research directions.

In this paper, we present a new method for solving the Motion Retargeting Problem, by using an intermediate skeleton. This allows us to convert movements between hierarchically and geometrically different characters. An Inverse Kinematics engine is then used to enforce Cartesian constraints while staying as close as possible to the captured motion.

Rendering and animating in real-time a multitude of articulated characters presents a real challenge, and few hardware systems are up to the task. Up to now, little research has been conducted to tackle the issue of real-time rendering of numerous virtual humans. This paper presents a hardware-independent technique that improves the display rate of animated characters by acting on the sole geometric and rendering information. We first review the acceleration techniques traditionally in use in computer graphics and highlight their suitability to articulated characters. We then show how impostors can be used to render virtual humans. We introduce concrete case studies that demonstrate the effectiveness of our approach. Finally, we tackle the visibility issue.

Agents in virtual environments require a combination of perception and action to behave in an autonomous way. We extend a software architecture for the management of actions blending, called AGENTlib, with a perception mechanism. The perception system provides a uniform interface to various techniques in the field of virtual perception, including synthetic vision, database access and perception persistency. We describe the framework we designed to efficiently filter valuable information from the scene and we address concerns about computation redundancy and data propagation through multiple filtering modules.

In this paper, we outline the creation of an Informed Environment, dedicated to urban life simulation.We propose methods and tools for creating and providing the information necessary for animating virtual humans in a city using an Informed Environment. The Informed Environment is based on a hierarchical decomposition of a urban scene into Environment Entities providing geometrical information as well as semantic notions, thus allowing a more realistic simulation of human behaviour. In this manner, virtual humans can integrate with a certain kind of urban knowledge.

We investigate how the movement measurement technologies developed for virtual reality applications can be applied to track in real-time the full body posture of a human being. The accuracy of this information is of definite importance to evaluate the feasibility of complex tasks involving human beings. We present a full body movement measurement approach. It provides a realistic conversion in real-time with a reasonable number of sensors. Associated with the hand movement measurement and correction algorithms we provide a pertinent visual and vibrotactile feedback to the performer.

We compare two formulations for the kinematic control of redundant manipulators, based on task prioritization. We also propose an incremental method to speed up the evaluation of the solution. This is especially interesting when dealing with multiple task-priority levels and with highly redundant structures. Finally, the effectiveness of the approach in the context of posture control of human-like articulated figures is illustrated.

We present an animation interface designed to conveniently control the motion capture process. The transition of performer's hand gestures tracked by a dataglove is recognized for software remote control. An intuitive camera metaphor allows one to specify the viewpoint location using the magnetic sensors strapped to the performer's head. The human motion capture is based on the Anatomical Converter, a toolkit to convert sensor measurements into human anatomical rotations in real time. An improved human motion capture technique, the multi-joint control, is introduced.

The recognition of daily human activities is a decisive interface component for more intuitive virtual reality (VR) interactions. In this paper, we describe a hierarchical model of body actions based on fine-grained action primitives. The associated recognition algorithm allows on-the-fly identification of simultaneous actions. Measurements highlight robustness to participants' variability and high detection rates when using the full potential of the action model. An example illustrates an interaction with a virtual character driven by the participant's action recognition.

Real-time animation of virtual humans requires a dedicated architecture for the integration of different motion control techniques running into so-called actions. In this paper, we describe a software architecture called AGENTlib for the management of action combination. Considered actions exploit various techniques from keyframe sequence playback to inverse kinematics and motion capture. Two major requirements have to be enforced from the end user viewpoint: first, that multiple motion controllers can control simultaneously some parts or whole of the virtual human, and second, that successive actions result in a smooth motion flow

We present a posture design paradigm for the positioning of complex characters. It is illustrated here on human figures. We exploit the inverse kinetics technique which allows the center of mass position control for postures with either single or multiple supports. For the multiple support case, we introduce a compatible flow model of the supporting influence. With this approach, we are able to handle continuous modification of the support distribution. By construction, inverse kinetics presents the same control architecture as inverse kinematics, and thus, it shows equivalent computing cost and similar intuitive concepts. Furthermore, inverse kinetics for the center of mass and inverse kinematics for fixed end effecters can be combined to generate a posture displaying static balance, goal oriented features, and an additional gravity optimization.

The manipulation of a virtual object by means of a digital glove is a tedious task for repetitive position and orientation adjustments. In this paper we try to overcome the lack of force feedback of general purpose digital gloves. We propose a solution based only on the interpretation of instantaneous hand posture variations. From this knowledge, a virtual contact model analog to the Coulomb law allows to evaluate the instantaneous nature of the contact, firm or sliding, between virtual sensors attached to the fingers and the virtual object. Although no force feedback is supplied, we are able to derive a visual restitution consistent with the user manipulative intentions while respecting the integrity of solid interaction with friction.

Abstract We describe the HUMANOID environment dedicated to human modeling and animation for general multimedia, VR, and CAD applications integrating virtual humans. We present the design of the system and the integration of the various features: generic modeling of a large class of entities with the BODY data structure, realistic skin deformation for body and hands, facial animation, collision detection, integrated motion control and parallelization of computation intensive tasks.

The scope of this paper is to propose a system for the automatic derivation of a human curved walking trajectory from the analysis provided by its synthetic vision module. The study context is a planar environment with still and sparse foothold locations. A general methodology associates the two low-level modules of vision and walking with a planification module which establishes the middle term path from the knowledge of the visualized environment. The planification is made under the constraint of minimizing the distance, the speed variation and the curvature cost. Moreover, the planification may trigger the alternate walking motion whenever the decreasing in curvature cost is higher than the associated increasing in speed variation cost due to the corresponding halt and restart. Finally, the characteristic of the next step location is derived in the context of a constant velocity walking motion along a circular trajectory.>

Abstract A new approach for the animation of articulated figures is presented. We propose a system of articulated motion design which offers a full combination of both direct and inverse kinematic control of the joint parameters. Such an approach allows an animator to specify interactively goal‐directed changes to existing sampled joint motions, resulting in a more general and expressive class of possible joint motions. The fundamental idea is to consider any desired‐joint space motion as a reference model inserted into the secondary task of an inverse kinematic control scheme. This approach profits from the use of half‐space Cartesian main tasks in conjunction with a parallel control of the articulated figure called the coach‐trainee metaphor. In addition, a transition function is introduced so as to guarantee the continuity of the control. The resulting combined kinematic control scheme leads to a new methodology of joint‐motion editing which is demonstrated through the improvement of a functional model of human walking.