Training machine learning models to identify human behavior is a difficult yet essential task to develop autonomous and adaptive systems such as smart homes. These models require large and diversified amounts of labeled data to be trained effectively. Due to the high variety of home environments and occupant behaviors, collecting datasets that are representative of all possible homes is a major challenge. In addition, privacy and cost are major hurdles to collect real home data. To avoid these difficulties, one solution consists of training these models using purely synthetic data, which can be generated through the simulation of home and their occupants. Two challenges arise from this approach: designing a methodology with a simulation able to generate credible simulated data and evaluating this credibility. In this article, we explain the methodology used to generate diversified synthetic data of daily activities, through the combination of an agent model to simulate an occupant and a simulated 3D house enriched with sensors and effectors to produce such data. We demonstrate the credibility of the generated synthetic data by comparing their efficacy for training human context understanding models against the efficacy generated by real data. To achieve this, we replicate a real dataset collection setting with our smart home simulator. The occupant is replaced by an autonomous agent following the same experimental protocol used for the real dataset collection. This agent is a BDI-based model enhanced with a scheduler designed to offer a balance between control and autonomy. This balance is useful in synthetic data generation since strong constraints can be imposed on the agent to simulate desired situations while allowing autonomous behaviors outside these constraints to generate diversified data. In our case, the constraints are those imposed during the real dataset collection that we want to replicate. The simulated sensors and effectors were configured to react to the agent’s behaviors similarly to the real ones. We experimentally show that data generated from this simulation are valuable for two human context understanding tasks: current human activity recognition and future human activity prediction. In particular, we show that models trained solely with simulated data can give reasonable predictions about real situations occurring in the original dataset. We also report experimental results regarding statistical analysis and C2ST to assess the credibility of generated data. We discuss the generality of our approach for evaluating the credibility of simulated data from their use as training data.

Autonomy and the ability to maintain social activities can be challenging for people with disabilities experiencing reduced mobility. In the case of disabilities that impact mobility, power wheelchairs can help such people retain or regain autonomy. Nonetheless, driving a power wheelchair is a complex task that requires a combination of cognitive, visual and visuo-spatial abilities. In practice, people need to pass prior ability tests and driving training before being prescribed a power wheelchair by their therapist. Still, conventional training in occupational therapy can be insufficient for some people with severe cognitive and/or visuo-spatial functions. As such, these people are often prevented from obtaining a power wheelchair prescription from their therapist due to safety concerns. In this context, driving simulators might be efficient and promising tools to provide alternative, adaptive, flexible, and safe training. In previous work, we proposed a Virtual Reality (VR) driving simulator integrating vestibular feedback to simulate wheelchair motion sensations. The performance and acceptability of a VR simulator rely on satisfying user Quality of Experience (QoE). Therefore, our simulator is designed to give the user a high Sense of Presence (SoP) and low Cybersickness. This paper presents a pilot study assessing the impact of the vestibular feedback provied on user QoE. Participants were asked to perform a driving task whilst in the simulator under two conditions: with and without vestibular feedback. User QoE is assessed through subjective questionnaires measuring user SoP and cybersickness. The results show that vestibular feedback activation increases SoP and decreases cybersickness. This study constitutes a mandatory step before clinical trials and, as such, only enrolled people without disabilities.

Due to the nature of Virtual Reality (VR) research, conducting experiments in order to validate the researcher's hypotheses is a must. However, the development of such experiments is a tedious and time-consuming task. In this work, we propose to make this task easier, more intuitive and faster with a method able to describe and generate the most tedious components of VR experiments. The main objective is to let experiment designers focus on their core tasks: designing, conducting, and reporting experiments. To that end, we propose the use of Domain-Specific Languages (DSLs) to ease the description and generation of VR experiments. An analysis of published VR experiments is used to identify the main properties that characterize VR experiments. This allowed us to design AGENT (Automatic Generation of ExperimeNtal proTocol runtime), a DSL for specifying and generating experimental protocol runtimes. We demonstrated the feasibility of our approach by using AGENT on two experiments published in the VRST'16 proceedings.

Does it feel the same when you touch an object in Augmented Reality (AR) or in Virtual Reality (VR)? In this paper we study and compare the haptic perception of stiffness of a virtual object in two situations: (1) a purely virtual environment versus (2) a real and augmented environment. We have designed an experimental setup based on a Microsoft HoloLens and a haptic force-feedback device, enabling to press a virtual piston, and compare its stiffness successively in either Augmented Reality (the virtual piston is surrounded by several real objects all located inside a cardboard box) or in Virtual Reality (the same virtual piston is displayed in a fully virtual scene composed of the same other objects). We have conducted a psychophysical experiment with 12 participants. Our results show a surprising bias in perception between the two conditions. The virtual piston is on average perceived stiffer in the VR condition compared to the AR condition. For instance, when the piston had the same stiffness in AR and VR, participants would select the VR piston as the stiffer one in 60% of cases. This suggests a psychological effect as if objects in AR would feel "softer" than in pure VR. Taken together, our results open new perspectives on perception in AR versus VR, and pave the way to future studies aiming at characterizing potential perceptual biases.

This paper presents the use of Virtual Reality (VR) for movie actors rehearsal of VFX-enhanced scenes. The impediment behind VFX scenes is that actors must be filmed in front of monochromatic green or blue screens with hardly any cue to the digital scenery that is supposed to surround them. The problem is worsens when the scene includes interaction with digital partners. The actors must pretend they are sharing the set with imaginary creatures when they are, in fact, on their own on an empty set. To support actors in this complicated task, we introduce the use of VR for acting rehearsals not only to immerse actors in the digital scenery but to provide them with advanced features for rehearsing their play. Indeed, our approach combines a fully interactive environment with a dynamic scenario feature to allow actors to become familiar with the virtual elements while rehearsing dialogue and action at their own speed. The interactive and creative rehearsals enabled by the system can be either single-user or multi-user. Moreover, thanks to the wide range of supported platforms, VR rehearsals can take place either on-set or off-set. We conducted a preliminary study to assess whether VR training can replace classical training. The results show that VR-trained actors deliver a performance just as good as ordinarily trained actors. Moreover, all the subjects in our experiment preferred VR training to classic training.

We propose a method to perform collision detection with cloths with ray-tracing. Our method is able to perform collision detection between cloths and volumetric objects (rigid or deformable) as well as collision detection between cloths (including auto-collision). Our method casts rays between objects to perform collision detection, and an inversion-handling algorithm is introduced to correct errors introduced by discrete simulations. GPU computing is used to improve the performances. Our implementation handles scenes containing deformable objects at an interactive frame-rate, with collision detection lasting a few milliseconds.

Summary form only given. The CORVETTE project aims at producing significant innovations in the field of collaborative virtual training. For that purpose, CORVETTE combines various technologies to enhance the effective collaboration between users and virtual humans performing a common task. First, CORVETTE proposes a model of collaborative interaction in virtual environments allowing actors to efficiently collaborate as a team whether they are controlled by a user or by a virtual human [4]. Moreover, the environment is simulated in real-time, at real scale and is using physics as well as physically-based humanoids to improve the realism of the training. Second, thanks to the interaction model, we defined a protocol of exchange of avatars [5, 3, 2]. Thus, an actor can dynamically exchange the control of his/her avatar with the one controlled by another user or by a virtual agent. Moreover, to improve the exchange protocol, we designed a new knowledge model embedded in each avatar. It allows users and virtual humans to retrieve knowledge previously gathered by an avatar following an exchange. The preservation of knowledge is indeed especially crucial for teamwork. Finally, we handle verbal communication between users and virtual humans with speech recognition and synthesis. Actors' knowledge is enhanced through dialogue and used for decision-making and conversation [1].

ABSTRACT Virtual agents are a real asset in collaborative virtual environment for training (CVET) as they can replace missing team members. Collaboration between such agents and users, however, is generally limited. We present here a whole integrated model of CVET focusing on the abstraction of the real or virtual nature of the actor to define a homogenous collaboration model. First, we define a new collaborative model of interaction. This model notably allows to abstract the real or virtual nature of a teammate. Moreover, we propose a new role exchange approach so that actors can swap their roles during training. The model also permits the use of physically based objects and characters animation to increase the realism of the world. Second, we design a new communicative agent model, which aims at improving collaboration with other actors using dialog to coordinate their actions and to share their knowledge. Finally, we evaluated the proposed model to estimate the resulting benefits for the users and we show that this is integrated in existing CVET applications. Copyright © 2014 John Wiley & Sons, Ltd.

The exchange of avatars, i.e. the actual fact of changing once avatar with another one, is a promising trend in multi-actor virtual environments. It provides new opportunities for users, such as controlling a different avatar for a specific action, retrieving knowledge belonging to a particular avatar, solving conflicts and deadlocks situations or even helping another user. Virtual Environments for Training are especially affected by this trend as a specific role derived from a scenario is usually assigned to a unique avatar. Despite the increasing use of avatar exchange, users' perception and understanding of this mechanism have not been studied. In this paper, we propose two complementary user-centered evaluations that aim at comparing several representations for the exchange of avatars; these are termed exchange metaphors. Our first experiment focuses on the perception of an exchange by a user who is not involved in the exchange, and the second experiment analyzes the perception of an exchange triggered by the user. Results show that the use of visual feedback globally aids better understanding of the exchange mechanism in both cases. Our first experiment suggests, however, that visual feedback is less efficient than a simple popup notification in terms of task duration. In addition, the second experiment shows that much simpler metaphors with no visual effect are generally preferred because of their efficiency.

We present IRTCD, a novel Iterative Ray-Traced Collision Detection algorithm that exploits spatial and temporal coherency. Our approach uses any existing standard ray-tracing algorithm and we propose an iterative algorithm that updates the previous time step results at a lower cost with some approximations. Applied for rigid bodies, our iterative algorithm accelerate the collision detection by a speedup up to 33 times compared to non-iterative algorithms on GPU.

In this paper we introduce novel 'Camera Motions' (CMs) to improve the sensations related to locomotion in virtual environments (VE). Traditional Camera Motions are artificial oscillating motions applied to the subjective viewpoint when walking in the VE, and they are meant to evoke and reproduce the visual flow generated during a human walk. Our novel camera motions are: (1) multistate, (2) personified, and (3) they can take into account the topography of the virtual terrain. Being multistate, our CMs can account for different states of locomotion in VE namely: walking, but also running and sprinting. Being personified, our CMs can be adapted to avatar's physiology such as to its size, weight or training status. They can then take into account avatar's fatigue and recuperation for updating visual CMs accordingly. Last, our approach is adapted to the topography of the VE. Running over a strong positive slope would rapidly decrease the advance speed of the avatar, increase its energy loss, and eventually change the locomotion mode, influencing the visual feedback of the camera motions. Our new approach relies on a locomotion simulator partially inspired by human physiology and implemented for a real-time use in Desktop VR. We have conducted a series of experiments to evaluate the perception of our new CMs by naive participants. Results notably show that participants could discriminate and perceive transitions between the different locomotion modes, by relying exclusively on our CMs. They could also perceive some properties of the avatar being used and, overall, very well appreciated the new CMs techniques. Taken together, our results suggest that our new CMs could be introduced in Desktop VR applications involving first-person navigation, in order to enhance sensations of walking, running, and sprinting, with potentially different avatars and over uneven terrains, such as for: training, virtual visits or video games.

The Walking-In-Place interaction technique was introduced to navigate infinitely in 3D virtual worlds by walking in place in the real world. The technique has been initially developed for users standing in immersive setups and was built upon sophisticated visual displays and tracking equipments.

In this paper we introduce a "spatialized haptic rendering" technique to enhance 6DOF haptic manipulation of virtual objects with impact position information using vibrations. This rendering technique uses our perceptive ability to determine the contact position by using the vibrations generated by the impact. In particular, the different vibrations generated by a beam are used to convey the impact position information. We present two experiments conducted to tune and evaluate our spatialized haptic rendering technique. The first experiment investigates the vibration parameters (amplitudes/frequencies) needed to enable an efficient discrimination of the force patterns used for spatialized haptic rendering. The second experiment is an evaluation of spatialized haptic rendering during 6DOF manipulation. Taken together, the results suggest that spatialized haptic rendering can be used to improve the haptic perception of impact position in complex 6DOF interactions.

The use of Virtual Environments for Training is strongly stimulated by important needs for training on sensitive equipments. Yet, developing such an application is often done without reusing existing components, which requires a huge amount of time. We present in this paper a full authoring platform to facilitate the development of both new virtual environments and pedagogical information for procedural training. This platform, named GVT (Generic Virtual Training) relies on innovative models and provides authoring tools which allow capitalizing on the developments realized. We present a generic model named STORM, used to describe reusable behaviors for 3D objects and reusable interactions between those objects. We also present a scenario language named LORA which allows non computer scientists to author various and complex sequences of tasks in a virtual scene. Based on those models, as an industrial validation with Nexter-Group, more than fifty operational scenarios of maintenance training on military equipments have been realized so far. We have also set up an assessment campaign, and we will expose in this paper the first results which show that GVT enables trainees to learn procedures efficiently. The platform keeps on evolving and training on collaborative procedures will soon be available.

In this paper, we describe a system used in the context of virtual training on collaborative maintenance procedures where the focus is on the learning of the industrial procedure rather than technical gestures. In existing collaborative virtual environments for training the distribution of scenario actions among actors is fixed: only one role can be associated with a given scenario action. In this paper, we propose to overcome this limitation and to add a mechanism to deal with this new flexibility. This mechanism is able to dynamically select the best actor for an action, based on various criteria, and to propose a distribution of actions among actors. We also propose to add collaborative profiles to virtual humans to guide them in order to select the next action to perform, possibly following the distribution suggestion. Trainees and virtual humans can then adapt their activities while respecting the reference procedure.

International audience

Using motion capture systems to animate humanlike figures still remains difficult when the movements are complex or need to be adapted to geometric constraints. We propose a new method to blend several captured movements while adapting the trajectories to new skeletons and to unknown environments. For each body part (considered as resources), a priority is defined for each movement (considered as consumers). The trajectories applied to the skeleton consist of a weighted sum of the motions trajectories. A new technique to compute the weights is proposed. Finally, the system adapts the resulting trajectories to the synthetic skeleton and to the environment. The results enabled to animate up to one hundred actors in interactive environments.

We present in this paper a new and original application for saliency maps, intending to simulate the visual perception of a synthetic actor. Within computer graphics field, simulating virtual humans has become a challenging task. Animating such an autonomous actor within a virtual environment requires most of the time in modeling of a perception-decision-action cycle. To model a part of the perception process, we have designed a new model of saliency map, based on geometric and depth information, allowing our synthetic humanoid to perceive its environment in a biologically plausible way.

We present an animation technique to control a humanoid in a virtual environment. The automatic generation of humanoid motion is difficult and needs to be simplified. The solution proposed in this paper consists of controlling humanoid motion through the image it "perceives": through-the-eyes control. The considered approach is based on the visual servoing concept. It allows the automatic generation of (virtual) camera motion by simply specifying the task in the image space. This approach is suited to highly reactive contexts (video games, virtual reality). We also discuss the integration of such techniques in a more complex behavioral system.

The paper introduces a novel method for the coordination of human motion based on planification and AI techniques. Motions are considered as black boxes that are activated according to pre-conditions and produce post-conditions in a hybrid continuous and discrete world. Each part of the body is an autonomous entity which cooperates with the others depending on global criteria such as occupation rate and distance to a goal (common to all the entities). This technique makes it possible to easily specify and solve the motion coordination problem of a juggler that deals with a dynamic number of balls in real time.

This work presents a simulation system designed to carry-out virtual experiments on human motion. 3D visualization, automatic code generation and generic control design patterns provide biomechanicians and medics with dynamic simulation tools. The paper first deals with the design of mechanical models of human beings. It also presents design patterns of controllers for an upper-limb model composed of 11 degrees of freedom. As an example, two controllers are presented in order to illustrate these design patterns. The paper also presents a user-friendly interface dedicated to medics that makes it possible to enter orders in natural language.

This paper presents a comprehensive system for adaptive replay of a computer animation sequence featuring the locomotion of a synthetic animal. The kinematic locomotion gait, which is supposed to be mostly stationary, is tuned before being injected on the dynamic model of the animated animal. The animator can take benefit from this new flexibility in order to fulfil some goals, like steering the animal towards a target. The reference gait kinematic trajectories are expanded on a wavelet redundant bases, and tuned through varying the wavelet parameters. Because the optimization process is time-consuming, we propose to carry-out all optimizations beforehand and send the outcomes in a training set for a neural network, whose role is to compress data and to interpolate in between the training samples. At runtime, the network emulates the behaviour of the optimization process and automatically tailors the gait parameters. This approach paves the way towards the definition of a ready-to-use model for animated gaits.>

This paper describes a new algorithm of space tracing. Scenes are modeled by a CSG tree. Space is subdivided regularly into 3D regions called boxes. With each box is associated a subtree which is the restriction of the whole scene CSG tree to primitives belonging to this box. A 3D grid is used to access boxes.