In this paper, we present a decision model aimed at determining when a social touch by a socially interactive agent (SIA) would be considered socially ‘correct’ : both coherent (meaningful given the current situation) and acceptable to the human interlocutor. Those decisions are computed by taking the context of interaction and the estimated level of rapport between the human and the agent into account. We then present a study based on a vicarious protocol where participants evaluated recorded interactions between an avatar (a human-controlled 3D character) and our agent. Results indicate that the estimations made by our decision model regarding the state of the situation mirror those made by human observers. Our initial hypothesis that the level of rapport would positively influence the acceptability of a touch receives mixed support. Social touch occurrences did not feel as coherent and acceptable as hoped yet. Avenues of improvement for human-agent interactions including social touch are discussed.

Endowing socially interactive virtual agents with the social touch modality could improve their emotional communication abilities and help them bond with human users. Touch is however a sensitive channel of communication that can have negative effects if used inappropriately. We present a first implementation of a system enabling an agent to perform social touch-based interactions and a preliminary study of the system dedicated to determining the factors that come into play for the social acceptability of an agent-initiated touch. The relationship between social touch with a virtual agent and sense of embodiment is also discussed. Based on the results of the study, we propose social (macro) and practical (micro) insights into how to produce meaningful, coherent and acceptable touch decisions (when to touch or not depending on the situation).

Unmanned Aerial Vehicles (UAVs) exhibit great agility but usually require an experienced pilot to operate them in certain applications such as inspection for disaster scenarios or buildings. The reduction of cognitive overload when driving this kind of aerial robot becomes a challenge and several solutions can be found in the literature. A new virtual control scheme for reducing this cognitive overload when controlling an aerial robot is proposed in this paper. The architecture is based on a novel interaction Drone Exocentric Advanced Metaphor (DrEAM) located in a Cave Automated Virtual Environment (CAVE) and a real robot containing an embedded controller based on quaternion formulation. The testing room, where real robots are evolving, is located away from the CAVE and they are connected via UDP in a ground station. The user controls manually a virtual drone through the DrEAM interaction metaphor, and the real robot imitates autonomously in real time the trajectory imposed by the user in the virtual environment. Experimental results illustrate the easy implementation and feasibility of the proposed scheme in two different scenarios. Results from these tests show that the mental effort when controlling a drone using the proposed virtual control scheme is lower than when controlling it in direct view. Moreover, the easy maneuverability and controllability of the real drone is also demonstrated in real time experiments.

International audience

In the past decade, Pedestrian Collision Warning Systems have been proposed to detect pedestrians and warn drivers of imminent collision. However, such systems are often limited by eye-off-road and cognitive overload problems. Head Up displays with augmented reality are being considered as a key technology for changing drivers’ user experiences. In this paper, we propose a new visual assistance system that can enhance drivers’ perception by dynamically directing attention to pedestrians to avoid collisions using Augmented Reality cues. The proposed system takes into account driver behaviors through vehicle driving signals analysis, in order to warn it at the right moments. To that end, we statistically model correct and incorrect driver behaviors in situations with pedestrians. Based on this model, a warning visual metaphor is displayed if unawareness is detected and a driving simulator was used to evaluate the concept. The experimental results suggest that our proposed adaptive visual aids can enhance driver awareness of pedestrians in critical situations.

Although virtual reality (VR) has many applications, only few studies have investigated user acceptance of this type of immersive technology. We propose an extended version of the Technology Acceptance Model (TAM) that addresses some aspects of VR. Our model includes variables from the TAM, user experience, variables specific to VR, and variables relating to user characteristics. This model was tested with 89 users who performed an aeronautical assembly task in VR. Results suggest that intention to use VR is positively influenced by perceived usefulness and negatively influenced by cybersickness. Hedonic quality-stimulation and personal innovativeness are predictors of perceived usefulness. Perceived ease of use does not have a significant impact on intention to use and it is only influenced by pragmatic quality. These findings have a number of implications regarding user acceptance of VR.

Augmented feedback has been shown to improve interaction in virtual environments and to facilitate motor learning. Recent studies proposed this type of feedback to guide users, to highlight specific areas or to help them to perform a specific task. They can follow a path, pass through specific waypoints or even mimic an avatar. However these approaches do not show the gap between learners' performance and the desired one. Our hypothesis is that by revealing this gap to the users, they will reduce it step by step and tend to the required performance. Thus, in this paper, we propose a new visual metaphor to guide trainees' gestures by showing trajectory errors instead of showing the path to follow. In a first study we evaluated trainees' improvement by measuring the mentioned gap. First results indicate that our approach allows an enhanced task performance.

Gesture training, especially for technical gestures, requires supervisors to point out errors made by trainees. Virtual reality (VR) makes it possible to reduce reliance on supervisors (fewer interventions and of shorter duration) and to reduce the length of training, using extrinsic feedback that provides training or learning assistance using different modalities (visual, auditory, and haptic). Visual feedback has received much attention in recent decades. Users can be guided by a metaphor in a virtual environment. This metaphor may be a 3D trace of canonical movements, a visual cue pointing in the right direction, or gestures by an avatar that the trainee must mimic. However, with many kinds of feedback, trainees are not aware of their errors while performing gestures. Our hypothesis is that guiding users with a dynamic metaphor based on the visualization of errors will reduce these errors and improve performance. To this end, in a previous work we designed and implemented a new 3D metaphor called EBAGG to guide users in real time.

Pedagogical Action Selection (PAS) is a major issue for intelligent tutoring and training systems. Expert knowledge provides useful insights to build strategies that relate students representation to PAS, but it can be difficult to collect. Furthermore, the influence of a specific action may vary across students, which is rarely reflected in expert knowledge. As part of an automatic gesture training system, we propose to model the co-evolution between a student and a training environment in order to provide personalized action selection. The proposed approach is based on three models representing the student, the environment, and the interactions between these two entities. The latter model sees the PAS as a multi-armed bandit problem, each arm representing a possible action. Thus, PAS personalization only relies on the interactions between the student and the learning environment, without any prior knowledge. Two experiments, one in a simulated environment and a second in a calligraphy training environment, highlight the model ability to personalize action selection, and the benefits of this ability on students skill acquisition.

Understanding driver behaviors is an important need for the Advanced Driver Assistance Systems. In particular, the pedestrian detection systems become extremely distracting and annoying when they inform the driver with unnecessary warning messages. In this paper, we propose to study the driver behaviors whenever a pedestrian appears in front of the vehicle. A method based on the driving actions and the Hidden Markov Model (HMM) algorithm is developed to classify the driver awareness of pedestrian and the driver unawareness of pedestrian. The method is successfully validated using the collected data from the experiments that are conducted on a driving simulator. Furthermore, two simple methods based on the static parameters such as the Time-To-Collision and the Required Deceleration Parameter are also applied to our problem and are compared to the proposed method. The result shows a significant improvement of the HMM-based method compared to the simple ones.

Using gestures for natural interactions in virtual environments require robust and smart recognition systems. In these contexts, gestures and gesticulations are part of the a continuous information stream: the first are sufficient to convey meaningful information such as commands and indications. On the contrary, gesticulations are unconscious body movements performed mainly, to support speech. In the majority of gestures recognition systems, the implicit assumption of "isolated patterns" is made. Indeed, following the Kendon's morpho-kinetics model, a gesture is the part of the armmovement contained between the pre-stroke and the post-stroke. This strong assumption shifts the recognition problem toward a clustering issue, e.g., recognizing an isolated temporal pattern. From the practical point of view, the isolated gestures hypothesis needs a cooperation from the user and the later should emphasize the pre and the post strokes. This removes the naturalness of the targeted interface. In this contribution, we focus on having a strong segmentation technique that clusters the body movements into consistent sequences. In this paper, we present a non-parametric stochastic segmentation algorithm that is able to cluster the continuous time series representing body movements into gestures and non-gestures segments. We show as well how this technique allows any novice user creating in a semi-supervised way, his or her, own gestures library. The proposed system is assessed through a real-life example, where a novice user creates an adhoc interface to control an artificial agent in a natural way.

Interacting naturally with artificial agents and environments including virtual avatars and robots rely mainly on gestures. However, the later require heavy setup procedures before any effective use. Specific and personalized training sessions are needed. In addition, performers have to indicate clear separations between gestures, namely, specifying pre- and post-strokes as well as the gesture used to perform the command. Thus, time series segmentation appears as a central problem. Indeed, clustering human motions into meaningful segments or isolating meaningful segments forming a continuous movement flow present the same problem: how to find the post and the pre strokes. For machine learning, this problem is solved by having training sets of carefully labeled data. Good segmentation improves the quality of the gesture recognition-based interface. In our contribution, we focus on developing a non-parametric stochastic segmentation algorithm. Once the segmentation has been validated, we show how any novice user can create in a semi-supervised way, his or her, own gestures library. In addition, we show how the obtained system is efficient in finding meaningful gestures (the once learned earlier) within continuous movements flow, thus removing the constraint of performing manual specification of respectively the beginning of the movement and its end. The proposed technique is assessed through a real-life example, where a novice user creates an ad-hoc interface to control a robot in a natural way.

As training systems are often designed for procedural task learning, we propose an adaptive training system (OSE) in this context to explore how informed interaction supported by a decision making module (GULLIVER) contributes to adaptive knowledge acquisition in a virtual environment. Visualization metaphors and audio messages are displayed as feedback to guide the learner, which is essential to this training. First results show that our system provides an adaptive guiding which is efficient for beginners. Though we focus on fluvial navigation training, the results may be relevant to other applications.

Taking into account the drivers' state is a major challenge for designing new advanced driver assistance systems. In this paper we present a driver assistance system strongly coupled to the user. DAARIA1stands for Driver Assistance by Augmented Reality for Intelligent Automobile. It is an augmented reality interface powered by several sensors. The detection has two goals: one is the position of obstacles and the quantification of the danger represented by them. The other is the driver's behavior. A suitable visualization metaphor allows the driver to perceive at any time the location of the relevant hazards while keeping his eyes on the road. First results show that our method could be applied to a vehicle but also to aerospace, fluvial or maritime navigation.

This paper presents an industrial application called “RVPI” [1] for training operators in a tyre manufacturing plant. This training is performed through an Informed Virtual Environment integrating different aspects of human activity and expertise. To create the system, knowledge extraction and a co-design of the system have been realized in collaboration with the German tyre manufacturer Continental. The system is deployed in real plants in five different countries and is now becoming a regular training tool. Results show that production costs are reduced with RVPI due to a shorter training period and a better understanding of the process.

This article describes the concept of knowledge integration in a 3D collaborative virtual environment. This concept characterizes collaborative design in terms of two models introduced in this article -an annotation model with a semantic metadata notion - and a knowledge model to capitalize and to manage annotations - MATRICS environment. We briefly describe this approach. Then we present the results of an empirical study in which subjects performed annotation tasks with the support of an ontology visualization 2D tool and through the navigation in the 3D annotations. This study shows that the knowledge model improves the quality of the understanding of design projects.

With the intensification and the expansion of collaboration modes between designers and engineers for better time and quality mastering, the communication efficiency between these two communities of practice have to be thought taking in to account both designers and engineer's representation systems. While designers participate to the concept research and deliver the product's scenario of use through images, sketches and renderings, the engineers participate to the physical definition and to the materialization of these concepts. There is a need for a common representation of the concepts and the product, i.e. explicit link between the subjective data coming from the designer and the objective data materializing the product. Nevertheless semantic product characterization have specific tools and methods usually found in 2D environments and 3D CAD systems do not answer to the needs of exchanges in a multi disciplinary design team. In this paper we introduce a virtual environment to integrate, capitalize and explore knowledge around the virtual mock up in order to facilitate subjective and objective product characterization for designers and engineers collaboration. This virtual environment (MATRICS) allows the user to post multi-media annotations on a virtual 3D mock up, to support intentions transferred from the designer to the engineer when communicating. Another aspect of this environment is to provide a shared visualization of knowledge connected to the model and users interaction with knowledge representation increasing the collaboration level while designing.

We analyze a real example of collaborative work in mechanical design. Two kinds of problems emerge from this analysis: the first is a lack of complete design environment very early before modeling. The second is the need for a flexible and very intuitive collaborative environment. In addition, there is no knowledge management of the exchanges, and the design method could be improved with information assistance during the project. From this analysis, we try to detect design and collaborative tasks for which students need help. Then a personal assistant agent based on a task ontology is proposed to offer them this help.