Cooperative gestures are gestures performed by multiple users to facilitate collaboration on large interactive surfaces, such as tabletops. While previous research suggested that such gestures enhance user experience, they overlooked important aspects of user experience and, in particular, the users’ sense of agency. This paper considers two types of cooperative gestures: (i) egalitarian, where both users have the same role, and (ii) hierarchical, where one user (the leader) leads the action, and the other user (the follower) follows the leader’s actions. In a user experiment, we evaluate these types of cooperative gestures by measuring the participants’ sense of agency using intentional binding and judgment of control. Results show a decrease in the sense of agency for the followers. By contrast, leaders and users with egalitarian roles maintained their agency at the implicit sensorimotor level. We conclude with “implications for design” of these findings.

International audience

Being able to produce facial expressions (FEs) that are adequate given a social context is key to harmonious social development, particularly in the case of children plagued with autism spectrum disorder (ASD). In this paper, we introduce JEMImE, a serious game solution that aims at teaching children how to produce FEs. JEMImE is based on a FE recognition module that is learned on a large video corpus of children performing FEs. This module is validated and incorporated through multiple scenarios of gradual difficulty, ranging from a training phase where children have to perform the FEs on request, with or without an avatar model, to an in-context phase that involves many emotion-eliciting social situations with virtual characters.

Gaze acts as a powerful nonverbal behavior that conveys emotional meaning and provides access to others' mental states. The present study sought to investigate how observing others' gaze behaviors can influence one's judgment of personality traits. Twenty four participants were placed face-to-face with a virtual human, alternately female or male, which they had to address verbally. Participants were made to believe that the virtual human was reproducing in real-time the gaze movements of true individuals whom they could not see. In fact, a computer program enabled the virtual human to react to the gaze of the participants that was detected via an eye-tracker. The virtual human displayed four different types of gaze behaviors: high amount of direct gaze, low amount of direct gaze, gaze avoidance and gaze following. After being exposed to each type of gaze behavior, participants were asked to rate the virtual human according to the five factor model of personality. The results showed that increasing eye-contact yielded higher judgments of conscientiousness. Gaze following was appraised as favoring extraversion compared to gaze avoidance. Additionally, personality judgments appeared to be modulated by the gender of the virtual human, with more negative traits attributed to the virtual female.

There are ongoing debates on whether learning involves the same mechanisms when it is mediated by social skills than when it is not [1]. Gaze cues serve as a strong communicative modality that is profoundly human. They have been shown to trigger automatic attentional orienting [2]. However, arrow cues have been shown to elicit similar effects [3]. Hence, gaze and arrow cues are often compared to investigate differences between social and non-social cognitive processes [4]. The present study sought to compare cued learning when the cue is provided by a social agent versus a nonsocial agent.

Many studies suggest that individuals are not able to build a sense of we-agency during joint actions with automated artificial systems. We sought to examine whether or not this lack of sense of control was linked with individuals' inability to represent automaton-generated action into their own cognitive system. Indeed, during human interactions, we automatically represent our partner's actions in our own sensorimotor system. This might sustain our capacity to build a sense of we-agency over our partner-generated actions. However, to our knowledge, no studies have investigated the potential relation between our ability to use our sensorimotor system to represent a partner's action and we-agency. Our approach consisted in performing a pilot study coupling together a Simon target detection task wherein RTs served as an index of action co-representation and an implicit measure of one's sense of agency as indicated by intentional binding phenomenon. The preliminary observations suggested that individuals could represent other-generated action and have a sense of agency over these actions provided that their partner was another human and not an artificial automated system.

In this article the role of different categories of postures in the detection, recognition, and interpretation of emotion in contextually rich scenarios, including ironic items, is investigated. Animated scenarios are designed with 3D virtual agents in order to test 3 conditions: In the “still” condition, the narrative content was accompanied by emotional facial expressions without any body movements; in the “idle” condition, emotionally neutral body movements were introduced; and in the “congruent” condition, emotional body postures congruent with the character's facial expressions were displayed. Those conditions were examined by 27 subjects, and their impact on the viewers’ attentional and emotional processes was assessed. The results highlight the importance of the contextual information to emotion recognition and irony interpretation. It is also shown that both idle and emotional postures improve the detection of emotional expressions. Moreover, emotional postures increase the perceived intensity of emotions and the realism of the animations.

This article analyses the issues pertaining to the simulation of joint attention with virtual humans. Gaze represents a powerful communication channel illustrated by the pivotal role of joint attention in social interactions. To our knowledge, there have been only few attempts to simulate gazing patterns associated with joint attention as a mean for developing empathic virtual agents. Eye-tracking technologies now enable creating non-invasive gaze-contingent systems that empower the user with the ability to lead a virtual human's focus of attention in real-time. Although gaze control can be deliberate, most of our visual behaviors in everyday life are not. This article reports empirical data suggesting that users only have partial awareness of controlling gaze-contingent displays. The technical challenges induced by detecting the user's focus of attention in virtual reality are reviewed and several solutions are compared. We designed and tested a platform for creating virtual humans endowed with the ability to follow the user's attention. The article discusses the advantages of simulating joint attention for improving interpersonal skills and user engagement. Joint attention plays a major role in the development of autism. The platform we designed is intended for research and treatment of autism and tests included participants with this disorder.