International audience

Motivational Interviewing (MI) is an approach to therapy that emphasizes collaboration and encourages behavioral change. To evaluate the quality of an MI conversation, client utterances can be classified using the MISC code as either Change Talk (CT), Sustain Talk (ST), or Follow/Neutral (F/N). The proportion of CT in an MI conversation positively correlates with therapy outcomes, making accurate classification of client utterances essential. This paper presents a classifier that accurately distinguishes between the three MISC classes (CT, ST, and F/N), leveraging multimodal features such as text, prosody, and facial expressivity. We annotate the publicly available AnnoMI dataset to train our model to collect multimodal information. Furthermore, we identify the modality that contributes most to the decision-making process, providing valuable insights into the interplay of different modalities during an MI conversation.

The demand for mental health services has outpaced available resources, resulting in long wait times for patients. A potential solution is to use virtual agents that perform motivational interviews. These agents can be rule-based, requiring expert knowledge, or data-driven, needing large datasets for training, which are often hard to obtain. Patient simulation can generate synthetic data as an alternative. Traditionally, this involved template utterances with a dialog manager or uncontrollable black box large language models LLMs. This study proposes a hybrid approach, leveraging both methods to see if LLMs can follow instructed dialog acts while generating natural, coherent utterances. Our study shows that the language model adheres to given conditions and that conditioning on dialog improves the naturalness and coherence of generated utterances, validating our approach for simulating patient responses.

Embodied Conversational Agents (ECA) present in Digital Health Solutions can contribute to making medical care processes more accessible and addressing the Treatment Gap, a public health issue. This study aims to evaluate the acceptability of an interaction with an ECA on a mobile application to screen for chronic insomnia. It was conducted among a clinical population followed for sleep complaints. Two designs were evaluated, one featuring an ECA and the other presenting only text. Questionnaires and semi-structured interviews allowed the collection of quantitative and qualitative data. We thus measured the level of acceptability of patients for each of the designs. The quantitative results obtained showed no significant results, but the processing of qualitative data allowed for the introduction of new elements, leading to the emergence of improvement avenues and the limitations of our experimental context.

The demand for mental health services has risen substantially in recent years, leading to challenges in meeting patient needs promptly.Virtual agents capable of emulating motivational interviews (MI) have emerged as a potential solution to address this issue, offering immediate support that is especially beneficial for therapy modalities requiring multiple sessions.However, developing effective patient simulation methods for training MI dialog systems poses challenges, particularly in generating syntactically and contextually correct, and diversified dialog acts while respecting existing patterns and trends in therapy data.This paper investigates data-driven approaches to simulate patients for training MI dialog systems.We propose a novel method that leverages time series models to generate diverse and contextually appropriate patient dialog acts, which are then transformed into utterances by a conditioned large language model.Additionally, we introduce evaluation measures tailored to assess the quality and coherence of simulated patient dialog.Our findings highlight the effectiveness of dialog act-conditioned approaches in improving patient simulation for MI, offering insights for developing virtual agents to support mental health therapy.

OBJECTIVE: Fully automated digital interventions show promise for disseminating evidence-based strategies to manage insomnia complaints. However, an important concept often overlooked concerns the extent to which users adopt the recommendations provided in these programs into their daily lives. Our objectives were evaluating users' adherence to the behavioral recommendations provided by an app, and exploring whether users' perceptions of the app had an impact on their adherence behavior. MATERIAL AND METHODS: Case series study of individuals completing a fully automated insomnia management program, conducted by a virtual agent, during December 2020 to September 2022. Primary outcome was self-reported adherence to the behavioral recommendations provided. Perceptions of the app and of the virtual agent were measured with the Acceptability E-Scale and ECA-Trust Questionnaire. Insomnia was evaluated with the Insomnia Severity Index at baseline (phase 1), after 7 days of sleep monitoring (phase 2) and post-intervention (phase 3). RESULTS: A total of 824 users were included, 62.7% female, mean age 51.85 (±12.55) years. Of them, 32.7% reported having followed at least one recommendation. Users' trust in the virtual agent and acceptance of the app were related to a pre-intervention effect in insomnia severity (phase 2). In turn, larger pre-intervention improvements predicted better adherence. Mediational analyses showed that higher levels of trust in the virtual agent and better acceptance of the app exerted statistically significant positive effects on adherence (β = 0.007, 95% CI, 0.001-0.017 and β = 0.003, 95% CI 0.0004-0.008, respectively). DISCUSSION: Users' adherence is motivated by positive perceptions of the app's features and pre-intervention improvements. CONCLUSIONS: Determinants of adherence should be assessed, and targeted, to increase the impact of fully automated digital interventions.

In this work, we present a socially interactive agent able to adapt its conversational strategies to maximize user's engagement during the interaction. For this purpose, we train our agent with simulated users using deep reinforcement learning. First, the agent estimates the simulated user's engagement depending on the latter's nonverbal behaviors and turn-taking status. This measured engagement is then used as a reward to balance the task of the agent (giving information) and its social goal (maintaining the user highly engaged). Agent's dialog acts may have different impact on the user's engagement depending on the latter's conversational preferences.

One potential solution to help people change their eating behavior is to develop conversational systems able to recommend healthy recipes. Beyond the intrinsic quality of the recommendations themselves, various factors might also influence users? perception of a recommendation. Two of these factors are the conversational skills of the system and users' interaction modality. In this paper, we present Cora, a conversational system that recommends recipes aligned with its users? eating habits and current preferences. Users can interact with Cora in two different ways. They can select predefined answers by clicking on buttons to talk to Cora or write text in natural language. On the other hand, Cora can engage users through a social dialogue, or go straight to the point. We conduct an experiment to evaluate the impact of Cora's conversational skills and users' interaction mode on users' perception and intention to cook the recommended recipes. Our results show that a conversational recommendation system that engages its users through a rapport-building dialogue improves users' perception of the interaction as well as their perception of the system.

One of the main challenges for conversational agents is to select the optimal dialogue policy based on the state of the interaction. This challenge becomes even harder when the conversational agent not only has to achieve a specific task, but also aims at building rapport. Although some work already tried to tackle this challenge using a Reinforcement Learning (RL) approach, they tend to consider one single optimal policy for all the users, regardless of their conversational goals. In this work, we describe a framework that allows us to build a RL-based agent able to adapt its dialogue policy depending on its user's conversational goals. After we build a rule-based agent and a user simulator communicating at the dialog-act level, we crowdsource the surface sentences authoring for both the simulated users and the agent, which allow us to generate a dataset of interactions in natural language. Then, we annotate each of these interactions with a single rapport score and analyze the links between simulated users' conversational goals, agent conversational policies, and rapport. Our results show that rapport was higher when both or none of the interlocutors tried to build rapport. We use this result to inform the design of a social reward function, and we rely on this social reward function to train a RL-based agent using an hybrid approach of supervised learning and reinforcement learning. We evaluate our approach by comparing two different versions of our RL-based agent: one that takes users' conversational goals into account and another that does not. The results show that an agent adapting its dialogue policy depending on users' conversational goals performs better.

A critical aspect of any recommendation process is explaining the reasoning behind each recommendation. These explanations can not only improve users' experiences, but also change their perception of the recommendation quality. This work describes our human-centered design for our conversational movie recommendation agent, which explains its decisions as humans would. After exploring and analyzing a corpus of dyadic interactions, we developed a computational model of explanations. We then incorporated this model in the architecture of a conversational agent and evaluated the resulting system via a user experiment. Our results show that social explanations can improve the perceived quality of both the system and the interaction, regardless of the intrinsic quality of the recommendations.

Over the last two decades, Reinforcement Learning (RL) has emerged the method of choice for data-driven dialog management. However, one of the limitations of RL methods for the optimization of dialog managers in the context of virtual conversational agents, is that they require a large amount of data, which is often unavailable, particularly when the dialog deals with complex discourse phenomena. User simulators help address this problem by generating synthetic data to train RL agents in an online fashion. In this work, we extend user simulators to the case of socially-aware conversational agents, that combine task and social functions. We propose a novel architecture that takes into consideration the user's conversational goals and generates both task and social behaviour. Our proposed architecture is general enough to be useful for training socially-aware conversational agents in any domain. As a proof of concept, we construct a user simulator for training a conversational recommendation agent and provide evidence towards the effectiveness of the approach.

Laughter is a social signal capable of facilitating interaction in groups of people: it communicates interest, helps to improve creativity, and facilitates sociability. This article focuses on: endowing virtual characters with computational models of laughter synthesis, based on an expressivity-copying paradigm; evaluating how the physically co-presence of the laughing character impacts on the user’s perception of an audio stimulus and mood. We adopt music as a means to stimulate laughter. Results show that the character presence influences the user’s perception of music and mood. Expressivity-copying has an influence on the user’s perception of music, but does not have any significant impact on mood.

In our demo, LoL, a user interacts with a virtual agentable to copy and to adapt its laughing and expressive behaviorson-the-fly. Our aim is to study copying capabilitiesparticipate in enhancing user’s experience in the interaction.User listens to funny audio stimuli in the presenceof a laughing agent: when funniness of audio increases, theagent laughs and the quality of its body movement (directionand amplitude of laughter movements) is modulated on-theflyby user’s body features.

User interaction with Embodied Conversational Agents (ECA) should involve a significant affective component to achieve realism in communication. This aspect has been studied through different frameworks describing the relationship between user and ECA, for instance alignment, rapport and empathy. We conducted an experiment to explore how an ECA's non-verbal expression can be controlled to respond to a single affective dimension generated by users as input. Our system is based on the mapping of a high-level affective dimension, approach/avoidance, onto a new ECA control mechanism in which Action Units (AU) are activated through a neural network. Since 'approach' has been associated to prefrontal cortex activation, we use a measure of prefrontal cortex left-asymmetry through fNIRS as a single input signal representing the user's attitude towards the ECA. We carried out the experiment with 10 subjects, who have been instructed to express a positive mental attitude towards the ECA. In return, the ECA facial expression would reflect the perceived attitude under a neurofeedback paradigm. Our results suggest that users are able to successfully interact with the ECA and perceive its response as consistent and realistic, both in terms of ECA responsiveness and in terms of relevance of facial expressions. From a system perspective, the empirical calibration of the network supports a progressive recruitment of various AUs, which provides a principled description of the ECA response and its intensity. Our findings suggest that complex ECA facial expressions can be successfully aligned with one high-level affective dimension. Furthermore, this use of a single dimension as input could support experiments in the fine-tuning of AU activation or their personalization to user preferred modalities.

The objective of my Ph.D is to develop a computational model of social relations for artificial companions. Such a model should consider the representation of the social relation, its initialization and its dynamics over time. Moreover, the influence of this relation on the agent's decision-making should be modeled. In the proposed computational model described in this paper, the representation, initialization, and dynamics of the social relation are based on the research in Human and Social Sciences, considering the influence of individual and social parameters such as the personality of the agent or its social role.