Errors in natural language generation, so-called hallucinations, remain a critical challenge, particularly in high-stakes domains such as healthcare or science communication. While several automatic metrics have been proposed to detect and quantify hallucinations, such as FactCC, QAGS, FEQA, and FactAcc, these metrics are often unavailable, difficult to reproduce, or incompatible with modern development workflows. We introduce MIRAGE, an open-source Python library designed to address these limitations. MIRAGE re-implements key hallucination evaluation metrics in a unified library built on the Hugging Face framework, offering modularity, reproducibility, and standardized inputs and outputs. By adhering to FAIR principles, MIRAGE promotes reproducibility, accelerates experimentation, and supports the development of future hallucination metrics. We validate MIRAGE by re-evaluating existing metrics on benchmark datasets, demonstrating comparable performance while significantly improving usability and transparency.

The general public often encounters complex texts but does not have the time or expertise to fully understand them, leading to the spread of misinformation. Automatic Text Simplification (ATS) helps make information more accessible, but its evaluation methods have not kept up with advances in text generation, especially with Large Language Models (LLMs). In particular, recent studies have shown that current ATS metrics do not correlate with the presence of errors. Manual inspections have further revealed a variety of errors, underscoring the need for a more nuanced evaluation framework, which is currently lacking. This resource paper addresses this gap by introducing a test collection for detecting and classifying errors in simplified texts. First, we propose a taxonomy of errors, with a formal focus on information distortion. Next, we introduce a parallel dataset of automatically simplified scientific texts. This dataset has been human-annotated with labels based on our proposed taxonomy. Finally, we analyze the quality of the dataset, and we study the performance of existing models to detect and classify errors from that taxonomy. These contributions give researchers the tools to better evaluate errors in ATS, develop more reliable models, and ultimately improve the quality of automatically simplified texts.

To improve communication and collaboration in virtual reality (VR), we suggest going beyond improving behavioral realism on participant avatars. Leveraging VR capabilities, our approach focuses on transforming and amplifying social interactions. VR technology prevents users from naturally expressing social signals, accurately detecting and transmitting them due to still limited technological capabilities, or even fully perceiving them. We propose an augmented model aimed at enhancing collaboration by displaying non-verbal engagement behaviors on listeners’ avatars, even if not executed. In a trio VR session organizing a cultural event, participants were represented by avatars, with or without the augmented model in a within-subject design. Results indicate significantly improved social presence and significant changes in participants’ behaviors, particularly their gaze, demonstrating the transformative potential of social augmentations for collaborative tasks in VR.

EXplainable AI (XAI) offers a wide range of algorithmic solutions to the problem of AI's opacity, but ensuring of their usefulness remains a challenge. In this study, we propose an multi-explanation XAI system using surrogate rules, LIME and nearest neighbor on a random forest. Through an experiment in an e-sports prediction task, we demonstrate the feasibility and measure the usefulness of working with multiple forms of explanation. Considering users' preferences, we offer new perspectives for XAI design and evaluation, highlighting the concept of data difficulty and of the idea of prior agreement between users and AI.

Human Activity Recognition (HAR) based on sen-sors data can be seen as a time series classification problem where the challenge is to handle both spatial and temporal dependencies, while focusing on the most relevant data variations. It can be done using 3D skeleton data extracted from a RGB+D camera. In this work, we propose to improve the spatio-temporal image encoding of 3D skeletons captured from a Kinect sensor, by studying the concept of motion energy which focuses mainly on skeleton joints that are the most solicited for an action. This encoding allows us to achieve a better discrimination for the detection of online activities by focusing on the most significant parts of the actions. The article presents this new encoding and its application for HAR using a deep learning model trained on the encoded 3D skeleton data. For this purpose, we proposed to investigate the knowledge transferability of several pre-trained CNNs provided by Keras. The article shows a significant improvement of the accuracy of the learning according to the state of the art.

This paper investigates the impact of displaying the facial expressions of a user in real time on the performance of the task, the social presence and the behavioral changes of the users interacting in a Virtual Reality environment. To evaluate this approach, we conducted a study where the users collaborated to build a TV stand in dyad including a novice and an expert assistant. The collaborative task was divided into two independent phases: a face-to-face discussion phase without object manipulation, and a furniture assembly phase with object manipulation. Our results indicate that the proposed approach can increase social presence and lead to gaze behavior changes in multi-user environments during the face-to-face phase.

Human Activity Recognition (HAR) using sensor data is a time series classification problem. The challenge lies in effectively managing spatial and temporal dependencies while highlighting crucial data variations. This can be accomplished through 3D skeleton data from an RGB+D camera. The hypothesis of this work is that representing 3D skeleton data as spatio-temporal graphs and utilizing graph embedding techniques and deep learning model, composed from Graph Convolutional Networks (GCN), to learn relationships between joints, and Transformers to focus attention on relevant parts, will improve the performance of HAR. We opted to design this model utilizing a Hill Climbing approach. To facilitate real-time action recognition, we employed the sliding window approach. This method involves segmenting the continuous data stream of 3D skeletons, potentially sourced from a Kinect device, into sequences of uniform length. The article shows a significant improvement of the accuracy of the learning according to the state of the art.

International audience

In this paper, we provided an unsupervised contrastive representation learning method which uses contrastive views in which both spatial and temporal similarity-contrast is balanced. The balanced views are created by taking pixels from the anchor sample and any randomly selected negative sample and balancing the ratio of number of pixels taken from the anchor and the negative. Then these balanced views are paired with the anchor to create the positive contrastive views and all other samples paired with the anchor are taken as negative contrastive views. We made the evaluation using reinforcement learning tasks on Atari games and Deep Mind Control suites (DMControl). Our evaluations on 26 Atari games and six DMControl tasks show that the proposed method is superior in learning spatio-temporally evolving factors of the environment by capturing the relevant task controlling generative factors from the agents’ raw observations.

Neural Architecture Search (NAS) algorithms are used to automate the design of deep neural networks. Finding the best architecture for a given dataset can be time consuming since these algorithms have to explore a large number of networks, and score them according to their performances to choose the most appropriate one. In this work, we propose a novel metric that uses the Intra-Cluster Distance (ICD) score to evaluate the ability of an untrained model to distinguish between data in order to approximate its quality. We also use an improved version of the FireFly algorithm, more robust to the local optimums problem than the baseline FireFly algorithm, as a search technique to find the best neural network model adapted to a specific dataset. Experimental results on the different NAS Benchmarks show that our metric is valid for either scoring CNNs and RNNs, and that our proposed FireFly algorithm can improve the result obtained by the state-of-art training-free methods.

In this article, we evaluate the ability of Hierarchical Temporal Memories (HTM) to process values coming from sensor chains. We present a study on the impact of the HTM parameterization on its ability to predict input values and its robustness to sensor faults. The HTM is evaluated on simulated multivariate time series comprising several causal relations between variables. The results show the ability of HTM to predict future values of multivariate time series and to be robust to sensor faults. We then present which parameters most impact HTM prediction performance and its robustness to faults.

In a medical environment, coordination between medical staff is imperative. In cases in which a human doctor or medical coordinator is not present, patient care, particularly from non-experts, becomes more difficult. The difficulty increases when care is completed at a remote site, for example, on a manned mission to Mars. Communication capability from medical experts on Mars is limited. To address this problem, a medical assistant remote system is proposed to act as a coordinator between the humans present and the remote medical experts. A virtual agent assuming such a role will accept feedback from both, running the situation without errors and additional stress. Leadership styles will be employed by the agent to develop trust and perception of competence among its followers. Additionally, prediction of behaviour and situational changes by both medical professionals and by the agent are necessary in order to combat a 10-minute latency affecting communication between Earth and Mars.

This paper presents an intelligent user interface model dedicated to the exploration of complex databases. This model is implemented on a 3D metaphor: a virtual museum. In this metaphor, the database elements are embodied as museum objects. The objects are grouped in rooms according to their semantic properties and relationships and the rooms organization forms the museum. Rooms? organization is not predefined but defined incrementally by taking into account not only the relationships between objects, but also the user's centers of interest. The latter are evaluated in real-time through user interactions within the virtual museum. This interface allows for a personal reading and favors the discovery of unsuspected links between data. In this paper, we present our model's formalization as well as its application to the context of cultural heritage.

This paper presents a model that provides adaptive and evolutive interaction between a human and a virtual agent. After introducing the theoretical justifications, the aliveness metaphor and the notion of coupling are presented. Then, we propose a formalization of the model that relies on the temporal evolution of the coupling between participants and the existence of phases during the interaction. An example on a fitness exergame is provided and some illustrations show the behavior of the model during an interaction. A video complements this example.

This paper presents a first step in the realization of an interactive user interface that organizes itself according to the user exploration of a database of cultural heritage objects. The first part makes a brief related works and lays the basis of this kind of system according to the enactive paradigm. The second part explains the mechanisms underlying the selforganization of the interface: keywords and cultural heritage objects are 3D graphical entities endowed with autonomous behaviors. They share a common virtual environment. Keywords behaviors are based on boïds flocking simulation while cultural heritage objects appears in an virtual museum which evolves and grows progressively. The result that is presented, is an incremental construction of an interactive and realtime 3D metaphor of virtual museum which is then user-specific.

ABSTRACT To be believable, virtual entities must be equipped with the ability to anticipate, that is, to predict the behavior of other entities and the subsequent consequences on the environment. For that purpose, we propose an original approach where the entity possesses an autonomous world of simulation within simulation, in which it can simulate itself (with its own model of behavior) and simulate the environment (with the representation of the behaviors of the other entities). This principle is illustrated by the development of an artificial juggler in 3D. In this application, the juggler predicts the motion of the balls in the air and uses its predictions to coordinate its own behavior to continue to juggle.Copyright © 2012 John Wiley & Sons, Ltd.

ABSTRACT In some video games, humans and computer programs can play together, each one controlling a virtual humanoid. These computer programs usually aim at replacing missing human players; however, they partially miss their goal, as they can be easily spotted by players as being artificial. Our objective is to find a method to create programs whose behaviors cannot be told apart from players when observed playing the game. We call this kind of behavior abelievable behavior. To achieve this goal, we choose models using Markov chains to generate the behaviors by imitation. Such models use probability distributions to find which decision to choose depending on the perceptions of the virtual humanoid. Then, actions are chosen depending on the perceptions and the decision. We propose a new model, calledChameleon, to enhance expressiveness and the associated imitation learning algorithm. We first organize the sensors and motors by semantic refinement and add a focus mechanism in order to improve the believability. Then, we integrate an algorithm to learn the topology of the environment that tries to best represent the use of the environment by the players. Finally, we propose an algorithm to learn parameters of the decision model. Copyright © 2013 John Wiley & Sons, Ltd.

Abstract This paper deals with simulations of real‐time interactive character behavior. The underlying idea is to take into account principles from cognitive science, in particular, the human ability to anticipate and simulate the world behavior. For that purpose, we propose a conceptual framework where the entity possesses an autonomous world of simulation within simulation, in which it can simulate itself (with its own model of behavior) and the environment (with an abstract representation, which can be learnt, of the other entities behaviors). This principle is illustrated by the development of an artificial juggler, which predicts the motion of balls in the air and uses its predictions to coordinate its own behavior while juggling. Thanks to this model it is possible to add a human user to launch balls that the virtual juggler can catch whilst juggling. Copyright © 2011 John Wiley & Sons, Ltd.

International audience

This article presents advantages of using context to set up a pedagogical assistance for recognition of collective situations in virtual environment for training (VET). We are focusing on generation of explanations to the learner. Two assistances types have been envisaged thanks to context using, the first one consists in guiding the learner before action and the second can be used during action. Those assistances have been set up thanks to contextual graph and consist of animations in the virtual environment

This study concerns virtual environments for training in operational conditions. The principal developed idea is that these environments are heterogeneous and open multi-agent systems. The MASCARET model is proposed to organize the interactions between agents and to provide them reactive, cognitive and social abilities to simulate the physical and social environment. The physical environment represents, in a realistic way, the phenomena that learners and teachers have to take into account. The social environment is simulated by agents executing collaborative and adaptive tasks. These agents realize, in team, procedures that they have to adapt to the environment. The users participate to the training environment through their avatar. In this article, we explain how we integrated in MASCARET models necessary to the creation of Intelligent Tutoring System. We notably incorporate pedagogical strategies and pedagogical actions. We present pedagogical agents. To validate our model, the SECUREVI application for fire-fighters training is developed.