This paper proposes a novel approach for the dynamic generation of training scenarios in Virtual Reality (VR), in the context of first-responders training. Our proposed model leverages an ontology, Large Language Models (LLMs) and Fuzzy Cognitive Maps (FCM) in order to produce scenarios meant to train first responders emotion regulation skills to be better prepared for complex, high-risk situations. These scenarios take into account users’ personal profiles, containing notably individual emotional vulnerabilities, i.e. which situational elements are susceptible to stress them more than others, in order to produce personalized, effective training experiences. The Fuzzy Cognitive Maps allow the system to model a degree of uncertainty when choosing story events, with probabilities for choosing the next scenario event being influenced by several factors, mainly the user’s profile and the virtual world state. Our evaluation shows that for randomized initial situations and user profiles, our system generates varied story events, offering diversity in generated scenarios.

International audience

International audience

Training people for facing dangerous situations has always been expensive and difficult or dangerous to organize. New approaches use virtual environments and simulation in which the role of collaborative people is played by cognitive agents, which allows making training sessions when they were not previously possible, or else decreasing their cost. Two problems arise: firstly, it is necessary to simulate teams, composed of virtual agents and the learner, interacting in a collaborative context. Secondly, we need to support the definition of a variety of explainable and professional behaviours (i.e. representative of the expected behaviours of an expert in a given profession). It is extremely complex and time consuming. To facilitate the design of tools allowing to build adequate agents model, we propose a meta-model of human behavior simulation. The originality of this meta-model is to make the cognitive models, used for explainability, and the agent model completely independent, to avoid repeated implementations of various behaviours. We show how task selection works on a simple agent model, by means of a graph of influences and preferences. This means has the advantage of being intelligible by its visual nature and of facilitating the addition of new behaviours impacting on the selection. A preliminary evaluation has shown that human users perceive the variability of behaviour and that the provided explanations are relevant.

Virtual environments offer the potential to train people for facing complex professional situations. A particularly interesting training context for virtual reality is when such situations involve risk taking and are potentially dangerous for human life. We are specifically interested in training commanding air force officers for military operations. Military training typically involves putting the learner in settings close to the field situation (simulation-based training). In order to facilitate the training we use a new model to orchestrate a generative scenario model maintaining both narrative coherence and control of pedagogical settings of the simulation.

Cognitive load (CL) when using Virtual Reality (VR) requires more experimental inputs, especially to determine how VR affects human psychophysiology depending on the task. Classifying humans’ physiological variations in a controlled setup is essential. We randomly assigned 92 participants to three experimental conditions: control, stereoscopy, and dual-task. Participants fulfilled a rest-baseline period, a Stroop task (25 congruent, 25 incongruent words), and a NASA-TLX questionnaire. We recorded behavioral and physiological data from eye-tracking(ET), electrocardiogram(ECG), and electrodermal activity (EDA). NASA-TLX scores of control and stereoscopy were statistically different with dual-task conditions. We used NASA-TLX scores to create three classes and train a CL classifier based on physiological variations. We deployed linear models penalized with the L1 norm to select the most relevant features correlated with subjective CL levels. The ECG sensor provided the most selected features compared to EDA and ET. We compared SVM, Logistic Regression, and Gradient boosting classifier models. The Gradient boosting method with 87.23% accuracy and an 87.13% F1 score is the most performant. Future works will try to compare such an approach with stressful stimuli.

Virtual Reality Head-Mounted Displays (HMDs) reached the consumer market and are used for learning purposes. Risks regarding visual fatigue and high cognitive load arise while using HMDs. These risks could impact learning efficiency. Visual fatigue and cognitive load can be measured with eye tracking, a technique that is progressively implemented in HMDs. Thus, we investigate how to assess visual fatigue and cognitive load via eye tracking. We conducted this review based on five research questions. We first described visual fatigue and possible cognitive overload while learning with HMDs. The review indicates that visual fatigue can be measured with blinks and cognitive load with pupil diameter based on thirty-seven included papers. Yet, distinguishing visual fatigue from cognitive load with such measures is challenging due to possible links between them. Despite measure interpretation issues, eye tracking is promising for live assessment. More researches are needed to make data interpretation more robust and document human factor risks when learning with HMDs.

Our work focuses on improving the management of stressful situations using virtual environments. We hypothesise that learners can improve how they deal with these situations by being confronted with a wide variety of scenarios. We want to create customised situations using different stressors, which is why we are interested in the representation and diagnosis of the learner ability to cope with said stressors. The diagnosis of a stress profile is carried out using stress sensors which provide measurements. However, these measurements are not quite spotless and uncertainties are present. Moreover, the situations we generate are complex and involve stressors that can impact each learner differently. Hence, many uncertainties also impact the diagnosis. In order to take these uncertainties into account, we rely on the transferable belief model, and we come up with stressors depicted in the form of a taxonomy that can be configured by the instructor. This will allow them to explore different levels of granularity.

Although virtual reality (VR) has many applications, only few studies have investigated user acceptance of this type of immersive technology. We propose an extended version of the Technology Acceptance Model (TAM) that addresses some aspects of VR. Our model includes variables from the TAM, user experience, variables specific to VR, and variables relating to user characteristics. This model was tested with 89 users who performed an aeronautical assembly task in VR. Results suggest that intention to use VR is positively influenced by perceived usefulness and negatively influenced by cybersickness. Hedonic quality-stimulation and personal innovativeness are predictors of perceived usefulness. Perceived ease of use does not have a significant impact on intention to use and it is only influenced by pragmatic quality. These findings have a number of implications regarding user acceptance of VR.

Health care delivery in military conflict, in peacekeeping missions or in the aftermath of disaster, implies high stress environments with danger exposures, life-threatening events and high levels of work demand. Crisis and emergency risk communication remains a real challenge. The rapid response of emergency medical teams has an important role to play in preventing serious adverse events. During critical events, medical errors can be related to human or system factors, including ineffective team leadership, non-standardized team communication, a lack of global situation awareness, poor use of resources and inappropriate triage and prioritization. The VICTEAMS project aims at building a virtual environment for training rescue team leaders to non-technical skills. Depending on the leader abilities, the virtual characters playing the subordinates (e.g. nurses) reproduce a variety of behaviors like erroneous actions, followership attitudes or stress-based behaviors. A pedagogical director tailors the difficulty according to the dynamic profile of the learner.

This paper discusses the design of training environments using a virtual reality space populated by virtual characters participating in collaborative work. We are particularly interested in training a single human person who needs to organize and run a team doing collaborative work. Many problems have to be solved, due to the requirements, e.g.: virtual characters must display a human behavior (possibly doing mistakes), be pro-active according to the situation and their physical, psychological and social profile; all exchanges of information must be done using natural language.

Under the project MacCoy Critical, we would like to train individuals, in virtual environments, to handle critical situations such as dilemmas. These latter refer to situations where there is no "good" solution. In other words, situations that lead to negative consequences whichever choice is made. Our objective is to develop a Scenario Orchestration System that generates dilemma situations dynamically without having to write them beforehand. The approach consists in using Knowledge Models to extract necessary properties for dilemmas to emerge. In this article we present this approach and expose a proof of concept of the generation process.

In this work, we propose a novel approach to solve probabilistic planning problems taking into account the risk that the decision maker is ready to accept regarding the probabilities of reaching the goals. Our approach guarantees that the probability of reaching a state satisfying the selected goals is above a certain limit threshold. To achieve this, we relax the constraints that all goals must be satisfied, and select the most valuable set of goals whose reachability probability is above the threshold. To this end, we propose a goal selection heuristic based on the reachability probability and the cost between goals that are estimated using an abstracted version of the problem. Finally, a planetary exploration problem will be used for illustrating the effectiveness of the proposed approach. Our results show that the obtained selections cover the most valuable possible goals and respect the reachability probability threshold.

This paper describes a new heuristic approach letting an agent in a multi-agent system make a decision for selecting tasks to undertake. The proposed approach consists in giving the agent a profile including preferences and physical parameters, and a description of its beliefs about the environment. The decision engine allowing the agent to do a specific task is local to the agent and takes into account the agent goals, physical and emotional state and beliefs by combining heuristics priorities for each possible task. The approach is flexible, allowing to model situations where agents represent humans. It can be extended to multiple agents in collaborative environments.

Virtual agents can play the trainee's teammates in Collaborative Virtual Environments for Training (CVET). In such teams, trust is essential since people rely on each others. We propose a decision making system for agents in CVET based on an agent model inspired from works on trust in organizational psychology. Agents dimensions are integrity, benevolence and abilities, and agents have trust-beliefs about their teammates integrity, benevolence and abilities. Based on their personal dimensions and trust beliefs, agents compute the importance of their goals and then the utility of actions to select the most appropriate action.

International audience

Virtual reality is a very interesting technology for professional training. We can mention in particular the ability to simulate the activity without real danger, the flexibility in the informations' presentation, or the exact control parameters of the simulation allows to reproduc specific situations. Today, technological maturity allows to plan increasingly a complex applications. However, in one hand, this complexity increases the difficulty, at the same time, to propose a pedagogical and narrative control (to ensure a given learning and narrative structure) and some freedom of actions (to promote the emergence of various, unique and suprised situations in order to ensure a learning-by-doing/errors). In other hand, this complexity makes difficult the tracking and understanding of learner's path. In this paper, we propose 1- a scripting model for training virtual environment combining both a pedagogical control and the emergence of pertinent learning situations and 2- tracking of the learner's actions, but also analysis and automatic diagnosis tools of the learner's performances.

Interacting with other persons implies having appropriate reactions to their actions. In the professional field, inappropriate responses can be very prejudicial. Many studies point out the importance of emotions in the construct of behavioral responses. This article proposes a model of emotional, social and cognitive Personal Assistant agent that can help its user to avoid conflicts.

This paper presents our virtual environment for training at high-risk sites (SEVESO sites). In order to enhance this environment, we have integrated into it an intelligent tutoring system called HERA. This system uses the print trace to which the learning process gives rise, and produces an activity trace using plan-recognition techniques. In this paper we present the main components (modules, models) of HERA, and its features for tracking the learner. We explain how HERA retrieves data and transmits them to the learner and the trainer. The data concern the task realized, the errors committed and their type, the potential risks, and comments and explanations about the errors and risky situations.