Training professionals in high-stakes, trauma-informed communication is critical across domains such as law enforcement, healthcare, and counseling. While live role-play with trained actors remains the gold standard, it is resource-intensive and emotionally demanding. Generative AI offers scalable alternatives, but what is gained or lost when training shifts to AI? We developed an AI-powered sexual assault victim interview training system and conducted a mixed-methods study with 35 police recruits, each completing both an AI-based and a live, actor-based training session. By varying the sequence (AI-first vs. human-first), we examined differences in self-efficacy, perceptions of the AI system, and perceived learning experience. Although both modalities supported learning, the order in which they were experienced significantly shaped learners’ emotional engagement, sense of preparedness, and interpretation of each simulation’s role. Building on these insights, we introduce a conceptual design framework that identifies social–emotional, temporal, and embodied distance as key pedagogical dimensions, and we offer implications for sequencing hybrid simulations to scaffold preparation, performance, and reflection. Our findings position AI not as a replacement for human realism, but as a complementary modality that expands opportunities for safe, scalable practice in sensitive communication training.

As AI support tools become more common in immersive medical training, designers face a critical interaction-design question: When should an AI facilitator take initiative, and when should it wait for the learner? To investigate this design tension, we compared two versions of an AI facilitator in a mixed-reality (XR) lumbar puncture simulator training conditions: one in which the AI proactively initiated guidance and encouragement, and another in which the AI responded only when prompted. Using a mixed-methods approach, we examined how medical students (n=22) engaged with, interpreted, and reacted to these two facilitation styles. We found no significant differences in learning outcomes, interaction frequency, or overall experience ratings. However, interviews and behavioral analyses revealed nuanced differences in how learners perceived AI interventions across distinct task phases. AI-initiated support was seen as helpful in some moments and disruptive in others, depending on task phase, cognitive load, and personal preferences. Based on these findings, we contribute a boundary framework which offers actionable design guidance for calibrating AI proactivity in immersive training systems, and extends HCI research on proactive agents and human–AI collaboration within high-cognitive-load environments.

Public speaking is a near universally anxiety inducing social situation. Applications recreating social interactions with autonomous agents in virtual reality have been proposed as tools to alleviate public speaking anxiety, recreating exposure therapy methods with virtual audiences. To be efficient, such applications rely on the precise induction of controlled amounts of social stress. We reviewed the literature that studied the effect of virtual audience behaviors on participants’ stress levels, and found contradictory results that we attempt to explain in this article. We examine those studies and propose that social presence is one of the important factors mediating the effect of audience behavior on stress levels, and through social presence we can explain the different effects of audience behavior on stress observed in past studies. We conducted a study to test this theory, and expected that high social presence would lead to larger difference in stress induced by positive and negative audiences, as opposed to lower social presence which would attenuate the effect of audience behavior. We compared two display mediums, VR headsets to induce high social presence and wall projection to induce low social presence. We find an interaction effect of social presence and audience behavior on subjectively reported stress levels, but not on physiological measures.

Digital twins, along with Internet of Things and Artificial Intelligence, have been identified as one of the key technologies for Industry 4.0. However, the definition of Digital Twin (DT) is still abstract and context-dependent. In this paper, we present a metamodel that supports concrete and operational descriptions of digital twin deployment. This metamodel encompasses the different aspects of deployment, including the definition of hardware and software components that compose the layered cyber-physical architectures of the digital twin, along with the installation and instantiation tasks that compose deployment processes. Multiple configurations can also be defined to support the deployment of a digital twin in different execution contexts. The relevance of this metamodel was evaluated by two case studies. The first consists in deploying the digital twin of a cobot in a simulation environment. The second applies the approach in a home automation environment. In both cases, our metamodel provides complete and precise descriptions of the deployment process and thus constitutes a viable first step towards a model-driven approach for digital twin deployment.

According to medical experts, haptic realism is difficult to achieve, and even more difficult to have inter-expert agreement on the haptic feedback of one simulation. However haptic feedback is important in medical training, and allows educators to share the forces felt during a procedure if they know and trust what a particular virtual simulator will provide to the trainee. A new approach is proposed to refine bio-mechanical models with experts' input, to closely match the forces felt during a simulated procedure with an expert trainer's expectations. By allowing experts to tune a training scenario's haptic feedback as they trial a newly developed case, the experts can replicate their haptic perception and match their expectations with the simulation.

Haptic sensation is difficult to describe verbally. An experiment was performed to evaluate the viability of using a contra lateral haptic replication task to elicit haptic information from subjects, bypassing verbal description. Subjects received force input of four magnitudes in four directions from one haptic device to the hand, and attempted to reproduce it in another haptic device that was held in the contra lateral hand. Results showed that although subjects tended to overshoot when reproducing the forces, it is a viable method for decoding and encoding haptic information without verbal description.

Leonardo is a prototype interactive augmented reality system designed to aid a surgeon in port-placement prior to performing robot-assisted laparoscopic surgery. The system components consist of a computer, a projector, and a motion and color tracking device. The human-computer interface through which the surgeon interacts with the system is the patient's abdomen itself. Based on the parameters of the patient body and surgical procedure, an optimized set of port locations is provided to the surgeon as colored icons projected onto the abdomen. The surgeon can adjust the suggested locations by direct manipulation with the gloved hand and fingers in real-time. This interactive decision-aid is expected to shorten the planning phase, and enhance performance of robot-assisted laparoscopic surgery.

This paper examined the relative effectiveness of three communication modalities in a collaborative virtual environment. Communication modalities of verbal only (V), haptic only (H), and both (HV), were used in a 2D pointing task. A total of 36 participants were paired into 18 dyads. In each dyad, there was a supervisor and an acting agent. The supervisor used one of the three modalities to guide the acting agent to reach the target location. The time to task completion and the trajectory were recorded and analyzed. The results indicate that subjects using verbal only and haptic+verbal communication performed equally well in the collaborative pointing task. Subjects using haptic only communication spent more time and had longer path lengths than verbal only and haptic+verbal communication. Nevertheless, all three modalities were effective in communicating in a virtual collaborative task.

The use of haptic devices to provide artificial force feedback in teleoperation has been shown to enhance performance. An experiment was conducted to examine how artificial (simulated) force feedback is utilized, compared with real force feedback, in a laparoscopic tissue-probing task Actual (real) forces in probing a double-layer silicon gel mass were replicated and exaggerated in a virtual environment using a haptic device. Ten subjects performed the probing task in four different artificial force feedback conditions: (1) high fidelity force feedback, (2) proportionately exaggerated feedback, (3) disproportionately exaggerated relative force feedback, and (4) reversed disproportionately exaggerated relative force feedback. Results showed that a higher maximum force was applied, detection time was longer, and distance error was greater in virtual probing compared to real probing. Detection time was significantly greater in the virtual high fidelity condition compared to the disproportionately exaggerated force feedback conditions. These results suggest that artificial force information may be processed differently than real haptic information, leading to higher force application that could potentially damage tissues, lower efficiency, and reduced accuracy in tissue probing tasks.

Indirect contact with the surgical environment using long laparoscopic instruments reduces haptic feedback in minimally invasive surgery. Furthermore, the friction force generated from the ports through which instruments are inserted into the body cavity masks the haptic information needed to perceive properties and structure of the tissue, complicating the control of the interaction forces between the instruments and tissue. It has been shown that friction forces inherent in the instrumentation increased the haptic perception threshold of naive subjects. A controlled experiment was conducted to examine the effects of surgical experience on force perception threshold in a simulated tissue-probing task. A mixed design was used, with friction, vision, and tissue softness as independent, within-subjects factors, and experience as a between-subjects factor. Applied force and detection time from fourteen participants (8 novices and 6 experienced surgeons) were recorded by a custom-built force-sensing system. Higher thresholds and longer detection times were observed when friction was present. Experienced surgeons applied greater force than novices, but were quicker to detect contact with tissue, suggesting that experience allowed surgeons to perform more efficiently while keeping within the limits of safety.

Laparoscopic surgery is performed using long instruments that enter the abdomen through small incisions while viewing the workspace on a video monitor. Because of the viewing limitations that are inherent in the imaging system, depth perception is severely limited compared to direct viewing in open surgery. Previous studies have demonstrated that the addition of shadows can improve performance in tasks under laparoscopic conditions. This study examined the effect of color-contrasting shadows on performance in a depth-perception-dependent laparoscopic task. It was hypothesized that the added contrast of colored shadows would make them more pronounced on the dark-red background found in the body, thus improving performance. Twenty-four novice participants performed a dynamic pick-and-place task under three different lighting conditions on two differently colored backgrounds. Results showed that the presence of both colored and black shadows improved performance by 10% compared to no shadows, but the colored shadows did not provide a significant advantage over the black shadows.

Lack of adequate force feedback for the surgeon in minimally invasive surgery (MIS) can lead to unnecessary trauma to tissue and adverse events during surgery. The successful use of vibrotactile stimulation to augment overloaded or deficient sensory modes in the human operator in other application areas warrants an investigation into its application in MIS. A vibrotactile force feedback system was designed, and its ability to provide useful force information to subjects performing a simulated MIS task was evaluated. Results showed that the system responds as predicted against the bottom surface of the foot, and that subjects were able to perceive a linear increase in force as linear increase in vibration intensity. Furthermore, vibrotacile force information increased one's sensitivity to tissue contact (1.3 N maximum force -no vibration, 1.0 N maximum force-fine step vibration feedback; p<0.001) and improved one's ability to consistently and accurately differentiate tissue softness in a simulated MIS task. Vibrotactile force feedback in MIS appears to have benefits which can lead to a decrease in trauma to tissue and adverse events.