Conversational Virtual Agents (CVAs) offer a promising approach for enhancing user task performance in Mixed Reality (MR) environments. This paper explores the integration of a CVA into an MR application designed to assist in solving a 2D physical puzzle, offering enhanced spatial cognitive capabilities. Using the CVA classification architecture and the MiRAS (Mixed Reality Agents) Cube Taxonomy, we developed an MR system with a state-aware assistant to guide users in a puzzle-solving task only when requested. The primary research question is whether or not the CVA needs to be embodied. We conducted a study with 34 participants to investigate the influence of Voice-only and Embodied CVAs on puzzle-solving performance, user interactions with the assistant, the assistant’s social presence, overall system cognitive workload, and users’ perceptions of future system use. Both modalities showed equivalent outcomes regarding the number of position- and orientation-related queries, perceived usability, message and affective understanding, performance, frustration, and usefulness. However, results showed that Voice-only CVA significantly enhanced task efficiency: participants completed puzzles more quickly and accurately, reporting lower effort than in the Embodied condition. These findings suggest that Voice-only CVAs may be more effective for tasks like puzzle solving, where auditory guidance alone appears sufficient to support better performance.

External Ventricular Drain (EVD) placement is a complex neurosurgical task that requires identifying a target point within the brain and accurately positioning a catheter at the appropriate angle. While Mixed Reality (MR) technologies have seen limited adoption in the operating room, they offer significant potential for developing training systems that enhance skill acquisition and retention in unaided conditions. A current gap in research concerns the effectiveness of multimodal guidance systems that incorporate both visual and audio-based MR cues. In this paper, we present an MR-based simulator for EVD placement training and evaluate the impact of three MR-guided training modalities: (1) a baseline condition using only 2D CT scans and a 2D catheter projection; (2) a visual guidance modality incorporating a 3D trajectory overlay; and (3) an embodied-audio guidance modality featuring a virtual agent delivering spoken instructions and feedback. Participants underwent a digital training phase using one of the three modalities, followed by an unaided EVD placement on a physical phantom with a real catheter to evaluate skill transfer and retention. Results indicate that both advanced MR modalities significantly improve procedural accuracy, execution speed and receive higher scores in usability and technology acceptance compared to the baseline. Notably, training with 3D visual trajectory guidance led to significantly higher unaided placement accuracy, indicating stronger skill retention. However, multimodal guidance demonstrated equivalent execution speed, while showing a trend toward lower overall cognitive load.

Educational practices are increasingly experimenting with eXtended Reality (XR) paradigms to offer novel opportunities for boundaryless learning experiences with real-time interactions in immersive environments. Digital Twins (DT) are also gaining traction in this field to facilitate personalized learning experiences. However, a still unexplored space in learning frameworks amounts to the one where XR intersects with DTs. This work wants to move a step in such a direction with the design, implementation, and test of a DT-driven XR framework to learn procedural tasks. The framework offers three distinct learning modalities where virtual and physical interactions enhance learning retention by engaging users actively in digital and real-world environments. We contextualize such a framework for procedural task learning through one of its pivotal use cases: learning Rubik’s Cube notations. To evaluate and compare the effectiveness of these modalities, we perform an experimental user campaign evaluating short-term skill retention, performance accuracy, usability, and cognitive load of each of them. We then provide an extensive statistical analysis to compare each kind of guidance while analyzing correlations between the examined variables, offering insights into optimizing instructional methodologies within XR-based educational frameworks.