Educational practices are increasingly experimenting with eXtended Reality (XR) paradigms to offer novel opportunities for boundaryless learning experiences with real-time interactions in immersive environments. Digital Twins (DT) are also gaining traction in this field to facilitate personalized learning experiences. However, a still unexplored space in learning frameworks amounts to the one where XR intersects with DTs. This work wants to move a step in such a direction with the design, implementation, and test of a DT-driven XR framework to learn procedural tasks. The framework offers three distinct learning modalities where virtual and physical interactions enhance learning retention by engaging users actively in digital and real-world environments. We contextualize such a framework for procedural task learning through one of its pivotal use cases: learning Rubik’s Cube notations. To evaluate and compare the effectiveness of these modalities, we perform an experimental user campaign evaluating short-term skill retention, performance accuracy, usability, and cognitive load of each of them. We then provide an extensive statistical analysis to compare each kind of guidance while analyzing correlations between the examined variables, offering insights into optimizing instructional methodologies within XR-based educational frameworks.

Abstract An invisible layer of knowledge is progressively growing with the emergence of situated visualizations and reality-based information retrieval systems. In essence, digital content will overlap with real-world entities, eventually providing insights into the surrounding environment and useful information for the user. The implementation of such a vision may appear close, but many subtle details separate us from its fulfillment. This kind of implementation, as the overlap between rendered virtual annotations and the camera’s real-world view, requires different computer vision paradigms for object recognition and tracking which often require high computing power and large-scale datasets of images. Nevertheless, these resources are not always available, and in some specific domains, the lack of an appropriate reference dataset could be disruptive for a considered task. In this particular scenario, we here consider the problem of wine recognition to support an augmented reading of their labels. In fact, images of wine bottle labels may not be available as wineries periodically change their designs, product information regulations may vary, and specific bottles may be rare, making the label recognition process hard or even impossible. In this work, we present augmented wine recognition, an augmented reality system that exploits optical character recognition paradigms to interpret and exploit the text within a wine label, without requiring any reference image. Our experiments show that such a framework can overcome the limitations posed by image retrieval-based systems while exhibiting a comparable performance.