Digital Twins are increasingly used in cultural heritage as integrative environments combining geometry, sensing and analysis. Their evolution into Reactive Heritage Digital Twins reveals a key gap, namely the lack of an interaction layer through which Cultural Heritage professionals can contribute spatially grounded and reusable knowledge. Existing Extended Reality tools remain limited in annotation expressiveness, often rely on asymmetrical collaboration and do not align with the working practices of heritage professionals. This paper presents our current work-in-progress Extended Reality annotator that attempts to address the gap by treating annotation, collaboration and interaction as a unified design challenge. Operating in Augmented Reality passthrough, it preserves co-presence while embedding a high-fidelity 3D replica in the shared workspace. The system introduces a structured annotation vocabulary organised into three functional families, combined with symmetric multi-user collaboration and a multimodal interaction model grounded in cognitive ergonomics. Early results are presented through one of the pilot cases of the Horizon Europe-funded ARTEMIS project focused on the Grand Maison building in the historic centre of Xanthi, Greece. The paper argues that the adoption of Extended Reality authoring in professional heritage practice depends on aligning interaction design with the conditions under which expert knowledge is produced. VAX positions the interaction layer not as an accessory to the digital twin, but as the medium through which it becomes usable, inhabitable and ultimately meaningful for interdisciplinary work.

Audio Augmented Reality (AAR) is a novel and unexplored area of AR representing the augmentation of reality with auditory as well as visual content. AAR’s interaction affordances as well as accurate real-world registration of visual as well as sound elements are challenging issues, especially in noisy, bright and busy environments outdoors. This paper presents a novel, mobile AAR experience that is deployed in a city environment while walking past six archaeological excavation sites in the city of Chania, Crete, Greece. The proposed AAR experience utilizes cutting-edge gamification techniques, non-linear storytelling, precise AR visualization and spatial audio, offering innovative AAR interaction while exploring the city’s archaeological sites and history, outdoors.

As digitization has transformed media and Augmented Reality (AR) is evolving from a research area to a commodity, museums are creating interactive AR experiences to digitally enhance their collection and increase audience engagement. Head-worn AR experiences, though, face interaction challenges as they are often employed in busy spaces and are in need of intuitive multimodal interfaces for users on the move. This paper presents an innovative, work-in-progress, multimodal AR experience integrating non-obtrusive dialogue, music, and sound as well as gesture and gaze-based interaction, while a user is wearing a head-worn AR display. Users are motivated to explore and interact with digital cultural artefacts superimposed onto the real-world museum setting and physical artefacts, while moving around in a museum setting. We initially analyze interactive AR experiences to identify specific user requirements related to head-worn AR experiences. We deploy these requirements for the design of interactive, multimodal AR in a museum setting.