In musical collaboration, digital musical instruments often hinder effective communication and engagement by restricting visibility and limiting gestural and non-verbal interactions.These challenges reduce musicians' situational awareness and complicate cohesive performance.To address this, we developed a head-mounted augmented reality (AR) system to enhance collaborative musical experiences by visualising musicians' hand movements, eye gaze positions, and instrument interactions in real-time.We conducted a user study involving four pairs of musicians performing live music using different AR interface configurations.The results suggest that the AR system can enhance situational awareness and assist collaboration, as reflected in questionnaire responses.Interviews indicated that real-time visualisations of bodily movements and interactions helped participants better understand the collaborative process and anticipate their collaborators' actions.These findings point to the potential of AR-assisted visualisation to support creative collaboration by tailoring visual information to different needs.Future research could explore its application in broader contexts of real-time creative cooperation.

To enable robots to comprehend high-level human instructions and perform complex tasks, a key challenge lies in achieving comprehensive scene understanding: interpreting and interacting with the 3D environment in a meaningful way. This requires a smart map that fuses accurate geometric structure with rich, human-understandable semantics. To address this, we introduce the 3D Queryable Scene Representation (3D QSR), a novel framework built on multimedia data that unifies three complementary 3D representations: (1) 3D-consistent novel view rendering and segmentation from panoptic reconstruction, (2) precise geometry from 3D point clouds, and (3) structured, scalable organization via 3D scene graphs. Built on an object-centric design, the framework integrates with large vision-language models to enable semantic queryability by linking multimodal object embeddings, and supporting object-level retrieval of geometric, visual, and semantic information. The retrieved data are then loaded into a robotic task planner for downstream execution.

Although exercising at home has benefits, it is not always engaging or motivating. Augmented Reality (AR) head-mounted displays (HMDs) offer the potential to make in-home exercising and exergaming more inclusive and immersive, but there is limited research investigating how such systems can be designed. We employed a participatory design approach involving semi-structured interviews to investigate how homes can be augmented to facilitate exercising experiences. We developed 10 recommendations for developing home-based exercising experiences using AR HMDs. Our results further contribute to the existing body of research on the use of AR for exercising, home applications, and everyday objects by presenting the first foundational study investigating the wide range of exercises that can be supported through AR HMDs in home environments and the different ways home elements may support these exercises, and laying the groundwork for future work developing home-based exergaming through AR HMDs to increase people’s physical activity levels.

Indoor asset tracking is an essential task in many areas of industry such as shipping and warehousing. Widely-used asset tracking technologies typically require supporting infrastructure (signal transponders) to communicate with active tags on the assets. Continuous asset tracking in large indoor spaces like warehouses can be costly, and reliable reference points are needed to calculate asset positions accurately. In this paper, we describe an indoor asset tracking technique for augmented reality (AR) that combines the use of (passive) fiducial markers together with flexible spatial anchors. Our approach, called SABIAT, continuously tracks the approximate location of an asset using spatial anchors and, when needed, the precise location of that asset using fiducial markers. We have applied our SABIAT technique to build a demonstrator system, AR-IPS, to show how assets can be tracked and located inside of a large, multi-level building.

Realistic non-player characters (NPC) are an important component of virtual environments. However, generating NPCs with humanlike behaviour can be difficult. A previously proposed pipeline used evacuation simulators to generate domain relevant NPC behaviour for virtual environments. However, the NPCs generated through this process had static behaviours and could not dynamically change pre-computed evacuation paths. In this paper, we extend this pipeline with an approach that supports path switching for NPCs in four situations and demonstrates the pipeline in a virtual environment modelled on a large real building. A scalability test on the large building showed overall evacuation time in both source evacuation simulator and target virtual environment was consistent regardless of building size and complexity. Three test-cases demonstrate path switching for NPCs with increasing evacuation success rates through different situations. The reuse of static paths to enable dynamic NPC behaviour supports ongoing work to develop low cost and realistic training systems using game engine technology.