Immersive technologies offer advantages for the visualization of and interaction with complex setups within manufacturing maintenance processes. The present work catalogs different applications of AR/VR in manufacturing maintenance practices as an extended version of a workshop paper presented at the International Workshop on eXtended Reality for Industrial and Occupational Supports (XRIOS). Through a scoping review in three computing and engineering digital libraries, we outline the key attributes of immersive solutions ( N p = 115 ) for industrial maintenance, categorizing functional prototypes with ten parameters related to interaction, visualization, and research methods. Moreover, we conducted a workshop with three manufacturing experts discussing the future of maintenance interfaces. By bringing forth their recommendations and insights, we targeted a key training challenge in maintenance. We designed and implemented a situated visualization prototype for a safety-critical procedure with real-time, in-depth, spatially relevant instructions. We compared the effects of 2D labels and 3D ghosts in a VR-simulated AR environment. In a preliminary between-subjects evaluation study ( N u = 24 ), we measured usability, workload, simulator sickness, completion time, and delayed recall. Although we did not find statistically significant differences between conditions, 3D ghosts showed slightly lower perceived workload and discomfort levels, along with shorter completion times. On the other hand, 2D labels produced higher usability. Overall, we contribute by a mapping out the state of the art and discovering knowledge gaps within immersive maintenance, presenting a design and preliminary user study that adheres to our recommendations.

Cross Reality (CR) is a new emerging field based on the current developments in Mixed Reality hardware, especially supported by the broad market penetration of video-based see-through Head-Mounted Displays. It refers to applications that span across different stages (real, Augmented Reality, Augmented Virtuality, Virtual Reality) of the reality-virtuality continuum, where users are interconnected between different stages and/or are able to transition between these stages. This publication follows the concept of other grand challenges publications and reflects the discussion of various researchers invested in CR. After an initial discussion at the 1stJoint Workshop on Cross Reality at IEEE ISMAR 2023, six topic groups have been identified, leading to 22 challenges, which were discussed in groups over the period of multiple months. The discussion of these challenges should act as a road map for future research in the area of CR.

In manufacturing environments, operators are expected to perform structured troubleshooting to ensure process stability and minimize downtime. However, the adoption of formal problem-solving protocols remains low in practice. Through a formative study in a pharmaceutical manufacturing setting, we identify key challenges in current troubleshooting practices: high cognitive load, fragmented documentation, and loss of tacit operational knowledge. We propose the exploration of a conversational AI (CAI) assistant to reframe these structured protocols as natural dialogues that support reasoning and learning rather than administrative compliance.

Understanding the interaction between cognitive load and features of video game play is important for the accurate measurement and application of this psychological construct in both research and industry scenarios to enhance the player experience. A challenge within this domain is the use of measurements developed in different areas in video games without first validating or testing the reliability of these tools. We present a holistic evaluation of methods used to measure cognitive load during naturalistic gameplay of a commercially available sandbox game. This study measures electroencephalography, electromyography, heart rate, heart rate variability, electrodermal activity, and eye blink rate from players during a building task in Minecraft whilst being assisted by a virtual companion. Various models reveal that a number of physiological measures can be used as a proxy of subjective cognitive load measurement, providing further insight into the player experience during gameplay. Our analysis also reveals some discriminant validity of the subjective measurement used. These results can help inform the choice of sensor in evaluating video game features, or when needing to accurately measure cognitive load for an adaptive situation or high-risk scenario.

This research proposes an evaluation of pitch-based sonification methods via user experiments in real-life scenarios, specifically vertical guidance, with the aim of standardizing the use of audio interfaces in AR in guidance tasks. Using literature on assistive technology for people who are blind or visually impaired, we aim to generalize their applicability to a broader population and for different use cases. We propose and test sonification methods for vertical guidance in a series of hand-navigation assessments with users without visual feedback. Including feedback from a visually impaired expert in digital accessibility, results (N=19) outlined that methods that do not rely on memorizing pitch had the most promising accuracy and self-reported workload performances. Ultimately, we argue for audio AR's ability to enhance user performance in different scenarios, from video games to finding objects in a pantry.

To promote empathy with people that have disabilities, we propose a multi-sensory interactive experience that allows sighted users to embody having a visual impairment whilst using assistive technologies. The experiment involves blindfolded sighted participants interacting with a variety of sonification methods in order to locate targets and place objects in a real kitchen environment. Prior to the tests, we enquired about the perceived benefits of increasing said empathy from the blind and visually impaired (BVI) community. To test empathy, we adapted an Empathy and Sympathy Response scale to gather sighted people's self-reported and perceived empathy with the BVI community from both sighted (N = 77) and BVI people (N = 20) respectively. We re-tested sighted people's empathy after the experiment and found that their empathetic and sympathetic responses (N = 15) significantly increased. Furthermore, survey results suggest that the BVI community believes the use of these empathy-evoking embodied experiences may lead to the development of new assistive technologies.

Whenever accessing indoor spaces such as classrooms or auditoriums, people might attempt to analyze and choose an appropriate place to stay while attending an event. Several criteria may be accounted for, and most are not always self-evident or trivial. This work proposes the use of data visualization allied to an Augmented Reality (AR) user interface to help users defining the most convenient seats to take. We consider sets of arbitrary demands and project information directly atop the seats and all around the room. Users can also narrow down the search by switching and combining the attributes being displayed, e.g., temperature, wheelchair accessibility. The proposed approach was tested against a comparable 2D interactive visualization of the same data in usability assessments of seat-choosing tasks with a set of users (N = 16) to validate the solution. Qualitative and quantitative data indicated that the AR-based solution is promising, suggesting that AR may help users make more accurate decisions, even in an ordinary daily task. Regarding Augmented Situated Visualization, our results open new avenues for the exploration of context-aware data.