Effective communication and teamwork are vital in high-stakes environments such as the operating room, where timely and accurate information exchange directly affects patient safety and surgical outcomes. Among intraoperative interactions, the collaboration between the surgeon and anesthesiologist is especially critical for maintaining smooth workflows and preventing adverse events. Despite its importance, little HCI research has explicitly examined the unique needs of this dyad or how AI-driven supportive systems might be designed to address them. In this work, we present a qualitative study of surgeon–anesthesiologist collaboration, drawing on focus groups with both specialties and in-situ observations of 45 surgeries spanning open, laparoscopic, and robotic procedures. Our findings uncover key challenges, unmet needs, and coordination breakdowns that shape this relationship. Based on these insights, we conceptualize a systems design to better support intraoperative collaboration.

Head-mounted displays (HMDs) are increasingly used in safety-critical fields such as surgery, aviation, and industrial manufacturing. As major manufacturers shift toward video-see through (VST) designs to deliver unified AR and VR experiences, they also replace direct visual access to the real world with a video feed. This design choice raises concerns about its impact on user performance. This study investigates the isolated impacts of VST and optical-see through HMDs on user real-world perceptual–motor performance by comparing two leading HMDs, Apple Vision Pro (AVP) and HoloLens 2 (MHL2) against unencumbered vision using the Purdue Pegboard Test (PPT), a standard assessment of manual dexterity. Twenty participants completed tasks across three conditions (AVP, MHL2, and Baseline), while we recorded dexterity scores, cognitive load, system usability, VR sickness, and subjective feedback. Movement data were also collected via Apple Watches. Study results with 20 participants revealed that dexterity scores significantly declined under the AVP condition across all subtests. This was accompanied by significantly higher cognitive load and a notable drop in RMS acceleration values (observed in the RMS analysis of a subset of 13 participants). The analysis on dexterity score yielded a significant difference between MHL2 and Baseline only for a single subtest of the PPT (Left Hand). Post-task interviews revealed greater discomfort, visual fatigue, and reduced task confidence with AVP. These findings suggest that current VST HMDs impose a hidden ergonomic cost undermining user performance in tasks where precision, and comfort are essential. For AR applications designed to enhance user performance, such as assistive tools, training systems, or task guidance interfaces, designers must account for and mitigate this performance degradation through counterbalancing strategies to offset the visual and cognitive burden introduced by VST HMDs.

The integration of Augmented Reality (AR) technology into surgical procedures offers significant potential to enhance clinical outcomes. While there are plenty of lab-proven prototypes, systems employed in actual clinical settings require specialized design and rigorous clinical evaluation of these AR-based solutions to meet the high demands of complex medical fields. Our research exposes these complex requirements emerging from clinical environments, such as operation theaters. To address the challenges, we introduce ARAS, an operational AR assistance system for live open pancreatic surgery. Employing a user-centric design methodology, we designed and refined ARAS through several iterations, ensuring its practical applicability and effectiveness in a real-world surgical setting during clinical trials. ARAS enables in situ and precise visualization of the patient’s 3D reconstructed vascular system and tumor during the surgical procedure. We evaluated ARAS through clinical trials (N=7) involving patients diagnosed with pancreatic cancer. Our interviews with the surgeons underscored the utility of ARAS for open pancreatic surgery, especially in critical and highly time-pressured phases of the surgery, as it proved to be exceptionally beneficial in aiding surgeons during in their decision-making process. In a post-surgery evaluation, the surgeons also certified the precise visualization accuracy of ARAS during those critical phases. Our findings showcased the heightened requirements for AR-based solutions in operational clinical use and proved that ARAS met the challenges emerging during live surgeries. Consequently, surgical AR assistance systems do have a transformative potential to revolutionize traditional practices, but their applicability is subject to high design constraints during critical medical procedures.

We investigate if providing shoppers with Augmented reality (AR) as a shopping tool can lead to an increase in purchase rate compared with conventional shopping applications. In a ”simulated shopping” study with two groups with a total number of 20 participants, a test group used a wearable AR device (HoloLens) as a primary shopping method while a control group used a tablet as a conventional 2D shopping device. According to the surveys collected from participants, 19 participants (95%) commented that the AR method was more joyful than conventional 2D method. Furthermore, all participants said that their experience with AR technology was more realistic. 16 participants (80%) believed that the AR method was more influential than the conventional method, but the other four participants (20%) said that neither of the methods had any impact on their buying intentions. Although the mean number of products added to the basket by each person in HoloLens experience and tablet experience was not significantly different (p=0.675), the mean number of bought items was significantly higher in HoloLens experience compared to the tablet experience (p=0.004). In conclusion, AR increased customer’s interest in shopping. While it had no significant impact on adding products to the shopping cart, it affected the rate of purchase.

We present a study comparing the effect of real-time wearable feedback with traditional training methods for cardiopulmonary resuscitation (CPR). The aim is to ensure that the students can deliver CPR with the right compression speed and depth. On the wearable side, we test two systems: one based on a combination of visual feedback and tactile information on a smart-watch and one based on visual feedback and audio information on a Google Glass. In a trial with 50 subjects (23 trainee nurses and 27 novices,) we compare those modalities to standard human teaching that is used in nurse training. While a single traditional teaching session tends to improve only the percentage of correct depth, it has less effect on the percentage of effective CPR (depth and speed correct at the same time). By contrast, in a training session with the wearable feedback device, the average percentage of time when CPR is effective improves by up to almost 25%.