Extended Reality (XR) technologies, including Mixed Reality (MR), Augmented Reality (AR), and Virtual Reality (VR), are blurring the lines between physical and digital environments, transcending the limitations of traditional two-dimensional (2D) interfaces. This shift toward embodied, often context-aware spatial interaction offers broad potential benefits, yet also introduces unique challenges, especially for certain user groups. For people who are Deaf or hard of hearing (DHH), XR’s immersive and multi-sensory environments provide unique opportunities to improve accessibility. However, design principles that work well in 2D interfaces may not always translate seamlessly into immersive contexts, creating new accessibility barriers. The launch of the Apple Vision Pro marks a significant moment in the mainstream adoption of spatial computing, yet little is known about its accessibility implications for deaf users. To explore this emerging area, we conducted an in-the-wild, open-ended study with five deaf participants who have diverse communication preferences, evaluating the Apple Vision Pro in everyday situations. Based on this exploratory evaluation, we identify key accessibility challenges and opportunities and provide practical recommendations to make spatial computing more inclusive for deaf users in the future.

Sign language interpreters play a crucial role in facilitating communication between Deaf individuals who primarily use sign language and people who use spoken language and lack proficiency in sign language. One method for providing this service is live video streaming, such as video remote interpreting (VRI) or video relay service (VRS) interpreting. In this case, during group sessions that involve multiple participants and various visual aids, interpreters often have limited access to the Deaf clients’ environment due to the constraints of their cameras’ field of view (FOV). This study examines whether enhancing interpreters’ access to a Deaf client’s environment using cameras with a wider FOV can improve the overall effectiveness of interpreting group sessions. To explore this, we developed an application using conference room webcams and conducted usability testing sessions with twenty experienced VRI/VRS interpreters, who evaluated different screen layouts through simulated group meetings. This paper presents the findings on nine different layout configurations, including screen layout, FOV, and the ability to observe the presenter’s movements during group sessions.

Captions make the audio content of videos accessible and understandable for deaf or hard-of-hearing people (DHH). However, in real-time captioning scenarios, captions alone can be challenging for DHH users to identify the active speaker in a real time in multiple-speaker scenarios. To enhance the accessibility of real-time captioning, we propose Haptic-Captioning which provides real-time vibration feedback on the wrist by directly translating the sound of content into vibrations. We conducted three experiments to examine: (1) the haptic perception (Preliminary Study), (2) the feasibility of the haptic modality along with real-time and non-real-time visual captioning methods (Study 1), and (3) the user experience of using the Haptic-Captioning system in different media contexts (Study 2). Our results highlight that the Haptic-Captioning complements visual captions by improving caption readability, maintaining media engagement, enhancing understanding of emotions, and assisting speaker indication in real-time captioning scenarios. Furthermore, we discuss design implications for the future development of Haptic-Captioning.

Sounds provide vital information such as spatial and interaction cues in virtual reality (VR) applications to convey more immersive experiences to VR users. However, it may be a challenge for deaf or hard-of-hearing (DHH) VR users to access the information given by sounds, which could limit their VR experience. To address this limitation, we present “SoundVizVR”, which explores visualizing sound characteristics and sound types for several types of sounds in VR experience. SoundVizVR uses Sound-Characteristic Indicators to visualize loudness, duration, and location of sound sources in VR and Sound-Type Indicators to present more information about the type of the sound. First, we examined three types of Sound-Characteristic Indicators (On-Object Indicators, Full Mini-Maps and Partial Mini-Maps) and their combinations in a study with 11 DHH participants. We identified that the combination of Full Mini-Map technique and On-Object Indicator was the most preferred visualization and performed best at locating sound sources in VR. Next, we explored presenting more information about the sounds using text and icons as Sound-Type Indicators. A second study with 14 DHH participants found that all Sound-Type Indicator combinations were successful at locating sound sources.

In this experience report, two deaf researchers with varying expertise, communication preferences, and technological skills document their experiences using Access on Demand (AoD), an Augmented Reality (AR) based accessibility application that provides on-demand real-time captioning and sign language interpretation services using the Vuzix Blade AR smart glasses. The researchers report their observations regarding using remote real-time American Sign Language (ASL) interpreting, captioning, and auto-captions offered by the AoD platform. The authors discuss the benefits and limitations of using AoD as an assistive technology device and how it would benefit the deaf community from the perspective of Deaf and Hard-of-Hearing (DHH) users.