Processing speech in noisy environments is a core challenge for children with attention deficit hyperactivity disorder (ADHD), and few studies investigated evidence-based interventions. Disrupted visual attention and audiovisual integration are key contributors to these difficulties. Virtual reality (VR) games offer potential to support attention and audiovisual integration training, yet few are designed for this purpose. Current approaches using VR games emphasize either clinical fidelity or entertainment, creating an imbalance between neural engagement and sustained motivation. We developed a VR game through a co-design approach, informed by neural mechanisms and clinical expertise, and embedding child-centered interaction to sustain engagement. The game includes variations in interaction modes and difficulty levels, iteratively refined with neurodevelopmental specialists and children. We conducted an exploratory study with 11 participants, including neurotypical and ADHD children. Findings highlight task performance and insights from the target user group, while also suggesting implications for balancing clinical potential with user engagement.

Recent research has begun adopting Large Language Model (LLM) agents to enhance Virtual Reality (VR) interactions, creating immersive chatbot experiences. However, while current studies focus on generating dialogue from user speech inputs, their abilities to generate richer experiences based on the perception of LLM agents’ VR environments and interaction cues remain unexplored. Hence, in this work, we propose an approach that enables LLM agents to perceive virtual environments and generate environment-aware interactions and conversations for an embodied human-AI interaction experience in VR environments. Here, we define a schema for describing VR environments and their interactions through text prompts. We evaluate the performance of our method through five role-play scenarios created using our approach in a study with 14 participants. The findings discuss the opportunities and challenges of our proposed approach for developing environment-aware LLM agents that facilitate spatial interactions and conversations within VR role-play scenarios.

Virtual Reality (VR) systems use immersive spatial audio to convey critical information, but these audio cues are often inaccessible to Deaf or Hard-of-Hearing (DHH) individuals. To address this, we developed SoundHapticVR, a head-based haptic system that converts audio signals into haptic feedback using multi-channel acoustic haptic actuators. We evaluated SoundHapticVR through three studies: determining the maximum tactile frequency threshold on different head regions for DHH users, identifying the ideal number and arrangement of transducers for sound localization, and assessing participants’ ability to differentiate sound sources with haptic patterns. Findings indicate that tactile perception thresholds vary across head regions, necessitating consistent frequency equalization. Adding a front transducer significantly improved sound localization, and participants could correlate distinct haptic patterns with specific objects. Overall, this system has the potential to make VR applications more accessible to DHH users.

Two-factor Authentication (also known as 2FA or two-step verification) is an authentication method that provides an extra layer of protection to ensure online account security. 2FA methods are used along with other primary authentication methods like PINs and Passwords to verify that the person trying to access any digital account is the person they are claiming to be. However, 2FA methods can be inaccessible for blind and low vision (BLV) users due to the requirement of multiple steps, apps, and/or devices for authentication. In addition, it can be a security risk as screen readers may read out the verification codes to bystanders. To address this, we present Haptic2FA, a haptic-based authentication method to improve 2FA accessibility for BLV users. Here, as a part of the 2FA process, the users are sent a 'haptic pattern' (similar to a one-time passcode in traditional 2FA methods) that they are required to enter or select for verification. Through a usability study with 10 BLV participants, we evaluated haptic patterns and input methods for the haptic patterns in the Haptic2FA method. Through the findings, we discuss the accessibility and usability of the Haptic2FA method.

Captions make the audio content of videos accessible and understandable for deaf or hard-of-hearing people (DHH). However, in real-time captioning scenarios, captions alone can be challenging for DHH users to identify the active speaker in a real time in multiple-speaker scenarios. To enhance the accessibility of real-time captioning, we propose Haptic-Captioning which provides real-time vibration feedback on the wrist by directly translating the sound of content into vibrations. We conducted three experiments to examine: (1) the haptic perception (Preliminary Study), (2) the feasibility of the haptic modality along with real-time and non-real-time visual captioning methods (Study 1), and (3) the user experience of using the Haptic-Captioning system in different media contexts (Study 2). Our results highlight that the Haptic-Captioning complements visual captions by improving caption readability, maintaining media engagement, enhancing understanding of emotions, and assisting speaker indication in real-time captioning scenarios. Furthermore, we discuss design implications for the future development of Haptic-Captioning.

Sounds provide vital information such as spatial and interaction cues in virtual reality (VR) applications to convey more immersive experiences to VR users. However, it may be a challenge for deaf or hard-of-hearing (DHH) VR users to access the information given by sounds, which could limit their VR experience. To address this limitation, we present “SoundVizVR”, which explores visualizing sound characteristics and sound types for several types of sounds in VR experience. SoundVizVR uses Sound-Characteristic Indicators to visualize loudness, duration, and location of sound sources in VR and Sound-Type Indicators to present more information about the type of the sound. First, we examined three types of Sound-Characteristic Indicators (On-Object Indicators, Full Mini-Maps and Partial Mini-Maps) and their combinations in a study with 11 DHH participants. We identified that the combination of Full Mini-Map technique and On-Object Indicator was the most preferred visualization and performed best at locating sound sources in VR. Next, we explored presenting more information about the sounds using text and icons as Sound-Type Indicators. A second study with 14 DHH participants found that all Sound-Type Indicator combinations were successful at locating sound sources.