Voice user interfaces (VUIs) are rapidly transitioning from accessibility features to mainstream interaction modalities. Yet most operating systems’ built-in voice commands remain underutilized despite possessing robust technical capabilities. Through our analysis of four commercial VUI systems and a formative study with 16 participants, we found that fixed command formats require exact phrasing, restrictive timeout mechanisms discard input during planning pauses, and insufficient feedback hampers multi-step interactions. To address these challenges, we developed VoiceAlign, an adaptive shimming layer that mediates between users and legacy VUI systems. VoiceAlign intercepts natural voice commands, transforms them to match the required syntax using a large language model, and transmits these adapted commands through a virtual audio channel that remains transparent to the underlying system. In our evaluation with 12 participants, VoiceAlign reduced command failures by half, required 25% fewer commands per task, and significantly lowered cognitive and temporal demands when paired with an existing legacy VUI system. Furthermore, we created a synthetic dataset informed by our studies and fine-tuned a small language model that achieves over 90% accuracy with 200 ms response time when served locally, eliminating dependence on third-party APIs while enabling real-time interaction on edge devices. This work demonstrates how modern AI techniques can unlock the underutilized potential of legacy VUI systems without requiring system modifications, offering a practical solution without replacing existing infrastructure.

This paper examines how the coding strategies of sighted and blind programmers differ when working with audio feedback alone.The goal is to identify challenges in mixed-ability collaboration, particularly when sighted programmers work with blind peers or teach programming to blind students.To overcome limitations of traditional blindness simulation studies, we proposed Task-Oriented Priming and Sensory Alignment (ToPSen), a design framework that reframes sensory constraints as technical requirements rather than as a disability.Through a study of 12 blind and 12 sighted participants coding non-visually, we found that expert blind programmers maintain more accurate mental models and process more information in working memory than sighted programmers using ToPSen.Our analysis revealed that blind and sighted programmers process structural information differently, exposing gaps in current IDE designs.These insights inform our guidelines for improving the accessibility of programming tools and fostering effective mixedability collaboration.

Sonification translates data into non-speech audio. Such auditory representations can make data visualization accessible to people who are blind or have low vision (BLV). This paper presents a sonification method for translating common data visualization into a blend of natural sounds. We hypothesize that people’s familiarity with sounds drawn from nature, such as birds singing in a forest, and their ability to listen to these sounds in parallel, will enable BLV users to perceive multiple data points being sonified at the same time. Informed by an extensive literature review and a preliminary study with 5 BLV participants, we designed an accessible data representation tool, Susurrus, that combines our sonification method with other accessibility features, such as keyboard interaction and text-to-speech feedback. Finally, we conducted a user study with 12 BLV participants and report the potential and application of natural sounds for sonification compared to existing sonification tools.

Sighted programmers often rely on visual cues (e.g., syntax coloring, keyword highlighting, code formatting) to perform common coding activities in text-based languages (e.g., Python). Unfortunately, blind and low-vision (BLV) programmers hardly benefit from these visual cues because they interact with computers via assistive technologies (e.g., screen readers), which fail to communicate visual semantics meaningfully. Prior work on making text-based programming languages and environments accessible mostly focused on code navigation and, to some extent, code debugging, but not much toward code editing, which is an essential coding activity.

Screen reader plugins are small pieces of code that blind users can download and install to enhance the capabilities of their screen readers. In this paper, we aim to understand the user experience of screen readers’ plugins, as well as their developers, distribution model, and maintenance. To this end, we conducted a study with 14 blind screen reader users. Our study revealed that screen reader users rely on plugins for various reasons, e.g., to improve the usability of both screen readers and application software, to make partially accessible applications accessible, and to enable custom shortcuts and commands. Furthermore, installing plugins is easy; uninstalling them is unlikely; and finding them online is ad hoc, challenging, and poses security threats. In addition, developing screen reader plugins is technically demanding; only a handful of people develop plugins, and they are well-recognized in the community. Finally, there is no central repository for plugins for most screen readers, and most plugins do not receive updates from their developers and become obsolete. The lack of financial incentives plays in the slow growth of the plugin ecosystem. Based on our findings, we recommend creating a central repository for all plugins, engaging third-party developers, and raising general awareness about the benefits and dangers of plugins. We believe our findings will inspire researchers to embrace the plugin-based distribution model as an effective way to combat application-level accessibility issues.