Tactile graphics present visual information to blind and visually-impaired individuals in an accessible way, through touch. Current methods for producing tactile graphics, such as embossing or swell-paper printing, have limitations such as durability - and the tools required to produce them are limited in expressiveness. In this project, we explore embroidery as a medium for producing tactile graphics. Embroidery, traditionally known for its variety and visual beauty, offers not just improved durability and ease of production - but the ability to convey information through a broad range of stitch types. Following an exploration of the design space of embroidered tactile graphics, we identify key perceptual properties that impact how embroidered textures are differentiated. Based on these differences, we introduce an optimization algorithm for assigning textures to regions of tactile graphics in a way that makes them diverse and legible. We implement an end-to-end pipeline for producing embroidered tactile graphics and evaluate the comprehensibility and legibility of our design with 6 blind participants. Our findings showed that embroidered tactile graphics present information accurately and comprehensively, and that measurable properties, such as the use of spacing and distinctiveness, were an important factor of expressive and legible design.

Figure 1: With A11yShape, (A) a blind or low-vision (BLV) user can create, interpret, and verify 3-D models through (B) a user interface composed of three parts: Code Editor Panel, AI Assistant Panel, and Model Panel.These panels are linked by a cross-representation highlighting mechanism that connects code, textual descriptions, hierarchical model abstractions, and 3-D visual renderings.The system supports the creation of (C) diverse, customized 3-D models created by BLV users.

Peer Reviewed

Sensory-substitution devices enable perceiving objects by translating one modality (e.g., vision) into another (e.g., tactile). While many explored the placement of the haptic-output (e.g., torso, forehead), the camera's location remains largely unexplored—typically seeing from the eyes’ perspective. Instead, we propose that seeing & feeling information from the hands’ perspective could enhance flexibility & expressivity of sensory-substitution devices to support manual interactions with physical objects. To this end, we engineered a back-of-the-hand electrotactile-display that renders tactile images from a wrist-mounted camera, allowing the user's hand to feel objects while reaching & hovering. We conducted a study with sighted/Blind-or-Low-Vision participants who used our eyes vs. hand tactile-perspectives to manipulate bottles and soldering-irons, etc. We found that while both tactile perspectives provided comparable performance, when offered the opportunity to choose, all participants found value in also using the hands’ perspective. Moreover, we observed behaviors when “seeing with the hands” that suggest a more ergonomic object-manipulation. We believe these insights extend the landscape of sensory-substitution devices.

Presentation software like Microsoft PowerPoint and Google Slides remains largely inaccessible for blind users because screen readers are not well suited to 2-D “artboards” that contain different objects in arbitrary arrangements lacking any inherent reading order. To investigate this problem, prior work by Zhang & Wobbrock (2023) developed multimodal interaction techniques in a prototype system called A11yBoard, but their system was limited to a single artboard in a self-contained prototype and was unable to support real-world use. In this work, we present a major extension of A11yBoard that expands upon its initial interaction techniques, addresses numerous real-world issues, and makes it deployable with Google Slides. We describe the new features developed for A11yBoard for Google Slides along with our participatory design process with a blind co-author. We also present two case studies based on real-world deployments showing that participants were able to independently complete slide reading and authoring tasks that were not possible without sighted assistance previously. We conclude with several design guidelines for making accessible digital content creation tools.

Electrical and computer engineering (ECE) theory and tooling are foundational prerequisites for learners’ success in designing and building systems (so called “making” activities) and in several STEM majors in higher education. However, ECE education remains significantly inaccessible to students who are blind or have low vision (BLV), due to the field's weighty reliance on visual content. In this work, we reflect on the autoethnographic account of the first blind student to complete the introductory ECE and making course at our educational institution, identifying three main areas pertaining to the overarching theme of inequitable challenges, including the student's experiences with (1) conceptual learning and tooling, (2) translating theory into practice, and (3) experimenting with learning methods and approaches. We then bring together our findings, previous works, and existing tools to propose a preliminary list of design considerations to inform the development of future accessible tools for ECE education and reduce access barriers to the field.

For non-technical domain experts and designers it can be a substantial challenge to create designs that meet domain specific goals. This presents an opportunity to create specialized tools that produce optimized designs in the domain. However, implementing domain-specific optimization methods requires a rare combination of programming and domain expertise. Creating flexible design tools with re-configurable optimizers that can tackle a variety of problems in a domain requires even more domain and programming expertise. We present OPTIMISM, a toolkit which enables programmers and domain experts to collaboratively implement an optimization component of design tools. OPTIMISM supports the implementation of metaheuristic optimization methods by factoring them into easy to implement and reuse components: objectives that measure desirable qualities in the domain, modifiers which make useful changes to designs, design and modifier selectors which determine how the optimizer steps through the search space, and stopping criteria that determine when to return results. Implementing optimizers with OPTIMISM shifts the burden of domain expertise from programmers to domain experts.

Online data visualizations play an important role in informing public opinion but are often inaccessible to screen reader users. To address the need for accessible data representations on the web that provide direct, multimodal, and up-to-date access to the data, we investigate audio data narratives –which combine textual descriptions and sonification (the mapping of data to non-speech sounds). We conduct two co-design workshops with screen reader users to define design principles that guide the structure, content, and duration of a data narrative. Based on these principles and relevant auditory processing characteristics, we propose a dynamic programming approach to automatically generate an audio data narrative from a given dataset. We evaluate our approach with 16 screen reader users. Findings show with audio narratives, users gain significantly more insights from the data. Users describe data narratives help them better extract and comprehend the information in both the sonification and description.

The ability to effectively read and interpret tactile graphics and charts is an essential part of a tactile learner’s path to literacy, but is a skill that requires instruction and training. Many teachers of the visual impaired (TVIs) report that blind and visually impaired students have trouble interpreting graphics independently without individual instruction. We present PantoGuide, a low-cost system that provides audio and haptic guidance, via skin-stretch feedback to the dorsum of a user’s hand while the user explores a tactile graphic overlaid on a touchscreen. This system allows programming of haptic guidance patterns and cues for tactile graphics that can be experienced by students learning remotely or that can be reviewed by a student independently. We propose two teaching scenarios (synchronous and asynchronous) and two guidance interactions (point-to-point and continuous) that the device can support – and demonstrate their use in a set of applications we co-design with one co-author who is blind and a tactile graphics user.