As social virtual reality (VR) grows more popular, addressing accessibility for blind and low vision (BLV) users is increasingly critical. Researchers have proposed an AI “sighted guide” to help users navigate VR and answer their questions, but it has not been studied with users. To address this gap, we developed a large language model (LLM)-powered guide and studied its use with 16 BLV participants in virtual environments with confederates posing as other users. We found that when alone, participants treated the guide as a tool, but treated it companionably around others, giving it nicknames, rationalizing its mistakes with its appearance, and encouraging confederate-guide interaction. Our work furthers understanding of guides as a versatile method for VR accessibility and presents design recommendations for future guides.

Virtual collaboration has transformed how people in mixed-ability teams, composed of disabled and non-disabled people, work together by offering greater flexibility. In these settings, accessibility practices, such as accommodations and inclusive norms, are essential for providing access to disabled people. However, we do not yet know how these practices shape broader facets of teamwork, such as productivity, participation, and camaraderie. To address this gap, we interviewed 18 participants (12 disabled, 6 non-disabled) who are part of mixed-ability teams. We found that beyond providing access, accessibility practices shaped how all participants coordinated tasks, sustained rapport, and negotiated responsibilities. Accessibility practices also introduced camaraderie challenges, such as balancing empathy and accountability. Non-disabled participants described allyship as a learning process and skill shaped by their disabled team members and team culture. Based on our findings, we present recommendations for team practices and design opportunities for virtual collaboration tools that reframe accessibility practices as a foundation for strong teamwork.

Multimodal large language models (MLLMs) are changing how Blind and Low Vision (BLV) people access visual information. Unlike traditional visual interpretation tools that only provide descriptions, MLLM-enabled applications offer conversational assistance, where users can ask questions to obtain goal-relevant details. However, evidence about their performance in the real-world and implications for BLV people’s daily lives remains limited. To address this, we conducted a two-week diary study, where we captured 20 BLV participants’ use of an MLLM-enabled visual interpretation application. Although participants rated the visual interpretations of the application as "trustworthy" (mean=3.76 out of 5, max=extremely trustworthy) and "somewhat satisfying" (mean=4.13 out of 5, max=very satisfying), the AI often produced incorrect answers (22.2%) or abstained (10.8%) from responding to users’ requests. Our findings show that while MLLMs can improve visual interpretations’ descriptive accuracy, supporting everyday use also depends on the “visual assistant” skill: behaviors for providing goal-directed, reliable assistance. We conclude by proposing the "visual assistant" skill and guidelines to help MLLM-enabled visual interpretation applications better support BLV people’s access to visual information.

Disabled people on social media often experience ableist hate and microaggressions. Prior work has shown that platform moderation often fails to remove ableist hate leaving disabled users exposed to harmful content. This paper examines how personalized moderation can safeguard users from viewing ableist comments. During interviews and focus groups with 23 disabled social media users, we presented design probes to elicit perceptions on configuring their filters of ableist speech (e.g. intensity of ableism and types of ableism) and customizing the presentation of the ableist speech to mitigate the harm (e.g. AI rephrasing the comment and content warnings). We found that participants preferred configuring their filters through types of ableist speech and favored content warnings. We surface participants distrust in AI-based moderation, skepticism in AI's accuracy, and varied tolerances in viewing ableist hate. Finally we share design recommendations to support users' agency, mitigate harm from hate, and promote safety.

Social virtual reality (VR) has become one of the most popular forms of VR. However, despite years of research on how VR interventions can be useful as diagnostic or therapeutic tools for neurodivergent (ND) users, there has been little examination of how accessible social VR may be for such ND individuals. In this paper, we describe an ongoing user study with participants who self-identify with both autism and ADHD (AuDHD) and also self-identify with facing frequent challenges with social interaction. So far, we have recruited four AuDHD participants; we had each participant briefly explore a world on a popular commercial social VR platform and then reflect on this experience afterward in a longer interview section. Through this process, we uncovered various accessibility challenges in social VR, such as difficulties with navigating social norms or managing certain sensory inputs. We also noted ideas on potential accommodations, like a text-based prompt system that can suggest “appropriate” conversation responses. Our work outlines opportunities to improve the accessibility of social VR for an often-overlooked user group.

The rapid growth of virtual reality (VR) has led to increased use of social VR platforms for interaction. However, these platforms lack adequate features to support blind and low vision (BLV) users, posing significant challenges in navigation, visual interpretation, and social interaction. One promising approach to these challenges is employing human guides in VR. However, this approach faces limitations with a lack of availability of humans to serve as guides, or the inability to customize the guidance a user receives from the human guide. We introduce an AI-powered guide to address these limitations. The AI guide features six personas, each offering unique behaviors and appearances to meet diverse user needs, along with visual interpretation and navigation assistance. We aim to use this AI guide in the future to help us understand BLV users’ preferences for guide forms and functionalities.

With the increasing adoption of social virtual reality (VR), it is critical to design inclusive avatars. While researchers have investigated how and why blind and d/Deaf people wish to disclose their disabilities in VR, little is known about the preferences of many others with invisible disabilities (e.g., ADHD, dyslexia, chronic conditions). We filled this gap by interviewing 15 participants, each with one to three invisible disabilities, who represented 22 different invisible disabilities in total. We found that invisibly disabled people approached avatar-based disclosure through contextualized considerations informed by their prior experiences. For example, some wished to use VR’s embodied affordances, such as facial expressions and body language, to dynamically represent their energy level or willingness to engage with others, while others preferred not to disclose their disability identity in any context. We define a binary framework for embodied invisible disability expression (public and private) and discuss three disclosure patterns (Activists, Non-Disclosers, and Situational Disclosers) to inform the design of future inclusive VR experiences.

With recent movements toward disability as a social identity, we explore whether pain associated with chronic pain conditions (e.g., arthritis, Crohn’s disease, lupus) is also linked to identity and representation preferences. Prior work showed social VR users with invisible disabilities noted preliminary interest in using their avatar’s body language to represent their disability-related identities. We examined movement-based social virtual reality (VR) avatar representation preferences by conducting semi-structured interviews with five participants with such chronic pain conditions. Participants incorporated social norms, cultural considerations, and internalized self-stigma into their decision-making about pain disclosure and representation in different contexts. Aligning with previous work on self-presence and embodiment, in order to avoid discomfort, most participants wanted to avoid experiences where their avatar moved in ways that they did not, or could not, move in the physical world (i.e., jumping, bending over from the spine). Two participants also wanted to be able to represent their personal use of clothing and fashion as accommodation in the physical world. We believe this study will further our understanding of how disability-related identities should be represented in social VR spaces.

Social VR has increased in popularity due to its affordances for rich, embodied, and nonverbal communication. However, nonverbal communication remains inaccessible for blind and low vision people in social VR. We designed accessible cues with audio and haptics to represent three nonverbal behaviors: eye contact, head shaking, and head nodding. We evaluated these cues in real-time conversation tasks where 16 blind and low vision participants conversed with two other users in VR. We found that the cues were effective in supporting conversations in VR. Participants had statistically significantly higher scores for accuracy and confidence in detecting attention during conversations with the cues than without. We also found that participants had a range of preferences and uses for the cues, such as learning social norms. We present design implications for handling additional cues in the future, such as the challenges of incorporating AI. Through this work, we take a step towards making interpersonal embodied interactions in VR fully accessible for blind and low vision people.

Cortical/cerebral visual impairment (CVI) is a neurological vision impairment that affects the visual processing centers of the brain. Digital learning tools are becoming increasingly popular in schools, but limited research has examined how tools can help students with CVI, who have distinct academic needs. To better understand how tools should be designed to help students with CVI, we interviewed 20 U.S.-based Teachers of the Visually Impaired (TVIs) who worked with students with CVI. We found that our participants analyzed highly specific information about students’ visual, learning, social, and environmental needs to create accommodations for academic materials, most of which they created on their own. Participants shared design critiques of existing digital learning tools and felt that well-designed tools could save them time and improve learning opportunities for their students. We provide design considerations for tools that can be effectively used by students with CVI.

Content creators (e.g., gamers, activists, vloggers) with marginalized identities are at-risk of experiencing hate and harassment. This paper examines the ableist hate and harassment that disabled content creators experience on social media. Through surveys (N=50) and interviews (N=20) with disabled creators, we developed a taxonomy of 11 types of ableist hate and harassment (e.g., eugenics-related speech, denial and stigmatization of accessibility) and outlined how ableism harms creators’ well-being and content creation practices. Using statistical modeling, we investigated differences in ableist experiences given creators’ intersecting identities such as race and sexuality. We found that LGBTQ disabled creators face significantly more ableist hate compared to non-LGBTQ disabled creators. Lastly, we discuss our findings through an infrastructure lens to highlight how disabled creators experience platform-enabled ableism, undergo labor to cope with hate, and develop strategies to safeguard against future hate.

While audio description (AD) is the standard approach for making videos accessible to blind and low vision (BLV) people, existing AD guidelines do not consider BLV users’ varied preferences across viewing scenarios. These scenarios range from how-to videos on YouTube, where users seek to learn new skills, to historical dramas on Netflix, where a user’s goal is entertainment. Additionally, the increase in video watching on mobile devices provides an opportunity to integrate nonverbal output modalities (e.g., audio cues, tactile elements, and visual enhancements). Through a formative survey and 15 semi-structured interviews, we identified BLV people’s video accessibility preferences across diverse scenarios. For example, participants valued action and equipment details for how-to videos, tactile graphics for learning scenarios, and 3D models for fantastical content. We define a six-dimensional video accessibility design space to guide future innovation and discuss how to move from “one-size-fits-all” paradigms to scenario-specific approaches.

"Scene description" applications that describe visual content in a photo are useful daily tools for blind and low vision (BLV) people. Researchers have studied their use, but they have only explored those that leverage remote sighted assistants; little is known about applications that use AI to generate their descriptions. Thus, to investigate their use cases, we conducted a two-week diary study where 16 BLV participants used an AI-powered scene description application we designed. Through their diary entries and follow-up interviews, users shared their information goals and assessments of the visual descriptions they received. We analyzed the entries and found frequent use cases, such as identifying visual features of known objects, and surprising ones, such as avoiding contact with dangerous objects. We also found users scored the descriptions relatively low on average, 2.76 out of 5 (SD=1.49) for satisfaction and 2.43 out of 4 (SD=1.16) for trust, showing that descriptions still need significant improvements to deliver satisfying and trustworthy experiences. We discuss future opportunities for AI as it becomes a more powerful accessibility tool for BLV users.

As augmented and virtual reality (XR) technologies gain traction, they remain almost completely inaccessible to people with disabilities. How can a blind person walk around a virtual bar in Horizon Worlds? How can a Deaf person have a conversation with a friend in VR Chat? It is critical to think about these challenges and address them proactively, before people with disabilities are left further behind. In this talk, I will present XR Access, which I co-founded to promote XR accessibility research and practice. For the last five years, XR Access has been bringing together practitioners, researchers, and advocates in conferences and seminars, developing resources, and supporting students in XR accessibility. I will present a taste of XR accessibility research, and outline ways in which you too can join this movement.

As social VR applications grow in popularity, blind and low vision users encounter continued accessibility barriers. Yet social VR, which enables multiple people to engage in the same virtual space, presents a unique opportunity to allow other people to support a user’s access needs. To explore this opportunity, we designed a framework based on physical sighted guidance that enables a guide to support a blind or low vision user with navigation and visual interpretation. A user can virtually hold on to their guide and move with them, while the guide can describe the environment. We studied the use of our framework with 16 blind and low vision participants and found that they had a wide range of preferences. For example, we found that participants wanted to use their guide to support social interactions and establish a human connection with a human-appearing guide. We also highlight opportunities for novel guidance abilities in VR, such as dynamically altering an inaccessible environment. Through this work, we open a novel design space for a versatile approach for making VR fully accessible.

As social virtual reality (VR) experiences become more popular, it is critical to design accessible and inclusive embodied avatars. At present, there are few, if any, customization features for invisible disabilities (e.g., chronic health conditions, mental health conditions, neurodivergence) in social VR platforms. To our knowledge, researchers have yet to explore how people with invisible disabilities want to self-represent and disclose disabilities through social VR avatars. We fill this gap in current accessibility research by centering the experiences and preferences of people with invisible disabilities. We conducted semi-structured interviews with nine participants and found that people with invisible disabilities used a unique, indirect approach to inform dynamic disclosure practices. Participants were interested in toggling representation on/off across contexts and shared ideas for representation through avatar design. In addition, they proposed ways to make the customization process more accessible (e.g., making it easier to import custom designs). We see our work as a vital contribution to the growing literature that calls for more inclusive social VR.

While audio description (AD) is a standard method for making traditional videos more accessible to blind and low vision (BLV) users, we lack an understanding of how to make 360° videos accessible while preserving their immersive nature. Through individual interviews and collaborative design workshops, we explored ways to improve 360° video accessibility with immersion and engagement in mind. Our design workshops presented a unique opportunity for participants with diverse backgrounds to build on each others’ personal and professional experiences and collaboratively develop accessible 360° video prototypes. Participants included both AD creators and users, with a focus on BLV AD creators as their perspectives are underrepresented in prior work. We found that immersive video accessibility went beyond an extension of traditional video accessibility techniques. Participants valued accurate vocabulary and different points of view for descriptions, preferred a variety of presentation locations for spatialized AD, appreciated sound effects for setting the mood and subtly guiding, and wished to engage multiple senses to boost engagement. We conclude with implications for immersive media accessibility and future research directions to support disabled people as creators of access technology.

Conversational assistants, an inherently accessible mode of interaction for blind and low vision (BLV) individuals, offer opportunities to support nonvisual access in new ways. In this paper, we explore whether and how human-like conversations can support access to interfaces, advancing the impersonal linear access provided by screen readers today. We first interviewed 10 BLV participants about this approach, but found it difficult to situate conversations around a future technology. So we turned to a speculative design approach and created four audio fictions: pre-recorded dialogues between users and their hypothetical screen reader assistants wherein assistants assumed distinct roles: a friend, butler, expert, and caregiver. We presented the audio fictions to 14 BLV participants and found that personable conversations can meaningfully extend the screen reader experience. We observed a tension between AI adaptation and screen reader customization. Participants further expressed a need to maintain control at three distinct levels: granular cursor movement, screen representation, and task assistance. Through the lens of assistant roles, we address fundamental questions about anthropomorphizing CAs and assistive technology broadly.

While past research has examined older adults’ voice assistant (VA) use, it is unclear whether VAs provide enough value to sustain use when compared to technologies such as smartphones. Research also suggests that barriers around structured command input may limit use. In order to investigate these gaps in adoption, we conducted interviews with ten older adults in a long-term care community who have adopted Alexa devices for at least one year. Participants learned to use Alexa through a training program that encouraged exploration. They used Alexa to complement their daily routines, improve their mood, engage in cognitively stimulating activities, and support socialization with others. We discuss our findings in the context of prior work, describe strategies to promote VA learning and adoption, and present design recommendations to support aging.

Drones (micro unmanned aerial vehicles) are becoming more prevalent in applications that bring them into close human spaces. This is made possible in part by clear drone-to-human communication strategies. However, current auditory and visual communication methods only work with strict environmental settings. To continue expanding the possibilities for drones to be useful in human spaces, we explore ways to overcome these limitations through physical touch. We present a new application for drones--physical instructive feedback. To do this we designed three different physical interaction modes for a drone. We then conducted a user study (N=12) to answer fundamental questions of where and how people want to physically interact with drones, and what people naturally infer the physical touch is communicating. We then used these insights to conduct a second user study (N=14) to understand the best way for a drone to communicate instructions to a human in a movement task. We found that continuous physical feedback is both the preferred mode and is more effective at providing instruction than incremental feedback.

Ableist microaggressions are subtle forms of discrimination that disabled people experience daily, perpetuating inequalities and maintaining their ongoing marginalization. Despite the importance of understanding such harms, little work has been done to examine how disabled people are discriminated against online. We address this gap by investigating how disabled people experience ableist microaggressions on social media and how they respond to and cope with these experiences. By conducting interviews with 20 participants with various disabilities, we uncover 12 archetypes of ableist microaggressions on social media, reveal participants’ coping mechanisms, and describe the long-term impact on their wellbeing and social media use. Lastly, we present design recommendations, re-evaluating how social media platforms can mitigate and prevent these harmful experiences.

Selfies are a pervasive form of communication in social media. While there has been some work on systems that guide people with visual impairments (PVI) in taking photos, nearly all has focused on using the camera on the back of the device. We do not know whether and how PVI take selfies. The aim of our work is to understand (1) PVI selfie-taking experiences and challenges, (2) what information do PVI need when taking selfies, and (3) what modalities do PVI prefer (e.g., tactile, verbal, or non-verbal audio) to support selfie-taking. To address this gap, we conducted interviews with 10 PVI. Our findings show that current selfie-taking applications do not provide enough assistance to meet the needs of PVI. We contribute design guidelines that researchers and designers can implement for creating accessible selfie-taking applications.

Scholars have recently drawn attention to a range of controversial issues posed by the use of computer vision for automatically generating descriptions of people in images. Despite these concerns, automated image description has become an important tool to ensure equitable access to information for blind and low vision people. In this paper, we investigate the ethical dilemmas faced by companies that have adopted the use of computer vision for producing alt text: textual descriptions of images for blind and low vision people. We use Facebook's automatic alt text tool as our primary case study. First, we analyze the policies that Facebook has adopted with respect to identity categories, such as race, gender, age, etc., and the company's decisions about whether to present these terms in alt text. We then describe an alternative---and manual---approach practiced in the museum community, focusing on how museums determine what to include in alt text descriptions of cultural artifacts. We compare these policies, using notable points of contrast to develop an analytic framework that characterizes the particular apprehensions behind these policy choices. We conclude by considering two strategies that seem to sidestep some of these concerns, finding that there are no easy ways to avoid the normative dilemmas posed by the use of computer vision to automate alt text.

Response bias has been framed as the tendency of a participant's response to be skewed by a variety of factors, including study design and participant-researcher dynamics. Response bias is a concern for all researchers who conduct studies with people — especially those working with participants with disabilities. This is because these participants’ diverse needs require methodological adjustments and differences in disability identity between the researcher and participant influence power dynamics. Despite its relevance, there is little literature that connects response bias to accessibility. We conducted semi-structured interviews with 27 accessibility researchers on how response bias manifested in their research and how they mitigated it. We present unique instances of response bias and how it is handled in accessibility research; insights into how response bias interacts with other biases like researcher or sampling bias; and philosophies and tensions around response bias such as whether to accept or address it. We conclude with guidelines on thinking about response bias in accessibility research.

Students with specific learning disabilities (SLDs) often experience negative emotions when solving math problems, which they have difficulty managing. This is one reason that current math e-learning tools, which elicit these negative emotions, are not effective for these students. We designed an intelligent math e-tutoring system that aims to reduce students’ negative emotional behaviors. The system automatically detects possible negative emotional behaviors by analyzing gaze, inputs on the touchscreen, and response time. It then uses one of four intervention methods (e.g., hints or brain breaks) to prevent students from being upset. To form this design, we conducted a formative study with five teachers for students with SLDs. The teachers thought that the design of four intervention methods would help students with SLDs. Among the four intervention methods, providing brain breaks is new and particularly useful for the students. The teachers also suggested that the system should personalize the detection of negative emotional behaviors to help students who have more severe learning disabilities.

Although some technology companies have made significant strides towards the accessibility of their products, most consumer-facing technology products still pose access barriers to people with disabilities. Prior research has established that accessibility expertise is limited to a small number of practitioners in companies, but we do not know how these practitioners can affect change across a large organization. We sought to address this gap and understand how large companies that produce consumer-facing technologies integrate accessibility into their product lifecycle. We conducted semi-structured interviews with 30 accessible technology practitioners working at 13 companies. We found accessibility expertise was centered in three main roles within the company: on a central accessibility team, in champions, and in accessibility teams embedded into large product teams. Much of the work of these practitioners centered around education and development of tools and resources to allow designers and developers throughout the organization to implement accessibility. Our study revealed current practices for embedding accessibility in large companies, highlighting the gap between accessibility research and practice. We conclude by presenting areas that need future research to understand how to better support accessibility practice.

Many students with specific learning disabilities (SLDs) have difficulty learning math. To succeed in math, they need to receive personalized support from teachers. Recently, math e-learning tools that provide personalized math skills training have gained popularity. However, we know little about how well these tools help teachers personalize instruction for students with SLDs. To answer this question, we conducted semi-structured interviews with 12 teachers who taught students with SLDs in grades five to eight. We found that participants used math e-learning tools that were not designed specifically for students with SLDs. Participants had difficulty using these tools because of text-intensive user interfaces, insufficient feedback about student performance, inability to adjust difficulty levels, and problems with setup and maintenance. Participants also needed assistive technology for their students, but they had challenges in getting and using it. From our findings, we distilled design implications to help shape the design of more inclusive and effective e-learning tools.

Tactile materials are powerful teaching aids for students with visual impairments (VIs). To design these materials, designers must use modeling applications, which have high learning curves and rely on visual feedback. Today, Orientation and Mobility (O&M) specialists and teachers are often responsible for designing these materials. However, most of them do not have professional modeling skills, and many are visually impaired themselves. To address this issue, we designed Molder, an accessible design tool for interactive tactile maps, an important type of tactile materials that can help students learn O&M skills. A designer uses Molder to design a map using tangible input techniques, and Molder provides auditory feedback and high-contrast visual feedback. We evaluated Molder with 12 participants (8 with VIs, 4 sighted). After a 30-minute training session, the participants were all able to use Molder to design maps with customized tactile and interactive information.

Wayfinding is a critical but challenging task for people who have low vision, a visual impairment that falls short of blindness. Prior wayfinding systems for people with visual impairments focused on blind people, providing only audio and tactile feedback. Since people with low vision use their remaining vision, we sought to determine how audio feedback compares to visual feedback in a wayfinding task. We developed visual and audio wayfinding guidance on smartglasses based on de facto standard approaches for blind and sighted people and conducted a study with 16 low vision participants. We found that participants made fewer mistakes and experienced lower cognitive load with visual feedback. Moreover, participants with a full field of view completed the wayfinding tasks faster when using visual feedback. However, many participants preferred audio feedback because of its shorter learning curve. We propose design guidelines for wayfinding systems for low vision.

To help both designers and people with tetraplegia fully realize the benefts of voice assistant technology, we conducted interviews with fve people with tetraplegia in the home to understand how this population currently uses voice-based interfaces as well as other technologies in their everyday tasks. We found that people with tetraplegia use voice assistants in specifc places, such as in their beds, or when traveling in their wheelchair. In addition, we note the inefciencies for people with tetraplegia when using voice assistance.

Tactile maps are important tools for people with visual impairments (VIs). Teachers and orientation and mobility (O&M) specialists often design tactile maps to help their VI students and clients learn about geographic areas. To design these maps, a designer must use modeling software applications, which require professional training and rely on visual feedback. However, most teachers and O&M specialists do not have professional modeling skills, and many have visual impairments. The complexity and inaccessibility of current modeling tools thus become major barriers for TVIs and O&M specialists when designing tactile maps. We present Molder, an accessible design tool for tactile maps. A designer creates a draft map model using Molder and prints the model. Then, she uses Molder to modify the draft model by directly interacting with it. Molder provides auditory feedback and high-contrast visuals to assist the designer in the design process.

Students with specific learning disabilities (SLD) typically struggle in their K-12 math classes, limiting the likelihood of success in STEM fields. Private tutoring is reported to be effective at helping them succeed in math, but it is not a scalable solution. While many recent e-learning tools have aimed at personalizing math support in ways that might be scalable, there remains much to be done. To better understand the gaps between current tools and the particular needs of students with SLD (and of their teachers), we conducted semi-structured interviews with 10 middle school math teachers. Our findings shed light on both the learning challenges faced by students with SLD and on the instructional challenges their teachers experience with e-learning tools. Further, we came to appreciate the importance of harnessing teacher perspectives in the design of effective e-learning tools for special students.

Students with visual impairments struggle to learn various concepts in the academic curriculum because diagrams, images, and other visual are not accessible to them. To address this, researchers have design interactive 3D printed models (I3Ms) that provide audio descriptions when a user touches components of a model. In prior work, I3Ms were designed on an ad hoc basis, and it is currently unknown what general guidelines produce effective I3M designs. To address this gap, we conducted two studies with Teachers of the Visually Impaired (TVIs). First, we led two design workshops with 35 TVIs, who modified sample models and added interactive elements to them. Second, we worked with three TVIs to design three I3Ms in an iterative instructional design process. At the end of this process, the TVIs used the I3Ms we designed to teach their students. We conclude that I3Ms should (1) have effective tactile features (e.g., distinctive patterns between components), (2) contain both auditory and visual content (e.g., explanatory animations), and (3) consider pedagogical methods (e.g., overview before details).

Navigating stairs is a dangerous mobility challenge for people with low vision, who have a visual impairment that falls short of blindness. Prior research contributed systems for stair navigation that provide audio or tactile feedback, but people with low vision have usable vision and don't typically use nonvisual aids. We conducted the first exploration of augmented reality (AR) visualizations to facilitate stair navigation for people with low vision. We designed visualizations for a projection-based AR platform and smartglasses, considering the different characteristics of these platforms. For projection-based AR, we designed visual highlights that are projected directly on the stairs. In contrast, for smartglasses that have a limited vertical field of view, we designed visualizations that indicate the user's position on the stairs, without directly augmenting the stairs themselves. We evaluated our visualizations on each platform with 12 people with low vision, finding that the visualizations for projection-based AR increased participants' walking speed. Our designs on both platforms largely increased participants' self-reported psychological security.

Tactile models are important learning materials for visually impaired students. With the adoption of 3D printing technologies, visually impaired students and teachers will have more access to 3D printed tactile models. We designed Talkit++, an iOS application that plays audio and visual content as a user touches parts of a 3D print. With Talkit++, a visually impaired student can explore a printed model tactilely, and use finger gestures and speech commands to get more information about certain elements in the model. Talkit++ detects the model and finger gestures using computer vision algorithms, simple accessories like paper stickers and printable trackers, and the built-in RGB camera on an iOS device. Based on the model's position and the user's input, Talkit++ speaks textual information, plays audio recordings, and displays visual animations.

Walking in environments with stairs and curbs is potentially dangerous for people with low vision. We sought to understand what challenges low vision people face and what strategies and tools they use when navigating such surface level changes. Using contextual inquiry, we interviewed and observed 14 low vision participants as they completed navigation tasks in two buildings and through two city blocks. The tasks involved walking in- and outdoors, across four staircases and two city blocks. We found that surface level changes were a source of uncertainty and even fear for all participants. Besides the white cane that many participants did not want to use, participants did not use technology in the study. Participants mostly used their vision, which was exhausting and sometimes deceptive. Our findings highlight the need for systems that support surface level changes and other depth-perception tasks; they should consider low vision people's distinct experiences from blind people, their sensitivity to different lighting conditions, and leverage visual enhancements.

Recognizing others is a major challenge for people with visual impairments (VIPs) and can hinder engagement in social activities. We present Accessibility Bot, a research prototype bot on Facebook Messenger, that leverages state-of-the-art computer vision and a user's friends' tagged photos on Facebook to help people with visual impairments recognize their friends. Accessibility Bot provides users information about identity and facial expressions and attributes of friends captured by their phone's camera. To guide our design, we interviewed eight VIPs to understand their challenges and needs in social activities. After designing and implementing the bot, we conducted a diary study with six VIPs to study its use in everyday life. While most participants found the Bot helpful, their experience was undermined by perceived low recognition accuracy, difficulty aiming a camera, and lack of knowledge about the phone's status. We discuss these real-world challenges, identify suitable use cases for Accessibility Bot, and distill design implications for future face recognition applications.

Knocking is a way of interacting with everyday objects. We introduce BeatIt, a novel technique that allows users to use passive, everyday objects to control a smart environment by recognizing the sounds generated from knocking on the objects. BeatIt uses a BeatSet, a series of percussive sound samples, to represent the sound signature of knocking on an object. A user associates a BeatSet with an event. For example, a user can associate the BeatSet of knocking on a door with the event of turning on the lights. Decoder, a signal-processing module, classifies the sound signals into one of the recorded BeatSets, and then triggers the associated event. Unlike prior work, BeatIt can be implemented on microphone-enabled commodity devices. Our user studies with 12 participants showed that our proof-of-concept implementation based on a smartwatch could accurately classify eight BeatSets using a user-independent classifier.

Three-dimensional printed models have the potential to serve as powerful accessibility tools for blind people. Recently, researchers have developed methods to further enhance 3D prints by making them interactive: when a user touches a certain area in the model, the model speaks a description of the area. However, these interactive models were limited in terms of their functionalities and interaction techniques. We conducted a two-section study with 12 legally blind participants to fill in the gap between existing interactive model technologies and end users' needs, and explore design opportunities. In the first section of the study, we observed participants' behavior as they explored and identified models and their components. In the second section, we elicited user-defined input techniques that would trigger various functions from an interactive model. We identified five exploration activities (e.g., comparing tactile elements), four hand postures (e.g., using one hand to hold a model in the air), and eight gestures (e.g., using index finger to strike on a model) from the participants' exploration processes and aggregate their elicited input techniques. We derived key insights from our findings including: (1) design implications for I3M technologies, and (2) specific designs for interactions and functionalities for I3Ms.

People with low vision have a visual impairment that affects their ability to perform daily activities. Unlike blind people, low vision people have functional vision and can potentially benefit from smart glasses that provide dynamic, always-available visual information. We sought to determine what low vision people could see on mainstream commercial augmented reality (AR) glasses, despite their visual limitations and the device's constraints. We conducted a study with 20 low vision participants and 18 sighted controls, asking them to identify virtual shapes and text in different sizes, colors, and thicknesses. We also evaluated their ability to see the virtual elements while walking. We found that low vision participants were able to identify basic shapes and read short phrases on the glasses while sitting and walking. Identifying virtual elements had a similar effect on low vision and sighted people's walking speed, slowing it down slightly. Our study yielded preliminary evidence that mainstream AR glasses can be powerful accessibility tools. We derive guidelines for presenting visual output for low vision people and discuss opportunities for accessibility applications on this platform.

The emergence of personal computing devices offers both a challenge and opportunity for displaying text: small screens can be hard to read, but also support higher resolution. To fit content on a small screen, text must be small. This small text size can make computing devices unusable, in particular to low-vision users, whose vision is not correctable with glasses. Usability is also decreased for sighted users straining to read the small letters, especially without glasses at hand. We propose animated scripts called livefonts for displaying English with improved legibility for all users. Because paper does not support animation, traditional text is static. However, modern screens support animation, and livefonts capitalize on this capability. We evaluate our livefont variations' legibility through a controlled lab study with low-vision and sighted participants, and find our animated scripts to be legible across vision types at approximately half the size (area) of traditional letters, while previous smartfonts (static alternate scripts) did not show a significant legibility advantage for low-vision users. We evaluate the learnability of our livefont with low-vision and sighted participants, and find it to be comparably learnable to static smartfonts after two thousand practice sentences.

As three-dimensional printers become more available, 3D printed models can serve as important learning materials, especially for blind people who perceive the models tactilely. Such models can be much more powerful when augmented with audio annotations that describe the model and their elements. We present Markit and Talkit, a low-barrier toolkit for creating and interacting with 3D models with audio annotations. Makers (e.g., hobbyists, teachers, and friends of blind people) can use Markit to mark model elements and associate then with text annotations. A blind user can then print the augmented model, launch the Talkit application, and access the annotations by touching the model and following Talkit's verbal cues. Talkit uses an RGB camera and a microphone to sense users' inputs so it can run on a variety of devices. We evaluated Markit with eight sighted "makers" and Talkit with eight blind people. On average, non-experts added two annotations to a model in 275 seconds (SD=70) with Markit. Meanwhile, with Talkit, blind people found a specified annotation on a model in an average of 7 seconds (SD=8).

Like sighted people, visually impaired people want to share photographs on social networking services, but find it difficult to identify and select photos from their albums. We aimed to address this problem by incorporating state-of-the-art computer-generated descriptions into Facebook's photo-sharing feature. We interviewed 12 visually impaired participants to understand their photo-sharing experiences and designed a photo description feature for the Facebook mobile application. We evaluated this feature with six participants in a seven-day diary study. We found that participants used the descriptions to recall and organize their photos, but they hesitated to upload photos without a sighted person's input. In addition to basic information about photo content, participants wanted to know more details about salient objects and people, and whether the photos reflected their personal aesthetic. We discuss these findings from the lens of self-disclosure and self-presentation theories and propose new computer vision research directions that will better support visual content sharing by visually impaired people.

Graphics like maps and models are important learning materials. With recently developed projects, we can use 3D printers to make tactile graphics that are more accessible to blind people. However, current 3D printed graphics can only convey limited information through their shapes and textures. We present Magic Touch, a computer vision-based system that augments printed graphics with audio files associated with specific locations, or hotspots, on the model. A user can access an audio file associated with a hotspot by touching it with a pointing gesture. The system detects the user's gesture and determines the hotspot location with computer vision algorithms by comparing a video feed of the user's interaction with the digital representation of the model and its hotspots. To enable MT, a model designer must add a single tracker with fiducial tags to a model. After the tracker is added, MT only requires an RGB camera, so it can be easily deployed on many devices such as mobile phones, laptops and smart glasses.

Low vision is a pervasive condition in which people have difficulty seeing even with corrective lenses. People with low vision frequently use mainstream computing devices, however how they use their devices to access information and whether digital low vision accessibility tools provide adequate support remains understudied. We addressed these questions with a contextual inquiry study. We observed 11 low vision participants using their smartphones, tablets, and computers when performing simple tasks such as reading email. We found that participants preferred accessing information visually than aurally (e.g., screen readers), and juggled a variety of accessibility tools. However, accessibility tools did not provide them with appropriate support. Moreover, participants had to constantly perform multiple gestures in order to see content comfortably. These challenges made participants inefficient-they were slow and often made mistakes; even tech savvy participants felt frustrated and not in control. Our findings reveal the unique needs of low vision people, which differ from those of people with no vision and design opportunities for improving low vision accessibility tools.

Three-dimensional models are important learning resources for blind people. With advances in 3D printing, 3D models are becoming more available. However, unlike visual or tactile graphics, there is no standard accessible way to label components in 3D models. We present a labeling toolkit that enables users to add and access audio labels to 3D printed models. The toolkit includes Tickers, small 3D printed percussion instruments added to 3D models, and Talker, a signal processing application that detects and classifies Ticker sounds. To use the toolkit, a model designer adds Tickers to a model using 3D modeling software. A user then prints the model with Tickers and records audio labels for each Ticker. Finally, users can strum the Tickers and Talker will play the corresponding labels. We evaluated Tickers and Talker with three models in a study with nine blind participants. Our toolkit achieved an accuracy of 93% across all participants and models. We discuss design implications and future work for accessible 3D printed models.

In this paper, we explore blind people's motivations, challenges, interactions, and experiences with visual content on Social Networking Services (SNSs). We present findings from an interview study of 11 individuals and a survey study of 60 individuals, all with little to no functional vision. Compared to sighted SNS users, our blind participants faced profound accessibility challenges, including the prevalence of photos without sufficient text descriptions. To overcome the challenges, they developed creative strategies, including using a variety of methods to access SNS features (e.g., opening the mobile site on a desktop browser), and inferring photo content from textual cues and social interactions. When strategies failed, participants reached out for help from trusted friends, or avoided certain features. We discuss our findings in the context of CSCW research and SNS accessibility as a design value. We highlight the social significance of photo interactions for blind people and suggest design practices.

Building service robots - robots that perform various services in buildings - are becoming more common in large buildings such as hotels and stores. We aim to leverage such robots to serve as guides for blind people. In this paper, we sought to design specifications that detail how a building service robot could interact with and guide a blind person through a building in an effective and socially acceptable way. We conducted participatory design sessions with three designers and five non-designers. Two of the designers and all of the non-designers had a vision disability. Primary features of the design include allowing the user to (1) summon the robot after entering the building, (2) choose from three modes of assistance (Sighted Guide, Escort, and Information Kiosk), and (3) receive information about the building's layout from the robot. We conclude with a discussion of themes and a reflection about our design process that can benefit robot design for blind people in general.

Visual impairments encompass a range of visual abilities. People with low vision have functional vision and thus their experiences are likely to be different from people with no vision. We sought to answer two research questions: (1) what challenges do low vision people face when performing daily activities and (2) what aids (high- and low-tech) do low vision people use to alleviate these challenges? Our goal was to reveal gaps in current technologies that can be addressed by the UbiComp community. Using contextual inquiry, we observed 11 low vision people perform a wayfinding and shopping task in an unfamiliar environment. The task involved wayfinding and searching and purchasing a product. We found that, although there are low vision aids on the market, participants mostly used their smartphones, despite interface accessibility challenges. While smartphones helped them outdoors, participants were overwhelmed and frustrated when shopping in a store. We discuss the inadequacies of existing aids and highlight the need for systems that enhance visual information, rather than convert it to audio or tactile.

Visual search is a major challenge for low vision people. Conventional vision enhancements like magnification help low vision people see more details, but cannot indicate the location of a target in a visual search task. In this paper, we explore visual cues---a new approach to facilitate visual search tasks for low vision people. We focus on product search and present CueSee, an augmented reality application on a head-mounted display (HMD) that facilitates product search by recognizing the product automatically and using visual cues to direct the user's attention to the product. We designed five visual cues that users can combine to suit their visual condition. We evaluated the visual cues with 12 low vision participants and found that participants preferred using our cues to conventional enhancements for product search. We also found that CueSee outperformed participants' best-corrected vision in both time and accuracy.

As small displays on devices like smartwatches become increasingly common, many people have difficulty reading the text on these displays. Vision conditions like presbyopia that result in blurry near vision make reading small text particularly hard. We design multiple different scripts for displaying English text, legible at small sizes even when blurry, for small screens such as smartphones and smartwatches. These "smartfonts" redesign visual character presentations to improve the reading experience. Like cursive, Grade 1 Braille, and ordinary fonts, they preserve orthography and spelling. They have the potential to enable people to read more text comfortably on small screens, e.g., without reading glasses. To simulate presbyopia, we blur images and evaluate their legibility using paid crowdsourcing. We also evaluate the difficulty of learning to read smartfonts and observe a learnability/legibility trade-off. Our most learnable smartfont can be read at roughly half the speed of Latin after two thousand practice sentences. It is also legible smaller than half the size of traditional Latin (i.e. "English") when blurry.

Most low vision people have functional vision and would likely prefer to use their vision to access information. Recently, there have been advances in head-mounted displays, cameras, and image processing technology that create opportunities to improve the visual experience for low vision people. In this paper, we present ForeSee, a head-mounted vision enhancement system with five enhancement methods: Magnification, Contrast Enhancement, Edge Enhancement, Black/White Reversal, and Text Extraction; in two display modes: Full and Window. ForeSee enables users to customize their visual experience by selecting, adjusting, and combining different enhancement methods and display modes in real time. We evaluated ForeSee by conducting a study with 19 low vision participants who performed near- and far-distance viewing tasks. We found that participants had different preferences for enhancement methods and display modes when performing different tasks. The Magnification Enhancement Method and the Window Display Mode were popular choices, but most participants felt that combining several methods produced the best results. The ability to customize the system was key to enabling people with a variety of different vision abilities to improve their visual experience.

There are many educational smartphone games for children, but few are accessible to blind children. We present BraillePlay, a suite of accessible games for smartphones that teach Braille character encodings to promote Braille literacy. The BraillePlay games are based on VBraille, a method for displaying Braille characters on a smartphone. BraillePlay includes four games of varying levels of difficulty: VBReader and VBWriter simulate Braille flashcards, and VBHangman and VBGhost incorporate Braille character identification and recall into word games. We evaluated BraillePlay with a longitudinal study in the wild with eight blind children. Through logged usage data and extensive interviews, we found that all but one participant were able to play the games independently and found them enjoyable. We also found evidence that some children learned Braille concepts. We distill implications for the design of games for blind children and discuss lessons learned.

Much recent work has explored the challenge of nonvisual text entry on mobile devices. While researchers have attempted to solve this problem with gestures, we explore a different modality: speech. We conducted a survey with 169 blind and sighted participants to investigate how often, what for, and why blind people used speech for input on their mobile devices. We found that blind people used speech more often and input longer messages than sighted people. We then conducted a study with 8 blind people to observe how they used speech input on an iPod compared with the on-screen keyboard with VoiceOver. We found that speech was nearly 5 times as fast as the keyboard. While participants were mostly satisfied with speech input, editing recognition errors was frustrating. Participants spent an average of 80.3% of their time editing. Finally, we propose challenges for future work, including more efficient eyes-free editing and better error detection methods for reviewing text.

Low-vision and blind bus riders often rely on known physical landmarks to help locate and verify bus stop locations (e.g., by searching for a shelter, bench, newspaper bin). However, there are currently few, if any, methods to determine this information a priori via computational tools or services. In this paper, we introduce and evaluate a new scalable method for collecting bus stop location and landmark descriptions by combining online crowdsourcing and Google Street View (GSV). We conduct and report on three studies in particular: (i) a formative interview study of 18 people with visual impairments to inform the design of our crowdsourcing tool; (ii) a comparative study examining differences between physical bus stop audit data and audits conducted virtually with GSV; and (iii) an online study of 153 crowd workers on Amazon Mechanical Turk to examine the feasibility of crowdsourcing bus stop audits using our custom tool with GSV. Our findings reemphasize the importance of landmarks in non-visual navigation, demonstrate that GSV is a viable bus stop audit dataset, and show that minimally trained crowd workers can find and identify bus stop landmarks with 82.5% accuracy across 150 bus stop locations (87.3% with simple quality control).

The time and labor demanded by a typical laboratory-based keyboard evaluation are limiting resources for algorithmic adjustment and optimization. We propose Remulation, a complementary method for evaluating touchscreen keyboard correction and recognition algorithms. It replicates prior user study data through real-time, on-device simulation. We have developed Octopus, a Remulation-based evaluation tool that enables keyboard developers to efficiently measure and inspect the impact of algorithmic changes without conducting resource-intensive user studies. It can also be used to evaluate third-party keyboards in a "black box" fashion, without access to their algorithms or source code. Octopus can evaluate both touch keyboards and word-gesture keyboards. Two empirical examples show that Remulation can efficiently and effectively measure many aspects of touch screen keyboards at both macro and micro levels. Additionally, we contribute two new metrics to measure keyboard accuracy at the word level: the Ratio of Error Reduction (RER) and the Word Score.

Eyes-free input usually relies on audio feedback that can be difficult to hear in noisy environments. We present DigiTaps, an eyes-free number entry method for touchscreen devices that requires little auditory attention. To enter a digit, users tap or swipe anywhere on the screen with one, two, or three fingers. The 10 digits are encoded by combinations of these gestures that relate to the digits' semantics. For example, the digit 2 is input with a 2-finger tap. We conducted a longitudinal evaluation with 16 people and found that DigiTaps with no audio feedback was faster but less accurate than with audio feedback after every input. Throughout the study, participants entered numbers with no audio feedback at an average rate of 0.87 characters per second, with an uncorrected error rate of 5.63%.

Blind mobile device users face security risks such as inaccessible authentication methods, and aural and visual eavesdropping. We interviewed 13 blind smartphone users and found that most participants were unaware of or not concerned about potential security threats. Not a single participant used optional authentication methods such as a password-protected screen lock. We addressed the high risk of unauthorized user access by developing PassChords, a non-visual authentication method for touch surfaces that is robust to aural and visual eavesdropping. A user enters a PassChord by tapping several times on a touch surface with one or more fingers. The set of fingers used in each tap defines the password. We give preliminary evidence that a four-tap PassChord has about the same entropy, a measure of password strength, as a four-digit personal identification number (PIN) used in the iPhone's Passcode Lock. We conducted a study with 16 blind participants that showed that PassChords were nearly three times as fast as iPhone's Passcode Lock with VoiceOver, suggesting that PassChords are a viable accessible authentication method for touch screens.

A new non-visual method of numeric entry into a smartphone is designed, implemented, and tested. Users tap the smartphone screen with one to three fingers or swipe the screen in order to enter numbers. No buttons are used--only simple, easy-to-remember gestures. A preliminary valuation with sighted users compares the method to a standard accessible numeric keyboard with a VoiceOver-like screen reader interface for non-visual entry. We found that users entered numbers faster and with higher accuracy with our number entry method than with a VoiceOver-like interface, showing there is potential for use among blind people as well. The Tapulator, a complete calculator based on this non-visual numeric entry that uses simple gestures for arithmetic operations and other calculator actions is described.

Text entry on smartphones is far slower and more error-prone than on traditional desktop keyboards, despite sophisticated detection and auto-correct algorithms. To strengthen the empirical and modeling foundation of smartphone text input improvements, we explore touch behavior on soft QWERTY keyboards when used with two thumbs, an index finger, and one thumb. We collected text entry data from 32 participants in a lab study and describe touch accuracy and precision for different keys. We found that distinct patterns exist for input among the three hand postures, suggesting that keyboards should adapt to different postures. We also discovered that participants' touch precision was relatively high given typical key dimensions, but there were pronounced and consistent touch offsets that can be leveraged by keyboard algorithms to correct errors. We identify patterns in our empirical findings and discuss implications for design and improvements of soft keyboards.

We explore using vibration on a smartphone to provide turn-by-turn walking instructions to people with visual impairments. We present two novel feedback methods called Wand and ScreenEdge and compare them to a third method called Pattern. We built a prototype and conducted a user study where 8 participants walked along a pre-programmed route using the 3 vibration feedback methods and no audio output. Participants interpreted the feedback with an average error rate of just 4 percent. Most preferred the Pattern method, where patterns of vibrations indicate different directions, or the ScreenEdge method, where areas of the screen correspond to directions and touching them may induce vibration.

Blind and deaf-blind people often rely on public transit for everyday mobility, but using transit can be challenging for them. We conducted semi-structured interviews with 13 blind and deaf-blind people to understand how they use public transit and what human values were important to them in this domain. Two key values were identified: independence and safety. We developed GoBraille, two related Braille-based applications that provide information about buses and bus stops while supporting the key values. GoBraille is built on MoBraille, a novel framework that enables a Braille display to benefit from many features in a smartphone without knowledge of proprietary, device-specific protocols. Finally, we conducted user studies with blind people to demonstrate that GoBraille enables people to travel more independently and safely. We also conducted co-design with a deaf-blind person, finding that a minimalist interface, with short input and output messages, was most effective for this population.

The Middle East Education Through Technology (MEET) program is a non-profit organization based in Jerusalem, that aims to empower future Israeli and Palestinian leaders by teaching them computer science and business. From the perspective of MEET's instructors, this paper describes how MEET uses computer science education to foster professional and personal contact among Israeli and Palestinian high school students, two groups who otherwise would have little or no interaction with each other. MEET's primary method of overcoming the barrier is teamwork: students are divided into groups that include both Israelis and Palestinians and are assigned software engineering tasks. We believe that the techniques used by MEET can serve as examples for other computer science programs that overcome barriers between groups in the United States and other countries.

We conducted interviews with blind and deaf-blind people to understand how they use the public transit system. In this paper, we discuss key challenges our participants faced and present a tool we developed to alleviate these challenges. We built this tool on MoBraille, a novel framework that enables a Braille display to benefit from many features in an Android phone without knowledge of proprietary, device-specific protocols. We conducted participatory design with a deaf-blind person and describe the lessons learned about designing an interface for a deaf-blind person.

Smart phones typically support a range of GPS-enabled navigation services. However, most navigation services on smart phones are of limited use to people with visual disabilities. In this paper, we present iWalk, a speech-enabled local search and navigation prototype for people with low vision. iWalk runs on smart phones. It supports speech input, and provides real-time turn-by-turn walking directions in speech and text, using distances and time-to-turn information in addition to street names so that users are not forced to read street signs. In between turns iWalk uses non-speech cues to indicate to the user that s/he is 'on-track'.