VLDB 2026 Research / reviewers in the wild / expert
Shaun K. Kane
dblp:63/6371
· DBLP profile ↗
78ranked-venue papers
17as first author
17since 2021 · last 2026
0000-0002-0276-9417ORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Human-computer interaction and ubiquitous computing · 78 · 17 first-author · 17 since 2021Artificial intelligence and machine learning · 1Graphics, computer vision, multimedia, augmented reality and games · 1
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | ADCanvas: Accessible and Conversational Audio Description Authoring for Blind and Low Vision CreatorsabstractAudio Description (AD) provides essential access to visual media for blind and low vision (BLV) audiences. Yet current AD production tools remain largely inaccessible to BLV video creators, who possess valuable expertise but face barriers due to visually-driven interfaces. We present ADCanvas, a multimodal authoring system that supports non-visual control over audio description (AD) creation. ADCanvas combines conversational interaction with keyboard-based playback control and a plain-text, screen reader–accessible editor to support end-to-end AD authoring and visual question answering (VQA). Combining screen-reader-friendly controls with a multimodal LLM agent, ADCanvas supports live VQA, script generation, and AD modification. Through a user study with 12 BLV video creators, we find that users adopt the conversational agent as an informational aide and drafting assistant, while maintaining agency through verification and editing. For example, participants saw themselves as curators who received information from the model and filtered it down for their audience. Our findings offer design implications for accessible media tools, including precise editing controls, accessibility support for creative ideation, and configurable rules for human-AI collaboration. Franklin Mingzhe Li, Michael Xieyang Liu, Cynthia L. Bennett, Shaun K. Kane |
CHI | 4 |
| 2026 | Ability Heuristics for Conducting Accessibility InspectionsabstractThe accessibility of interactive technologies is often evaluated using checklists that are low-level, numerous, and platform-specific. Such checklists are typically used by accessibility experts, leaving everyday designers and developers with little support for assessing their own interfaces. To make accessibility evaluations easier to conduct, we devised a set of nine “ability heuristics” that prompt designers to engage with accessibility throughout the design process. We empirically evaluated these ability heuristics with 37 design students, comparing them to usability heuristics and WCAG. The ability heuristics emphasized the quality of accessibility features compared to the other methods, and surfaced issues that were more broadly dispersed across disability groups. Further, the students found the heuristics were as easy to use as the alternative methods. We argue that the heuristics help to move beyond binary notions of accessibility, pushing designers to consider the quality of features across diverse disabilities and the range of abilities within. Claire L. Mitchell, Junhan Kong, Jesse J. Martinez, Shaun K. Kane, Amy J. Ko, Alexis Hiniker, Jacob O. Wobbrock |
CHI | 4 |
| 2025 | Making Street View Accessible Using Context-Aware, Multimodal AI: A Demo of StreetReaderAI
Jon Froehlich, Alexander Fiannaca, Nimer Jaber, Victor Tsaran, Shaun K. Kane |
ASSETS | 5 |
| 2025 | StreetViewAI: Making Street View Accessible Using Context-Aware Multimodal AI
Jon Froehlich, Alexander Fiannaca, Nimer Jaber, Victor Tsaran, Shaun K. Kane |
UIST | 5 |
| 2024 | Painting with Cameras and Drawing with Text: AI Use in Accessible CreativityabstractGenerative AI (GAI) is proliferating, and among its many applications are to support creative work (e.g., generating text, images, music) and to enhance accessibility (e.g., captions of images and audio). As GAI evolves, creatives must consider how (or how not) to incorporate these tools into their practices. In this paper, we present interviews at the intersection of these applications. We learned from 10 creatives with disabilities who intentionally use and do not use GAI in and around their creative work. Their mediums ranged from audio engineering to leatherwork, and they collectively experienced a variety of disabilities, from sensory to motor to invisible disabilities. We share cross-cutting themes of their access hacks, how creative practice and access work become entangled, and their perspectives on how GAI should and should not fit into their workflows. In turn, we offer qualities of accessible creativity with responsible AI that can inform future research. Cynthia L. Bennett, Renee Shelby, Negar Rostamzadeh, Shaun K. Kane |
ASSETS | 4 |
| 2024 | "It's an independent living skill, but covered with fun!": Prompting At-Home Skill Development for Children with Vision ImpairmentabstractThis study examined how to design tools that build independence with Blind or Visually Impaired (BVI) children and their families. Beyond core academics, BVI children require instruction on independent living skills, with their curriculum necessitating parent-school cooperation to support continued education at home. However, most technology for BVI children focus on academics, spatial orientation, and physical mobility. In this work, we aim to design a tool for independence that aligns with existing familial structures and activities. Through interviews and diary studies with five families, we explored development practices parents used with their BVI children, parent-teacher relationships, and how a prompting and reflection tool supported family goals. This study highlights home routines and independence skills that benefit from customized prompting, how activity prompts can encourage parents to scale back their assistance and propel independence, and how reflection builds optimism and empowers parents in the learning process. Vinitha Gadiraju, Lucia Jayne, Shaun K. Kane |
ASSETS | 3 |
| 2024 | A Recipe for Success? Exploring Strategies for Improving Non-Visual Access to Cooking InstructionsabstractCooking is an essential activity that enhances quality of life by enabling individuals to prepare their own meals. However, cooking often requires multitasking between cooking tasks and following instructions, which can be challenging to cooks with vision impairments if recipes or other instructions are inaccessible. To explore the practices and challenges of recipe access while cooking, we conducted semi-structured interviews with 20 people with vision impairments who have cooking experience and four cooking instructors at a vision rehabilitation center. We also asked participants to edit and give feedback on existing recipes. We revealed unique practices and challenges to accessing recipe information at different cooking stages, such as the heavy burden of hand-washing to interact with recipe readers. We also presented the preferred information representation and structure of recipes. We then highlighted design features of technological supports that could facilitate the development of more accessible kitchen technologies for recipe access. Our work contributes nuanced insights and design guidelines to enhance recipe accessibility for people with vision impairments. Franklin Mingzhe Li, Ashley Wang, Patrick Carrington, Shaun K. Kane |
ASSETS | 4 |
| 2024 | From Provenance to Aberrations: Image Creator and Screen Reader User Perspectives on Alt Text for AI-Generated ImagesabstractAI-generated images are proliferating as a new visual medium. However, state-of-the-art image generation models do not output alternative (alt) text with their images, rendering them largely inaccessible to screen reader users (SRUs). Moreover, less is known about what information would be most desirable to SRUs in this new medium. To address this, we invited AI image creators and SRUs to evaluate alt text prepared from various sources and write their own alt text for AI images. Our mixed-methods analysis makes three contributions. First, we highlight creators’ perspectives on alt text, as creators are well-positioned to write descriptions of their images. Second, we illustrate SRUs’ alt text needs particular to the emerging medium of AI images. Finally, we discuss the promises and pitfalls of utilizing text prompts written as input for AI models in alt text generation, and areas where broader digital accessibility guidelines could expand to account for AI images. Maitraye Das, Alexander Fiannaca, Meredith Ringel Morris, Shaun K. Kane, Cynthia L. Bennett |
CHI | 4 |
| 2024 | A Contextual Inquiry of People with Vision Impairments in CookingabstractIndividuals with vision impairments employ a variety of strategies for object identification, such as pans or soy sauce, in the culinary process. In addition, they often rely on contextual details about objects, such as location, orientation, and current status, to autonomously execute cooking activities. To understand how people with vision impairments collect and use the contextual information of objects while cooking, we conducted a contextual inquiry study with 12 participants in their own kitchens. This research aims to analyze object interaction dynamics in culinary practices to enhance assistive vision technologies for visually impaired cooks. We outline eight different types of contextual information and the strategies that blind cooks currently use to access the information while preparing meals. Further, we discuss preferences for communicating contextual information about kitchen objects as well as considerations for the deployment of AI-powered assistive technologies. Franklin Mingzhe Li, Michael Xieyang Liu, Shaun K. Kane, Patrick Carrington |
CHI | 3 |
| 2024 | "They only care to show us the wheelchair": disability representation in text-to-image AI modelsabstractThis paper reports on disability representation in images output from text-to-image (T2I) generative AI systems. Through eight focus groups with 25 people with disabilities, we found that models repeatedly presented reductive archetypes for different disabilities. Often these representations reflected broader societal stereotypes and biases, which our participants were concerned to see reproduced through T2I. Our participants discussed further challenges with using these models including the current reliance on prompt engineering to reach satisfactorily diverse results. Finally, they offered suggestions for how to improve disability representation with solutions like showing multiple, heterogeneous images for a single prompt and including the prompt with images generated. Our discussion reflects on tensions and tradeoffs we found among the diverse perspectives shared to inform future research on representation-oriented generative AI system evaluation metrics and development processes. Kelly Mack, Rida Qadri, Remi Denton, Shaun K. Kane, Cynthia L. Bennett |
CHI | 4 |
| 2023 | "I just thought it was me": How Smartphones Fail Users with Mild-to-Moderate Dexterity DifferencesabstractAccessibility solutions often focus on the experiences of people with more severe disabilities, such as those who are unable to perform certain tasks unassisted. However, disability exists on a spectrum, and people with moderate disabilities may be overlooked when studying or sampling for differences. As a result, these individuals and their needs are excluded from relevant research. In this study, we interviewed 12 adults with mild-to-moderate dexterity impairments about their experiences using smartphones and other mobile devices. While our participants frequently experienced accessibility challenges, they struggled to know where to find help, in part because of discomfort with traditional labels of disability and accessibility. We found four key themes: (1) There were large gaps in available and usable accessibility tools for this population, (2) Users were unlikely to seek out accessibility features due to complex disability identity, (3) Contextual concerns impacted mobile device use, and (4) Users relied on self-created adaptations and modifications to improve usability. We suggest that individuals with mild-to-moderate dexterity challenges are a unique cohort that would benefit from further consideration from the accessibility community and accessibility features that support their needs. Molly Bowman, Jerry Robinson, Erin Buehler, Shaun K. Kane |
ASSETS | 4 |
| 2023 | "The less I type, the better": How AI Language Models can Enhance or Impede Communication for AAC UsersabstractUsers of augmentative and alternative communication (AAC) devices sometimes find it difficult to communicate in real time with others due to the time it takes to compose messages. AI technologies such as large language models (LLMs) provide an opportunity to support AAC users by improving the quality and variety of text suggestions. However, these technologies may fundamentally change how users interact with AAC devices as users transition from typing their own phrases to prompting and selecting AI-generated phrases. We conducted a study in which 12 AAC users tested live suggestions from a language model across three usage scenarios: extending short replies, answering biographical questions, and requesting assistance. Our study participants believed that AI-generated phrases could save time, physical and cognitive effort when communicating, but felt it was important that these phrases reflect their own communication style and preferences. This work identifies opportunities and challenges for future AI-enhanced AAC devices. Stephanie Valencia, Richard Cave, Krystal Kallarackal, Katie Seaver, Michael Terry, Shaun K. Kane |
CHI | 6 |
| 2023 | Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI ChallengesabstractTechnology companies continue to invest in efforts to incorporate responsibility in their Artificial Intelligence (AI) advancements, while efforts to audit and regulate AI systems expand. This shift towards Responsible AI (RAI) in the tech industry necessitates new practices and adaptations to roles—undertaken by a variety of practitioners in more or less formal positions, many of whom focus on the user-centered aspects of AI. To better understand practices at the intersection of user experience (UX) and RAI, we conducted an interview study with industrial UX practitioners and RAI subject matter experts, both of whom are actively involved in addressing RAI concerns throughout the early design and development of new AI-based prototypes, demos, and products, at a large technology company. Many of the specific practices and their associated challenges have yet to be surfaced in the literature, and distilling them offers a critical view into how practitioners’ roles are adapting to meet present-day RAI challenges. We present and discuss three emerging practices in which RAI is being enacted and reified in UX practitioners’ everyday work. We conclude by arguing that the emerging practices, goals, and types of expertise that surfaced in our study point to an evolution in praxis, with associated challenges that suggest important areas for further research in HCI. Qiaosi Wang, Michael A. Madaio, Shaun K. Kane, Shivani Kapania, Michael Terry, Lauren Wilcox |
CHI | 3 |
| 2022 | Knitting Access: Exploring Stateful Textiles with People with DisabilitiesabstractWearables are often a primary means of collecting data on the body and in-situ. The data collected upon wearables can shape or record interactions in real time, prompting practices like self-care and reflection. In this work, we became intrigued by textile structures that were non-digital but in themselves “stateful”. We explored how these textile interfaces can fit meaningfully into the lives of people with disabilities as sensors and display. Our study revealed interesting practices that emerged for self-tracking that were qualitatively unique in their close relationship to the body and deeply physical modes of engagement. Our findings offer insights into (1) qualities of textile interfaces that are important to people with disabilities, (2) new forms of data that people found to be worthwhile in tracking, and (3) knitted interfaces for sensing and display. Annika Muehlbradt, Gregory Whiting, Shaun K. Kane, Laura Devendorf |
Conference on Designing Interactive Systems | 3 |
| 2022 | Towards augmented reality coaching for daily routines: Participatory design with individuals with cognitive disabilities and their caregivers
Varsha Koushik, Shaun K. Kane |
Int. J. Hum. Comput. Stud. | 2 |
| 2021 | "It is fascinating to make these beasts fly": Understanding Visually Impaired People's Motivations and Needs for Drone PilotingabstractDrones have become fixtures in commerce, safety efforts, and in homes as a leisure activity. Researchers have started to explore how drones can support people with disabilities in piloting and serve as assistive devices. Our work focuses on people with vision impairment and investigates what motivates them to fly drones. We administered a survey to visually impaired adults that gauged general interest in drone piloting and previous experience with drones. From the 59 survey responses, we interviewed 13 participants to elaborate on how they envision using drones and how different feedback and modes of piloting can make the flying experience more accessible. We found that our participants had overarching interests in aviation, trying new technology, environment exploration, and finding collaborative activities to do with their sighted family members, which extended to an interest in piloting drones. This research helps lay groundwork for design scenarios and accessible features for future drones. Vinitha Gadiraju, Jérémie Garcia, Shaun K. Kane, Anke M. Brock |
ASSETS | 3 |
| 2021 | Exploring Technology Design for Students with Vision Impairment in the Classroom and RemotelyabstractTeachers of the Visually Impaired (TVIs) teach academic and functional living skills simultaneously to prepare students with vision impairment to be successful and independent. Current educational tools primarily focus on academic instruction rather than this multifaceted approach needed for students. Our work aims to understand how technology can integrate behavioral skills, like independence, and support TVIs in their preferred teaching strategy. We observed elementary classrooms at a school for the blind for six weeks to study how educators design lessons and use technology to supplement their instruction in different subjects. After the observational study, we conducted remote interviews with educators to understand how technology can support students in building academic and behavioral skills in-person and remotely. Educators suggested incorporating audio feedback that motivates students to play and learn consistently, student progress tracking for parents and educators, and designing features that help students build independence and develop collaborative skills. Vinitha Gadiraju, Olwyn Doyle, Shaun K. Kane |
CHI | 3 |
| 2020 | Sense and Accessibility: Understanding People with Physical Disabilities' Experiences with Sensing SystemsabstractSensing technologies that implicitly and explicitly mediate digital experiences are an increasingly pervasive part of daily living; it is vital to ensure that these technologies work appropriately for people with physical disabilities. We conducted on online survey with 40 adults with physical disabilities, gathering open-ended descriptions about respondents’ experiences with a variety of sensing systems, including motion sensors, biometric sensors, speech input, as well as touch and gesture systems. We present findings regarding the many challenges status quo sensing systems present for people with physical disabilities, as well as the ways in which our participants responded to these challenges. We conclude by reflecting on the significance of these findings for defining a future research agenda for creating more inclusive sensing systems. Shaun K. Kane, Anhong Guo, Meredith Ringel Morris |
ASSETS | 1 |
| 2020 | "I just went into it assuming that I wouldn't be able to have the full experience": Understanding the Accessibility of Virtual Reality for People with Limited MobilityabstractVirtual reality (VR) has the potential to transform many aspects of our daily lives, including work, entertainment, communication, and education. However, there has been little research into understanding the usability of VR for people with mobility limitations. In this paper, we present the results of an exploration to understand the accessibility of VR for people with limited mobility. We conducted semi-structured interviews with 16 people with limited mobility about their thoughts on, and experiences with, VR systems. We identified 7 barriers related to the physical accessibility of VR devices that people with limited mobility might encounter, ranging from the initial setup of a VR system to keeping VR controllers in view of cameras embedded in VR headsets. We also elicited potential improvements to VR systems that would address some accessibility concerns. Based on our findings, we discuss the importance of considering the abilities of people with limited mobility when designing VR systems, as the abilities of many participants did not match the assumptions embedded in the design of current VR systems. Martez E. Mott, John C. Tang, Shaun K. Kane, Edward Cutrell, Meredith Ringel Morris |
ASSETS | 3 |
| 2020 | BrailleBlocks: Computational Braille Toys for Collaborative LearningabstractBraille literacy has fallen in recent years, and many blind children now grow up without learning Braille. However, learning Braille can increase employment chances and improve literacy skills. We introduce BrailleBlocks, a system to help visually impaired children learn and practice Braille alongside a sighted parent. BrailleBlocks comprises a set of tangible blocks and pegs, each block representing a Braille cell, and an associated application with games. The system automatically tracks and recognizes the blocks so that parents can follow along even if they cannot read Braille. We conducted a user study to test BrailleBlocks with five families, with five parents and six visually impaired children. The contributions of this work are a novel approach to Braille education toys, observations of how visually impaired children and sighted parents used this system together, their insights on current issues with Braille educational tools, and actionable feedback for future Braille-based learning tools. Vinitha Gadiraju, Annika Muehlbradt, Shaun K. Kane |
CHI | 3 |
| 2019 | Youth Learning Machine Learning through Building Models of Athletic MovesabstractMachine Learning-based (ML) technologies impact many facets of our lives. Given ML's ubiquity, and the ways it offers creative computational possibilities distinct from programming, we believe it could be a powerful tool for youth to leverage in making, creativity, and play. We investigate how youth with no programming experience can incorporate ML classifiers into athletic practice by building models of their own physical activity. In this paper, we describe a design experiment exploring how to introduce youth to making ML models within the context of their athletic interests. We present AlpacaML, an iOS application that connects to wearable sensors and allows young people to model physical movement using an ML classifier, and detail its use in a three-hour workshop with middle- and high-school athletes. We found the youth were able to collect data, build models, test and evaluate models, and quickly iterate on this process. We finish with a discussion of why this is a promising direction for the incorporation of Machine Learning into novice youth making, exploration, and play. Abigail Zimmermann-Niefield, Makenna Turner, Bridget Murphy, Shaun K. Kane, R. Benjamin Shapiro |
IDC | 4 |
| 2019 | RoboGraphics: Dynamic Tactile Graphics Powered by Mobile RobotsabstractTactile graphics are a common way to present information to people with vision impairments. Tactile graphics can be used to explore a broad range of static visual content but aren't well suited to representing animation or interactivity. We introduce a new approach to creating dynamic tactile graphics that combines a touch screen tablet, static tactile overlays, and small mobile robots. We introduce a prototype system called RoboGraphics and several proof-of-concept applications. We evaluated our prototype with seven participants with varying levels of vision, comparing the RoboGraphics approach to a flat screen, audio-tactile interface. Our results show that dynamic tactile graphics can help visually impaired participants explore data quickly and accurately. Darren Guinness, Annika Muehlbradt, Daniel Szafir, Shaun K. Kane |
ASSETS | 4 |
| 2019 | RoboGraphics: Using Mobile Robots to Create Dynamic Tactile GraphicsabstractTactile graphics are a common way to present information to people with vision impairments. Tactile graphics can be used to explore a broad range of content including presenting data, telling stories, and conveying map information, but aren't well suited to representing dynamic or moving information. We introduce and demonstrate a new approach RoboGraphics for creating dynamic tactile graphics by combining static tactile overlays, touch screen tablets, and off-the-shelf tangible robots. To evaluate the RoboGraphics approach for conveying information to blind users, we created a set of reference applications, including an interface for exploring graph data, braille characters, the story of the tortoise and the hare, the analog clock, and the ruminant digestive process. A design probe demonstrated that haptic graphics can be used to help people with vision impairments explore data using the audio-tactile display. Darren Guinness, Annika Muehlbradt, Daniel Szafir, Shaun K. Kane |
ASSETS | 4 |
| 2019 | StoryBlocks: A Tangible Programming Game To Create Accessible Audio StoriesabstractBlock-based programming languages can support novice programmers through features such as simplified code syntax and user-friendly libraries. However, most block-based programming languages are highly visual, which makes them inaccessible to blind and visually impaired students. To address the inaccessibility of block-based languages, we introduce StoryBlocks, a tangible block-based game that enables blind programmers to learn basic programming concepts by creating audio stories. In this paper, we document the design of StoryBlocks and report on a series of design activities with groups of teachers, Braille experts, and students. Participants in our design sessions worked together to create accessible stories, and their feedback offers insights for the future development of accessible, tangible programming tools. Varsha Koushik, Darren Guinness, Shaun K. Kane |
CHI | 3 |
| 2019 | "It Broadens My Mind": Empowering People with Cognitive Disabilities through Computing EducationabstractComputer science education is widely viewed as a path to empowerment for young people, potentially leading to higher education, careers, and development of computational thinking skills. However, few resources exist for people with cognitive disabilities to learn computer science. In this paper, we document our observations of a successful program in which young adults with cognitive disabilities are trained in computing concepts. Through field observations and interviews, we identify instructional strategies used by this group, accessibility challenges encountered by this group, and how instructors and students leverage peer learning to support technical education. Our findings lead to guidelines for developing tools and curricula to support young adults with cognitive disabilities in learning computer science. Varsha Koushik, Shaun K. Kane |
CHI | 2 |
| 2019 | EyeDescribe: Combining Eye Gaze and Speech to Automatically Create Accessible Touch Screen ArtworkabstractMany images on the Web, including photographs and artistic images, feature spatial relationships between objects that are inaccessible to someone who is blind or visually impaired even when a text description is provided. While some tools exist to manually create accessible image descriptions, this work is time consuming and requires specialized tools. We introduce an approach that automatically creates spatially registered image labels based on how a sighted person naturally interacts with the image. Our system collects behavioral data from sighted viewers of an image, specifically eye gaze data and spoken descriptions, and uses them to generate a spatially indexed accessible image that can then be explored using an audio-based touch screen application. We describe our approach to assigning text labels to locations in an image based on eye gaze. We then report on two formative studies with blind users testing EyeDescribe. Our approach resulted in correct labels for all objects in our image set. Participants were able to better recall the location of objects when given both object labels and spatial locations. This approach provides a new method for creating accessible images with minimum required effort. Kyle Reinholt, Darren Guinness, Shaun K. Kane |
ISS | 3 |
| 2018 | Bonk: accessible programming for accessible audio gamesabstractIntroductory computer programming presents a number of challenges for blind and visually impaired screen reader users. In addition to the challenges of navigating complex code documents using a screen reader, novice programmers who are blind are often unable to experience fun coding projects such as programming games or animations. To address these accessibility barriers, we developed Bonk, an accessible programming environment that enables the creation of interactive audio games using a subset of the JavaScript programming language. Bonk enables novice programmers to create, share, play, and remix accessible audio games. In this paper, we introduce the Bonk programming toolkit and describe its use in a week-long programming workshop with blind and visually impaired high school students. Students in the workshop were able to create and share original audio games using Bonk, and expressed enthusiasm about furthering their programming knowledge. Shaun K. Kane, Varsha Koushik, Annika Muehlbradt |
IDC | 1 |
| 2018 | Exploring the Design of Audio-Kinetic Graphics for EducationabstractCreating tactile representations of visual information, especially moving images, is difficult due to a lack of available tactile computing technology and a lack of tools for authoring tactile information. To address these limitations, we developed a software framework that enables educators and other subject experts to create graphical representations that combine audio descriptions with kinetic motion. These audio-kinetic graphics can be played back using off-the-shelf computer hardware. We report on a study in which 10 educators developed content using our framework, and in which 18 people with vision impairments viewed these graphics on our output device. Our findings provide insights on how to translate knowledge of visual information to non-visual formats. Annika Muehlbradt, Madhur Atreya, Darren Guinness, Shaun K. Kane |
ICMI | 4 |
| 2018 | The Haptic Video Player: Using Mobile Robots to Create Tangible Video AnnotationsabstractVideo and animation are common ways of delivering concepts that cannot be easily communicated through text. This visual information is often inaccessible to blind and visually impaired people, and alternative representations such as Braille and audio may leave out important details. Audio-haptic displays with along with supplemental descriptions allow for the presentation of complex spatial information, along with accompanying description. We introduce the Haptic Video Player, a system for authoring and presenting audio-haptic content from videos. The Haptic Video Player presents video using mobile robots that can be touched as they move over a touch screen. We describe the design of the Haptic Video Player system, and present user studies with educators and blind individuals that demonstrate the ability of this system to render dynamic visual content non-visually. Darren Guinness, Annika Muehlbradt, Daniel Szafir, Shaun K. Kane |
ISS | 4 |
| 2018 | Supporting Remote Real-Time Expert Help: Opportunities and Challenges for Novice 3D ModelersabstractWe investigate how novice 3D modelers can remotely leverage real-time expert help to aid their learning tasks. We first carried out an observational study of remote novice-expert pairs of 3D modelers to understand traditional chat-based assistance in the context of learning 3D modeling. Next, we designed MarmalAid, a web-based 3D modeling tool with a novel real-time, in-context help feature that allows users to embed real-time chat conversations at any location within the 3D geometry of their models. Our user study with 12 novices who used both MarmalAid's real-time, in-context chat and an external chat tool to seek help, showed that novices found the real-time, in-context chat to be more useful and easier to use, and that experts asked for fewer clarifications, allowing the novices to ask more task-related questions. Our findings suggest to several design opportunities to utilize and extend the real-time, in-con-text heln concept in 3D modeline aoolications and bevond. Parmit K. Chilana, Nathaniel Hudson 0002, Srinjita Bhaduri, Prashant Shashikumar, Shaun K. Kane |
VL/HCC | 5 |
| 2018 | Interacting with Distant Objects in Augmented RealityabstractAugmented reality (AR) applications can leverage the full space of an environment to create immersive experiences. However, most empirical studies of interaction in AR focus on interactions with objects close to the user, generally within arms reach. As objects move farther away, the efficacy and usability of different interaction modalities may change. This work explores AR interactions at a distance, measuring how applications may support fluid, efficient, and intuitive interactive experiences in room-scale augmented reality. We conducted an empirical study (N = 20) to measure trade-offs between three interaction modalities-multimodal voice, embodied freehand gesture, and handhelds devices-for selecting, rotating, and translating objects at distances ranging from 8 to 16 feet (2.4m-4.9m). Though participants performed comparably with embodied freehand gestures and handheld remotes, they perceived embodied gestures as significantly more efficient and usable than device-mediated interactions. Our findings offer considerations for designing efficient and intuitive interactions in room-scale AR applications. Matt Whitlock, Ethan Harnner, Jed R. Brubaker, Shaun K. Kane, Danielle Albers Szafir |
VR | 4 |
| 2017 | GUI Robots: Using Off-the-Shelf Robots as Tangible Input and Output Devices for Unmodified GUI ApplicationsabstractTraditional GUI applications provide limited support for tangible interaction, as most applications are not programmed to support tangible input, and most input devices do not provide haptic feedback. To address this limitation, we introduce GUI Robots, a software framework that enables developers to repurpose off-the-shelf robots as tangible input and haptic output devices, and to connect them to unmodified desktop applications. We introduce the GUI Robots framework and present several proof-of-concept applications, including a haptic scroll wheel, force feedback game controllers, a 3D mouse, and a self-driving notification robot. To evaluate whether GUI Robots can be used to prototype tangible interfaces for existing applications, we conducted a user study in which developers created customized tangible interfaces for two applications. Study participants were able to create tangible user interfaces for these applications in less than an hour. GUI Robots allows developers to easily extend applications with tangible input and haptic output. Darren Guinness, Daniel Szafir, Shaun K. Kane |
Conference on Designing Interactive Systems | 3 |
| 2017 | Let's Talk About X: Combining Image Recognition and Eye Gaze to Support Conversation for People with ALSabstractCommunicating at a natural speed is a significant challenge for users of augmentative and alternative communication (AAC) devices, especially when input is provided by eye gaze, as is common for people with ALS and similar conditions. One way to improve AAC throughput is by drawing on contextual information from the outside world. Toward this goal, we present SceneTalk, a prototype gaze-based AAC system that uses computer vision to identify objects in the user's field of view and suggests words and phrases related to the current scene. We conducted a formative evaluation of SceneTalk with six people with ALS, in which we evaluated their preference for user interface modes and output preferences. Participants agreed that integrating contextual awareness into their AAC device could be helpful across a diverse range of situations. Shaun K. Kane, Meredith Ringel Morris |
Conference on Designing Interactive Systems | 1 |
| 2017 | Tangibles + Programming + Audio Stories = FunabstractBlock-based programming languages enable novice programmers, including children, to learn the basics of programming. However, most block-based programming languages are not accessible to blind and visually impaired users because they rely upon visual drag-and-drop interaction, and because they typically create visual output. To improve access to block-based programming languages, we introduce Story Blocks, a programming toolkit that uses tangible blocks to represent story components, and which produces output in the form of accessible audio stories and games. Story Blocks provides an introductory programming environment that can be enjoyed by people of all abilities. Varsha Koushik, Shaun K. Kane |
ASSETS | 2 |
| 2017 | Goby: A Wearable Swimming Aid for Blind AthletesabstractWe introduce Goby, a swimming aid that provides audio feedback for blind and visually impaired athletes. Goby's activity tracker is worn on the thigh and uses a downward-facing camera to track the swimmer's position in the pool. Goby detects when the user is swimming outside the lane or approaching a wall, and warns the user via an audio notification. In this paper, we introduce the Goby prototype and share formative feedback about the prototype from blind and sighted swimmers. Annika Muehlbradt, Varsha Koushik, Shaun K. Kane |
ASSETS | 3 |
| 2017 | Fabrication Games: Using 3D Printers to Explore New Interactions for Tabletop GamesabstractPersonal fabrication technologies such as 3D printers are becoming increasingly affordable, enabling many to own and use 3D printers in their own homes. Yet we have little understanding of how fabrication tools and technologies can be used and appropriated within the home. In this paper, we explore the opportunities and challenges related to using personal fabrication technologies as part of play, specifically in the context of board and tabletop games. We present an overview of existing uses of 3D printers in the context of gaming, which has largely focused on creating and replacing pieces for existing games. Drawing on existing uses of 3D printing in games, and on prior research in interacting with fabrication tools, we then introduce a set of gameplay elements that use the affordances of the 3D printer to enhance and extend gameplay. We evaluated these gameplay elements through a focus group with 9 gaming hobbyists, who provided feedback on these elements and designed new games that used these elements. Our contributions include an extended set of gameplay elements that leverage fabrication tools, a set of reference games, and guidelines for augmenting existing fabrication tools to support playful interactions. Srinjita Bhaduri, Jesús G. Ortiz Tovar, Shaun K. Kane |
Creativity & Cognition | 3 |
| 2017 | Understanding the Role Fluidity of Stakeholders During Assistive Technology Research "In the Wild"abstractDeploying novel technologies requires the coordinated efforts of the research team, research participants, and a variety of community members and project stakeholders. To ensure that the project is completed successfully, these disparate groups of people engage in articulation work, which is the meta-work that supports the use of collaborative systems. In this paper, we examine the articulation work surrounding the deployment of systems that have found limited long-term adoption: assistive technology. Specifically, we examine three research deployments of a collaborative game for children with autism. Analysis of the articulation work performed during these studies demonstrates how research deployments of technologies create conditions in which stakeholders must take on additional roles to make the deployment work. By understanding the articulation work surrounding deployment studies engendered in this role fluidity, we can improve both research design and the analysis of data emergent from these studies. Louanne E. Boyd, Kyle Rector, Halley Profita, Abigale Stangl, Annuska Z. Perkins, Shaun K. Kane, Gillian R. Hayes |
CHI | 6 |
| 2017 | Toward Everyday Gaze Input: Accuracy and Precision of Eye Tracking and Implications for DesignabstractFor eye tracking to become a ubiquitous part of our everyday interaction with computers, we first need to understand its limitations outside rigorously controlled labs, and develop robust applications that can be used by a broad range of users and in various environments. Toward this end, we collected eye tracking data from 80 people in a calibration-style task, using two different trackers in two lighting conditions. We found that accuracy and precision can vary between users and targets more than six-fold, and report on differences between lighting, trackers, and screen regions. We show how such data can be used to determine appropriate target sizes and to optimize the parameters of commonly used filters. We conclude with design recommendations and examples how our findings and methodology can inform the design of error-aware adaptive applications. Anna Maria Feit, Shane Williams, Arturo Toledo, Ann Paradiso, Harish Kulkarni, Shaun K. Kane, Meredith Ringel Morris |
CHI | 6 |
| 2017 | "At times avuncular and cantankerous, with the reflexes of a mongoose": Understanding Self-Expression through Augmentative and Alternative Communication DevicesabstractAmyotrophic Lateral Sclerosis (ALS) is a disease that causes individuals to lose their ability to control their muscles, eventually leaving them unable to speak or write. People with ALS often transition to using an augmentative and alternative communication device (AAC), which requires both the AAC user and their conversation partners to adjust to new and different communication patterns. We conducted interviews with seven individuals with ALS and their partners, focusing on how AAC use has impacted their communication and personal expression. Our participants experienced a range of communication difficulties, including conversational pacing, personality expression, and interacting with unfamiliar conversational partners. Participants worked to adapt their communication behaviors to the AAC device, but still experienced challenges in expressing themselves, and sometimes felt compelled to withdraw from social interaction. By improving our understanding of how people transition to using AAC, we may inform improved designs for future AAC devices. Shaun K. Kane, Meredith Ringel Morris, Ann Paradiso, Jon Campbell |
CSCW | 1 |
| 2016 | Promoting Strategic Research on Inclusive Access to Rich Online Content and ServicesabstractHow can the broader field of computer science research be harnessed to address challenges and opportunities in accessibility? This poster summarizes the findings of a workshop, sponsored by the Computing Community Consortium (USA), that brought together computer scientists, representatives of disability advocacy organizations, people from industry, and government employees to develop an agenda for strategic research. Members of the ASSETS community may find the reports useful in organizing collaborative projects with others in the computer science community. Clayton H. Lewis, Shaun K. Kane, Richard E. Ladner |
ASSETS | 2 |
| 2016 | Nothing to Hide: Aesthetic Customization of Hearing Aids and Cochlear Implants in an Online CommunityabstractHearing aids and cochlear implants can improve accessibility and quality of life for people with hearing impairments. However, use of these devices may cause concern amongst some users due to sociocultural issues such as unwanted attention and perceived stigma. While some individuals may respond to these concerns by attempting to conceal their devices, or even abandoning their devices, others have responded by making their devices more visible through aesthetic customization, and some have begun to share these customizations online. In this paper, we describe community interactions in an online forum dedicated to customized hearing aids and cochlear implants. We found that community members discussed customization tools and techniques, shared their customizations, and provided each other with encouragement and support. Community members customized their devices as a means of self-expression that demonstrated the wearer's fashion sense, revealed favorite sports teams and characters, and marked holidays and personal milestones. Our findings may inform the design of assistive technologies that better support personalization, customization, and self-expression. Halley Profita, Abigale Stangl, Laura Matuszewska, Sigrunn Sky, Shaun K. Kane |
ASSETS | 5 |
| 2016 | "Holy Starches Batman!! We are Getting Walloped!": Crowdsourcing Comic Book TranscriptionsabstractComic books are among the most popular forms of popular media, but most comics are not provided in an accessible format. Creating an accessible transcript of a comic book may be more challenging than simply describing the images, as comics involve complex interplay between words and images, and often feature long-running and complex storylines. In this poster we describe a pilot study exploring the feasibility of crowdsourcing transcriptions of comic book pages. We recruited 60 crowd workers and asked them to transcribe a page of a comic book; half were told that the description was for a blind person, and half were not. We found that people who knew that they were transcribing for a blind person produced longer, more detailed descriptions. Our results also suggest that comic book knowledge may have at least some small impact on description detail. Christine Samson, Casey Fiesler, Shaun K. Kane |
ASSETS | 3 |
| 2016 | "With most of it being pictures now, I rarely use it": Understanding Twitter's Evolving Accessibility to Blind UsersabstractSocial media is an increasingly important part of modern life. We investigate the use of and usability of Twitter by blind users, via a combination of surveys of blind Twitter users, large-scale analysis of tweets from and Twitter profiles of blind and sighted users, and analysis of tweets containing embedded imagery. While Twitter has traditionally been thought of as the most accessible social media platform for blind users, Twitter's increasing integration of image content and users' diverse uses for images have presented emergent accessibility challenges. Our findings illuminate the importance of the ability to use social media for people who are blind, while also highlighting the many challenges such media currently present this user base, including difficulty in creating profiles, in awareness of available features and settings, in controlling revelations of one's disability status, and in dealing with the increasing pervasiveness of image-based content. We propose changes that Twitter and other social platforms should make to promote fuller access to users with visual impairments. Meredith Ringel Morris, Annuska Z. Perkins, Catherine Yao, Sina Bahram, Jeffrey P. Bigham, Shaun K. Kane |
CHI | 6 |
| 2016 | Smart Touch: Improving Touch Accuracy for People with Motor Impairments with Template MatchingabstractWe present two contributions toward improving the accessibility of touch screens for people with motor impairments. First, we provide an exploration of the touch behaviors of 10 people with motor impairments, e.g., we describe how touching with the back or sides of the hand, with multiple fingers, or with knuckles creates varied multi-point touches. Second, we introduce Smart Touch, a novel template-matching technique for touch input that maps any number of arbitrary contact-areas to a user's intended (x,y) target location. The result is that users with motor impairments can touch however their abilities allow, and Smart Touch will resolve their intended touch point. Smart Touch therefore allows users to touch targets in whichever ways are most comfortable and natural for them. In an experimental evaluation, we found that Smart Touch predicted (x,y) coordinates of the users' intended target locations over three times closer to the intended target than the native Land-on and Lift-off techniques reported by the built-in touch sensors found in the Microsoft PixelSense interactive tabletop. This result is an important step toward improving touch accuracy for people with motor impairments and others for whom touch screen operation was previously impossible. Martez E. Mott, Radu-Daniel Vatavu, Shaun K. Kane, Jacob O. Wobbrock |
CHI | 3 |
| 2016 | The AT Effect: How Disability Affects the Perceived Social Acceptability of Head-Mounted Display UseabstractWearable computing devices offer new possibilities to increase accessibility and independence for individuals with disabilities. However, the adoption of such devices may be influenced by social factors, and useful devices may not be adopted if they are considered inappropriate to use. While public policy may adapt to support accommodations for assistive technology, emerging technologies may be unfamiliar or unaccepted by bystanders. We surveyed 1200 individuals about the use of a head-mounted display in a public setting, examining how information about the user's disability affected judgments of the social acceptability of the scenario. Our findings reveal that observers considered head-mounted display use more socially acceptable if the device was being used to support a person with a disability. Halley Profita, Reem Albaghli, Leah Findlater, Paul T. Jaeger, Shaun K. Kane |
CHI | 5 |
| 2015 | The Invisible Work of Accessibility: How Blind Employees Manage Accessibility in Mixed-Ability WorkplacesabstractOver the past century, people who are blind and their allies have developed successful public policies and technologies in support of creating more accessible workplaces. However, simply creating accessible technologies does not guarantee that these will be available or adopted. Because much work occurs within shared workspaces, decisions about assistive technology use may be mediated by social interactions with, and expectations of, sighted coworkers. We present findings from a qualitative field study of five workplaces from the perspective of blind employees. Although all participants were effective employees, they expressed that working in a predominantly sighted office environment produces impediments to a blind person's independence and to their integration as an equal coworker. We describe strategies employed by our participants to create and maintain an accessible workplace and present suggestions for future technology that better supports blind workers as equal peers in the workplace. Stacy M. Branham, Shaun K. Kane |
ASSETS | 2 |
| 2015 | Collaborative Accessibility: How Blind and Sighted Companions Co-Create Accessible Home SpacesabstractIn recent decades, great technological strides have been made toward enabling people who are blind to live independent, successful lives. However, there has been relatively little progress towards understanding the social, collaborative needs of this population, particularly in the domestic setting. We conducted semi-structured interviews in the homes of 10 pairs of close companions in which one partner was blind and one was not. We found that partners engaged in collaborative accessibility by taking active roles in co-creating an accessible environment. Due to their different visual abilities, however, partners sometimes encountered difficulties managing divergent needs and engaging in shared experiences. We describe outstanding challenges to creating accessible shared home spaces and outline new research and technology opportunities for supporting collaborative accessibility in the home. Stacy M. Branham, Shaun K. Kane |
CHI | 2 |
| 2015 | Sharing is Caring: Assistive Technology Designs on ThingiverseabstractAn increasing number of online communities support the open-source sharing of designs that can be built using rapid prototyping to construct physical objects. In this paper, we examine the designs and motivations for assistive technology found on Thingiverse.com, the largest of these communities at the time of this writing. We present results from a survey of all assistive technology that has been posted to Thingiverse since 2008 and a questionnaire distributed to the designers exploring their relationship with assistive technology and the motivation for creating these designs. The majority of these designs are intended to be manufactured on a 3D printer and include assistive devices and modifications for individuals with disabilities, older adults, and medication management. Many of these designs are created by the end-users themselves or on behalf of friends and loved ones. These designers frequently have no formal training or expertise in the creation of assistive technology. This paper discusses trends within this community as well as future opportunities and challenges. Erin Buehler, Stacy M. Branham, Abdullah X. Ali, Jeremy J. Chang, Megan Hofmann, Amy Hurst, Shaun K. Kane |
CHI | 7 |
| 2015 | Accessible Crowdwork?: Understanding the Value in and Challenge of Microtask Employment for People with DisabilitiesabstractWe present the first formal study of crowdworkers who have disabilities via in-depth open-ended interviews of 17 people (disabled crowdworkers and job coaches for people with disabilities) and a survey of 631 adults with disabilities. Our findings establish that people with a variety of disabilities currently participate in the crowd labor marketplace, despite challenges such as crowdsourcing workflow designs that inadvertently prohibit participation by, and may negatively affect the worker reputations of, people with disabilities. Despite such challenges, we find that crowdwork potentially offers different opportunities for people with disabilities relative to the normative office environment, such as job flexibility and lack of a need to rely on public transit. We close by identifying several ways in which crowd labor platform operators and/or individual task requestors could improve the accessibility of this increasingly important form of employment. Kathryn Zyskowski, Meredith Ringel Morris, Jeffrey P. Bigham, Mary L. Gray, Shaun K. Kane |
CSCW | 5 |
| 2014 | Everyday making: identifying future uses for 3D printing in the homeabstractLow-cost and commercially available 3D printers are predicted to be the next disruptive innovation in technology. However, little research has examined how non-designers might interact with fabrication tools in their homes. To explore the potential uses of 3D printers and other fabrication devices in the home, we conducted a study in which 10 households (with 28 individuals) kept a faux 3D printer in their homes for four weeks. Participants kept a log of items that they would want to print, and completed a series of design probes. We found that participants' use of the fabrication tools involved three activities: replicating existing objects, modifying and customizing existing objects, and creating new custom objects. Our study also provides insights on the types of objects that individuals wish to create, and how the faux 3D printer was situated in our participants' homes. Rita Shewbridge, Amy Hurst, Shaun K. Kane |
Conference on Designing Interactive Systems | 3 |
| 2014 | ABC and 3D: opportunities and obstacles to 3D printing in special education environmentsabstractConsumer-grade digital fabrication such as 3D printing is on the rise, and we believe it can be leveraged to great benefit in the arena of special education. Although 3D printing is beginning to infiltrate mainstream education, little to no research has explored 3D printing in the context of students with special support needs. We present a formative study exploring the use of 3D printing at three locations serving populations with varying ability, including individuals with cognitive, motor, and visual impairments. We found that 3D design and printing performs three functions in special education: developing 3D design and printing skills encourages STEM engagement; 3D printing can support the creation of educational aids for providing accessible curriculum content; and 3D printing can be used to create custom adaptive devices. In addition to providing opportunities to students, faculty, and caregivers in their efforts to integrate 3D printing in special education settings, our investigation also revealed several concerns and challenges. We present our investigation at three diverse sites as a case study of 3D printing in the realm of special education, discuss obstacles to efficient 3D printing in this context, and offer suggestions for designers and technologists. Erin Buehler, Shaun K. Kane, Amy Hurst |
ASSETS | 2 |
| 2014 | Evaluating the accessibility of crowdsourcing tasks on Amazon's mechanical turkabstractCrowd work web sites such as Amazon Mechanical Turk enable individuals to work from home, which may be useful for people with disabilities. However, the web sites for finding and performing crowd work tasks must be accessible if people with disabilities are to use them. We performed a heuristic analysis of one crowd work site, Amazon's Mechanical Turk, using the Web Content Accessibility Guidelines 2.0. This paper presents the accessibility problems identified in our analysis and offers suggestions for making crowd work platforms more accessible Rocío Calvo, Shaun K. Kane, Amy Hurst |
ASSETS | 2 |
| 2014 | The gest-rest: a pressure-sensitive chairable input pad for power wheelchair armrestsabstractInteracting with touch screen-based computing devices can be difficult for individuals with mobility impairments that affect their hands, arms, neck, or head. These problems may be especially difficult for power wheelchair users, as the frame of their wheelchair may obstruct the users' range of motion and reduce their ability to reach objects in the environment. The concept of chairable input devices refers to input devices that are designed to fit with the form of an individual's wheelchair, much like wearable technology fits with an individual's clothing. In this paper, we introduce a new chairable input device, the Gest-Rest, which provides a pressure-sensitive input surface that fits over a standard power wheelchair armrest. The Gest-Rest enables users to perform traditional touch screen gestures, such as press and flick, as well as pressure-based gestures such as squeezing and punching. The Gest-Rest enables multiple inputs, unlike most switches, and does not substantially change the shape of the wheelchair armrest. We present a formative evaluation in which nine wheelchair users and three clinicians tested multiple gestures using the Gest-Rest prototype, and provided recommendations for integrating the Gest-Rest with computing applications. Our study showed that our motor impaired participants were each able to perform multiple gestures using the prototype, but had some difficulty with the pre-set sensitivity settings, and would thus benefit from a more robust gesture recognizer. Patrick Carrington, Amy Hurst, Shaun K. Kane |
ASSETS | 3 |
| 2014 | Tactile aids for visually impaired graphical design educationabstractIn this demonstration, we describe our exploration in making graphic design theory accessible to a visually impaired student with the use of rapid prototyping tools. We created over 10 novel aids with the use of a laser cutter and 3D printer to demonstrate tangible examples of color theory, type face, web page layouts, and web design. These tactile aids were inexpensive and fabricated in a relatively small amount of time, suggesting the feasibility of our approach. The participant's feedback concluded an increased understanding of the class material and confirmed the potential of tactile aids and rapid prototyping in an educational environment Samantha McDonald, Joshua Dutterer, Ali Abdolrahmani, Shaun K. Kane, Amy Hurst |
ASSETS | 4 |
| 2014 | "just let the cane hit it": how the blind and sighted see navigation differentlyabstractSighted people often have the best of intentions when they want to help a blind person navigate, but their well meaning is also often coupled with a lack of knowledge and understanding about how a person navigates without vision. As a result what sighted people think is the right feedback is too often the wrong feedback to give to a person with a visual impairment. Understanding how to provide feedback to blind navigators is crucial to the design of assistive technologies for navigation. In our research investigating the design of a personal pedestrian navigation device, we observed firsthand the ways that sighted people seemingly misunderstand how many blind people navigate when using a white cane mobility aid. Throughout our qualitative end user studies that included focus groups and observations (including couple-based observations with a close companion) we gathered data that explicitly shows how the language and understanding of sighted vs. blind pedestrians differs greatly and even how it can be dangerous when people interfere in the wrong way. From our findings we discuss why it is difficult for a blind person to navigate like a sighted person to ensure designers are aware of the difficulties and designing with new training in mind, not simply designing from their own point of view. We also want to encourage advocacy and empathy amongst the sighted community towards this activity of walking around independently. Michele A. Williams, Caroline Marie Galbraith, Shaun K. Kane, Amy Hurst |
ASSETS | 3 |
| 2014 | Wearables and chairables: inclusive design of mobile input and output techniques for power wheelchair usersabstractPower wheelchair users often use and carry multiple mobile computing devices. Many power wheelchair users have some upper body motor impairment that can make using these devices difficult. We believe that mobile device accessibility could be improved through designs that take into account users' functional abilities and take advantage of available space around the wheelchair itself. In this paper we present findings from multiple design sessions and interviews with 13 power wheelchair users and 30 clinicians, exploring the placement and form factor possibilities for input and output on a power wheelchair. We found that many power wheelchair users could benefit from chairable technology that is designed to work within the workspace of the wheelchair, whether worn on the body or mounted on he wheelchair frame. We present participants' preferences for chairable input and output devices, and identify possible design configurations for wearable and chairable devices. Patrick Carrington, Amy Hurst, Shaun K. Kane |
CHI | 3 |
| 2014 | Framing the conversation: the role of facebook conversations in shopping for eyeglassesabstractIndividuals often leverage their social network to receive feedback from their peers about various decisions. Capitalizing on this trend, fashion retailers, such as the eyeglass frame retailer Warby Parker, have created 'home try-on' programs and encourage customers to solicit feedback from their friends before making a purchasing decision. In this paper, we investigate Warby Parker's Facebook page and explore the ways customers formulate questions and conversations as they shop for new eyeglass frames. Our research presents insights derived from a dataset of over 10,000 Facebook posts, consisting of photos, comments, and 'likes'. Using statistical analyses and qualitative coding techniques, we examine trends and patterns of conversation on the Warby Parker Facebook page. We find that contributors produce a broad range of content, and use the Warby Parker Facebook page as a platform for creative self-expression, and socially driven decision-making. Karim Said, Michele A. Burton, Amy Hurst, Shaun K. Kane |
CSCW | 4 |
| 2014 | Tracking @stemxcomet: teaching programming to blind students via 3D printing, crisis management, and twitterabstractIntroductory programming activities for students often include graphical user interfaces or other visual media that are inaccessible to students with visual impairments. Digital fabrication techniques such as 3D printing offer an opportunity for students to write programs that produce tactile objects, providing an accessible way of exploring program output. This paper describes the planning and execution of a four-day computer science education workshop in which blind and visually impaired students wrote Ruby programs to analyze data from Twitter regarding a fictional ecological crisis. Students then wrote code to produce accessible tactile visualizations of that data. This paper describes outcomes from our workshop and suggests future directions for integrating data analysis and 3D printing into programming instruction for blind students. Shaun K. Kane, Jeffrey P. Bigham |
SIGCSE | 1 |
| 2013 | Making "making" accessibleabstractAssistive technologies empower individuals to accomplish tasks they might not be able to do otherwise. Unfortunately, a large percentage of assistive devices that are purchased (35% or more) end up unused or abandoned [8], leaving many people with assistive technology that is inappropriate for their needs. This paper describes our ongoing work to help more people gain access to the assistive technology they need by empowering non-engineers to "Do-It-Yourself" (DIY), and thus create, modify, or build their own assistive technology. We discuss how a new generation of rapid prototyping tools and online communities can empower more individuals, and we describe two technologies we have developed to enable novices to prototype and create physical objects. Amy Hurst, Shaun K. Kane |
IDC | 2 |
| 2013 | How power wheelchair users choose computing devicesabstractPeople with motor impairments experience a range of challenges when interacting with computers. While much prior research has explored the effects of motor impairments on accessing computer input devices, such as keyboards, mice, and touch screens, we know relatively little about how real world use of a wheelchair affects why people in power wheelchairs choose specific computing devices, and how they switch between such devices. We interviewed 8 power wheelchair users about their use of computers and mobile devices. We found that participants often had difficulty switching between the various devices in their life, and that technology use was especially challenging on the go. Our findings suggest that there are numerous opportunities to make computing more wheelchair-friendly, by consolidating devices, improving reachability and portability of devices, and by creating technology that is robust to the challenges of moving around in a wheelchair. Patrick Carrington, Amy Hurst, Shaun K. Kane |
ASSETS | 3 |
| 2013 | Touchplates: low-cost tactile overlays for visually impaired touch screen usersabstractAdding tactile feedback to touch screens can improve their accessibility to blind users, but prior approaches to integrating tactile feedback with touch screens have either offered limited functionality or required extensive (and typically expensive) customization of the hardware. We introduce touchplates, carefully designed tactile guides that provide tactile feedback for touch screens in the form of physical guides that are overlaid on the screen and recognized by the underlying application. Unlike prior approaches to integrating tactile feedback with touch screens, touchplates are implemented with simple plastics and use standard touch screen software, making them versatile and inexpensive. Touchplates may be customized to suit individual users and applications, and may be produced on a laser cutter, 3D printer, or made by hand. We describe the design and implementation of touchplates, a "starter kit" of touchplates, and feedback from a formative evaluation with 9 people with visual impairments. Touchplates provide a low-cost, adaptable, and accessible method of adding tactile feedback to touch screen interfaces. Shaun K. Kane, Meredith Ringel Morris, Jacob O. Wobbrock |
ASSETS | 1 |
| 2013 | Follow that sound: using sonification and corrective verbal feedback to teach touchscreen gesturesabstractWhile sighted users may learn to perform touchscreen gestures through observation (e.g., of other users or video tutorials), such mechanisms are inaccessible for users with visual impairments. As a result, learning to perform gestures can be challenging. We propose and evaluate two techniques to teach touchscreen gestures to users with visual impairments: (1) corrective verbal feedback using text-to-speech and automatic analysis of the user's drawn gesture; (2) gesture sonification to generate sound based on finger touches, creating an audio representation of a gesture. To refine and evaluate the techniques, we conducted two controlled lab studies. The first study, with 12 sighted participants, compared parameters for sonifying gestures in an eyes-free scenario and identified pitch + stereo panning as the best combination. In the second study, 6 blind and low-vision participants completed gesture replication tasks with the two feedback techniques. Subjective data and preliminary performance findings indicate that the techniques offer complementary advantages. Uran Oh, Shaun K. Kane, Leah Findlater |
ASSETS | 2 |
| 2013 | "Pray before you step out": describing personal and situational blind navigation behaviorsabstractPersonal navigation tools have greatly impacted the lives of people with vision impairments. As people with vision impairments often have different requirements for technology, it is important to understand users' ever-changing needs. We conducted a formative study exploring how people with vision impairments used technology to support navigation. Our findings from interviews with 30 adults with vision impairments included insights about experiences in Orientation & Mobility (O&M) training, everyday navigation challenges, helpful and unhelpful technologies, and the role of social interactions while navigating. We produced a set of categorical data that future technologists can use to identify user requirements and usage scenarios. These categories consist of Personality and Scenario attributes describing navigation behaviors of people with vision impairments. We demonstrate the usefulness of these attributes by introducing navigation-style personas backed by our data. This work demonstrates the complex choices individuals with vision impairments undergo when leaving their home, and the many factors that affect their navigation behavior. Michele A. Williams, Amy Hurst, Shaun K. Kane |
ASSETS | 3 |
| 2013 | Access lens: a gesture-based screen reader for real-world documentsabstractGesture-based touch screen user interfaces, when designed to be accessible to blind users, can be an effective mode of interaction for those users. However, current accessible touch screen interaction techniques suffer from one serious limitation: they are only usable on devices that have been explicitly designed to support them. Access Lens is a new interaction method that uses computer vision-based gesture tracking to enable blind people to use accessible gestures on paper documents and other physical objects, such as product packages, device screens, and home appliances. This paper describes the development of Access Lens hardware and software, the iterative design of Access Lens in collaboration with blind computer users, and opportunities for future development. Shaun K. Kane, Brian Frey, Jacob O. Wobbrock |
CHI | 1 |
| 2012 | What we talk about: designing a context-aware communication tool for people with aphasiaabstractMany people with aphasia experience difficulty recalling words extemporaneously, but can recognize those words when given an image, text, or audio prompt. Augmented and alternative communication (AAC) systems can help address this problem by enabling people with aphasia to browse and select from a list of vocabulary words. However, these systems can be difficult to navigate, especially when they contain large amounts of content. In this paper, we describe the design of TalkAbout, a context-aware, adaptive AAC system that provides users with a word list that is adapted to their current location and conversation partner. We describe the design and development of TalkAbout, which we conducted in collaboration with 5 adults with aphasia. We then present guidelines for developing and evaluating context-aware technology for people with aphasia. Shaun K. Kane, Barbara Linam-Church, Kyle Althoff, Denise McCall |
ASSETS | 1 |
| 2011 | Usable gestures for blind people: understanding preference and performanceabstractDespite growing awareness of the accessibility issues surrounding touch screen use by blind people, designers still face challenges when creating accessible touch screen interfaces. One major stumbling block is a lack of understanding about how blind people actually use touch screens. We conducted two user studies that compared how blind people and sighted people use touch screen gestures. First, we conducted a gesture elicitation study in which 10 blind and 10 sighted people invented gestures to perform common computing tasks on a tablet PC. We found that blind people have different gesture preferences than sighted people, including preferences for edge-based gestures and gestures that involve tapping virtual keys on a keyboard. Second, we conducted a performance study in which the same participants performed a set of reference gestures. We found significant differences in the speed, size, and shape of gestures performed by blind people versus those performed by sighted people. Our results suggest new design guidelines for accessible touch screen interfaces. Shaun K. Kane, Jacob O. Wobbrock, Richard E. Ladner |
CHI | 1 |
| 2011 | Access overlays: improving non-visual access to large touch screens for blind usersabstractMany touch screens remain inaccessible to blind users, and those approaches to providing access that do exist offer minimal support for interacting with large touch screens or spatial data. In this paper, we introduce a set of three software-based access overlays intended to improve the accessibility of large touch screen interfaces, specifically interactive tabletops. Our access overlays are called edge projection, neighborhood browsing, and touch-and-speak. In a user study, 14 blind users compared access overlays to an implementation of Apple's VoiceOver screen reader. Our results show that two of our techniques were faster than VoiceOver, that participants correctly answered more questions about the screen's layout using our techniques, and that participants overwhelmingly preferred our techniques. We developed several applications demonstrating the use of access overlays, including an accessible map kiosk and an accessible board game. Shaun K. Kane, Meredith Ringel Morris, Annuska Z. Perkins, Daniel J. Wigdor, Richard E. Ladner, Jacob O. Wobbrock |
UIST | 1 |
| 2010 | Augmenting on-screen instructions with micro-projected guides: when it works, and when it failsabstractWe present a study that evaluates the effectiveness of augmenting on-screen instructions with micro-projection for manual task guidance unlike prior work, which replaced screen instructions with alternative modalities (e.g., head-mounted displays). In our study, 30 participants completed 10 trials each of 11 manual tasks chosen to represent a set of common task-components (e.g., cutting, folding) found in many everyday activities such as crafts, cooking, and hobby electronics. Fifteen participants received only on-screen instructions, and 15 received both on-screen and micro-projected instructions. In contrast to prior work, which focused only on whole tasks, our study examines the benefit of augmenting common task instructions. The augmented instructions improved participants' performance overall; however, we show that in certain cases when projected guides and physical objects visually interfered, projected elements caused increased errors. Our results demonstrate that examining effectiveness at an instruction level is both useful and necessary, and provide insight into the design of systems that help users perform everyday tasks. Stephanie Rosenthal, Shaun K. Kane, Jacob O. Wobbrock, Daniel Avrahami |
UbiComp | 2 |
| 2009 | Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilitiesabstractMobile devices provide people with disabilities new opportunities to act independently in the world. However, these empowering devices have their own accessibility challenges. We present a formative study that examines how people with visual and motor disabilities select, adapt, and use mobile devices in their daily lives. We interviewed 20 participants with visual and motor disabilities and asked about their current use of mobile devices, including how they select them, how they use them while away from home, and how they adapt to accessibility challenges when on the go. Following the interviews, 19 participants completed a diary study in which they recorded their experiences using mobile devices for one week. Our results show that people with visual and motor disabilities use a variety of strategies to adapt inaccessible mobile devices and successfully use them to perform everyday tasks and navigate independently. We provide guidelines for more accessible and empowering mobile device design. Shaun K. Kane, Chandrika Jayant, Jacob O. Wobbrock, Richard E. Ladner |
ASSETS | 1 |
| 2009 | Exploring Cross-Device Web Use on PCs and Mobile Devices
Shaun K. Kane, Amy K. Karlson, Brian Meyers, Paul Johns, Andy Jacobs, Greg Smith |
INTERACT (1) | 1 |
| 2009 | Bonfire: a nomadic system for hybrid laptop-tabletop interactionabstractWe present Bonfire, a self-contained mobile computing system that uses two laptop-mounted laser micro-projectors to project an interactive display space to either side of a laptop keyboard. Coupled with each micro-projector is a camera to enable hand gesture tracking, object recognition, and information transfer within the projected space. Thus, Bonfire is neither a pure laptop system nor a pure tabletop system, but an integration of the two into one new nomadic computing platform. This integration (1) enables observing the periphery and responding appropriately, e.g., to the casual placement of objects within its field of view, (2) enables integration between physical and digital objects via computer vision, (3) provides a horizontal surface in tandem with the usual vertical laptop display, allowing direct pointing and gestures, and (4) enlarges the input/output space to enrich existing applications. We describe Bonfire's architecture, and offer scenarios that highlight Bonfire's advantages. We also include lessons learned and insights for further development and use. Shaun K. Kane, Daniel Avrahami, Jacob O. Wobbrock, Beverly L. Harrison, Adam D. Rea, Matthai Philipose, Anthony LaMarca |
UIST | 1 |
| 2008 | Envisioning systemic effects on persons and society throughout interactive system designabstractThe design, development, and deployment of interactive systems can substantively impact individuals, society, and the natural environment, now and potentially well into the future. Yet, a scarcity of methods exists to support long-term, emergent, systemic thinking in interactive design practice. Toward addressing this gap, we propose four envisioning criteria --- stakeholders, time, values, and pervasiveness -- distilled from prior work in urban planning, design noir, and Value Sensitive Design. We characterize how the criteria can support systemic thinking, illustrate the integration of the envisioning criteria into established design practice (scenariobased design), and provide strategic activities to serve as generative envisioning tools. We conclude with suggestions for use and future work. Key contributions include: 1) four envisioning criteria to support systemic thinking, 2) value scenarios (extending scenario-based design), and 3) strategic activities for engaging the envisioning criteria in interactive system design practice. Lisa P. Nathan, Batya Friedman, Predrag V. Klasnja, Shaun K. Kane, Jessica K. Miller |
Conference on Designing Interactive Systems | 4 |
| 2008 | Slide rule: making mobile touch screens accessible to blind people using multi-touch interaction techniquesabstractRecent advances in touch screen technology have increased the prevalence of touch screens and have prompted a wave of new touch screen-based devices. However, touch screens are still largely inaccessible to blind users, who must adopt error-prone compensatory strategies to use them or find accessible alternatives. This inaccessibility is due to interaction techniques that require the user to visually locate objects on the screen. To address this problem, we introduce Slide Rule, a set of audio-based multi-touch interaction techniques that enable blind users to access touch screen applications. We describe the design of Slide Rule, our interaction techniques, and a user study in which 10 blind people used Slide Rule and a button-based Pocket PC screen reader. Results show that Slide Rule was significantly faster than the button-based system, and was preferred by 7 of 10 users. However, users made more errors when using Slide Rule than when using the more familiar button-based system. Shaun K. Kane, Jeffrey P. Bigham, Jacob O. Wobbrock |
ASSETS | 1 |
| 2008 | Design patterns for sociality in human-robot interactionabstractWe propose that Christopher Alexander's idea of design patterns can benefit the emerging field of HRI. We first discuss four features of design patterns that appear particularly useful. For example, a pattern should be specified abstractly enough such that many different instantiations of the pattern can be uniquely realized in the solution to specific problems in context. Then, after describing our method for generating patterns, we offer and describe eight possible design patterns for sociality in human robot interaction: initial introduction, didactic communication, in motion together, personal interests and history, recovering from mistakes, reciprocal turn-taking in game context, physical intimacy, and claiming unfair treatment or wrongful harms. We also discuss the issue of validation of design patterns. If a design pattern program proves successful, it will provide HRI researchers with basic knowledge about human robot interaction, and save time through the reuse of patterns to achieve high levels of sociality. Peter H. Kahn Jr., Nathan G. Freier, Takayuki Kanda 0001, Hiroshi Ishiguro, Jolina H. Ruckert, Rachel L. Severson, Shaun K. Kane |
HRI | 7 |
| 2008 | TrueKeys: identifying and correcting typing errors for people with motor impairmentsabstractPeople with motor impairments often have difficulty typing using desktop keyboards. We developed TrueKeys, a system that combines models of word frequency, keyboard layout, and typing error patterns to automatically identify and correct typing mistakes. In this paper, we describe the TrueKeys algorithm, compare its performance to existing correction algorithms, and report on a study of TrueKeys with 9 motor-impaired and 9 non-impaired participants. Running in non-interactive mode, TrueKeys performed more corrections than popular commercial and open source spell checkers. Used interactively, both motor-impaired and non-impaired users performed typing tasks significantly more accurately with TrueKeys than without. However, typing speed was reduced while TrueKeys was enabled. Shaun K. Kane, Jacob O. Wobbrock, Mark Harniss, Kurt L. Johnson |
IUI | 1 |
| 2008 | Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spacesabstractUsing a mobile device while moving limits attention and motor ability and can result in reduced performance. Mobile devices that can sense and adapt to contextual factors such as movement may reduce this performance deficit. We performed two studies evaluating the feasibility of walking user interfaces (WUIs) that adapt their layout when the user is moving. In a pilot study with 6 users, we evaluated the effects of different button sizes on performance when walking while using a portable music player. Results showed significant interactions between size and movement. In the second study, 29 users evaluated the performance of a WUI that dynamically changed button sizes as the user moved. Results show that our dynamic user interface performs at the level of its component static interfaces without any additional penalty due to adaptation. This work adds to our design knowledge about walking user interfaces and provides lessons learned in evaluating mobile devices while walking in public spaces. Shaun K. Kane, Jacob O. Wobbrock, Ian E. Smith |
Mobile HCI | 1 |
| 2007 | Barrier pointing: using physical edabstractMobile phones and personal digital assistants (PDAs) are incredibly popular pervasive technologies. Many of these devices contain touch screens, which can present problems for users with motor impairments due to small targets and their reliance on tapping for target acquisition. In order to select a target, users must tap on the screen, an action which requires the precise motion of flying into a target and lifting without slipping. In this paper, we propose a new technique for target acquisition called barrier pointing, which leverages the elevated physical edges surrounding the screen to improve pointing accuracy. After designing a series of barrier pointing techniques, we conducted an initial study with 9 able bodied users and 9 users with motor impairments in order to discover the parameters that make barrier pointing successful. From this data, we offer an in-depth analysis of the performance of two motor impaired users for whom barrier pointing was especially beneficial. We show the importance of providing physical stability by allowing the stylus to press against the screen and its physical edge. We offer other design insights and lessons learned that can inform future attempts at leveraging the physical properties of mobile devices to improve accessibility. Jon Froehlich, Jacob O. Wobbrock, Shaun K. Kane |
ASSETS | 3 |
| 2007 | Engaging student programmers as inclusive designersabstractWhile awareness of accessibility standards and guidelines can help students learn to produce accessible web pages, overcoming real-world accessibility issues often requires creativity and flexibility. One way to develop these skills is to engage student programmers in design practice. This poster describes an undergraduate web programming course that used pedagogical techniques drawn from architecture and industrial design education to enable students to empathize with users with disabilities and to creatively address accessibility issues. Evaluative feedback from students indicates that this approach is effective in encouraging accessible design practices. Shaun K. Kane |
ITiCSE | 1 |