People who are blind or have low-vision (BLV) may hesitate to travel independently in unfamiliar environments due to uncertainty about the physical landscape. While most tools focus on in-situ navigation assistance, those supporting pre-travel assistance typically provide information about only landmarks and turn-by-turn instructions, lacking detailed visual context. Street level imagery, which contains rich visual information and has the potential to reveal numerous environmental details, remains inaccessible to BLV people. In this work, we present SceneScout, a multimodal large language model (MLLM)-driven prototype that enables accessible interactions with street level imagery. SceneScout supports two modes: (1) Route Preview, enabling users to familiarize themselves with visual details along a route, and (2) Virtual Exploration, enabling free, user-driven movement within street level imagery. Our user study (N = 10) demonstrates that SceneScout helps BLV users uncover visual information otherwise unavailable through existing means. An initial analysis of AI-generated descriptions suggests that the majority are accurate and describe stable visual elements even in older imagery, though occasional subtle and plausible errors make them difficult to verify without sight. We discuss future opportunities and challenges of street level imagery-based navigation experiences.

Generative text-to-image (T2I) models often output images that have stereotypes of people with disabilities. One possibility to mitigate the risk of these biases is to intervene at the user level, supporting T2I users themselves in being able to identify biases and act accordingly. To understand how to design such support and its potential effectiveness, we implemented two interventions: (1) an education module to inform users of disability stereotypes in T2I images and (2) AI-generated feedback about potential stereotypes in a given image. We evaluated these options alone and in combination through a controlled experiment (N = 103) and a qualitative study (N = 10). Our results demonstrate that interface-based interventions can help users identify stereotypes, but that people do not always desire to avoid stereotypes. Participants wanted image subjects to “look” disabled, which sometimes inadvertently perpetuated stereotypes. Our results indicate clear ways for T2I interfaces to support users in prompting for and assessing images.

We propose a novel task, hierarchical instance tracking, which entails tracking all instances of predefined categories of objects and parts, while maintaining their hierarchical relationships. We introduce the first benchmark dataset supporting this task, consisting of 2,765 unique entities that are tracked in 552 videos and belong to 40 categories (across objects and parts). Evaluation of seven variants of four models tailored to our novel task reveals the new dataset is challenging. Our dataset is available at https://vizwiz.org/tasks-and-datasets/hierarchical-instance-tracking/

Insights into the unique security and privacy practices, risks, and solutions for people with disabilities are currently fragmented across disciplines.In this work, we present a literature review of 33 papers published at leading human-computer interaction, accessibility, and usable security and privacy venues.We categorize the contributions of these papers-ranging from interventions to empirical studies of risks and behaviors-and identify key themes and implications.Papers in this corpus highlight 1) the opportunities and risks of the data collected by assistive technologies and security and privacy tools, 2) the inaccessibility or low usability of security and privacy solutions for people with disabilities, and 3) the utility of customized, contextual security and privacy solutions.We conclude with best practices for collecting data from disabled communities and implications for the design of assistive technologies and security/privacy tools.

Accessibility research has a broad mandate: use technology to make the world more accessible to disabled people. Yet, as a field, accessibility research lacks a clear characterization of what "accessibility" is. Furthermore, it has been historically limited in who is designed for, focusing on specific types of disability and often failing to consider how disability intersects with other identities. We set out to explicate what it means to make something accessible, grounded in the lived experiences of a diverse group of 25 disabled people. From our empirical findings, we develop a process for modeling accessibility. First, an individual assesses their experience of inaccess, specifically, the type of barrier they face, the technology repertoire they possess, and the contextual factors that shape how they address accessibility barriers. Then, having assessed an access barrier, they perform consequence calculus, weighing all available options to achieve access and deciding upon the option that best matches their priorities. We highlight the situated nature of access; people's identities, contextual factors, repertoires, and priorities all dictate their experience of accessibility.

Blind and low vision (BLV) individuals use Generative AI (GenAI) tools to interpret and manage visual content in their daily lives.While such tools can enhance the accessibility of visual content and enable greater user independence, they also introduce complex challenges around visual privacy.In this paper, we investigate the current practices and future design preferences of blind and low vision individuals through an interview study with 21 participants.Our findings reveal a range of current practices with GenAI that balance privacy, efficiency, and emotional agency, with users accounting for privacy risks across six key scenarios: selfpresentation, indoor spatial privacy, outdoor spatial privacy, social media sharing, sharing with employer or professional setup, and handling professional content as employers.Our findings reveal design preferences, including on-device processing, zero-retention guarantees, sensitive content redaction, privacy-aware appearance indicators, and multimodal tactile mirrored interaction methods.We conclude with actionable design recommendations to support user-centered visual privacy through GenAI, expanding the notion of privacy and responsible handling of others' information.

From curating the layout of a resume to selecting filters for social media, creating and configuring visual content allows individuals to express identity, communicate intent, and engage socially, yet blind individuals often face significant barriers to such expressive practices.Prior accessibility research primarily addresses functional content configuration, leaving little understanding of blind individuals' expressive visual creation needs.To better understand and support these needs, we conducted a two-stage study: first, we interviewed 10 blind participants to understand their motivations, current practices, and barriers in visual expression, and to ideate on potential visual editing support; second, based on interview insights, we developed an interactive prototype (VizXpress) that provides real-time feedback on visual aesthetics using a vision-language model and supports automated and manual visual editing controls.We used VizXpress as a design probe to further explore accessible design opportunities for visual expression.Our findings highlight many blind users' strong interest in creating visually expressive content, nuanced informational requirements for subjective aesthetics (e.g., color, mood, lighting), and ongoing accessibility challenges with visual creative tools.Grounded in these insights, we propose design implications including richer aesthetic feedback, controlled intelligent editing, and accessible manual editing mechanisms.

a reproduction of a tactile graphic made through thermoforming over the original.Images from the National Braille Press [29].

Record soundsTrain personalized model Iteratively test Figure 1: Overview of the SPECTRA pipeline.In an interactive machine learning training workfow, users collect audio data samples (left), flter their data into a training dataset (center), and assess their model's performance in a live environment (right).The design includes key elements to support the needs of DHH users during this process, including spectrogram and waveform audio visualizations of audio, data annotating to save useful contextual information, and an interactive clustering visualization of their dataset.

As children increasingly consume media on devices, parents look for ways this usage can support learning and growth, especially in domains like social-emotional learning. We introduce eaSEL, a system that (a) integrates social-emotional learning (SEL) curricula into children's video consumption by generating reflection activities and (b) facilitates parent-child discussions around digital media without requiring co-consumption of videos. We present a technical evaluation of our system's ability to detect social-emotional moments within a transcript and to generate high-quality SEL-based activities for both children and parents. Through a user study with N=20 parent-child dyads, we find that after completing an eaSEL activity, children reflect more on the emotional content of videos. Furthermore, parents find that the tool promotes meaningful active engagement and could scaffold deeper conversations around content. Our work paves directions in how AI can support children's social-emotional reflection of media and family connections in the digital age.

Individuals who are blind or have low vision (BLV) are at a heightened risk of sharing private information if they share photographs they have taken. To facilitate developing technologies that can help them preserve privacy, we introduce BIV-Priv-Seg, the first localization dataset originating from people with visual impairments that shows private content. It contains 1,028 images with segmentation annotations for 16 private object categories. We first characterize BIV-Priv-Seg and then evaluate modern models' performance for locating private content in the dataset. We find modern models struggle most with locating private objects that are not salient, small, and lack text as well as recognizing when private content is absent from an image. We facilitate future extensions by sharing our new dataset with the evaluation server at https://vizwiz.org/tasks-and-datasets/object-localization/

Much of early literacy education happens at home with caretakers reading books to young children. Prior research demonstrates how having dialogue with children during co-reading can develop critical reading readiness skills, but most adult readers are unsure if and how to lead effective conversations. We present ContextQ, a tablet-based reading application to unobtrusively present auto-generated dialogic questions to caretakers to support this dialogic reading practice. An ablation study demonstrates how our method of encoding educator expertise into the question generation pipeline can produce high-quality output; and through a user study with 12 parent-child dyads (child age: 4–6), we demonstrate that this system can serve as a guide for parents in leading contextually meaningful dialogue, leading to significantly more conversational turns from both the parent and the child and deeper conversations with connections to the child’s everyday life.

A significant body of human-computer interaction accessibility research explores ways technology can improve communication access. Yet, this research infrequently engages other fields with complementary expertise – namely disability studies, Deaf studies, disability justice, and communication studies. To facilitate interdisciplinary communication access research, we synthesize thinking from these four fields into a framework of collective communication access. We then analyze human-centered accessibility-focused captioning research published between 2013 and 2023, investigating how collective communication access principles are or are not employed. We find that, while the majority of captioning research does not demonstrate a collective communication access approach, it reaches a baseline of targeting change toward inaccessible technical infrastructures and engaging d/Deaf and hard of hearing people as captioning experts. The small body of work that aligns with our framework, however, demonstrates that designing to change discriminatory social conditions and engaging conversation partners in access is a promising direction for future work.

As user-generated video dominates media landscapes, it poses an accessibility challenge. While disability advocacy groups globally have secured hard-won accessibility regulations for broadcast media, no such regulation of user-generated content exists. Yet, one major player in this shift, TikTok, has a culture of user-generated, creative captioning. We sought to understand how TikTok videos are captioned and the impact current practices have on those who need captions to access audio content. Therefore, we conducted a content analysis of 300 open-captioned TikToks and contextualized these findings by interviewing nine caption users. We found that the current state of TikTok captioning does facilitate access to the platform but that a user-generated, social video-specific standard for captioning could improve caption quality and expand access. We contribute an empirical account of the state of TikTok captioning and outline steps toward a standard for user-generated captioning.

Blind individuals commonly share photos in everyday life. Despite substantial interest from the blind community in being able to independently obfuscate private information in photos, existing tools are designed without their inputs. In this study, we prototyped a preliminary screen reader-accessible obfuscation interface to probe for feedback and design insights. We implemented a version of the prototype through off-the-shelf AI models (e.g., SAM, BLIP2, ChatGPT) and a Wizard-of-Oz version that provides human-authored guidance. Through a user study with 12 blind participants who obfuscated diverse private photos using the prototype, we uncovered how they understood and approached visual private content manipulation, how they reacted to frictions such as inaccuracy with existing AI models and cognitive load, and how they envisioned such tools to be better designed to support their needs (e.g., guidelines for describing visual obfuscation effects, co-creative interaction design that respects blind users’ agency).

Simultaneous interpretation is a cognitively taxing task, and even seasoned professionals benefit from real-time assistance. However, both recruiting professional interpreters and evaluating new assistance techniques are difficult. We present a novel, realistic simultaneous interpretation task that mimics the cognitive load of interpretation with crowdworker surrogates. Our task tests different real-time assistance methods in a Wizard-of-Oz experiment with a large pool of proxy users and compares against professional interpreters. Both professional and proxy participants respond similarly to changes in interpreting conditions, including improvement with two assistance interventions—translation of specific terms and of numbers—compared to a no-assistance control.

Collaborative ideation plays a vital role in driving creativity and innovation across various professional and educational contexts. This study investigates the experiences of disabled individuals within the collaborative ideation process, specifically examining their utilization of digital whiteboarding tools. Through interviews with 19 professionals and academics with disabilities, alongside a thematic analysis of online forum posts for two popular digital whiteboarding platforms (Miro and Figma), we delve into the access barriers encountered by disabled individuals and the strategies they employ to create access in collaborative ideation. Our findings illuminate the multifaceted nature of access barriers, encompassing issues such as inaccessible visual features, technology-induced discomfort, unstructured nature of freeform content, and complex communication setups. Furthermore, we uncover the intricate dynamics involved in negotiating diverse access needs and conflicts within teams involving people with different disabilities. Through this analysis, we highlight tensions around proficiency with inaccessible technologies stemming from ableist standards of professional success and discuss the implications of our findings for the design of accessible collaborative ideation systems.

The privacy dimensions of accessibility technologies are often understudied and overlooked. Very little prior research has investigated the privacy concerns of disabled people, and much less has studied the barriers of privacy-preserving techniques. In order to address this gap and bridge between two separate communities (accessibility and privacy), our one-day workshop explores how researchers might design and build technologies that are both accessible and privacy-preserving.

Creative activities play an essential role in everyday life. Recently, there has been increasing interest in the accessibility community to support blind and low vision (BLV) people’s digital creative experiences. We conducted a mixed-method study to gain a comprehensive understanding of their creative needs to inform and focus this line of research. Through a large-scale survey (N = 165) and follow-up interviews (N = 15), we learned that BLV people are interested in a more diverse range of creative tasks than what is currently accessible. In particular, many forms of visual content creation and advanced expressions still remain challenging. Participants pointed out both accessibility improvements and social changes needed to fulfill personal creative pursuits. In turn, we discuss potential design ideas to move toward more inclusive creative practices, such as developing alternative, non-visual information-sharing methods and establishing visual information presentation guidelines specific to creative contexts.

Consumer speech recognition systems do not work as well for many people with speech differences, such as stuttering, relative to the rest of the general population. However, what is not clear is the degree to which these systems do not work, how they can be improved, or how much people want to use them. In this paper, we first address these questions using results from a 61-person survey from people who stutter and find participants want to use speech recognition but are frequently cut off, misunderstood, or speech predictions do not represent intent. In a second study, where 91 people who stutter recorded voice assistant commands and dictation, we quantify how dysfluencies impede performance in a consumer-grade speech recognition system. Through three technical investigations, we demonstrate how many common errors can be prevented, resulting in a system that cuts utterances off 79.1% less often and improves word error rate from 25.4% to 9.9%.

Visual arts play an important role in cultural life and provide access to social heritage and self-enrichment, but most visual arts are inaccessible to blind people. Researchers have explored different ways to enhance blind people’s access to visual arts (e.g., audio descriptions, tactile graphics). However, how blind people adopt these methods remains unknown. We conducted semi-structured interviews with 15 blind visual arts patrons to understand how they engage with visual artwork and the factors that influence their adoption of visual arts access methods. We further examined interview insights in a follow-up survey (N=220). We present: 1) current practices and challenges of accessing visual artwork in-person and online (e.g., Zoom tour), 2) motivation and cognition of perceiving visual arts (e.g., imagination), and 3) implications for designing visual arts access methods. Overall, our findings provide a roadmap for technology-based support for blind people’s visual arts experiences.

With improvements in automated speech recognition and increased use of videoconferencing, real-time captioning has changed significantly. This shift toward broadly available but less accurate captioning invites exploration of the role hearing conversation partners play in shaping the accessibility of a conversation to d/Deaf and hard of hearing (DHH) captioning users. While recent work has explored DHH individuals’ videoconferencing experiences with captioning, we focus on established groups’ current practices and priorities for future tools to support more accessible online conversations. Our study consists of three codesign sessions, conducted with four groups (17 participants total, 10 DHH, 7 hearing). We found that established groups crafted social accessibility norms that met their relational contexts. We also identify promising directions for future captioning design, including the need to standardize speaker identification and customization, opportunities to provide behavioral feedback during a conversation, and ways that videoconferencing platforms could enable groups to set and share norms.

We present the design and creation of a disability-first dataset, “BIV-Priv,” which contains 728 images and 728 videos of 14 private categories captured by 26 blind participants to support downstream development of artificial intelligence (AI) models. While best practices in dataset creation typically attempt to eliminate private content, some applications require such content for model development. We describe our approach in creating this dataset with private content in an ethical way, including using props rather than participants’ own private objects and balancing multi-disciplinary perspectives (e.g., accessibility, privacy, computer vision) to meet the tangible metrics (e.g., diversity, category, amount of content) to support AI innovations. We observed challenges that our participants encountered during the data collection, including accessibility issues (e.g., understanding foreground vs. background object placement) and issues due to the sensitive nature of the content (e.g., discomfort in capturing some props such as condoms around family members).

Accessible design and technology could support the large and growing group of people with chronic illnesses. However, human computer interactions (HCI) has largely approached people with chronic illnesses through a lens of medical tracking or treatment rather than accessibility. We describe and demonstrate a framework for designing technology in ways that center the chronically ill experience. First, we identify guiding tenets: 1) treating chronically ill people not as patients but as people with access needs and expertise, 2) recognizing the way that variable ability shapes accessibility considerations, and 3) adopting a theoretical understanding of chronic illness that attends to the body. We then illustrate these tenets through autoethnographic case studies of two chronically ill authors using technology. Finally, we discuss implications for technology design, including designing for consequence-based accessibility, considering how to engage care communities, and how HCI research can engage chronically ill participants in research.

Recent advances have enabled automatic sound recognition systems for deaf and hard of hearing (DHH) users on mobile devices. However, these tools use pre-trained, generic sound recognition models, which do not meet the diverse needs of DHH users. We introduce ProtoSound, an interactive system for customizing sound recognition models by recording a few examples, thereby enabling personalized and fine-grained categories. ProtoSound is motivated by prior work examining sound awareness needs of DHH people and by a survey we conducted with 472 DHH participants. To evaluate ProtoSound, we characterized performance on two real-world sound datasets, showing significant improvement over state-of-the-art (e.g., +9.7% accuracy on the first dataset). We then deployed ProtoSound's end-user training and real-time recognition through a mobile application and recruited 19 hearing participants who listened to the real-world sounds and rated the accuracy across 56 locations (e.g., homes, restaurants, parks). Results show that ProtoSound personalized the model on-device in real-time and accurately learned sounds across diverse acoustic contexts. We close by discussing open challenges in personalizable sound recognition, including the need for better recording interfaces and algorithmic improvements.

Auditory interfaces increasingly support access to website content, through recent advances in voice interaction. Typically, however, these interfaces provide only limited audio styling, collapsing rich visual design into a static audio output style with a single synthesized voice. To explore the potential for more aesthetic and intuitive sound design for websites, we prompted 14 professional sound designers to create auditory website mockups and interviewed them about their designs and rationale. Our findings reveal their prioritized design considerations (aesthetics and emotion, user engagement, audio clarity, information dynamics, and interactivity), specific sound design ideas to support each consideration (e.g., replacing spoken labels with short, memorable audio expressions), and challenges with applying sound design practices to auditory websites. These findings provide promising direction for how to support designers in creating richer auditory website experiences.

Voice assistants have become an essential tool for people with various disabilities because they enable complex phone-or tablet-based interactions without the need for fine-grained motor control, such as with touchscreens. However, these systems are not tuned for the unique characteristics of individuals with speech disorders, including many of those who have a motor-speech disorder, are deaf or hard of hearing, have a severe stutter, or are minimally verbal. We introduce an alternative voice-based input system which relies on sound event detection using fifteen nonverbal mouth sounds like "pop", "click", or "eh." This system was designed to work regardless of ones’ speech abilities and allows full access to existing technology. In this paper, we describe the design of a dataset, model considerations for real-world deployment, and efforts towards model personalization. Our fully-supervised model achieves segment-level precision and recall of 88.6% and 88.4% on an internal dataset of 710 adults, while achieving 0.31 false positives per hour on aggressors such as speech. Five-shot personalization enables satisfactory performance in 84.5% of cases where the generic model fails.

We present a landscape analysis of commercially available visual assistance technologies (VATs) that provide auditory descriptions of image and video content found online, as well as those taken by people who are blind and have visual questions. Through structured web-based searches, we identified 20 VATs released by 17 companies, and analyzed how these companies communicate to users about their technical innovation and service offerings. Our results can orient new researchers, UX professionals, and developers to trends within commercial VAT development.

Accessibility research has grown substantially in the past few decades, yet there has been no literature review of the field. To understand current and historical trends, we created and analyzed a dataset of accessibility papers appearing at CHI and ASSETS since ASSETS' founding in 1994. We qualitatively coded areas of focus and methodological decisions for the past 10 years (2010-2019, N=506 papers), and analyzed paper counts and keywords over the full 26 years (N=836 papers). Our findings highlight areas that have received disproportionate attention and those that are underserved--for example, over 43% of papers in the past 10 years are on accessibility for blind and low vision people. We also capture common study characteristics, such as the roles of disabled and nondisabled participants as well as sample sizes (e.g., a median of 13 for participant groups with disabilities and older adults). We close by critically reflecting on gaps in the literature and offering guidance for future work in the field.

Real-time captioning is a critical accessibility tool for many d/Deaf and hard of hearing (DHH) people. While the vast majority of captioning work has focused on formal settings and technical innovations, in contrast, we investigate captioning for informal, interactive small-group conversations, which have a high degree of spontaneity and foster dynamic social interactions. This paper reports on semi-structured interviews and design probe activities we conducted with 15 DHH participants to understand their use of existing real-time captioning services and future design preferences for both in-person and remote small-group communication. We found that our participants' experiences of captioned small-group conversations are shaped by social, environmental, and technical considerations (e.g., interlocutors' pre-established relationships, the type of captioning displays available, and how far captions lag behind speech). When considering future captioning tools, participants were interested in greater feedback on non-speech elements of conversation (e.g., speaker identity, speech rate, volume) both for their personal use and to guide hearing interlocutors toward more accessible communication. We contribute a qualitative account of DHH people's real-time captioning experiences during small-group conversation and future design considerations to better support the groups being captioned, both in person and online.?

With advances in expressive speech synthesis and conversational understanding, an ever-increasing amount of digital content---including social and personal content---can be consumed through voice. Voice has long been known to convey personal characteristics and emotional states, both of which are prominent aspects of social media. Yet, no study has investigated voice design requirements for social media platforms. We interviewed 15 active social media users about their preferences on using synthesized voices to represent their profiles. Our findings show that participants want to have control over how a voice delivers their content, such as the personality and emotion with which the voice speaks, because these prosodic variations can impact users' online personas and interfere with impression management. We report motivations behind customizing or not customizing voice characteristics in different scenarios, and uncover key challenges around usability and the potential for stereotyping. We argue that synthesized speech for social media should be evaluated not only on listening experience and voice quality but also on its expressivity, degree of customizability, and ability to adapt to contexts (e.g., social media platforms, groups, individual posts). We discuss how our contribution confirms and extends knowledge of voice technology design and online self-presentation, and offer design considerations for voice personalization related to social interactions.

Age and motor ability are well-known to impact input performance. Past work examining these factors, however, has tended to focus on samples of 20-40 participants and has binned participants into a small set of age groups (e.g., “younger” vs. “older”). To foster a more nuanced understanding of how age and motor ability impact input performance, this short paper contributes: (1) a dataset from a large-scale study that captures input performance with a mouse and/or touchscreen from over 700 participants, as well as (2) summary analysis of a subset of 318 participants who range in age from 18 to 83 years old and of whom 53% reported a motor impairment. The analysis demonstrates the continuous relationship between age and input performance for users with and without motor impairments, but also illustrates that knowing a user's age and self-reported motor ability should not lead to assumptions about their input performance. The dataset, which contains mouse and touchscreen input traces, should allow for further exploration by other researchers.

Head-mounted displays can provide private and glanceable speech and sound feedback to deaf and hard of hearing people, yet prior systems have largely focused on speech transcription. We introduce HoloSound, a HoloLens-based augmented reality (AR) prototype that uses deep learning to classify and visualize sound identity and location in addition to providing speech transcription. This poster paper presents a working proof-of-concept prototype, and discusses future opportunities for advancing AR-based sound awareness.

Smartwatches have the potential to provide glanceable, always-available sound feedback to people who are deaf or hard of hearing. In this paper, we present a performance evaluation of four low-resource deep learning sound classification models: MobileNet, Inception, ResNet-lite, and VGG-lite across four device architectures: watch-only, watch+phone, watch+phone+cloud, and watch+cloud. While direct comparison with prior work is challenging, our results show that the best model, VGG-lite, performed similar to the state of the art for non-portable devices with an average accuracy of 81.2% (SD=5.8%) across 20 sound classes and 97.6% (SD=1.7%) across the three highest-priority sounds. For device architectures, we found that the watch+phone architecture provided the best balance between CPU, memory, network usage, and classification latency. Based on these experimental results, we built and conducted a qualitative lab evaluation of a smartwatch-based sound awareness app, called SoundWatch (Figure 1), with eight DHH participants. Qualitative findings show support for our sound awareness app but also uncover issues with misclassifications, latency, and privacy concerns. We close by offering design considerations for future wearable sound awareness technology.

We present a qualitative study with 16 deaf and hard of hearing (DHH) participants examining reactions to smartwatch-based visual + haptic sound feedback designs. In Part 1, we conducted a Wizard-of-Oz (WoZ) evaluation of three smartwatch feedback techniques (visual alone, visual + simple vibration, and visual + tacton) and investigated vibrational patterns (tactons) to portray sound loudness, direction, and identity. In Part 2, we visited three public or semi-public locations where we demonstrated sound feedback on the smartwatch in situ to examine contextual influences and explore sound filtering options. Our findings characterize uses for vibration in multimodal sound awareness, both for push notification and for immediately actionable sound information displayed through vibrational patterns (tactons). In situ experiences caused participants to request sound filtering - particularly to limit haptic feedback - as a method for managing soundscape complexity. Additional concerns arose related to learnability, possibility of distraction, and system trust. Our findings have implications for future portable sound awareness systems.

We introduce HomeSound, an in-home sound awareness system for Deaf and hard of hearing (DHH) users. Similar to the Echo Show or Nest Hub, HomeSound consists of a microphone and display, and uses multiple devices installed in each home. We iteratively developed two prototypes, both of which sense and visualize sound information in real-time. Prototype 1 provided a floorplan view of sound occurrences with waveform histories depicting loudness and pitch. A three-week deployment in four DHH homes showed an increase in participants' home- and self-awareness but also uncovered challenges due to lack of line of sight and sound classification. For Prototype 2, we added automatic sound classification and smartwatch support for wearable alerts. A second field deployment in four homes showed further increases in awareness but misclassifications and constant watch vibrations were not well received. We discuss findings related to awareness, privacy, and display placement and implications for future home sound awareness technology.

Automatically generated explanations of how machine learning (ML) models reason can help users understand and accept them. However, explanations can have unintended consequences: promoting over-reliance or undermining trust. This paper investigates how explanations shape users' perceptions of ML models with or without the ability to provide feedback to them: (1) does revealing model flaws increase users' desire to "fix" them; (2) does providing explanations cause users to believe - wrongly - that models are introspective, and will thus improve over time. Through two controlled experiments - varying model quality - we show how the combination of explanations and user feedback impacted perceptions, such as frustration and expectations of model improvement. Explanations without opportunity for feedback were frustrating with a lower quality model, while interactions between explanation and feedback for the higher quality model suggest that detailed feedback should not be requested without explanation. Users expected model correction, regardless of whether they provided feedback or received explanations.

Wayfinding is a critical but challenging task for people who have low vision, a visual impairment that falls short of blindness. Prior wayfinding systems for people with visual impairments focused on blind people, providing only audio and tactile feedback. Since people with low vision use their remaining vision, we sought to determine how audio feedback compares to visual feedback in a wayfinding task. We developed visual and audio wayfinding guidance on smartglasses based on de facto standard approaches for blind and sighted people and conducted a study with 16 low vision participants. We found that participants made fewer mistakes and experienced lower cognitive load with visual feedback. Moreover, participants with a full field of view completed the wayfinding tasks faster when using visual feedback. However, many participants preferred audio feedback because of its shorter learning curve. We propose design guidelines for wayfinding systems for low vision.

Cross-lingual word embeddings transfer knowledge between languages: models trained on high-resource languages can predict in low-resource languages.We introduce CLIME, an interactive system to quickly refine cross-lingual word embeddings for a given classification problem.First, CLIME ranks words by their salience to the downstream task.Then, users mark similarity between keywords and their nearest neighbors in the embedding space.Finally, CLIME updates the embeddings using the annotations.We evaluate CLIME on identifying health-related text in four low-resource languages: Ilocano, Sinhalese, Tigrinya, and Uyghur.Embeddings refined by CLIME capture more nuanced word semantics and have higher test accuracy than the original embeddings.CLIME often improves accuracy faster than an active learning baseline and can be easily combined with active learning to improve results.

We explore predictability and control in interactive systems where controls are easy to validate. Human-in-the-loop techniques allow users to guide unsupervised algorithms by exposing and supporting interaction with underlying model representations, increasing transparency and promising fine-grained control. However, these models must balance user input and the underlying data, meaning they sometimes update slowly, poorly, or unpredictably---either by not incorporating user input as expected (adherence) or by making other unexpected changes (instability). While prior work exposes model internals and supports user feedback, less attention has been paid to users' reactions when transparent models limit control. Focusing on interactive topic models, we explore user perceptions of control using a study where 100 participants organize documents with one of three distinct topic modeling approaches. These approaches incorporate input differently, resulting in varied adherence, stability, update speeds, and model quality. Participants disliked slow updates most, followed by lack of adherence. Instability was polarizing: some participants liked it when it surfaced interesting information, while others did not. Across modeling approaches, participants differed only in whether they noticed adherence.

Text representations are critical for modern natural language processing. One form of text representation, sense-specific embeddings, reflect a word’s sense in a sentence better than single-prototype word embeddings tied to each type. However, existing sense representations are not uniformly better: although they work well for computer-centric evaluations, they fail for human-centric tasks like inspecting a language’s sense inventory. To expose this discrepancy, we propose a new coherence evaluation for sense embeddings. We also describe a minimal model (Gumbel Attention for Sense Induction) optimized for discovering interpretable sense representations that are more coherent than existing sense embeddings.

This panel will provide practical advice on what and how to teach accessibility in a variety of settings. In this context, teaching accessibility means teaching about computer technologies that people with various disabilities can use and be productive with. At the undergraduate level it could mean teaching about how to design and build accessible web sites and applications. At the graduate level it could be teaching about building applications that can help people with disabilities with specific tasks. An entire course could focus on accessibility or it could be just part of an existing course. It is also important to learn about the diversity of consumers of technologies: what their abilities are and what access infrastructures they use every day. All the panelists have extensive experience in teaching accessibility. They will provide the audience of the panel deep insights into what they might do to teach accessibility in their own courses.

Voice assistants embodied in smart speakers (e.g., Amazon Echo, Google Home) enable voice-based interaction that does not necessarily rely on expertise with mobile or desktop computing. Hence, these voice assistants offer new opportunities to different populations, including individuals who are not interested or able to use traditional computing devices such as computers and smartphones. To understand how older adults who use technology infrequently perceive and use these voice assistants, we conducted a 3-week field deployment of the Amazon Echo Dot in the homes of seven older adults. While some types of usage dropped over the 3-week period (e.g., playing music), we observed consistent usage for finding online information. Given that much of this information was health-related, this finding emphasizes the need to revisit concerns about credibility of information with this new interaction medium. Although features to support memory (e.g., setting timers, reminders) were initially perceived as useful, the actual usage was unexpectedly low due to reliability concerns. We discuss how these findings apply to other user groups along with design implications and recommendations for future work on voice-user interfaces.

To address the lack of comparative evaluation of Human-in-the-Loop Topic Modeling (HLTM) systems, we implement and evaluate three contrasting HLTM modeling approaches using simulation experiments.These approaches extend previously proposed frameworks, including constraints and informed prior-based methods.Users should have a sense of control in HLTM systems, so we propose a control metric to measure whether refinement operations' results match users' expectations.Informed prior-based methods provide better control than constraints, but constraints yield higher quality topics.

A gender stereotype that has some basis in research is that men are more reluctant to ask for directions than women. We wanted to investigate whether this stereotype applies to technology-related contexts, affecting older adults' abilities to learn new technologies. To explore how help seeking and gender might relate for older adults, we conducted a controlled experiment with 36 individuals, of whom 18 identified as men and 18 identified as women, and observed how often they asked for help when learning new applications. We also conducted post-experiment interviews with participants. We found that although most participants stereotyped older men as being reluctant to ask for help in the interview, the gender difference was minimal in the experiment. Instead, individual differences had a greater effect: older participants took longer to complete tasks and participants with lower technology self-efficacy asked significantly more questions.

To understand whether people can identify their optimal touchscreen target sizes, we asked older and younger adults to identify optimal target sizes on a questionnaire and compared these chosen sizes to performance on a target acquisition task. We found that older individuals (60+) were better than younger adults at choosing their optimal target sizes. In fact, younger adults underestimated the smallest target size they could accurately touch by almost 6mm. This study suggests that older adults may be able to better configure target size settings than younger adults.

To investigate how older adults perceive ability changes (e.g., sensory, physical, cognitive) and how attitudes toward those changes affect perception and adoption of built-in mobile accessibility features (such as those found on Apple iOS and Google Android smartphones and tablets), we conducted an interview study with 14 older adults and six of their family members. Accessibility features were difficult for participants to find and configure, which were issues compounded by a reluctance to use trial-and-error. At 4-6 weeks after the interview, however, some participants had adopted new accessibility features that we had showed them, suggesting a willingness to adopt once features are made visible. The older adults who did already use accessibility features had experienced a disability earlier in life, suggesting that those experiencing progressive ability changes later in life might not be as aware of accessibility features, or might not have the know-how to adapt technologies to their changing needs. Our findings provide support for creating technologies that can detect older adults' abilities and recommend or enact interface changes to match.

Travel experiences offer a diverse view into an individual's interactions with different cultures, societies, and places. In this paper, we present a 2.5-year autoethnographic travel account of a hard of hearing individual-Jain. Through retrospective journals and field notes, we reveal the tensions and nuances in his travel, including the magnified difficulty of social conversations, issues with navigating unfamiliar environments and cultural contexts, and changes in the relationship to personal assistive technologies. By exploring the longitudinal travel experiences of a single individual, we uncover evocative and personal insights rarely available through participant-based research methods. Based on these lived experiences and post hoc reflections, we present two design explorations of personalized technology the autoethnographer created for aiding his travel. Finally, we offer reflections for customized travel technologies for deaf and hard of hearing users, and methodological guidelines for performing first-person research in the context of disability.

Smartwatches present inherent input difficulties due to the small touchscreen. In a controlled experiment with 14 participants with upper body motor impairments, we compared smartwatch touchscreen input to input on the bezel of the watch, the latter of which should at least theoretically stabilize user input due to its hard edge. Results demonstrate a speed-accuracy tradeoff whereby the touchscreen is faster but the bezel is more accurate.

People with visual impairments often rely on screen readers when interacting with computer systems. Increasingly, these individuals also make extensive use of voice-based virtual assistants (VAs). We conducted a survey of 53 people who are legally blind to identify the strengths and weaknesses of both technologies, and the unmet opportunities at their intersection. We learned that virtual assistants are convenient and accessible, but lack the ability to deeply engage with content (e.g., read beyond the first few sentences of an article), and the ability to get a quick overview of the landscape (e.g., list alternative search results and suggestions). In contrast, screen readers allow for deep engagement with content (when content is accessible), and provide fine-grained navigation and control, but at the cost of reduced walk-up-and-use convenience. Based on these findings, we implemented VERSE (Voice Exploration, Retrieval, and SEarch), a prototype that extends a VA with screen-reader-inspired capabilities, and allows other devices (e.g., smartwatches) to serve as optional input accelerators. In a usability study with 12 blind screen reader users we found that VERSE meaningfully extended VA functionality. Participants especially valued having access to multiple search results and search verticals.

To investigate preferences for mobile and wearable sound awareness systems, we conducted an online survey with 201 DHH participants. The survey explores how demographic factors affect perceptions of sound awareness technologies, gauges interest in specific sounds and sound characteristics, solicits reactions to three design scenarios (smartphone, smartwatch, head-mounted display) and two output modalities (visual, haptic), and probes issues related to social context of use. While most participants were highly interested in being aware of sounds, this interest was modulated by communication preference--that is, for sign or oral communication or both. Almost all participants wanted both visual and haptic feedback and 75% preferred to have that feedback on separate devices (e.g., haptic on smartwatch, visual on head-mounted display). Other findings related to sound type, full captions vs. keywords, sound filtering, notification styles, and social context provide direct guidance for the design of future mobile and wearable sound awareness systems.

The home is filled with a rich diversity of sounds from mundane beeps and whirs to dog barks and children's shouts. In this paper, we examine how deaf and hard of hearing (DHH) people think about and relate to sounds in the home, solicit feedback and reactions to initial domestic sound awareness systems, and explore potential concerns. We present findings from two qualitative studies: in Study 1, 12 DHH participants discussed their perceptions of and experiences with sound in the home and provided feedback on initial sound awareness mockups. Informed by Study 1, we designed three tablet-based sound awareness prototypes, which we evaluated with 10 DHH participants using a Wizard-of-Oz approach. Together, our findings suggest a general interest in smarthome-based sound awareness systems particularly for displaying contextually aware, personalized and glanceable visualizations but key concerns arose related to privacy, activity tracking, cognitive overload, and trust.

People with visual impairments often rely on screen readers when interacting with computer systems. Increasingly, these individuals also make extensive use of voice-based virtual assistants (VAs). We conducted a survey of 53 people who are legally blind to identify the strengths and weaknesses of both technologies, as well as the unmet opportunities at their intersection. We learned that virtual assistants are convenient and accessible, but lack the ability to deeply engage with content (e.g., read beyond the first few sentences of Wikipedia), and the ability to get a quick overview of the landscape (list alternative search results & suggestions). In contrast, screen readers allow for deep engagement with content (when content is accessible), and provide fine-grained navigation & control, but at the cost of increased complexity, and reduced walk-up-and-use convenience. In this demonstration, we showcase VERSE, a system that combines the positive aspects of VAs and screen readers, and allows other devices (e.g., smart watches) to serve as optional input accelerators. Together, these features allow people with visual impairments to deeply engage with web content through voice interaction.

As voice-based conversational agents such as Amazon Alexa and Google Assistant move into our homes, researchers have studied the corresponding privacy implications, embeddedness in these complex social environments, and use by specific user groups. Yet it is unknown how users categorize these devices: are they thought of as just another object, like a toaster? As a social companion? Though past work hints to human-like attributes that are ported onto these devices, the anthropomorphization of voice assistants has not been studied in depth. Through a study deploying Amazon Echo Dot Devices in the homes of older adults, we provide a preliminary assessment of how individuals 1) perceive having social interactions with the voice agent, and 2) ontologically categorize the voice assistants. Our discussion contributes to an understanding of how well-developed theories of anthropomorphism apply to voice assistants, such as how the socioemotional context of the user (e.g., loneliness) drives increased anthropomorphism. We conclude with recommendations for designing voice assistants with the ontological category in mind, as well as implications for the design of technologies for social companionship for older adults.

Prior work has explored communication challenges faced by people who are deaf and hard of hearing (DHH) and the potential role of new captioning and support technologies to address these challenges; however, the focus has been on stationary contexts such as group meetings and lectures. In this paper, we present two studies examining the needs of DHH people in moving contexts (e.g., walking) and the potential for mobile captions on head-mounted displays (HMDs) to support those needs. Our formative study with 12 DHH participants identifies social and environmental challenges unique to or exacerbated by moving contexts. Informed by these findings, we introduce and evaluate a proof-of-concept HMD prototype with 10 DHH participants. Results show that, while walking, HMD captions can support communication access and improve attentional balance between the speakers(s) and navigating the environment. We close by describing open questions in the mobile context space and design guidelines for future technology.

Augmented reality (AR) systems that enhance visual capabilities could make text and other fine details more accessible for low vision users, improving independence and quality of life. Prior work has begun to investigate the potential of assistive AR, but recent advancements enable new AR visualizations and interactions not yet explored in the context of assistive technology. In this paper, we follow an iterative design process with feedback and suggestions from seven visually impaired participants, designing and testing AR magnification ideas using the Microsoft HoloLens. Participants identified several advantages to the concept of head-worn magnification (e.g., portability, privacy, ready availability), and to our AR designs in particular (e.g., a more natural reading experience and the ability to multitask). We discuss the strengths and weaknesses of this AR magnification approach and summarize lessons learned throughout the process.

Color identification tools do not identify visual patterns or allow users to quickly inspect multiple locations, which are both important for identifying clothing. We are exploring the use of a finger-based camera that allows users to query clothing colors and patterns by touch. Previously, we demonstrated the feasibility of this approach using a small, highly-controlled dataset and combining two image classification techniques commonly used for object recognition. Here, to improve scalability and robustness, we collect a dataset of fabric images from online sources and apply transfer learning to train an end-to-end deep neural network to recognize visual patterns. This new approach achieves 92% accuracy in a general case and 97% when tuned for images from a finger-mounted camera.

Speech has become an increasingly common means of text input, from smartphones and smartwatches to voice-based intelligent personal assistants. However, reviewing the recognized text to identify and correct errors is a challenge when no visual feedback is available. In this paper, we first quantify and describe the speech recognition errors that users are prone to miss, and investigate how to better support this error identification task by manipulating pauses between words, speech rate, and speech repetition. To achieve these goals, we conducted a series of four studies. Study 1, an in-lab study, showed that participants missed identifying over 50% of speech recognition errors when listening to audio output of the recognized text. Building on this result, Studies 2 to 4 were conducted using an online crowdsourcing platform and showed that adding a pause between words improves error identification compared to no pause, the ability to identify errors degrades with higher speech rates (300 WPM), and repeating the speech output does not improve error identification. We derive implications for the design of audio-only speech dictation.

Smartwatches are always-available, provide quick access to information in a mobile setting, and can collect continuous health and fitness data. However, the small interaction space of these wearables may pose challenges for people with upper body motor impairments. To investigate accessible smartwatch interactions for this user group, we conducted two studies. First, we assessed the accessibility of existing smartwatch gestures with 10 participants with motor impairments. We found that not all participants were able to complete button, swipe and tap interactions. In a second study, we adopted a participatory approach to explore smartwatch gesture preferences and to gain insight into alternative, more accessible smartwatch interaction techniques. Eleven participants with motor impairments created gestures for 16 common smartwatch actions on both touchscreen and non-touchscreen (bezel, wristband) areas of the watch and the user's body. We present results from both studies and provide design recommendations.

From an accessibility perspective, voice-controlled, home-based intelligent personal assistants (IPAs) have the potential to greatly expand speech interaction beyond dictation and screen reader output. To examine the accessibility of off-the-shelf IPAs (e.g., Amazon Echo) and to understand how users with disabilities are making use of these devices, we conducted two exploratory studies. The first, broader study is a content analysis of 346 Amazon Echo reviews that include users with disabilities, while the second study more specifically focuses on users with visual impairments, through interviews with 16 current users of home-based IPAs. Findings show that, although some accessibility challenges exist, users with a range of disabilities are using the Amazon Echo, including for unexpected cases such as speech therapy and support for caregivers. Richer voice-based applications and solutions to support discoverability would be particularly useful to users with visual impairments. These findings should inform future work on accessible voice-based IPAs.

Human-in-the-loop topic modeling allows users to guide the creation of topic models and to improve model quality without having to be experts in topic modeling algorithms. Prior work in this area has focused either on algorithmic implementation without understanding how users actually wish to improve the model or on user needs but without the context of a fully interactive system. To address this disconnect, we implemented a set of model refinements requested by users in prior work and conducted a study with twelve non-expert participants to examine how end users are affected by issues that arise with a fully interactive, user-centered system. As these issues mirror those identified in interactive machine learning more broadly, such as unpredictability, latency, and trust, we also examined interactive machine learning challenges with non-expert end users through the lens of human-in-the-loop topic modeling. We found that although users experience unpredictability, their reactions vary from positive to negative, and, surprisingly, we did not find any cases of distrust, but instead noted instances where users perhaps trusted the system too much or had too little confidence in themselves.

Tactile overlays with audio annotations can increase the accessibility of touchscreens for blind users; however, preparing these overlays is complex and labor intensive. We introduce TacTILE, a novel toolchain to more easily create tactile overlays with audio annotations for arbitrary touchscreen graphics (e.g., graphs, pictures, maps). The workflow includes: (i) an annotation tool to add audio to graphical elements, (ii) a fabrication process that generates 3D-printed tactile overlays, and (iii) a custom app for the user to explore graphics with these overlays. We close with a pilot study with one blind participant who explores three examples (floor plan, photo, and chart), and a discussion of future work.

Precisely guiding a blind person's hand can be useful for a range of applications from tracing printed text to learning and understanding shapes and gestures. In this paper, we evaluate wrist-worn haptics as a directional hand guide. We implemented and evaluated the following haptic wristband variations: (1) four versus eight vibromotor designs; (2) vibration from only a single motor at a time versus from two adjacent motors using interpolation. To evaluate our designs, we conducted two studies: Study 1 (N=13, 2 blind) showed that participants could non-visually find targets and trace paths more quickly and accurately with single-motor feedback than with interpolated feedback, particularly when only four motors were used. Study 2 (N=14 blind or visually impaired participants) found that single-motor feedback with four motors was faster, more accurate, and most preferred compared to similar feedback with eight motors. We derive implications for the design of wrist-worn directional haptic feedback and discuss future work.

We investigate clothing color and visual texture recognition using images from a finger-mounted camera to support people with visual impairments. Our approach mitigates issues with distance and lighting that can impact the accuracy of existing color and texture recognizers and allows for easy touch-based interrogation to better understand clothing appearance. We classify image textures by combining two off-the-shelf techniques commonly used for object recognition achieving 99.4% accuracy on a dataset of 520 clothing images across 9 texture categories. We close with a discussion of potential applications, user evaluation plans, and open questions.

For screenreader users who are blind or visually impaired (VI), today's mobile devices, while reasonably accessible, are not necessarily efficient. This inefficiency may be especially problematic for microinteractions, which are brief but high-frequency interactions that take only a few seconds for sighted users to complete (e.g., checking the weather or for new messages). One potential solution to support efficient non-visual microinteractions is on-body input, which appropriates the user's own body as the interaction medium. In this paper, we address two related research questions: How well are microinteractions currently supported for VI users' How should on-body interaction be designed to best support microinteractions for this user group? We conducted two studies: (1) an online survey to compare current microinteraction use between VI and sighted users (N=117); and (2) an in-person study where 12 VI screenreader users qualitatively evaluated a real-time on-body interaction system that provided three contrasting input designs. Our findings suggest that efficient microinteractions are not currently well-supported for VI users, at least using manual input, which highlights the need for new interaction approaches. On-body input offers this potential and the qualitative evaluation revealed tradeoffs with different on-body interaction techniques in terms of perceived efficiency, learnability, social acceptability, and ability to use on the go.

Recent technical advances have enabled new wearable augmented reality (AR) solutions that can aid people with visual impairments (VI) in their everyday lives. Here, we investigate an AR-based magnification solution that combines a small finger-worn camera with a transparent augmented reality display (the Microsoft Hololens). The image from the camera is processed and projected on the Hololens to magnify visible content below the user's finger such as text and images. Our approach offers: (i) a close-up camera view (similar to a CCTV system) with the portability and processing power of a smartphone magnifier app, (ii) access to content through direct touch, and (iii) flexible placement of the magnified image within the wearer's field of view. We present three proof-of-concept interfaces and plans for a user evaluation.

Controlled studies of touchscreen input performance for users with upper body motor impairments remain relatively sparse. To address this gap, we present a controlled lab study of mouse vs. touchscreen performance with 32 participants (16 with upper body motor impairments and 16 without). Our study examines: (1) how touch input compares to an indirect pointing device (a mouse); (2) how performance compares across a range of standard interaction techniques; and (3) how these answers differ for users with and without motor impairments. While the touchscreen was faster than the mouse overall, only participants without motor impairments benefited from a lower error rate on the touchscreen. Indeed, participants with motor impairments had a three-fold increase in pointing (tapping) errors on the touchscreen compared to the mouse. Our findings also highlight the high frequency of spurious touches for users with motor impairments and update past accessibility recommendations for minimum touchscreen target sizes to at least 18mm.

Research on the viability of using crowdsourcing for HCI performance experiments has concluded that online results are similar to those achieved in the lab---at least for desktop interactions. However, mobile devices, the most popular form of online access today, may be more problematic due to variability in the user's posture and in movement of the device. To assess this possibility, we conducted two experiments with 30 lab-based and 303 crowdsourced participants using basic mouse and touchscreen tasks. Our findings show that: (1) separately analyzing the crowd and lab data yields different study conclusions-touchscreen input was significantly less error prone than mouse input in the lab but more error prone online; (2) age-matched crowdsourced participants were significantly faster and less accurate than their lab-based counterparts, contrasting past work; (3) variability in mobile device movement and orientation increased as experimenter control decreased--a potential factor affecting the touchscreen error differences. This study cautions against assuming that crowdsourced data for performance experiments will directly reflect lab-based data, particularly for mobile devices.

Topic modeling is a common tool for understanding large bodies of text, but is typically provided as a “take it or leave it” proposition. Incorporating human knowledge in unsupervised learning is a promising approach to create high-quality topic models. Existing interactive systems and modeling algorithms support a wide range of refinement operations to express feedback. However, these systems’ interactions are primarily driven by algorithmic convenience, ignoring users who may lack expertise in topic modeling. To better understand how non-expert users understand, assess, and refine topics, we conducted two user studies—an in-person interview study and an online crowdsourced study. These studies demonstrate a disconnect between what non-expert users want and the complex, low-level operations that current interactive systems support . In particular, our findings include: (1) analysis of how non-expert users perceive topic models; (2) characterization of primary refinement operations expected by non-expert users and ordered by relative preference; (3) further evidence of the benefits of supporting users in directly refining a topic model; (4) design implications for future human-in-the-loop topic modeling interfaces.

Probabilistic topic models are important tools for indexing, summarizing, and analyzing large document collections by their themes. However, promoting end-user understanding of topics remains an open research problem. We compare labels generated by users given four topic visualization techniques—word lists, word lists with bars, word clouds, and network graphs—against each other and against automatically generated labels. Our basis of comparison is participant ratings of how well labels describe documents from the topic. Our study has two phases: a labeling phase where participants label visualized topics and a validation phase where different participants select which labels best describe the topics’ documents. Although all visualizations produce similar quality labels, simple visualizations such as word lists allow participants to quickly understand topics, while complex visualizations take longer but expose multi-word expressions that simpler visualizations obscure. Automatic labels lag behind user-created labels, but our dataset of manually labeled topics highlights linguistic patterns (e.g., hypernyms, phrases) that can be used to improve automatic topic labeling algorithms.

Effective text classification requires experts to annotate data with labels; these training data are time-consuming and expensive to obtain.If you know what labels you want, active learning can reduce the number of labeled documents needed.However, establishing the label set remains difficult.Annotators often lack the global knowledge needed to induce a label set.We introduce ALTO: Active Learning with Topic Overviews, an interactive system to help humans annotate documents: topic models provide a global overview of what labels to create and active learning directs them to the right documents to label.Our forty-annotator user study shows that while active learning alone is best in extremely resource limited conditions, topic models (even by themselves) lead to better label sets, and ALTO's combination is best overall.

Current symbol-based dictionaries providing vocabulary support for persons with the language disorder, aphasia, are housed on smartphones or other portable devices. To employ the support on these external devices requires the user to divert their attention away from their conversation partner, to the neglect of conversation dynamics like eye contact or verbal inflection. A prior study investigated head-worn displays (HWDs) as an alternative form factor for supporting glanceable, unobtrusive, and always-available conversation support, but it did not directly compare the HWD to a control condition. To address this limitation, we compared vocabulary support on a HWD to equivalent support on a smartphone in terms of overall experience, perceived focus, and conversational success. Lastly, we elicited critical discussion of how each device might be better designed for conversation support. Our work contributes (1) evidence that a HWD can support more efficient communication, (2) preliminary results that a HWD can provide a better overall experience using assistive vocabulary, and (3) a characterization of the design features persons with aphasia value in portable conversation support technologies. Our findings should motivate further work on head-worn conversation support for persons with aphasia.

Wearable computing devices offer new possibilities to increase accessibility and independence for individuals with disabilities. However, the adoption of such devices may be influenced by social factors, and useful devices may not be adopted if they are considered inappropriate to use. While public policy may adapt to support accommodations for assistive technology, emerging technologies may be unfamiliar or unaccepted by bystanders. We surveyed 1200 individuals about the use of a head-mounted display in a public setting, examining how information about the user's disability affected judgments of the social acceptability of the scenario. Our findings reveal that observers considered head-mounted display use more socially acceptable if the device was being used to support a person with a disability.

Skin-based biometrics rely on the distinctiveness of skin patterns across individuals for identification. In this paper, we investigate whether small image patches of the skin can be localized on a user's body, determining not “who?” instead “where?” Applying techniques from biometrics and computer vision, we introduce a hierarchical classifier that estimates a location from the image texture and refines the estimate with keypoint matching and geometric verification. To evaluate our approach, we collected 10,198 close-up images of 17 hand and wrist locations across 30 participants. Within-person algorithmic experiments demonstrate that an individual's own skin features can be used to localize their skin surface image patches with an F1score of 96.5%. As secondary analyses, we assess the effects of training set size and between-person classification. We close with a discussion of the strengths and limitations of our approach and evaluation methods as well as implications for future applications using a wearable camera to support touch-based, location-specific taps and gestures on the surface of the skin.

The HandSight project investigates how wearable micro-cameras can be used to augment a blind or visually impaired user--s sense of touch with computer vision. Our goal is to support an array of activities of daily living by sensing and feeding back non-tactile information (e.g., color, printed text, patterns) about an object as it is touched. In this poster paper, we provide an overview of the project, our current proof-of-concept prototype, and a summary of findings from finger-based text reading studies. As this is an early-stage project, we also enumerate current open questions.

Persons with hearing loss use visual signals such as gestures and lip movement to interpret speech. While hearing aids and cochlear implants can improve sound recognition, they generally do not help the wearer localize sound necessary to leverage these visual cues. In this paper, we design and evaluate visualizations for spatially locating sound on a head-mounted display (HMD). To investigate this design space, we developed eight high-level visual sound feedback dimensions. For each dimension, we created 3-12 example visualizations and evaluated these as a design probe with 24 deaf and hard of hearing participants (Study 1). We then implemented a real-time proof-of-concept HMD prototype and solicited feedback from 4 new participants (Study 2). Study 1 findings reaffirm past work on challenges faced by persons with hearing loss in group conversations, provide support for the general idea of sound awareness visualizations on HMDs, and reveal preferences for specific design options. Although preliminary, Study 2 further contextualizes the design probe and uncovers directions for future work.

Head-mounted displays provide relatively hands-free interaction that could improve mobile computing access for users with motor impairments. To investigate this largely unexplored area, we present two user studies. The first, smaller study evaluated the accessibility of Google Glass, a head-mounted display, with 6 participants. Findings revealed potential benefits of a head-mounted display yet demonstrated the need for alternative means of controlling Glass-3 of the 6 participants could not use it at all. We then conducted a second study with 12 participants to evaluate a potential alternative input mechanism that could allow for accessible control of a head-mounted display: switch-based wearable touchpads that can be affixed to the body or wheelchair. The study assessed input performance with three sizes of touchpad, investigated personalization patterns when participants were asked to place the touchpads on their body or wheelchair, and elicited subjective responses. All 12 participants were able to use the touchpads to control the display, and patterns of touchpad placement point to the value of personalization in providing support for each user's motor abilities.

Symbol-based dictionaries of text, images and sound can help individuals with aphasia find the words they need, but are often seen as a last resort because they tend to replace rather than augment the user's natural speech. Through two design investigations, we explore head-worn displays as a means of providing unobtrusive, always-available, and glanceable vocabulary support. The first study used narrative storyboards as a design probe to explore the potential benefits and challenges of a head-worn approach over traditional augmented alternative communication (AAC) tools. The second study then evaluated a proof-of-concept prototype in both a lab setting with the researcher and in situ with unfamiliar conversation partners at a local market. Findings suggest that a head-worn approach could better allow wearers to maintain focus on the conversation, reduce reliance on the availability of external tools (e.g., paper and pen) or people, and minimize visibility of the support by others. These studies should motivate further investigation of head-worn conversational support.

The present study used a flat keyboard without moving keys and enabled with haptic keyclick feedback to examine the effect of haptic keyclick feedback on touch typing performance. We investigated, with well-controlled stimuli and a within-participant design, how haptic keyclick feedback might improve typing performance in terms of typing speed, typing efficiency and typing errors. Of the three kinds of haptic feedback we tested, all increased typing speed and decreased typing errors compared to a condition without haptic feedback. We did not find significant differences among the types of haptic feedback. We also found that auditory keyclick feedback alone is not as effective as haptic keyclick feedback, and the addition of auditory feedback to haptic feedback does not lead to any significant improvement in typing performance. We also learned that global haptic keyclick feedback simulated through local keyclick feedback on each key (as opposed to haptic feedback all over the keyboard) might have the additional and unexpected benefit of helping a typist to locate keys on a keyboard. Furthermore, the participants preferred auditory or haptic keyclick feedback to no feedback, and haptic feedback restricted to the typing finger alone is preferred to that over a larger area of the keyboard.

Physical movement has the potential to enhance learning activities. To investigate how movement can be incorporated into children's mobile language learning, we designed and evaluated two versions of a German vocabulary game called Scenic Words. The first version used movementbased dynamic peephole navigation, which requires physical movement of the arms, while the second version used touchbased static peephole navigation, which only requires standard touchscreen interactions; static peepholes are the status quo interaction technique for navigation, commonly found, for example, in map applications and games. To compare the two types of navigation and to assess children's reactions to dynamic peepholes, we conducted an inhome study with 16 children (ages 89). The children participated in pairs but individually played each version of the game on a mobile device. While results showed that the more familiar static peepholes were the preferred interaction style overall, participants became accustomed to the movementbased dynamic peepholes during the study. Participants noted that the dynamic peephole interaction became easier over time, and that it had some advantages such as for dragginganddropping elements in the game.

Creating a predefined set of touchscreen gestures that caters to all users and age groups is difficult. To inform the design of intuitive and easy to use gestures specifically for children, we adapted a userdefined gesture study by Wobbrock et al. [12] that had been designed for adults. We then compared gestures created on an interactive tabletop by 12 children and 14 adults. Our study indicates that previous touchscreen experience strongly influences the gestures created by both groups; that adults and children create similar gestures; and that the adaptations we made allowed us to successfully elicit userdefined gestures from both children and adults. These findings will aid designers in better supporting touchscreen gestures for children, and provide a basis for further userdefined gesture studies with children.

Head-mounted displays such as Google Glass offer potential advantages for persons with motor impairments (MI). For example, they are always available and offer relatively hands-free interaction compared to a mobile phone. Despite this potential, there is little prior work examining the accessibility of such devices. In this poster paper, we perform a preliminary assessment of the accessibility of Google Glass for users with MI and the potential impacts of a head-mounted interactive computer. Our findings show that, while the touchpad is particularly difficult to use-impossible for three participants-advantages over a phone include that it is relatively hands free, does not require looking down at the display, and cannot be easily dropped.

Lab-based studies on touchscreen use by people with motor impairments have identified both positive and negative impacts on accessibility. Little work, however, has moved beyond the lab to investigate the truly mobile experiences of users with motor impairments. We conducted two studies to investigate how smartphones are being used on a daily basis, what activities they enable, and what contextual challenges users are encountering. The first study was a small online survey with 16 respondents. The second study was much more in depth, including an initial interview, two weeks of diary entries, and a 3-hour contextual session that included neighborhood activities. Four expert smartphone users participated in the second study and we used a case study approach for analysis. Our findings highlight the ways in which smartphones are enabling everyday activities for people with motor impairments, particularly in overcoming physical accessibility challenges in the real world and supporting writing and reading. We also identified important situational impairments, such as the inability to retrieve the phone while in transit, and confirmed many lab-based findings in the real-world setting. We present design implications and directions for future work.

For users with visual impairments, who do not necessarily need the visual display of a mobile device, non-visual on-body interaction (e.g., Imaginary Interfaces) could provide accessible input in a mobile context. Such interaction provides the potential advantages of an always-available input surface, and increased tactile and proprioceptive feedback compared to a smooth touchscreen. To investigate preferences for and design of accessible on-body interaction, we conducted a study with 12 visually impaired participants. Participants evaluated five locations for on-body input and compared on-phone to on-hand interaction with one versus two hands. Our findings show that the least preferred areas were the face/neck and the forearm, while locations on the hands were considered to be more discreet and natural. The findings also suggest that participants may prioritize social acceptability over ease of use and physical comfort when assessing the feasibility of input at different locations of the body. Finally, tradeoffs were seen in preferences for touchscreen versus on-body input, with on-body input considered useful for contexts where one hand is busy (e.g., holding a cane or dog leash). We provide implications for the design of accessible on-body input.

With the increasing popularity of mainstream wearable devices, it is critical to assess the accessibility implications of such technologies. For people with visual impairments, who do not always need the visual display of a mobile phone, alternative means of eyes-free wearable interaction are particularly appealing. To explore the potential impacts of such technology, we conducted two studies. The first was an online survey that included 114 participants with visual impairments and 101 sighted participants; we compare the two groups in terms of current device use. The second was an interview and design probe study with 10 participants with visual impairments. Our findings expand on past work to characterize a range of trends in smartphone use and accessibility issues therein. Participants with visual impairments also responded positively to two eyes-free wearable device scenarios: a wristband or ring and a glasses-based device. Discussions on projected use of these devices suggest that small, easily accessible, and discreet wearable input could positively impact the ability of people with visual impairments to access information on the go and to participate in certain social interactions.

Touchscreen devices have become one of the most pervasive video game platforms in the world and, in turn, an integral part of popular culture; however, little work exists on comprehensively examining their accessibility. In this poster paper, we present initial findings from a survey and qualitative analysis of popular iPad touchscreen games with a focus on exploring factors relevant to persons with motor impairments. This paper contributes a novel qualitative codebook with which to examine the accessibility of touchscreen games for users with motor impairments and the results from applying this codebook to 72 iPad games.

While sighted users may learn to perform touchscreen gestures through observation (e.g., of other users or video tutorials), such mechanisms are inaccessible for users with visual impairments. As a result, learning to perform gestures can be challenging. We propose and evaluate two techniques to teach touchscreen gestures to users with visual impairments: (1) corrective verbal feedback using text-to-speech and automatic analysis of the user's drawn gesture; (2) gesture sonification to generate sound based on finger touches, creating an audio representation of a gesture. To refine and evaluate the techniques, we conducted two controlled lab studies. The first study, with 12 sighted participants, compared parameters for sonifying gestures in an eyes-free scenario and identified pitch + stereo panning as the best combination. In the second study, 6 blind and low-vision participants completed gesture replication tasks with the two feedback techniques. Subjective data and preliminary performance findings indicate that the techniques offer complementary advantages.

Most work on the usability of touchscreen interaction for people with motor impairments has focused on lab studies with relatively few participants and small cross-sections of the population. To develop a richer characterization of use, we turned to a previously untapped source of data: YouTube videos. We collected and analyzed 187 non-commercial videos uploaded to YouTube that depicted a person with a physical disability interacting with a mainstream mobile touchscreen device. We coded the videos along a range of dimensions to characterize the interaction, the challenges encountered, and the adaptations being adopted in daily use. To complement the video data, we also invited the video uploaders to complete a survey on their ongoing use of touchscreen technology. Our findings show that, while many people with motor impairments find these devices empowering, accessibility issues still exist. In addition to providing implications for more accessible touchscreen design, we reflect on the application of user-generated content to study user interface design.

Despite the apparent popularity of touchscreens for older adults, little is known about the psychomotor performance of these devices. We compared performance between older adults and younger adults on four desktop and touchscreen tasks: pointing, dragging, crossing and steering. On the touchscreen, we also examined pinch-to-zoom. Our results show that while older adults were significantly slower than younger adults in general, the touchscreen reduced this performance gap relative to the desktop and mouse. Indeed, the touchscreen resulted in a significant movement time reduction of 35% over the mouse for older adults, compared to only 16% for younger adults. Error rates also decreased.

The vast majority of work on understanding and supporting the gesture creation process has focused on professional designers. In contrast, gesture customization by end users' - which may offer better memorability, efficiency and accessibility than pre-defined gestures - has received little attention. To understand the end-user gesture creation process, we conducted a study where 20 participants were asked to: (1) exhaustively create new gestures for an open-ended use case; (2) exhaustively create new gestures for 12 specific use cases; (3) judge the saliency of different touchscreen gesture features. Our findings showed that even when asked to create novel gestures, participants tended to focus on the familiar. Misconceptions about the gesture recognizer's abilities were also evident, and in some cases constrained the range of gestures that participants created. Finally, as a calibration point for future research, we used a simple gesture recognizer ($N) to analyze recognition accuracy of the participants' custom gesture sets: accuracy was 68-88% on average, depending on the amount of training and the customization scenario. We conclude with implications for the design of a mixed-initiative approach to support custom gesture creation.

Although many techniques have been proposed to improve text input on touch screens, the vast majority of this research ignores non-alphanumeric input (i.e., punctuation, symbols, and modifiers). To support this input, widely adopted commercial touch-screen interfaces require mode switches to alternate keyboard layouts for most punctuation and symbols. Our approach is to augment existing ten-finger QWERTY keyboards with multi-touch gestural input that can exist as a complement to the moded-keyboard approach. To inform our design, we conducted a study to elicit user-defined gestures from 20 participants. The final gesture set includes both multi-touch and single-touch gestures for commonly used non-alphanumeric text input. We implemented and conducted a preliminary evaluation of a touch-screen keyboard augmented with this technique. Findings show that using gestures for non-alphanumeric input is no slower than using keys, and that users strongly prefer gestures to a moded-keyboard interface.

Although typing on touchscreens is slower than typing on physical keyboards, touchscreens offer a critical potential advantage: they are software-based, and, as such, the keyboard layout and classification models used to interpret key presses can dynamically adapt to suit each user's typing pattern. To explore this potential, we introduce and evaluate two novel personalized keyboard interfaces, both of which adapt their underlying key-press classification models. The first keyboard also visually adapts the location of keys while the second one always maintains a visually stable rectangular layout. A three-session user evaluation showed that the keyboard with the stable rectangular layout significantly improved typing speed compared to a control condition with no personalization. Although no similar benefit was found for the keyboard that also offered visual adaptation, overall subjective response to both new touchscreen keyboards was positive. As personalized keyboards are still an emerging area of research, we also outline a design space that includes dimensions of adaptation and key-press classification features.

Few means currently exist for home occupants to learn about their water consumption: e.g., where water use occurs, whether such use is excessive and what steps can be taken to conserve. Emerging water sensing systems, however, can provide detailed usage data at the level of individual water fixtures (i.e., disaggregated usage data). In this paper, we perform formative evaluations of two sets of novel eco-feedback displays that take advantage of this disaggregated data. The first display set isolates and examines specific elements of an eco-feedback design space such as data and time granularity. Displays in the second set act as design probes to elicit reactions about competition, privacy, and integration into domestic space. The displays were evaluated via an online survey of 651 North American respondents and in-home, semi-structured interviews with 10 families (20 adults). Our findings are relevant not only to the design of future water eco-feedback systems but also for other types of consumption (e.g., electricity and gas).

The lack of tactile feedback on touch screens makes typing difficult, a challenge exacerbated when situational impairments like walking vibration and divided attention arise in mobile settings. We introduce WalkType, an adaptive text entry system that leverages the mobile device's built-in tri-axis accelerometer to compensate for extraneous movement while walking. WalkType's classification model uses the displacement and acceleration of the device, and inference about the user's footsteps. Additionally, WalkType models finger-touch location and finger distance traveled on the screen, features that increase overall accuracy regardless of movement. The final model was built on typing data collected from 16 participants. In a study comparing WalkType to a control condition, WalkType reduced uncorrected errors by 45.2% and increased typing speed by 12.9% for walking participants.

Community health workers (CHWs) have been shown to be an effective and powerful intervention for improving community health. Routine visits, for example, can lower maternal and neonatal mortality rates. Despite these benefits, many challenges, including supervision and support, make CHW programs difficult to maintain. An increasing number of mHealth projects are providing CHWs with mobile phones to support their work, which opens up opportunities for real-time supervision of the program. Taking advantage of this potential, we evaluated the impact of SMS reminders to improve the promptness of routine CHW visits, first in a pilot study in Dodoma, Tanzania, followed by two larger studies with 87 CHWs in Dar es Salaam, Tanzania. The first Dar es Salaam study evaluated an escalating reminder system that sent SMS reminders directly to the CHW before notifying the CHW's supervisor after several overdue days. The reminders resulted in an 86% reduction in the average number of days a CHW's clients were overdue (9.7 to 1.4 days), with only a small number of cases ever escalating to the supervisor. However, when the step of escalating to the supervisor was removed in the second study, CHW performance significantly decreased.

Touch screen surfaces large enough for ten-finger input have become increasingly popular, yet typing on touch screens pales in comparison to physical keyboards. We examine typing patterns that emerge when expert users of physical keyboards touch-type on a flat surface. Our aim is to inform future designs of touch screen keyboards, with the ultimate goal of supporting touch-typing with limited tactile feedback. To study the issues inherent to flat-glass typing, we asked 20 expert typists to enter text under three conditions: (1) with no visual keyboard and no feedback on input errors, then (2) with and (3) without a visual keyboard, but with some feedback. We analyzed touch contact points and hand contours, looking at attributes such as natural finger positioning, the spread of hits among individual keys, and the pattern of non-finger touches. We also show that expert typists exhibit spatially consistent key press distributions within an individual, which provides evidence that eyes-free touch-typing may be possible on touch surfaces and points to the role of personalization in such a solution. We conclude with implications for design.

Nonparametric data from multi-factor experiments arise often in human-computer interaction (HCI). Examples may include error counts, Likert responses, and preference tallies. But because multiple factors are involved, common nonparametric tests (e.g., Friedman) are inadequate, as they are unable to examine interaction effects. While some statistical techniques exist to handle such data, these techniques are not widely available and are complex. To address these concerns, we present the Aligned Rank Transform (ART) for nonparametric factorial data analysis in HCI. The ART relies on a preprocessing step that "aligns" data before applying averaged ranks, after which point common ANOVA procedures can be used, making the ART accessible to anyone familiar with the F-test. Unlike most articles on the ART, which only address two factors, we generalize the ART to N factors. We also provide ARTool and ARTweb, desktop and Web-based programs for aligning and ranking data. Our re-examination of some published HCI results exhibits advantages of the ART.

Eco-feedback technology provides feedback on individual or group behaviors with a goal of reducing environmental impact. The history of eco-feedback extends back more than 40 years to the origins of environmental psychology. Despite its stated purpose, few HCI eco-feedback studies have attempted to measure behavior change. This leads to two overarching questions: (1) what can HCI learn from environmental psychology and (2) what role should HCI have in designing and evaluating eco-feedback technology? To help answer these questions, this paper conducts a comparative survey of eco-feedback technology, including 89 papers from environmental psychology and 44 papers from the HCI and UbiComp literature. We also provide an overview of predominant models of proenvironmental behaviors and a summary of key motivation techniques to promote this behavior.

Computer users with motor impairments face major challenges with conventional mouse pointing. These challenges are mostly due to fine pointing corrections at the final stages of target acquisition. To reduce the need for correction-phase pointing and to lessen the effects of small target size on acquisition difficulty, we introduce four enhanced area cursors, two of which rely on magnification and two of which use goal crossing. In a study with motor-impaired and able-bodied users, we compared the new designs to the point and Bubble cursors, the latter of which had not been evaluated for users with motor impairments. Two enhanced area cursors, the Visual-Motor-Magnifier and Click-and-Cross, were the most successful new designs for users with motor impairments, reducing selection time for small targets by 19%, corrective submovements by 45%, and error rate by up to 82% compared to the point cursor. Although the Bubble cursor also improved performance, participants with motor impairments unanimously preferred the enhanced area cursors.

Multimodal interfaces with little or no text have been shown to be useful for users with low literacy. However, this research has not differentiated between the needs of the fully illiterate and semiliterate - those who have basic literacy but cannot read and write fluently. Text offers a fast and unambiguous mode of interaction for literate users and the exposure to text may allow for incidental improvement of reading skills. We conducted two studies that explore how semiliterate users with very little education might benefit from a combination of text and audio as compared to illiterate and literate users. Results show that semiliterate users reduced their use of audio support even during the first hour of use and over several hours this reduction was accompanied by a gain in visual word recognition; illiterate users showed no similar improvement. Semiliterate users should thus be treated differently from illiterate users in interface design.

We introduce ephemeral adaptation, a new adaptive GUI technique that improves performance by reducing visual search time while maintaining spatial consistency. Ephemeral adaptive interfaces employ gradual onset to draw the user's attention to predicted items: adaptively predicted items appear abruptly when the menu is opened, but non-predicted items fade in gradually. To demonstrate the benefit of ephemeral adaptation we conducted two experiments with a total of 48 users to show: (1) that ephemeral adaptive menus are faster than static menus when accuracy is high, and are not significantly slower when it is low and (2) that ephemeral adaptive menus are also faster than adaptive highlighting. While we focused on user-adaptive GUIs, ephemeral adaptation should be applicable to a broad range of visually complex tasks.

Non-governmental organizations (NGOs) working in disadvantaged communities have a variety of data-collection and analysis needs, for example, for performing surveys or monitoring programs. Because much of this data collection occurs in environments with insufficient IT support and infrastructure, and among populations not always comfortable with technology, paper forms rather than electronic methods remain the predominant means for data collection.

Adaptive personalization, where the system adapts the interface to a user's needs, has the potential for significant performance benefits on small screen devices. However, research on adaptive interfaces has almost exclusively focused on desktop displays. To explore how well previous findings generalize to small screen devices, we conducted a study with 36 subjects to compare adaptive interfaces for small and desktop-sized screens. Results show that high accuracy adaptive menus have an even larger positive impact on performance and satisfaction when screen real estate is constrained. The drawback of the high accuracy menus, however, is that they reduce the user's awareness of the full set of items in the interface, potentially making it more difficult for users to learn about new features.

Coarse-grained approaches to customization allow the user to enable or disable groups of features at once, rather than individual features. While this may reduce the complexity of customization and encourage more users to customize, the research challenges of designing such approaches have not been fully explored. To address this limitation, we conducted an interview study with 14 professional software developers who use an integrated development environment that provides a role-based, coarse-grained approach to customization. We identify challenges of designing coarse-grained customization models, including issues of functionality partitioning, presentation, and individual differences. These findings highlight potentially critical design choices, and provide direction for future work.

Large classrooms have traditionally provided multiple blackboards on which an entire lecture could be visible. In recent decades, classrooms were augmented with a data projector and screen, allowing computer-generated slides to replace hand-written blackboard presentations and overhead transparencies as the medium of choice. Many lecture halls and conference rooms will soon be equipped with multiple projectors that provide large, high-resolution displays of comparable size to an old fashioned array of blackboards. The predominant presentation software, however, is still designed for a single medium-resolution projector. With the ultimate goal of designing rich presentation tools that take full advantage of increased screen resolution and real estate, we conducted an observational study to examine current practice with both traditional whiteboards and blackboards, and computer-generated slides. We identify several categories of observed usage, and highlight differences between traditional media and computer slides. We then present design guidelines for presentation software that capture the advantages of the old and the new and describe a working prototype based on those guidelines that more fully utilizes the capabilities of multiple displays.

Cooking is a daily activity for many people. However, traditional text recipes are often prohibitively difficult to follow for people with language disorders, such as aphasia. We have developed a multi-modal application that leverages the retained ability of aphasic individuals to recognize image-based representations of objects, providing a presentation format that can be more easily followed than a traditional text recipe. Through a systematic approach to developing a visual language for cooking, and the subsequent case study evaluation of a prototype developed according to this language, we show that a combination of visual instructions and navigational structure can help individuals with relatively large language deficits to cook more independently.

Software applications continue to grow in terms of the number of features they offer, making personalization increasingly important. Research has shown that most users prefer the control afforded by an adaptable approach to personalization rather than a system-controlled adaptive approach. No study, however, has compared the efficiency of the two approaches. In a controlled lab study with 27 subjects we compared the measured and perceived efficiency of three menu conditions: static, adaptable and adaptive. Each was implemented as a split menu, in which the top four items remained static, were adaptable by the subject, or adapted according to the subject's frequently and recently used items. The static menu was found to be significantly faster than the adaptive menu, and the adaptable menu was found to be significantly faster than the adaptive menu under certain conditions. The majority of users preferred the adaptable menu overall. Implications for interface design are discussed.

We describe a method for spatio-temporal data mining based on expected distribution domain generalization (ExGen) graphs. Using familiar calendar and geographical concepts, such as workdays, weeks, climatic regions, and countries, spatio-temporal data can be aggregated into summaries in many ways. We automatically search for a summary with a distribution that is anomalous, i.e., far from user expectations. We repeatedly ranked possible summaries according to current expectations, and then allow the user to adjust these expectations.