Recent advances in general-purpose AI underscore the urgent need to align AI systems with human goals and values. Yet, the lack of a clear, shared understanding of what constitutes "alignment" limits meaningful progress and cross-disciplinary collaboration. In this position paper, we argue that the research community should explicitly define and critically reflect on "alignment" to account for the bidirectional and dynamic relationship between humans and AI. Through a systematic review of over 400 papers spanning HCI, NLP, ML, and more, we examine how alignment is currently defined and operationalized. Building on this analysis, we introduce the Bidirectional Human-AI Alignment framework, which not only incorporates traditional efforts to align AI with human values but also introduces the critical, underexplored dimension of aligning humans with AI – supporting cognitive, behavioral, and societal adaptation to rapidly advancing AI technologies. Our findings reveal significant gaps in current literature, especially in long-term interaction design, human value modeling, and mutual understanding. We conclude with three central challenges and actionable recommendations to guide future research toward more nuanced, reciprocal, and human-AI alignment approaches.

The development of increasingly agentic and human-like AI assistants, capable of performing a wide range of tasks on user's behalf over time, has sparked heightened interest in the nature and bounds of human interactions with AI. Such systems may indeed ground a transition from task-oriented interactions with AI, at discrete time intervals, to ongoing relationships -- where users develop a deeper sense of connection with and attachment to the technology. This paper investigates what it means for relationships between users and advanced AI assistants to be appropriate and proposes a new framework to evaluate both users' relationships with AI and developers' design choices. We first provide an account of advanced AI assistants, motivating the question of appropriate relationships by exploring several distinctive features of this technology. These include anthropomorphic cues and the longevity of interactions with users, increased AI agency, generality and context ambiguity, and the forms and depth of dependence the relationship could engender. Drawing upon various ethical traditions, we then consider a series of values, including benefit, flourishing, autonomy and care, that characterise appropriate human interpersonal relationships. These values guide our analysis of how the distinctive features of AI assistants may give rise to inappropriate relationships with users. Specifically, we discuss a set of concrete risks arising from user--AI assistant relationships that: (1) cause direct emotional or physical harm to users, (2) limit opportunities for user personal development, (3) exploit user emotional dependence, and (4) generate material dependencies without adequate commitment to user needs. We conclude with a set of recommendations to address these risks.

A goal of accessible technology (AT) design is often to increase independence, i.e., to enable people with disabilities to accomplish tasks on their own without help. Recent work challenges this view by recognizing the role of ‘help’ in addressing the access needs of people with disabilities. However, empirical evidence examining help is limited to the Global North; we address this gap using a case study of how people with visual impairments (PVI) navigate indoor environments in India. Using interviews with PVI and their companions and a video-diary study, we find that help is a key practice that PVI use to navigate indoor environments. We uncover how help is a situated phenomenon shaped by socio-material and cultural factors unique to the Indian context. We discuss the value of help in the context of broader HCI and Accessibility literature on mixed-ability and collaborative interactions. We also discuss implications our findings on help have for AT design.

AI-generated images are proliferating as a new visual medium. However, state-of-the-art image generation models do not output alternative (alt) text with their images, rendering them largely inaccessible to screen reader users (SRUs). Moreover, less is known about what information would be most desirable to SRUs in this new medium. To address this, we invited AI image creators and SRUs to evaluate alt text prepared from various sources and write their own alt text for AI images. Our mixed-methods analysis makes three contributions. First, we highlight creators’ perspectives on alt text, as creators are well-positioned to write descriptions of their images. Second, we illustrate SRUs’ alt text needs particular to the emerging medium of AI images. Finally, we discuss the promises and pitfalls of utilizing text prompts written as input for AI models in alt text generation, and areas where broader digital accessibility guidelines could expand to account for AI images.

We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors. This framework introduces levels of AGI performance, generality, and autonomy, providing a common language to compare models, assess risks, and measure progress along the path to AGI. To develop our framework, we analyze existing definitions of AGI, and distill six principles that a useful ontology for AGI should satisfy. With these principles in mind, we propose “Levels of AGI” based on depth (performance) and breadth (generality) of capabilities, and reflect on how current systems fit into this ontology. We discuss the challenging requirements for future benchmarks that quantify the behavior and capabilities of AGI models against these levels. Finally, we discuss how these levels of AGI interact with deployment considerations such as autonomy and risk, and emphasize the importance of carefully selecting Human-AI Interaction paradigms for responsible and safe deployment of highly capable AI systems.

Presentation slides commonly use visual patterns for structural navigation, such as titles, dividers, and build slides. However, screen readers do not capture such intention, making it time-consuming and less accessible for blind and visually impaired (BVI) users to linearly consume slides with repeated content. We present Slide Gestalt, an automatic approach that identifies the hierarchical structure in a slide deck. Slide Gestalt computes the visual and textual correspondences between slides to generate hierarchical groupings. Readers can navigate the slide deck from the higher-level section overview to the lower-level description of a slide group or individual elements interactively with our UI. We derived side consumption and authoring practices from interviews with BVI readers and sighted creators and an analysis of 100 decks. We performed our pipeline with 50 real-world slide decks and a large dataset. Feedback from eight BVI participants showed that Slide Gestalt helped navigate a slide deck by anchoring content more efficiently, compared to using accessible slides.

Believable proxies of human behavior can empower interactive applications ranging from immersive environments to rehearsal spaces for interpersonal communication to prototyping tools. In this paper, we introduce generative agents: computational software agents that simulate believable human behavior. Generative agents wake up, cook breakfast, and head to work; artists paint, while authors write; they form opinions, notice each other, and initiate conversations; they remember and reflect on days past as they plan the next day. To enable generative agents, we describe an architecture that extends a large language model to store a complete record of the agent’s experiences using natural language, synthesize those memories over time into higher-level reflections, and retrieve them dynamically to plan behavior. We instantiate generative agents to populate an interactive sandbox environment inspired by The Sims, where end users can interact with a small town of twenty-five agents using natural language. In an evaluation, these generative agents produce believable individual and emergent social behaviors. For example, starting with only a single user-specified notion that one agent wants to throw a Valentine’s Day party, the agents autonomously spread invitations to the party over the next two days, make new acquaintances, ask each other out on dates to the party, and coordinate to show up for the party together at the right time. We demonstrate through ablation that the components of our agent architecture—observation, planning, and reflection—each contribute critically to the believability of agent behavior. By fusing large language models with computational interactive agents, this work introduces architectural and interaction patterns for enabling believable simulations of human behavior.

Prior work has explored the writing challenges experienced by people with dyslexia, and the potential for new spelling, grammar, and word retrieval technologies to address these challenges. However, the capabilities for natural language generation demonstrated by the latest class of large language models (LLMs) highlight an opportunity to explore new forms of human-AI writing support tools. In this paper, we introduce LaMPost, a prototype email-writing interface that explores the potential for LLMs to power writing support tools that address the varied needs of people with dyslexia. LaMPost draws from our understanding of these needs and introduces novel AI-powered features for email-writing, including: outlining main ideas, generating a subject line, suggesting changes, rewriting a selection. We evaluated LaMPost with 19 adults with dyslexia, identifying many promising routes for further exploration (including the popularity of the “rewrite” and “subject line” features), but also finding that the current generation of LLMs may not surpass the accuracy and quality thresholds required to meet the needs of writers with dyslexia. Surprisingly, we found that participants’ awareness of the AI had no effect on their perception of the system, nor on their feelings of autonomy, expression, and self-efficacy when writing emails. Our findings yield further insight into the benefits and drawbacks of using LLMs as writing support for adults with dyslexia and provide a foundation to build upon in future research.

Auditory interfaces increasingly support access to website content, through recent advances in voice interaction. Typically, however, these interfaces provide only limited audio styling, collapsing rich visual design into a static audio output style with a single synthesized voice. To explore the potential for more aesthetic and intuitive sound design for websites, we prompted 14 professional sound designers to create auditory website mockups and interviewed them about their designs and rationale. Our findings reveal their prioritized design considerations (aesthetics and emotion, user engagement, audio clarity, information dynamics, and interactivity), specific sound design ideas to support each consideration (e.g., replacing spoken labels with short, memorable audio expressions), and challenges with applying sound design practices to auditory websites. These findings provide promising direction for how to support designers in creating richer auditory website experiences.

Few images on the Web receive alt-text descriptions that would make them accessible to blind and low vision (BLV) users.Imagebased NLG systems have progressed to the point where they can begin to address this persistent societal problem, but these systems will not be fully successful unless we evaluate them on metrics that guide their development correctly.Here, we argue against current referenceless metrics -those that don't rely on human-generated ground-truth descriptions -on the grounds that they do not align with the needs of BLV users.The fundamental shortcoming of these metrics is that they do not take context into account, whereas contextual information is highly valued by BLV users.To substantiate these claims, we present a study with BLV participants who rated descriptions along a variety of dimensions.An in-depth analysis reveals that the lack of context-awareness makes current referenceless metrics inadequate for advancing image accessibility.As a proof-of-concept, we provide a contextual version of the referenceless metric CLIPScore which begins to address the disconnect to the BLV data.

Shanqing Cai, Subhashini Venugopalan, Katrin Tomanek, Ajit Narayanan, Meredith Morris, Michael Brenner. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Social computing prototypes probe the social behaviors that may arise in an envisioned system design. This prototyping practice is currently limited to recruiting small groups of people. Unfortunately, many challenges do not arise until a system is populated at a larger scale. Can a designer understand how a social system might behave when populated, and make adjustments to the design before the system falls prey to such challenges? We introduce social simulacra, a prototyping technique that generates a breadth of realistic social interactions that may emerge when a social computing system is populated. Social simulacra take as input the designer’s description of a community’s design—goal, rules, and member personas—and produce as output an instance of that design with simulated behavior, including posts, replies, and anti-social behaviors. We demonstrate that social simulacra shift the behaviors that they generate appropriately in response to design changes, and that they enable exploration of “what if?” scenarios where community members or moderators intervene. To power social simulacra, we contribute techniques for prompting a large language model to generate thousands of distinct community members and their social interactions with each other; these techniques are enabled by the observation that large language models’ training data already includes a wide variety of positive and negative behavior on social media platforms. In evaluations, we show that participants are often unable to distinguish social simulacra from actual community behavior and that social computing designers successfully refine their social computing designs when using social simulacra.

Virtual reality (VR) leverages human sight, hearing and touch senses to convey virtual experiences. For d/Deaf and hard of hearing (DHH) people, information conveyed through sound may not be accessible. To help with future design of accessible VR sound representations for DHH users, this paper contributes a consistent language and structure for representing sounds in VR. Using two studies, we report on the design and evaluation of a novel taxonomy for VR sounds. Study 1 included interviews with 10 VR sound designers to develop our taxonomy along two dimensions: sound source and intent. To evaluate this taxonomy, we conducted another study (Study 2) where eight HCI researchers used our taxonomy to document sounds in 33 VR apps. We found that our taxonomy was able to successfully categorize nearly all sounds (265/267) in these apps. We also uncovered additional insights for designing accessible visual and haptic-based sound substitutes for DHH users.

Disaggregated evaluations of AI systems, in which system performance is assessed and reported separately for different groups of people, are conceptually simple. However, their design involves a variety of choices. Some of these choices influence the results that will be obtained, and thus the conclusions that can be drawn; others influence the impacts---both beneficial and harmful---that a disaggregated evaluation will have on people, including the people whose data is used to conduct the evaluation. We argue that a deeper understanding of these choices will enable researchers and practitioners to design careful and conclusive disaggregated evaluations. We also argue that better documentation of these choices, along with the underlying considerations and tradeoffs that have been made, will help others when interpreting an evaluation's results and conclusions.

A diverse representation of different demographic groups in AI training data sets is important in ensuring that the models will work for a large range of users. To this end, recent efforts in AI fairness and inclusion have advocated for creating AI data sets that are well-balanced across race, gender, socioeconomic status, and disability status. In this paper, we contribute to this line of work by focusing on the representation of age by asking whether older adults are represented proportionally to the population at large in AI data sets. We examine publicly-available information about 92 face data sets to understand how they codify age as a case study to investigate how the subjects' ages are recorded and whether older generations are represented. We find that older adults are very under-represented; five data sets in the study that explicitly documented the closed age intervals of their subjects included older adults (defined as older than 65 years), while only one included oldest-old adults (defined as older than 85 years). Additionally, we find that only 24 of the data sets include any age-related information in their documentation or metadata, and that there is no consistent method followed across these data sets to collect and record the subjects' ages. We recognize the unique difficulties in creating representative data sets in terms of age, but raise it as an important dimension that researchers and engineers interested in inclusive AI should consider.

Alternative (alt) text provides access to descriptions of digital images for people who use screen readers. While prior work studied screen reader users’ (SRUs’) preferences about alt text and automatic alt text (i.e., alt text generated by artificial intelligence), little work examined the alt text author’s experience composing or editing these descriptions. We built two types of prototype interfaces for two tasks: authoring alt text and providing feedback on automatic alt text. Through combined interview-usability testing sessions with alt text authors and interviews with SRUs, we tested the effectiveness of our prototypes in the context of Microsoft PowerPoint. Our results suggest that authoring interfaces that support authors in choosing what to include in their descriptions result in higher quality alt text. The feedback interfaces highlighted considerable differences in the perceptions of authors and SRUs regarding “high-quality” alt text. Finally, authors crafted significantly lower quality alt text when starting from the automatic alt text compared to starting from a blank box. We discuss the implications of these results on applications that support alt text.

Image descriptions are how people who are blind or have low vision (BLV) access information depicted within images. To our knowledge, no prior work has examined how a description for an image should be designed for different scenarios in which users encounter images. Scenarios consist of the information goal the person has when seeking information from or about an image, paired with the source where the image is found. To address this gap, we interviewed 28 people who are BLV to learn how the scenario impacts what image content (information) should go into an image description. We offer our findings as a foundation for considering how to design next-generation image description technologies that can both (A) support a departure from one-size-fits-all image descriptions to context-aware descriptions, and (B) reveal what content to include in minimum viable descriptions for a large range of scenarios.

Priming is used as a way of increasing the diversity of proposals in end-user elicitation studies, but priming has not been investigated thoroughly in this context. We conduct a distributed end-user elicitation study with 167 participants, which had three priming groups: a no-priming control group, sci-fi priming, and a creative mindset group. We evaluated the gestures proposed by these groups in a distributed learnability and memorability study with 18 participants. We found that the user-elicited gestures from the sci-fi group were significantly faster to learn, requiring an average of 1.22 viewings to learn compared to 1.60 viewings required to learn the control gestures, and 1.56 viewings to learn the gestures elicited from the creative mindset group. In addition, both primed gesture groups had higher memorability with 80% of the sci-fi-primed gestures and 73% of the creative mindset group gestures were recalled correctly after one week without practice compared to 43% of the control group gestures.

Virtual reality (VR) leverages sight, hearing, and touch senses to convey virtual experiences. For d/Deaf and hard of hearing (DHH) people, however, information conveyed through sound may not be accessible. While prior work has explored making every day sounds accessible to DHH users, the context of VR is, as yet, unexplored. In this paper, we provide a first comprehensive investigation of sound accessibility in VR. Our primary contributions include a design space for developing visual and haptic substitutes of VR sounds to support DHH users and prototypes illustrating several points within the design space. We also characterize sound accessibility in commonly used VR apps and discuss findings from early evaluations of our prototypes with 11 DHH users and 4 VR developers.

The World Health Organization estimates that over one billion people worldwide are disabled. Innovations at the intersection of AI and HCI have the potential to increase the accessibility of the digital and physical worlds for people experiencing long-term, temporary, and/or situational disabilities. Considering accessibility scenarios can illuminate opportunities and challenges for designers of intelligent user interfaces. In this keynote, I will use two scenarios to illustrate this concept: automatic alt text generation for images and augmentative and alternative communication technologies.

With advances in expressive speech synthesis and conversational understanding, an ever-increasing amount of digital content---including social and personal content---can be consumed through voice. Voice has long been known to convey personal characteristics and emotional states, both of which are prominent aspects of social media. Yet, no study has investigated voice design requirements for social media platforms. We interviewed 15 active social media users about their preferences on using synthesized voices to represent their profiles. Our findings show that participants want to have control over how a voice delivers their content, such as the personality and emotion with which the voice speaks, because these prosodic variations can impact users' online personas and interfere with impression management. We report motivations behind customizing or not customizing voice characteristics in different scenarios, and uncover key challenges around usability and the potential for stereotyping. We argue that synthesized speech for social media should be evaluated not only on listening experience and voice quality but also on its expressivity, degree of customizability, and ability to adapt to contexts (e.g., social media platforms, groups, individual posts). We discuss how our contribution confirms and extends knowledge of voice technology design and online self-presentation, and offer design considerations for voice personalization related to social interactions.

Despite the large body of work in accessibility concerning the design of novel navigation technologies, little is known about commonly available technologies that people with visual impairments currently use for navigation. We address this gap with a qualitative study consisting of interviews with 23 people with visual impairments, ten of whom also participated in a follow-up diary study. We develop the idea of complementarity first introduced by Williams et al. [53] and find that in addition to using apps to complement mobility aids, technologies and apps complemented each other and filled in for the gaps inherent in one another. Furthermore, the complementarity between apps and other apps/aids was primarily the result of the differences in information and modalities in which this information is communicated by apps, technology and mobility aids. We propose design recommendations to enhance this complementarity and guide the development of improved navigation experiences for people with visual impairments.

Sensing technologies that implicitly and explicitly mediate digital experiences are an increasingly pervasive part of daily living; it is vital to ensure that these technologies work appropriately for people with physical disabilities. We conducted on online survey with 40 adults with physical disabilities, gathering open-ended descriptions about respondents’ experiences with a variety of sensing systems, including motion sensors, biometric sensors, speech input, as well as touch and gesture systems. We present findings regarding the many challenges status quo sensing systems present for people with physical disabilities, as well as the ways in which our participants responded to these challenges. We conclude by reflecting on the significance of these findings for defining a future research agenda for creating more inclusive sensing systems.

Virtual reality (VR) has the potential to transform many aspects of our daily lives, including work, entertainment, communication, and education. However, there has been little research into understanding the usability of VR for people with mobility limitations. In this paper, we present the results of an exploration to understand the accessibility of VR for people with limited mobility. We conducted semi-structured interviews with 16 people with limited mobility about their thoughts on, and experiences with, VR systems. We identified 7 barriers related to the physical accessibility of VR devices that people with limited mobility might encounter, ranging from the initial setup of a VR system to keeping VR controllers in view of cameras embedded in VR headsets. We also elicited potential improvements to VR systems that would address some accessibility concerns. Based on our findings, we discuss the importance of considering the abilities of people with limited mobility when designing VR systems, as the abilities of many participants did not match the assumptions embedded in the design of current VR systems.

Access to digital images is important to people who are blind or have low vision (BLV). Many contemporary image description efforts do not take into account this population's nuanced image description preferences. In this paper, we present a qualitative study that provides insight into 28 BLV people's experiences with descriptions of digital images from news websites, social networking sites/platforms, eCommerce websites, employment websites, online dating websites/platforms, productivity applications, and e-publications. Our findings reveal how image description preferences vary based on the source where digital images are encountered and the surrounding context. We provide recommendations for the development of next-generation image description technologies inspired by our empirical analysis.

The ability to find information online using a search engine is a key modern literacy skill. However, information seeking remains particularly challenging for many people with disabilities, a sizable population comprising over a billion people worldwide, including approximately 20% of adults in the U.S. In this talk, I will describe how various types of disability impact information seeking, including sensory disabilities such as vision impairment and cognitive differences such as dyslexia. I will present research at the intersection of human-computer interaction and information retrieval that aims to address the challenges that search engine use presents for these populations. These examples illustrate how considering the needs of end-users with disabilities can result in the creation of accessible search tools that benefit us all.

Web search is a key digital literacy skill that can be particularly challenging for people with dyslexia, a common learning disability that affects reading and spelling skills in about 15% of the English-speaking population. In this paper, we collected and analyzed eye-tracking, search log, and self-report data from 27 participants (14 with dyslexia) to confirm that searchers with dyslexia struggle with all stages of the search process and have markedly different gaze patterns and search behavior that reflect the strategies used and challenges faced. Based on these findings, we discuss design implications to improve the cognitive accessibility of web search.

This joint panel between UIST and CSCW brings together leading researchers at the intersection of the conferences-systems researchers in collaborative and social computing-to engage in a discussion and retrospective. Pairs of panelists will represent each decade since the founding of the conferences, sharing a brief retrospective that surveys the most influential papers of that decade, the zeitgeist of the problems that were popular that decade and why, and what each decade's work has to say to the decades that came before and after. The panel is intended as a space to celebrate advances in the field, and reflect on the burdens and opportunities that it faces ahead.

Social media platforms support the sharing of written text, video, and audio. All of these formats may be inaccessible to people who are deaf or hard of hearing (DHH), particularly those who primarily communicate via sign language, people who we call Deaf signers. We study how Deaf signers engage with social platforms, focusing on how they share content and the barriers they face. We employ a mixed-methods approach involving seven in-depth interviews and a survey of a larger population (n = 60). We find that Deaf signers share the most in written English, despite their desire to share in sign language. We further identify key areas of difficulty in consuming content (e.g., lack of captions for spoken content in videos) and producing content (e.g., captioning signed videos, signing into a phone camera) on social media platforms. Our results both provide novel insights into social media use by Deaf signers and reinforce prior findings on DHH communication more generally, while revealing potential ways to make social media platforms more accessible to Deaf signers.

Developing successful sign language recognition, generation, and translation systems requires expertise in a wide range of fields, including computer vision, computer graphics, natural language processing, human-computer interaction, linguistics, and Deaf culture. Despite the need for deep interdisciplinary knowledge, existing research occurs in separate disciplinary silos, and tackles separate portions of the sign language processing pipeline. This leads to three key questions: 1) What does an interdisciplinary view of the current landscape reveal? 2) What are the biggest challenges facing the field? and 3) What are the calls to action for people working in the field? To help answer these questions, we brought together a diverse group of experts for a two-day workshop. This paper presents the results of that interdisciplinary workshop, providing key background that is often overlooked by computer scientists, a review of the state-of-the-art, a set of pressing challenges, and a call to action for the research community.

Despite the major role of Global Positioning Systems (GPS) as a navigation tool for people with visual impairments (VI), a crucial missing aspect of point-to-point navigation with these systems is the last-few-meters wayfinding problem. Due to GPS inaccuracy and inadequate map data, systems often bring a user to the vicinity of a destination but not to the exact location, causing challenges such as difficulty locating building entrances or a specific storefront from a series of stores. In this paper, we study this problem space in two parts: (1) A formative study (N=22) to understand challenges, current resolution techniques, and user needs; and (2) A design probe study (N=13) using a novel, vision-based system called Landmark AI to understand how technology can better address aspects of this problem. Based on these investigations, we articulate a design space for systems addressing this challenge, along with implications for future systems to support precise navigation for people with VI.

Gaze-tracking technologies can enable computer access for users who are unable to use standard input devices. However, using gaze as input poses challenges for interactions that require visual planning, like playing a digital instrument. We explore how multimodality can support eye-controlled musical expression by designing different multi-modal gaze interactions around a digital instrument we call Dueto. We tackle three design goals: creating an instrument that is explorable, easy to learn, and allows feature controllability. We showcase three different multimodal interactions for music playing such as eye gaze only, gaze + switch, and gaze + partner mode.

People with visual impairments often rely on screen readers when interacting with computer systems. Increasingly, these individuals also make extensive use of voice-based virtual assistants (VAs). We conducted a survey of 53 people who are legally blind to identify the strengths and weaknesses of both technologies, and the unmet opportunities at their intersection. We learned that virtual assistants are convenient and accessible, but lack the ability to deeply engage with content (e.g., read beyond the first few sentences of an article), and the ability to get a quick overview of the landscape (e.g., list alternative search results and suggestions). In contrast, screen readers allow for deep engagement with content (when content is accessible), and provide fine-grained navigation and control, but at the cost of reduced walk-up-and-use convenience. Based on these findings, we implemented VERSE (Voice Exploration, Retrieval, and SEarch), a prototype that extends a VA with screen-reader-inspired capabilities, and allows other devices (e.g., smartwatches) to serve as optional input accelerators. In a usability study with 12 blind screen reader users we found that VERSE meaningfully extended VA functionality. Participants especially valued having access to multiple search results and search verticals.

End-user elicitation studies are a popular design method. Currently, such studies are usually confined to a lab, limiting the number and diversity of participants, and therefore the representativeness of their results. Furthermore, the quality of the results from such studies generally lacks any formal means of evaluation. In this paper, we address some of the limitations of elicitation studies through the creation of the Crowdlicit system along with the introduction of end-user identification studies, which are the reverse of elicitation studies. Crowdlicit is a new web-based system that enables researchers to conduct online and in-lab elicitation and identification studies. We used Crowdlicit to run a crowd-powered elicitation study based on Morris's "Web on the Wall" study (2012) with 78 participants, arriving at a set of symbols that included six new symbols different from Morris's. We evaluated the effectiveness of 49 symbols (43 from Morris and six from Crowdlicit) by conducting a crowd-powered identification study. We show that the Crowdlicit elicitation study resulted in a set of symbols that was significantly more identifiable than Morris's.

As reading increasingly shifts from paper to online media, many web browsers now provide a "Reader View,'' which modifies web page layout and design for better readability. However, research has yet to establish whether Reader Views are effective in improving readability and how they might change the user experience. We characterize how Mozilla Firefox's Reader View significantly reduces the visual complexity of websites by excluding menus, images, and content. We then conducted an online study with 391 participants (including 42 who self-reported having been diagnosed with dyslexia), showing that compared to standard websites the Reader View increased reading speed by 5% for readers on average, and significantly improved perceived readability and visual appeal. We suggest guidelines for the design of websites and browsers that better support people with varying reading skills.

Current virtual reality applications do not support people who have low vision, i.e., vision loss that falls short of complete blindness but is not correctable by glasses. We present SeeingVR, a set of 14 tools that enhance a VR application for people with low vision by providing visual and audio augmentations. A user can select, adjust, and combine different tools based on their preferences. Nine of our tools modify an existing VR application post hoc via a plugin without developer effort. The rest require simple inputs from developers using a Unity toolkit we created that allows integrating all 14 of our low vision support tools during development. Our evaluation with 11 participants with low vision showed that SeeingVR enabled users to better enjoy VR and complete tasks more quickly and accurately. Developers also found our Unity toolkit easy and convenient to use.

To make images on Twitter and other social media platforms accessible to screen reader users, image descriptions (alternative text) need to be added that describe the information contained within the image. The lack of alternative text has been an enduring accessibility problem since the “alt” attribute was added in HTML 2.0 over 20 years ago, and the rise of user-generated content has only increased the number of images shared. As of 2016, Twitter provides users the ability to turn on a feature that allows descriptions to be added to images in their tweets, presumably in an effort to combat this accessibility problem. What has remained unknown is whether simply enabling users to provide alternative text has an impact on experienced accessibility. In this paper, we present a study of 1.09 million tweets with images, finding that only 0.1% of those tweets included descriptions. In a separate analysis of the timelines of 94 blind Twitter users, we found that these image tweets included descriptions more often. Even users with the feature turned on only write descriptions for about half of the images they tweet. To better understand why users provide alternative text descriptions (or not), we interviewed 20 Twitter users who have written image descriptions. Users did not remember to add alternative text, did not have time to add it, or did not know what to include when writing the descriptions. Our findings indicate that simply making it possible to provide image descriptions is not enough, and reveal future directions for automated tools that may support users in writing high-quality descriptions.

People with visual impairments often rely on screen readers when interacting with computer systems. Increasingly, these individuals also make extensive use of voice-based virtual assistants (VAs). We conducted a survey of 53 people who are legally blind to identify the strengths and weaknesses of both technologies, as well as the unmet opportunities at their intersection. We learned that virtual assistants are convenient and accessible, but lack the ability to deeply engage with content (e.g., read beyond the first few sentences of Wikipedia), and the ability to get a quick overview of the landscape (list alternative search results & suggestions). In contrast, screen readers allow for deep engagement with content (when content is accessible), and provide fine-grained navigation & control, but at the cost of increased complexity, and reduced walk-up-and-use convenience. In this demonstration, we showcase VERSE, a system that combines the positive aspects of VAs and screen readers, and allows other devices (e.g., smart watches) to serve as optional input accelerators. Together, these features allow people with visual impairments to deeply engage with web content through voice interaction.

Video calling (VC) aims to create multi-modal, collaborative environments that are "just like being there." However, we found that autistic individuals, who exhibit atypical social and cognitive processing, may not share this goal. We interviewed autistic adults about their perceptions of VC compared to other computer- mediated communications (CMC) and face-to-face interactions. We developed a neurodiversity-sensitive model of CMC that describes how stressors such as sensory sensitivities, cognitive load, and anxiety, contribute to their preferences for CMC channels. We learned that they apply significant effort to construct coping strategies to support their sensory, cognitive, and social needs. These strategies include moderating their sensory inputs, creating mental models of conversation partners, and attempting to mask their autism by adopting neurotypical behaviors. Without effective strategies, interviewees experience more stress, have less capacity to interpret verbal and non-verbal cues, and feel less empowered to participate. Our findings reveal critical needs for autistic users. We suggest design opportunities to support their ability to comfortably use VC, and in doing so, point the way towards making VC more comfortable for all.

We contribute a qualitative investigation of how teens with visual impairments (VIP) access smartphone photography, from the time they take photos through editing and sharing them on social media. We observed that they largely want to engage with photos visually, similarly to their sighted peers, and have developed strategies around photo capture, editing, sharing, and consumption that attempt to mitigate usability limitations of current photography and social media apps. We demonstrate the need for more work examining how young people with low vision engage with smartphone photography and social media, as they are heavy users of such technologies and have challenges distinct from their totally blind counterparts. We conclude with design considerations to alleviate the usability barriers we uncovered and for making smartphone photography and social media more accessible and relevant for VIPs.

Alternative and augmentative communication (AAC) systems used by people with speech disabilities rely on text-to-speech (TTS) engines for synthesizing speech. Advances in TTS systems allowing for the rendering of speech with a range of emotions have yet to be incorporated into AAC systems, leaving AAC users with speech that is mostly devoid of emotion and expressivity. In this work, we describe voicesetting as the process of authoring the speech properties of text. We present the design and evaluation of two voicesetting user interfaces: the Expressive Keyboard, designed for rapid addition of expressivity to speech, and the Voicesetting Editor, designed for more careful crafting of the way text should be spoken. We evaluated the perceived output quality, requisite effort, and usability of both interfaces; the concept of voicesetting and our interfaces were highly valued by end-users as an enhancement to communication quality. We close by discussing design insights from our evaluations.

Accessing images online is often difficult for users with vision impairments. This population relies on text descriptions of images that vary based on website authors' accessibility practices. Where one author might provide a descriptive caption for an image, another might provide no caption for the same image, leading to inconsistent experiences. In this work, we present the Caption Crawler system, which uses reverse image search to find existing captions on the web and make them accessible to a user's screen reader. We report our system's performance on a set of 481 websites from alexa.com's list of most popular sites to estimate caption coverage and latency, and also report blind and sighted users' ratings of our system's output quality. Finally, we conducted a user study with fourteen screen reader users to examine how the system might be used for personal browsing.

As many as 20% of English speakers have dyslexia, a language disability that impacts reading and spelling. Web search is an important modern literacy skill, yet the accessibility of this language-centric endeavor to people with dyslexia is largely unexplored. We interviewed ten adults with dyslexia and conducted an online survey with 81 dyslexic and 80 non-dyslexic adults, in which participants described challenges they face in various stages of web search (query formulation, search result triage, and information extraction). We also report the findings of an online study in which 174 adults with dyslexia and 172 without dyslexia rated the readability and relevance of sets of search query results. Our findings demonstrate differences in behaviors and preferences between dyslexic and non-dyslexic searchers, and indicate that factoring readability into search engine rankings and/or interfaces may benefit both dyslexic and non-dyslexic users.

Alt text (short for "alternative text") is descriptive text associated with an image in HTML and other document formats. Screen reader technologies speak the alt text aloud to people who are visually impaired. Introduced with HTML 2.0 in 1995, the alt attribute has not evolved despite significant changes in technology over the past two decades. In light of the expanding volume, purpose, and importance of digital imagery, we reflect on how alt text could be supplemented to offer a richer experience of visual content to screen reader users. Our contributions include articulating the design space of representations of visual content for screen reader users, prototypes illustrating several points within this design space, and evaluations of several of these new image representations with people who are blind. We close by discussing the implications of our taxonomy, prototypes, and user study findings.

We present the results of an exploration to understand the accessibility of smartphone photography for people with motor impairments. We surveyed forty-six people and interviewed twelve people about capturing, editing, and sharing photographs on smartphones. We found that people with motor impairments encounter many challenges with smartphone photography, resulting in users capturing fewer photographs than they would like. Participants described various strategies they used to overcome challenges in order to capture a quality photograph. We also found that photograph quality plays a large role in deciding which photographs users share and how often they share, with most participants rating their photographs as average or poor quality compared to photos shared on their social networks. Additionally, we created design probes of two novel photography interfaces and received feedback from our interview participants about their usefulness and functionality. Based on our findings, we propose design recommendations for how to improve the accessibility of mobile photoware for people with motor impairments.

Traditional virtual reality (VR) mainly focuses on visual feedback, which is not accessible for people with visual impairments. We created Canetroller, a haptic cane controller that simulates white cane interactions, enabling people with visual impairments to navigate a virtual environment by transferring their cane skills into the virtual world. Canetroller provides three types of feedback: (1) physical resistance generated by a wearable programmable brake mechanism that physically impedes the controller when the virtual cane comes in contact with a virtual object; (2) vibrotactile feedback that simulates the vibrations when a cane hits an object or touches and drags across various surfaces; and (3) spatial 3D auditory feedback simulating the sound of real-world cane interactions. We designed indoor and outdoor VR scenes to evaluate the effectiveness of our controller. Our study showed that Canetroller was a promising tool that enabled visually impaired participants to navigate different virtual spaces. We discuss potential applications supported by Canetroller ranging from entertainment to mobility training.

We study how real-time crowdsourcing can be used both for evaluating the value provided by existing automated approaches and for enabling workflows that provide scalable and useful alt text to blind users. We show that the shortcomings of existing AI image captioning systems frequently hinder a user's understanding of an image they cannot see to a degree that even clarifying conversations with sighted assistants cannot correct. Based on analysis of clarifying conversations collected from our studies, we design experiences that can effectively assist users in a scalable way without the need for real-time interaction. Our results provide lessons and guidelines that the designers of future AI captioning systems can use to improve labeling of social media imagery for blind users.

Standards organizations, (e.g., the World Wide Web Consortium), are placing increased importance on the cognitive accessibility of online systems, including web search. Previous work has shown an association between query-document relevance judgments, and query-independent assessments of document readability. In this paper we study the lexical and aesthetic features of web documents that may underlie this relationship. Leveraging a data set consisting of relevance and readability judgments for 200 web pages as assessed by 174 adults with dyslexia and 172 adults without dyslexia, we answer the following research questions: (1) Which web page features are most associated with readability? (2) To what extent are these features also associated with relevance? And, (3) are any features associated with the differences in readability/relevance judgments observed between dyslexic and non-dyslexic populations? Our findings have implications for improving the cognitive accessibility of search systems and web documents.

End-user elicitation studies are a popular design method, but their data require substantial time and effort to analyze. In this paper, we present Crowdsensus, a crowd-powered tool that enables researchers to efficiently analyze the results of elicitation studies using subjective human judgment and automatic clustering algorithms. In addition to our own analysis, we asked six expert researchers with experience running and analyzing elicitation studies to analyze an end-user elicitation dataset of 10 functions for operating a web-browser, each with 43 voice commands elicited from end-users for a total of 430 voice commands. We used Crowdsensus to gather similarity judgments of these same 430 commands from 410 online crowd workers. The crowd outperformed the experts by arriving at the same results for seven of eight functions and resolving a function where the experts failed to agree. Also, using Crowdsensus was about four times faster than using experts.

Communicating at a natural speed is a significant challenge for users of augmentative and alternative communication (AAC) devices, especially when input is provided by eye gaze, as is common for people with ALS and similar conditions. One way to improve AAC throughput is by drawing on contextual information from the outside world. Toward this goal, we present SceneTalk, a prototype gaze-based AAC system that uses computer vision to identify objects in the user's field of view and suggests words and phrases related to the current scene. We conducted a formative evaluation of SceneTalk with six people with ALS, in which we evaluated their preference for user interface modes and output preferences. Participants agreed that integrating contextual awareness into their AAC device could be helpful across a diverse range of situations.

For eye tracking to become a ubiquitous part of our everyday interaction with computers, we first need to understand its limitations outside rigorously controlled labs, and develop robust applications that can be used by a broad range of users and in various environments. Toward this end, we collected eye tracking data from 80 people in a calibration-style task, using two different trackers in two lighting conditions. We found that accuracy and precision can vary between users and targets more than six-fold, and report on differences between lighting, trackers, and screen regions. We show how such data can be used to determine appropriate target sizes and to optimize the parameters of commonly used filters. We conclude with design recommendations and examples how our findings and methodology can inform the design of error-aware adaptive applications.

Research advancements allow computational systems to automatically caption social media images. Often, these captions are evaluated with sighted humans using the image as a reference. Here, we explore how blind and visually impaired people experience these captions in two studies about social media images. Using a contextual inquiry approach (n=6 blind/visually impaired), we found that blind people place a lot of trust in automatically generated captions, filling in details to resolve differences between an image's context and an incongruent caption. We built on this in-person study with a second, larger online experiment (n=100 blind/visually impaired) to investigate the role of phrasing in encouraging trust or skepticism in captions. We found that captions emphasizing the probability of error, rather than correctness, encouraged people to attribute incongruence to an incorrect caption, rather than missing details. Where existing research has focused on encouraging trust in intelligent systems, we conclude by challenging this assumption and consider the benefits of encouraging appropriate skepticism.

Mainstream crowdwork platforms treat microtasks as indivisible units; however, in this article, we propose that there is value in re-examining this assumption. We argue that crowdwork platforms can improve their value proposition for all stakeholders by supporting subcontracting within microtasks. After describing the value proposition of subcontracting, we then define three models for microtask subcontracting: real-time assistance, task management, and task improvement, and reflect on potential use cases and implementation considerations associated with each. Finally, we describe the outcome of two tasks on Mechanical Turk meant to simulate aspects of subcontracting. We reflect on the implications of these findings for the design of future crowd work platforms that effectively harness the potential of subcontracting workflows.

We present cascading dwell gaze typing, a novel approach to dwell-based eye typing that dynamically adjusts the dwell time of keys in an on-screen keyboard based on the likelihood that a key will be selected next, and the location of the key on the keyboard. Our approach makes unlikely keys more difficult to select and likely keys easier to select by increasing and decreasing their required dwell times, respectively. To maintain a smooth typing rhythm for the user, we cascade the dwell time of likely keys, slowly decreasing the minimum allowable dwell time as a user enters text. Cascading the dwell time affords users the benefits of faster dwell times while causing little disruption to users' typing cadence. Results from a longitudinal study with 17 non-disabled participants show that our dynamic cascading dwell technique was significantly faster than a static dwell approach. Participants were able to achieve typing speeds of 12.39 WPM on average with our cascading technique, whereas participants were able to achieve typing speeds of 10.62 WPM on average with a static dwell time approach. In a small evaluation conducted with five people with ALS, participants achieved average typing speeds of 9.51 WPM with our cascading dwell approach. These results show that our dynamic cascading dwell technique has the potential to improve gaze typing for users with and without disabilities.

"Remind me to get milk later this afternoon." In communications and planning, people often express uncertainty about time using imprecise temporal expressions (ITEs). Unfortunately, modern virtual assistants often lack system support to capture the intents behind these expressions. This can result in unnatural interactions and undesirable interruptions (e.g., having a work reminder delivered at 12pm when out at lunch, because the user said "this afternoon"). In this paper we explore existing practices, expectations, and preferences surrounding the use of ITEs. Our mixed methods approach employs surveys, interviews, and an analysis of a large corpus of written communications. We find that people frequently use a diverse set of ITEs in both communication and planning. These uses reflect a variety of motivations, such as conveying uncertainty or task priority. In addition, we find that people have a variety of expectations about time input and management when interacting with virtual assistants. We conclude with design implications for future virtual assistants.

Augmentative and alternative communication (AAC) devices are a critical technology for people with disabilities that affect their speech. One challenge with AAC systems is their inability to portray aspects of nonverbal communication that typically accent, complement, regulate, or substitute for verbal speech. In this paper, we explore the design space of awareness displays that can supplement AAC devices, considering their output features and their effects on the perceptions of interlocutors. Through designing prototypes and getting feedback on our designs from people with ALS, their primary caregivers, and other communication partners, we consider (1) the consistent tensions that arose between abstractness and clarity in meaning for these designs and (2) the ways in which these designs can further mark users as "other." Overall, we contribute a generative understanding of designing AAC awareness displays to augment and contextualize communication.

Current eye-tracking input systems for people with ALS or other motor impairments are expensive, not robust under sunlight, and require frequent re-calibration and substantial, relatively immobile setups. Eye-gaze transfer (e-tran) boards, a low-tech alternative, are challenging to master and offer slow communication rates. To mitigate the drawbacks of these two status quo approaches, we created GazeSpeak, an eye gesture communication system that runs on a smartphone, and is designed to be low-cost, robust, portable, and easy-to-learn, with a higher communication bandwidth than an e-tran board. GazeSpeak can interpret eye gestures in real time, decode these gestures into predicted utterances, and facilitate communication, with different user interfaces for speakers and interpreters. Our evaluations demonstrate that GazeSpeak is robust, has good user satisfaction, and provides a speed improvement with respect to an e-tran board; we also identify avenues for further improvement to low-cost, low-effort gaze-based communication technologies.

Gaze-based alternative and augmentative communication (AAC) devices provide users with neuromuscular diseases the ability to communicate with other people through only the movement of their eyes. These devices suffer from slow input, causing a host of communication breakdowns to occur during face-to-face conversations. These breakdowns lead to decreased user autonomy, conversation quality, and communication partner engagement. Attempts to improve communication through these devices has mainly focused on throughput and rate enhancement, though this has only attained meager results to date. In this work, we address this issue from the top down by considering AAC devices as a form of groupware and designing interactions around this groupware that facilitate better conversations for all involved communicators. We first present qualitative findings on issues with gaze-based AAC and end-user communication preferences; we identify several design guidelines for improving these systems and then present AACrobat, a system that embodies these guidelines and introduces novel interactions by extending gaze-based AAC devices with a mobile companion app. Finally, we present early feedback on AACrobat through three case studies of users with ALS.

Amyotrophic Lateral Sclerosis (ALS) is a disease that causes individuals to lose their ability to control their muscles, eventually leaving them unable to speak or write. People with ALS often transition to using an augmentative and alternative communication device (AAC), which requires both the AAC user and their conversation partners to adjust to new and different communication patterns. We conducted interviews with seven individuals with ALS and their partners, focusing on how AAC use has impacted their communication and personal expression. Our participants experienced a range of communication difficulties, including conversational pacing, personality expression, and interacting with unfamiliar conversational partners. Participants worked to adapt their communication behaviors to the AAC device, but still experienced challenges in expressing themselves, and sometimes felt compelled to withdraw from social interaction. By improving our understanding of how people transition to using AAC, we may inform improved designs for future AAC devices.

In this work, we present EduFeed, a system that enables preliterate children to explore an algorithmically-mediated social feed of learning exercises, select activities to engage in, and share them with their peers. We deployed EduFeed as a technology probe to three classrooms of ESL students in the first year of elementary school in India who had limited English literacy and limited experience with touchscreen technology. We found that children were able to self-direct their engagement with the system and were initially motivated by digital sharing, while social context surrounding the system in physical space was also important. Based on our design and probe deployment, we reflect on issues relevant to adapting the social feed paradigm to this context, such as ranking algorithms, physicality, immediacy, and bridging physical and digital collaborative experiences.

The access of visually impaired users to imagery in social media is constrained by the availability of suitable alt text. It is unknown how imperfections in emerging tools for automatic caption generation may help or hinder blind users' understanding of social media posts with embedded imagery. In this paper, we study how crowdsourcing can be used both for evaluating the value provided by existing automated approaches and for enabling workflows that provide scalable and useful alt text to blind users. Using real-time crowdsourcing, we designed experiences that varied the depth of interaction of the crowd in assisting visually impaired users at caption interpretation, and measured trade-offs in effectiveness, scalability, and reusability. We show that the shortcomings of existing AI image captioning systems frequently hinder a user's understanding of an image they cannot see to a degree that even clarifying conversations with sighted assistants cannot correct. Our detailed analysis of the set of clarifying conversations collected from our studies led to the design of experiences that can effectively assist users in a scalable way without the need for real-time interaction. They also provide lessons and guidelines that human captioners and the designers of future iterations of AI captioning systems can use to improve labeling of social media imagery for blind users.

Many people rely on web search engines to check the spelling or grammatical correctness of input phrases. For example, one might search [recurring or reoccurring] to decide between these similar words. While language-related queries are common, they have low click-through rates, lack a strong intent signal, and are generally challenging to study. Perhaps for these reasons, they have yet to be characterized in the literature. In this paper we report the results of two surveys that investigate how, when, and why people use web search to support low-level, language-related tasks. The first survey was distributed by email, and asked participants to reflect on a recent search task. The second survey was embedded directly in search result pages, and captured information about searchers' intents in-situ. Our analysis confirms that language-related search tasks are indeed common, accounting for at least 2.7% of all queries posed by our respondents. Survey responses also reveal: (1) the range of language-related tasks people perform with search, (2) the contexts in which these tasks arise, and (3), the reasons why people elect to use web search rather than relying on traditional proofing tools (e.g., spelling and grammar checkers).

Diversifying participation in crowd work can benefit the worker and requester. Increasing numbers of older adults are online, but little is known about their awareness of or how they engage in mainstream crowd work. Through an online survey with 505 seniors, we found that most have never heard of crowd work but would be motivated to complete tasks by earning money or working on interesting or stimulating tasks. We follow up results from the survey with interviews and observations of 14 older adults completing crowd work tasks. While our survey data suggests that financial incentives are encouraging, in-depth interviews reveal that a combination of personal and social incentives may be stronger drivers of participation, but only if older adults can overcome accessibility issues and understand the purpose of crowd work. This paper contributes insights into how crowdsourcing sites could better engage seniors and other users.

Social media is an increasingly important part of modern life. We investigate the use of and usability of Twitter by blind users, via a combination of surveys of blind Twitter users, large-scale analysis of tweets from and Twitter profiles of blind and sighted users, and analysis of tweets containing embedded imagery. While Twitter has traditionally been thought of as the most accessible social media platform for blind users, Twitter's increasing integration of image content and users' diverse uses for images have presented emergent accessibility challenges. Our findings illuminate the importance of the ability to use social media for people who are blind, while also highlighting the many challenges such media currently present this user base, including difficulty in creating profiles, in awareness of available features and settings, in controlling revelations of one's disability status, and in dealing with the increasing pervasiveness of image-based content. We propose changes that Twitter and other social platforms should make to promote fuller access to users with visual impairments.

Technology workers are often stereotyped as being socially awkward or having difficulty communicating, often with humorous intent; however, for many technology workers with atypical cognitive profiles, such issues are no laughing matter. In this paper, we explore the hidden lives of neurodiverse technology workers, e.g., those with autism spectrum disorder (ASD), attention deficit hyperactivity disorder (ADHD), and/or other learning disabilities, such as dyslexia. We present findings from interviews with 10 neurodiverse technology workers, identifying the challenges that impede these employees from fully realizing their potential in the workplace. Based on the interview findings, we developed a survey that was taken by 846 engineers at a large software company. In this paper, we reflect on the differences between the neurotypical (N = 781) and neurodiverse (N = 59) respondents. Technology companies struggle to attract, develop, and retain talented software developers; our findings offer insight into how employers can better support the needs of this important worker constituency.

Crowd-powered systems that help people are difficult to scale and sustain because human labor is expensive and worker pools are difficult to grow. To address this problem we introduce the idea of social microvolunteering, a type of intermediated friendsourcing in which a person can provide access to their friends as potential workers for microtasks supporting causes that they care about. We explore this idea by creating Visual Answers, an exemplar social microvolunteering application for Facebook that posts visual questions from people who are blind. We present results of a survey of 350 participants on the concept of social microvolunteering, and a deployment of the Visual Answers application with 91 participants, which collected 618 high-quality answers to questions asked over 12 days, illustrating the feasibility of the approach.

Educators have developed an effective technique to get feedback after in-person lectures, called "muddy cards." Students are given time to reflect and write the "muddiest" (least clear) point on an index card, to hand in as they leave class. This practice of assigning end-of-lecture reflection tasks to generate explicit student feedback is well suited for adaptation to the challenge of supporting feedback in online video lectures. We describe the design and evaluation of Mudslide, a prototype system that translates the practice of muddy cards into the realm of online lecture videos. Based on an in-lab study of students and teachers, we find that spatially contextualizing students' muddy point feedback with respect to particular lecture slides is advantageous to both students and teachers. We also reflect on further opportunities for enhancing this feedback method based on teachers' and students' experiences with our prototype.

Teachers in conventional classrooms often ask learners to express themselves and show their thought processes by speaking out loud, drawing on a whiteboard, or even using physical objects. Despite the pedagogical value of such activities, interactive exercises available in most online learning platforms are constrained to multiple-choice and short answer questions. We introduce RIMES, a system for easily authoring, recording, and reviewing interactive multimedia exercises embedded in lecture videos. With RIMES, teachers can prompt learners to record their responses to an activity using video, audio, and inking while watching lecture videos. Teachers can then review and interact with all the learners' responses in an aggregated gallery. We evaluated RIMES with 19 teachers and 25 students. Teachers created a diverse set of activities across multiple subjects that tested deep conceptual and procedural knowledge. Teachers found the exercises useful for capturing students' thought processes, identifying misconceptions, and engaging students with content.

We present the first formal study of crowdworkers who have disabilities via in-depth open-ended interviews of 17 people (disabled crowdworkers and job coaches for people with disabilities) and a survey of 631 adults with disabilities. Our findings establish that people with a variety of disabilities currently participate in the crowd labor marketplace, despite challenges such as crowdsourcing workflow designs that inadvertently prohibit participation by, and may negatively affect the worker reputations of, people with disabilities. Despite such challenges, we find that crowdwork potentially offers different opportunities for people with disabilities relative to the normative office environment, such as job flexibility and lack of a need to rely on public transit. We close by identifying several ways in which crowd labor platform operators and/or individual task requestors could improve the accessibility of this increasingly important form of employment.

We present SWARM, a wearable affective technology designed to help a user to reflect on their own emotional state, modify their affect, and interpret the emotional states of others. SWARM aims for a universal design (inclusive of people with various disabilities), with a focus on modular actuation components to accommodate users' sensory capabilities and preferences, and a scarf form-factor meant to reduce the stigma of accessible technologies through a fashionable embodiment. Using an iterative, user-centered approach, we present SWARM's design. Additionally, we contribute findings for communicating emotions through technology actuations, wearable design techniques (including a modular soft circuit design technique that fuses conductive fabric with actuation components), and universal design considerations for wearable technology.

Search engines and social media are two of the most com-monly used online services; in this paper, we examine how users appropriate these platforms for online health activi-ties via both large-scale log analysis and a survey of 210 people. While users often turn to search engines to learn about serious or highly stigmatic conditions, a surprising amount of sensitive health information is also sought and shared via social media, in our case the public social plat-form Twitter. We contrast what health content people seek via search engines vs. share on social media, as well as why they choose a particular platform for online health activi-ties. We reflect on the implications of our results for design-ing search engines, social media, and social search tools that better support people's health information seeking and sharing needs.

Every day users of social networking services ask their followers and friends millions of questions. These friendsourced questions not only provide informational benefits, but also may reinforce social bonds. However, there is a limit to how much a person may want to friendsource. They may be uncomfortable asking questions that are too private, might not want to expend others' time or effort, or may feel as though they have already accrued too many social debts. These perceived social costs limit the potential benefits of friendsourcing. In this paper we explore the perceived social costs of friendsourcing on Twitter via a monetary choice. We develop a model of how users value the attention and effort of their social network while friendsourcing, compare and contrast it with paid question answering in a crowdsourced labor market, and provide future design considerations for better supporting friendsourcing.

In addition to posting news and status updates, many Twitter users post questions that seek various types of subjective and objective information. These questions are often labeled with 'Q&A' hashtags, such as #lazyweb or #twoogle. We surveyed Twitter users and found they employ these Q&A hashtags both as a topical signifier (this tweet needs an answer!) and to reach out to those beyond their immediate followers (a community of helpful tweeters who monitor the hashtag). However, our log analysis of thousands of hashtagged Q&A exchanges reveals that nearly all replies to hashtagged questions come from a user's immediate follower network, contradicting users' beliefs that they are tapping into a larger community by tagging their question tweets. This finding has implications for designing next-generation social search systems that reach and engage a wide audience of answerers.

In this paper, we present the first formal study of how mothers of young children (aged three and under) use social networking sites, particularly Facebook and Twitter, including mothers' perceptions of which SNSes are appropriate for sharing information about their children, changes in post style and frequency after birth, and the volume and nature of child-related content shared in these venues. Our findings have implications for improving the utility and usability of SNS tools for mothers of young children, as well as for creating and improving sociotechnical systems related to maternal and child health.

Consumers shopping in "brick-and-mortar" (non-virtual) stores often use their mobile phones to consult with others about potential purchases. Via a survey (n = 200), we detail current practices in seeking remote shopping advice. We then consider how emerging social platforms, such as social networking sites and crowd labor markets, could offer rich next-generation remote shopping advice experiences. We conducted a field experiment in which shoppers shared photographs of potential purchases via MMS, Facebook, and Mechanical Turk. Paid crowdsourcing, in particular, proved surprisingly useful and influential as a means of augmenting in-store shopping. Based on our findings, we offer design suggestions for next-generation remote shopping advice systems.

In status message question asking (SMQA), members of social networking sites make use of status messages to express information needs to friends and contacts. We present findings from a laboratory study that examined 82 participants' SMQA behaviors in the broader context of online information seeking. When given the option of using a search engine and/or a social network, participants leveraged SMQA for 20% of their information needs, most often posing a question to their network in addition to issuing a query. We show the important roles played by the specificity of the information need and the perceived audience of a given network on routing decisions. We then demonstrate that routing decisions have varied effects on participants' satisfaction, information value, and trust of outcomes. In addition to highlighting the complementary advantages and disadvantages of search and SMQA, our findings suggest that search engines can better address a meaningful portion of people's information needs by integrating SMQA capabilities into their systems.

People with disabilities can be reluctant to friendsource help from their own friends for fear of appearing dependent or annoying. Our social microvolunteering approach has volunteers post friendsourcing tasks on behalf of people with disabilities. We demonstrate this approach via a Facebook application that answers visual questions on behalf of blind users.

Adding tactile feedback to touch screens can improve their accessibility to blind users, but prior approaches to integrating tactile feedback with touch screens have either offered limited functionality or required extensive (and typically expensive) customization of the hardware. We introduce touchplates, carefully designed tactile guides that provide tactile feedback for touch screens in the form of physical guides that are overlaid on the screen and recognized by the underlying application. Unlike prior approaches to integrating tactile feedback with touch screens, touchplates are implemented with simple plastics and use standard touch screen software, making them versatile and inexpensive. Touchplates may be customized to suit individual users and applications, and may be produced on a laser cutter, 3D printer, or made by hand. We describe the design and implementation of touchplates, a "starter kit" of touchplates, and feedback from a formative evaluation with 9 people with visual impairments. Touchplates provide a low-cost, adaptable, and accessible method of adding tactile feedback to touch screen interfaces.

The challenges faced by blind people in their everyday lives are not well understood. In this paper, we report on the findings of a large-scale study of the visual questions that blind people would like to have answered. As part of this year-long study, 5,329 blind users asked 40,748 questions about photographs that they took from their iPhones using an application called VizWiz Social. We present a taxonomy of the types of questions asked, report on a number of features of the questions and accompanying photographs, and discuss how individuals changed how they used VizWiz Social over time. These results improve our understanding of the problems blind people face, and may help motivate new projects more accurately targeted to help blind people live more independently in their everyday lives.

Recent work has shown the potential of having remote humans answer visual questions that blind users have. On the surface social networking sites (SNSs) offer an attractive free source of human-powered answers that can be personalized to the user. In this paper, we explore the potential of blind users asking visual questions to their social networks. We present the first formal study of how blind people use social networking sites via a survey of 191 blind adults. We also explore whether blind users find SNSs an appropriate venue for Q&A through a log analysis of questions asked using VizWiz Social, an iPhone app with over 5,000 users, which lets blind users ask questions to either the crowd or friends. We then report findings of a field experiment with 23 blind VizWiz Social users, which explored question asking on VizWiz Social in the presence of monetary costs for non-social sources. We find that blind people have a large presence on social networking sites, but do not see them as an appropriate venue for asking questions due to high perceived social costs.

Despite recent innovations in technologies supporting collaborative web search [11, 13, 25, 34, 35, 37], the features of the primary tools for digital information seeking (web browsers and search engines) continue to reflect a presumption that search is a single-user activity. In this paper, we present the findings of a survey of 167 diverse users' collaborative web search practices, including the prevalence and frequency of such activities, the information needs motivating collaboration, the methods and tools employed in such tasks, and users' satisfaction with the status quo. We find an increased prevalence and frequency of collaborative search, particularly by younger users, and an appropriation of "old" technologies like e-mail as well as "new" technologies like smartphones and social networking sites, rather than the use of dedicated collaborative search tools. We reflect on how and why collaborative search practices have changed in the six years since the first survey detailing this phenomenon was conducted [22], and synthesize our findings to offer suggestions for the design of future collaborative search technologies.

Microblogs have become an increasingly important source of information, both in the U.S. (Twitter) and in China (Weibo). However, the brevity of microblog updates, combined with increasing access of microblog content through search rather than through direct network connections, makes it challenging to assess the credibility of news relayed in this manner [34]. This paper reports on experimental and survey data that compare the impact of several features of microblog updates (author's gender, name style, profile image, location, and degree of network overlap with the reader) on credibility perceptions among U.S. and Chinese audiences. We reveal the complex mechanism of credibility perceptions, identify several key differences in how users from each country critically consume microblog content, and discuss how to incorporate these findings into the design of improved user interfaces for accessing microblogs in different cultural settings.

Twitter is now used to distribute substantive content such as breaking news, increasing the importance of assessing the credibility of tweets. As users increasingly access tweets through search, they have less information on which to base credibility judgments as compared to consuming content from direct social network connections. We present survey results regarding users' perceptions of tweet credibility. We find a disparity between features users consider relevant to credibility assessment and those currently revealed by search engines. We then conducted two experiments in which we systematically manipulated several features of tweets to assess their impact on credibility ratings. We show that users are poor judges of truthfulness based on content alone, and instead are influenced by heuristics such as user name when making credibility assessments. Based on these findings, we discuss strategies tweet authors can use to enhance their credibility with readers (and strategies astute readers should be aware of!). We propose design improvements for displaying social search results so as to better convey credibility.

Co-located collaboration can be extremely valuable during complex visual analytics tasks. We present an exploratory study of a system designed to support collaborative visual analysis tasks on a digital tabletop display. Fifteen participant pairs employed Cambiera, a visual analytics system, to solve a problem involving 240 digital documents. Our analysis, supported by observations, system logs, questionnaires, and interview data, explores how pairs approached the problem around the table. We contribute a unique, rich understanding of how users worked together around the table and identify eight types of collaboration styles that can be used to identify how closely people work together while problem solving. We show how the closeness of teams’ collaboration and communication influenced how they performed on the task overall. We further discuss the role of the tabletop for visual analytics tasks and derive design implications for future co-located collaborative tabletop problem solving systems.

We explore the use of social learning - improving knowledge skills by observing peer behavior - in the domain of Web search skill acquisition, focusing specifically on co-located classroom scenarios. Through a series of interviews, pilot studies, and classroom deployments, we conclude that a peripheral display of Web search activity within a classroom facilitates both social learning and teacher-led discourse. We present the ClassSearch system for shared awareness of Web search activity, which embodies principles gleaned from our iterative design process, and show results from a ClassSearch deployment in twelve middle-school classroom sessions. Finally, we highlight design suggestions and opportunities for future work while taxonomizing the space of co-located search pedagogies.

The presence (and, sometimes, prominence) of incorrect and misleading content on the Web can have serious consequences for people who increasingly rely on the internet as their information source for topics such as health, politics, and financial advice. In this paper, we identify and collect several page features (such as popularity among specialized user groups) that are currently difficult or impossible for end users to assess, yet provide valuable signals regarding credibility. We then present visualizations designed to augment search results and Web pages with the most promising of these features. Our lab evaluation finds that our augmented search results are particularly effective at increasing the accuracy of users'" credibility assessments, highlighting the potential of data aggregation and simple interventions to help people make more informed decisions as they search for information online.

Synchronous, explicit search has some interesting characteristics that distinguish it from other types of interaction: there is much more emphasis on interaction, as the system has to not only communicate search results to the user, but also mediate some forms of communication and data sharing among its users. There are new algorithms that need to be invented that use inputs from multiple people to produce search results, and new evaluation metrics need to be invented that reflect the collaborative and interactive nature of the task. Finally, we need to integrate the expertise of library and information science researchers and practitioners by revisiting real-world information seeking situations with an eye for explicit, synchronous collaborative search.

Many touch screens remain inaccessible to blind users, and those approaches to providing access that do exist offer minimal support for interacting with large touch screens or spatial data. In this paper, we introduce a set of three software-based access overlays intended to improve the accessibility of large touch screen interfaces, specifically interactive tabletops. Our access overlays are called edge projection, neighborhood browsing, and touch-and-speak. In a user study, 14 blind users compared access overlays to an implementation of Apple's VoiceOver screen reader. Our results show that two of our techniques were faster than VoiceOver, that participants correctly answered more questions about the screen's layout using our techniques, and that participants overwhelmingly preferred our techniques. We developed several applications demonstrating the use of access overlays, including an accessible map kiosk and an accessible board game.

Social networking Web sites are not just places to maintain relationships; they can also be valuable information sources. However, little is known about how and why people search socially-generated content. In this paper we explore search behavior on the popular microblogging/social networking site Twitter. Using analysis of large-scale query logs and supplemental qualitative data, we observe that people search Twitter to find temporally relevant information (e.g., breaking news, real-time content, and popular trends) and information related to people (e.g., content directed at the searcher, information about people of interest, and general sentiment and opinion). Twitter queries are shorter, more popular, and less likely to evolve as part of a session than Web queries. It appears people repeat Twitter queries to monitor the associated search results, while changing and developing Web queries to learn about a topic. The results returned from the different corpora support these different uses, with Twitter results including more social chatter and social events, and Web results containing more basic facts and navigational content. We discuss the implications of these findings for the design of next-generation Web search tools that incorporate social media.

Sensemaking is an important aspect of information-seeking tasks but has mostly been studied at the individual level. We conducted a study of sensemaking in collaborative Web search using SearchTogether and found that collaborators face several challenges in making sense of information during collaborative search tasks. We built and evaluated a new tool, CoSense, which enhanced sensemaking in SearchTogether. The evaluation of CoSense provided insights into how collaborative sensemaking differed from individual sensemaking in terms of the different kinds of information that collaborators needed to make sense of. In this article we discuss findings about how sensemaking occurs in synchronous and asynchronous collaboration and the challenges participants face in handling handoffs. We found that participants had two different strategies of handling handoffs—search-lead and sensemaking-lead—and that participants with these two strategies exhibited different procedural knowledge of sensemaking. We also discuss how complex and varied the products of sensemaking are during a collaborative search task. Through our evaluation of CoSense we provide insights into the design of tools that can enhance sensemaking in collaborative search tasks.

People often turn to their friends, families, and colleagues when they have questions. The recent, rapid rise of online social networking tools has made doing this on a large scale easy and efficient. In this paper we explore the phenomenon of using social network status messages to ask questions. We conducted a survey of 624 people, asking them to share the questions they have asked and answered of their online social networks. We present detailed data on the frequency of this type of question asking, the types of questions asked, and respondents' motivations for asking their social networks rather than using more traditional search tools like Web search engines. We report on the perceived speed and quality of the answers received, as well as what motivates people to respond to questions seen in their friends' status messages. We then discuss the implications of our findings for the design of next-generation search tools.

A recent trend in interface design for classrooms in developing regions has many students interacting on the same display using mice. Text entry has emerged as an important problem preventing such mouse-based singledisplay groupware systems from offering compelling interactive activities. We explore the design space of mouse-based text entry and develop 13 techniques with novel characteristics suited to the multiple mouse scenario. We evaluated these in a 3-phase study over 14 days with 40 students in 2 developing region schools. The results show that one technique effectively balanced all of our design dimensions, another was most preferred by students, and both could benefit from augmentation to support collaborative interaction. Our results also provide insights into the factors that create an optimal text entry technique for single-display groupware systems.

This paper introduces an interactive tabletop system that enhances creative collaboration across physical and digital artifacts. Pictionaire offers capture, retrieval, annotation, and collection of visual material. It enables multiple designers to fluidly move imagery from the physical to the digital realm; work with found, drawn and captured imagery; organize items into functional collections; and record meeting histories. These benefits are made possible by a large interactive table augmented with high-resolution overhead image capture. Summative evaluations with 16 professionals and four student pairs validated discoverability and utility of interactions, uncovered emergent functionality, and suggested opportunities for transitioning content to and from the table.

Groups of users often have shared information needs -- for example, business colleagues need to conduct research relating to joint projects and students must work together on group homework assignments. In this paper, we introduce WeSearch, a collaborative Web search system designed to leverage the benefits of tabletop displays for face-to-face collaboration and organization tasks. We describe the design of WeSearch and explain the interactions it affords. We then describe an evaluation in which eleven groups used WeSearch to conduct real collaborative search tasks. Based on our study's findings, we analyze the effectiveness of the features introduced by WeSearch.

An understanding of how people allocate their visual attention when viewing Web pages is very important for Web authors, interface designers, advertisers and others. Such knowledge opens the door to a variety of innovations, ranging from improved Web page design to the creation of compact, yet recognizable, visual representations of long pages. We present an eye-tracking study in which 20 users viewed 361 Web pages while engaged in information foraging and page recognition tasks. From this data, we describe general location-based characteristics of visual attention for Web pages dependent on different tasks and demographics, and generate a model for predicting the visual attention that individual page elements may receive. Finally, we introduce the concept of fixation impact, a new method for mapping gaze data to visual scenes that is motivated by findings in vision research.

Multi-touch interactions are a promising means of control for interactive tabletops. However, a lack of precision and tactile feedback makes multi-touch controls a poor fit for tasks where precision and feedback are crucial. We present an approach that offers precise control and tactile feedback for tabletop systems through the integration of dynamically re-mappable physical controllers with the multi-touch environment, and we demonstrate this approach in our collaborative tabletop audio editing environment. An observational user study demonstrates that our approach can provide needed precision and feedback, while preserving the collaborative benefits of a shared direct-manipulation surface. Our observations also suggest that direct touch and physical controllers can offer complementary benefits, and that providing both allows users to adjust their control strategy based on considerations including precision, convenience, visibility, and user role.

Making sense of the information found during an investigational Web search task can be daunting. With the recent emergence of tools to support collaborative Web search, the associated sensemaking task has become even more complex, requiring sense to be made not only of the products of a search (i.e., results found) but of the process, as well (i.e., group division of labor and decision-making).

Many surface computing prototypes have employed gestures created by system designers. Although such gestures are appropriate for early investigations, they are not necessarily reflective of user behavior. We present an approach to designing tabletop gestures that relies on eliciting gestures from non-technical users by first portraying the effect of a gesture, and then asking users to perform its cause. In all, 1080 gestures from 20 participants were logged, analyzed, and paired with think-aloud data for 27 commands performed with 1 and 2 hands. Our findings indicate that users rarely care about the number of fingers they employ, that one hand is preferred to two, that desktop idioms strongly influence users' mental models, and that some commands elicit little gestural agreement, suggesting the need for on-screen widgets. We also present a complete user-defined gesture set, quantitative agreement scores, implications for surface technology, and a taxonomy of surface gestures. Our results will help designers create better gesture sets informed by user behavior.

This note examines the role traditional input devices can play in surface computing. Mice and keyboards can enhance tabletop technologies since they support high fidelity input, facilitate interaction with distant objects, and serve as a proxy for user identity and position. Interactive tabletops, in turn, can enhance the functionality of traditional input devices: they provide spatial sensing, augment devices with co-located visual content, and support connections among a plurality of devices. We introduce eight interaction techniques for a table with mice and keyboards, and we discuss the design space of such interactions.

Personalized Web search takes advantage of information about an individual to identify the most relevant results for that person. A challenge for personalization lies in collecting user profiles that are rich enough to do this successfully. One way an individual's profile can be augmented is by using data from other people. To better understand whether groups of people can be used to benefit personalized search, we explore the similarity of query selection, desktop information, and explicit relevance judgments across people grouped in different ways. The groupings we explore fall along two dimensions: the longevity of the group members' relationship, and how explicitly the group is formed. We find that some groupings provide valuable insight into what members consider relevant to queries related to the group focus, but that it can be difficult to identify valuable groups implicitly. Building on these findings, we explore an algorithm to "groupize" (versus "personalize") Web search results that leads to a significant improvement in result ranking on group-relevant queries.

Web search is often viewed as a solitary task; however, there are many situations in which groups of people gather around a single computer to jointly search for information online. We present the findings of interviews with teachers, librarians, and developing world researchers that provide details about users' collaborative search habits in shared-computer settings, revealing several limitations of this practice. We then introduce CoSearch, a system we developed to improve the experience of co-located collaborative Web search by leveraging readily available devices such as mobile phones and extra mice. Finally, we present an evaluation comparing CoSearch to status quo collaboration approaches, and show that CoSearch enabled distributed control and division of labor, thus reducing the frustrations associated with shared-computer searches, while still preserving the positive aspects of communication and collaboration associated with joint computer use.

Today's Web browsers provide limited support for rich information-seeking and information-sharing scenarios. A survey we conducted of 204 knowledge workers at a large technology company has revealed that a large proportion of users engage in searches that include collaborative activities. We present the results of the survey, and then review the implications of these findings for designing new Web search interfaces that provide tools for sharing.

Current user interfaces for Web search, including browsers and search engine sites, typically treat search as a transient activity. However, people often conduct complex, multi-query investigations that may span long durations and may be interrupted by other tasks. In this paper, we first present the results of a survey of users' search habits, which show that many search tasks span long periods of time. We then introduce SearchBar, a system for proactively and persistently storing query histories, browsing histories, and users' notes and ratings in an interrelated fashion. SearchBar supports multi-session investigations by assisting with task context resumption and information re-finding. We describe a user study comparing use of SearchBar to status-quo tools such as browser histories, and discuss our findings, which show that users find SearchBar valuable for task reacquisition. Our study also reveals the strategies employed by users of status-quo tools for handling multi-query, multi-session search tasks.

Collaboration on Web search is common in many domains, such as education and knowledge work; recently, HCI researchers have begun to introduce prototype collaborative search tools to support such scenarios. We analyze data from a collaborative search experiment, and based on these data we propose three techniques that can enhance the value of collaborative search tools using personalization: groupization, smart splitting, and group hit-highlighting.

Studies of search habits reveal that people engage in many search tasks involving collaboration with others, such as travel planning, organizing social events, or working on a homework assignment. However, current Web search tools are designed for a single user, working alone. We introduce SearchTogether, a prototype that enables groups of remote users to synchronously or asynchronously collaborate when searching the Web. We describe an example usage scenario, and discuss the ways SearchTogether facilitates collaboration by supporting awareness, division of labor, and persistence. We then discuss the findings of our evaluation of SearchTogether, analyzing which aspects of its design enabled successful collaboration among study participants.

Multi-user, touch-sensing input devices create opportunities for the use of cooperative gestures -- multi-user gestural interactions for single display groupware. Cooperative gestures are interactions where the system interprets the gestures of more than one user as contributing to a single, combined command. Cooperative gestures can be used to enhance users' sense of teamwork, increase awareness of important system events, facilitate reachability and access control on large, shared displays, or add a unique touch to an entertainment-oriented activity. This paper discusses motivating scenarios for the use of cooperative gesturing and describes some initial experiences with CollabDraw, a system for collaborative art and photo manipulation. We identify design issues relevant to cooperative gesturing interfaces, and present a preliminary design framework. We conclude by identifying directions for future research on cooperative gesturing interaction techniques.

We explore how the placement of control widgets (such as menus) affects collaboration and usability for co-located tabletop groupware applications. We evaluated two design alternatives: a centralized set of controls shared by all users, and separate per-user controls replicated around the borders of the shared tabletop. We conducted this evaluation in the context of TeamTag, a system for collective annotation of digital photos. Our comparison of the two design alternatives found that users preferred replicated over shared controls. We discuss the cause of this preference, and also present data on the impact of these interface design variants on collaboration, as well as the role that orientation, co-touching, and the use of different regions of the table played in shaping users' behavior and preferences.

This paper presents a design case study of SIDES: Shared Interfaces to Develop Effective Social Skills. SIDES is a tool designed to help adolescents with Asperger's Syndrome practice effective group work skills using a four-player cooperative computer game that runs on tabletop technology. We present the design process and evaluation of SIDES conducted over six months with a middle school social group therapy class. Our findings indicate that cooperative tabletop computer games are a motivating and supportive tool for facilitating effective group work among our target population and reveal several design lessons to inform the development of similar systems.

DiamondSpin is a toolkit for the efficient prototyping of and experimentation with multi-person, concurrent interfaces for interactive shared displays. In this paper, we identify the fundamental functionality that tabletop user interfaces should embody, then present the toolkit's architecture and API. DiamondSpin provides a novel real-time polar to Cartesian transformation engine that has enabled new, around-the-table interaction metaphors to be implemented. DiamondSpin allows arbitrary document positioning and orientation on a tabletop surface. Polygonal tabletop layouts such as rectangular, octagonal, and circular tabletops can easily be constructed. DiamondSpin also supports multiple work areas within the same digital tabletop. Multi-user operations are offered through multi-threaded input event streams, multiple active objects, and multiple concurrent menus. We also discuss insights on tabletop interaction issues we have observed from a set of applications built with DiamondSpin.

We introduce a system that allows four users to each receive sound from a private audio channel while using a shared tabletop display. In order to explore how private audio channels affect a collaborative work environment, we conducted a user study with this system. The results reveal differences in work strategies when groups are presented with individual versus public audio, and suggest that the use of private audio does not impede group communication and may positively impact group dynamics. We discuss the findings, as well as their implications for the design of future audio-based "single display privacyware" systems.

The status quo for co-located groupware is to assume that "social protocols" (standards of polite behavior) are sufficient to coordinate the actions of a group of users; however, prior studies of groupware use as well as our own observations of groups using a shared tabletop display suggest potential for improving groupware interfaces by incorporating coordination policies - direct manipulation mechanisms for avoiding and resolving conflicts. We discuss our observations of group tabletop usage and present our coordination framework. We conclude with example usage scenarios and discuss future research suggested by this framework.

Interactive tabletops have been previously proposed and studied in the domain of co-located group applications. However, little fundamental research has been done to explore the issue of size. In this paper we identify a number of size considerations for tabletop design, and present an experiment to explore some of these issues, in particular the effects of group size and table size on the speed at which the task was performed, the distribution of work among group members, issues of shared resources, and user preference for table size. Our findings shed light on (1) how work strategies are affected by group size, (2) how social interaction varies with respect to table size, and (3) how the speed of task performance is influenced by group size but not by table size. In addition, our experiments revealed that for larger groups, designers might need to add additional vertical displays for shared information. This finding opens the door for extending singledisplay groupware to shared-display groupware settings that involve multiple, shared displays.

The iStuff toolkit of physical devices, and the flexible software infrastructure to support it, were designed to simplify the exploration of novel interaction techniques in the post-desktop era of multiple users, devices, systems and applications collaborating in an interactive environment. The toolkit leverages an existing interactive workspace in-frastructure, making it lightweight and platform independent. The supporting software framework includes a dynamically configurable intermediary to simplify the mapping of devices to applications. We describe the iStuff architecture and provide several examples of iStuff, organized into a design space of ubiquitous computing interaction components. The main contribution is a physical toolkit for distributed, heterogeneous environments with run-time retargetable device data flow. We conclude with some insights and experiences derived from using this toolkit and framework to prototype experimental interaction techniques for ubiquitous computing environments.