Child-agent interaction research has largely focused on automated voice agents, although privacy concerns and quality of interactions serve as a rationale for using teleoperated agents in some settings. Research on neurodivergent children's interactions with voice agents has largely focused on therapy and classroom contexts as opposed to home contexts. We conducted a remote case study via Zoom observing six families with neurodivergent children playing with a tangible, parent-operated voice agent at home for up to eight weeks. We used quantitative measures of agent utilization, qualitative analysis of video recordings, and parent survey responses to identify implications for teleoperation interface design for playful voice agents. Based on our analysis, we argue joint media engagement principles are applicable to a broader set of play activities, and teleoperated agents present affordances and challenges distinct from those of automated agents.

People with visual impairments (PVIs) are less likely to participate in physical activity than their sighted peers. One barrier is the lack of accessible group-based aerobic exercise classes, often due to instructors not giving accessible verbal instructions. While there is research in exercise tracking, these tools often require vision or familiarity with the exercise. There are accessible solutions that give personalized verbal feedback in slower-paced exercises, not generalizing to aerobics. In response, we have developed an algorithm that detects shoeprints on a sensor mat using computer vision and a CNN. We can infer whether a person is following along with a step aerobics workout and are designing reactive verbal feedback to guide the person to rejoin the class. Future work will include finishing development and conducting a user study to assess the effectiveness of the reactive verbal feedback.

Physical activity is an important part of quality life, however people with visual impairments (PVIs) are less likely to participate in physical activity than their sighted peers. One barrier is that exercise instructors may not give accessible verbal instructions. There is a potential for text analysis to determine these phrases, and in response provide more accessible instructions. First, a taxonomy of accessible phrases needs to be developed. To address this problem, we conducted user studies with 10 PVIs exercising along with audio and video aerobic workouts. We analyzed video footage of their exercise along with interviews to determine a preliminary set of phrases that are helpful or confusing. We then conducted an iterative qualitative analysis of six other exercise videos and sought expert feedback to derive our taxonomy. We hope these findings inform systems that analyze instructional phrases for accessibility to PVIs.

Vehicle manufacturers and government agencies are considering using vehicle-to-pedestrian (V2P) communication to improve pedestrian safety. However, there are unanswered questions about whether people will heed alerts and warnings presented through a smartphone. We conducted between-subject studies with younger and older adults where they physically crossed a virtual street. They received either permissive alerts (safe to cross), prohibitive warnings (not safe to cross), or no alerts or warnings (control). We found that both older and younger adults were highly likely to heed permissive alerts, even when this meant taking gaps between two vehicles that were smaller than they would typically take on their own. We also found that we could shift participants’ road-crossing behavior toward greater caution when they were only alerted to cross very large gaps between two vehicles. Participants stated that alerts and warnings were useful, but that prohibitive warnings were annoying. These findings give insights into V2P design and pedestrian behavior when smartphone assistance is provided.

Research predicts that 196 million people will be diagnosed with Age-Related Macular Degeneration (AMD) by 2020. People who experience AMD and other vision loss face barriers that affect their Quality of Life (QoL). People experience only modest improvement from technologies (e.g., screen readers, CCTV), tools (e.g., magnifying glasses, tactile buttons), and human help (e.g., friends, blindness organizations). Further, there are issues to accessing these resources based on one's place of residence. To explore these challenges and determine design implications to support people who have experienced vision loss (PVL), we conducted a qualitative semi-structured interview study exploring QoL with 10 PVL. We uncovered themes of supporting creative work, recognizing the impact of one's living in a non-urban setting on QoL, and increasing efficiency at accomplishing tasks. We motivate the inclusion of PVL in the design process because they learned skills while sighted and are now low vision or blind.

Virtual Reality (VR) is a growing source of entertainment, but people who are visually impaired have not been effectively included. Audio cues are motivated as a complement to visuals, making experiences more immersive, but are not a primary cue. To address this, we implemented a VR game called Virtual Showdown. We based Virtual Showdown on an accessible real-world game called Showdown, where people use their hearing to locate and hit a ball against an opponent. Further, we developed Verbal and Verbal/Vibration Scaffolds to teach people how to play Virtual Showdown. We assessed the acceptability of Virtual Showdown and compared our scaffolds in an empirical study with 34 youth who are visually impaired. Thirty-three participants wanted to play Virtual Showdown again, and we learned that participants scored higher with the Verbal Scaffold or if they had prior Showdown experience. Our empirical findings inform the design of future accessible VR experiences.

Access to a variety of exercises is important for maintaining a healthy lifestyle. This variety includes physical activity in public spaces. A 400-meter jogging track is not accessible because it provides solely visual cues for people to remain in their lane. As a first step toward making exercise spaces accessible, we conducted an ecologically valid Wizard of Oz study to compare the accuracy and user experience of human guide, verbal, wrist vibration, and head beat feedback while people walked around the track. The technology conditions did not affect accuracy, but the order of preference was human guide, verbal, wrist vibration, and head beat. Participants had a difficult time perceiving vibrations when holding their cane or guide dog, and lower frequency sounds made it difficult to focus on their existing navigation strategies.

Deploying novel technologies requires the coordinated efforts of the research team, research participants, and a variety of community members and project stakeholders. To ensure that the project is completed successfully, these disparate groups of people engage in articulation work, which is the meta-work that supports the use of collaborative systems. In this paper, we examine the articulation work surrounding the deployment of systems that have found limited long-term adoption: assistive technology. Specifically, we examine three research deployments of a collaborative game for children with autism. Analysis of the articulation work performed during these studies demonstrates how research deployments of technologies create conditions in which stakeholders must take on additional roles to make the deployment work. By understanding the articulation work surrounding deployment studies engendered in this role fluidity, we can improve both research design and the analysis of data emergent from these studies.

Every child should have an equal opportunity to learn, play, and participate in his or her life. In this work, we investigate how interactive technology design features support children with and without disabilities with inclusion during play. We developed four versions of Incloodle, a two-player picture-taking tablet application, designed to be inclusive of children with different abilities and needs. Each version of the application varied in (1) whether or not it enforced co-operation between children; and in (2) whether it prompted interactions through in-app characters or more basic instructions. A laboratory study revealed technology-enforced cooperation was helpful for child pairs who needed scaffolding, but character-based prompting had little effect on children's experiences. We provide an empirical evaluation of interactive technology for inclusive play and offer guidance for designing technology that facilitates inclusive play between young neurotypical and neurodiverse children.

People who are blind or low-vision may have a harder time participating in exercise due to inaccessibility or lack of experience. We employed Value Sensitive Design (VSD) to explore the potential of technology to enhance exercise for people who are blind or low-vision. We conducted 20 semi-structured interviews about exercise and technology with 10 people who are blind or low-vision and 10 people who facilitate fitness for people who are blind or low-vision. We also conducted a survey with 76 people to learn about outsider perceptions of hypothetical exercise with people who are blind or low-vision. Based on our interviews and survey, we found opportunities for technology development in four areas: 1) mainstream exercise classes, 2) exercise with sighted guides, 3) rigorous outdoors activity, and 4) navigation of exercise spaces. Design considerations should include when and how to deliver auditory or haptic information based on exercise and context, and whether it is acceptable to develop less mainstream technologies if they enhance mainstream exercise. The findings of this work seek to inform the design of accessible exercise technologies.

Students learning American Sign Language (ASL) have trouble searching for the meaning of unfamiliar signs. ASL signs can be differentiated by a small set of simple features including hand shape, orientation, location, and movement. In a feature-based ASL-to-English dictionary, users search for a sign by providing a query, which is a set of observed features. Because there is natural variability in the way signs are executed, and observations are error-prone, an approach other than exact matching of features is needed. We propose ASL-Search, an ASL-to-English dictionary entirely powered by its users. ASL-Search utilizes Latent Semantic Analysis (LSA) on a database of feature-based user queries to account for variability. To demonstrate ASL-Search's viability, we created ASL-Flash, a learning tool that presents online flashcards to ASL students and provides query data. Our simulations on this data serve as a proof of concept, demonstrating that our dictionary's performance improves with use and performs well for users with varied levels of ASL experience.

Email is the primary method of digital communication for most people, but the overwhelming quantity has led to a poverty of attention. Existing manual and automatic solutions that aim to save important emails from falling through the cracks have begun to address this problem, but may increase user workload, sacrifice efficiency, or fail to identify high value communications. In response, we developed MinEMail, an alert system that uses a text message (SMS) to remind and notify users of critical emails that may have been missed or forgotten. MinEMail provides an alert infrastructure as well as accurately labeling and predicting which emails are critical, and when and how they need to be addressed. To motivate our system, we also present an up-front study with 777 participants that aims to understand the state and limitations of email and SMS in enterprise. We conduct an experience sampling study of over 3000 emails in order to construct MinEMail's predictive models. Finally, we present the results from a 15 user ecologically valid real-world deployment of MinEMail in enterprise.

While email is a major conduit for information sharing in enterprise, there has been little work on exploring the files sent along with these messages -- attachments. These accompanying documents can be large (multiple megabytes), lengthy (multiple pages), and not optimized for the smaller screen sizes, limited reading time, and expensive bandwidth of mobile users. Thus, attachments can increase data storage costs (for both end users and email servers), drain users' time when irrelevant, cause important information to be missed when ignored, and pose a serious access issue for mobile users. To address these problems we created AttachMate, a novel email attachment summarization system. AttachMate can summarize the content of email attachments and automatically insert the summary into the text of the email. AttachMate also stores all files in the cloud, reducing file storage costs and bandwidth consumption. In this paper, the primary contribution is the AttachMate client/server architecture. To ground, support and validate the AttachMate system we present two upfront studies (813 participants) to understand the state and limitations of attachments, a novel algorithm to extract representative concept sentences (tested through two validation studies), and a user study of AttachMate within an enterprise.

People who are blind or low vision may have a harder time participating in exercise classes due to inaccessibility, travel difficulties, or lack of experience. Exergames can encourage exercise at home and help lower the barrier to trying new activities, but there are often accessibility issues since they rely on visual feedback to help align body positions. To address this, we developed Eyes-Free Yoga, an exergame using the Microsoft Kinect that acts as a yoga instructor, teaches six yoga poses, and has customized auditory-only feedback based on skeletal tracking. We ran a controlled study with 16 people who are blind or low vision to evaluate the feasibility and feedback of Eyes-Free Yoga. We found participants enjoyed the game, and the extra auditory feedback helped their understanding of each pose. The findings of this work have implications for improving auditory-only feedback and on the design of exergames using depth cameras.

Many theories of human debugging rely on complex mental constructs that offer little practical advice to builders of software engineering tools. Although hypotheses are important in debugging, a theory of navigation adds more practical value to our understanding of how programmers debug. Therefore, in this paper, we reconsider how people go about debugging in large collections of source code using a modern programming environment. We present an information foraging theory of debugging that treats programmer navigation during debugging as being analogous to a predator following scent to find prey in the wild. The theory proposes that constructs of scent and topology provide enough information to describe and predict programmer navigation during debugging, without reference to mental states such as hypotheses. We investigate the scope of our theory through an empirical study of 10 professional programmers debugging a real-world open source program. We found that the programmers' verbalizations far more often concerned scent-following than hypotheses. To evaluate the predictiveness of our theory, we created an executable model that predicted programmer navigation behavior more accurately than comparable models that did not consider information scent. Finally, we discuss the implications of our results for enhancing software engineering tools.

Blind mobile device users face security risks such as inaccessible authentication methods, and aural and visual eavesdropping. We interviewed 13 blind smartphone users and found that most participants were unaware of or not concerned about potential security threats. Not a single participant used optional authentication methods such as a password-protected screen lock. We addressed the high risk of unauthorized user access by developing PassChords, a non-visual authentication method for touch surfaces that is robust to aural and visual eavesdropping. A user enters a PassChord by tapping several times on a touch surface with one or more fingers. The set of fingers used in each tap defines the password. We give preliminary evidence that a four-tap PassChord has about the same entropy, a measure of password strength, as a four-digit personal identification number (PIN) used in the iPhone's Passcode Lock. We conducted a study with 16 blind participants that showed that PassChords were nearly three times as fast as iPhone's Passcode Lock with VoiceOver, suggesting that PassChords are a viable accessible authentication method for touch screens.

Despite decades of research into how professional programmers debug, only recently has work emerged about how end-user programmers attempt to debug programs. Without this knowledge, we cannot build tools to adequately support their needs. This article reports the results of a detailed qualitative empirical study of end-user programmers' sensemaking about a spreadsheet's correctness. Using our study's data, we derived a sensemaking model for end-user debugging and categorized participants' activities and verbalizations according to this model, allowing us to investigate how participants went about debugging. Among the results are identification of the prevalence of information foraging during end-user debugging, two successful strategies for traversing the sensemaking model, potential ties to gender differences in the literature, sensemaking sequences leading to debugging progress, and sequences tied with troublesome points in the debugging process. The results also reveal new implications for the design of spreadsheet tools to support end-user programmers' sensemaking during debugging.

Although there has been significant research into gender regarding educational and workplace practices, there has been little awareness of gender differences as they pertain to software tools, such as spreadsheet applications, that try to support end users in problem-solving tasks. Although such software tools are intended to be gender agnostic, we believe that closer examination of this premise is warranted. Therefore, in this paper, we report an end-to-end investigation into gender differences with spreadsheet software. Our results showed gender differences in feature usage, feature-related confidence, and tinkering (playful exploration) with features. Then, drawing implications from these results, we designed and implemented features for our spreadsheet prototype that took the gender differences into account. The results of an evaluation on this prototype showed improvements for both males and females, and also decreased gender differences in some outcome measures, such as confidence. These results are encouraging, but also open new questions for investigation. We also discuss how our results compare to generalization studies performed with a variety of other software platforms and populations.

In recent years, systems have emerged that enable end users to “mash” together existing web services to build new web sites. However, little is known about how well end users succeed at building such mashups, or what they do if they do not succeed at their first attempt. To help fill this gap, we took a fresh look, from a debugging perspective, at the approaches of end users as they attempted to create mashups. Our results reveal the end users' debugging strategies and strategy barriers, the gender differences between the debugging strategies males and females followed and the features they used, and finally how their debugging successes and difficulties interacted with their design behaviors.

In recent years, the software engineering community has begun to study program navigation and tools to support it. Some of these navigation tools are very useful, but they lack a theoretical basis that could reduce the need for ad hoc tool building approaches by explaining what is fundamentally necessary in such tools. In this paper, we present PFIS (Programmer Flow by Information Scent), a model and algorithm of programmer navigation during software maintenance. We also describe an experimental study of expert programmers debugging real bugs described in real bug reports for a real Java application. We found that PFIS' performance was close to aggregated human decisions as to where to navigate, and was significantly better than individual programmers' decisions.

Recent research has begun to report that female end-user programmers are often more reluctant than males to employ features that are useful for testing and debugging. These earlier findings suggest that, unless such features can be changed in some appropriate way, there are likely to be important gender differences in end-user programmerspsila benefits from these features. In this paper, we compare end-user programmerspsila feature usage in an environment that supports end-user debugging, against an extension of the same environment with two features designed to help ameliorate the effects of low self-efficacy. Our results show ways in which these features affect female versus male enduser programmerspsila self-efficacy, attitudes, usage of testing and debugging features, and performance.

Previous findings have revealed the ability of information foraging to model or predict where developers will navigate within source code. However, the previous investigation did not consider whether the places developers went were the right places to go. In this paper, we present afield study in which we investigated over 200 open source bug reports and feature requests. We analyzed the textual similarity of these issues in relation to the source code, and determined what files developers had changed to fix these issues. Our results demonstrate that information scent can narrow down quite well where developers should make fixes, implying that future software navigation tools can predict the appropriate places to make fixes based solely on the contents of the issue and the source code.

Although there have been a number of studies of end-user software development tasks, few of them have considered gender issues for real end-user developers in real-world environments for end-user programming. In order to be trusted, the results of such laboratory studies must always be re-evaluated with fewer controls, more closely reflecting real-world conditions. Therefore, the research question in this paper is whether the results of a gender HCI controlled study generalize - to real-world end-user developers, in a real-world spreadsheet environment, using a real-world spreadsheet. Our findings are that the concepts revealed by the original laboratory study appear to be quite robust, being demonstrated in multiple ways in this real-world environment.

There has been little research into how end-user programming environments can provide explanations that could fill a critical information gap for end-user debuggers - help with debugging strategy. To address this need, we designed and prototyped a video-based approach for explaining debugging strategy, and accompanied it with a text-only approach. We then conducted a qualitative empirical study with end-user debuggers. The results reveal the influences of the explanations on end-user debuggers' decision making, how users reacted to the video versus textual media, and the information gaps the explanations closed. The results also reveal issues of particular importance to explanations of this type.