Technology-mediated communication tools are widely used to maintain connections between older adults and their remote family members. However, challenges often arise due to differences in their life rhythms and communication preferences, such as time zones, daily routines, or preferred platforms. To address these challenges, we propose using Ditto, a mimetic embodied agent, in video-call-like interactions between an older adult and a remote family member. When direct interaction is difficult, FamilyDitto can represent either party, providing a strong social presence and a personalized experience to the other person. To explore the potential of Ditto in supporting intergenerational communication, we conducted seven co-design workshops (n=27) with older adults and younger family members. Our thematic analysis reveals Ditto's potential roles as both a temporal bridge and an emotional proxy, identifies the tension between faithful and idealized representations, and emphasizes the importance of personalization to support unique family dynamics. We distill our findings into implications for designing mediated semi-synchronous communication, using idealized representation responsibly, and addressing the asymmetric motivations and comfort levels with Ditto across roles and ages. This study provides a foundation for mimetic AI technologies that enhance, rather than replace, human connection in remote intergenerational relationships.

We introduce ChartA11y, an app developed to enable accessible 2-D visualizations on smartphones for blind users through a participatory and iterative design process involving 13 sessions with two blind partners. We also present a design journey for making accessible touch experiences that go beyond simple auditory feedback, incorporating multimodal interactions and multisensory data representations. Together, ChartA11y aimed at providing direct chart accessing and comprehensive chart understanding by applying a two-mode setting: a semantic navigation framework mode and a direct touch mapping mode. By re-designing traditional touch-to-audio interactions, ChartA11y also extends to accessible scatter plots, addressing the under-explored challenges posed by their non-linear data distribution. Our main contributions encompass the detailed participatory design process and the resulting system, ChartA11y, offering a novel approach for blind users to access visualizations on their smartphones.

Imagine being able to send a personalized embodied agent to meetings you are unable to attend. This paper explores the idea of a Ditto —an agent that visually resembles a person, sounds like them, possesses knowledge about them, and can represent them in meetings. This paper reports on results from two empirical investigations: 1) focus group sessions with six groups (n=24) and 2) a Wizard of Oz (WOz) study with 10 groups (n=39) recruited from within a large technology company. Results from the focus group sessions provide insights on what contexts are appropriate for Dittos, and issues around social acceptability and representation risk. The focus group results also provide feedback on visual design characteristics for Dittos. In the WOz study, teams participated in meetings with two different embodied agents: a Ditto and a Delegate (an agent which did not resemble the absent person). Insights from this research demonstrate the impact these embodied agents can have in meetings and highlight that Dittos in particular show promise in evoking feelings of presence and trust, as well as informing decision making. These results also highlight issues related to relationship dynamics such as maintaining social etiquette, managing one's professional reputation, and upholding accountability. Overall, our investigation provides early evidence that Dittos could be beneficial to represent users when they are unable to be present but also outlines many factors that need to be carefully considered to successfully realize this vision.

The opportunity for artificial intelligence, or AI, to enable accessibility is rapidly growing, but widely impactful applications can be challenging to build given the diversity of user need within and across disability communities. Teachable AI systems give users with disabilities a way to leverage the power of AI to personalize applications for their own specific needs, as long as the effort of providing examples is balanced with the benefit of the personalization received. As an example, this paper presents the design and evaluation of Find My Things, an end-to-end application that can be taught by people who are blind or low vision to find their personal things. Through synthesis of the design process, this paper offers design considerations for the teaching loop that is so critical to realizing the power of teachable AI for accessibility.

Profile pictures can convey rich social signals that are often inaccessible to blind and low vision screen reader users. Although there have been efforts to understand screen reader users’ preferences for alternative (alt) text descriptions when encountering images online, profile pictures evoke distinct information needs. We conducted semi-structured interviews with 16 screen reader users to understand their preferences for various styles of profile picture image descriptions in different social contexts. We also interviewed seven sighted individuals to explore their thoughts on authoring alt text for profile pictures. Our findings suggest that detailed image descriptions and user narrated alt text can provide screen reader users enjoyable and informative experiences when exploring profile pictures. We also identified mismatches between how sighted individuals would author alt text with what screen reader users prefer to know about profile pictures. We discuss the implications of our findings for social applications that support profile pictures.

Even though screen readers are a core accessibility tool for blind and low vision individuals (BLVIs), most visualizations are incompatible with screen readers. To improve accessible visualization experiences, we partnered with 10 BLV screen reader users (SRUs) in an iterative co-design study to design and develop accessible visualization experiences that afford SRUs the autonomy to interactively read and understand visualizations and their underlying data. During the five-month study, we explored accessible visualization prototypes with our design partners for three one-hour sessions. Our results provide feedback on the synthesized design concepts we explored, why (or why not) they aid comprehension and exploration for SRUs, and how differing design concepts can fit into cohesive accessible visualization experiences. We contribute both Chart Reader, a web-based accessibility engine resulting from our design iterations, and our distilled study findings—organized by design dimensions—in the creation of comprehensive accessible visualization experiences.

Only a small percentage of blind and low-vision people use traditional mobility aids such as a cane or a guide dog. Various assistive technologies have been proposed to address the limitations of traditional mobility aids. These devices often give either the user or the device majority of the control. In this work, we explore how varying levels of control affect the users' sense of agency, trust in the device, confidence, and successful navigation. We present Glide, a novel mobility aid with two modes for control: Glide-directed and User-directed. We employ Glide in a study (N=9) in which blind or low-vision participants used both modes to navigate through an indoor environment. Overall, participants found that Glide was easy to use and learn. Most participants trusted Glide despite its current limitations, and their confidence and performance increased as they continued to use Glide. Users' control mode preferences varied in different situations; no single mode "won" in all situations.

Alternative (alt) text provides access to descriptions of digital images for people who use screen readers. While prior work studied screen reader users’ (SRUs’) preferences about alt text and automatic alt text (i.e., alt text generated by artificial intelligence), little work examined the alt text author’s experience composing or editing these descriptions. We built two types of prototype interfaces for two tasks: authoring alt text and providing feedback on automatic alt text. Through combined interview-usability testing sessions with alt text authors and interviews with SRUs, we tested the effectiveness of our prototypes in the context of Microsoft PowerPoint. Our results suggest that authoring interfaces that support authors in choosing what to include in their descriptions result in higher quality alt text. The feedback interfaces highlighted considerable differences in the perceptions of authors and SRUs regarding “high-quality” alt text. Finally, authors crafted significantly lower quality alt text when starting from the automatic alt text compared to starting from a blank box. We discuss the implications of these results on applications that support alt text.

Novel AI-infused educational technologies can give children with blindness the opportunity to explore concepts learned incidentally through vision by using alternative perceptual modalities. However, more effort is needed to support the meaningful use of such technological innovations for evaluations at scale and later wide-spread adoption. This paper presents the development and pilot evaluation of a curriculum to enable educators to support blind learners’ self-exploration of social attention using the PeopleLens technology. We reflect on these learnings to present four design guidelines for creating curricula aimed to enable meaningful use. We then consider how formulations of “success” by our participants can help us think about ways of assessing efficacy in low-incidence disability groups. We conclude by arguing for our community to widen the scope of discourse around assistive technologies from design and engineering to include supporting their meaningful use.

Artificial Intelligence (AI) for accessibility is a rapidly growing area, requiring datasets that are inclusive of the disabled users that assistive technology aims to serve. We offer insights from a multi-disciplinary project that constructed a dataset for teachable object recognition with people who are blind or low vision. Teachable object recognition enables users to teach a model objects that are of interest to them, e.g., their white cane or own sunglasses, by providing example images or videos of objects. In this paper, we make the following contributions: 1) a disability-first procedure to support blind and low vision data collectors to produce good quality data, using video rather than images; 2) a validation and evolution of this procedure through a series of data collection phases and 3) a set of questions to orient researchers involved in creating datasets toward reflecting on the needs of their participant community.

AI technologies are often used to aid people in performing discrete tasks with well-defined goals (e.g., recognising faces in images). Emerging technologies that provide continuous, real-time information enable more open-ended AI experiences. In partnership with a blind child, we explore the challenges and opportunities of designing human-AI interaction for a system intended to support social sensemaking. Adopting a research-through-design perspective, we reflect upon working with the uncertain capabilities of AI systems in the design of this experience. We contribute: (i) a concrete example of an open-ended AI system that enabled a blind child to extend his own capabilities; (ii) an illustration of the delta between imagined and actual use, highlighting how capabilities derive from the human-AI interaction and not the AI system alone; and (iii) a discussion of design choices to craft an ongoing human-AI interaction that addresses the challenge of uncertain outputs of AI systems.

People enjoy listening to music as part of their life. This makes music an excellent choice for designing a user-friendly brain-computer interface (BCI) for long-term use. We propose a novel BCI system using music stimuli that relies on brain signals collected via Smartfones, an EEG recording device integrated into a pair of headphones. In a user study of the proposed system, participants were asked to pay attention to one of three musical instruments playing simultaneously from separate spatial directions. We used a stimulus reconstruction method to decode attention from EEG signals. Results show that the proposed system can achieve good decoding accuracy (>70%) while providing superior user-friendliness compared to a traditional EEG setup.

Object recognition has made great advances in the last decade, but predominately still relies on many high-quality training examples per object category. In contrast, learning new objects from only a few examples could enable many impactful applications from robotics to user personalization. Most few-shot learning research, however, has been driven by benchmark datasets that lack the high variation that these applications will face when deployed in the real-world. To close this gap, we present the ORBIT dataset and benchmark, grounded in the real-world application of teachable object recognizers for people who are blind/low-vision. The dataset contains 3,822 videos of 486 objects recorded by people who are blind/low-vision on their mobile phones. The benchmark reflects a realistic, highly challenging recognition problem, providing a rich playground to drive research in robustness to few-shot, high-variation conditions. We set the benchmark’s first state-of-the-art and show there is massive scope for further innovation, holding the potential to impact a broad range of real-world vision applications including tools for the blind/low-vision community. We release the dataset at https://doi.org/10.25383/city.14294597 and benchmark code at https://github.com/microsoft/ORBIT-Dataset.

Through an iterative design process using Wizard of Oz (WOz) prototypes, we designed a video calling application for people with Autism Spectrum Disorder. Our Video Calling for Autism prototype provided an Expressiveness Mirror that gave feedback to autistic people on how their facial expressions might be interpreted by their neurotypical conversation partners. This feedback was in the form of emojis representing six emotions and a bar indicating the amount of overall expressiveness demonstrated by the user. However, when we built a working prototype and conducted a user study with autistic participants, their negative feedback caused us to reconsider how our design process led to a prototype that they did not find useful. We reflect on the design challenges around developing AI technology for an autistic user population, how Wizard of Oz prototypes can be overly optimistic in representing AI-driven prototypes, how autistic research participants can respond differently to user experience prototypes of varying fidelity, and how designing for people with diverse abilities needs to include that population in the development process.

Despite the large body of work in accessibility concerning the design of novel navigation technologies, little is known about commonly available technologies that people with visual impairments currently use for navigation. We address this gap with a qualitative study consisting of interviews with 23 people with visual impairments, ten of whom also participated in a follow-up diary study. We develop the idea of complementarity first introduced by Williams et al. [53] and find that in addition to using apps to complement mobility aids, technologies and apps complemented each other and filled in for the gaps inherent in one another. Furthermore, the complementarity between apps and other apps/aids was primarily the result of the differences in information and modalities in which this information is communicated by apps, technology and mobility aids. We propose design recommendations to enhance this complementarity and guide the development of improved navigation experiences for people with visual impairments.

Virtual reality (VR) has the potential to transform many aspects of our daily lives, including work, entertainment, communication, and education. However, there has been little research into understanding the usability of VR for people with mobility limitations. In this paper, we present the results of an exploration to understand the accessibility of VR for people with limited mobility. We conducted semi-structured interviews with 16 people with limited mobility about their thoughts on, and experiences with, VR systems. We identified 7 barriers related to the physical accessibility of VR devices that people with limited mobility might encounter, ranging from the initial setup of a VR system to keeping VR controllers in view of cameras embedded in VR headsets. We also elicited potential improvements to VR systems that would address some accessibility concerns. Based on our findings, we discuss the importance of considering the abilities of people with limited mobility when designing VR systems, as the abilities of many participants did not match the assumptions embedded in the design of current VR systems.

Current Virtual Reality (VR) technologies focus on rendering visuospatial effects, and thus are inaccessible for blind or low vision users. We examine the use of a novel white cane controller that enables navigation without vision of large virtual environments with complex architecture, such as winding paths and occluding walls and doors. The cane controller employs a lightweight three-axis brake mechanism to provide large-scale shape of virtual objects. The multiple degrees-of-freedom enables users to adapt the controller to their preferred techniques and grip. In addition, surface textures are rendered with a voice coil actuator based on contact vibrations; and spatialized audio is determined based on the progression of sound through the geometry around the user. We design a scavenger hunt game that demonstrates how our device enables blind users to navigate a complex virtual environment. Seven out of eight users were able to successfully navigate the virtual room (6x6m) to locate targets while avoiding collisions. We conclude with design consideration on creating immersive non-visual VR experiences based on user preferences for cane techniques, and cane material properties.

Despite the major role of Global Positioning Systems (GPS) as a navigation tool for people with visual impairments (VI), a crucial missing aspect of point-to-point navigation with these systems is the last-few-meters wayfinding problem. Due to GPS inaccuracy and inadequate map data, systems often bring a user to the vicinity of a destination but not to the exact location, causing challenges such as difficulty locating building entrances or a specific storefront from a series of stores. In this paper, we study this problem space in two parts: (1) A formative study (N=22) to understand challenges, current resolution techniques, and user needs; and (2) A design probe study (N=13) using a novel, vision-based system called Landmark AI to understand how technology can better address aspects of this problem. Based on these investigations, we articulate a design space for systems addressing this challenge, along with implications for future systems to support precise navigation for people with VI.

Current virtual reality applications do not support people who have low vision, i.e., vision loss that falls short of complete blindness but is not correctable by glasses. We present SeeingVR, a set of 14 tools that enhance a VR application for people with low vision by providing visual and audio augmentations. A user can select, adjust, and combine different tools based on their preferences. Nine of our tools modify an existing VR application post hoc via a plugin without developer effort. The rest require simple inputs from developers using a Unity toolkit we created that allows integrating all 14 of our low vision support tools during development. Our evaluation with 11 participants with low vision showed that SeeingVR enabled users to better enjoy VR and complete tasks more quickly and accurately. Developers also found our Unity toolkit easy and convenient to use.

Nonvisually Accessible Video Calling (NAVC) is a prototype that detects visual conversation cues in a video call and uses audio cues to convey them to a user who is blind or low-vision. NAVC uses audio cues inspired by movie soundtracks to convey Attention, Agreement, Disagreement, Happiness, Thinking, and Surprise. When designing NAVC, we partnered with people who are blind or low-vision through a user-centered design process that included need-finding interviews and design reviews. To evaluate NAVC, we conducted a user study with 16 participants. The study provided feedback on the NAVC prototype and showed that the participants could easily discern some cues, like Attention and Agreement, but had trouble distinguishing others. The accuracy of the prototype in detecting conversation cues emerged as a key concern, especially in avoiding false positives and in detecting negative emotions, which tend to be masked in social conversations. This research identified challenges and design opportunities in using AI models to enable accessible video calling.

In rural parts of the developing world, spreading awareness about critical issues in health, governance, and other topics is challenging and costly. Traditional media such as print, radio and TV each have limitations and offer little guarantee that new information is absorbed or retained by the target population. This paper describes Learn2Earn, a system that leverages mobile payments to bolster public awareness campaigns in rural India. Users call an Interactive Voice Response (IVR) system, listen to a brief audio tutorial, and take a multiple-choice quiz to check their understanding. People who pass the quiz receive a mobile top-up (about $0.14) and have the opportunity to earn additional credits by referring others to the system. We describe a pilot deployment of Learn2Earn in rural India that spread via word-of-mouth to over 15,000 people within seven weeks. Usage was concentrated among young men, many of them students. In a mixed-methods study, we draw upon call logs, electronic surveys, qualitative interviews, and other sources of data to suggest that Learn2Earn could be an effective way to build awareness about important topics.

Video calling (VC) aims to create multi-modal, collaborative environments that are "just like being there." However, we found that autistic individuals, who exhibit atypical social and cognitive processing, may not share this goal. We interviewed autistic adults about their perceptions of VC compared to other computer- mediated communications (CMC) and face-to-face interactions. We developed a neurodiversity-sensitive model of CMC that describes how stressors such as sensory sensitivities, cognitive load, and anxiety, contribute to their preferences for CMC channels. We learned that they apply significant effort to construct coping strategies to support their sensory, cognitive, and social needs. These strategies include moderating their sensory inputs, creating mental models of conversation partners, and attempting to mask their autism by adopting neurotypical behaviors. Without effective strategies, interviewees experience more stress, have less capacity to interpret verbal and non-verbal cues, and feel less empowered to participate. Our findings reveal critical needs for autistic users. We suggest design opportunities to support their ability to comfortably use VC, and in doing so, point the way towards making VC more comfortable for all.

We contribute a qualitative investigation of how teens with visual impairments (VIP) access smartphone photography, from the time they take photos through editing and sharing them on social media. We observed that they largely want to engage with photos visually, similarly to their sighted peers, and have developed strategies around photo capture, editing, sharing, and consumption that attempt to mitigate usability limitations of current photography and social media apps. We demonstrate the need for more work examining how young people with low vision engage with smartphone photography and social media, as they are heavy users of such technologies and have challenges distinct from their totally blind counterparts. We conclude with design considerations to alleviate the usability barriers we uncovered and for making smartphone photography and social media more accessible and relevant for VIPs.

Accessing images online is often difficult for users with vision impairments. This population relies on text descriptions of images that vary based on website authors' accessibility practices. Where one author might provide a descriptive caption for an image, another might provide no caption for the same image, leading to inconsistent experiences. In this work, we present the Caption Crawler system, which uses reverse image search to find existing captions on the web and make them accessible to a user's screen reader. We report our system's performance on a set of 481 websites from alexa.com's list of most popular sites to estimate caption coverage and latency, and also report blind and sighted users' ratings of our system's output quality. Finally, we conducted a user study with fourteen screen reader users to examine how the system might be used for personal browsing.

Alt text (short for "alternative text") is descriptive text associated with an image in HTML and other document formats. Screen reader technologies speak the alt text aloud to people who are visually impaired. Introduced with HTML 2.0 in 1995, the alt attribute has not evolved despite significant changes in technology over the past two decades. In light of the expanding volume, purpose, and importance of digital imagery, we reflect on how alt text could be supplemented to offer a richer experience of visual content to screen reader users. Our contributions include articulating the design space of representations of visual content for screen reader users, prototypes illustrating several points within this design space, and evaluations of several of these new image representations with people who are blind. We close by discussing the implications of our taxonomy, prototypes, and user study findings.

We present the results of an exploration to understand the accessibility of smartphone photography for people with motor impairments. We surveyed forty-six people and interviewed twelve people about capturing, editing, and sharing photographs on smartphones. We found that people with motor impairments encounter many challenges with smartphone photography, resulting in users capturing fewer photographs than they would like. Participants described various strategies they used to overcome challenges in order to capture a quality photograph. We also found that photograph quality plays a large role in deciding which photographs users share and how often they share, with most participants rating their photographs as average or poor quality compared to photos shared on their social networks. Additionally, we created design probes of two novel photography interfaces and received feedback from our interview participants about their usefulness and functionality. Based on our findings, we propose design recommendations for how to improve the accessibility of mobile photoware for people with motor impairments.

This research takes an orientation to visual impairment (VI) that does not regard it as fixed or determined alone in or through the body. Instead, we consider (dis)ability as produced through interactions with the environment and configured by the people and technology within it. Specifically, we explore how abilities become negotiated through video ethnography with six VI athletes and spectators during the Rio 2016 Paralympics. We use generated in-depth examples to identify how technology can be a meaningful part of ability negotiations, emphasizing how these embed into the social interactions and lives of people with VI. In contrast to treating technology as a solution to a 'sensory deficit', we understand it to support the triangulation process of sense-making through provision of appropriate additional information. Further, we suggest that technology should not try and replace human assistance, but instead enable people with VI to better identify and interact with other people in-situ.

Traditional virtual reality (VR) mainly focuses on visual feedback, which is not accessible for people with visual impairments. We created Canetroller, a haptic cane controller that simulates white cane interactions, enabling people with visual impairments to navigate a virtual environment by transferring their cane skills into the virtual world. Canetroller provides three types of feedback: (1) physical resistance generated by a wearable programmable brake mechanism that physically impedes the controller when the virtual cane comes in contact with a virtual object; (2) vibrotactile feedback that simulates the vibrations when a cane hits an object or touches and drags across various surfaces; and (3) spatial 3D auditory feedback simulating the sound of real-world cane interactions. We designed indoor and outdoor VR scenes to evaluate the effectiveness of our controller. Our study showed that Canetroller was a promising tool that enabled visually impaired participants to navigate different virtual spaces. We discuss potential applications supported by Canetroller ranging from entertainment to mobility training.

There has been a surge in artificial intelligence (AI) technologies co-opted by or designed for people with visual disabilities. Researchers and engineers have pushed technical boundaries in areas such as computer vision, natural language processing, location inference, and wearable computing. But what do people with visual disabilities imagine as their own technological future? To explore this question, we developed and carried out tactile ideation workshops with participants in the UK and India. Our participants generated a large and diverse set of ideas, most focusing on ways to meet needs related to social interaction. In some cases, this was a matter of recognizing people. In other cases, they wanted to be able to participate in social situations without foregrounding their disability. It was striking that this finding was consistent across UK and India despite substantial cultural and infrastructural differences. In this paper, we describe a new technique for working with people with visual disabilities to imagine new technologies that are tuned to their needs and aspirations. Based on our experience with these workshops, we provide a set of social dimensions to consider in the design of new AI technologies: social participation, social navigation, social maintenance, and social independence. We offer these social dimensions as a starting point to forefront users' social needs and desires as a more deliberate consideration for assistive technology design.

Research advancements allow computational systems to automatically caption social media images. Often, these captions are evaluated with sighted humans using the image as a reference. Here, we explore how blind and visually impaired people experience these captions in two studies about social media images. Using a contextual inquiry approach (n=6 blind/visually impaired), we found that blind people place a lot of trust in automatically generated captions, filling in details to resolve differences between an image's context and an incongruent caption. We built on this in-person study with a second, larger online experiment (n=100 blind/visually impaired) to investigate the role of phrasing in encouraging trust or skepticism in captions. We found that captions emphasizing the probability of error, rather than correctness, encouraged people to attribute incongruence to an incorrect caption, rather than missing details. Where existing research has focused on encouraging trust in intelligent systems, we conclude by challenging this assumption and consider the benefits of encouraging appropriate skepticism.

Augmentative and alternative communication (AAC) devices are a critical technology for people with disabilities that affect their speech. One challenge with AAC systems is their inability to portray aspects of nonverbal communication that typically accent, complement, regulate, or substitute for verbal speech. In this paper, we explore the design space of awareness displays that can supplement AAC devices, considering their output features and their effects on the perceptions of interlocutors. Through designing prototypes and getting feedback on our designs from people with ALS, their primary caregivers, and other communication partners, we consider (1) the consistent tensions that arose between abstractness and clarity in meaning for these designs and (2) the ways in which these designs can further mark users as "other." Overall, we contribute a generative understanding of designing AAC awareness displays to augment and contextualize communication.

In this work, we present EduFeed, a system that enables preliterate children to explore an algorithmically-mediated social feed of learning exercises, select activities to engage in, and share them with their peers. We deployed EduFeed as a technology probe to three classrooms of ESL students in the first year of elementary school in India who had limited English literacy and limited experience with touchscreen technology. We found that children were able to self-direct their engagement with the system and were initially motivated by digital sharing, while social context surrounding the system in physical space was also important. Based on our design and probe deployment, we reflect on issues relevant to adapting the social feed paradigm to this context, such as ranking algorithms, physicality, immediacy, and bridging physical and digital collaborative experiences.

Mobile and ubiquitous computing researchers have long envisioned future worlds for users in developed regions. Steered by such visions, they have innovated devices and services exploring the value of alternative propositions with and for individuals, groups and communities. Meanwhile, such radical and long-term explorations are uncommon for what have been termed emergent users; users, that is, for whom advanced technologies are just within grasp. Rather, a driving assumption is that today’s high-end mobile technologies will “trickle down” to these user groups in due course. In this paper, we open the debate about what mobile technologies might be like if emergent users were directly involved in creating their visions for the future 5–10 years from now. To do this, we report on a set of envisioning workshops in India, South Africa and Kenya that provide a roadmap for valued, effective devices and services for these regions in the next decade.

Estimating eye centers is an important computer vision problem with several applications. In the past, eye center localization was constrained by the use of special hardware such as infrared cameras. Methods that estimate eye centers based on visible light have also been suggested in the literature, but these methods are inaccurate when used with low resolution images and wide ranges of lighting. We propose a novel method that can be used to detect eye centers efficiently, even with low resolution images provided by a webcam. Our method takes into consideration the circular nature of the iris and its intensity difference as compared to the sclera. It then uses an energy based active contour called Snakuscule for capturing the iris. We test our algorithm's robustness towards changes in pose, illumination and scale using the BioID database and Yale Face Database B. Our method compares well with existing state-of-the-art techniques in terms of accuracy, is easy to implement and exhibits real-time performance.

We present the first analysis of the use and non-use of social media platforms by low-income blind users in rural and peri-urban India. Using a mixed-methods approach of semi-structured interviews and observations, we examine the benefits received by low-income blind people from Facebook, Twitter and WhatsApp and investigate constraints that impede their social media participation. We also present a detailed analysis of how low-income blind people used a voice-based social media platform deployed in India that received significant traction from low-income people in rural and peri-urban areas. In eleven-weeks of deployment, fifty-three blind participants in our sample collectively placed 4784 voice calls, contributed 1312 voice messages, cast 33,909 votes and listened to the messages 46,090 times. Using a mixed-methods analysis of call logs, qualitative interviews, and phone surveys, we evaluate the strengths and weaknesses of the platform and benefits it offered to low-income blind people.

Interactive voice forums have emerged as a promising platform for people in developing regions to record and share audio messages using low-end mobile phones. However, one of the barriers to the scalability of voice forums is the process of screening and categorizing content, often done by a dedicated team of moderators. We present Sangeet Swara, a voice forum for songs and cultural content that relies on the community of callers to curate high-quality posts that are prioritized for playback to others. An 11-week deployment of Sangeet Swara found broad and impassioned usage, especially among visually impaired users. We also conducted a follow-up experiment, called Talent Hunt, that sought to reduce reliance on toll-free telephone lines. Together, our deployments span about 53,000 calls from 13,000 callers, who submitted 6,000 posts and 150,000 judgments of other content. Using a mixed-methods analysis of call logs, audio content, comparison with outside judges, and 204 automated phone surveys, we evaluate the user experience, the strengths and weaknesses of community moderation, financial sustainability, and the implications for future systems.

With the wide penetration of mobile internet, social networking (SN) systems are becoming increasingly popular in the developing world. However, most SN sites are text heavy, and are therefore unusable by low-literate populations. Here we ask what would an SN application for low-literate users look like and how would it be used? We designed and deployed KrishiPustak, an audio-visual SN mobile application for low-literate farming populations in rural India. Over a four month deployment, 306 farmers registered through the phones of eight agricultural mediators making 514 posts and 180 replies. We conducted interviews with farmers and mediators and analyzed the content to understand system usage and to drive iterative design. The context of mediated use and agricultural framing had a powerful impact on system understanding (what it was for) and usage. Overall, KrishiPustak was useful and usable, but none-the-less we identify a number of design recommendations for similar SN systems.

Though many researchers have studied how to incentivize people to respond to surveys, little is known about how these incentives impact respondents' willingness to recruit others to participate as well. In this paper, we show that the incentives offered for individual survey responses can have a dramatic impact on the overall reach of a survey through a network of peers. In a field experiment in India, we made a survey accessible via mobile phones and offered respondents either a fixed incentive (guaranteed payment of about $0.17) or a lottery incentive (1% chance of winning $17). When asked to choose, a significant fraction of respondents preferred the lottery incentive. However, when encouraged to spread the survey, the fixed incentive spread over 100 times further, reaching about 800 people in a day. We interpret this surprising result and discuss the implications for HCI.

We measure the effectiveness of a traditional honor code at deterring cheating in an online examination, and we compare it to that of a stern warning. Through experimental evaluation in a 409-student online course, we find that a pre-task warning leads to a significant decrease in the rate of cheating while an honor code has a smaller (non-significant) effect. Unlike much prior work, we measure the rate of cheating directly and we do not rely on potentially inaccurate post-examination surveys. Our findings demonstrate that replacing traditional honor codes with warnings could be a simple and effective way to deter cheating in online courses.

Students in the developing world are frequently cited as being among the most important beneficiaries of online education initiatives such as massive open online courses (MOOCs). While some predict that online classrooms will replace physical classrooms, our experience suggests that blending online and in-person instruction is more likely to succeed in developing regions. However, very little research has actually been done on the effects of online education or blended learning in these environments. In this paper we describe a blended learning initiative that combines videos from a large online course with peer-led sessions for undergraduate technical education in India. We performed a randomized controlled trial (RCT) that indicates our intervention was associated with a small but significant improvement in performance on a summative exam. We discuss the results of the RCT and an ethnographic study of the intervention to make recommendations for future, scalable blended learning initiatives for places such as India.

While most MOOCs rely on world-famous experts to teach the masses, in many circumstances students may learn more from people who share their context such as local teachers or peers. Here, we describe an experiment to explore how the "source" of video content, the teacher, affects online learning, specifically in the context of higher education in Indian colleges. The proposed experiment will compare three content sources -- a local lecturer (teacher from an Indian engineering college), a local peer (both male and female students similar to the targeted audience), and an internationally recognized expert (a Stanford lecturer). Students will watch videos by the various source authors, after which we will measure differences in their preference, engagement, and learning. In addition, we discuss our experiences with helping students prepare video lectures and describe the support and processes we used to curate interesting and clear peer-generated content.

Many Internet services depend on the integrity of their users, even when these users have strong incentives to behave dishonestly. Drawing on experiments in two different online contexts, this study measures the prevalence of cheating and evaluates two different methods for deterring it. Our first experiment investigates cheating behavior in a pair of online exams spanning 632 students in India. Our second experiment examines dishonest behavior on Mechanical Turk through an online task with 2,378 total participants. Using direct measurements that are not dependent on self-reports, we detect significant rates of cheating in both environments. We confirm that honor codes--despite frequent use in massive open online courses (MOOCs)--lead to only a small and insignificant reduction in online cheating behaviors. To overcome these challenges, we propose a new intervention: a stern warning that spells out the potential consequences of cheating. We show that the warning leads to a significant (about twofold) reduction in cheating, consistent across experiments. We also characterize the demographic correlates of cheating on Mechanical Turk. Our findings advance the understanding of cheating in online environments, and suggest that replacing traditional honor codes with warnings could be a simple and effective way to deter cheating in online courses and online labor marketplaces.

Videos are becoming an increasingly popular medium for communicating information, especially for online education. Recent efforts by organizations like Coursera, edX, Udacity and Khan Academy have produced thousands of educational videos with hundreds of millions of views in their attempt to make high quality teaching available to the masses. As a medium, videos are time-consuming to produce and cannot be easily modified after release. As a result, errors or problems with legibility are common. While text-based information platforms like Wikipedia have benefitted enormously from crowdsourced contributions for the creation and improvement of content, the various limitations of video hinder the collaborative editing and improvement of educational videos. To address this issue, we present VidWiki, an online platform that enables students to iteratively improve the presentation quality and content of educational videos. Through the platform, users can improve the legibility of handwriting, correct errors, or translate text in videos by overlaying typeset content such as text, shapes, equations, or images. We conducted a small user study in which 13 novice users annotated and revised Khan Academy videos. Our results suggest that with only a small investment of time on the part of viewers, it may be possible to make meaningful improvements in online educational videos.

A fundamental tenet of user-centered design is that the needs, wants, limitations, and contexts of end users are central to the process of creating products and services that can be used and understood by the people who will use them. Most of the time these end users aren’t all that different from the people designing the technology. But as the differences increase between designers and the people they’re designing for, understanding and empathizing with users becomes harder and even more important. As we build software for people and communities with vastly diverse backgrounds, cultures, languages, and education, we need to stretch our ideas of what users want and need and how best to serve them.

Due to the recent emergence of massive open online courses (MOOCs), students and teachers are gaining unprecedented access to high-quality educational content. However, many questions remain on how best to utilize that content in a classroom environment. In this small-scale, exploratory study, we compared two ways of using a recorded video lecture. In the online learning condition, students viewed the video on a personal computer, and also viewed a follow-up tutorial (a quiz review) on the computer. In the blended learning condition, students viewed the video as a group in a classroom, and received the follow-up tutorial from a live lecturer. We randomly assigned 102 students to these conditions, and assessed learning outcomes via a series of quizzes. While we saw significant learning gains after each session conducted, we did not observe any significant differences between the online and blended learning groups. We discuss these findings as well as areas for future work.

Recent years have seen enormous growth of online educational videos, spanning K-12 tutorials to university lectures. As this content has grown, so too has grown the number of presentation styles. Some educators have strong allegiance to handwritten recordings (using pen and tablet), while others use only typed (PowerPoint) presentations. In this paper, we present the first systematic comparison of these two presentation styles and how they are perceived by viewers. Surveys on edX and Mechanical Turk suggest that users enjoy handwriting because it is personal and engaging, yet they also enjoy typeface because it is clear and legible. Based on these observations, we propose a new presentation style, TypeRighting, that combines the benefits of handwriting and typeface. Each phrase is written by hand, but fades into typeface soon after it appears. Our surveys suggest that about 80% of respondents prefer TypeRighting over handwriting. The same fraction of respondents prefer TypeRighting over typeface, for videos in which the handwriting is sufficiently legible.

Designing ICT systems for rural users in the developing world is difficult for a variety of reasons ranging from problems with infrastructure to wide differences in user contexts and capabilities. Developing regions may include huge variability in spoken languages, and users are often low- or non-literate, with very little experience interacting with digital technologies. Researchers have explored the use of text-free graphical interfaces as well as speech-based applications to overcome some of the issues related to language and literacy. While there are benefits and drawbacks to each of these approaches, they can be complementary when used together. In this work, we present VideoKheti, a mobile system using speech, graphics, and touch interaction for low-literate farmers in rural India. VideoKheti helps farmers to find and watch agricultural extension videos in their own language and dialect. In this paper, we detail the design and development of VideoKheti and report on a field study with 20 farmers in rural India who were asked to find videos based on a scenario. The results show that farmers could use VideoKheti, but their success still greatly depended on their education level. While participants were enthusiastic about using the system, the multimodal interface did not overcome many obstacles for low-literate users.

One of the greatest challenges in designing applications for economically poor communities is that potential users may have little or no education. We investigated how limited education appears to impact the ability to navigate a hierarchical UI, even when it has no text. We scored 60 participants from low-income communities in India using tests of textual literacy and Raven's Progressive Matrices. These were used as proxies for educational level and a subset of cognitive abilities. We then evaluated participants' performance on a UI task involving hierarchical navigation. First, our results confirm that textual literacy is correlated with scores on the Raven's test. In addition, we found that performance on both instruments are predictive of performance in navigating UI hierarchies, even when the UI is text-free. This provides statistically significant confirmation of previous anecdotal hypotheses. We conclude with design recommendations for UI hierarchies for people with limited education.

While e-governance is acclaimed as a means to decentralisation, and an efficiency and accountability enhancing mechanism, it can be implemented in different ways. In a strong centralized state like the Indian state, decentralization is often pursued in a centralized manner through top-down interventions. This paper, traces the implementation of two centrally driven e-governance interventions in the state of Karnataka, India i.e. Helpline and Aasthi to argue that while 'centralized decentralization' may be justified on grounds of standardization, it can have divergent outcomes, many of which are often contrary to the objectives of decentralization. The experience of Helpline and Aasthi belies the claim of e-governance being an efficiency and accountability enhancing mechanism. On the contrary, the centralized approach to decentralization in implementing Helpline and Aasthi has weakened the accountability of the state and limited the efficiency gains of urban decentralization.

For years, researchers have explored the use of mobile phone reminders to improve adherence to medication. However, few studies have measured the direct medical benefit of those reminders, especially for low-literate populations in the developing world. This paper describes the use of automated voice calls to promote adherence to iron supplements among pregnant women in urban India. Unlike prior studies, we assess impact via a direct measurement of hemoglobin (Hb) levels in the blood. We enrolled 130 pregnant women from a low-income area of Mumbai, India and randomly assigned them to control and treatment groups. Both groups received a counseling session and a free supply of medication. The treatment group also received short audio messages, three times per week for a period of three months, encouraging them to take iron supplements. Results suggest that automated calls positively impacted Hb levels. However, because we could only recover 79 women for follow-up, and the effect size was small, our results lack statistical power (average change in Hb = 0.43 g/dL, 95% CI = -0.13 0.98 g/dL, p=0.13). We conclude that automated calls deserve further consideration for reducing maternal anemia, and we share our lessons learned for the benefit of future interventions.

Mobile devices are increasingly powerful and flexible tools for computing and communication. When ICTD workers are given a mobile phone "for work", what else do they do? And to what extent can or should an employer shape that use? This note presents research in progress, focused on rules that development projects impose to govern use of mobile devices. This work maps these rules against actual instrumental (work-related, non-prescribed) and non-instrumental (personal) device use, and enforcement of these rules, in eight projects using a popular mobile-based job aid, CommCare. We present early insights from qualitative analysis of two such deployments in India identifying a range of often conflicting policy choices that affect device use for project mission and/or professional and personal empowerment. We explore tradeoffs for morale, work quality, mission, and device integrity. We identify user remote availability, soft intimidation, and validation as mechanisms to shift authority and credibility of information sources. The implications of our findings are increasingly important as governments and NGOs arm frontline workers with mobile devices as tools to improve service delivery.

Although HCI researchers and practitioners frequently work with groups of people that differ significantly from themselves, little attention has been paid to the effects these differences have on the evaluation of HCI systems. Via 450 interviews in Bangalore, India, we measure participant response bias due to interviewer demand characteristics and the role of social and demographic factors in influencing that bias. We find that respondents are about 2.5x more likely to prefer a technological artifact they believe to be developed by the interviewer, even when the alternative is identical. When the interviewer is a foreign researcher requiring a translator, the bias towards the interviewer's artifact increases to 5x. In fact, the interviewer's artifact is preferred even when it is degraded to be obviously inferior to the alternative. We conclude that participant response bias should receive more attention within the CHI community, especially when designing for underprivileged populations.

Global crowdsourcing platforms could offer new employment opportunities to low-income workers in developing countries. However, the impact to date has been limited because poor communities usually lack access to computers and the Internet.

Shared access to airtime is a prominent mode of connectivity access in the developing world. We seek to understand airtime sharing among low-income microenterprises in India (small, low-capital businesses, such as flower sellers and milkmen), that constitute 90% of the total enterprises in India. We introduce social negotiation as the foundation of airtime sharing. We highlight negotiation mechanisms in the microenterprise, showing how shared resources are used towards personal interests amidst tensions and value conflicts, by adapting, modifying, subverting, and repurposing airtime. We then explore the design space of airtime and bandwidth sharing in low-income communities, including designing for negotiation and improving readability of airtime.

We present the results of two surveys and a qualitative interview-based study with users of screen readers in India. Our early interviews moved us in the direction of examining patterns that differentiate users of two particular software applications -- the dominant market standard JAWS and the free, open source challenger NVDA. A comparison between the two is timely and particularly relevant to issues elsewhere in the developing world. In the short term, the question of choosing one application over another could be based on price and support for custom-made applications, but in the long term, issues of language support are likely to be of concern as well. We explore software adoption behavior and present results that show the relationship between the quality of audio and peoples' willingness to use one software over another. We also compare the switch from JAWS to NVDA to other kinds of switches from dominant software to open source options. In conclusion, we discuss the business aspects of screen readers and examine why the comparison between these two applications is particularly important in the discussion on accessible personal computing for people with vision impairments in the developing world.

In this paper we present results from an anthropological study of everyday mobile internet adoption among teenagers in a low-income urban setting. We attempt to use this study to explore how information about everyday ICT use may be relevant for development research even if it is largely dominated by entertainment uses. To understand how ICT tools are used, we need to study the spaces users inhabit, even if these spaces are dominated by mundane, non-instrumental and entertainment-driven needs. The key here is for ICTD discourse to situate insights from anthropological studies (such as this one) within an understanding of what drives a specific user population to adopt technologies in particular ways. Clearly there is a link between context and use, and understanding this may be invaluable for development research. Adopting a narrow development lens of technology use may miss the actual engagements and ingenious strategies marginal populations use to instate technologies into their everyday.

Electronic response systems known as "clickers" have demonstrated educational benefits in well-resourced classrooms, but remain out-of-reach for most schools due to their prohibitive cost. We propose a new, low-cost technique that utilizes computer vision for real-time polling of a classroom. Our approach allows teachers to ask a multiple-choice question. Students respond by holding up a qCard: a sheet of paper that contains a printed code, similar to a QR code, encoding their student IDs. Students indicate their answers (A, B, C or D) by holding the card in one of four orientations. Using a laptop and an off-the-shelf webcam, our software automatically recognizes and aggregates the students' responses and displays them to the teacher. We built this system and performed initial trials in secondary schools in Bangalore, India. In a 25-student classroom, our system offers 99.8% recognition accuracy, captures 97% of responses within 10 seconds, and costs 15 times less than existing electronic solutions.

In this paper, we present the design, implementation, and deployment of a phone-based broadcasting system designed for reaching out to at-risk populations in urban India. We worked in collaboration with Pragati, a non-governmental organization dedicated to assisting Urban Sex Workers (USWs) in Bangalore, India, with the goal of improving Pragati's outreach to the women they serve. We conducted ethnographic action research to understand and address the needs of Pragati and the lifestyles of USWs. Responding to the unique design constraints of the USW community such as specific privacy and timing constraints, a desire to remain invisible, and the unusually high rate of mobile phone use, we designed a phone-based broadcasting system for Pragati. We then deployed the system on four different occasions and application areas. We present the results and key findings from our study, and conclude with a discussion on how designing for particularly difficult cases such as USWs can shed new light on the design of mobile applications for the developing world in general, such as challenging ubiquity and phone numbers as identity.

The Technology for Emerging Markets (TEM) group at Microsoft Research India seeks to address the needs and aspirations of people in the world's developing communities. Our research targets people who are just beginning to use computing technologies and services as well as those for whom access to computing still remains largely out of reach. Most of our work falls under the rubric of the relatively young field of Information and Communication Technology for Development (ICTD or ICT4D). Reflecting the interdisciplinary nature of ICTD, TEM is a multidisciplinary group engaged in a range of technical and social-science research. We work in the areas of cultural anthropology, sociology, political science, economics, and psychology, all of which help us understand the social context of technology and how it relates to communities and individual users. We combine this understanding with technical research in hardware and software to devise solutions for underserved communities in rural and urban environments around the world.

The developing world faces infrastructural challenges in providing Western-style educational computing technologies, but on the other hand observes very high cell phone penetration. However, the use of mobile technology has not been extensively explored in the context of collaborative learning. New projection and display technologies for mobile devices raise the important question of whether to use single or multiple displays in these environments. In this paper, we explore two mobile-based techniques for using co-located collaborative game-play to supplement ESL (English as a Second Language) education in a developing region: (1) Mobile Single Display Groupware: a pico-projector connected to a cell phone, with a handheld controller for each child to interact, and (2) Mobile Multiple Display Groupware: a phone for each child. We explore the types of interaction that occur in both of these conditions and the impact on learning outcomes.

We explore the potential of using camera phones and pico projectors in rapid creation and presentation of digital content in a development context. A camera phone based content authoring application was designed and deployed with three different user populations in the domains of classroom education and health care. Our findings show that despite the variations in education levels, cultural background, and technology exposure, users successfully created and presented different forms of digital content using the camera phone and pico projector.

Reading text from document images can be difficult on mobile devices due to the limited screen width available on them. While there exist solutions for reflowing Latin-script texts on such devices, these solutions do not work well for images of other scripts or combinations of scripts, since they rely on script-specific characteristics or OCR. We present a technique that reflows text in document images in a manner that is agnostic to the script used to compose them. Our technique achieved over 95% segmentation accuracy for a corpus of 139 images containing text in 4 genetically-distant languages-English, Hindi, Kannada and Arabic. A preliminary user study with a prototype implementation of the technique provided evidence of some of its usability benefits.

Researchers have long been interested in the potential of ICTs to enable positive change in developing regions communities. In these environments, ICT interventions often fail because political, social and cultural forces work against the changes ICTs entail. We argue that familiar uses of ICTs for information services in these contexts are less potent than their use for persuasion and motivation in order to facilitate change. We focus on India's rural maternal health system where health workers are employed in villages to persuade pregnant women to utilize health services. Health workers face challenges due to resistance to change in the village, and because of their limited education, training and status. These factors appear to reduce the motivation of health workers and impair their performance. For two months, we deployed short videos on mobile phones designed to persuade village women and motivate health workers. We also asked health workers to record their own videos. While our results are preliminary, they show evidence that the creation and use of videos did help (1) engage village women in dialogue, (2) show positive effects toward health worker motivation and learning, and (3) motivate key community influencers to participate in promoting the health workers.

We describe ViralVCD: a low cost method for tracing paths of information diffusion in developing communities using physical media. We instituted a participatory video framework for creation and dissemination of developmental videos in seven urban slums and peri-urban communities of Bangalore, India. By combining a call-in contest with Video CDs, we were able to measure developmental impact as well as elicit data on social networks and technology usage practices. In particular, our technique was able to extract data from multiple layers-social, technological, and developmental. ViralVCD allowed us to identify key actors and map information diffusion, as well as technology ownership and access. These findings have implications for HCI initiatives targeting low income locales and populations.

We describe a prevalent mode of information access in low-income communities of the developing world--intermediated interactions. They enable persons for whom technology is inaccessible due to non-literacy, lack of technology-operation skills, or financial constraints, to benefit from technologies through digitally skilled users--thus, expanding the reach of technologies. Reporting the results of our ethnography in two urban slums of Bangalore, India, we present three distinct intermediated interactions: inputting intent into the device in proximate enabling, interpretation of device output in proximate translation, and both input of intent and interpretation of output in surrogate usage. We present some requirements and challenges in interface design of these interactions and explain how they are different from direct interactions. We then explain the broader effects of these interactions on low-income communities, and present some implications for design.

The Multiple Mice project demonstrated the financial [1] and learning [2] benefits of enabling students in resource-constrained schools to share one computer. In India, the lack of Multiple Mice authoring tools coupled with teachers' low computer proficiency means little or no customized content is created. This is problematic as the capability to create digital content enables teachers to prepare digital lessons that address the particular learning needs of their students [3], [4], [5].

We present a study on using storytelling for teaching skills to low-income workers in the developing world. Taking a cue from work on using dramatized stories and video to promote technology use and agricultural and HIV/AIDS education, we investigated storytelling's ability for teaching low-literacy populations. We created a series of videos to teach domestic workers in urban India bed-making and vacuuming. We tested the effect on learning of a) embedding instructional content in narratives and b) adding motivational content on the benefits of learning these skills. We compared: 1) instruction-only videos, 2) instructional videos book-ended with voice-overs describing skills' benefits, 3) combined instructional and narrative videos showing no skill learning benefits; and 4) combined instructional and narrative videos which portray benefits for learning a skill. Narrative framing and motivational content each improved learning, but combining them resulted in dramatic improvement.

One of the greatest challenges in designing applications for developing communities is that potential users may have limited literacy. Past work in UI design for low-literate users has focused on illiteracy as the inability to read per se, with little recognition to other cognitive differences between literate and non-literate users.

The Indian government has undertaken an ambitious strategy for financial inclusion (FI) as part of its development agenda. With the aid of technology-enabled branchless banking initiatives, this drive has been successful in regards to extending access---nearly 60% of the Indian population is banked. However, empirical evidence suggests that the majority of bank accounts are not being utilized, especially not by the poor who are the target of FI. This paper examines the reasons for such underutilization and also recommends ways to improve the FI drive. The paper contributes to the strand of ICTD literature that focuses on FI in two ways. First, it makes clear that the measures of FI success should not be focused on access alone. The real impact comes from appropriate usage of these accounts. Second, it argues that financial education (FE) should be integrated into the FI drive. This would help the poor to more effectively exploit their links to formal financial services and decrease their reliance on costly informal alternatives.

We present Collage, a software application designed for classroom presentations for school teachers in the developing world. An in-depth investigation of teaching practices in several schools in India led us to believe that a simple tool that enabled the display of images and textbook materials while facilitating blackboard-like interactions would be very helpful for these teachers. Collage is a simple media viewer with a small number of features that enables teachers to prepare lessons with little overhead and then present them in classrooms with maximum flexibility. The tool was piloted in three schools in suburban India for use in real-world classroom teaching. All teachers who used Collage uniformly praised it, and students' learning of visual concepts seemed to improve through it. Interestingly, some teachers continue to use the tool now for their own teaching needs and have spontaneously shared it with colleagues from other schools.

We investigate how people interact with Web search engine result pages using eye-tracking. While previous research has focused on the visual attention devoted to the 10 organic search results, this paper examines other components of contemporary search engines, such as ads and related searches. We systematically varied the type of task (informational or navigational), the quality of the ads (relevant or irrelevant to the query), and the sequence in which ads of different quality were presented. We measured the effects of these variables on the distribution of visual attention and on task performance. Our results show significant effects of each variable. The amount of visual attention that people devote to organic results depends on both task type and ad quality. The amount of visual attention that people devote to ads depends on their quality, but not the type of task. Interestingly, the sequence and predictability of ad quality is also an important factor in determining how much people attend to ads. When the quality of ads varied randomly from task to task, people paid little attention to the ads, even when they were good. These results further our understanding of how attention devoted to search results is influenced by other page elements, and how previous search experiences influence how people attend to the current page.

This paper explores user authentication schemes for banking systems implemented over mobile phone networks in the developing world. We analyze an authentication scheme currently deployed by an Indian mobile banking service provider which uses a combination of PINs and printed codebooks for authenticating users. As a first step, we report security weaknesses in that scheme and show that it is susceptible to easy and efficient PIN recovery attacks. We then propose a new scheme which offers better secrecy of PINs, while still maintaining the simplicity and scalability advantages of the original scheme. Finally, we investigate the usability of the two schemes with a sample of 34 current and potential customers of the banking system. Our findings suggest that the new scheme is more efficient, less susceptible to human error and better preferred by the target consumers.

Users find it hard to delete unimportant personal information which often results in cluttered workspaces. We present a full design cycle for GrayArea, a novel interface that allows users to demote unimportant files by dragging them to a gray area at the bottom of their file folders. Demotion is an intermediate option between keeping and deleting. It combines the advantages of deletion (unimportant files don't compete for attention) and keeping (files are retrieved in their folder context). We developed the GrayArea working prototype using thorough iterative design. We evaluated it by asking 96 participants to 'clean' two folders with, and without, GrayArea. Using GrayArea reduced folder clutter by 13%. Further, 81% of participants found it easier to demote than delete files, and most indicated they would use GrayArea if provided in their operating systems. The results provide strong evidence for the demotion principle suggested by the user-subjective approach.

An understanding of how people allocate their visual attention when viewing Web pages is very important for Web authors, interface designers, advertisers and others. Such knowledge opens the door to a variety of innovations, ranging from improved Web page design to the creation of compact, yet recognizable, visual representations of long pages. We present an eye-tracking study in which 20 users viewed 361 Web pages while engaged in information foraging and page recognition tasks. From this data, we describe general location-based characteristics of visual attention for Web pages dependent on different tasks and demographics, and generate a model for predicting the visual attention that individual page elements may receive. Finally, we introduce the concept of fixation impact, a new method for mapping gaze data to visual scenes that is motivated by findings in vision research.

Mischief is a system for classroom interaction that allows multiple children to use individual mice and cursors to interact with a single large display [20]. While the system can support large groups of children, it is unclear how children's performance is affected as group size increases. We explore this question via a study involving two tasks, with children working in group sizes ranging from 1 to 32. The first required reciprocal selection of two on-screen targets, resembling a swarm pointing scenario that might be used in educational applications. The second, a more temporally and spatially distributed pointing task, had children entering different words by selecting characters on an on-screen keyboard. Results indicate that performance is significantly affected by group size only when targets are small. Further, group size had a smaller effect when pointing was spatially and temporally distributed than when everyone was concurrently aiming at the same targets.

People regularly interact with different representations of Web pages. A person looking for new information may initially find a Web page represented as a short snippet rendered by a search engine. When he wants to return to the same page the next day, the page may instead be represented by a link in his browser history. Previous research has explored how to best represent Web pages in support of specific task types, but, as we find in this paper, consistency in representation across tasks is also important. We explore how different representations are used in a variety of contexts and present a compact representation that supports both the identification of new, relevant Web pages and the re-finding of previously viewed pages.

This paper attempts to re-imagine ubiquitous computing for populations in low-income and information-challenged environments. We examine information infrastructures in mid-sized urban slums of Mumbai and Bangalore in three ways--1) highlighting technologies supporting social networks, 2) examining underlying notions of trust and privacy in building information networks, and 3) discussing protocols and practices around shared access. We then discuss our thoughts on designing for low-income, low-literacy, and resource-challenged communities, presenting new ways to think about the design of ubiquitous technologies for international development. We argue for collaborative exchange between the established strengths of the Information and Communication Technologies for Development (ICT4D) and Ubicomp communities to generate new ways of shaping technologies towards poverty alleviation in previously neglected socio-economic contexts--Ubicomp4D.

Acquiring small targets on a tablet or touch screen can be challenging. To address the problem, researchers have proposed techniques that enlarge the effective size of targets by extending targets into adjacent screen space. When applied to targets organized in clusters, however, these techniques show little effect because there is no space to grow into. Unfortunately, target clusters are common in many popular applications. We present Starburst, a space partitioning algorithm that works for target clusters. Starburst identifies areas of available screen space, grows a line from each target into the available space, and then expands that line into a clickable surface. We present the basic algorithm and extensions. We then present 2 user studies in which Starburst led to a reduction in error rate by factors of 9 and 3 compared to traditional target expansion.

For decades, Fitts' law (1954) has been used to model pointing time in user interfaces. As with any rapid motor act, faster pointing movements result in increased errors. But although prior work has examined accuracy as the "spread of hits," no work has formulated a predictive model for error rates (0-100%) based on Fitts' law parameters. We show that Fitts' law mathematically implies a predictive error rate model, which we derive. We then describe an experiment in which target size, target distance, and movement time are manipulated. Our results show a strong model fit: a regression analysis of observed vs. predicted error rates yields a correlation of R2=.959 for N=90 points. Furthermore, we show that the effect on error rate of target size (W) is greater than that of target distance (A), indicating a departure from Fitts' law, which maintains that W and A contribute proportionally to index of difficulty (ID). Our error model can be used with Fitts' law to estimate and predict error rates along with speeds, providing a framework for unifying this dichotomy.

Web search services are among the most heavily used applications on the World Wide Web. Perhaps because search is used in such a huge variety of tasks and contexts, the user interface must strike a careful balance to meet all user needs. We describe a study that used eye tracking methodologies to explore the effects of changes in the presentation of search results. We found that adding information to the contextual snippet significantly improved performance for informational tasks but degraded performance for navigational tasks. We discuss possible reasons for this difference and the design implications for better presentation of search results.

Web search engines present search results in a rank ordered list. This works when what a user wants is near the top, but sometimes the information that the user really wants is located at the bottom of the page. This study examined how users' search behaviors vary when target results were displayed at various positions for informational and navigational tasks. We found that when targets were placed relatively low in the first page of search results, people spent more time searching and were less successful in finding the target, especially for informational tasks. Further analysis of eye movements showed that the decrease in search performance was partially due to the fact that users rarely looked at lower ranking results. The large decrease in performance for informational search is probably because users have high confidence in the search engine's ranking; in contrast to navigational tasks, where the target is more obvious from information presented in the results, in informational tasks, users try out the top ranked results even if these results are perceived as less relevant for the task.

Using a notebook to sketch designs, reflect on a topic, or capture and extend creative ideas are examples of active note taking tasks. Optimal experience for such tasks demands concentration without interruption. Yet active note taking may also require reference documents or emails from team members. InkSeine is a Tablet PC application that supports active note taking by coupling a pen-and-ink interface with an in situ search facility that flows directly from a user's ink notes (Fig. 1). InkSeine integrates four key concepts: it leverages preexisting ink to initiate a search; it provides tight coupling of search queries with application content; it persists search queries as first class objects that can be commingled with ink notes; and it enables a quick and flexible workflow where the user may freely interleave inking, searching, and gathering content. InkSeine offers these capabilities in an interface that is tailored to the unique demands of pen input, and that maintains the primacy of inking above all other tasks.

Systems for fast search of personal information are rapidly becoming ubiquitous. Such systems promise to dramatically improve personal information management, yet most are modeled on Web search in which users know very little about the content that they are searching. We describe the design and deployment of a system called Phlat that optimizes search for personal information with an intuitive interface that merges search and browsing through a variety of associative and contextual cues. In addition, Phlat supports a unified tagging (labeling) scheme for organizing personal content across storage systems (files, email, etc.). The system has been deployed to hundreds of employees within our organization. We report on both quantitative and qualitative aspects of system use. Phlat is available as a free download at http://research.microsoft.com/adapt/phlat.

Modes allow a few inputs to invoke many operations, yet if a user misclassifies or forgets the state of a system, modes can result in errors. Spring-loaded modes (quasimodes) maintain a mode while the user holds a control such as a button or key. The Springboard is an interaction technique for tablet computers that extends quasimodes to encompass multiple tool modes in a single spring-loaded control. The Springboard allows the user to continue holding down a nonpreferred-hand command button after selecting a tool from a menu as a way to repeatedly apply the same tool. We find the Springboard improves performance for both a local marking menu and for a non-local marking menu ("lagoon") at the lower left corner of the screen. Despite the round-trip costs incurred to move the pen to a tool lagoon, a keystroke-level analysis of the true cost of each technique reveals the local marking menu is not significantly faster.

Snapping is a widely used technique that helps users position graphical objects precisely, e.g., to align them with a grid or other graphical objects. Unfortunately, whenever users want to position a dragged object close to such an aligned location, they first need to deactivate snapping. We propose snap-and-go, a snapping technique that overcomes this limitation. By merely stopping dragged objects at aligned positions, rather than "warping" them there, snap-and-go helps users align objects, yet still allows placing dragged objects anywhere else. While this approach of inserting additional motor space renders snap-and-go slightly slower than traditional snapping, snap-and-go simplifies the user interface by eliminating the need for a deactivation option and thereby allows introducing snapping to application scenarios where traditional snapping is inapplicable. In our user studies, participants were able to align objects up to 138% (1D) and 231% (2D) faster with snap-and-go than without and snap-and-go proved robust against the presence of distracting snap targets.

ZoneZoom is an input technique that lets users traverse large information spaces on smartphones. Our technique ZoneZoom, segments a given view of an information space into nine sub-segments, each of which is mapped to a key on the number keypad of the smartphone. This segmentation can be hand-crafted by the information space author or dynamically created at run-time. ZoneZoom supports "spring-loaded" view shifting which allows users to easily "glance" at nearby areas and then quickly return to their current view. Our ZoneZoom technique lets users gain an overview and compare information from different parts of a dataset. SmartPhlow is an optimized application for browsing a map of local-area road traffic conditions.

The Implicit Query (IQ) prototype is a system which automatically generates context-sensitive searches based on a user's current computing activities. In the demo, we show IQ running when users are reading or composing email. Queries are automatically generated by analyzing the email message, and results are presented in a small pane adjacent to the current window to provide peripheral awareness of related information.

Most information retrieval technologies are designed to facilitate information discovery. However, much knowledge work involves finding and re-using previously seen information. We describe the design and evaluation of a system, called Stuff I've Seen (SIS), that facilitates information re-use. This is accomplished in two ways. First, the system provides a unified index of information that a person has seen, whether it was seen as email, web page, document, appointment, etc. Second, because the information has been seen before, rich contextual cues can be used in the search interface. The system has been used internally by more than 230 employees. We report on both qualitative and quantitative aspects of system use. Initial findings show that time and people are important retrieval cues. Users find information more easily using SIS, and use other search tools less frequently after installation.

We propose a formal experimental paradigm designed to help evaluate scrolling interaction techniques. Such a method is needed by interaction designers to quantify scrolling performance, thereby providing a tool to evaluate and improve upon new techniques. We systematically vary the scrolling distance as well as the required tolerance of scrolling. Distance and tolerance are the parameters of Fitts' Law, which traditionally has been applied to the evaluation of pointing devices in tasks involving rapid, aimed movement to visible targets. Scrolling involves acquisition of targets well beyond the edges of the screen, yet Fitts' Law models our experimental data very wellWe apply our paradigm to the IBM ScrollPoint and the IntelliMouse Wheel. Our experimental approach reveals a crossover effect in performance versus distance, with the Wheel performing best at short distances but the ScrollPoint performing best at long distances. We also demonstrate that the performance of the Wheel can be significantly improved using an acceleration algorithm. These results show that our approach yields a practical and rigorous method for the evaluation of scrolling techniques.

We developed and evaluated seven interfaces for integrating semantic category information with Web search results. List interfaces were based on the familiar ranked-listing of search results, sometimes augmented with a category name for each result. Category interfaces also showed page titles and/or category names, but re-organized the search results so that items in the same category were grouped together visually. Our user studies show that all Category interfaces were more effective than List interfaces even when lists were augmented with category names for each result. The best category performance was obtained when both category names and individual page titles were presented. Either alone is better than a list presentation, but both together provide the most effective means for allowing users to quickly examining search results. These results provide a better understanding of the perceptual and cognitive factors underlying the advantage of category groupings and provide some practical guidance to Web search interface designers. Keywords User Interface, World Wide Web, Search, User Study, Usability, Text Categorization, Focus-In-Context