Finding employment is extremely challenging for returning citizens – formerly incarcerated individuals – as they face additional barriers due to their criminal records, gaps in employment, and limited experience with digital technology. Using design studio workshops and focus groups, we explored how returning citizens in the United States experience digital job search and how digital platforms and job search tools could be improved. We find that returning citizens are uncertain about when and how to disclose their felony status, and develop unique tactics in response. When faced with concerns about employment discrimination, especially racism, returning citizens maintained a practical outlook. And, participants expressed specific features they sought in technology platforms for job search: clear, simple job descriptions; auto-fill for commonly occurring input fields; and options for human assistance. Throughout, we pay special attention to Black/African American returning citizens, who have historically been discriminated by the criminal justice system in the United States.

This article theorizes and proposes a novel construct, community digital capacity , to measure collective digital capacity at a community level. Community digital capacity is the extent to which the culture, infrastructure, and digital competence of family and community enable and support digital practices. We address a critical gap in individual digital literacy assessments and address limitations with existing theories that do not show digital inequities in the context of underlying systemic and structural challenges posed by one's social position. Building on insights from Computer Supportive Cooperative Work and Social Computing and Human-Computer Interaction for Development communities, we recognize that digital training initiatives must shift toward critical cultural and social practices that encourage full participation in community affairs. Accordingly, we created 28 items covering three domains---individual, social, and infrastructure. We conducted cognitive interviews with a public housing community to refine the items and capture the construct fully. We assessed their factor structure in two Southeastern Michigan cohorts. After dropping eight items based on contribution to Standardized Root Mean Square Residual (SRMR), the public housing residents exhibit a two-factor structure (SRMR=0.09) consisting of nearly independent factors for the individual and social domains, with all items loading positively on their respective domain. We contribute an initial measure for researchers and practitioners to assess community members' access to shared digital resources and support, offering a tool to assess broader social and structural factors contributing to the digital divide.

When ethnic minorities in the Global South use social media to document human rights abuses against them by majority groups or state actors, platforms often remove this content, thus suppressing these groups' ability to raise awareness about their oppression. To understand ethnic minorities in the Global South's experiences with content moderation, we conducted interviews with ethnic minority Nigerian journalists, activists, politicians, and lawyers, and digital ethnographic observation of social media posts about human rights violations. We found that participants use social media as immediate and urgent archival tools to document violence and human rights abuses against them and their communities, which would not be possible with traditional media due to systemic state exclusion. Further, we identify cultural complexities and context gaps that are frequently ignored by social media companies, who impose Global North norms and values when moderating diverse populations in the Global South. When ethnic minorities post about violence and human rights concerns, they often experience over-moderation, while harmful content that targets them with violence is often under-moderated. We argue for a specialized ''social justice archive moderation'' as a way to account for the documentary and archival functions that social media serves for marginalized populations in conflict zones. We argue that this approach helps to support ethnically, linguistically, culturally, and politically vulnerable populations when moderating Global South populations in high-stakes violence and human rights contexts.

While Israeli suppression of Palestinian voices is well-understood, much less is known about the Palestinian authorities’ repression of Palestinians – the very people they are supposed to represent. This paper investigates digital repression by Hamas and the Palestinian Authority through semi-structured interviews – in-person and online – with 19 Palestinian activists who post on social media. Many of our findings echo those from other repressive contexts, but the unusual Palestinian context also gives rise to several unique elements. For example, Palestinian authorities, while incorporating some high-tech methods, appear to rely primarily on a low-tech, labor-intensive apparatus to monitor, intimidate, and censor their targets, some of which involves highly personalized forms of repression. We also heard credible accusations of Palestinian authorities’ collaboration with Iranian and Israeli governments, the latter typically viewed as an adversary by Palestinians. We consider the implications of these findings and offer recommendations both for activists and social media platforms.

Non-Consensual Image-Disclosure Abuse (NCIDA) represents a subset of technology-facilitated sexual abuse where imagery and video with romantic or sexual connotations are used to control, extort, and otherwise harm victims. Despite considerable research on NCIDA, little is known about them in non-Western contexts. We investigate NCIDA in Pakistan, through interviews with victims, their relatives, and investigative officers; and observations of NCIDA cases being processed at a law enforcement agency. We find, first, that what constitutes NCIDA is much broader in Pakistan’s patriarchal society, and that its effects can be more severe than in Western contexts. On every dimension – types of content, perpetrators, impact on victims, and desired response by victims – our findings suggest an expansion of the concepts associated with NCIDA. We conclude by making technical and policy-level recommendations, both to address the specific context of Pakistan, and to enable a more global conception of NCIDA.

Like other communication forums, social media can be the site of cultural dominance, where some cultures have greater visibility and influence than other cultures.A particular form of such dominance is cultural hegemony, when a dominant group reinforces its economic and political dominance by propagating values and norms that uphold it.In this study, we explored how young adults from two sub-Saharan countries -Namibia and Nigeria -perceived cultural dominance and hegemony on social media.We find that participants generally see the West as dominant on social media, both in terms of productive usage and cultural or economic cachet.Our participants also exhibited differences in how they felt about this dominance, with Namibian participants tending to voice resentment about the lack of representation of their own culture, while Nigerian participants often expressed admiration for successful Global North influencers online.Yet, even as participants hoped for representation and economic opportunity through social media, they also seemed unaware of the platforms' extractive potential, suggesting that social media effectively strengthens the global capitalist order's cultural hegemony.

Friendships are a key element of mental health, yet modern life increasingly involves "enforced online regimes," which can inhibit friendship formation. One example is provided by residential university students under COVID-19. Through interviews with 17 graduate students at a U.S. university, we investigate how new friendships were made and maintained under the pandemic. While some of our individual findings echo previous work with online social interaction, our analysis reveals a novel 7-phase friendship formation process that extends Levinger & Snoek's classic pair-relatedness theory. The model enables pinpoint diagnoses. For our participants, three specific phases were blocked -- Physical Awareness (apprehension of another's physical characteristics); Personal Contact (exchange of personal information); and Ongoing Mutuality (repeat interactions to build friendship). The model also explains divergent results under similar but different situations (e.g., residential students under COVID eventually made friends, but students of purely online courses do not), and enables targeted recommendations.

Online harassment is a global problem. This article examines perceptions of harm and preferences for remedies associated with online harassment with nearly 4000 participants in 14 countries around the world. The countries in this work reflect a range of identities and values, with a focus on those outside of North American and European contexts. Results show that perceptions of harm are higher among participants from all countries studied compared to the United States. Non-consensual sharing of sexual photos is consistently rated as harmful in all countries, while insults and rumors are perceived as more harmful in non-U.S. countries, especially harm to family reputation. Lower trust in other people and lower trust in sense of safety in one’s neighborhood correlate with increased perceptions of harm of online harassment. In terms of remedies, participants in most countries prefer monetary compensation, apologies, and publicly revealing offender’s identities compared to the U.S. Social media platform design and policy must consider regional values and norms, which may depart from U.S. centric-approaches.

Co-design collects insights from multiple stakeholders collaboratively making it a powerful method to design with marginalized populations. In the latter context, stakeholders have varying levels of power causing asymmetry and possible suppression of one group over another. Such power differentials can hinder co-design’s effectiveness. Through thirteen semi-structured interviews with co-design facilitators who have worked with marginalized communities in 43 different countries, we discovered that despite efforts to mitigate power differentials, significant disparities in educational and cultural backgrounds, language barriers, and gender imbalances prevent true collaboration. Tools for prototyping, analysis and evaluation often require literacy, advanced training, and resources. When these are inaccessible, co-design fails to materialize in the design analysis, implementation, and evaluation phases. We found this failure occurred with marginalized groups. We also found that experienced facilitators were aware of their own privilege as well as the power differentials of outside stakeholders such as donors, and they prioritized strategies to address them ahead of time.

As with irregular migrants elsewhere, Hondurans seeking to cross the border into the United States use digital technologies, particularly mobile phones. Based on participant observation and semi-structured interviews in Honduras with 26 irregular migrants, we explore how they relate to digital devices at the onset of their journey and throughout. Contrary to studies on migrants’ experiences at or near their desired destination, we find that participants strongly prefer to use mobile phones to the extent allowed by their modes of migration – specifically, travel with human smugglers called coyotes, with caravans, or on their own. Irregular migrants encounter dangerous threats along their path, which they fear could be heightened by mobile phones. Nevertheless, despite their concerns, we find that their adaptive strategies are irrelevant or ineffective in preventing deportation and extortion. These findings lead to recommendations that consider a broader digital ecology beyond personal devices to support migrants’ needs and dispel misinformation about the dangers of device access.

Social change projects, such as those undertaken under the banner of information & communication technologies for development (ICT4D), frequently address problems using a design-based approach and therefore inherit the values of what could be called “classical design.” One of those values, as we demonstrate through an examination of popular design textbooks, is to accommodate people’s needs, abilities, and behaviors. Accommodating people as they are, however, is very different from encouraging the internal changes in people – as either individuals or societies – that are required for meaningful social change. The inclinations of classical design are thus poorly matched to efforts intended to lead to social change. We flesh out this argument through an examination of classical design’s underlying ethos, an articulation of the difference between social good and social change, case studies of three ICT4D design efforts, and analysis of the implications of “giving people what they want.” We conclude with a call to draw a clear line between design for convenience or efficiency and innovation for social change in ICT4D pedagogy, research, and practice.

In order to sustain everyday life in India during pandemic induced lockdowns, home service gig platforms materialized to provide essential services for urban society. As unemployment worsened, these gig platforms also emerged as key sources of paid work for gig workers, with some platforms promising an unusual degree of health and financial support for their gig workforce. Through semi-structured interviews, we examine how women beauty workers engaged with the infrastructural promise extended by home service gig platforms during the pandemic. While gig platforms promoted the potential of stable income and social security in the context of the Global South, we investigate the reality behind this image. We find that various breakdowns, from miscommunication around localized travel restrictions to limited platform helpline access, introduces day-to-day unpredictability for gig workers, hindering access to paid work as well as other platform extended benefits. We suggest that home service gig platforms actually serve as 'faux infrastructure,' in which the privatized logics work to enclose public value, while pushing the burden of access onto gig workers who must perform additional, often unpaid labors, in order to fill last-mile service gaps.

Front-line workers in global development are often responsible for data collection and record-keeping about their own work. The authenticity of such data and the role of mid-level supervisors, however, remains understudied. We report on the case of immunization in Pakistan, where, through interviews with 30 mid-level vaccination managers in Punjab district, we find that data falsification by vaccinators is common, though not necessarily rampant. Because of an intricate protocol for record-keeping, supervisors can detect data falsification, and we find they have devised an array of methods, broadly classifiable into four types: triangulation, supplementary data collection, anomaly detection, and interrogation. We also find that the strategies that supervisors use to detect falsification seem linked to their style of management, with authoritarian supervisors preferring supplementary data collection and spot checks, while supportive supervisors use triangulation. Our findings lead to recommendations for designing technologies intended to monitor and manage front-line data.

With the rise of social media, entrepreneurs are feeling the pressure to adopt digital tools for their work. However, the upfront effort and resources needed to participate on these platforms is ever more complex, particularly in underresourced contexts. Through participatory action research over two years in Detroit's Eastside, we found that local entrepreneurs preferred to become engaged digitally through a community collective, which involved (a) resource-connecting organizations, (b) regular in-person meetings, (c) paper planning tools, and (d) practice and validation. Together, these elements combined to provide (1) awareness and willingness to use digital tools, (2) regular opportunities to build internet self-efficacy, and (3) ways to collectively overcome digital obstacles. We discuss our findings in the context of digital engagement and entrepreneurship, and outline recommendations for digital platforms seeking to better support economic mobility more broadly.

Meditative and contemplative practices are common among U.S. adults, but the impact of digital technology use on these practices and on associated transcendent experiences is poorly understood. Through semi-structured interviews with sixteen experienced practitioners from a variety of traditions, we find that practitioners consider digital technology to be a mixed blessing. While they see its practical value, they are wary of its stimulation-based effects and find minimal usefulness in commercial meditation apps. They also feel that digital technology use may interfere with possible transcendent experiences. The practitioners, however, applied insights from their respective practices to strategically mitigate digital technology's negative effects in three ways: limiting its use to instrumental purposes, using technology interactions as grist for self-reflection, and integrating technology itself into a site for practice. Specific design recommendations are discussed.

We examine the opportunities and challenges in designing for maternal health in low-income, low-resource communities in patriarchal and religious contexts. Pakistan faces a crisis in maternal health with a maternal mortality ratio of 178 deaths per 100,000 live births, as compared to the developed-country average of just 12 deaths per 100,000. Through a 6-month long qualitative, empirical study we examine the prevalent beliefs and practices around maternal health in Pakistan, the access women have to health-care, the existing religious practices that influence them and the agency they exert in their own health-care decision making. We reveal the rampant misinformation among mothers and health workers, house-hold power dynamics that impact maternal health and the deep link between maternal health and religious beliefs. We also show how current maternal health care interventions fit poorly into this context and discuss alternate design recommendations for meeting the maternal health needs of these women.

The human-computer interaction community has made some efforts toward racial diversity, but the outcomes remain meager. We introduce critical race theory and adapt it for HCI to lay a theoretical basis for race-conscious efforts, both in research and within our community. Building on the theory's original tenets, we argue that racism is pervasive in everyday socio-technical systems; that the HCI community is prone to "interest convergence", where concessions to inclusion require benefits to those in power; and that the neoliberal underpinnings of the technology industry itself propagate racism. Critical race theory uses storytelling as a means to upend deep-seated assumptions, and we relate several personal stories to highlight ongoing problems of race in HCI. The implications: all HCI research must be attuned to issues of race; participation of underrepresented minorities must be sought in all of our activities; and as a community, we cannot become comfortable while racial disparities exist.

In the United States, African-American women are three times more likely to die of pregnancy-related issues, and 57% more likely to have miscarriages than White women. In part, this is due to deficiencies in the information ecology serving pregnant mothers. We report on a qualitative study with 16 low-income, African-American, pregnant women from the Detroit area, focused on understanding their informational needs and information-seeking behavior. We find that our participants consumed information from multiple sources voraciously; treated formal medical sources as no more trustworthy than other information sources; sought corroborative evidence when making decisions; relied on video and social media sources; and voiced requests for material and social support more than for medical information. Among our recommendations are increased use of video as a means for pregnancy-related information dissemination -- particularly testimonials by experienced mothers with similar backgrounds -- and better integration of medically sound voices in spaces where mothers already congregate online.

Despite some successes, Rwanda's ambitious plan to become an ICT-led economy faces challenges. Through 31 semi-structured interviews with stakeholders, we explored the reported strengths, weaknesses, and potential lessons of Rwanda's ICT sector, especially as relevant to the country's economic development. We find that (1) there is sincere, widely shared aspiration toward ICT-led national development; (2) support for ICT entrepreneurship is successful at helping to start ICT-based businesses, but not at continuing or succeeding with them; and (3) some of the challenges of continuing ICT-based businesses have to do with Rwandan higher education, availability of start-up financing, and a weak sense for entrepreneurial initiative. We situate these findings in the literature on global technology hubs and national development, and discuss policy recommendations for countries seeking ICT-based economic growth.

An understanding of medical diagnosis as it is practiced is essential for those seeking to support it using intelligent systems. Through the case of epilepsy, we show that diagnosis is a situated, relational, and evolving process that accounts for information well beyond the patient's physiology, even for physiological phenomena like seizures. Through observations and interviews with neurologists, we show that the meaning of brainwaves and other physiological data depends upon a range of patient-specific and contextual factors, such as age, comorbidities, and mealtimes. Further, we show that diagnosis is partly determined by social factors such as the activities of caregivers and other clinicians, and environmental factors such as faulty electrical wiring. Additionally, diagnostic classifications can evolve in response to new information: events that were once considered seizures can be reinterpreted as clinically irrelevant and vice versa. We contribute a broader sociotechnical perspective to literature on intelligent decision making in healthcare and discuss implications for the design of decision support systems that can better support the work of medical diagnosis.

Fourth-age residents in long-term care facilities (LTCF) are known to suffer declines in well-being due to their advanced age and resulting loss of independence. Using an action research approach, we set up a makerspace in a New Jersey LTCF for eight weeks to see whether it could improve well-being for residents. Based on engaged observation over 280 hours and semi-structured interviews with participants, we find that people aged 80-99 years will spend (sometimes significant) time in a makerspace for the purposes of making and companionship; that makerspaces can contribute to both autonomy and well-being for older residents; and participants produced not only decorative art, but novel artifacts that solved real challenges in their daily lives. We situate these findings in the literature on art and activity therapy for fourth-age people, and make recommendations for makerspaces in long-term care facilities.

Returning citizens (formerly incarcerated individuals) face great challenges finding employment, and these are exacerbated by the need for digital literacy in modern job search. Through 23 semi-structured interviews and a pilot digital literacy course with returning citizens in the Greater Detroit area, we explore tactics and needs with respect to job search and digital technology. Returning citizens exhibit great diversity, but overall, we find our participants to have striking gaps in digital literacy upon release, even as they are quickly introduced to smartphones by friends and family. They tend to have employable skills and ability to use offline social networks to find opportunities, but have little understanding of formal job search processes, online or offline. They mostly mirror mainstream use of mobile technology, but they have various reasons to avoid social media. These and other findings lead to recommendations for digital literacy programs for returning citizens.

Saxenian's concept of "brain circulation" suggests that the information technology sectors of developing countries benefit from having their brightest talent move and work abroad, as some then return home with new skills and networks. This accelerates economic growth within their country of origin.

We present the design and preliminary evaluation of a mobile text-based system and that provides homeless people in the United States with information about the availability of free meals. The system, piloted in Ann Arbor, Michigan, provides information about when and where free meals are offered based on zipcode as entered by users. It has features to address issues around information access, incompleteness, and irrelevance. We found that SMS text messages are preferred by our homeless users even when they are non-literate and own smartphones, and that formal systems are useful even in contexts where word-of-mouth predominates.

This paper explores computing education as a potential site for intercultural learning and encounter in post-conflict environments. It reports on ethnographic fieldwork from the Nairobi Play Project, a constructionist educational program serving adolescents aged 14-18 in urban and rural multi-ethnic refugee communities in Kenya. While the program offers programming and game design instruction, an equal goal is to foster interaction, collaboration, dialogue and understanding across cultural backgrounds. Based on fieldwork from two project cycles involving 5 after-school classes of 12-24 students each, we describe key affordances for encounter, important resistances to be managed or overcome, and emergent complications in the execution of such programs. We argue that many important accomplishments of intercultural learning occur through moments of friction, breakdowns, and gaps -- for example, technical challenges that produce sites of shared humour; frictions between intercultural activities and computing activities; acts of disrupting order; and unstructured time that students collaboratively fill in. We also describe significant complications in such programs, including pressures to adopt norms and practices consistent with dominant or majority cultures, and instances of intercultural bonding over artefacts with xenophobic themes. We reflect on the implications of these phenomena for the design of future programs that use computing as a backbone for intercultural learning or diversity and inclusion efforts in CSCW, ICTD, and allied fields of work.

Online technologies are increasingly hailed as enablers of entrepreneurship and income generation. Recent evidence suggests, however, that even the best such tools disproportionately favor those with pre-existing entrepreneurial advantages. Despite intentions, the technology on its own seems far from addressing socio-economic inequalities. Using participatory action research, we investigated why this might be, in an intimate, close-up context. Over a 1-year period, we--a collaborative team of university researchers and residents of Detroit's East Side--worked to establish a neighborhood tour whose initial goal was to raise supplementary income and fundraise for community block clubs. We found that in addition to technical requirements, such as communication tools, a range of non-technological efforts is needed to manage projects, build self-efficacy, and otherwise support community participants. Our findings widen Ackerman's "socio-technical gap" for some contexts and offer a counterpoint to overgeneralized claims about well-designed technologies being able to address certain classes of social challenges.

Undocumented immigrants in the United States face risks of discrimination, surveillance, and deportation. We investigate their technology use, risk perceptions, and protective strategies relating to their vulnerability. Through semi-structured interviews with Latinx undocumented immigrants, we find that while participants act to address offline threats, this vigilance does not translate to their online activities. Their technology use is shaped by needs and benefits rather than risk perceptions. While our participants are concerned about identity theft and privacy generally, and some raise concerns about online harassment, their understanding of government surveillance risks is vague and met with resignation. We identify tensions among self-expression, group privacy, and self-censorship related to their immigration status, as well as strong trust in service providers. Our findings have implications for digital literacy education, privacy and security interfaces, and technology design in general. Even minor design decisions can substantially affect exposure risks and well-being for such vulnerable communities.

Qualitative researchers perform an important and painstaking data annotation process known as coding. However, much of the process can be tedious and repetitive, becoming prohibitive for large datasets. Could coding be partially automated, and should it be? To answer this question, we interviewed researchers and observed them code interview transcripts. We found that across disciplines, researchers follow several coding practices well-suited to automation. Further, researchers desire automation after having developed a codebook and coded a subset of data, particularly in extending their coding to unseen data. Researchers also require any assistive tool to be transparent about its recommendations. Based on our findings, we built prototypes to partially automate coding using simple natural language processing techniques. Our top-performing system generates coding that matches human coders on inter-rater reliability measures. We discuss implications for interface and algorithm design, meta-issues around automating qualitative research, and suggestions for future work.

In India, women represent 45% of total computer science enrollment in universities, almost three times the rate in the United States, where it is 17%. At the same time, women make up an estimated 25-30% of the HCI community in India, half the rate in the U.S. We investigate the complexities of these surprising phenomena through qualitative research of Indian computer science and human-computer interaction researchers and professionals at various life stages. We find among other things that Indian familial norms play a significant role in pressuring young women into computing as a field; that familial pressures and workplace discrimination then cause a precipitous exit of women from computing at the onset of marriage; and that HCI occupies an interstitial space between art and technology that affects women's careers. Our findings underscore the societal influence on women's representation in the tech sector and invite further participation by the HCI community in related questions.

Entrepreneurship has long been used to create self-employment opportunities to guard against career uncertainty. Yet, little is known about how social technologies impact the day-to-day work of entrepreneurs in resource-constrained contexts. We performed a qualitative study involving interviews with 26 micro-entrepreneurs in Detroit and observations of entrepreneurship events. We found that micro-entrepreneurs in Detroit are often pushed into entrepreneurship in response to unexpected life disruptions, barriers to employment, and desire to benefit the community. Their resource-constrained contexts shape how they use social technologies, such as sharing economy tools and social media groups, particularly with respect to privacy, safety, and professional agency. We expand the discussion in CSCW around what it means to be an entrepreneur and provide implications for how social technologies can be designed to better meet the employment needs of people in resource-constrained communities.

Religious institutions hold a significant place in daily life for the vast majority of people in the world, especially in developing countries. Yet despite their social prominence, and despite HCI's emphasis on the social context of technology, organized religion is neglected in both the HCI and ICTD literature. This paper explores the relationship that mosques in Bangladesh have with their constituencies and with technology, with an eye toward the integration of technology with existing religious institutions as a way to achieve positive social ends. We first describe a qualitative exploration of several mosque communities in Bangladesh, where we find that skepticism and pragmatism about modern technology interact in a complex way that nevertheless leaves room for technical interventions. We then describe a randomized controlled trial to study the relative value of SMS messages infused with overtly religious or secularly altruistic frames for the purpose of mosque fundraising. We find that SMS messages increase donations overall, but that their framing is significant. Messages with secular altruistic framing increased donations by 9.5%, while those with religious sentiment increased donations by 57.3%. Our findings demonstrate how technologies like SMS amplify underlying religious forces and suggest the possibility of working with religious institutions in applying positive ICT interventions.

We present a three-phase exploration of mobile messaging to address the high rate of maternal mortality in low-income, urban Pakistan, with a focus on identifying deviations from previously published findings about mobile interventions in developing-world health.

Based on a survey of 525 papers published in three ICTD-related conferences during the ten-year period between 2006 and 2015, we find that there has been a general trend toward increasing geographical diversification of ICTD research sites. This manifests as an overall decline in both absolute number and proportion of papers reporting on studies in India -- which retained the top spot throughout the interval of this analysis -- as well as gains in other countries such as Kenya and Ghana. In addition, we find that the total number of countries represented in the literature has also increased, leading to what appears to be a geographically broadening range of ICTD research activity. We offer several hypotheses to explain this trend's underlying causes.

During the first decade of the 21st century, the rise of mobile feature phones in India saw the development of both an economy of informal media exchange and a culture of active media sharing for entertainment. Mobile phone owners paid for pirated movies and music on the grey market, and they traded them with one another, even using poorly designed mechanisms such as Bluetooth file exchange.

One of the greatest challenges in designing applications for economically poor communities is that potential users may have little or no education. We investigated how limited education appears to impact the ability to navigate a hierarchical UI, even when it has no text. We scored 60 participants from low-income communities in India using tests of textual literacy and Raven's Progressive Matrices. These were used as proxies for educational level and a subset of cognitive abilities. We then evaluated participants' performance on a UI task involving hierarchical navigation. First, our results confirm that textual literacy is correlated with scores on the Raven's test. In addition, we found that performance on both instruments are predictive of performance in navigating UI hierarchies, even when the UI is text-free. This provides statistically significant confirmation of previous anecdotal hypotheses. We conclude with design recommendations for UI hierarchies for people with limited education.

Grameen Telecom's famous Village Phone program in Bangladesh has been hailed as an innovative model of ICT-based international development, and promoted by international aid agencies in a wide range of developing countries. Despite implementation at scale, and success both as a business model and in its impact on the poor, the Village Phone programs appear time-limited. Decline of Village Phone model is driven in part by mobile penetration obviating the need for a specialized service selling phone usage, but there are other intertwined factors.

While mobile phones have found broad application in bringing health, financial, and other services to the developing world, usability remains a major hurdle for novice and low-literacy populations. In this article, we take two steps to evaluate and improve the usability of mobile interfaces for such users. First, we offer an ethnographic study of the usability barriers facing 90 low-literacy subjects in India, Kenya, the Philippines, and South Africa. Then, via two studies involving over 70 subjects in India, we quantitatively compare the usability of different points in the mobile design space. In addition to text interfaces such as electronic forms, SMS, and USSD, we consider three text-free interfaces: a spoken dialog system, a graphical interface, and a live operator. Our results confirm that textual interfaces are unusable by first-time low-literacy users, and error prone for literate but novice users. In the context of healthcare, we find that a live operator is up to ten times more accurate than text-based interfaces, and can also be cost effective in countries such as India. In the context of mobile banking, we find that task completion is highest with a graphical interface, but those who understand the spoken dialog system can use it more quickly due to their comfort and familiarity with speech. We synthesize our findings into a set of design recommendations.

We describe ViralVCD: a low cost method for tracing paths of information diffusion in developing communities using physical media. We instituted a participatory video framework for creation and dissemination of developmental videos in seven urban slums and peri-urban communities of Bangalore, India. By combining a call-in contest with Video CDs, we were able to measure developmental impact as well as elicit data on social networks and technology usage practices. In particular, our technique was able to extract data from multiple layers-social, technological, and developmental. ViralVCD allowed us to identify key actors and map information diffusion, as well as technology ownership and access. These findings have implications for HCI initiatives targeting low income locales and populations.

We describe a prevalent mode of information access in low-income communities of the developing world--intermediated interactions. They enable persons for whom technology is inaccessible due to non-literacy, lack of technology-operation skills, or financial constraints, to benefit from technologies through digitally skilled users--thus, expanding the reach of technologies. Reporting the results of our ethnography in two urban slums of Bangalore, India, we present three distinct intermediated interactions: inputting intent into the device in proximate enabling, interpretation of device output in proximate translation, and both input of intent and interpretation of output in surrogate usage. We present some requirements and challenges in interface design of these interactions and explain how they are different from direct interactions. We then explain the broader effects of these interactions on low-income communities, and present some implications for design.

We present the results of a qualitative study of the sharing and consumption of entertainment media on low-cost mobile phones in urban India, a practice which has evolved into a vibrant, informal socio-technical ecosystem. This wide-ranging phenomenon includes end users, mobile phone shops, and content distributors, and exhibits remarkable ingenuity. Even more impressive is the number of obstacles which have been surmounted in its establishment, from the technical (interface complexity, limited Internet access, viruses), to the broader socioeconomic (cost, language, legality, institutional rules, lack of privacy), all seemingly due to a strong desire to be entertained.

A recent trend in interface design for classrooms in developing regions has many students interacting on the same display using mice. Text entry has emerged as an important problem preventing such mouse-based singledisplay groupware systems from offering compelling interactive activities. We explore the design space of mouse-based text entry and develop 13 techniques with novel characteristics suited to the multiple mouse scenario. We evaluated these in a 3-phase study over 14 days with 40 students in 2 developing region schools. The results show that one technique effectively balanced all of our design dimensions, another was most preferred by students, and both could benefit from augmentation to support collaborative interaction. Our results also provide insights into the factors that create an optimal text entry technique for single-display groupware systems.

One of the greatest challenges in designing applications for developing communities is that potential users may have limited literacy. Past work in UI design for low-literate users has focused on illiteracy as the inability to read per se, with little recognition to other cognitive differences between literate and non-literate users.

We present Collage, a software application designed for classroom presentations for school teachers in the developing world. An in-depth investigation of teaching practices in several schools in India led us to believe that a simple tool that enabled the display of images and textbook materials while facilitating blackboard-like interactions would be very helpful for these teachers. Collage is a simple media viewer with a small number of features that enables teachers to prepare lessons with little overhead and then present them in classrooms with maximum flexibility. The tool was piloted in three schools in suburban India for use in real-world classroom teaching. All teachers who used Collage uniformly praised it, and students' learning of visual concepts seemed to improve through it. Interestingly, some teachers continue to use the tool now for their own teaching needs and have spontaneously shared it with colleagues from other schools.

India's extensive Self-Help Group (SHG) microfinance network brings formal savings and credit services to 86 million poor households. Yet, the inability to maintain high-quality records remains a persistent weakness in SHG functioning. We study this problem and present a financial record management application built on a low-cost digital slate prototype. The solution directly accepts handwritten input on ordinary paper forms and provides immediate electronic feedback. A field trial with 200 SHG members in rural India shows that the use of the digital slate solution results in shorter data recording time, fewer incorrect entries, and more complete records. The paper-pen-slate solution performs as well as, and is strongly preferred over, a purely electronic alternative. The digital slate solution is able to comfortably move between paper and digital worlds, achieving efficiency and quality gains while catering to the preferences and budgets of low-income low-literate clients.

Multimodal interfaces with little or no text have been shown to be useful for users with low literacy. However, this research has not differentiated between the needs of the fully illiterate and semiliterate - those who have basic literacy but cannot read and write fluently. Text offers a fast and unambiguous mode of interaction for literate users and the exposure to text may allow for incidental improvement of reading skills. We conducted two studies that explore how semiliterate users with very little education might benefit from a combination of text and audio as compared to illiterate and literate users. Results show that semiliterate users reduced their use of audio support even during the first hour of use and over several hours this reduction was accompanied by a gain in visual word recognition; illiterate users showed no similar improvement. Semiliterate users should thus be treated differently from illiterate users in interface design.

Due to the increasing penetration of mobile phones even into poor communities, mobile payment schemes could bring formal financial services to the "unbanked". However, because poverty for the most part also correlates with low levels of formal education, there are questions as to whether electronic access to complex financial services is enough to bridge the gap, and if so, what sort of UI is best.

Featherweight multimedia devices combine audio with non-electronic visual displays (e.g., paper). Because of their low cost, customizability, durability, storage capacity, and energy efficiency, they are well-suited for education and information dissemination among illiterate and semi-literate people. We present a taxonomy of featherweight multimedia devices and also derive design recommendations from our experiences deploying featherweight multimedia in the agriculture and health domains in India. We found that with some initial guidance, illiterate users can quickly learn to use and enjoy the device, especially if they are taught by peers.

It is estimated that there are 100 million “street children” in the world [1]. Street children typically live independent of families in makeshift living arrangements, and survive on little but their wits and the camaraderie of small gangs.

Almost all formal organizations employ service staff for tasks such as housekeeping, security, maintenance, and transport at their office facility. Many of these workers earn wages in line with menial-labor salaries in their respective countries. They have few on-the-job opportunities to upgrade their skills or learn new ones. Kelsa+ is an initiative through which organizations in developing countries can increase digital literacy and skill development among such low-income workers, through the provision of an Internet-connected PC for the service staffs free, unrestricted use when off duty. We study a Kelsa+ pilot implementation in Bangalore, India, involving an office facility with 35 service staff. In a preliminary exploration over 18 months, we find that at a cost that is negligible for the organization, workers use of the Kelsa+ PC is high and can deliver benefits both to themselves and to the office. For workers, broad gains were seen in confidence, self-esteem, and basic digital literacy, while a few individuals experienced improvements in second-language (English) proficiency and career opportunities. These early results point in the direction of a cost-effective ICT4D initiative that could be run in the developing-country offices of the very organizations promoting development off-site.

Non-governmental organizations (NGOs) working in disadvantaged communities have a variety of data-collection and analysis needs, for example, for performing surveys or monitoring programs. Because much of this data collection occurs in environments with insufficient IT support and infrastructure, and among populations not always comfortable with technology, paper forms rather than electronic methods remain the predominant means for data collection.

We present work that explores whether the asynchronous peer-to-peer communication capabilities of email can be made accessible to illiterate populations in the developing world. Building on metaphors from traditional communication systems such as postal mail, and relevant design principles established by previous research into text-free interfaces, we designed and evaluated a prototype asynchronous communication application built on standard email protocols. We considered different message formats -- text, freeform ink, audio, and video + audio -- and via iterative usage and design sessions, determined that video + audio was the most viable. Design alternatives for authentication processes were also explored. Our prototype was refined over three usability iterations, and the final version evaluated in a two-stage study with 20 illiterate users from an urban slum in Bangalore, India. Our results are mixed: On the one hand, the results show that users can understand the concept of video mail. They were able to successfully complete tasks ranging from account setup to login to viewing and creating mail, but required assistance from an online audio assistant. On the other hand, there were some surprising challenges such as a consistent difficulty understanding the notion of asynchronicity. The latter suggests that more work on the paradigm is required before the benefits of email can be brought to illiterate users.

The full range of possibilities for spoken-language technologies (SLTs) to impact poor communities has been investigated on partially, despite what appears to be strong potential. Voice interfaces raise fewer barriers for the illiterate, require less training to use, and are a natural choice for applications on cell phones, which have far greater penetration, in the developing world than PCs. At the same time, critical lessons of existing technology projects in development still apply and require careful attention. We suggest how to expand the view of SLT for development, and discuss how its potential can realistically be explored.

This study evaluates single-mouse and multiple-mice configurations for computer-aided learning in schools where access to computers is limited due to resource constraints. Multimouse, a single display groupware solution, developed to allow multiple mice to be used simultaneously on a single PC, is compared with single-user-single-mouse and multiple-user-single-mouse scenarios. Multimouse itself is trialed with two unique interaction designs -- one where competitive interaction among students is encouraged, and another where more collaborative interaction is expected. Experiments were conducted with 238 schoolchildren from underprivileged households in rural India on an English vocabulary retention task. On the whole, Multimouse configurations (five users each) were found to be at par with single-user scenarios in terms of actual words learned by students. This suggests that the value of a PC can be inexpensively multiplied by employing a multi-input shared-use design. Gender effects were found, where boys show significant differences in learning depending on interaction modality, whereas girls learned at similar rates across configurations. In addition, a comparison of the two Multimouse modes -- collaborative and competitive -- showed the striking difference in learning outcomes and user behavior that is possible due to even slight variations in interaction designs for multiple-mice.

Robust, global, address geocoding is challenging because there is no single address format that applies to all geographies, and in any case, users may not restrict themselves to well-formed addresses. Particularly in online mapping systems, users frequently enter queries with missing or conflicting information, misspellings, address transpositions, and other such variations.

Digital green is a research project that seeks to disseminate targeted agricultural information to small and marginal farmers in India using digital video. The unique components of digital green are (1) a participatory process for content production, (2) a locally generated digital video database, (3) human-mediated instruction for dissemination and training, and (4) regimented sequencing to initiate a new community. Unlike some systems that expect information or communication technology alone to deliver useful knowledge to marginal farmers, digital green works with existing, people-based extension systems and aims to amplify their effectiveness. While video provides a point of focus, it is people and social dynamics that ultimately make digital green work. Local social networks are tapped to connect farmers with experts; the thrill of appearing "on TV" motivates farmers; and homophily is exploited to minimize the distance between teacher and learner. In a four-month trial involving 16 villages (1070 households), digital green was seen to increase adoption of certain agriculture practices by a factor of six to seven times over classical person-only agriculture extension. The hardware investment was a TV and a DVD-player per village, and one digital camera and PC shared among all 16 villages. These results are very preliminary, but promising.

This paper presents the use of full-context video to motivate and aid non-literate, first-time users of PCs to successfully navigate a computer application with minimal assistance. Following previous work focused on non-literate users, we observed that in spite of our subjects' understanding of the UI mechanics, they experienced barriers beyond illiteracy in interacting with the computer: lack of awareness of what the PC could deliver, fear and mistrust of the technology, and lack of comprehension about how information relevant to them was embedded in the PC. In this paper, we address these challenges with full-context video, which includes dramatizations of how a user might use the application and how relevant information comes to be contained in the computer, in addition to a tutorial of the UI. In experiments conducted with 35 non-literate residents of Bangalore slums, the introduction of full-context video dramatically improved task completion for a job-search task.

Computer centers in rural Indian public schools raise questions about the value of expensive modern technology in starkly poor environments. Arguments for or against spending on computers in low-income schools have appeared in policy circles, academia, teacher conferences, and philanthropic discussions, with passionate rhetoric from all sides. One shortcoming of the debate has been the absent voice of parents and children themselves. We present the results of a qualitative study of computer-aided learning centers in four districts of rural Karnataka, South India, where we discussed with parents issues such as aspirations, quality of schooling, and the perception of computers more generally. The research reveals a range of voices on hopes for the next generation, perceived value of computer courses and higher education, and views on the arrival of computers into their village schools. It emerges that in the minds of many parents, the computer has an immense symbolic value - separate from its functional value - that is tied to social and economic ascendancy. We find that this symbolic value derives from associations that parents imbibe from their various interactions with people using computers in a range of situations. Despite a large number of parents not entirely clear on what a computer does, the sense of mystical quality about technology is a feature we find across the board. In discussions with rural parents, we find an environment of great fear about the future of agriculture, because of which computer-aided learning centers have become a symbol of future aspirations of jobs for their children. The research also reveals varied related concerns of parents from the dowry implications of having computer-trained daughters to the relative value of English versus computer literacy as the key to social mobility.

In this paper, we present what we believe to be the first documented experiment to replace an existing PC-based system that had goals of "bridging the digital divide" for an agricultural district with a mobile-phone-based system in which a small, but relevant amount of data is transferred to farmers via SMS text messaging. Implemented in rural Maharashtra, Warana Unwired sought to replace an existing PC-based system for managing information in a sugarcane cooperative with an SMS-based mobile-phone system. In an eight-month trial involving seven villages, Warana Unwired successfully replicated all of the PC-based functionality, and was found to be less expensive, more convenient, and more popular with farmers than the previous PC-based system. This paper discusses the early investigations of the Warana Wired Village Project that led to the conception and implementation of the Warana Unwired project. The new system is described in detail, and results, both quantitative and qualitative are analyzed.

In this paper we describe the experimental set-up and execution of a public art project. The aim was to explore the use of SMSBlogging for the purpose of community building and creative self-expression. We also discuss the results from this experiment and show our findings from six blog-on-the-street art acts in Bangalore, India, that introduce SMSBlogging technology in an iterative design process.

We present research leading toward an understanding of the optimal audio-visual representation for illustrating concepts for illiterate and semi-literate users of computers. In our user study, which to our knowledge is the first of its kind, we presented to 200 illiterate subjects each of 13 different health symptoms in one representation randomly selected among the following ten: text, static drawings, staticphotographs, hand-drawn animations, and video, each with and without voice annotation. The goal was to see how comprehensible these representation types were for an illiterate audience. We used a methodology for generating each of the representations tested in a way that fairly stacks one representational type against the others.

Abstract Geographic maps have existed from early stages of human civilization. Various styles of visualizing the geographic information have evolved depending on the nature of information and the technology available for visualization. This has led to innumerable map styles. In this work we develop a technique to create maps by combining two‐dimensional and three‐dimensional information such that the resulting maps are both functional and aesthetically appealing. Our technique requires geographical information in vector form and aerial images as inputs. We use computer vision based approaches and user defined inputs to augment the vector data with information that is required to create stylized maps. We define procedural graphics methods to generate a range of geographic elements that can be composed together into a stylized map. We demonstrate our technique by generating example maps of a region in Las Vegas. Copyright © 2007 John Wiley & Sons, Ltd.

CAM is a user interface toolkit that allows a camera-equipped mobile phone to interact with paper documents. It is designed to automate inefficient, paper-intensive information processes in the developing world. In this paper we present a usability evaluation of an application built using CAM for collecting data from microfinance groups in rural India. This application serves an important and immediate need in the microfinance industry. Our quantitative results show that the user interface is efficient, accurate and can quickly be learned by rural users. The results were competitive with an equivalent PC-based UI. Qualitatively, the interface was found easy to use by almost all users. This shows that, with a properly designed user interface, mobile phones can be a preferred platform for many rural computing applications. Voice feedback and numeric data entry were particularly well-received by users. We are conducting a pilot of this application with 400 microfinance groups in India.

This paper examines the social and political challenges related to the implementation of information and communication technology (ICT) kiosk projects for rural development in India. Specifically, the paper focuses on the Akshaya project, a franchise of rural computer-service kiosks, which was implemented in Kerala as a public-private sector collaboration. The Akshaya project has the twin goals of social development through increased access to computers for rural people and financial viability through market-driven entrepreneurship. Using interview and participant observation methods, we examine the challenges that state actors and entrepreneurs face in simultaneously addressing social and financial sustainability. The preliminary evidence suggests that there is a tension between these goals at a macro level (within the state) and a micro level (for entrepreneurs and potential consumers) that makes it difficult to run a financially self-sustaining ICT kiosk project that also meets social development goals. The paper demonstrates that the implementation of ICTs for development is not simply a technical process of delivering services to the poor, but is a highly political process that involves tradeoffs and prioritization of particular goals to attain sustainability. Branding this project is a challenge for the state and entrepreneurs due to consumer perceptions of what development is, with particular expectations of state provided services, versus what business is

We describe work toward the goal of a user interface designed such that even novice, illiterate users require absolutely no intervention from anyone at all to use. Our text-free user interface is based on many hours of ethnographic design conducted in collaboration with a community of illiterate domestic labourers in three Bangalore slums. An ethnographic design process was used to understand what kind of application subjects would be interested in, how they respond to computing technology, and how they react to specific UI elements. We built two applications using these principles, one for job search for domestic labourers, and another for a generic map that could be used for navigating a city. The resulting designs are based on key lessons that we gained through the design process. The paper describes the design process, the design principles which evolved out of the process, the final application design, and results from initial user testing. Our results confirm previous work that emphasizes the need for semi-abstracted graphics and voice feedback, but we additionally find that some aspects of design for illiterate users that have been previously overlooked (such as a consistent help feature). Results also show that the text-free designs are strongly preferred over standard text-based interfaces by the communities which we address, and that they are potentially able to bring even complex computer functions within the reach of users who are unable to read

Understanding technology adoption in emerging regions is challenging given the complex interrelations among socioeconomic factors that affect it directly and indirectly. The issue of impact assessment of technology adoption projects, especially the kind implemented in areas where prior technology has been very limited, is highly problematic and open to many methodological difficulties. Ethnographic evaluations have provided insight into the quality of interactions and into conceptions of technology and its adoption, whereas some quantitative analysis has been useful for high-level abstraction. In this paper, we examine the use of Bayesian networks as tools that can be used in revealing the structure of the relationships between demographic, social, and economic factors, and penetration for various technologies. Our hypothesis is that technology adoption cases in emerging regions display unique aggregated characteristics that make Bayesian network-based analysis a useful starting point in defining relationships between variables in project analysis. We compare the usability of Bayesian networks in analyzing two data sets: (1) a detailed survey focusing on 500 respondents across 14 favelas in Rio de Janeiro; and (2) a comprehensive survey of 998 users of the Akshaya tele-kiosk initiative in Kerala, India. Our illustrations show how Bayesian networks can be useful as statistical analysis tools that reveal new hypotheses, suggest unintended correlations in data, and confirm standing hypotheses

A distinct feature observed in computer use in schools or rural kiosks in developing countries is the high student-to-computer ratio. It is not unusual to see more than five children crowding around a single display, as schools are rarely funded to afford one PC per child in a classroom. One child controls the mouse, while others are passive onlookers, without operational control of the computer. Learning benefits appear to accrue primarily to the child with the mouse, with the other children missing out. The obvious technical solution is to provide each child with a mouse and cursor on screen, thus effectively multiplying the amount of interaction per student per PC for the cost of a few extra mice. To our surprise, both the concept and the implementation appear to be unique to date, for the specific application to computers in education in resource-strapped communities, with previous work restricting studies to two mice, or for largely non-educational applications. We have developed software that allows multiple coloured cursors to co-exist on the monitor, along with two sample games with some educational content. Initial trials with both single-mouse and multiple-mice scenarios suggest that children are more engaged when in control of a mouse, and that more mice increases overall engagement. Our results suggest new areas of research in pedagogy for computers in education

Rural PC kiosks have become prominent recently as a way to impact socio-economic development through computing technology. Despite the significant backing these projects receive from governments and other large organizations, there are very few rigorous studies which measure their actual impact and utility. We have developed and deployed a software PC logging tool that allows us to gain exact quantitative insight into the usage statistics of kiosks on which the tool is installed. In field trials in Maharashtra and Uttar Pradesh, India, we collected over 120 days of software tool logging data from 13 separate kiosks, while we also questioned the kiosk operators during the same period. We show some evidence that the software based logging tool complements the existing survey based and other ethnographic approaches for data collection. We also show that the tool does a better job in gathering certain usage statistics as compared to questioning the kiosk operator

Computer-aided learning is fast gaining traction in developing regions as a means to augment classroom instruction. Reasons for using computer-aided learning range from supplementing teacher shortages to starting underprivileged children off in technology, and funding for such initiatives range from state education funds to international agencies and private groups interested in child development. The interaction of children with computers is seen at various levels, from unsupervised self-guided learning at public booths without specific curriculum to highly regulated in-class computer applications with modules designed to go with school curriculum. Such learning is used at various levels from children as young as 5 year-old to high-schoolers. This paper uses field observations of primary school children in India using computer-aided learning modules, and finds patterns by which children who perform better in classroom activities seat themselves in front of computer monitors, and control the mouse, in cases where children are required to share computer resources. We find that in such circumstances, there emerges a pattern of learning, unique to multi-user environments - wherein certain children tend to learn better because of their control of the mouse. This research also shows that while computer aided learning software for children is primarily designed for single-users, the implementation realities of resource-strapped learning environments in developing regions presents a strong case for multi-user design.

Classifying pictures into one of several semantic categories is a classical image understanding problem. In this paper, we present a stratified approach to both binary (outdoor-indoor) and multiple category of scene classification. We first learn mixture models for 20 basic classes of local image content based on color and texture information. Once trained, these models are applied to a test image, and produce 20 probability density response maps (PDRM) indicating the likelihood that each image region was produced by each class. We then extract some very simple features from those PDRMs, and use them to train a bagged LDA classifier for 10 scene categories. For this process, no explicit region segmentation or spatial context model are computed. To test this classification system, we created a labeled database of 1500 photos taken under very different environment and lighting conditions, using different cameras, and from 43 persons over 5 years. The classification rate of outdoor-indoor classification is 93.8%, and the classification rate for 10 scene categories is 90.1%. As a byproduct, local image patches can be contextually labeled into the 20 basic material classes by using loopy belief propagation (Yedidia et al., 2001) as an anisotropic filter on PDRMs, producing an image-level segmentation if desired.

Researchers have noted conflicting trends in collaboration technologies between delivering more information on larger displays and exploiting mobility on smaller devices. Large, shared displays provide greater choice in the presentation of information, but mobile devices offer greater flexibility in the access of information. We describe a platform that leverages the best of both worlds by allowing multiple users to access and interact with a large, shared display using their own personal mobile devices, such as a cell phone, laptop, or wireless PDA. We highlight three applications built on top of the platform that demonstrate its generality and utility in a variety of group settings: namely, web browsing, polling, and entertainment.

A new algorithm is proposed for removing large objects from digital images. The challenge is to fill in the hole that is left behind in a visually plausible way. In the past, this problem has been addressed by two classes of algorithms: 1) "texture synthesis" algorithms for generating large image regions from sample textures and 2) "inpainting" techniques for filling in small image gaps. The former has been demonstrated for "textures"--repeating two-dimensional patterns with some stochasticity; the latter focus on linear "structures" which can be thought of as one-dimensional patterns, such as lines and object contours. This paper presents a novel and efficient algorithm that combines the advantages of these two approaches. We first note that exemplar-based texture synthesis contains the essential process required to replicate both texture and structure; the success of structure propagation, however, is highly dependent on the order in which the filling proceeds. We propose a best-first algorithm in which the confidence in the synthesized pixel values is propagated in a manner similar to the propagation of information in inpainting. The actual color values are computed using exemplar-based synthesis. In this paper, the simultaneous propagation of texture and structure information is achieved by a single, efficient algorithm. Computational efficiency is achieved by a block-based sampling process. A number of examples on real and synthetic images demonstrate the effectiveness of our algorithm in removing large occluding objects, as well as thin scratches. Robustness with respect to the shape of the manually selected target region is also demonstrated. Our results compare favorably to those obtained by existing techniques.

Digital photography has made it possible to quickly and easily take a pair of images of low-light environments: one with flash to capture detail and one without flash to capture ambient illumination. We present a variety of applications that analyze and combine the strengths of such flash/no-flash image pairs. Our applications include denoising and detail transfer (to merge the ambient qualities of the no-flash image with the high-frequency flash detail), white-balancing (to change the color tone of the ambient image), continuous flash (to interactively adjust flash intensity), and red-eye removal (to repair artifacts in the flash image). We demonstrate how these applications can synthesize new images that are of higher quality than either of the originals.

A new algorithm is proposed for removing large objects from digital images. The challenge is to fill in the hole that is left behind in a visually plausible way. In the past, this problem has been addressed by two classes of algorithms: (i) "texture synthesis" algorithms for generating large image regions from sample textures, and (ii) "inpainting" techniques for filling in small image gaps. The former work well for "textures" - repeating two dimensional patterns with some stochasticity; the latter focus on linear "structures" which can be thought of as one dimensional patterns, such as lines and object contours. This paper presents a novel and efficient algorithm that combines the advantages of these two approaches. We first note that exemplar-based texture synthesis contains the essential process required to replicate both texture and structure; the success of structure propagation, however, is highly dependent on the order in which the filling proceeds. We propose a best-first algorithm in which the confidence in the synthesized pixel values is propagated in a manner similar to the propagation of information in inpainting. The actual color values are computed using exemplar-based synthesis. Computational efficiency is achieved by a block-based sampling process. A number of examples on real and synthetic images demonstrate the effectiveness of our algorithm in removing large occluding objects as well as thin scratches. Robustness with respect to the shape of the manually selected target region is also demonstrated. Our results compare favorably to those obtained by existing techniques.

We describe an end-to-end system that capitalizes on geographic location tags for digital photographs. The World Wide Media eXchange (WWMX) database indexes large collections of image media by several pieces of metadata including timestamp, owner, and critically, location stamp. The location where a photo was shot is important because it says much about its semantic content, while being relatively easy to acquire, index, and search.The process of building, browsing, and writing applications for such a database raises issues that have heretofore been un- addressed in either the multimedia or the GIS community. This paper brings all of these issues together, explores different options, and offers novel solutions where necessary. Topics include acquisition of location tags for image media, data structures for location tags on photos, database optimization for location-tagged image media, and an intuitive UI for browsing a massive location-tagged image database. We end by describing an application built on top of the WWMX, a lightweight travelogue-authoring tool that automatically creates appropriate context maps for a slideshow of location-tagged photographs.

A new exemplar-based, probabilistic paradigm for visual tracking is presented. Probabilistic mechanisms are attractive because they handle fusion of information, especially temporal fusion, in a principled manner. Exemplars are selected representatives of raw training data, used here to represent probabilistic mixture distributions of object configurations. Their use avoids tedious hand-construction of object models and problems with changes of topology. Using exemplars in place of a parameterized model poses several challenges, addressed here with what we call the "Metric Mixture" (M/sup 2/) approach. The M/sup 2/ model has several valuable properties. Principally, it provides alternatives to standard learning algorithms by allowing the use of metrics that are not embedded in a vector space. Secondly, it uses a noise model that is learned from training data. Lastly, it eliminates any need for an assumption of probabilistic pixelwise independence. Experiments demonstrate the effectiveness of the M/sup 2/ model in two domains tracking walking people using chamfer distances on binary edge images and tracking mouth movements by means of a shuffle distance.

A precise analysis of an entire image is computationally wasteful if one is interested in finding a target object located in a subregion of the image. A useful "attention strategy" can reduce the overall computation by carrying out fast but approximate image measurements and using their results to suggest a promising subregion. The paper proposes a maximum-likelihood attention mechanism that does this. The attention mechanism recognizes that objects are made of parts and that parts have different features. It works by proposing object part and image feature pairings which have the highest likelihood of coming from the target. The exact calculation of the likelihood as well as approximations are provided. The attention mechanism is adaptive, that is, its behavior adapts to the statistics of the image features. Experimental results suggest that, on average, the attention mechanism evaluates less than 2 percent of all part-feature pairs before selecting the actual object, showing a significant reduction in the complexity of visual search.

We present an algorithm for estimation of head orientation, given cropped images of a subject's head from any viewpoint. Our algorithm handles dramatic changes in illumination, applies to many people without per-user initialization, and covers a wider range (e.g., side and back) of head orientations than previous algorithms. The algorithm builds an ellipsoidal model of the head, where points on the model maintain probabilistic information about surface edge density. To collect data for each point on the model, edge-density features are extracted from hand-annotated training images and projected into the model. Each model point learns a probability density function from the training observations. During pose estimation, features are extracted from input images; then, the maximum a posteriori pose is sought, given the current observation.

Background maintenance is a frequent element of video surveillance systems. We develop Wallflower, a three-component system for background maintenance: the pixel-level component performs Wiener filtering to make probabilistic predictions of the expected background; the region-level component fills in homogeneous regions of foreground objects; and the frame-level component detects sudden, global changes in the image and swaps in better approximations of the background. We compare our system with 8 other background subtraction algorithms. Wallflower is shown to outperform previous algorithms by handling a greater set of the difficult situations that can occur. Finally, we analyze the experimental results and propose normative principles for background maintenance.

In the past several years, the speed of standard processors has reached the point where interesting problems requiring visual tracking can be carried out on standard workstations. However, relatively little attention has been devoted to developing visual tracking technology in its own right. In this article, we describe X Vision, a modular, portable framework for visual tracking. X Vision is designed to be a programming environment for real-time vision which provides high performance on standard workstations outfitted with a simple digitizer. X Vision consists of a small set of image-level tracking primitives, and a framework for combining tracking primitives to form complex tracking systems. Efficiency and robustness are achieved by propagating geometric and temporal constraints to the feature detection level, where image warping and specialized image processing are combined to perform feature detection quickly and robustly. Over the past several years, we have used X Vision to construct several vision-based systems. We present some of these applications as an illustration of how useful, robust tracking systems can be constructed by simple combinations of a few basic primitives combined with the appropriate task-specific constraints.

We present the Incremental Focus of Attention (IFA) architecture for adding robustness to software-based, real-time, motion trackers. The framework provides a structure which, when given the entire camera image to search, efficiently focuses the attention of the system into a narrow set of possible slates that includes the target state. IFA offers a means for automatic tracking initialization and reinitialization when environmental conditions momentarily deteriorate and cause the system to lose track of its target. Systems based on the framework degrade gracefully as various assumptions about the environment are violated. In particular, multiple tracking algorithms are layered so that the failure of a single algorithm causes another algorithm of less precision to take over, thereby allowing the system to return approximate feature state information.

We introduce Servomatic, a modular system for robot motion control based on calibration-insensitive visual servoing. A small number of generic motion control operations referred to as primitive skills use stereo visual feedback to enforce a specific task-space kinematic constraint between a robot end-effector and a set of target features. Primitive skills are able to position with an accuracy that is independent of errors in hand-eye calibration and are easily combined to form more complex kinematic constraints as required by different applications. The system has been applied to a number of example problems, showing that modular, high precision, vision-based motion control is easily achieved with off-the-shelf hardware. Our continuing goal is to develop a system where low-level robot control ceases to be a concern to higher-level robotics researchers.

Visual tracking is prone to distractions, where features similar to the target features guide the track away from its intended object. Global shape models and dynamic models are necessary for completely distraction-free contour tracking, but there are cases when component feature trackers alone can be expected to avoid distraction. We define the tracking problem in general and devise a method for local, window-based, feature trackers to track accurately in spite of background distractions. The algorithm is applied to a generic line tracker and a snake-like contour tracker which are then analyzed with respect to previous contour-trackers. We discuss the advantages and disadvantages of our approach and suggest that existing model-based trackers can be improved by incorporating similar techniques at the local level.