Generative AI is rapidly diffusing worldwide, yet access remains uneven. In informal settlements, barriers of cost, literacy and connectivity can exclude residents from AI-enabled self-expression. This paper presents Street Scenes: a public appliance for walk-up interaction with generative AI video in Dharavi, Mumbai. Inspired by the “Hole in the Wall” computers and previous Dharavi speech deployments, the system lets passers-by capture phone images, add voice-, button- and dial-based prompts, and generate short videos to view and leave locally. We report on ideation workshops, two Wizard-of-Oz prototypes and a 13-day in-situ deployment across Dharavi street locations. Findings show residents appropriating AI for play, self-presentation, small business promotion and community messaging, while also raising concerns about privacy, trust and misuse. We contribute: (1) a model for public AI appliances; (2) empirical insights into community engagement with generative AI; and, (3) design lessons for accessible, equitable and community-governed AI systems.

Computer vision systems are increasingly used by blind individuals to navigate their lives, helping, for example, locate objects such as doors or chairs. Yet these recognition systems do not work for many personal objects a blind user might want to find, such as keys or a special notebook. In response, efforts created personalized recognition systems, where individuals train their phones to identify and locate things, like a coffee mug or white cane, using example images/videos. However, these tools are trained on data from high-resource contexts, not necessarily reflecting India’s material culture. This paper discusses the contribution of the ORBIT-India dataset, which extends these tools to the Indian context, home of the world’s largest blind population. The ORBIT-India dataset comprises 105,243 images from 587 videos, representing 76 unique objects. We use this experience to examine dataset collection practices translated from high- to low-resource settings, providing recommendations to support cross-geography dataset collection.

Sketch-to-image systems let users transform simple line drawings into realistic images, but current workflows force users into tedious redraw-regenerate cycles that slow creative exploration. We introduce two complementary interaction techniques that reduce iteration friction: AutoSketch, which extends partial sketches through AI-driven completions (pre-generation support), and BackSketch, which transforms generated images back into editable sketches at multiple abstraction levels (post-generation support). In a study with 30 participants, the results indicate that both techniques can improve exploration and expressiveness compared to a baseline sketch-to-image system, while AutoSketch also can increase users’ sense of agency and co-creation with the AI. We contribute new evidence that shifting support before or after generation opens distinct pathways for balancing user control and system initiative. Together, our results establish pre- and post-generation assistance as a design space for co-creative sketch-to-image systems.

Assistive technologies, such as smartphone-based object-recognition (OR) apps, provide visual assistance to people who are blind or lowvision to enable increased independent participation in society.While previous research has explored the functional accessibility of object-recognition technologies, little attention has been given to their social accessibility, particularly in interdependent sociocultural contexts of the Global South.Through a mixed-methods approach, employing a seven-day diary study followed by one-onone interviews with seven OR app users in India, we explore their experiences in depth.Our findings highlight the nuances of what interdependence looks like in a multicultural, Indian society, as people navigate public and private spheres with a camera-based assistive technology designed for independent, western contexts.We argue for the necessity to design assistive technologies following the interdependence framework that accommodates the social and cultural context of the Global South.Additionally, we propose design guidelines for assistive technologies in community-oriented societies, emphasizing community-centered approaches, cultural alignment, and locally adaptable designs.

We report on community-centered, collaborative research that weaves together HCI, natural language processing, linguistic, and design insights to develop spoken language technologies for unwritten languages. Across three visits to a Banjara farming community in India, we use participatory, technical, and creative methods to engage community members, collect spoken language photo annotations, and develop an information retrieval (IR) system. Drawing on orality theory, we interrogate assumptions and biases of current speech interfaces and create a simple application that leverages our IR system to match fluidly spoken queries with recorded annotations and surface corresponding photos. In-situ evaluations show how our novel approach returns reliable results and inspired the co-creation of media retrieval use-cases that are more appropriate in oral contexts. The very low (< 4h) spoken data requirements makes our approach adaptable to other contexts where languages are unwritten or have no digital language resources available.

In this paper we develop approaches to automatic speech recognition (ASR) development that suit the needs and functions of under-heard language speakers. Our novel contribution to HCI is to show how community-engagement can surface key technical and social issues and opportunities for more effective speech-based systems. We introduce a bespoke toolkit of technologies and showcase how we utilised the toolkit to engage communities of under-heard language speakers; and, through that engagement process, situate key aspects of ASR development in community contexts. The toolkit consists of (1) an information appliance to facilitate spoken-data collection on topics of community interest, (2) a mobile app to create crowdsourced transcripts of collected data, and (3) demonstrator systems to showcase ASR capabilities and to feed back research results to community members. Drawing on the sensibilities we cultivated through this research, we present a series of challenges to the orthodoxy of state-of-the-art approaches to ASR development.

What do pedestrian crossings, ATMs, elevators and ticket machines have in common? These are just a few of the ubiquitous yet essential elements of public-space infrastructure that rely on physical buttons or touchscreens; common interactions that, until recently, were considered perfectly safe to perform. This work investigates how we might integrate touchless technologies into public-space infrastructure in order to minimise physical interaction with shared devices in light of the ongoing COVID-19 pandemic. Drawing on an ethnographic exploration into how public utilities are being used, adapted or avoided, we developed and evaluated a suite of technology probes that can be either retrofitted into, or replace, these services. In-situ community deployments of our probes demonstrate strong uptake and provide insight into how hands-free technologies can be adapted and utilised for the public domain; and, in turn, used to inform the future of walk-up-and use public technologies.

Automatic Speech Recognition (ASR) researchers are turning their attention towards supporting low-resource languages, such as isiXhosa or Marathi, with only limited training resources. We report and reflect on collaborative research across ASR & HCI to situate ASR-enabled technologies to suit the needs and functions of two communities of low-resource language speakers, on the outskirts of Cape Town, South Africa and in Mumbai, India. We build on longstanding community partnerships and draw on linguistics, media studies and HCI scholarship to guide our research. We demonstrate diverse design methods to: remotely engage participants; collect speech data to test ASR models; and ultimately field-test models with users. Reflecting on the research, we identify opportunities, challenges, and use-cases of ASR, in particular to support pervasive use of WhatsApp voice messaging. Finally, we uncover implications for collaborations across ASR & HCI that advance important discussions at CHI surrounding data, ethics, and AI.

Many of us daily encounter shadow and reflected light patterns alongside macro-level changes in ambient light levels. These are caused by elements—opaque objects, glass, mirrors, even clouds—in our environment interfacing with sunlight or artificial indoor lighting. Inspired by these phenomena, we explored ways of creating digitally-supported displays that use light, shade and reflection for output and harness the energy they need to operate from the sun or indoor ambient light. Through a set of design workshops we developed exemplar devices: SolarPix, ShadMo and GlowBoard. We detail their function and implementation, as well as evidencing their technical viability. The designs were informed by material understandings from the Global North and Global South and demonstrated in a cross-cultural workshop run in parallel in India and South Africa where community co-designers reflected on their uses and value given lived experience of their communication practices and unreliable energy networks.

Working with emergent users in two of Mumbai’s slums, we explored the value and uses of photovoltaic (PV) self-powering digital materials. Through a series of co-design workshops, a diary study and responses by artists and craftspeople, we developed the PV-Pix concept for inter-home connections. Each PV-Pix element consists of a deformable energy harvesting material that, when actuated by a person in one home, changes its physical state both there and in a connected home. To explore the concept we considered two forms of PV-Pix: one uses rigid materials and the other flexible ones. We deployed two low-fidelity prototypes, each constructed of a grid of one PV-Pix type, in four slum homes over a four week period to further understand the usability and uses of the materials, eliciting interesting inter-family communication practices. Encouraged by these results we report on a first-step towards working prototypes and demonstrate the technical viability of the approach.

The interactive, digital future with its seductive vision of Internet-of-Things connected sensors, actuators and displays comes at a high cost in terms of both energy demands and the clutter it brings to the physical world. But what if such devices were made of materials that enabled them to self-power their interactive features? And, what if those materials were directly used to build aesthetically pleasing environments and objects that met practical physical needs as well as digital ones? In this paper we introduce PV-Tiles ? a novel material that closely couples photovoltaic energy harvesting and light sensing materials with digital interface components. We consider potential contexts, use-cases and light gestures surfaced through co-creation workshops; and, present initial technological designs and prototypes. The work opens a new set of opportunities and collaborations between HCI and material science, stimulating technical and design pointers to accommodate and exploit the material's properties.

There is widespread concern over the ways speech assistant providers currently use humans to listen to users' queries without their knowledge. We report two iterations of the TalkBack smart speaker, which transparently combines machine and human assistance. In the first, we created a prototype to investigate whether people would choose to forward their questions to a human answerer if the machine was unable to help. Longitudinal deployment revealed that most users would do so when given the explicit choice. In the second iteration we extended the prototype to draw upon spoken answers from previous deployments, combining machine efficiency with human richness. Deployment of this second iteration shows that this corpus can help provide relevant, human-created instant responses. We distil lessons learned for those developing conversational agents or other AI-infused systems about how to appropriately enlist human-in-the-loop information services to benefit users, task workers and system performance.

Multiple fingers are often used for efficient interaction with handheld computing devices. Currently, any tactile feedback provided is felt on the finger pad or the palm with coarse granularity. In contrast, we present a new tactile feedback technique, Active PinScreen, that applies localised stimuli on multiple fingers with fine spatial and temporal resolution. The tactile screen uses an array of solenoid-actuated magnetic pins with millimetre scale form-factor which could be deployed for back-of-device handheld use without instrumenting the user. As well as presenting a detailed description of the prototype, we provide the potential design configurations and the applications of the Active PinScreen and evaluate the human factors of tactile interaction with multiple fingers in a controlled user evaluation. The results of our study show a high recognition rate for directional and patterned stimulation across different grip orientations as well as within- and between- fingers. We end the paper with a discussion of our main findings, limitations in the current design and directions for future work.

This paper explores the use of conversational speech question and answer systems in the challenging context of public spaces in slums. A major part of this work is a comparison of the source and speed of the given responses; that is, either machine-powered and instant or human-powered and delayed. We examine these dimensions via a two-stage, multi-sited deployment. We report on a pilot deployment that helped refine the system, and a second deployment involving the installation of nine of each type of system within a large Mumbai slum for a 40-day period, resulting in over 12,000 queries. We present the findings from a detailed analysis and comparison of the two question-answer corpora; discuss how these insights might help improve machine-powered smart speakers; and, highlight the potential benefits of multi-sited public speech installations within slum environments.

Touchscreens are the predominant medium for interactions with digital services; however, their current fixed form factor narrows the scope for rich physical interactions by limiting interaction possibilities to a single, planar surface. In this paper we introduce the concept of PickCells, a fully re-configurable device concept composed of cells, that breaks the mould of rigid screens and explores a modular system that affords rich sets of tangible interactions and novel across-device relationships. Through a series of co-design activities -- involving HCI experts and potential end-users of such systems -- we synthesised a design space aimed at inspiring future research, giving researchers and designers a framework in which to explore modular screen interactions. The design space we propose unifies existing works on modular touch surfaces under a general framework and broadens horizons by opening up unexplored spaces providing new interaction possibilities. In this paper, we present the PickCells concept, a design space of modular touch surfaces, and propose a toolkit for quick scenario prototyping.

Current haptic feedback techniques on handheld devices are applied to the finger pad or the palm of the user. These state-of-the-art approaches are coarse-grained and tend to be intrusive, rather than subtle. In contrast, we present a new feedback technique that applies stimuli around the periphery of the finger pulp, demonstrating how this can provide rich, nuanced haptic information. We use a reconfigurable haptic device employing a ferromagnetic marble for back-of-the device handheld use, which, for the first time, probes, without instrumenting the user, the periphery of the distal phalanx with localised stimulation. We present the design-space afforded by this new technique and evaluate the human-factors of finger-peripheral touch interaction in a controlled user-study. We report results with marbles of different diameters, speeds and a combination of poking, lateral vibration and patterns; present the resulting design guidelines for finger-periphery haptic feedback; and, illustrate its potential with use case scenarios.

Mainstream digital interactions are spread over a plethora of devices and form-factors, from mobiles to laptops; printouts to large screens. For emergent users, however, such abundance of choice is rarely accessible or affordable. In particular, viewing mobile content on a larger screen, or printing out copies, is often not available. In this paper we present Sustainabot - a small robot printer that uses everyday materials to print shapes and patterns from mobile phones. Sustainabot was proposed and developed by and with emergent users through a series of co-creation workshops. We begin by discussing this process, then detail the open-source mobile printer prototype. We carried out two evaluations of Sustainabot, the first focused on printing with materials in situ, and the second on understandability of its output. We present these results, and discuss opportunities and challenges for similar developments. We conclude by highlighting where and how similar devices could be used in future.

“Designed in California” is a brand statement used by high-tech manufacturers to denote provenance and cachet of digital innovation and modernity. In this paper, we explore philosophically alternate design perspectives to those this statement embodies, reporting and reflecting on a long-term multi-sited project that seeks to diversify future-making by engaging communities of “emergent” users in “developing” regions. We argue that digital technologies are typically created with a design lens firmly focused on “first world” populations, assuming a base set of cultural norms, resource availabilities, and technological experience levels that do not strongly align with those of emergent users. We discuss and argue for inclusive technology design methods, present our approach, and detail indicative results and case studies as an example of the potential of these perspectives in uncovering radical innovations. Distilling findings and lessons learned, we present a methodology—itinerative design—that pivots between emergent user communities across multiple regions, driving digital innovation through the periphery of mainstream design’s current remit.

Millions of homes worldwide enjoy access to digital content and services through smart speakers such as Amazon's Echo and Google's Home. Promotional materials and users' own videos typically show homes that have many well-resourced rooms, with good power and data infrastructures. Over the last several years, we have been working with slum communities in India, whose dwellings are usually very compact (one or two rooms), personal home WiFi is almost unheard of, power infrastructures are far less robust, and financial resources put such smart speakers out of individual household reach. Inspired by the "hole in the wall" internet-kiosk programme, we carried out workshops with slum inhabitants to uncover issues and opportunities for providing a smart-speaker-type device in public areas and passageways. We designed and deployed a simple probe that allowed passers-by to ask and receive answers to questions. In this paper, we present the findings of this work, and a design space for such devices in these settings.

We present APPropriate -- a novel mobile design to allow users to temporarily annex any Android device for their own use. APPropriate is a small, cheap storage pod, designed to be easily carried in a pocket or hidden within clothing. Its purpose is simple: to hold a copy of the local content an owner has on their mobile, liberating them from carrying a phone, or allowing them to use another device that provides advantages over their own. Picking up another device when carrying APPropriate transfers all pertinent content to the borrowed device (using local no-cost WiFi from the APPropriate device), transforming it to give the impression that they are using their own phone. While APPropriate is useful for a wide range of contexts, the design was envisaged through a co-design process with resource-constrained emergent users in three countries. Lab studies and a subsequent deployment on participants' own devices identified key benefits of the approach in these contexts, including for security, resource sharing, and privacy.

We present Tangible Drops, a visio-tactile display that for the first time provides physical visualization and tactile feedback using a planar liquid interface. It presents digital information interactively by tracing dynamic patterns on horizontal flat surfaces using liquid metal drops on a programmable electrode array. It provides tactile feedback with directional information in the 2D vector plane using linear locomotion and/or vibration of the liquid metal drops. We demonstrate move, oscillate, merge, split and dispense-from-reservoir functions of the liquid metal drops by consuming low power (450 mW per electrode) and low voltage (8--15 V). We report on results of our empirical study with 12 participants on tactile feedback using 8 mm diameter drops, which indicate that Tangible Drops can convey tactile sensations such as changing speed, varying direction and controlled oscillation with no visual feedback. We present the design space and demonstrate the applications of Tangible Drops, and conclude by suggesting potential future applications for the technique.

Digital painting is an increasingly popular medium of expression for many artists, yet when compared to its traditional equivalents of physical brushes and viscous paint it lacks a dimension of tangibility. We conducted observations and interviews with physical and digital artists, which gave us a strong understanding of the types of interactions used to create both physical and digital art, and the important role tangibility plays within these experiences. From this, we developed a unique liquid-like tangible display for mobile, digital colour mixing. Using a chemical hydrogel that changes its viscosity depending on temperature, we are able to create some resemblances to the feeling of mixing paint with a finger. This paper documents the information gathered from working with artists, how this process informed the development of a mobile painting attachment, and an exploration of its capabilities. After returning with our prototype, we found that it provided artists with sensations of oil and acrylic paint mixing and also successfully mimicked how paints are laid out on a paint palette.

Many users value the ability to have quick and frequent sight of their mobiles when in public settings. However, in doing so, they expose themselves to potential risks, ranging from being targets of robbery to the more subtle social losses through being seen to be rude or inattentive to those around them. In nature, some animals can blend into their environments to avoid being eaten or to reduce their impact on the ecosystem around them. Taking inspiration from these evolved systems we investigate the notion of chameleon approaches for mobile interaction design. Our probes were motivated, inspired and refined through extended interactions with people drawn from contexts with differing ranges of security and privacy concerns. Through deployments on users' own devices, our prototypes show the value of the concept. The encouraging results motivate further research in materials and form factors that can provide more effective automatic plain-sight hiding.

This research forms part of a wider body of work focused around involving emergent users---those just beginning to get access to mobile devices---in the development and refinement of far-future technologies. In this paper we present an evaluation of a new type of deformable slider with emergent users, designed to investigate whether shape-changing interfaces provide any benefit over touchscreens for this type of user. Our trials, which took place in two contexts and three disparate regions, revealed that while there was a clear correlation between performance and technology exposure, emergent users had similar ability with both touchscreen and deformable controls.

Mainstream mobile interactions are focused around individual devices, with any collaboration happening via `the cloud'. We carried out design workshops with emergent users, revealing opportunities for novel collocated collaborative interactions. In this paper we present Better Together - a framework for disaggregating services, splitting interaction elements over separate mobiles. This distribution supports both sharing of resources (such as screen real-estate, or mobile data); and, scaffolding of inclusive interaction in mixed groups (e.g., in terms of literacy or prior technology exposure). We developed two prototypes to explore the concept, trialling the first---collocated group-based shopping list making---with emergent users in South Africa and India. We deployed the second probe, which splits YouTube into its constituent parts across separate mobiles, in a longitudinal study with users in Kenya, South Africa and India. We describe the concept and design process, and report on the design's suitability for emergent users based on our results.

In this paper, we demonstrate a method for the dynamic 2D transformation of liquid matter and present unique organic animations based on spatio-temporally controlled electric fields. In particular, we deploy a droplet of liquid metal (Gallium indium eutectic alloy) in a 7x7 electrode array prototype system, featuring an integrated image tracking system and a simple GUI. Exploiting the strong dependance of EGaIn's surface tension on external electric voltages, we control multiple electrodes dynamically to manipulate the liquid metal into a fine-grained desired shape. Taking advantage of the high conductivity of liquid metals, we introduce a shape changing, reconfigurable smart circuit as an example of unique applications. We discuss system constraints and the overarching challenge of controlling liquid metals in the presence of phenomena such as splitting, self-electrode interference and finger instabilities. Finally, we reflect on the broader vision of this project and discuss our work in the context of the wider scope of programmable materials.

We present a programmable liquid matter which can dynamically transform its 2D shape into a variety of forms and present unique organic animations based on spatio-temporally controlled electric fields. We deployed a EGaIn (Gallium indium eutectic alloy) liquid metal as our smart liquid material since it features a superior electric conductivity in spite of a liquid state and presents a high dynamic range of surface tension and 2D area controlled by applied voltage strength and polarity. Our proposed liquid metal shape and motion control algorithms with dynamically patterned electric fields realize path tracing organic animation. We demonstrate an interactive 7x7 electrode array control system with a computer vision based GUI system to enable novice users to physically draw alphabet letters and 2D shapes by unique animatronics of liquid metals.

Mobile and ubiquitous computing researchers have long envisioned future worlds for users in developed regions. Steered by such visions, they have innovated devices and services exploring the value of alternative propositions with and for individuals, groups and communities. Meanwhile, such radical and long-term explorations are uncommon for what have been termed emergent users; users, that is, for whom advanced technologies are just within grasp. Rather, a driving assumption is that today’s high-end mobile technologies will “trickle down” to these user groups in due course. In this paper, we open the debate about what mobile technologies might be like if emergent users were directly involved in creating their visions for the future 5–10 years from now. To do this, we report on a set of envisioning workshops in India, South Africa and Kenya that provide a roadmap for valued, effective devices and services for these regions in the next decade.

In this paper we present the concept of Emergeables -- mobile surfaces that can deform or 'morph' to provide fully-actuated, tangible controls. Our goal in this work is to provide the flexibility of graphical touchscreens, coupled with the affordance and tactile benefits offered by physical widgets. In contrast to previous research in the area of deformable displays, our work focuses on continuous controls (e.g., dials or sliders), and strives for fully-dynamic positioning, providing versatile widgets that can change shape and location depending on the user's needs. We describe the design and implementation of two prototype emergeables built to demonstrate the concept, and present an in-depth evaluation that compares both with a touchscreen alternative. The results show the strong potential of emergeables for on-demand, eyes-free control of continuous parameters, particularly when comparing the accuracy and usability of a high-resolution emergeable to a standard GUI approach. We conclude with a discussion of the level of resolution that is necessary for future emergeables, and suggest how high-resolution versions might be achieved.

Rural developing regions are often defined in terms of their resource constraints, including limited technology exposure, lack of power, and low access to data connections (leading to an inability to access information from digital or physical sources), as well as being amongst the most socio-economically disadvantaged and least literate in their countries’ populations. This article is focused around information access in such regions, aiming to build upon and extend the audio-based services that are already widely used in order to provide access to further types of media. In this article, then, we present an extended exploration of AudioCanvas —an interactive telephone-based audio information system that allows cameraphone users to interact directly with their own photos of physical media to receive narration or description. Our novel approach requires no specialist hardware, literacy, or data connectivity, making it far more likely to be a suitable solution for users in such regions.

Current uses of smartwatches are focused solely around the wearer's content, viewed by the wearer alone. When worn on a wrist, however, watches are often visible to many other people, making it easy to quickly glance at their displays. We explore the possibility of extending smartwatch interactions to turn personal wearables into more public displays. We begin opening up this area by investigating fundamental aspects of this interaction form, such as the social acceptability and noticeability of looking at someone else's watch, as well as the likelihood of a watch face being visible to others. We then sketch out interaction dimensions as a design space, evaluating each aspect via a web-based study and a deployment of three potential designs. We conclude with a discussion of the findings, implications of the approach and ways in which designers in this space can approach public wrist-worn wearables.

In this paper we present a novel interface for collaborative creation of evolving audio-visual documents. PaperChains allows users to sketch on paper and then augment with digital audio, allowing both the physical and digital objects to evolve simultaneously over time. The technique we have developed focuses on affordability and accessibility in its design, using standard cameraphones and telephone connections, which allow it to be used in regions where literacy, technological experience and data connections cannot necessarily be taken for granted. The main use-case that we focus on in this paper is for collaborative storytelling, an area which has been well studied and previously proven to be of value in resource constrained environments. To investigate the relevance of the approach in these contexts, we undertook two usability evaluations in India and South Africa. Results from these investigations indicate users' ability to both create and interpret stories using the software, as well as demonstrating high overall usability and enjoyment. We end with a discussion of the implications of our design and opportunities for use in other contexts.

The world is faced with a growing number of people who live with chronic medical conditions. There have been numerous digital interventions into personal management of these diseases in recent years, yet gaps remain in the HCI literature. In particular, we lack a systematic understanding of user requirements in tools that support independent management while away from external influences. This paper presents a first investigation into low-intervention support for self-management. A mobile application enabled individuals to capture contextual information related to their health in the form of photographs. Through a month-long user study, we identify four management trends amongst our participants and describe their influence on mobile application adoption.

Digital markers such as barcodes and QR codes are ubiquitous. However, these codes are normally used only for retrieving a small amount of information, such as a product identifier or a web link. Much previous work has investigated the value of associating digital content with physical objects in everyday scenarios, but has so far relied primarily on adding new markers to existing items, or studied only short-term usage. In this paper, we explore the benefits of "commandeering" existing object labels to support this interaction. We add a social layer to existing digital codes, allowing users to "tag" any marker in their environment with their own messages, which can then be viewed by any other user. The core contribution of this work is the findings and insights that were collected in user studies. We explored the use of our design via two deployments that demonstrate the potential of such a system beyond its playful starting point. We conclude the work by drawing out a number of key design elements for future appropriation designs.

Barcodes are all around us--on books, groceries and other products--but these everyday markers are typically used for a single focused purpose. In this paper we explore the concept of "piggybacking" on ubiquitous markers to facilitate indoor navigation. Our initial probe--BookMark--allows library visitors to scan any nearby book to provide a custom map to the location of a desired item. In contrast to previous indoor navigation systems, our approach repurposes existing markers on physical items that are already in the navigation space, meaning that no additional infrastructure is required. We evaluated the BookMark probe in a large university library, showing its potential with real library users. In addition, we illustrate how the general technique shows further potential in other similar barcode-rich environments.

In this paper we present a novel interaction technique that helps to make textual information more accessible to those with low or no textual literacy skills. AudioCanvas allows cameraphone users to interact directly with their own photos of printed media to receive audio feedback or narration. The use of a remote telephone-based service also allows our design to be used over a standard phone line, removing the need for data connections, which can be problematic in developing regions. We show the value of the technique via user evaluations in both a rural Indian village and a South African township.

In this paper we introduce Acoustic Quick Response codes to facilitate sharing between Interactive Voice Response (IVR) service users. IVRs are telephone-based, and similar to the world wide web in many aspects, but currently lack support for content sharing. Our approach uses 'audio codes' to let people share their call positions, and allows callers to hold their normal (low-end) handsets together to synchronise. The technique uses remote generation and recognition of audio codes to ensure that sharing is possible on any type of phone without the need for textual literacy or an internet connection. We begin by exploring existing user needs for sharing, then evaluate the technical robustness of our audio-based design. We demonstrate the value of the approach for voice service users over several separate studies--including an eight-month extended field deployment--then conclude with a discussion of future possibilities for such scenarios.

Mobile reading is becoming evermore popular with the introduction of eInk devices such as the Kindle, as well as the many reading applications available on slate PCs and cellular handsets. The portable nature and large storage capacity of these modern mobile devices is making reading a more technology orientated activity. One aspect of mobile reading that has been given surprisingly little attention is collective reading - which is a common activity with paper documents. We investigate the support of group reading using slate PCs, focussing on collective annotation. In the past, desktop PCs have proved inferior in many ways for reading, when compared to paper. Notably, user evaluations of our new system, BuddyBooks, demonstrate that the slate PC form factor can, in contrast, provide advantages for group reading. While annotation practices change with the new format, coordinating within the group can be improved when touch-interaction is carefully exploited.

Attentive reading is a complex and cognitively demanding task that uses note-taking and annotation to support the reader’s interpretation of the document. When reading on paper, extensive use of highlighting and other activities are conducted to support attentive reading, but this rich behaviour is not used with digital documents. Many users therefore print digital documents and then interact with them in physical form. This paper presents the “Digital Reading Desk,” an enhanced digital reading environment that provides support for attentive reading, providing a large working space for notes that mimics the use of desk space in a conventional physical environment. The Reading Desk uses a single tool to support both annotation and bookmarking, simplifying both the user’s learning of the system and their use of tools. Evaluation of the Reading Desk indicates preference for it, as well as considerable behavioural differences between our approach and traditional digital reading tools.

Reading is increasingly being performed interactively on-screen; for instance, new novels are now routinely released in electronic format for viewing on PCs and mobile devices. Unfortunately, on-screen reading loses many of the natural features of conventional physical media, such as the ability to annotate, slip in bookmarks, turn page corners, and so on. How best should these features be represented electronically? Can computerized representations give benefits that excel the conventional benefits of paper? We describe the design and implementation of a novel reading system that mimics key properties of paper and surpasses them by incorporating digital techniques. A comparative user study evaluating the system confirmed the effectiveness of the features and the value of the system as a whole.