This article presents an account of the Interaction Research Studio’s distinctive approach to incorporating data into design, unpacking a vital but previously unexamined facet of our two-decade-long research programme. We demonstrate how data have served as a pragmatic tool in creating and engaging contextually aware computational products that provide alternatives to dominant task-oriented and consumerist narratives by promoting interpretation, imagination, spirituality and socio-cultural reflection. Leveraging an annotated portfolio approach, we revisit our diverse portfolio of designs, starting from individual reflections before surfacing overarching themes. This article foregrounds our strategies for utilising data, the associated labour and the challenges that have arisen throughout our design process. This emergent and empirical account may complement more theoretical treatments of designing with data and provide valuable insights for researchers, designers and practitioners navigating the complexities of data-integrated design in the contemporary technological landscape.

We are a mother and son who have been using a pair of simple, self-build communication devices to maintain a feeling of connection while separated by over 5,000 miles. The devices, called Light Touch, only allow us to send one another slowly-fading, coloured lights, yet we have been surprised by how much our ongoing interaction with them means to us. This paper contributes an autoethnographical account of our experiences over the last two years, including our initial experiences with the devices, and focusing on various aspects of our day-to-day use. Based on our observations, we discuss the features that have proven important in mediating our feelings of connection. We point out, however, that their success is contingent on our context of use and the nature of our bond, and suggest that simple systems like Light Touch may support emotional communication, but only if they are well-matched to settings and relationships.

Practice-based design research is often emergent. Methods, tactics, goals and even topics can unfold and change as researchers adapt and learn in the course of their projects. This adaptability is one of the strengths of design as an approach to research, but it seems to contradict assumptions about research as a systematic, inquiry-led investigation. This leads to a tension in practicing and reporting research that we unpack here, before making a series of suggestions for practice-based researchers and reviewers about how to better navigate emergence when pursuing, reporting and evaluating practice-based design research.

Yo–Yo Machines are playful communication devices designed to help people feel socially connected while physically separated. We designed them to reach as many people as possible, both to make a positive impact during the COVID-19 pandemic and to assess a self-build approach to circulating research products and the appeal of peripheral and expressive communication devices. A portfolio of four distinct designs, based on over 30 years of research, were made available for people to make by following simple online instructions (yoyomachines.io). Each involves connecting a pair of identical devices over the internet to allow simple communication at a distance. This paper describes our motivation for the project, previous work in the area, the design of the devices, supporting website and publicity, and how users have made and used Yo-Yo Machines. Finally, we reflect on what we learned about peripheral and expressive communication devices and implications for the self-build approach.

In this paper, we extend the concept of annotated portfolios to include designs for new domains. Although annotated portfolios were intentionally left open to interpretation and appropriation, most of the published research that uses this method to articulate intermediate knowledge focuses on annotation strategies that abstract new knowledge about the qualities of interaction and about the design domain. We suggest that annotations can do more than pull towards concerns regarding abstraction and show that several not so very theoretical, but relevant findings can be achieved using other strategies. Two additional strategies are brought forward to illustrate this: a chronological design trajectory that shows the historical account of new domain explorations, and a design ecosystem strategy that aims to show how artefacts can work together. We apply all four strategies mentioned above and discuss how they contribute to revealing features of the design space for people with Myalgic Encephalomyelitis.

My Naturewatch Camera is an inexpensive wildlife camera that we designed for people to make themselves as a way of promoting engagement with nature and digital making. We aligned its development to the interests of the BBC's Natural History Unit as part of an orchestrated engagement strategy also involving our project website and outreach to social media. Since June 2018, when the BBC featured the camera on one of their Springwatch 2018 broadcasts, over 1000 My Naturewatch Cameras have been constructed using instructions and software from our project website and commercially available components, without direct contact with our studio. In this paper, we describe the project and outcomes with a focus on its success in promoting engagement with nature, engagement with digital making, and the effectiveness of this strategy for sharing research products outside traditional commercial channels.

TaskCams are simple digital cameras intended to serve as a tool for Cultural Probe studies and made available by the Interaction Research Studio via open-source distribution. In conjunction with an associated website, instructions and videos, they represent a novel strategy for disseminating and facilitating a research methodology. At the same time, they provide a myriad of options for customisation and modification, allowing researchers to adopt and adapt them to their needs. In the first part of this paper, the design team describes the rationale and design of the TaskCams and the tactics developed to make them publicly available. In the second part, the story is taken up by designers from the Everyday Design Studio, who assembled their own TaskCams and customised them extensively for a Cultural Probe study they ran for an ongoing project. Rather than discussing the results of their study, we focus on how their experiences reveal some of the issues both in producing and using open-source products such as these. These suggest the potential of TaskCams to support design-led user studies more generally.

In this paper we describe the Datacatcher, a location-aware, tangible and embodied mobile device that displays a continuous stream of statements about its location that are drawn from a large number of data sources and which speak to sociopolitical issues. We describe how the design and our underlying research interests emerged and changed over the course of three distinct phases of development: the device's conceptual design, its refinement to a final design, and the final detailing leading a batch production of 130 of the devices. We discuss the Datacatcher as resonant with many current issues in HCI, including augmented reality, environmental issues, political systems and using data as a design material.

This paper reports the results of a field trial of 130 bespoke devices as well as our methodological approach to the undertaking. Datacatchers are custom-built, location-aware devices that stream messages about the area they are in. Derived from a large number of 'big data' sources, the messages simultaneously draw attention to the socio-political topology of the lived environment and to the nature of big data itself. We used a service design consultancy to deploy the devices, and two teams of documentary filmmakers to capture participants' experiences. Here we discuss the development of this approach and how people responded to the Datacatchers as products, as revealing sociopolitical issues, and as purveyors of big data that might be open to question.

The Energy Babble is a kind of automated talk-radio that is obsessed with energy and the environment. We developed it with, and deployed it to, a number of existing "energy communities" in the UK. The system gathers content from a variety of online sources, including Twitter? feeds from the communities, from governmental departments, and from the National Grid, and chats about it continually using a number of synthesised voices interspersed with a variety of jingles and sound effects. Designed to playfully reflect and comment on the existing state of discourse and reports of practice in the UK, the Babble can be considered both as a product and as a research tool, in which role it worked to highlight issues, understandings, practices and difficulties in the communities with whom we worked.

The term 'critical design' is on the upswing in HCI. We analyze how discourses around 'critical design' are diverging in Design and HCI. We argue that this divergence undermines HCI's ability to learn from and appropriate the design approaches signaled by this term. Instead, we articulate two ways to broaden and deepen connections between Design and HCI: (1) develop a broader collective understanding of what these design approaches can be, without forcing them to be about 'criticality' or 'critical design,' narrowly construed; and (2) shape a variation of design criticism to better meet Design practices, terms, and ways of knowing.

In this project, we investigated how a ludic approach might open new possibilities for environmental HCI by designing three related devices that encourage environmental awareness while eschewing utilitarian or persuasive agendas. In addition, we extended our methodological approach by batch-producing multiple copies of each device and deploying them to 20 households for several months, gathering a range of accounts about how people engaged and used them. The devices, collectively called the 'Indoor Weather Stations', reveal the home's microclimate by highlighting small gusts of wind, the colour of ambient light, and temperature differentials within the home. We found that participants initially tended to relate to the devices in line with two 'orienting narratives' of environmental tools or ludic designs, finding the devices disappointing from either perspective. Most of our participants showed lingering affection for the devices, however, for a variety of reasons. We discuss the implications of this 'sporadic interaction', and the more general lessons from the project, both for environmental HCI and ludic design.

In this essay, I explore several facets of research through design in order to contribute to discussions about how the approach should develop. The essay has three parts. In the first, I review two influential theories from the Philosophy of Science to help reflect on the nature of design theory, concluding that research through design is likely to produce theories that are provisional, contingent, and aspirational. In the second part, I discuss three possible interpretations for the diversity of approaches to research through design, and suggest that this variation need not be seen as a sign of inadequate standards or a lack of cumulative progress in the field, but may be natural for a generative endeavour. In the final section, I suggest that, rather than aiming to develop increasingly comprehensive theories of design, practice based research might better view theory as annotation of realised design examples, and particularly portfolios of related pieces. Overall, I suggest that the design research community should be wary of impulses towards convergence and standardisation, and instead take pride in its aptitude for exploring and speculating, particularising and diversifying, and - especially - its ability to manifest the results in the form of new, conceptually rich artefacts.

In this paper, I discuss design workbooks, collections of design proposals and related materials, both as a method for design and as a design methodology. In considering them as a method, I describe a number of examples of design workbooks we have developed in our studio and describe some of the practical techniques we have used in developing them. More fundamentally, I discuss design workbooks as embodiments of a methodological approach which recognises that ideas may emerge slowly over time, that important issues and perspectives may emerge from multiple concrete ideas, potentially generated by multiple members of a team, rather than being theory-driven, and that maintaining the provisionality and vagueness of early proposals can be useful in supporting a quasi-participatory design approach that allows participants to interpret, react to and elaborate upon the ideas they present.

The Photostroller is a device designed for use by residents of a care home for older people. It shows a continuous slideshow of photographs retrieved from the Flickr™ image website using a set of six predefined categories modified by a tuneable degree of 'semantic drift'. In this paper, we describe the design process that led to the Photostroller, and summarise observations made during a deployment in the care home that has lasted over two months at the time of writing. We suggest that the Photostroller balances constraint with openness, and control with drift, to provide an effective resource for the ludic engagement of a diverse group of older people with each other and the world outside their home.

The "older old" (people over eighty) are a largely invisible group for those not directly involved in their lives; this project explores the ways that technology might strengthen links between different generations. This paper describes findings from a two-year study of a residential care home and develops the notion of cross-generational engagement through ludic systems which encourage curiosity and playfulness. It outlines innovative ways of engaging the older old through "digital curios" such as Bloom, the Tenori On and Google Earth. The use of these curios was supplemented with portraiture by three local artists, nine school children and the field researcher. The paper describes four technological interventions: "video window", "projected portraiture", "blank canvas", and "soundscape radio". These interventions attempt to reposition "off the shelf technologies to provide a space for cross-generational engagement The notion of inter-passivity (the obverse of interaction) is explored in relation to each intervention.

In this paper we describe the Prayer Companion, a device we developed as a resource for the spiritual activity of a group of cloistered nuns. The device displays a stream of information sourced from RSS news feeds and social networking sites to suggest possible topics for prayers. The nuns have engaged with the device enthusiastically over the first ten months of an ongoing deployment, and, notwithstanding some initial irritation with the balance of content, report that it plays a significant and continuing role in their prayer life. We discuss how we balanced specificity in the design with a degree of openness for interpretation to create a resource that the nuns could both understand and appropriate, describe the importance of materiality to the device's successful adoption, consider its implications as a design for older people, and reflect on the example it provides of how computation may serve spirituality.

In this paper, we describe the failure of a novel sensor-based system intended to evoke user interpretation and appropriation in domestic settings. We contrast participants' interactions in this case study with those observed during more successful deployments to identify 'symptoms of failure' under four themes: engagement, reference, accommodation, and surprise and insight. These themes provide a set of sensitivities or orientations that may complement traditional task-based approaches to evaluation as well as the more open-ended ones we describe here. Our system showed symptoms of failure under each of these themes. We examine the reasons for this at three levels: problems particular to the specific design hypothesis; problems relevant for input-output mapping more generally; and problems in the design process we used. We conclude by noting that, although interpretive systems such as the one we describe here may succeed in a myriad of different ways, it is reassuring to know that they can also fail, and fail incontrovertibly, yet instructively.

Threshold devices present information gathered from the home's surroundings to give new views on the domestic situation. We built two prototypes of different threshold devices and studied them in field trials with participant households. The Local Barometer displays online text and images related to the home's locality depending on the local wind conditions to give an impression of the sociocultural surroundings. The Plane Tracker tracks aircraft passing overhead and imagines their flights onscreen to resource an understanding of the home's global links. Our studies indicated that the experiences they provided were compelling, that participants could and did interpret the devices in various ways, that their form designs were appropriate for domestic environments, that using ready-made information contributed to the richness of the experiences, and that situating the information they provided with respect to the home and its locality was important for the ways people engaged with them.

Domestic ubiquitous computing systems often rely on inferences about activities in the home, but the open-ended, dynamic and heterogeneous nature of the home poses serious problems for such systems. In this paper, we propose that by shifting the responsibility for interpretation from the system to the user, we can build systems that interact with people at humanly meaningful levels, preserve privacy, and encourage engagement with suggested topics. We describe a system that embodies this hypothesis, using sensors and inferencing software to assess 'domestic wellbeing' and presenting the results to inhabitants through an output chosen for its ambiguity. In a three-month field study of the system, customised for a particular volunteer household, users engaged extensively with the system, discussing and challenging its outputs and responding to the particular topics it raised.

This article describes field trials of 3 electronic furniture prototypes designed to encourage ludic engagement in the home. The Drift Table shows slowly scrolling aerial photography controlled by the weight of the objects on its surface. The History Tablecloth creates slowly growing “halos” around things left on it. The Key Table measures the force with which people put things on it and tilts a picture frame to indicate their mood. The pieces were loaned to different households for periods of 1 to 3 months. Because they were designed for user appropriation, a hypothesis-testing paradigm is inappropriate for evaluating their success. The focus instead was on gathering rich, multilayered accounts of people's experience through ethnographic observations and documentary videos. The results helped assess the particular designs, draw lessons for ludic design more generally, and reflect on field methods for evaluating open-ended designs.

The History Tablecloth is a flexible substrate screen-printed with electroluminescent material forming a grid of lace-like elements. When objects are left on the table, cells beneath them light to form a halo that grows over a period of hours, highlighting the flow of objects in the home. The Tablecloth explores an approach to design that emphasises engaging, open-ended situations over defined utilitarian purposes. Long-term deployment of the History Tablecloth in a volunteer household revealed complex ways that people experienced and interacted with the Tablecloth. Beyond evoking reflection on the flow of objects over a particular table, the Tablecloth served as a ground for interpretative reflection about technology, an asset for social interaction, and an aesthetic object. Even behaviours we saw as system errors were interpreted by the users as interactively rich. Their experience highlights the subtlety of domestic ubiquitous computing, illustrating alternatives to traditional views of technology's domestic role.

This paper shows how we can take inspiration and use techniques from documentary film in pursuing user research. Documentary filmmaking has a long history of portraying everyday life in ways that leave the erratic, elusive fabric of the everyday intact. This may be valuable as interaction design currently embraces issues of engagement, expression and aesthetics. We discuss key documentary formats, and suggest that a purely observational approach may not be most valuable for design research. Three design documentaries are discussed to show how different documentary approaches can be used in practice to inform the early stages of design. We suggest that, for design research in HCI, film can be much more than a note-taking tool; we can use it as a means to explore, understand and present the everyday, and benefit from film's capabilities to preserve ambiguities and paradoxes instead of resolving them into univocal conclusions.

Human-Computer Interaction (HCI) often focuses on how designers can develop systems that convey a single, specific, clear interpretation of what they are for and how they should be used and experienced. New domains such as domestic and public environments, new influences from the arts and humanities, and new techniques in HCI itself are converging to suggest that multiple, potentially competing interpretations can fruitfully co-exist. In this paper, we lay out the contours of the new space opened by a focus on multiple interpretations, which may more fully address the complexity, dynamics and interplay of user, system, and designer interpretation. We document how design and evaluation strategies shift when we abandon the presumption that a specific, authoritative interpretation of the systems we build is necessary, possible or desirable.

The new usability agenda is driving empirical and experimental studies into a growing range of quality criteria such as engagement, user experience, and aesthetics. Some see this as a positive move to theorise about the nature of good design qualities, and to objectively test such hypotheses on the new usability theme. However, others (e.g. [1], [2]) have argued for interpretation-based inquiry into user engagement and experience on the grounds that such phenomena can only be understood by investigations into contexts of use which defy quantitative approaches. Many in the design community would agree with them and go further to argue that quality in design is a matter of creativity and can not be measured or theorised per se; instead, research should focus on understanding and improving the process of design. This panel will debate the tensions between these positions and explore possible common ground between them as a contribution towards the research agenda that is being debated in the DIS conference series.

Movements of interfaces can be analyzed in terms of whether they are expected, sensed, and desired. Expected movements are those that users naturally perform; sensed are those that can be measured by a computer; and desired movements are those that are required by a given application. We show how a systematic comparison of expected, sensed, and desired movements, especially with regard to how they do not precisely overlap, can reveal potential problems with an interface and also inspire new features. We describe how this approach has been applied to the design of three interfaces: pointing flashlights at walls and posters in order to play sounds; the Augurscope II, a mobile augmented reality interface for outdoors; and the Drift Table, an item of furniture that uses load sensing to control the display of aerial photographs. We propose that this approach can help to build a bridge between the analytic and inspirational approaches to design and can help designers meet the challenges raised by a diversification of sensing technologies and interface forms, increased mobility, and an emerging focus on technologies for everyday life.

Ambiguity is usually considered anathema in Human Computer Interaction. We argue, in contrast, that it is a resource for design that can be used to encourage close personal engagement with systems. We illustrate this with examples from contemporary arts and design practice, and distinguish three broad classes of ambiguity according to where uncertainty is located in the interpretative relationship linking person and artefact. Ambiguity of information finds its source in the artefact itself, ambiguity of context in the sociocultural discourses that are used to interpret it, and ambiguity of relationship in the interpretative and evaluative stance of the individual. For each of these categories, we describe tactics for emphasising ambiguity that may help designers and other practitioners understand and craft its use.

As a way of mapping a design space for a project on information appliances, we produced a workbook describing about twenty conceptual design proposals. On the one hand, they serve as suggestions that digital devices might embody values apart from those traditionally associated with functionality and usefulness. On the other, they are examples of research through design, balancing concreteness with openness to spur the imagination, and using multiplicity to allow the emergence of a new design space. Here we describe them both in terms of content and process, discussing first the values they address and then how they were crafted to encourage a broad discussion with our partners that could inform future stages of design.

As a part of a European Union sponsored project, we have proposed a system which aggregates peoples expressions over a widening network of public electronic displays in a massive Dutch housing development. Reflecting ideas from contemporary arts as well as from research on media spaces, this is an example of a conceptual design intended to produce meaningful effects on a local culture. In this paper, we describe the methods and ideas that led to this proposal, as an example of research on technologies from the traditions of artist-designers.

The Virtual Window system uses iiead movements in a local office to control camera movement in a remote office.The result is like a window allowing exploration of remote scenes rather than a flat screen showing moving pictures.Our analysis of the system, experience implementing a prototype, and observations of people using it, combine to suggest that it may help overcome the limitations of typical media space configurations.In particular, it seems useful in offering an expanded field of view, reducing visual discontinuities, allowing mutual negotiation of orientation, providing depth information, and supporting camera awareness.The prototype we built is too large, noisy, slow and inaccurate for extended use, but it is valuable in opening a space of possibilities for the design of systems that allow richer access to remote colleagues.

This panel addresses issues in designing audio-based user interactions for small, personal computing devices, or PDAs. One issue is the nature of interacting with an auditory PDA and the interplay of affordances and form factors. Another issue is how both new and traditional metaphors and interaction concepts might be applied to auditory PDAs. The utility and design of nonspeech cues are discussed, as are the aesthetic issues of persona and narrative in designing sounds. Also discussed are commercially available sound and speech components and related hardware tradeoffs. Finally, the social implications of auditory interactions are explored, including privacy, fashion and novel social interactions.

The design of richly informative interfaces would benefit from an account of how visual forms convey information.In this paper we suggest that the study of~orwz-giving in Industrial Engineering might provide a foundation for such an account.We present three studies of designed synesthesia, in which objects' forms indicate non-visible attributes such as taste or smell.These studies illustrate the rich possibilities for conveying information with form, possibilities which are routinely exploited in industrial design.We believe that similar opportunities exist for interface design, and that further studies of form-giving may help in taking advantage of them.Results of a student exercise expressing computer metaphors in 3D forms will be discussed.

ENO is an audio server designed to make it easy for applications in the Unix environment to incorporate non-speech audio cues. At the physical level, ENO manages a shared resource, namely the audio hardware. At the logical level, it manages a sound space that is shared by various client applications. Instead of dealing with sound in terms of its physical description (i.e., sampled sounds), ENO allows sounds to be presented and controlled in terms of higher-level descriptions of sources, interactions, attributes, and sound space. Using this structure, ENO can facilitate the creation of consistent, rich systems of audio cues. In this paper, we discuss the justification, design, and implementation of ENO.

At EuroPARC, we have been exploring ways to allow physically separated colleagues to work together effectively and naturally. In this paper, we briefly discuss several examples of our work in the context of three themes that have emerged: the need to support the full range of shared work; the desire to ensure privacy without giving up unobtrusive awareness; and the possibility of creating systems which blur the boundaries between people, technologies and the everyday world.

Ecological approaches to psychology suggest succinct accounts of easily-used artifacts. Affordances are properties of the world that are compatible with and relevant for people's interactions. When affordances are perceptible, they offer a direct link between perception and action; hidden and false affordances lead to mistakes. Complex actions can be understood in terms of groups of affordances that are sequential in time or nested in space, and in terms of the abilities of different media to reveal them. I illustrate this discussion with several examples of interface techniques, and suggest that the concept of affordances can provide a useful tool for user-centered analyses of technologies. (http://doi.acm.org/10.1145/108844.108856)

We designed an ecology of auditory icons which worked together to convey information about a complex, demanding simulation task, and observed users collaborating on it with and without sound.Our observations suggest that audio cues can provide useful information about processes and problems, and support the perceptual integration of a number of separate processes into one complex one.In addition, they can smooth the transition between division of labour and collaboration by providing a new dimension of reference.These results suggest that auditory icons can play a significant role in future multiprocessing and collaborative systems.

The appropriate use of nonspeech sounds has the potential to add a great deal to the functionality of computer interfaces. Sound is a largely unexploited medium of output, even though it plays an integral role in our everyday encounters with the world, a role that is complementary to vision. Sound should be used in computers as it is in the world, where it conveys information about the nature of sound-producing events. Such a strategy leads to auditory icons, which are everyday sounds meant to convey information about computer events by analogy with everyday events. Auditory icons are an intuitively accessible way to use sound to provide multidimensional, organized information to users. These ideas are instantiated in the SonicFinder, which is an auditory interface I developed at Apple Computer, Inc. In this interface, information is conveyed using auditory icons as well as standard graphical feedback. I discuss how events are mapped to auditory icons in the SonicFinder, and illustrate how sound is used by describing a typical interaction with this interface. Two major gains are associated with using sound in this interface: an increase in direct engagement with the model world of the computer and an added flexibility for users in getting information about that world. These advantages seem to be due to the iconic nature of the mappings used between sound and the information it is to convey. I discuss sound effects and source metaphors as methods of extending auditory icons beyond the limitations implied by literal mappings, and I speculate on future directions for such interfaces.

There is growing interest in the use of sound to convey information in computer interfaces. The strategies employed thus far have been based on an understanding of sound that leads to either an arbitrary or metaphorical relation between the sounds used and the data to be represented. In this article, an alternative approach to the use of sound in computer interfaces is outlined, one that emphasizes the role of sound in conveying information about the world to the listener. According to this approach, auditory icons, caricatures of naturally occurring sounds, could be used to provide information about sources of data. Auditory icons provide a natural way to represent dimensional data as well as conceptual objects in a computer system. They allow categorization of data into distinct families, using a single sound. Perhaps the most important advantage of this strategy is that it is based on the way people listen to the world in their everyday lives.