Despite a long tradition of ‘non-expert’ participation in ecoacoustics research, asymmetrical distribution of resources and engagement between the Global North and Global South continue, extending to ecoacoustic sensing and design. Whilst there exists a growing body of work in Participatory Design (PD) addressing the technical and social challenges of ecoacoustic research, we find that popular PD methods inadequately address design justice and decolonising agendas. Through participatory ecoacoustic sensing and design engagements with a forest community in Ghana, we highlight the tensions that emerge when employing visual and written modes of PD in a context where an oral approach to creativity and communication is more appropriate. We present Justice-oriented Design Listening, an acoustically-mediated approach to PD, described through three modes: polyphony, pace and transformation. This work contributes to calls for design justice by developing a methodological approach that facilitates pluralistic participation in design when developing conservation technologies in non-Western contexts.

We present Multiverse Explorer, a domain-specific probabilistic programming language presented as a visual language integrated with a domain world model. The interactive visualisation presents a Monte Carlo simulation over a causal graph, allowing the user to gain an overview and query alternative outcomes in a counterfactual manner. Separate graphs express the policies attributed to multiple heterogeneous agents. The outcomes of actions are visualised in an interactive 3D animation of the environment; in this work, we apply the Multiverse Explorer to multi-agent driving scenarios by extending the CARLA simulator. The Multiverse Explorer has been evaluated with a sample of technical non-specialists, demonstrating the potential of this approach to be used in design, audit, policy, litigation, and other contexts where the outcome of multi-agent decision scenarios must be investigated by professionals beyond a specialist AI audience.

Machine learning algorithms often need to be trained with sample datasets, prepared by humans who label the content manually. However, data labeling is tedious and repetitive, with humans often rep...

Across the world, policy initiatives are being developed to engage children with computer programming and computational thinking. Diversity and inclusion has been a strong force in this agenda, but children with disabilities have largely been omitted from the conversation. Currently, there are no age appropriate tools for teaching programming concepts and computational thinking to primary school children with visual disabilities. We address this gap through presenting the design and implementation of Torino, a tangible programming language for teaching programming concepts to children age 7–11 regardless of level of vision. In this paper, we: (1) describe the design process done in conjunction with children with visual disabilities; (2) articulate the design decisions made; and (3) report insights generated from an evaluation with 10 children with mixed visual abilities that considers how children are able to trace (read) and create (write) programs with Torino. We discuss key design trade-offs: (1) readability versus extensibility; and (2) size versus liveness. We conclude by reflecting upon how an inclusive design approach shaped the final result.

Abstract An unfortunate tendency in previous HCI research has been to give the impression that it aims to ‘fix the problem’ of human ageing, suggesting a ‘deficit’ model of ageing or a ‘prosthetic’ model of technology. We conducted diary-aided interviews to investigate how technology use is situated in active, healthy older adults’ meaningful participation in community music. We argue that recognizing community music practices and technology use as situated action provides opportunities to grasp the subtleties of social participation and design for active ageing. We identified technology-mediated music practices, such as music sharing and revisiting, and how they evolved through the reconfiguration of connections between technology, competence, and forward-facing identities. We found that identity development, via routes such as exercising control, role transitions, and social spaces, had psychological significance and implications for active ageing. We explore how HCI leverages the perspective of active ageing and might facilitate older adults’ meaningful participation enhanced by technologies.

We ask how users interact with `knowledge' in the context of artificial intelligence systems. Four examples of visual interfaces demonstrate the need for such systems to allow room for negotiation between domain experts, automated statistical models, and the people who are involved in collecting and providing data.

My 1996 paper [1] challenged the VL community to ask What do we think we are doing? It might now be called a Systematic Literature Review, although formal procedures for SLR were not developed until later [5]. It made a textual analysis of publications in which authors described a cognitive rationale for VL research, observing that many relied on insights from folk psychology, from introspection, or speculative computer analogies to the brain. This was a study of metacognition - beliefs about one's cognitive ability that shape the mental strategies we choose. In the case of programming language designers, the choices being shaped were not their own problem-solving strategies (something we all do), but the design rationale for new languages (which will affect others). The paper was written in the first year of my PhD, which should be encouraging to students wondering what will happen to their own work in 20 years time. It was inspired by research I had been doing for Hitachi, in a team designing a new VL. Sadly `AppGallery' was not a commercial success. More interestingly, my own scientific assumptions turned out to be wrong - as reported in later VL papers [2], in my PhD dissertation, and in a ToCHI publication where I tried to reconstruct why we had believed it was a good idea in the first place [3]. At VL'96, scientific attention to human factors was a minority interest, and few user studies were reported. I remember that Judy Gurka seemed to ask the right questions about testing effectiveness of algorithm animation [4]. And the two keynote speakers - Thomas Green and Ben Shneiderman - demonstrated growing engagement of VL with HCI. However, within a few years, the whole VL community became convinced that these were the central questions in our work - first with a radical change of name to Human-Centric Computing, and then to the integration of VL and HCC One might assume it is easy to recognize influential papers. But this paper was controversial. It didn't include any technical work, let alone report user studies. It was a textual critique, but written by an engineer who simply needed better information about users, and had set out to ask what guidance was available for the design of VLs. Yet although the motivation came from engineering, the paper didn't make an engineering contribution, so people wondered whether some other publication venue would be more appropriate. When aiming to demonstrate conference quality through low acceptance ratios, it is often easier to reject a discussion paper like this one in favour of research that offers simpler conclusions, hard evidence and scientific consensus. This question of whether papers at a conference like VL should only present scientific or engineering results continues to be active 20 years later. In 1996, VL had been proceeding on the assumption that the primary focus of the conference should be on engineering work. The fact that engineering work relied on hidden psychological assumptions had not been noticed. However, bringing in more insights from human-facing fields such as HCI, and broadening the base of disciplinary enquiry, did help to overturn fallacies that had been acquired by holding to a strictly engineering mindset. We continue to face similar problems today. HCI has changed greatly over the past 20 years. In particular, HCI researchers now understand that hypothesis-testing experiments are not sufficient for interaction design: different kinds of empirical data must be acquired and integrated into explanatory and predictive theories of human behavior. But there are still advocates for approaches to programming language research that reject theories of design, relying on common sense and experimental measurements. It is hard for young researchers to argue for better theory in the face of demands for engineering evidence, so I hope that this award is a reminder that we still need to have informed debate about what we think we are doing.

As computing technologies become ubiquitous in social life, social science increasingly becomes the study of those technosystems. Similarly, as technology corporations compete to design new ubicomp products, social science research is recruited as a design method. This paper presents an interpretive bricolage, exploring the relationship between social science and corporate technology research. It draws on a specific case study: the history of ethnomethodology during the lifetime of the Xerox EuroPARC laboratory. This interpretation relates to the ways that social and organisational authority is maintained through scientific knowledge, and the ways in which interdisciplinary design research engages with product users, customers and markets.

We present a live, multiple-representation novice environment for probabilistic programming based on the Infer.NET language. When compared to a text-only editor in a controlled experiment on 16 participants, our system showed a significant reduction in keystrokes during introductory probabilistic programming exercises, and subsequently, a significant improvement in program description and debugging tasks as measured by task time, keystrokes and deletions.

How diverse are the ways that programming is done? While a variety of accounts exist, each appears in isolation, neither framed in terms of a distinct practice, nor as one of many such practices. In this work we explore accounts spanning software engineering, bricolage/tinkering, sketching, live coding, code-bending, and hacking. These practices of programming are analyzed in relation to ongoing research, and in particular HCI's `practice turn', offering connections to accounts of practice in other contexts than programming. The conceptualization of practice helps to interpret recent interest in program code as craft material, and also offers potential to inform programming education, tools and work as well as future research.

Data wrangling is the term used by data scientists for the work of re-organising data into a new structure, before work starts on reporting or analysis. We present a prototype that applies programming by example methods to data wrangling in spreadsheets. The Data Noodles system guides the user through constructing a simple example that illustrates how they would like their spreadsheet to look. A transformation program is then synthesised and executed to produce the final reshaped spreadsheet.

Gatherminer is an interactive visual tool for analysing time series data with two key strengths. First, it facilitates bottom-up analysis, i.e., the detection of trends and patterns whose shapes are not known beforehand. Second, it integrates data mining algorithms to explain such patterns in terms of the time series' metadata attributes - an extremely difficult task if the space of attribute-value combinations is large. To accomplish these aims, Gatherminer automatically rearranges the data to visually expose patterns and clusters, whereupon users can select those groups they deem `interesting.' To explain the selected patterns, the visualisation is tightly coupled with automated classification techniques, such as decision tree learning. We present a brief evaluation with telecommunications experts comparing our tool against their current commercial solution, and conclude that Gatherminer significantly improves both the completeness of analyses as well as analysts' confidence therein.

The application of mobile computing is currently altering patterns of our behavior to a greater degree than perhaps any other invention. In combination with the introduction of power-efficient wireless communication technologies, such as Bluetooth Low Energy (BLE), designers are today increasingly empowered to shape the way we interact with our physical surroundings and thus build entirely new experiences. However, our evaluations of BLE and its abilities to facilitate mobile location-based experiences in public environments revealed a number of potential problems. Most notably, the position and orientation of the user in combination with various environmental factors, such as crowds of people traversing the space, were found to cause major fluctuations of the received BLE signal strength. These issues are rendering a seamless functioning of any location-based application practically impossible. Instead of achieving seamlessness by eliminating these technical issues, we thus choose to advocate the use of a seamful approach, that is, to reveal and exploit these problems and turn them into a part of the actual experience. In order to demonstrate the viability of this approach, we designed, implemented, and evaluated the Ghost Detector —an educational location-based museum game for children. By presenting a qualitative evaluation of this game and by motivating our design decisions, this article provides insight into some of the challenges and possible solutions connected to the process of developing location-based BLE-enabled experiences for public cultural spaces.

We present "FingAR Puppet", an Augmented Reality (AR) system enhancing social pretend play by young children. Unlike goal-oriented AR systems that augment reality with informative instructions, FingAR Puppet helps children associate expressive interpretations with immediate reality. Empirical results show that FingAR Puppet promotes reasoning about emotional states, communication and divergent thinking during social pretend play for children 4-6 years old. We suggest that this study opens an interesting space for future AR systems to support complex cognitive and social development in early childhood. We also identify broader implications from using theories of cognitive development to guide the design of tangible and augmented interactions.

We explore the factors that determine whether individuals are likely to experience intrinsic motivation in end-user programming (EUP). We report two experiments: one that tests whether there are reliable psychometric constructs that describe different aspects of intrinsic motivation, and one that tests whether these constructs are successful in predicting individuals' own self-reported intrinsic motivation after using a popular EUP product. We conclude that there are identifiable and distinct motivational factors in EUP, and that these are associated with particular psychometric personality traits. We offer several suggestions for future research that could apply these findings to improve uptake and quality of user experience for educational and general-purpose EUP applications.

BrainCel is an interactive visual system for performing general-purpose machine learning in spreadsheets, building on end-user programming and interactive machine learning. BrainCel features multiple coordinated views of the model being built, explaining its current confidence in predictions as well as its coverage of the input domain, thus helping the user to evolve the model and select training examples. Through a study investigating users' learning barriers while building models using BrainCel, we found that our approach successfully complements the Teach and Try system [1] to facilitate more complex modelling activities.

Children with autism spectrum condition (ASC) suffer from deficits or developmental delays in symbolic thinking. In particular, they are often found lacking in pretend play during early childhood. Researchers believe that they encounter difficulty in generating and maintaining mental representation of pretense coupled with the immediate reality. We have developed an interactive system that explores the potential of Augmented Reality (AR) technology to visually conceptualize the representation of pretense within an open-ended play environment. Results from an empirical study involving children with ASC aged 4 to 7 demonstrated a significant improvement of pretend play in terms of frequency, duration and relevance using the AR system in comparison to a non computer-assisted situation. We investigated individual differences, skill transfer, system usability and limitations of the proposed AR system. We discuss design guidelines for future AR systems for children with ASC and other pervasive developmental disorders.

The modern economy increasingly relies on exploratory data analysis. Much of this is dependent on data scientists - expert statisticians who process data using statistical tools and programming languages. Our goal is to offer some of this analytical power to end-users who have no statistical training through simple interaction techniques and metaphors. We describe a spreadsheet-based interaction technique that can be used to build and apply sophisticated statistical models such as neural networks, decision trees, support vector machines and linear regression. We present the results of an experiment demonstrating that our prototype can be understood and successfully applied by users having no professional training in statistics or computing, and that the experience of interacting with the system leads them to acquire some understanding of the concepts underlying exploratory statistical modelling.

There are two alternative display metaphors for Augmented Reality (AR) screens: a see-through window or a magic mirror. Commonly used by task-support AR applications, the see-through display has not been compared with the mirror display in terms of user's task performance, even though the “mirror” hardware is more accessible to general users. We conducted a novel experiment to compare participants' performance when following object rotation cues with the two display metaphors. Results show that participants' overall performance under the mirror view was comparable to the see-through view, which indicates that the augmented mirror display may be a promising alternative to the window display for AR applications which guide moderately complex three-dimensional manipulations with physical objects.

Lack of spontaneous pretend play is an early diagnostic indicator of autism spectrum conditions (ASC) along with impaired communication and social interaction. In a previous ISMAR poster [2] we proposed an Augmented Reality (AR) system to encourage pretend play, based on an analogy between imaginative interpretation of physical objects (pretense) and the superimposition of virtual content on the physical world in AR. This paper reports an empirical experiment evaluating that proposal, involving children between the ages of 4 and 7 who have been diagnosed with ASC. Results find significantly more pretend play, and higher engagement, using the AR system by comparison to a non-augmented condition. We also discuss usability issues and design implications for AR systems that aim to support children with ASC and other pervasive developmental disorders.

Cognitive neuroscience defines the sense of agency as the experience of controlling one's own actions and, through this control, affecting the external world. We believe that the sense of personal agency is a key factor in how people experience interactions with technology. This paper draws on theoretical perspectives in cognitive neuroscience and describes two implicit methods through which personal agency can be empirically investigated. We report two experiments applying these methods to HCI problems. One shows that a new input modality - skin-based interaction - can substantially increase users' sense of agency. The second demonstrates that variations in the parameters of assistance techniques such as predictive mouse acceleration can have a significant impact on users' sense of agency. The methods presented provide designers with new ways of evaluating and refining empowering interaction techniques and interfaces, in which users experience an instinctive sense of control and ownership over their actions.

Children with autism are often found to lack facility for pretend play. It is believed that this deficit is linked to linguistic, social and creativity competencies in autism. We observe that both Augmented Reality (AR) and pretend play involve processing of information that is coupled with real scenes while not necessarily being directly perceived. This research therefore examines the potential of using AR technologies to promote pretend play behaviors in children with autism. As an initial outcome, we present the design and implementation of an AR system that aims to enhance the comprehension and flexibility of object substitution during pretend play.

End-user programmers have experiences of computation that are different from those of professional software engineers or people trained in computer science. We now understand that these different experiences arise from different motivations, different approaches to the task of programming, and different expectations about the outcomes. This talk reports results from a 10 year programme of research, engaging with collaborators who have been sought out precisely because their motivations, work methods and intentions are as different as possible from typical engineering programmers. These investigations have involved the development of a wide range of novel programmable systems and notations designed for use by choreographers, composers, musicians, and visual artists. The findings extend previous analyses of computational experience such as the attention investment model of abstraction use, and the cognitive dimensions of notations, drawing attention to ways of programming that are not only utilitarian, but playful and engaging. Research of this kind also helps us to understand that there are not necessarily different kinds of people in the world - those who are creative or not - but rather different ways of engaging with technology, available to all designers painting from the palette of human-centric computing systems.

There has been a rapid increase in research evaluating usability of Augmented Reality (AR) systems in recent years. Although many different styles of evaluation are used, there is no clear consensus on the most relevant approaches. We report a review of papers published in International Symposium of Mixed and Augmented Reality (ISMAR) proceedings in the past decade, building on the previous work of Swan and Gabbard (2005). Firstly, we investigate the evaluation goal, measurement and method of ISMAR papers according to their usability research in four categories: performance, perception and cognition, collaboration and User Experience (UX). Secondly, we consider the balance of evaluation approaches with regard to empirical–analytical, quantitative–qualitative and participant demographics. Finally we identify potential emphases for usability study of AR systems in the future. These analyses provide a reference point for current evaluation techniques, trends and challenges, which benefit researchers intending to design, conduct and interpret usability evaluations for future AR systems.

Lucy A. Suchman. Human–Machine Reconfigurations: Plans and Situated Actions, 2nd ed.Cambridge, U.K.: Cambridge University Press, 2006. ISBN-10 052167588X, ISBN-13 978-0521675888. 326 pages. - Volume 17 Issue 1

The majority of Tangible User Interfaces (TUIs) consist of rigid objects that are either held in the hands, or arranged relative to each other on a horizontal or vertical surface. In this paper we consider the design space of TUIs that can be created by moving parts of an assembly relative to each other -- creating articulated tangible interfaces. An analytic approach to this design space allows us to identify the potential applications and trade-offs of TUIs that include mechanically articulated parts.

Tangible User Interfaces (TUIs) are commonly accepted as those in which the configuration of physical objects embodies digital system state, providing "graspable" digital media that can be manipulated in the focus of users' attention. In this paper we offer an alternative perspective on the use of tangibility in interaction, in which meaning is created not through precise manipulations of a computationally-interpreted spatial syntax, but through imprecise interactions with independently meaningful, digitally-augmented physical tokens. Users are free to arrange such tokens around the periphery of their workspace, away from their normal centre of attention, ready to selectively and fluidly engage them in loosely related, dispersed episodes of use. We call this concept "peripheral tangible interaction", and in this paper we describe both our analytic approach to designing a personal desktop TUI supporting such an interaction style, and user responses to its analytically-inspired features during extended deployment in a real office context.

In the context of tangibility, mobile phones are rapidly becoming sensor-rich handheld computers with the potential to take better advantage of our physical capabilities and our lifetime of experiences interacting both in and with the world around us. In this paper, we analyse four different ways in which mobiles can be used to represent and control digital information, showing that each resulting interaction style is characterized by a unique coordination of the user's attention and two hands in relation to the mobile device. We present our analysis in terms of a framework that can be used to critically examine future schemes of bimanual interaction with mobile phones.

Children who are active on the Internet are performing significant design and programming activity without realising it, in the course of hacking little animations, game scripts and so on. What does such effortless learning suggest about how to support end-user programming? This paper presents observations of 'unwitting' design and programming activity by a small group of teenagers, aged 12-17. It analyses their adoption and appropriation of technology, and discusses how such practices are embedded in social networks.

This article is concerned with the problem of improving software products and investigates how to base that process on solid empirical foundations. Our key contribution is a contextual method that provides a means of identifying new features to support discovered and currently unsupported ways of working and a means of evaluating the usefulness of proposed features. Standard methods of discovery and evaluation, such as interviews and usability testing, gather some of the necessary data but fall short of covering important aspects. The shortcomings of these approaches are overcome by applying an integrated and iterative method for collecting and interpreting data about product usage in context. This article demonstrates its effectiveness when applied to the discovery and evaluation of new features for standard Web clients.

Earlier research on gender effects with software features intended to help problem-solvers in end-user debugging environments has shown that females are less likely to use unfamiliar software features. This poses a serious problem because these features may be key to helping them with debugging problems. Contrasting this with research documenting males' inclination for tinkering in unfamiliar environments, the question arises as to whether encouraging tinkering with new features would help females overcome the factors, such as low self-efficacy, that led to the earlier results. In this paper, we present an experiment with males and females in an end-user debugging setting, and investigate how tinkering behavior impacts several measures of their debugging success. Our results show that the factors of tinkering, reflection, and self-efficacy, can combine in multiple ways to impact debugging effectiveness differently for males than for females.

Despite causing many debates in human-computer interaction (HCI), the term “metaphor” remains a central element of design practice. This article investigates the history of ideas behind user-interface (UI) metaphor, not only technical developments, but also less familiar perspectives from education, philosophy, and the sociology of science. The historical analysis is complemented by a study of attitudes toward metaphor among HCI researchers 30 years later. Working from these two streams of evidence, we find new insights into the way that theories in HCI are related to interface design, and offer recommendations regarding approaches to future UI design research.

Summary form only given on an article discussing cognitive dimensions of notations.

We describe Webkit, an application which uses a large-screen graphical user interface and a tangible user interface to teach children important rhetorical skills. We discuss our evaluation of this application and possible future directions for computer-supported rhetorical applications.

Although a variety of evaluation techniques are available to researchers of visual and end-user programming systems, they are primarily suited to evaluation of research systems. It is important to have evaluation techniques suitable for real-world programming environments, in order to satisfy real-world product managers of the usefulness of proposed new features. To help fill this gap, we present a new evaluation technique, based in part on Cognitive Dimensions and Attention Investment, called "Champagne Prototyping". The technique is an early-evaluation technique that is inexpensive to do, yet features the credibility that comes from being based on the real commercial environment of interest, and from working with real users of the environment.

This paper presents the design and user evaluation of SmartBack, a feature that complements the standard Back button by enabling users to jump directly to key pages in their navigation session, making common navigation activities more efficient. Defining key pages was informed by the findings of a user study that involved detailed monitoring of Web usage and analysis of Web browsing in terms of navigation trails. The pages accessible through SmartBack are determined automatically based on the structure of the user's navigation trails or page association with specific user's activities, such as search or browsing bookmarked sites. We discuss implementation decisions and present results of a usability study in which we deployed the SmartBack prototype and monitored usage for a month in both corporate and home settings. The results show that the feature brings qualitative improvement to the browsing experience of individuals who use it.

In this paper we describe a technique of Curriculum-Focused Design, and the aspects of our research experience on which the technique is based. Our technique is a variant of Druin's Cooperative Inquiry. Cooperative Inquiry is a well-developed design practice for children, but it has been practised largely outside the classroom. Druin's technique has also been developed in American schools, which have greater curriculum flexibility than English schools, which are highly curriculum-focused. We studied the English curriculum and identified an area that we believed could fruitfully be augmented by technology. Our design approach was novel insofar as our evaluation sessions doubled as lessons for students. Our interdisciplinary design team, including a former teacher with over 10 years' classroom experience, evaluated the interface in a classroom setting, providing strong environmental validity to the design process.

We describe extensions to the Excel spreadsheet that integrate userdefined functions into the spreadsheet grid, rather than treating them as a "bolt-on". Our first objective was to bring the benefits of additional programming language features to a system that is often not recognised as a programming language. Second, in a project involving the evolution of a well-established language, compatibility with previous versions is a major issue, and maintaining this compatibility was our second objective. Third and most important, the commercial success of spreadsheets is largely due to the fact that many people find them more usable than programming languages for programming-like tasks. Thus, our third objective (with resulting constraints) was to maintain this usability advantage.

Existing devices for communicating information to computers are bulky, slow to use, or unreliable. Dasher is a new interface incorporating language modelling and driven by continuous two-dimensional gestures, e.g. a mouse, touchscreen, or eye-tracker. Tests have shown that this device can be used to enter text at around 25 words per minute, compared with typical ten-finger keyboard typing of 40--60 words per minute. Although the interface is slower than a conventional keyboard, it is small and simple, and could be used on personal data assistants and by handicapped computer users. KEYWORDS: Adaptive, Text, Entry, Language, Modelling 1 INTRODUCTION A conventional Scholes (qwerty) keyboard is often inconvenient; the interaction device may be too small or too big [8], or may have to be operated with one hand because it is wristmounted or hand-held [17]. There are many competing methods of text entry for handheld devices, but none is clearly superior. This is largely because text entry...