The fertility industry has rapidly adopted AI as both a clinical and marketing tool. However, the biases embedded within these systems remain largely invisible to the people making intimate decisions based on the outputs of the AI models. Baby Lottery is an interactive demonstration that invites conference attendees to generate AI images of a hypothetical baby by combining their own photograph with a randomly selected AI-generated donor profile created from a dataset of over 1,000 sperm donor profiles collected from U.S. and European sperm banks. The results are printed on a thermal receipt printer, producing an immediate physical artifact. By making the generation process participatory and its outputs tangible, the demonstration invites critical examination of how AI systems encode bias in deeply personal contexts. This work extends the investigation of AI bias propagation in fertility marketing documented in our recent work, and engages attendees as active subjects in the ongoing inquiry.

Musicians are using neural audio models to synthesize new sounds, but the sound space accessible through such models is often constrained to essentially interpolated approximations of existing training data, limiting the true sonic novelty musicians can elicit from generative models. To address this problem, we present Latent Novelification (LN), a machine learning method which learns to transform vectors in any generative model’s latent space into new, contrasting vectors according to an objective function which we define. The LN objective enforces that these contrasting vectors explicitly diverge from existing training data while remaining semantically meaningful to the generative model, facilitating generative novelty beyond simple interpolation. LN supports divergent creativity by helping musicians generate new and novel sounds, integrating into any neural audio model interface to controllably and interactively expand accessible sound space. This work contributes a new algorithmic method—implemented in open-source software—which musicians working with neural audio synthesis can use to synthesize new sounds to incorporate into their art.

Sound offers unique affordances for interaction designers, enabling immediate, embodied, and expressive experiences. Yet sound is often treated as peripheral, limited to simple notifications rather than being explored as a central design material. Designing meaningful auditory feedback typically requires low-level programming, audio-specific expertise, and fragmented toolchains that hinder early-stage experimentation. We present TESI (Tangible and Embodied Sound Interaction), a modular toolkit that reduces these barriers by combining sensor-integrated hardware, open-source middleware, and a visual programming environment for real-time sensor-to-sound mapping. TESI enables interaction designers to prototype adaptive soundscapes across networked tangible interfaces with minimal setup. We describe the toolkit’s architecture, illustrate its use through examples, and discuss future directions for embodied sound design in distributed systems.

This paper describes a workshop conducted as part of practice-based research that aims at critiquing online tracking algorithms commonly found in everyday web environments. The workshop introduced participants to online tracking algorithms using a series of choreographic exercises that informed a discussion on the topic and strategies to counteract data-driven extractivist technologies. We analysed the outcomes of our workshop and showed that it allowed individuals to become more aware of their lack of agency over data harvesting and its use by digital services, and enabled them to develop strategies for reclaiming agency over their personal data. We discuss how the choreographic approach used in the workshop contributes to engaging people in a critical examination of online tracking in their everyday lives and to inspire forms of countering extractive algorithmic systems. Our paper contributes empirical insights on how choreography can be used to raise awareness of data tracking online.

Systems to interactively generate audio from human movement are used by artists including dancers to support their performances and practice. However, current real-time movement-to-sound systems require specialized hardware or expertise, or map only very simple movement-to-audio relationships. We present a new technique and system implementation for interactive sonification of human movement through unsupervised machine learning. Our system maps between latent spaces, linking a pose estimator to a neural audio generator to enable sonification of human bodies. This may lower barriers to entry for artists to generate sound from their embodied movement through complex mappings. Our system requires no specialized hardware or niche AI expertise, minimal data to learn a user's custom movements, and trains extremely fast. It represents a new method for mapping custom data to a latent space through unsupervised learning, and advances state-of-the-art interactive movement sonification through its increased accessibility and ease of use relative to its complexity.

Human-Computer Counter-Choreographies is an artistic research project that combines critical design, choreography and embodied sense-making with data tracking. The project has evolved into various formats such as workshops, a live-coding performance piece, web-based tools and artworks. In this talk, Joana Chicau will introduce the motivations behind the project and how it has impacted audiences by raising awareness of data tracking. She will also demo a custom version of the open-source DuckDuckGo privacy extension which unveils online tracking algorithms through audio and visual feedback.

We invite technology practitioners to join us in the collaborative exploration of discomfort associated with technology in the age of surveillance capitalism. With the help of body-based exercises inspired by choreography we will articulate the discomforts of living and designing with extractivist technology. Our studio is aimed at technology practitioners of a broad range of expertise who have experienced discomfort in relation to data-driven extractivist systems. In the first part of the studio participants will share their experiences of resisting such systems both as users and creators of technology. In the second part, participants will engage in an ideation session to propose forms of countering existing technologies. Embodied methods and choreographic approaches will be used for making digital discomfort tangible and for guiding the exploration of the topics at stake. As an outcome, participants will collectively design a toolbox to conceptualise discomfort in a tangible, embodied way, and form a network to continue discuss these matters post-studio in an online community discussion group.

Modern digitally-augmented boardgames have, with few exceptions, relied primarily on mobile devices rather than taking advantage of other components such as microcontrollers, sensors, and actuators. What alternative shapes might such games take? And what types of player experiences might such games support? To begin to map out answers to these questions, we collected information about how 31 hobbyist boardgame players envision interacting with technology in future analogue-digital hybrid games, using a rapid idea generation activity. We employed a qualitative content analysis approach to identify their envisioned game components, player interactions with these components, and game effects triggered by these interactions. From this analysis, we constructed a taxonomy of analogue-digital hybrid board games as envisioned by players. This paper uses the taxonomy to organise a detailed discussion of players' imagined interactions. This work thus contributes a player-centric exploration of the design space of hybrid digital-analogue games, with the aim of inspiring new, alternative approaches to boardgame design. Based on the taxonomy, we have additionally released a free and open-source ideation card deck to support new avenues into the design of future hybrid games.

What expectations exist in the minds of dancers when interacting with a generative machine learning model? During two workshop events, experienced dancers explore these expectations through improvisation and role-play, embodying an imagined AI-dancer. The dancers explored how intuited flow, shared images, and the concept of a human replica might work in their imagined AI-human interaction. Our findings challenge existing assumptions about what is desired from generative models of dance, such as expectations of realism, and how such systems should be evaluated. We further advocate that such models should celebrate non-human artefacts, focus on the potential for serendipitous moments of discovery, and that dance practitioners should be included in their development. Our concrete suggestions show how our findings can be adapted into the development of improved generative and interactive machine learning models for dancers’ creative practice.

Board gaming is a popular hobby that increasingly features the inclusion of technology, yet little research has sought to understand how board game player experience is impacted by digital augmentation or to inform the design of new technology-enhanced games. We present a mixed-methods study exploring how the presence of music and sound effects impacts the player experience of a board game. We found that the soundtrack increased the enjoyment and tension experienced by players during game play. We also found that a soundtrack provided atmosphere surrounding the gaming experience, though players did not necessarily experience this as enhancing the world-building capabilities of the game. We discuss how our findings can inform the design of new games and soundtracks as well as future research into board game player experience.

As immersive technologies are increasingly being adopted by artists, dancers and developers in their creative work, there is a demand for tools and methods to design compelling ways of embodied interaction within virtual environments. Interactive Machine Learning allows creators to quickly and easily implement movement interaction in their applications by performing examples of movement to train a machine learning model. A key aspect of this training is providing appropriate movement data features for a machine learning model to accurately characterise the movement then recognise it from incoming data. We explore methodologies that aim to support creators’ understanding of movement feature data in relation to machine learning models and ask how these models hold the potential to inform creators’ understanding of their own movement. We propose a 5-day hackathon, bringing together artists, dancers and designers, to explore designing movement interaction and create prototypes using new interactive machine learning tool InteractML.

Interactive Machine Learning offers a method for designing movement interaction that supports creators in implementing even complex movement designs in their immersive applications by simply performing them with their bodies. We introduce a new tool, InteractML, and an accompanying ideation method, which makes movement interaction design faster, adaptable and accessible to creators of varying experience and backgrounds, such as artists, dancers and independent game developers. The tool is specifically tailored to non-experts as creators configure and train machine learning models via a node-based graph and VR interface, requiring minimal programming. We aim to democratise machine learning for movement interaction to be used in the development of a range of creative and immersive applications.

Videogame systems incorporate varied sensors to increase the range of player interactions and improve player experience. However, implementing robust recognisers for player actions with sensors presents significant challenges to developers. Further, sensor-based controls offer little player customisation compared to traditional input interfaces (gamepads, keyboards and joysticks). Past research on motion-driven music systems has successfully used interactive machine learning (IML) techniques to facilitate the development and customisation of sensor-based interfaces, both by developers and end users. However, existing standalone software tools for IML are not ideal for use in game development and distribution. In order to support more effective and flexible use of sensors by game developers and players, we developed an integrated IML solution for Unity3D in the form of a visual node system supporting classification, regression and time series analysis of sensor data.

This article aims to lay a foundation for the research and practice of machine learning education for creative practitioners. It begins by arguing that it is important to teach machine learning to creative practitioners and to conduct research about this teaching, drawing on related work in creative machine learning, creative computing education, and machine learning education. It then draws on research about design processes in engineering and creative practice to motivate a set of learning objectives for students who wish to design new creative artifacts with machine learning. The article then draws on education research and knowledge of creative computing practices to propose a set of teaching strategies that can be used to support creative computing students in achieving these objectives. Explanations of these strategies are accompanied by concrete descriptions of how they have been employed to develop new lectures and activities, and to design new experiential learning and scaffolding technologies, for teaching some of the first courses in the world focused on teaching machine learning to creative practitioners. The article subsequently draws on data collected from these courses—an online course as well as undergraduate and masters-level courses taught at a university—to begin to understand how this curriculum supported student learning, to understand learners’ challenges and mistakes, and to inform future teaching and research.

editorial Free Access Share on Introduction to the Special Section: Launching an Agenda for Research on Learning Machine Learning Authors: R. Benjamin Shapiro Department of Computer Science, University of Colorado Boulder Department of Computer Science, University of Colorado BoulderView Profile , Rebecca Fiebrink Department of Computing, Goldsmiths University of London Department of Computing, Goldsmiths University of LondonView Profile Authors Info & Claims ACM Transactions on Computing EducationVolume 19Issue 4December 2019 Article No.: 30pp 1–6https://doi.org/10.1145/3354136Published:10 October 2019Publication History 2citation570DownloadsMetricsTotal Citations2Total Downloads570Last 12 Months133Last 6 weeks16 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF

Machine learning is one of the most important and successful techniques in contemporary computer science. Although it can be applied to myriad problems of human interest, research in machine learning is often framed in an impersonal way, as merely algorithms being applied to model data. However, this viewpoint hides considerable human work of tuning the algorithms, gathering the data, deciding what should be modeled in the first place, and using the outcomes of machine learning in the real world. Examining machine learning from a human-centered perspective includes explicitly recognizing human work, as well as reframing machine learning workflows based on situated human working practices, and exploring the co-adaptation of humans and intelligent systems. A human-centered understanding of machine learning in human contexts can lead not only to more usable machine learning tools, but to new ways of understanding what machine learning is good for and how to make it more useful. This special issue brings together nine articles that present different ways to frame machine learning in a human context. They represent very different application areas (from medicine to audio) and methodologies (including machine learning methods, human-computer interaction methods, and hybrids), but they all explore the human contexts in which machine learning is used. This introduction summarizes the articles in this issue and draws out some common themes.

We have applied interactive machine learning (IML) to the creation and customisation of gesturally controlled musical interfaces in six workshops with people with learning and physical disabilities. Our observations and discussions with participants demonstrate the utility of IML as a tool for participatory design of accessible interfaces. This work has also led to a better understanding of challenges in end-user training of learning models, of how people develop personalised interaction strategies with different types of pre-trained interfaces, and of how properties of control spaces and input devices influence people's customisation strategies and engagement with instruments. This work has also uncovered similarities between the musical goals and practices of disabled people and those of expert musicians.

Research on students' learning in computing typically investigates how to enable individuals to develop concepts and skills, yet many forms of computing education, from peer instruction to robotics competitions, involve group work in which understanding may not be entirely locatable within individuals' minds. We need theories and methods that allow us to understand learning in cognitive systems: culturally and historically situated groups of students, teachers, and tools. Accordingly, we draw on Hutchins' Distributed Cognition [16] theory to present a qualitative case study analysis of interaction and learning within a small group of middle school students programming computer music. Our analysis shows how a system of students, teachers, and tools, working in a music classroom, is able to accomplish conceptually demanding computer music programming. We show how the system does this by 1) collectively drawing on individuals' knowledge, 2) using the physical and virtual affordances of different tools to organize work, externalize knowledge, and create new demands for problem solving, and 3) reconfiguring relationships between individuals and tools over time as the focus of problem solving changes. We discuss the implications of this perspective for research on teaching, learning and assessment in computing.

Interactive end-user training of machine learning systems has received significant attention as a tool for personalizing recognizers. However, most research limits end users to training a fixed set of application-defined concepts. This paper considers additional challenges that arise in end-user support for defining the number and nature of concepts that a system must learn to recognize. We develop BeatBox, a new system that enables end-user creation of custom beatbox recognizers and interactive adaptation of recognizers to an end user's technique, environment, and musical goals. BeatBox proposes rapid end-user exploration of variations in the number and nature of learned concepts, and provides end users with feedback on the reliability of recognizers learned for different potential combinations of percussive vocalizations. In a preliminary evaluation, we observed that end users were able to quickly create usable classifiers, that they explored different combinations of concepts to test alternative vocalizations and to refine classifiers for new musical contexts, and that learnability feedback was often helpful in alerting them to potential difficulties with a desired learning concept.

Model evaluation plays a special role in interactive machine learning (IML) systems in which users rely on their assessment of a model's performance in order to determine how to improve it. A better understanding of what model criteria are important to users can therefore inform the design of user interfaces for model evaluation as well as the choice and design of learning algorithms. We present work studying the evaluation practices of end users interactively building supervised learning systems for real-world gesture analysis problems. We examine users' model evaluation criteria, which span conventionally relevant criteria such as accuracy and cost, as well as novel criteria such as unexpectedness. We observed that users employed evaluation techniques---including cross-validation and direct, real-time evaluation---not only to make relevant judgments of algorithms' performance and interactively improve the trained models, but also to learn to provide more effective training data. Furthermore, we observed that evaluation taught users about what types of models were easy or possible to build, and users sometimes used this information to modify the learning problem definition or their plans for using the trained models in practice. We discuss the implications of these findings with regard to the role of generalization accuracy in IML, the design of new algorithms and interfaces, and the scope of potential benefits of incorporating human interaction in the design of supervised learning systems.

Multi-touch interactions are a promising means of control for interactive tabletops. However, a lack of precision and tactile feedback makes multi-touch controls a poor fit for tasks where precision and feedback are crucial. We present an approach that offers precise control and tactile feedback for tabletop systems through the integration of dynamically re-mappable physical controllers with the multi-touch environment, and we demonstrate this approach in our collaborative tabletop audio editing environment. An observational user study demonstrates that our approach can provide needed precision and feedback, while preserving the collaborative benefits of a shared direct-manipulation surface. Our observations also suggest that direct touch and physical controllers can offer complementary benefits, and that providing both allows users to adjust their control strategy based on considerations including precision, convenience, visibility, and user role.