Recent advances in generative AI have enabled new forms of audio synthesis for musical creation, yet model training remains underexplored as an artistic and collaborative practice. While prior work has focused on composition tools and live performance interfaces, how artists train their models have received less attention. This paper presents a qualitative study of collaborative training practices around audio synthesis models. We conducted semi-structured interviews with 13 artists across 8 musical projects relying on custom-trained models for musical performance or installations. Using thematic analysis, our findings show that artists approach training as a situated, craft-like practice rather than a purely technical task. Collectives develop shared languages and embodied practices to communicate and guide training, using iterative feedback and dataset sculpting to shape model behavior, with musical engagement as a primary method for steering models. We discuss implications to better support model training as key site of interaction and collaboration.

Recent advances in Artificial Intelligence have enabled powerful generative models, yet few are tailored to dancers’ practices. We present a long-term collaboration with a Voguing and Dancehall collective to design movement generation models trained on their repertoire. Our initial study with the dancers revealed that, despite limited physical realism, the generated movements inspired them. Iterative development led to Korai, an interactive tool for monitoring training, visualizing motion data, and prompting generation, which improved output quality. A subsequent structured observation study compared three model variants with high, medium, and low fidelity to the original dataset’s style. Results show that dancers favored either highly faithful or highly unfaithful outputs, rejecting medium fidelity as neither authentic to their style nor creatively stimulating. Our findings highlight how direct collaboration with dancers not only informs model design but also deepens understanding of AI’s role in supporting creative movement practices.

Non-experts increasingly engage in user-driven algorithm auditing, interacting directly with AI systems to probe, document, and reflect on biased behavior. Yet, auditing remains challenging due to model opacity and limited support for navigating and interpreting outputs. This paper explores the design and evaluation of interfaces grounded in the sensemaking framework to support non-experts in auditing gender bias in image captioning. In a between-subjects study, 60 participants audited an image captioning model using one of three interface conditions: a Baseline interface, a Masking Tool for image manipulation, or a Filtering Tool for organizing captions. Our findings show that interface design shaped what participants noticed, how they interpreted model behavior, and supported their hypotheses. The Image Masking Tool enabled fine-grained testing of visual cues and context, while the Text Filtering Tool revealed broader asymmetries in gendered language. We argue that incorporating sensemaking into auditing practices can advance accountability and transparency in machine learning systems.

Transformer-based ranking models, such as MonoBERT, are central to Information Retrieval; yet their inner workings remain largely opaque. This hinders not only our understanding of the systems implementing them, but also our ability to improve them. To alleviate this limitation, we introduce IR Lens, a new interpretability tool tailored to cross-encoders based on two key components: 1) Neuron Integrated Gradients to expose the contributions of model parts at multiple levels, and 2) targeted ablations to support hypothesis tracking. With its interactive graphical interface, IR Lens enables IR practitioners to explore, analyze, and manipulate neuron-level mechanisms in cross-encoders, facilitating a deeper understanding of neural ranking models. By extending the reach of existing interpretability methods, we believe IR Lens has the potential to support the improvement of cross-encoders.

While human-centered approaches to machine learning explore various human roles within the interaction loop, the notion of Interactive Machine Teaching (IMT) emerged with a focus on leveraging the teaching skills of humans as a teacher to build machine learning systems. However, most systems and studies are devoted to single users. In this article, we study collaborative interactive machine teaching in the context of image classification to analyze how people can structure the teaching process collectively and to understand their experience. Our contributions are threefold. First, we developed a web application called TeachTOK that enables groups of users to curate data and train a model together incrementally. Second, we conducted a study in which ten participants were divided into three teams that competed to build an image classifier in nine days. Qualitative results of participants’ discussions in focus groups reveal the emergence of collaboration patterns in the machine teaching task, how collaboration helps revise teaching strategies and participants’ reflections on their interaction with the TeachTOK application. From these findings we provide implications for the design of more interactive, collaborative and participatory machine learning-based systems.

We present a performance-led inquiry that involved a live coder programming movement-based interactive sound and two dance improvisers. During two years of collaboration, we developed a joint improvisation practice where the interactions between the dancers’ movement and the sound feedback are programmed on the fly through live coding and movement sensing. To that end, we designed a new live coding environment called CO/DA that facilitates the real-time manipulation of continuous streams of the dancers’ motion data for interactive sound synthesis. Through an autoethnographic inquiry, we describe our practice of sound and movement improvisation where live coding dynamically changes how the dancers’ movements generate sound, which in turn influences the dancers’ improvisation. We then discuss the value, potential and challenges of our dance/code improvisation practice, along with its implications as a design method.

Human-centered approaches to machine learning have established theoretical foundations, design principles and interaction techniques to facilitate end-user interaction with machine learning systems. Yet, general-purpose toolkits supporting the design of interactive machine learning systems are still missing, despite their potential to foster reuse, appropriation and collaboration between different stakeholders including developers, machine learning experts, designers and end users. In this paper, we present an architectural model for toolkits dedicated to the design of human interactions with machine learning. The architecture is built upon a modular collection of interactive components that can be composed to build interactive machine learning workflows, using reactive pipelines and composable user interfaces. We introduce Marcelle, a toolkit for the design of human interactions with machine learning that implements this model. We illustrate Marcelle with two implemented case studies: (1) a HCI researcher conducts user studies to understand novice interaction with machine learning, and (2) a machine learning expert and a clinician collaborate to develop a skin cancer diagnosis system. Finally, we discuss our experience with the toolkit, along with its limitation and perspectives.

Machine learning systems became pervasive in modern interactive technology but provide users with little, if any, agency with respect to how their models are trained from data. In this paper, we are interested in the way novices handle learning algorithms, what they understand from their behavior and what strategy they may use to "make it work". We developed a web-based sketch recognition algorithm based on Deep Neural Network (DNN), called Marcelle-Sketch, that end-users can train incrementally. We present an experimental study that investigate people's strategies and (mis)understandings in a realistic algorithm-teaching task. Our study involved 12 participants who performed individual teaching sessions using a think-aloud protocol. Our results show that participants adopted heterogeneous strategies in which variability affected the model performances. We highlighted the importance of sketch sequencing, particularly at the early stage of the teaching task. We also found that users' understanding is facilitated by simple operations on drawings, while confusions are caused by certain inherent properties of DNN. From these findings, we propose implications for design of IML systems dedicated to novices and discuss the socio-cultural aspect of this research.

Busy daily lives and ongoing distractions often make people feel disconnected from their bodies and experiences. Guided attention to self can alleviate this disconnect as in focused-attention meditation, in which breathing often constitutes the primary object on which to focus attention. In this context, sustained breath awareness plays a crucial role in the emergence of the meditation experience. We designed an immersive virtual environment (iVE) with a generative soundtrack that supports sustained attention on breathing by employing the users' breathing in interaction. Both sounds and visuals are directly mapped to the user's breathing patterns, thus bringing the awareness researched. We conducted micro-phenomenology interviews to unfold the process in which breath awareness can be induced and sustained in this environment. The findings revealed the mechanisms by which audio and visual cues in VR can elicit and foster breath-awareness, and unfolded the nuances of this process through subjective experiences of the study participants. Finally, the results emphasize the important role that a sense of agency and control have in shaping the overall quality of the experience. This can in turn inform the design specifications of future mindfulness-based designs focused on breath awareness.

Technologies for sensing movement are expanding toward everyday use in virtual reality, gaming, and artistic practices. In this context, there is a need for methodologies to help designers and users create meaningful movement experiences. This article discusses a user-centered approach for the design of interactive auditory feedback using interactive machine learning. We discuss Mapping through Interaction, a method for crafting sonic interactions from corporeal demonstrations of embodied associations between motion and sound. It uses an interactive machine learning approach to build the mapping from user demonstrations, emphasizing an iterative design process that integrates acted and interactive experiences of the relationships between movement and sound. We examine Gaussian Mixture Regression and Hidden Markov Regression for continuous movement recognition and real-time sound parameter generation. We illustrate and evaluate this approach through an application in which novice users can create interactive sound feedback based on coproduced gestures and vocalizations. Results indicate that Gaussian Mixture Regression and Hidden Markov Regression can efficiently learn complex motion-sound mappings from few examples.

Human movement has historically been approached as a functional component of interaction within human computer interaction. Yet movement is not only functional, it is also highly expressive. In our research, we explore how movement expertise as articulated in Laban Movement Analysis (LMA) can contribute to the design of computational models of movement's expressive qualities as defined in the framework of Laban Efforts. We include experts in LMA in our design process, in order to select a set of suitable multimodal sensors as well as to compute features that closely correlate to the definitions of Efforts in LMA. Evaluation of our model shows that multimodal data combining positional, dynamic and physiological information allows for a better characterization of Laban Efforts. We conclude with implications for design that illustrate how our methodology and our approach to multimodal capture and recognition of Effort qualities can be integrated to design interactive applications.

We consider kinesthetic awareness, the perception of our own body position and movement in space, as a critical value for embodied design within third wave HCI. We designed an interactive sound installation that supports kinesthetic awareness of a participant's micro-movements. The installation's interaction design uses continuous auditory feedback and leverages an adaptive mapping strategy, refining its sensitivity to increase sonic resolution at lower levels of movement activity. The installation uses field recordings as rich source materials to generate a sound environment that attunes to a participant's micro-movements. Through a qualitative study using a second-person interview technique, we gained nuanced insights into the participants' subjective experiences of the installation. These reveal consistent temporal patterns, as participants build on a gradual process of integration to increase the complexity and capacity of their kinesthetic awareness during interaction.

We investigate the use of interactive sound feedback for dance pedagogy based on the practice of vocalizing while moving. Our goal is to allow dancers to access a greater range of expressive movement qualities through vocalization. We propose a methodology for the sonification of Effort Factors, as defined in Laban Movement Analysis, based on vocalizations performed by movement experts. Based on the experiential outcomes of an exploratory workshop, we propose a set of design guidelines that can be applied to interactive sonification systems for learning to perform Laban Effort Factors in a dance pedagogy context.

In this paper we address the issue of mapping between gesture and sound in interactive music systems. Our approach, we call mapping by demonstration, aims at learning the mapping from examples provided by users while interacting with the system. We propose a general framework for modeling gesture--sound sequences based on a probabilistic, multimodal and hierarchical model. Two orthogonal modeling aspects are detailed and we describe planned research directions to improve and evaluate the proposed models.

We address the issue of mapping between gesture and sound for gesture-based control of physical modeling sound synthesis. We propose an approach called mapping by demonstration, allowing users to design the mapping by performing gestures while listening to sound examples. The system is based on a multimodal model able to learn the relationships between gestures and sounds.

In this paper, we propose a multimodal approach to create the mapping between gesture and sound in interactive music systems. Specifically, we propose to use a multimodal HMM to conjointly model the gesture and sound parameters. Our approach is compatible with a learning method that allows users to define the gesture--sound relationships interactively. We describe an implementation of this method for the control of physical modeling sound synthesis. Our model is promising to capture expressive gesture variations while guaranteeing a consistent relationship between gesture and sound.