Explainable AI (XAI) is frequently positioned as a technical problem of revealing the inner workings of an AI model. This position is affected by unexamined onto-epistemological assumptions: meaning is treated as immanent to the model, the explainer is positioned outside the system, and a causal structure is presumed recoverable through computational techniques. In this paper, we draw on Barad’s agential realism to develop an alternative onto-epistemology of XAI. We propose that interpretations are material-discursive performances that emerge from situated entanglements of the AI model with humans, context, and the interpretative apparatus. To develop this position, we read a comprehensive set of XAI methods through agential realism and reveal the assumptions and limitations that underpin several of these methods. We then articulate the framework’s ethical dimension and propose design directions for XAI interfaces that support emergent interpretation, using a speculative text-to-music interface as a case study.

As our most advanced technologies, such as AI, become both infrastructural and opaque, experts must educate and engage the broader public.To that end, we developed an Augmented Reality * (AR) museum installation about facial recognition and data collection that served both as a medium of public education and as a platform for collecting multiple different kinds of data-though, notably, not facial or other biometric data-from more than 100,000 museum visitors.We explain our design process through four animating tensions: comfort/discomfort, simplicity/complexity, neutrality/critique, and the individual/communal.Using thematic analysis of interviews and surveys, we draw insights on how people exposed to problematic technologies in a 'safe space' such as a museum make sense of these issues: with levity and resignation but also reverence, often specifically rooted in local cultures.We conclude with implications of the guiding principle derived from this work: "using problematic technology to teach about problematic technology."

The practice of sound design involves creating and manipulating environmental sounds for music, films, or games. Recently, an increasing number of studies have adopted generative AI to assist in sound design co-creation. Most of these studies focus on the needs of novices, and less on the pragmatic needs of sound design practitioners. In this paper, we aim to understand how generative AI models might support sound designers in their practice. We designed two interactive generative AI models as Creative Support Tools (CSTs) and invited nine professional sound design practitioners to apply the CSTs in their practice. We conducted semi-structured interviews and reflected on the challenges and opportunities of using generative AI in mixed-initiative interfaces for sound design. We provide insights into sound designers’ expectations of generative AI and highlight opportunities to situate generative AI-based tools within the design process. Finally, we discuss design considerations for human-AI interaction researchers working with audio.

Abstract This article presents an empirical investigation into spectator interpretation of multimodal artworks. Specifically, this article explores perception of the artwork in two different sensorial modalities — sight and sound — and the effect of their interaction. We selected four abstract paintings and created four acousmatic musical pieces that were composed to reflect the content of the artwork. We then ran a between-subjects experimental study in three conditions: visual only, music only, and a combination of both. A total of 48 participants completed an online survey in which they were asked to report their interpretations of the shown artworks. Following a thematic analysis on the collected data, we clustered participants’ interpretation into two main categories:reflectiveandperceptive. The combination increased spectators’ attention to the artworks, affected the temporality of the artworks, and created richer understandings of the multimodal works. The study provides knowledge for the inclusion of multimodal experiences in the presentation of art expanding the possibilities for inter-sensory dialogue in the arts.

Software updates are commonly perceived as tools for fixing flaws and improving functionality. In this paper, we problematise this view by showing how software updates may also be used by vendors to create new norms of use that control user behaviour and reduce their agency. We explore the nature and aftermath of a controversial software update that was released by Spotify in June 2019. By analysing almost 3,500 reactions to this update, we show how it removed and modified several features in ways that severely affected users' capability to organise, navigate, and maintain their music libraries, while it pushed modes of listening that delegate song selection to Spotify. Elaborating upon our results, we discuss how updates may be used as political tools that privilege certain forms of behaviour while restricting others. We also portray updates as sites where ongoing struggles and negotiations regarding user agency and digital ownership take place.

This paper explores the process by which designers come to terms with an unfamiliar and ambiguous sensor material. Drawing on craft practice and material-driven interaction design, we developed a simple yet flexible sensor technology based on the movement of conductive elements within a magnetic field. Variations in materials and structure give rise to objects which produce a complex time-varying signal in response to physical interaction. Sonifying the signal yields nuanced and intuitive action-sound correspondences which nonetheless defy easy categorisation in terms of conventional types of sensors. We reflect on a craft-based exploration of the material by one of the authors, then report on two workshops with groups of designers of varying background. Through examining the objects produced and the experience of the participants, we explore the tension between tacit and explicit understanding of unfamiliar materials and the ways that material thinking can create new design opportunities.

This paper presents a model to codesign interpretively flexible artefacts. We present the case study of Beatfield, a multimodal system that allows users to control audiovisual material by means of tangible interaction. The design of the system was coauthored by individuals with different background and interests to encourage a range of difference interpretations. The capability of Beatfield to foster multiple interpretations was evaluated in a qualitative study with 21 participants. Elaborating on the outcome of this study, we present a new design model that can be used to stimulate heterogeneous interpretations of interactive artefacts.

This paper reflects on the dynamics and practices of building a maker community around a new hardware platform. We examine the factors promoting the successful uptake of a maker platform from two perspectives: first, we investigate the technical and user experience considerations that users identify as the most important. Second, we explore the specific activities that help attract a community and encourage sustained participation. We present an inductive approach based on the case study of Bela, an embedded platform for creating interactive audio systems. The technical design and community building processes are detailed, culminating in a successful crowdfunding campaign. To further understand the community dynamics, the paper also presents an intensive three-day workshop with eight digital musical instrument designers. From observations and interviews, we reflect on the relationship between the platform and the community and offer suggestions for HCI researchers and practitioners interested in establishing their own maker communities.

Most of musicians can control their performance by relying on different sensorial modalities that complement the auditory cue. Vision, in particular, offers most instrumentalists an essential support: it helps them developing techniques, identifying errors, correcting expressiveness, and memorise complex passages. By contrast, when performing a piece, singers can almost exclusively rely on the auditory feedback coming from their voice to adjust their singing. This paper frames this issue and proposes possible alternatives to improve singers' awareness by adding visual and tangible feedback to their performance.

The recent increase in interest for online multimedia streaming platforms has availed massive amounts of multimedia information that need to be indexed to be searchable and retrievable. User-centric implicit affective indexing employing emotion detection based on psycho-physiological signals, such as electrocardiography (ECG), galvanic skin response (GSR), electroencephalography (EEG) and face tracking, has recently gained attention. However, real world psycho-physiological signals obtained from wearable devices and facial trackers are contaminated by various noise sources that can result in spurious emotion detection. Therefore, in this paper we propose the development of psycho-physiological signal quality estimators for unimodal affect recognition systems. The presented systems perform adequately in classifying users affect however, they resulted in high failure rates due to rejection of bad quality samples. Thus, to reduce the affect recognition failure rate, a quality adaptive multimodal fusion scheme is proposed. The proposed scheme yields no failure, while at the same time classify the users' arousal/valence and liking with significantly above chance weighted F1-scores in a cross-user experiment. Another finding of this study is that head movements encode liking perception of users in response to music snippets. This work also includes the release of the employed dataset including psycho-physiological signals, their quality annotations, and users' affective self-assessments.