We chart the creative journeys of two groups of musicians as they explored how to engage with an AI system developed for the live performance of Irish Traditional Dance Music. A group of expert Irish musicians participated in an extended workshop to refine the system while a professional contemporary folk duo composed and rehearsed material for a public show. Drawing on the standard definition of creativity, we chart their creative journeys across a landscape we define by the two dimensions of originality and effectiveness. We reveal the importance of authenticity to a traditional practice as an opposing force to creativity and consider how the tension between these shaped their musical journeys. We generalise five strategies that move creative practitioners across this landscape in different directions: proving your chops, finding your place, perfecting the nuances, innovating, and glitching.

This paper discusses the findings from a cross-sector research project investigating how a digital score created using AI and robotics might stimulate new creative opportunities and relationships within the practices of an inclusive music ensemble. Through the concept of a digital score [65], AI and a robotic arm were introduced into an ensemble’s musical practice to evaluate the impact and benefits of using autonomous systems to challenge barriers around a disabled musician’s access to creative music-making. Throughout the development process we placed an emphasis on involvement and togetherness of not only the AI and robots’ contribution to shared creativity amongst the ensemble, but also to the social aspects of the creative process across the team of musicians, developers, researchers and supporting organisations. The findings were surprising with many aspects of the project exceeding the expectations of the original aims. In short, all the musicians benefited from the introduction of these unfamiliar technologies with practices enhanced and relationships transformed.

Volunteering benefits recipients, volunteers, communities, and society, while digital technologies establish new opportunities for virtual volunteering. We describe how volunteers transitioned the UK's long-established Oxjam grassroots music festival online in response to the COVID pandemic, delivering a local pilot before scaling up nationwide. We adopt an infrastructural perspective to reveal how two teams of volunteers defined a flexible festival format, knitted together diverse technologies into a technical platform, and operated this to deliver the festival. We highlight the need for teams of volunteers to orchestrate both audience and performer trajectories through festivals. We argue for deliberately designing in volunteer labour rather than automating it out by translating traditional roles online while defining new digital ones. We propose to make these roles rewarding through a more social volunteer experience, including privileged backstage access. We highlight the challenges of using social media for such events, including complying with algorithmic policing of rights.

Losing Her Voice is a new opera which highlights the challenges of subtly interweaving digital technologies into established cultural forms. Audience members were encouraged to use their own mobile phones to interact with on-stage projections before, during and after the performance. We chart the trajectories of participation that were designed into the premiere performances of this work, and how these unfolded in practice. We identify the strategies that were effective to: encourage the adoption of the mobile app; interweave use with the other elements of the opera performance; make it consistent with the content; and ensure it complemented but was not essential to the show. We highlight the way in which three canonical trajectories must be woven together in this experience: the standard trajectory of “a night at the opera”; the dramatic arc of the specific work, and the audience member's use of their own device.

Access to healthcare advice is crucial to promote healthy societies. Many factors shape how access might be constrained, such as economic status, education or, as the COVID-19 pandemic has shown, remote consultations with health practitioners. Our work focuses on providing pre/post-natal advice to maternal women. A salient factor of our work concerns the design and deployment of embodied conversation agents (ECAs) which can sense the (health) literacy of users and adapt to scaffold user engagement in this setting. We present an account of a Wizard of Oz user study of 'ALTCAI', an ECA with three modes of interaction (i.e., adaptive speech and text, adaptive ECA, and non-adaptive ECA). We compare reported engagement with these modes from 44 maternal women who have differing levels of literacy. The study shows that a combination of embodiment and adaptivity scaffolds reported engagement, but matters of health-literacy and language introduce nuanced considerations for the design of ECAs.

As computer interfaces spread across and even into our bodies, so we find ourselves engaged in ever more deeply connected bodily interactions. From the mouse and keyboard, to wearables and gestural...

This article presents the results of a study based on a group of participants' interactions with an experimental sound installation at the National Science and Media Museum in Bradford, UK. The installation used audio augmented reality to attach virtual sound sources to a vintage radio receiver from the museum's collection, with a view to understanding the potentials of this technology for promoting exploration and engagement within museums and galleries. We employ a practice-based design ethnography, including a thematic analysis of our participants' interactions with spatialised interactive audio, and present an identified sequence of interactional phases. We discuss how audio augmented artefacts can communicate and engage visitors beyond their traditional confines of line-of-sight, and how visitors can be drawn to engage further, beyond the realm of their original encounter. Finally, we provide evidence of how contextualised and embodied interactions, along with authentic audio reproduction, evoked personal memories associated with our museum artefact, and how this can promote interest in the acquisition of declarative knowledge. Additionally, through the adoption of a functional and theoretical aura-based model, we present ways in which this could be achieved, and, overall, we demonstrate a material object's potential role as an interface for engaging users with, and contextualising, immaterial digital audio archival content.

Hybrid gifting combines physical artefacts and experiences with digital interactivity to generate new kinds of gifts. Our review details how gifting is a complex social phenomenon and how digital gifting is less engaging than physical gifting for both givers and receivers. Employing a Research Through Design approach, we developed a portfolio of four hybrid gifting experiences: an augmented advent calendar; edible music tracks; personalized museum tours; and a narrated city walk. Our reflection addresses three concepts: hybrid wrapping where physical gifts become wrapped in digital media and vice versa; the importance of effortful interactions that are visible and pleasurable; and the need to consider social obligation, including opportunities for acknowledgement and reciprocation, dealing with embarrassment, and recognizing the distinction between giving and sharing. Our concepts provide guidance to practitioners who wish to design future gifting experiences while helping HCI researchers engage with the concept of gifting in a nuanced way.

Failure is a common artefact of challenging experiences, a fact of life for interactive systems but also a resource for aesthetic and improvisational performance. We present a study of how three professional pianists performed an interactive piano composition that included playing hidden codes within the music so as to control their path through the piece and trigger system actions. We reveal how apparent failures to play the codes occurred for diverse reasons including mistakes in their playing, limitations of the system, but also deliberate failures as a way of controlling the system, and how these failures provoked aesthetic and improvised responses from the performers. We propose that creative and performative interfaces should be designed to enable aesthetic failures and introduce a taxonomy that compares human approaches to failure with approaches to capable systems, revealing new creative design strategies of gaming, taming, riding and serving the system.

Guitars are physical instruments that require skillful two-handed use. Their use is also supported by diverse digital and physical resources, such as videos and chord charts. To understand the challenges of interacting with supporting resources at the same time as playing we conducted an ethnographic study of the preparation activities of working musicians. We observe successive stages of individual and collaborative preparation, in which working musicians engage with a diverse range of digital and physical resources to support their preparation. Interaction with this complex ecology of digital and physical resources is finely interwoven into their embodied musical practices, which are usually encumbered by having their instrument in hand, and often by playing. We identify challenges for augmenting guitars within the rehearsal process by supporting interaction that is encumbered, contextual and connected, and suggest a range of possible responses.

We report on the design, premiere and public evaluation of a multifaceted audience interface for a complex non-linear musical performance called Climb! which is particularly suited to being experienced more than once. This interface is designed to enable audiences to understand and appreciate the work, and integrates a physical instrument and staging, projected visuals, personal devices and an online archive. A public premiere concert comprising two performances of Climb! revealed how the audience reoriented to the second performance through growing understanding and comparison to the first. Using trajectories as an analytical framework for the audience 'journey' made apparent: how the trajectories of a single performance are embedded within the larger trajectories of a concert and the creative work as a whole; the distinctive demands of understanding and interpretation; and the potential of the archive in enabling appreciation across repeated performances.

We present our case study on gifting digital music, The Rough Mile, as an example of a Framework of Immersive Practice, intended for researchers and practitioners in HCI and interaction design. Although immersion is a frequently used term in the HCI and related literatures, we find no definitions or frameworks that are robust enough to capture the full range of multi-sensory, emotional, and cognitive engagement that the richest of these experiences can entail. We therefore turn to the theatrical performance literature to distil a theory-based framework of practices that can apply to interdisciplinary projects as well as works with an entirely aesthetic aim. The design choices and findings of The Rough Mile are presented in terms of this framework, leading to a discussion of the design guidelines that can shape its use in any HCI or interaction design project aiming for a deep, personal engagement through technology.

We consider the potential for novel in-vehicle user-interfaces that alter speed perception at a subliminal level through the spatial adaption of music. In a fixed-base simulator, twenty-six participants drove on a motorway and were asked to maintain a speed of 70mph. At specific points, the speedometer was turned off. Music at a constant tempo was played throughout but periodically changed in balance from a 50:50 front:rear speaker split to a 25:75 ratio. Without the speedometer, participants drove significantly slower after the music had faded from the front to rear speakers (mean speed 71.5mph) compared to when no change occurred (mean speed 73.1mph). Post study interviews revealed that participants were not aware of alterations in the spatial positioning of the music. Such results suggest drivers naturally slowed when the music faded from front to rear speakers in an unconscious attempt to re-envelope themselves within the sound bubble.

We report lessons from iteratively developing a music recognition system to enable a wide range of musicians to embed musical codes into their typical performance practice. The musician composes fragments of music that can be played back with varying levels of embellishment, disguise and looseness to trigger digital interactions. We collaborated with twenty-three musicians, spanning professionals to amateurs and working with a variety of instruments. We chart the rapid evolution of the system to meet their needs as they strove to integrate music recognition technology into their performance practice, introducing multiple features to enable them to trade-off reliability with musical expression. Collectively, these support the idea of deliberately introducing "looseness" into interactive systems by addressing the three key challenges of control, feedback and attunement, and highlight the potential role for written notations in other recognition-based systems.

We explore the crafting of interactive decoration for everyday artefacts. This involves adorning them with decorative patterns that enhance their beauty while triggering digital interactions when scanned with cameras. These are realized using an existing augmented reality technique that embeds computer readable codes into the topological structures of hand-drawn patterns. We describe a research through design process that engaged artisans to craft a portfolio of interactive artefacts, including ceramic bowls, embroidered gift cards, fabric souvenirs, and an acoustic guitar. We annotate this portfolio with reflections on the crafting process, revealing how artisans addressed pattern, materials, form and function, and digital mappings throughout their craft process. Further reflection on our portfolio reveals how they bridged between human and system perceptions of visual patterns and engaged in a deep embedding of digital interactions into physical materials. Our findings demonstrate the potential for interactive decoration, distilling the craft knowledge involved in creating aesthetic and functional decoration, highlight the need for transparent computer vision technologies, and raise wider issues for HCI’s growing engagement with craft.

This paper explores the design of digital musical instruments (DMIs) for exploratory play. Based on Gaver's principles of ludic design, we examine the ways in which people come to terms with an unfamiliar musical interface. We describe two workshops with the D-Box, a DMI designed to be modified and hacked by the user. The operation of the D-Box is deliberately left ambiguous to encourage users to develop their own meanings and interaction techniques. During the workshops we observed emergent patterns of exploration which revealed a rich process of exploratory play. We discuss our observations in relation to previous literature on appropriation, ambiguity and ludic engagement, and we provide recommendations for the design of playful and exploratory interfaces.

We explore how physical artefacts can be connected to digital records of where they have been, who they have encountered and what has happened to them, and how this can enhance their meaning and utility. We describe how a travelling technology probe in the form of an augmented acoustic guitar engaged users in a design conversation as it visited homes, studios, gigs, workshops and lessons, and how this revealed the diversity and utility of its digital record. We describe how this record was captured and flexibly mapped to the physical guitar and proxy artefacts. We contribute a conceptual framework for accountable artefacts that articulates how multiple and complex mappings between physical artefacts and their digital records may be created, appropriated, shared and interrogated to deliver accounts of provenance and use as well as methodological reflections on technology probes.

Non-digital forms of play that allow players to direct brute force directly upon each other, such as martial arts, boxing and full contact team sports, are very popular. However, inter-player brutality has largely been unexplored as a feature of digital gaming. In this paper, we describe the design and study of 2 multi-player games that encourage players to use brute force directly against other players. Balance of Power is a tug-of-war style game implemented with Xbox Kinect, while Bundle is a playground-inspired chasing game implemented with smartphones. Two groups of five participants (n=10) played both games while being filmed, and were subsequently interviewed. A thematic analysis identified five key components of the brutal multiplayer video game experience, which informs a set of seven design considerations. This work aims to inspire the design of engaging game experiences based on awareness and enjoyment of our own and others' physicality.

Listening to music on the move is an everyday activity for many people. This paper proposes geotracks and geolists, music tracks and playlists of existing music that are aligned and adapted to specific journeys. We describe how everyday walking journeys such as commutes to work and existing popular music tracks can each be analysed, decomposed and then brought together, using musical adaptations including skipping and repeating parts of tracks, dynamically remixing tracks and cross-fades. Using a naturalistic experiment we compared walking while listening to geotracks (dynamically adapted using GPS location information) to walking while listening to a fixed playlist. Overall, participants enjoyed the walk more when listening to the adaptive geotracks. However adapting the lengths of tracks appeared to detract from the experience of the music in some situations and for some participants, revealing trade-offs in achieving fine-grained alignment of music and walking journeys.

We present an in-the-wild project to design and study a mobile musical soundtrack that enhances the experience of visiting a sculpture park. As with soundtracks for films and games, the goal was to enhance the emotional and narrative aspects of the experience while remaining in the background. We describe a compositional approach in which we first established a broad musical landscape before treating specific exhibits with detailed musical trajectories. Our study reveals how our soundtrack dramatically shaped visitors' experiences while they remained largely unaware of its operation. We distil seven experiential factors to be addressed by mobile soundtracks alongside ten compositional guidelines.

Musical soundtracks will be important features of future locative experiences from tours to games. We present a study designed to uncover potential relationships between higher-level musical structures such as harmony, melody, timbre, dynamic intensity and punctuation and users' spatial experiences. We observed twenty-two participants exploring an open field while listening to four contrasting musical compositions, and then interviewed them afterwards. We report their different approaches to interpreting the music, strategies for mapping zones, choice of stopping destinations, and their awareness and appreciation of the music. Our discussion of these findings in relation to the literature leads us to propose six initial principles to guide the composition of mobile and locative soundtracks, and also to articulate a three-layer framework of global, regional and local attachment to help guide the attachment of musical features to different regions within a locative experience.