Entrainment offers a rich theoretical understanding of how rhythmic stimuli are embodied in movement. Behavioural research examines how entrained coordination arises in the presence of sound and interpersonal interactions, which are reflected in sensorimotor and physiological responses. This paper explores the affordances of generative models for sound-movement mapping designed to support dyadic embodied interaction. We adopt a set of physical partner-based exercises that emphasise spatio-temporal coordination, guided by a moving prop. A tangible artefact is orchestrated around this prop, integrating two mapping strategies: an explicit, one-to-one mapping, and a latent mapping using a deep generative model for neural synthesis. A study compares these approaches as six dyads perform the exercises. Our findings suggest that sound feedback influenced participants’ peripheral awareness, and that the latent mapping offered relevant design opportunities for evoking familiar embodied sensations and bringing attention to emergent qualities of dyadic movement.

There is a growing interest in interactive visuals for dance performance. Recent research has identified potential in using interactive visuals to convey to the audience otherwise non-visible elements of performances. Informed by soma design, and with a co-design perspective, we aim to make apparent non-visible bodily aspects of dancers. We propose to design interactive visuals from body maps, following two approaches – Machine Learning and Composite Animation. We conducted a multi-stage study involving 12 dancers. We present and discuss the results of our evaluations, confirming that both prototypes were successful in addressing our aim, with some limitations. We discuss our two approaches, different uses and actors in different stages, tensions between research and dance creation, and potential applications. Our main contributions are the two approaches for designing interactive visuals from body maps and their analysis. These are materialized in two software systems released as open-source and in their design framework descriptions.

There has been a growing interest in interactive visuals in contemporary dance performances. These visuals often rely on embodied interaction techniques, such as motion capture or biosignal sensors. However, there is a lack of research into how audience members experience these interactive visuals, and how to enhance that experience. We conducted an audience study, involving four different dance performances. Each of the performances explored a different approach for interaction involving visuals. We collected data from audience members, regarding their experience of the performances, using questionnaires and interviews. The analysis of this data allows us to identify implications for design: balancing trade-offs within a mapping clarity spectrum; connecting layers; visuals as co-creative mediator; defined territory and individuality of visuals; exploration of perspective shift and abstract fragmentation. We argue that these considerations are relevant for designers of systems visualizing embodied interaction, not just for dance, but also for other related applications.

This paper argues that a significant paradigm change in contemporary dance can offer further opportunities for HCI researchers interested in embodied interaction and interactive system design. Based on the analysis of 42 HCI papers in our data set, resulting from searches in two computing research libraries, we suggest seven thematic categories that reflect how HCI researchers have been engaging with contemporary dance. Moreover, we propose a standardized usage of contemporary dance terminology in HCI literature, and discuss the current state of engagement with publications from the field of performance theory. We identify three opportunities for HCI, which can arise through further engagement with the knowledge produced in contemporary dance and performance: to engage with the field of embodied interaction from the perspective of performance research and theory; to employ contemporary dance methods and practices in HCI research; and to integrate contemporary dance choreographers and performers as researchers in interdisciplinary projects.

The combined use of sound and image has a rich history, from audiovisual artworks to research exploring the potential of data visualization and sonification. However, we lack standard tools or guidelines for audiovisual (AV) interaction design, particularly for live performance. We propose the AVUI (AudioVisual User Interface), where sound and image are used together in a cohesive way in the interface; and an enabling technology, the ofxAVUI toolkit. AVUI guidelines and ofxAVUI were developed in a three-stage process, together with AV producers: 1) participatory design activities; 2) prototype development; 3) encapsulation of prototype as a plug-in, evaluation, and roll out. Best practices identified include: reconfigurable interfaces and mappings; object-oriented packaging of AV and UI; diverse sound visualization; flexible media manipulation and management. The toolkit and a mobile app developed using it have been released as open-source. Guidelines and toolkit demonstrate the potential of AVUI and offer designers a convenient framework for AV interaction design.

There is growing interest in the application of crossmodal perception to interface design.However, most research has focused on task performance measures and often ignored user experience and engagement.We present an examination of crossmodal congruence in terms of performance and engagement in the context of a memory task of audio, visual, and audio-visual stimuli.Participants in a first study showed improved performance when using a visual congruent mapping that was cancelled by the addition of audio to the baseline conditions, and a subjective preference for the audio-visual stimulus that was not reflected in the objective data.Based on these findings, we designed an audio-visual memory game to examine the effects of crossmodal congruence on user experience and engagement.Results showed higher engagement levels with congruent displays with some reported preference for potential challenge and enjoyment that an incongruent display may support, particularly for increased task complexity.

The present study aims to address the following research question: how to create a tool for audiovisual performance, allowing for real-time usage of shared online visual resources, which can be customizable, and used across a variety of different hardware platforms? To address this issue, we have developed AVVX (AudioVisual Vector eXchange), a novel application for audiovisual performances, based on open web technologies such as HTML5, JavaScript and SVG (Scalable Vector Graphics). This paper contextualizes AVVX with related work and technologies, and then presents the design and development of the software. Taking as a starting point a workshop conducted with AVVX, the project has been evaluated by means of a questionnaire and user tests. The results of the tests indicate that the web browser, together with open web technologies, can provide a foundation for a customizable, content-sharing and multi-platform approach to audiovisual performance.