What would happen if you could catch a sound in your hands? Push or pull it, stretch it out, mix it up, or toss it at the ceiling like a spaghetti noodle? This work explores how children engage with sound in their environment, and as part of their social connection with others, supported by tangible sound toys. By looking at children’s playful processes of listening to and engaging with the isolation of sounds, we developed three sound-based toys that leverage tangible embodied metaphors. This paper presents our sound toy designs, and contributes to a deeper awareness of the affect of sound as a focused component in tangible design, to playfully challenge children to consider how sound impacts their environment, and how they can impact sound.

This paper explores how sound-based tangible toys can encourage children to engage with sounds in their environment through active listening and collaboration with their peers. Twenty-eight children, aged 3 to 4.5 years old, explored sound in their environment through three toy prototypes. One toy focused on hearing sounds in relation to their environment; such as traffic and children playing. Another toy explored the recording and playback of their own sounds, being "caught" in a racket and blown out. The third toy explored a combination of shaking in sounds, stirring them to manipulate them, and pouring the mix out. This project uses a mixed-methods approach and is presented as a step towards further studies comparing toys with different approaches to sound.

Interactive computer systems have been explored in the performing arts for the past two decades, incorporating several models of the Performer-Technology relationship. As computer systems are afforded greater autonomy and agency in performance and everyday interactions, an interface that reflects the collaborative nature of the Performer-Performer relationship is required. One potential solution utilizes the embodied knowledge of the performer's interaction by modeling breathing patterns in a computer agent as cues to the system's intended action. This choice is supported by theories from cognitive science, performing arts and philosophy illustrating that our embodied perception of other's intention provides the basis for human interaction. We engage the performer's awareness of the computer's 'intention' by modeling breathing patterns as embodied cues, engaging the performer's sense of intuition and empathy. We present our evaluation of a Performer-System interaction model with data collected from performers improvising in studio sessions with a system designed to project its intention through a simulated breath.

This research investigates audience experience of empathy with a performer during a digitally mediated performance. Theatrical performance necessitates social interaction between performers and audience. We present a performance-based study that explores audience awareness of performer's kinaesthetic activity in 2 ways: by isolating the audience's senses (visual, auditory, and kinaesthetic) and by focusing audience perception through defamiliarization. By positioning the performer behind the audience: in their 'backspace', we focus the audience's attention to the performer in an unfamiliar way. We describe two research contributions to the study of audience empathic experience during performance. The first is the development of a phenomenological interview method designed for extracting empirical evaluations of experience of audience members in a performance scenario. The second is a descriptive model for a poetics of reception. Our model is based on an empathetic audience-performer relationship that includes 3 components of audience awareness: contextual, interpersonal, and sense-based. Our research contributions are of particular benefit to performances involving digital media, and can provide insight into audience experience of empathy.

Supporting users' reasoned imagination in sense making during interaction with tangible and embedded computation involves supporting the application of their existing mental schemata in understanding new forms of interaction. Recent studies that include an embodied metaphor in the interaction model, which relates action-based inputs to digital outputs, have provided evidence that this approach is beneficial. Yet the design of such systems has been difficult and full of setbacks. Wide spread adoption of this approach requires a better understanding of how to design such embodied metaphor-based interactional models. We analyze three recent design-based research studies in which we have been involved in order to derive design knowledge that may inform others. Following a case study methodology we identify kernels or points in the design process where discontinuities between predicted and actual interaction highlight important design knowledge.

We discuss three design strategies for improving the quality of social interaction and learning with interactive museum guides: 1) embodied interaction; 2) game-learning; 3) a hybrid system. We used these strategies in our prototype Kurio, which is aimed at supporting families visiting museums. The results of our evaluation show positive implications of implementing the design strategies: closing the social gap, naturalizing technology, and supporting exploration and discovery in learning.

A recent trend in ubiquitous computing is the development of new forms of interfaces, which rely on embodied interaction. We focus on the definition of embodiment that refers to the ways in which abstract concepts rely on metaphorical extensions of embodied schemata shaped by processes below the level of conscious awareness as explored by Lakoff and Johnson [Lakoff, G., Johnson, M., 1980. Metaphors We Live By. Chicago Press, Chicago, IL, USA]. Our inquiry focuses on understanding the role embodied metaphors may play in supporting people to understand the possibilities for physical interaction in augmented spaces. We explore this issue through the development and evaluation of an interactive audio environment. We instantiate metaphor theory by using embodied schemata as the basis for the interactional metaphor that relates full-body input actions to audio output responses. We demonstrate and explore the benefits of this approach through a comparative experiment in which adults and children learn to use our audio environment. The results from our experiment indicated that embodied metaphors improve usability however, other factors including discoverability, perceivability of feedback and duplicity of structural isomorphism may mediate these metaphor-based benefits. We have generalized our main findings as a set of suggestions for the design of embodied style interfaces that rely on physical interaction.

In this paper we present the results of a comparative study that explores the potential benefits of using embodied interaction to help children, aged 7 to 10, learn abstract concepts related to musical sounds. Forty children learned to create musical sound sequences using an interactive sound making environment. Half the children used a version of the system that instantiated a body-based metaphor in the mapping layer connecting body movements to output sounds. The remaining children used a version of the same environment that did not instantiate a metaphor in the mapping layer. In general, children were able to more accurately demonstrate sound sequences in the embodied metaphor based system version. However, we observed that children often resorted to spatial rather than body-based metaphors and that the mapping must be easily discoverable as well as metaphorical to provide benefit.

'nite_aura' has been developed as both an artistic work and an instrument for research into immersive environments. 'nite_aura' is an audio-visual, interactive installation exploring physical motion within an alternative immersive environment. The enchantment of the night's sky has inspires flights of imagination and whispered wishes in humans of all ages. 'nite_aura' embodies these memories and experiences allowing for visitors to whisper to the air and play with fields of stars surrounding them in an interactive immersive art installation focusing on sensual and physical interaction. The work investigates the effect of the texture of space, light and sound in providing a comforting, relaxing, and meditative quality of immersion.