The use of wearable sensor technology opens up exciting avenues for both art and HCI research, providing new ways to explore the invisible link between audience and performer. To be effective, such work requires close collaboration between performers and researchers. In this article, we report on the co-design process and research insights from our work integrating physiological sensing and live performance. We explore the connection between the audience’s physiological data and their experience during the performance, analyzing a multi-modal dataset collected from 98 audience members. We identify notable moments based on HRV and EDA, and show how the audience’s physiological responses can be linked to the choreography. The longitudinal changes in HRV features suggest a strong connection to the choreographer’s intended narrative arc, while EDA features appear to correspond with short-term audience responses to dramatic moments. We discuss the physiological phenomena and implications for designing feedback systems and interdisciplinary collaborations.

Ethereal Phenomena is an interactive installation in which a user meditates in front of a digital illustration based on Tibetan thangka art that reacts to the way they breathe. The biofeedback of the breath makes the artwork an extension of the user’s body and a reflection of their mind. By combining art, sound, and biofeedback, Ethereal Phenomena becomes a unique meditative experience that aims at producing physical and mental wellness and that has the potential to induce a state of self-transcendence or pure consciousness.

What we wear (our clothes and wearable accessories) can represent our mood at the moment. We developed Emolleia to explore how to make aesthetic wears more expressive to become a novel form of non-verbal communication to express our emotional feelings. Emolleia is an open wearable kinetic display in form of three 3D printed flowers that can dynamically open and close at different speeds. With our open-source platform, users can define their own animated motions. In this paper, we described the prototype design, hardware considerations, and user surveys (n=50) to evaluate the expressiveness of 8 pre-defined animated motions of Emolleia. Our initial results showed animated motions are feasible to communicate different emotional feelings especially at the valence and arousal dimensions. Based on the findings, we mapped eight pre-defined animated motions to the reported, user-perceived valence, arousal and dominance and discussed possible directions for future work.

We present our first steps towards an umbrella-based novel multimedia platform using physiological data as an integrated feedback loop. In this paper, we demonstrate the viability of using an umbrella handle as a form factor to measure electrodermal activity(EDA) and heart rate(HR) in real-time. We compared the performance of the device with that of a more conventional finger sensor placement. Although the finger sensor placement is more widespread and considered to be more reliable, yet we are able to derive meaningful data from the umbrella handle in both stationary and dynamic contexts in the presented feasibility study.

Insufficient social cues between distributed learners in online learning could result in lack of engagement and social bonds. With the development of wearable sensing, sharing physiological data can be used to enhance mutual understanding and connectedness among sharers. Our work aims to explore the potential of sharing heart rate (HR) and heart rate variability (HRV) collected from distributed learners to enhance their online learning experiences. We implemented a physiological streaming system and conducted a field study with 11 learners in online classes. This paper describes the study and discusses our interview findings by contrasting the influence of visualized collective physiological data from viewpoints of data contributors and viewers. Our exploratory results suggest streaming collective HR and HRV from multiple distributed learners could be used in online classes to improve engagement and sense of community.

In stage performances, an invisible wall in front of the stage often weakens the connections between the audience and performers. To amplify this performative connection, we present the concept ”Boiling Mind”. Our design concept is based on streaming sensor data related to heart and electrodermal activities from audience members and integrating this data into staging elements, such as visual projections, music, and lighting. Thus, the internal states of the audience directly influence the staging. Artists can have a more direct perception of the inner reactions of audience members and can create physical expressions in response to them. In this paper, we present the wearable sensing system as well as design considerations of mapping heart and electrodermal activity to changes in the staging elements. We evaluated our design and setup over three live performances.