The ’Footsteps Illusion’ shows that pitch-altering footstep sounds in real-time affects body perception, gait, and emotion: high versus low frequencies evoke a lighter versus heavier body. We tested whether this illusion extends beyond the lab, where environmental factors matter. Using a mixed-methods approach, twenty-eight participants used a minimal setup to synchronize with three prerecorded pitch-altered footstep soundtracks: Control, High-Frequency, and Low-Frequency. In Experiment 1, they walked a fixed path, with gait recordings, questionnaires, and body visualizations. In Experiment 2, they walked freely across campus routes with High/Low-Frequency soundtracks, followed by post-walk interviews using a novel spatial mapping tool linking body sensations to context. Results replicate the illusion outdoors, showing synchronization with prerecorded sounds as a viable alternative to real-time feedback, and revealing that contextual factors modulate illusion effects. We contribute insights and novel tools (prototype, spatial mapping method, and footstep soundbase) for studying multisensory body perception in everyday contexts.

Researching body experiences poses challenges due to their inherently subjective and intangible nature. In TEI, more broadly in HCI, methods like body maps capture these experiences. We advance body maps by developing and applying two extensions to body maps to address important dimensions of body experience. These were tested in two studies. The first focuses on design processes aimed at achieving more generalizable data through quantitative methods, while enabling analysis from an individual perspective. Here, we introduce a segmentation approach of body maps for quantifying and comparing body experiences. These maps feature subdivisions of various body areas, derived from a study reporting the effects of movement sonification intervention on body perception. The segmentation allows for precise measurement and comparison of these changes. In the second, we shift to the context in which body experiences take place, presenting context-sensitive body maps anchored in the environment. These maps explore the relationship between body and environment. We present methodologies for researching embodied interactions within TEI.

Understanding the relationship between movement variability and body image is essential in disciplines where motor control and body perception are closely linked, such as dance. The present study with 24 ballet dancers presents a computational approach for detecting moments of loss of control in elbow movement fluidity. This serves as an objective proxy for body image and body awareness, as subtle disruptions in movement, especially in joints that are not the primary focus of technical training, can reflect increased self-monitoring or discomfort, factors commonly associated with negative body image in dancers. Through movement analysis during two common rehearsal contexts (i.e., marking choreography and full execution with music) we examined how movement variability differs based on task demands. Our findings show that loss of control moments are more frequent during marking, underscoring the influence of rehearsal context on both motor performance and body-related self-perception. Furthermore, we demonstrate that movement instability in the elbow joint can be associated with self-reported measures of body image and awareness, reinforcing the connection between motor variability and psychological well-being. By identifying the elbow as a sensitive and reliable indicator of instability during ballet practice, our approach offers a lightweight alternative to full-body kinematic analysis, supporting practical applications beyond laboratory settings. From a Human-Computer Interaction (HCI) and Ubiquitous Computing (Ubicomp) perspective, this work contributes design insights for systems that integrate body image and movement behavior metrics. These findings open new possibilities for interactive technologies aimed at enhancing body image and improving movement precision in dancers and other movement practitioners. • A computational approach to detect moments of loss of control in elbow movement fluidity, offering an objective proxy for assessing body image factors during ballet practice. • Loss of control differs between marking and choreography execution with music. • The isolation of the elbow joint can reliably capture motor variability and psychological influences during ballet practice, providing an efficient alternative to full-body assessments. • Key insights into how moments of loss of control in movement relate to body image disturbances.

Abstract Movement-based design methods have gained increased attention across various research fields and practices, particularly in interaction design. By engaging the body in movement, these methods have the potential to explore a richer, more intuitive, and immersive user experience. A recent project MeCaMinD1 brought together researchers from interaction and sports design to explore, collect, and document movement-based methods and make them practically useful across domains. The methods were explored through a series of workshops, furthering the team’s understanding of their use and practical applicability. This understanding was compiled into a set of design cards that we present here. We discuss the experience of using the resulting cards in movement-based design sessions. We found that while the cards are mostly used in designing, planning, and preparing sessions, they also function as support during the design sessions, used by both facilitators and participants. Based on a final ideation session with both novice and experienced facilitators, we sketch ways to support managing the cards during sessions, integrating them with movement and physical action.

Movement-based design foregrounds the moving and sentient body, fostering holistic engagement with the surrounding physical, material, and sociospatial contexts. Over the years, this approach has yielded multiple methodologies, tools, and exemplars to support body-based ideation. In this studio, we explore tools that facilitate embodied thinking and the creative processes of designing with, through, and for the body. In particular, we focus on: i) embodied ideation tools, kits, and technology probes to prompt ideation; ii) sketching and other documentation techniques to materialize ephemeral embodied action during ideation. We will bring tools and techniques to engage with both aspects, and we will invite participants to bring their own, which can be physical or technological, low or high-fidelity. This hands-on studio will provide a space to collectively engage in embodied ideation and sketching; exploring, analysing, and engaging deeply with the available objects and methods. The studio will culminate in a rich set of visual material and an annotated portfolio, which will be shared with the broader community, fostering connections among designers interested in movement-based and tangible design.

We present the design and evaluation of the Movits, a minimalist toolkit for embodied sketching design explorations. The toolkit includes technology probes featuring minimalist wearable digital units that support the hands-on exploration and design of movement-driven interactions using multisensory feedback. The Movits are self-contained and generate audiovisual or vibrotactile patterns in response to movement-based inputs. We present the theoretical and empirical grounding driving our design process. We discuss the findings of using the Movits during four co-design workshops with design students, technologists, dancers and physiotherapists, where they resulted in being generative and adaptable to a range of embodied design approaches. We contend that the Movits can be favourable for those interested in a holistic design approach to wearables in general and specifically for those targeting movement-based application domains. © The Authors, 2024. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in DIS '24: Proceedings of the 2024 ACM Designing Interactive Systems Conference, https://dx.doi.org/10.1145/3643834

Sensory technologies alter how we perceive our body, which can have profound implications for multiple domains. Prior work has contributed a myriad of artefacts and evaluative studies, but we still lack design knowledge to design meaningful body perception alterations facilitated by sensory technologies. To address this gap, we draw from soma design to propose a methodological approach centered on body sensations as design material. We articulate our approach based on a project on co-designing wearables to alter body perception together with professional dancers. Our approach involves engaging participants in articulating and sharing body sensations to others, and exploring somatically sensory stimuli to co-design concepts for future technologies. We contribute experiential facets of body sensations, movement and sensory potentials to alter sensations, methods and design strategies, and a collection of ideas. Our work can be relevant to design communities interested in sensory technologies, perceptual alterations and body sensations as material.

Changes in body perception influence behavior and emotion and can be induced through multisensory feedback. Auditory feedback to one’s actions can trigger such alterations; however, it is unclear which individual factors modulate these effects. We employ and evaluate SoniWeight Shoes, a wearable device based on literature for altering one’s weight perception through manipulated footstep sounds. In a healthy population sample across a spectrum of individuals (n=84) with varying degrees of eating disorder symptomatology, physical activity levels, body concerns, and mental imagery capacities, we explore the effects of three sound conditions (low-frequency, high-frequency and control) on extensive body perception measures (demographic, behavioral, physiological, psychological, and subjective). Analyses revealed an impact of individual differences in each of these dimensions. Besides replicating previous findings, we reveal and highlight the role of individual differences in body perception, offering avenues for personalized sonification strategies. Datasets, technical refinements, and novel body map quantification tools are provided.

Inspired by the strong concept of Intercorporeal Biofeedback, here we present the design and development of a minimalist embodied sketching toolkit for designing wearables for motor learning. The toolkit includes technology probes featuring minimalist wearable digital units that support hands-on explorations and the design of potential future interactions driven by movement with multisensory feedback. These units are self-contained and generate audiovisual or vibrotactile patterns in response to body inputs such as movement, spatial orientation, and touch. Here, we present and characterise the toolkit, together with its theoretical and empirical grounding and the design values driving the design process. The toolkit can be useful for those interested in an embodied design approach to designing wearables in general, and specifically for those targeting movement, such as in motor learning application domains. It can complement and be used with other assorted non-digital objects during Embodied Sketching sessions, like bodystorming.

Body perception has a significant impact on people’s motor, emotional, and social functioning. We evaluated the potential of four different existing wearable prototypes, which provide sound or haptic bodily feedback to alter body perception. In a Research through Design workshop, we invited professional dancers as expert study participants to explore and assess our prototypes. Based on the workshop’s insights, we articulate the experiential quality of Body Transformation, which characterizes how dancers perceive and experience their bodies while interacting with such prototypes. The quality encompasses a perceptual and holistic transformation, impacting the feelings about body, movement and emotions, and where the sensory feedback’s evocative power is crucial. Additionally, it elicits different transformation valuations. We contribute a deeper understanding of the impact of sensory feedback on body perception, its potential for transforming people’s overall body experience, and methodological insights on the potential of working with dancers to evaluate wearable sensory technology.

Wearables integrating movement sonification can support body-perception changes and related physical activity; yet, we lack design principles for such sonifications. Through two mixed-methods studies, we investigate sound pitch and movement direction interaction effects on self-perception during squat exercises. We measured effects on body perception, affective quality of the experience, and actual and perceived movement, and compared them with two control conditions: no-sound and vibrotactile feedback. Results show that regardless of movement direction, ascending pitch enhances several body feelings and overall experience quality, while descending pitch increases movement acceleration. These effects were moderated by exercise physical demand. Sound and vibrotactile feedback enhanced flexibility and strength feelings, respectively, and contributed to exercise completion in different ways. Sound was perceived as an internal-to-body force while vibrotactile feedback was perceived as an external-to-body force. Feedback effects were stronger in people with lower fitness levels. We discuss results in terms of malleability of body perceptions and highlight opportunities to support demanding physical activity through wearable devices.

Movement-based design methods are increasingly adopted to help design rich embodied experiences. While there are well-known methods in the field, there is no systematic overview to help designers choose among them, adapt them, or create their own. We collected 41 methods used by movement design researchers and employed a practice-based, bottom-up approach to analyze and characterize their properties. We found 17 categories and arranged them into five main groups: Design Resources, Activities, Delivery, Framing, and Context. In this paper, we describe these groups in general and then focus on Design Resources containing the categories of Movement, Space, and Objects. We ground the characterization with examples from empirical material provided by the design researchers and references to previous work. Additionally, we share recommendations and action points to bring these into practice. This work can help novice and seasoned design researchers who want to employ movement-based design methods in their practice.

Negative body perceptions are a major predictor of physical inactivity, a serious health concern. Sensory feedback can be used to alter such body perceptions; movement sonification, in particular, has been suggested to affect body perception and levels of physical activity (PA) in inactive people. We investigated how metaphorical sounds impact body perception and PA. We report two qualitative studies centered on performing different strengthening/flexibility exercises using SoniBand, a wearable that augments movement through different sounds. The first study involved physically active participants and served to obtain a nuanced understanding of the sonifications’ impact. The second, in the home of physically inactive participants, served to identify which effects could support PA adherence. Our findings show that movement sonification based on metaphors led to changes in body perception (e.g., feeling strong) and PA (e.g., repetitions) in both populations, but effects could differ according to the existing PA-level. We discuss principles for metaphor-based sonification design to foster PA.

Natural interaction with virtual objects relies on two major technology components: hand tracking and hand-object physics simulation. There are functional solutions for these two components, but their hand representations may differ in size and skeletal morphology, hence making the connection non-trivial. In this paper, we introduce a pose retargeting strategy to connect the tracked and simulated hand representations, and we have formulated and solved this hand retargeting as an optimization problem. We have also carried out a user study that demonstrates the effectiveness of our approach to enable fine manipulations that are slow and awkward with naïve approaches.

Physical inactivity is an increasing problem. It has been linked to psychological and emotional barriers related to the perception of one's body, such as physical capabilities. It remains a challenge to design technologies to increase physical activity in inactive people. We propose the use of a sound interactive system where inputs from movement sensors integrated in shoes are transformed into sounds that evoke body sensations at a metaphorical level. Our user study investigates the effects of various gesture-sound mappings on the perception of one's body and its movement qualities (e.g. being flexible or agile), the related emotional state and movement patterns, when people performed two exercises, walking and thigh stretch. The results confirm the effect of the “metaphor” conditions vs. the control conditions in feelings of body weight; feeling less tired and more in control; or being more comfortable, motivated, and happier. These changes linked to changes in affective state and body movement. We discuss the results in terms of how acting upon body perception and affective states through sensory feedback may in turn enhance physical activity, and the opportunities opened by our findings for the design of wearable technologies and interventions in inactive populations.

Supporting exercise adherence through technology remains an important HCI challenge. Recent works showed that altering walking sounds leads people perceiving themselves as thinner/lighter, happier and walking more dynamically. While this novel approach shows potential for physical activity, it raises critical questions impacting technology design. We ran two studies in the context of exertion (gym-step, stairs-climbing) to investigate how individual factors impact the effect of sound and the duration of the after-effects. The results confirm that the effects of sound in body-perception occur even in physically demanding situations and through ubiquitous wearable devices. We also show that the effect of sound interacted with participants' body weight and masculinity/femininity aspirations, but not with gender. Additionally, changes in body-perceptions did not hold once the feedback stopped; however, body-feelings or behavioural changes appeared to persist for longer. We discuss the results in terms of malleability of body-perception and highlight opportunities for supporting exercise adherence.

In self-directed chronic pain physical rehabilitation it is important that the individual can progress as physical capabilities and confidence grow. However, people with chronic pain often struggle to pass what they have identified as safe boundaries. At the same time, over-activity due to the desire to progress fast or function more normally, may lead to setbacks. We investigate how musically-informed movement sonification can be used as an implicit mechanism to both avoid overdoing and facilitate progress during stretching exercises. We sonify an end target-point in a stretch exercise, using a stable sound (i.e., where the sonification is musically resolved) to encourage movements ending and an unstable sound (i.e., musically unresolved) to encourage continuation. Results on healthy participants show that instability leads to progression further beyond the target-point while stability leads to a smoother stop beyond this point. We conclude discussing how these findings should generalize to the CP population.

Chronic (persistent) pain (CP) affects 1 in 10 adults; clinical resources are insufficient, and anxiety about activity restricts lives. Technological aids monitor activity but lack necessary psychological support. This article proposes a new sonification framework, Go-with-the-Flow, informed by physiotherapists and people with CP. The framework proposes articulation of user-defined sonified exercise spaces (SESs) tailored to psychological needs and physical capabilities that enhance body and movement awareness to rebuild confidence in physical activity. A smartphone-based wearable device and a Kinect-based device were designed based on the framework to track movement and breathing and sonify them during physical activity. In control studies conducted to evaluate the sonification strategies, people with CP reported increased performance, motivation, awareness of movement, and relaxation with sound feedback. Home studies, a focus group, and a survey of CP patients conducted at the end of a hospital pain management session provided an in-depth understanding of how different aspects of the SESs and their calibration can facilitate self-directed rehabilitation and how the wearable version of the device can facilitate transfer of gains from exercise to feared or demanding activities in real life. We conclude by discussing the implications of our findings on the design of technology for physical rehabilitation.

An ever more sedentary lifestyle is a serious problem in our society. Enhancing people's exercise adherence through technology remains an important research challenge. We propose a novel approach for a system supporting walking that draws from basic findings in neuroscience research. Our shoe-based prototype senses a person's footsteps and alters in real-time the frequency spectra of the sound they produce while walking. The resulting sounds are consistent with those produced by either a lighter or heavier body. Our user study showed that modified walking sounds change one's own perceived body weight and lead to a related gait pattern. In particular, augmenting the high frequencies of the sound leads to the perception of having a thinner body and enhances the motivation for physical activity inducing a more dynamic swing and a shorter heel strike. We here discuss the opportunities and the questions our findings open.

Physical activity is important for improving quality of life in people with chronic pain. However, actual or anticipated pain exacerbation, and lack of confidence when doing physical activity, make it difficult to maintain and build towards long-term activity goals. Research guiding the design of interactive technology to motivate and support physical activity in people with chronic pain is lacking. We conducted studies with: (1) people with chronic pain, to understand how they maintained and increased physical activity in daily life and what factors deterred them; and (2) pain-specialist physiotherapists, to understand how they supported people with chronic pain. Building on this understanding, we investigated the use of auditory feedback to address some of the psychological barriers and needs identified and to increase self-efficacy, motivation and confidence in physical activity. We conclude by discussing further design opportunities based on the overall findings.