The ’Footsteps Illusion’ shows that pitch-altering footstep sounds in real-time affects body perception, gait, and emotion: high versus low frequencies evoke a lighter versus heavier body. We tested whether this illusion extends beyond the lab, where environmental factors matter. Using a mixed-methods approach, twenty-eight participants used a minimal setup to synchronize with three prerecorded pitch-altered footstep soundtracks: Control, High-Frequency, and Low-Frequency. In Experiment 1, they walked a fixed path, with gait recordings, questionnaires, and body visualizations. In Experiment 2, they walked freely across campus routes with High/Low-Frequency soundtracks, followed by post-walk interviews using a novel spatial mapping tool linking body sensations to context. Results replicate the illusion outdoors, showing synchronization with prerecorded sounds as a viable alternative to real-time feedback, and revealing that contextual factors modulate illusion effects. We contribute insights and novel tools (prototype, spatial mapping method, and footstep soundbase) for studying multisensory body perception in everyday contexts.

Changes in body perception influence behavior and emotion and can be induced through multisensory feedback. Auditory feedback to one’s actions can trigger such alterations; however, it is unclear which individual factors modulate these effects. We employ and evaluate SoniWeight Shoes, a wearable device based on literature for altering one’s weight perception through manipulated footstep sounds. In a healthy population sample across a spectrum of individuals (n=84) with varying degrees of eating disorder symptomatology, physical activity levels, body concerns, and mental imagery capacities, we explore the effects of three sound conditions (low-frequency, high-frequency and control) on extensive body perception measures (demographic, behavioral, physiological, psychological, and subjective). Analyses revealed an impact of individual differences in each of these dimensions. Besides replicating previous findings, we reveal and highlight the role of individual differences in body perception, offering avenues for personalized sonification strategies. Datasets, technical refinements, and novel body map quantification tools are provided.

In this article, we evaluateb-Reactable, a digital music instrument that combines implicit physiology-based interaction through EEG and ECG, and explicit gestural interaction for sound generation and control. This multimodality is embodied in tangible objects namedphysiopucks,which are driven by biosignals. We hypothesize that multimodality increases users’ motivation in a musical task, compared to the use of a gesture-only tabletop system (the Reactable). We compared motivational aspects in dyads collaborating in three experimental groups (N = 56): the Physio group (one physiology- and one gesture-based user), the Sham group (one prerecorded physiology- and one gesture-based user), and the Control group (two gesture users). Between-group comparisons showed that motivation dimensions of Confidence and Satisfaction were higher inb-Reactablethan in the gesture-only tangible interface, and that fake physiology-based feedback significantly reduced these effects. Our study also shows the potential of combined implicit and explicit interaction modes in multiuser HCI scenarios.

Humans have the ability to use a complex code of non-verbal behavior to communicate their internal states to others. Conversely, the understanding of intentions and emotions of others is a fundamental aspect of human social interaction. In the study presented here we investigate how people perceive the expression of emotional states based on the observation of different styles of locomotion. Our goal is to find a small set of canonical parameters that allow to control a wide range of emotional expressions. We generated different classes of walking behavior by varying the head/torso inclination, the walking speed, and the viewing angle of an animation of a virtual character. 18 subjects rated the observed walking person using the two-dimensional circumplex model of arousal and valence. The results show that, independent of the viewing angle, participants perceived distinct states of arousal and valence. Moreover, we could show that parametrized body posture codes emotional states, irrespective of the contextual influence or facial expressions. These findings suggest that human locomotion transmits basic emotional cues that can be directly related to canonical parameters of different dimensions of the expressive behavior. These findings are important as they allow us to build virtual characters whose emotional expression is recognizable at large distance and during extended periods of time.

While rotating visual and auditory stimuli have long been known to elicit self-motion illusions (“circular vection”), audiovisual interactions have hardly been investigated. Here, two experiments investigated whether visually induced circular vection can be enhanced by concurrently rotating auditory cues that match visual landmarks (e.g., a fountain sound). Participants sat behind a curved projection screen displaying rotating panoramic renderings of a market place. Apart from a no-sound condition, headphone-based auditory stimuli consisted of mono sound, ambient sound, or low-/high-spatial resolution auralizations using generic head-related transfer functions (HRTFs). While merely adding nonrotating (mono or ambient) sound showed no effects, moving sound stimuli facilitated both vection and presence in the virtual environment. This spatialization benefit was maximal for a medium (20° × 15°) FOV, reduced for a larger (54° × 45°) FOV and unexpectedly absent for the smallest (10° × 7.5°) FOV. Increasing auralization spatial fidelity (from low, comparable to five-channel home theatre systems, to high, 5° resolution) provided no further benefit, suggesting a ceiling effect. In conclusion, both self-motion perception and presence can benefit from adding moving auditory stimuli. This has important implications both for multimodal cue integration theories and the applied challenge of building affordable yet effective motion simulators.