Advances in emerging technologies, such as on-body mechanical actuators and electrical muscle stimulation, have allowed computers to take control over our bodies. This presents opportunities as well as challenges, raising fundamental questions about agency and the role of our bodies when interacting with technology. To advance this research field as a whole, we brought together expert perspectives in a week-long seminar to articulate the grand challenges that should be tackled when it comes to the design of computers’ control over our bodies. These grand challenges span technical, design, user, and ethical aspects. By articulating these grand challenges, we aim to begin initiating a research agenda that positions bodily control not only as a technical feature but as a central, experiential, and ethical concern for future human–computer interaction endeavors.

Sounds play a crucial role in shaping dining experiences. Recently, designers have increasingly integrated them into interfaces that connect diners with food. Yet, little is known about how sounds can function as culinary materials to enrich chefs’ creativity, particularly in creating meaningful auditory interactions that resonate with their culinary creations. Hence, we present GastroConcerto, an auditory dining system that combines a magnetic contact microphone equipped under the plate with a companion mobile application. This system delves into the interactive space between diners and their food to introduce a novel interactive mechanism of dining–sound pairing, enabling chefs to design specific auditory interactions that respond to diners’ individual interactions with the food on dining containers. Through a within-subjects study and field deployment, we examined how GastroConcerto enriches chefs’ creative practices in crafting "sonic dish" experiences. Ultimately, our goal is to shift the ownership of auditory interaction design from interface designers to chefs, thereby supporting their culinary creativity.

Drinking is an inherently multisensory activity, yet the potential of immersive technology to dynamically shape flavor experiences remains underexplored in Human-Food Interaction (HFI) research. We introduce “XTea”, an adaptive beverage cup-based system that integrates large language models to translate natural language input into modifications of a parameterized immersive environment experienced through a headset when drinking bubble tea. Through a study with 12 bubble tea enthusiasts, we derived themes that demonstrate how “XTea” can enrich sensory engagement, support personalized and agentic experiences, and foster social qualities of drinking, pointing toward new explorations for multisensory HFI design. We also present four design strategies for multisensory beverage experiences. Ultimately, we aim to contribute to the advancement of HFI research on how multisensory interaction design can enrich flavor perception and engagement.

Physical augmentation technologies can extend human abilities beyond biological limitations, creating "superpower" experiences. However, as these technologies integrate with the human body, they can also introduce unfortunate negative effects due to the body’s constraints. Inspired by biology, especially the fly-catching mechanism of the flytrap plant, we designed "Flytrap Hand", a provocative artifact that grants users the superpower of accelerated grasping through a distance sensor and electrical muscle stimulation, while resulting in uncertainty. A mixed-method study (N = 12) revealed that participants appreciated the fortunate superpower effects, but also highlighted the unfortunate effects of discomfort, skepticism toward bodily automation, and ethical concerns. Building on these findings, we propose a design framework for designing more thoughtful superpower experiences that account for potential negative effects. Ultimately, with our work, we aim to help designers consider not only superpowers’ positive but also negative effects early on as they may be difficult to rectify in hindsight.

Designing superpowers in Human-Computer Interaction (HCI), often inspired by science fiction, has garnered increased attention. However, it is important to ask whether such superpower designs might have inherent negative side effects, especially considering that technological advances allow going beyond short demos to integrate these superpowers into everyday life. To understand the positive and negative side effects of superpower design, we created "EmoPals"and studied it in everyday life. EmoPals is a novel system inspired by telepathy, where one user's emotions are detected through a brain-computer interface and replicated on the other user's face through electrical muscle stimulation, therefore one user's happiness makes the other smile and vice versa. A 5-day field study with 12 participants suggests that EmoPals can strengthen emotional connections and facilitate empathy, however, it also highlights the negative side effects of amplifying negative emotions and social discomfort. We propose five design recommendations for designing superpowers that account for negative side effects. Ultimately, we aim to deepen our understanding of superpower design for everyday life.

DIS ’25, Funchal, Portugal

Figure 1: We review and categorize VMIs aimed at enhancing context awareness.Our key contribution is a Macro-Micro-Macro level (whole-detail-whole) system design framework, providing actionable references from a Data Modality-Driven perspective: (1) Macro-level contextual factors: considerations for context understanding (Section 3); (2) Micro-level system foundations: input data modality (visual + other modalities), data integration stages, multimodal data processing and evaluation strategies (Sections 4, 5); (3) Macro-level design synthesis: application domains, design considerations and key challenges (Sections 6, 7).

Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.

Traditional games like "Tag" rely on shared control via inter-body interactions (IBIs) – touching, pushing, and pulling – that foster emotional and social connection. Digital games largely limit IBIs, with players using their bodies as input to control virtual avatars instead. Our “Shared Bodily Fusion” approach addresses this by fusing players’ bodies through a mediating computer, creating a shared input and output system. We demonstrate this approach with "Hidden Touch", a game where a novel social electrical muscle stimulation system transforms touch (input) into muscle actuations (output), facilitating IBIs. Through a study (n=27), we identified three player experience themes. Informed by these findings and our design process, we mapped their trajectories across our three experiential spaces – threshold, tolerance, and precision – which collectively form our design framework. This framework facilitates the creation of future digital games where IBIs are intrinsic, ultimately promoting the many benefits of social play.

The field of Sports Human-Computer Interaction (SportsHCI) investigates interaction design to support a physically active human being. Despite growing interest and dissemination of SportsHCI literature over the past years, many publications still focus on solving specific problems in a given sport. We believe in the benefit of generating fundamental knowledge for SportsHCI more broadly to advance the field as a whole. To achieve this, we aim to identify the grand challenges in SportsHCI, which can help researchers and practitioners in developing a future research agenda. Hence, this paper presents a set of grand challenges identified in a five-day workshop with 22 experts who have previously researched, designed, and deployed SportsHCI systems. Addressing these challenges will drive transformative advancements in SportsHCI, fostering better athlete performance, athlete-coach relationships, spectator engagement, but also immersive experiences for recreational sports or exercise motivation, and ultimately, improve human well-being.

People with a fear of being in water rarely engage in water activities and hence miss out on the associated health benefits. Prior research suggested virtual exposure to treat fears. However, when it comes to a fear of being in water, virtual water might not capture water’s immersive qualities, while real water can pose safety risks. We propose extended reality to combine both advantages: We conducted a study (N=12) where participants with a fear of being in water interacted with playful water-inspired virtual reality worlds while floating inside a floatation tank. Our findings, supported quantitatively by heart rate variability and qualitatively by interviews, suggest that playful extended reality could mitigate fear responses in an entertaining way. We also present insights for the design of future systems that aim to help people with a fear of being in water and other phobias by using the best of the virtual and physical worlds.

Recent combinations of interactive technology, humans, and water have resulted in “WaterHCI”. WaterHCI design seeks to complement the many benefits of engagement with the aquatic domain, by offering, for example, augmented reality systems for snorkelers, virtual reality in floatation tanks, underwater musical instruments for artists, robotic systems for divers, and wearables for swimmers. We conducted a workshop in which WaterHCI experts articulated the field's grand challenges, aiming to contribute towards a systematic WaterHCI research agenda and ultimately advance the field.

Underlying humanity’s social abilities is the brain’s capacity to interpersonally synchronize. Experimental, lab-based neuropsychological studies have demonstrated that inter-brain synchrony can be technologically mediated. However, knowledge in deploying these technologies in-the-wild and studying their user experience, an area HCI excels in, is lacking. With advances in mobile brain sensing and stimulation, we identify an opportunity for HCI to investigate the in-the-wild augmentation of inter-brain synchrony. We designed “PsiNet,” the first wearable brain-to-brain system aimed at augmenting inter-brain synchrony in-the-wild. Participant interviews illustrated three themes that describe the user experience of modulated inter-brain synchrony: hyper-awareness; relational interaction; and the dissolution of self. We contribute these three themes to assist HCI theorists’ discussions of inter-brain synchrony experiences. We also present three practical design tactics for HCI practitioners designing inter-brain synchrony, and hope that our work guides a HCI future of brain-to-brain experiences which fosters human connection.

This studio aims to collaboratively build foot augmentations, experiment with different materials and techniques, and create new designs for low-cost, wearable, and accessible devices that can be used by researchers, makers, designers, and artists. Considering the heightened focus on the human body with the rise of AR/VR/XR technologies, foot augmentation has great potential. To explore this potential, we invite researchers, designers and artists to share their applications, experiences, and ideas while designing foot augmentations. Participants will share knowledge, brainstorm ideas, and explore tools and materials for rapid prototyping. Finally, they will tinker and explore, discussing their design strategies to derive common approaches and best practices. Based on the hands-on session results, we will write a paper on design strategies for foot augmentation that will help facilitate more sustainable investigations and design of future foot interfaces.

Everyday, billions of people use footwear for walking, running, or exercise. Of emerging interest are “smart footwear”, which help users track gait, count steps or even analyse performance. However, such nascent footwear lack fine-grain ground surface context awareness, which could allow them to adapt to the conditions and create usable functions and experiences. Hence, this research aims to recognize the walking surface using a radar sensor embedded in a shoe, enabling ground context-awareness. Using data collected from 23 participants from an in-the-wild setting, we developed several classification models. We show that our model can detect five common terrain types with an accuracy of 80.0% and further ten terrain types with an accuracy of 66.3%, while moving. Importantly, it can detect the gait motion types such as ‘walking’, ‘stepping up’, ‘stepping down’, ‘still’, with an accuracy of 90%. Finally, we present potential use cases and insights for future work based on such ground-aware smart shoes.

User engagement in Virtual Reality (VR) games is crucial for creating immersive and captivating gaming experiences that meet the expectations of players. However, understanding and measuring these levels in VR games presents a challenge for game designers, as current methods, such as self-reports, may be limited in capturing the full extent of user engagement. Additionally, approaches based on biological signals to measure engagement in VR games present complications and challenges, including signal complexity, interpretation difficulties, and ethical concerns. This study explores body movements, as a novel approach to measure user engagement in VR gaming. We employ E4, emteqPRO, and off-the-shelf IMUs to measure the body movements from diverse participants engaged in multiple VR games. Further, we examine the simultaneous occurrence of player motivation and physiological responses to explore potential associations with body movements. Our findings suggest that body movements hold promise as a reliable and objective indicator of user engagement, offering game designers valuable insights on generating more engaging and immersive experiences.

Emotional Self-Awareness (ESA) plays a vital role in physical and mental well-being. Recent advancements in artificial intelligence technologies have shown promising emotion recognition results, opening new opportunities to build systems to support ESA. However, little research has been done to understand users' perspectives on artificial-intelligence-based emotion recognition systems. We introduce Troi, an automatic emotion recognition mobile app using wearable signals. With Troi, we ran a multi-day user study with 12 users to understand user preference parameters, such as perceived accuracy, confidence, preferred emotion representations, effect of self-awareness of emotions, and real-time use cases. Further, we extend our study to evaluate the machine learning model in-the-wild to understand behaviours in-the-wild. We found that users perceived accuracy of the emotion recognition model is higher than the actual model prediction accuracy; there was no strong preference for one specific emotion representation, and users' self-awareness of emotions improved over time.

Tickling is a type of sensation that is associated with laughter, smiling, or other similar reactions. Psychology research has shown that tickling and laughter can significantly relieve stress. Although several tickling artifacts have been suggested in prior work, limited knowledge is available if those artifacts could evoke laughter. In this article, we aim at filling this gap by designing and developing a novel foot-tickling mechanism that can evoke laughter. We first developed an actuator that can create tickling sensations along the sole of the foot utilising magnet-driven brushes. Then, we conducted two studies to identify the most ticklish locations of the foot’s sole and stimulation patterns that can evoke laughter. In a follow-up study with a new set of participants, we confirmed that the identified stimuli could evoke laughter. From the participants’ feedback, we derived several applications that such a simulation could be useful. Finally, we embedded our actuators into a flexible insole, demonstrating the potential of a wearable tickling insole.

Self-Tracking stress at work is an important aspect of stress management and is often the first step to improving mental health and personal well-being. We introduce StressShoe, a DIY toolkit for self-tracking stress and just-in-time stress interventions for office workers performing sedentary tasks. Informed by a focus group study and a pilot study, we designed and tested a toolkit that consists of a single, off-the-shelf, shoe-mounted inertial measurement unit (IMU), a machine learning model that estimates acute stress, and a companion mobile app. The mobile app allows for user-defined just-in-time interventions on the estimated stress level. To demonstrate the benefits of our system we evaluated StressShoe with 10 users over 4 weeks. We identified several effective just-in-time personalised intervention users created and found that StressShoe is easy to use and helped them reflect on their daily stressful experiences.

We present a wearable forearm augmentation that enables the recreation of natural touch sensation by applying shear-forces onto the skin. In contrast to previous approaches, we arrange light-weight and stretchable 3x3cm plasters in a matrix onto the skin. Individual plasters were embedded with lines of shape-memory alloy (SMA) wires to generate shear-forces. Our design is informed by a series of studies investigating the perceptibility of different sizes, spacings, and attachments of plasters on the forearm. Our matrix arrangement enables the perception of touches, for instance, feeling ones wrist being grabbed or the arm being stroked. Users rated the recreated touch sensations as being fairly similar to a real touch (4.1/5). Even without a visual representation, users were able to correctly distinguish them with an overall accuracy of 94.75%. Finally, we explored two use cases showing how AR and VR could be empowered with experiencing recreated touch sensations on the forearm.

The correct execution of exercises, such as squats and dead-lifts, is essential to prevent various bodily injuries. Existing solutions either rely on expensive motion tracking or multiple Inertial Measurement Units (IMU) systems require an extensive set-up and individual calibration. This paper introduces a proof of concept, GymSoles, an insole prototype that provides feedback on the Centre of Pressure (CoP) at the feet to assist users with maintaining the correct body posture, while performing squats and dead-lifts. GymSoles was evaluated with 13 users in three conditions: 1) no feedback, 2) vibrotactile feedback, and 3) visual feedback. It has shown that solely providing feedback on the current CoP, results in a significantly improved body posture.

Currently, smartwatches are equipped with Photoplethysmography (PPG) sensors to measure Heart Rate (HR) and Heart Rate Variability (HRV). However, PPG sensors consume considerably high energy, making it impractical to monitor HR & HRV continuously for an extended period. Utilising low power accelerometers to estimate HR has been broadly discussed in previous decades. Inspired by prior work, we introduce CompRate, an alternative method to measure HR continuously for an extended period in low-intensity physical activities. CompRate model calibrated for individual users only has an average performance of Root Mean Squared Error (RMSE) 1.58 Beats Per Minute (BPM). Further, CompRate used 3.75 times less energy compared to the built-in PPG sensor. We also demonstrate that CompRate model can be extended to predict HRV. We will demonstrate CompRate in several application scenarios: self-awareness of fatigue and just-in-time interruption while driving; enabling teachers to be aware of students’ mental effort during a learning activity; and the broadcasting of the location of live victims in a disaster situation.

Rhythm is the first musical concept deaf people learn in music classes. However, hearing loss limits the amount of information that allows a deaf person to evaluate his or her performance and stay in sync with other musicians. In this paper, we investigated how a visual and vibrotactile music-sensory-substitution device, MuSS-Bits++, affects rhythm discrimination, reproduction, and expressivity of deaf people. We conducted a controlled study with 11 deaf children and found that most participants felt more confident wearing the device in vibration mode even when it did not objectively improve their accuracy. Furthermore, we studied how MuSS-Bits++ can be used in music classes at deaf schools and what challenges and opportunities arise in such a setting. Based on these studies, we discuss insights and future directions that support the design and development of music-sensory-substitution systems for music making.