With growing research on haptic interfaces, Mediated Social Touch (MST) technologies offer the potential to record, synthesise, and reproduce (RSR) touch experiences across space and time, enabling, for instance, a hug from afar and from the past. Although much of the existing research highlights the direct benefits of these systems, such as reducing loneliness and providing emotional support, little attention has been paid to their broader sociotechnical impacts. To address this gap, we used the Future Ripples method to speculate on possible effects of MST. We conducted three workshops with 24 participants, including potential users, domain experts, and haptics researchers. Throughout these sessions, participants collectively envisioned possible future scenarios, alongside opportunities and threats, and proposed actionable responses. Our qualitative analysis organised these insights into four themes and three distinctive challenges. These findings offer haptics researchers intervention points across the RSR pipeline to inform MST design, alongside methodological insights from applying Future Ripples to MST technology.

Embodiment can enhance conversational agents, such as increasing their perceived presence. This is typically achieved through visual representations of a virtual body; however, visual modalities are not always available, such as when users interact with agents using headphones or display-less glasses. In this work, we explore auditory embodiment. By introducing auditory cues of bodily presence - through spatially localized voice and situated Foley audio from environmental interactions - we investigate how audio alone can convey embodiment and influence perceptions of a conversational agent. We conducted a 2 (spatialization: monaural vs. spatialized) x 2 (Foley: none vs. Foley) within-subjects study, where participants (n=24) engaged in conversations with agents. Our results show that spatialization and Foley increase co-presence, but reduce users' perceptions of the agent's attention and other social attributes.

Spatial tactile feedback can enhance the realism of geometry exploration in virtual reality applications. Current vibrotactile approaches often face challenges with the spatial and temporal resolution needed to render different 3D geometries. Inspired by the natural deformation of finger pads when exploring 3D objects and surfaces, we propose TactDeform, a parametric approach to render spatio-temporal tactile patterns using a finger-worn electro-tactile interface. The system dynamically renders electro-tactile patterns based on both interaction contexts (approaching, contact, and sliding) and geometric contexts (geometric features and textures), emulating deformations that occur during real-world touch exploration. Results from a user study \rr{(N=24)} show that the proposed approach enabled high texture discrimination and geometric feature identification compared to a baseline. Informed by results from a free 3D-geometry exploration phase, we provide insights that can inform future tactile interface designs.

Virtual reality has been effectively used for eliciting emotions, yet most research focuses on the intensity of affective responses rather than on how interaction influences those experiences. To address this gap, we advance a validated VR emotion-elicitation dataset through two key extensions. First, we add a new high-arousal, high-valence scene and validate its effectiveness in a within-subject study (N=24). Second, we incorporate interactive elements into each scene, creating both interactive and non-interactive versions to examine the impact of interaction on emotional responses. We evaluate interaction through a multimodal approach combining subjective ratings and physiological signals to capture both conscious and unconscious affective responses. Our evaluation study (N=84) shows that interaction not only amplifies emotions but modulates them in context, supporting coping in negative scenes and enhancing enjoyment in positive scenes. These findings highlight the potential of scene-tailored interaction for different applications, where regulating emotions is as important as eliciting them.

Wearable supernumerary robotic limbs (SRLs) sit at the intersection of human augmentation and embodied AI, promising to function as extensions of the human body. However, their movements within the intimate near-body space raise unresolved challenges for perceived safety, user control, and trust. In this paper, we present results from a Wizard-of-Oz study (n=18), where participants completed near-body collaboration tasks with SRLs to explore these challenges. We collected qualitative data through think-aloud protocols and semi-structured interviews, complemented by physiological signals and post-task ratings. Findings indicate that greater autonomy did not inherently enhance perceived safety or trust. Instead, participants identified near-body zones and paired them with clear coordination rules. They also expressed expectations for how different arm components should behave, shaping preferences around autonomy, perceived safety, and trust. Building on these insights, we introduce SRL Proxemics, a zone- and segment-level design framework showing that autonomy is not monolithic: perceived safety hinges on spatially calibrated, legible behaviors, not on autonomy level alone.

Remote VR teleoperation with supernumerary robotic limbs enables distant users to operate in another’s local space. While a shared first-person view aids hand-eye coordination, locking the guest’s camera to the host’s head can degrade comfort, embodiment, and coordination. Based on a formative study (N=10) using a virtual supernumerary robotic limbs configuration to stress-test coordination, we propose guest-driven perspective switching from a shared first-person baseline (Shared Embodied View) to two alternatives: (a) a stabilized view with guest-controlled rotation (Embedded Anchored View), and (b) a fully decoupled third-person view (Out-of-body View). We ran a user study with 24 pairs (N=48), who switched between the baseline and proposed views as task demands changed. We measured performance, embodiment, fatigue, physiological arousal, and switching behaviors. Our results reveal role-dependent trade-offs: Out-of-body View improves navigation efficiency and reduces errors, while Embedded Anchored View supports embodiment. We conclude with guidelines: use Embedded Anchored View for hand-centric adjustments, Out-of-body View for navigation and object placement, and ensure smooth transitions.

Figure 1: The consumed endurance at 60 seconds.(1) attaching 1 kg weight boosts arm exertion to 80%; (2) bare-hand only consumed 20%.A more intense red coloration on the shoulder indicates a higher level of fatigue experienced by the user.

Creating humorous content has been shown to improve an individual’s emotional well-being by decreasing stress, overcoming anxiety, and enhancing interpersonal relationships. However, it is common knowledge that a good sense of humor is not common. In this paper, we propose a natural language processing (NLP) driven collaborative tool based on appropriate incongruity theory to assist novices in writing humorous content. We use cartoon-caption writing as the use case since it is a popular method where people engage in creating humorous content. The paper describes the design of our co-authoring tool and findings from a two-part user study where (1) 20 participants used our tool to co-author cartoon captions and (2) 66 participants evaluated those captions. Our findings show that the tool helped participants to identify incongruous visual elements in the cartoon, support ideation, and expand the narrative. This resulted in co-authored captions more frequently rated funnier than those written without the tool. This approach can be appropriated to other humor generation applications including creative writing, creating memes, sketch comedy, and advertising.

There is a growing interest in sustainable fabrication approaches, including the exploration of material conservation and utilisation of waste materials. Particularly, recent work has applied organic myco-materials, made from fungi, to develop tangible, interactive devices. However, a systematic approach for 3D fabrication using myco-materials is under-explored. In this paper, we present a parametric design tool and a fabrication pipeline to grow 3D designs using the mycelia of edible fungi species, such as Reishi or Oyster mushrooms. The proposed tool is designed based on empirical results from a series of technical evaluations of the geometric and material qualities of 3D-grown myco-objects. Furthermore, the paper introduces an easy-to-replicate fabrication process that can recycle different organic waste material combinations such as sawdust and coffee grounds to grow mycelia. Through a series of demonstration applications, we identify the challenges and opportunities for working with myco-materials in the HCI context.

Embedding technology into day-to-day wearables and creating smart devices such as smartwatches and smart-glasses has been a growing area of interest. In this paper, we explore the interaction around face masks, a common accessory worn by many to prevent the spread of infectious diseases. Particularly, we propose a method of using the straps of a face mask as an input medium. We identified a set of plausible gestures on mask straps through an elicitation study (N = 20), in which the participants proposed different gestures for a given referent. We then developed a prototype to identify the gestures performed on the mask straps and present the recognition accuracy from a user study with eight participants. Our results show the system achieves 93.07% classification accuracy for 12 gestures.

For grasping, tactile stimuli to multiple fingertips are crucial for realistic shape rendering and precise manipulation. Pinching is particularly important in virtual reality since it is frequently used to grasp virtual objects. However, the interaction space of tactile feedback around pinching is underexplored due to a lack of means to provide co-located but different stimulation to finger pads. We propose a double-sided electrotactile device with a thin and flexible form factor to fit within pinched fingerpads, comprising two overlapping 3 × 3 electrode arrays. Using this new tactile interface, we define a new concept of double-sided tactile interactions with three feedback modes: (1) single-sided stimulation, (2) simultaneous double-sided stimulation, and (3) spatiotemporal double-sided stimulation. Through two user studies, we (1) demonstrate that participants can accurately discriminate between single-sided and double-sided stimulation and find a qualitative difference in tactile sensation; and (2) confirm the occurrence of apparent tactile motion between fingers and present optimal parameters for continuous or discrete movements. Based on these findings, we demonstrate five VR applications to exemplify how double-sided tactile interactions can produce spatiotemporal movement of a virtual object between fingers and enrich touch feedback for UI operation.

For the care of neonatal infants, abdominal auscultation is considered a safe, convenient, and inexpensive method to monitor bowel conditions. With the help of early automated detection of bowel dysfunction, neonatologists could create a diagnosis plan for early intervention. In this article, a novel technique is proposed for automated peristalsis sound detection from neonatal abdominal sound recordings and compared to various other machine learning approaches. It adopts an ensemble approach that utilises handcrafted as well as one and two dimensional deep features obtained from Mel Frequency Cepstral Coefficients (MFCCs). The results are then refined with the help of a hierarchical Hidden Semi-Markov Models (HSMM) strategy. We evaluate our method on abdominal sounds collected from 49 newborn infants admitted to our tertiary Neonatal Intensive Care Unit (NICU). The results of leave-one-patient-out cross validation show that our method provides an accuracy of 95.1% and an Area Under Curve (AUC) of 85.6%, outperforming both the baselines and the recent works significantly. These encouraging results show that our proposed Ensemble-based Deep Learning model is helpful for neonatologists to facilitate tele-health applications.

We introduce MyoSpring, a novel technique to fabricate customised mechanomyographic sensors with 3D printed springs. MyoSpring averts from a traditional ‘one-size-fits-all’ approach and facilitates the development of functionally customisable wrist-worn sensors, which can be used to recognise subtle finger gestures with high accuracy. We automate the design process of MyoSpring sensors with an intuitive Graphical User Interface enabling novices to quickly design and fabricate customised sensors. We conducted a user study with 15 participants and 12 gestures, including partial and full flexion finger gestures, to verify the functionality of MyoSpring. Through the evaluation, we show that 1) MyoSpring can achieve high average accuracy of finger gesture sensing of 94.11% (SD = 9.73%), 2) mechanical customisation allows MyoSpring to significantly improve the average accuracy of overall gesture sensing and 3) MyoSpring better supports partial finger flexion gesture sensing, achieving an accuracy of 91.44% (SD = 11.76%) for partial gestures.

Abdominal auscultation is a convenient, safe and inexpensive method to assess bowel conditions, which is essential in neonatal care. It helps early detection of neonatal bowel dysfunctions and allows timely intervention. This paper presents a neonatal bowel sound detection method to assist the auscultation. Specifically, a Convolutional Neural Network (CNN) is proposed to classify peristalsis and non-peristalsis sounds. The classification is then optimized using a Laplace Hidden Semi-Markov Model (HSMM). The proposed method is validated on abdominal sounds from 49 newborn infants admitted to our tertiary Neonatal Intensive Care Unit (NICU). The results show that the method can effectively detect bowel sounds with accuracy and area under curve (AUC) score being 89.81% and 83.96% respectively, outperforming 13 baseline methods. Furthermore, the proposed Laplace HSMM refinement strategy is proven capable to enhance other bowel sound detection models. The outcomes of this work have the potential to facilitate future telehealth applications for neonatal care. The source code of our work can be found at: https://bitbucket.org/chirudeakin/neonatal-bowel-sound-classification/.

We present Flowcuits, a DIY fabrication method to prototype tangible, interactive and functional electrical components by manipulating liquid metals. The prototypes afford both physical and visual interactions to demonstrate the inner working mechanics of fundamental electronic elements, which enables tangible and playful learning. The fabrication process follows simple imprinting and sealing of fluidic circuits with a 3D-printed stamp on an accessible moldable-substrates such as ‘Blu Tack’. Utilizing conductive gallium indium liquid metal, we demonstrated interactive and re-configurable electronic components such as switches, variable resistors, variable capacitors, logic gates and pressure sensors. In this paper, we present the design analogy of Flowcuits, DIY fabrication approach including a parametric 3D stamp design toolkit and results from a technical evaluation. The stamps are printed with a low-cost 3D printer and all the materials are inexpensive and reusable, enabling Flowcuits to be easily used without any advanced lab facilities.

We contribute a novel user- and activity-independent kinematics-based regressive model for continuously predicting ballistic hand movements in virtual reality (VR). Compared to prior work on end-point prediction, continuous hand trajectory prediction in VR enables an early estimation of future events such as collisions between the user’s hand and virtual objects such as UI widgets. We developed and validated our prediction model through a user study with 20 participants. The study collected hand motion data with a 3D pointing task and a gaming task with three popular VR games. Results show that our model can achieve a low Root Mean Square Error (RMSE) of 0.80 cm, 0.85 cm and 3.15 cm from future hand positions ahead of 100 ms, 200 ms and 300 ms respectively across all the users and activities. In pointing tasks, our predictive model achieves an average angular error of 4.0° and 1.5° from the true landing position when 50% and 70% of the way through the movement. A follow-up study showed that the model can be applied to new users and new activities without further training.

Eyes-free operation of mobile devices is critical in situations where the visual channel is either unavailable or attention is needed elsewhere. In such situations, vibrotactile tracing along paths or lines can help users to navigate and identify symbols and shapes without visual information. In this paper, we investigated the applicability of different metrics that can measure the effectiveness of vibrotactile line tracing methods on touch screens. In two user studies, we compare trace Length Error, Area Error, and Fréchet Distance as alternatives to commonly used trace Time. Our results show that a lower Fréchet distance is correlated better with the comprehension of a line trace. Furthermore, we show that distinct feedback methods perform differently with varying geometric features in lines and propose a segmented line design for tactile line tracing studies. We believe the results will inform future designs of eyes-free operation techniques and studies.

Rapid prototyping of haptic output on 3D objects promises to enable a more widespread use of the tactile channel for ubiquitous, tangible, and wearable computing. Existing prototyping approaches, however, have limited tactile output capabilities, require advanced skills for design and fabrication, or are incompatible with curved object geometries. In this paper, we present a novel digital fabrication approach for printing custom, high-resolution controls for electro-tactile output with integrated touch sensing on interactive objects. It supports curved geometries of everyday objects. We contribute a design tool for modeling, testing, and refining tactile input and output at a high level of abstraction, based on parameterized electro-tactile controls. We further contribute an inventory of 10 parametric Tactlet controls that integrate sensing of user input with real-time electro-tactile feedback. We present two approaches for printing Tactlets on 3D objects, using conductive inkjet printing or FDM 3D printing. Empirical results from a psychophysical study and findings from two practical application cases confirm the functionality and practical feasibility of the Tactlets approach.

Mutual capacitance-based multi-touch sensing is now a ubiquitous and high-fidelity input technology. However, due to the complexity of electrical and signal processing requirements, it remains very challenging to create interface prototypes with custom-designed multi-touch input surfaces. In this paper, we introduce Multi-Touch Kit, a technique enabling electronics novices to rapidly prototype customized capacitive multi-touch sensors. In contrast to existing techniques, it works with a commodity microcontroller and open-source software and does not require any specialized hardware. Evaluation results show that our approach enables multi-touch sensors with a high spatial and temporal resolution and can accurately detect multiple simultaneous touches. A set of application examples demonstrates the versatile uses of our approach for sensors of different scales, curvature, and materials.

Skin-based touch input opens up new opportunities for direct, subtle, and expressive interaction. However, existing skin-worn sensors are restricted to single-touch input and limited by a low resolution. We present the first skin overlay that can capture high-resolution multi-touch input. Our main contributions are: 1) Based on an exploration of functional materials, we present a fabrication approach for printing thin and flexible multi-touch sensors for on-skin interactions. 2) We present the first non-rectangular multi-touch sensor overlay for use on skin and introduce a design tool that generates such sensors in custom shapes and sizes. 3) To validate the feasibility and versatility of our approach, we present four application examples and empirical results from two technical evaluations. They confirm that the sensor achieves a high signal-to-noise ratio on the body under various grounding conditions and has a high spatial accuracy even when subjected to strong deformations.

This paper introduces Tacttoo, a feel-through interface for electro-tactile output on the user's skin. Integrated in a temporary tattoo with a thin and conformal form factor, it can be applied on complex body geometries, including the fingertip, and is scalable to various body locations. At less than 35µm in thickness, it is the thinnest tactile interface for wearable computing to date. Our results show that Tacttoo retains the natural tactile acuity similar to bare skin while delivering high-density tactile output. We present the fabrication of customized Tacttoo tattoos using DIY tools and contribute a mechanism for consistent electro-tactile operation on the skin. Moreover, we explore new interactive scenarios that are enabled by Tacttoo. Applications in tactile augmented reality and on-skin interaction benefit from a seamless augmentation of real-world tactile cues with computer-generated stimuli. Applications in virtual reality and private notifications benefit from high-density output in an ergonomic form factor. Results from two psychophysical studies and a technical evaluation demonstrate Tacttoo's functionality, feel-through properties and durability.

Recent advances in printed electronics have enabled the design and fabrication of thin, flexible and customizable interactive surfaces. These interfaces create opportunities for a variety of novel interactions leveraging on the unique form factor, flexibility and customization options. Previous research has demonstrated the possibility of customizable multi-touch sensors, conformal on-skin interfaces and printable shape changing displays. The aim of this tutorial is to acquire basic conceptual and practical skills in developing interactive surfaces with printed electronics.

This paper describes the design of "ArmSleeve", a patient monitoring system to support occupational therapists in their upper limb rehabilitation work with stroke patients. Occupational therapists can provide rehabilitation in clinics, but they have limited insights into how much their patients use their affected arm and hand in daily life, which is critical for effective recovery to occur. Our work addresses this problem through three interrelated studies: (1) interviews with therapists to examine their current rehabilitation practices; (2) the design of the "ArmSleeve Sensor" to monitor a patient's upper limb movement; and (3) the design and evaluation of the "ArmSleeve Dashboard" to visualize this information for therapists. The findings show the importance of collecting objective data to assess exercise and activities outside therapy, but also a lack of contextual information to interpret this data. We discuss considerations for how to address this issue through patient engagement as well as considerations for designing wearable sensor technology that is usable in everyday life.

In this paper we present zSense, which provides greater input expressivity for spatially limited devices such as smart wearables through a shallow depth gesture recognition system using non-focused infrared sensors. To achieve this, we introduce a novel Non-linear Spatial Sampling (NSS) technique that significantly cuts down the number of required infrared sensors and emitters. These can be arranged in many different configurations; for example, number of sensor emitter units can be as minimal as one sensor and two emitters. We implemented different configurations of zSense on smart wearables such as smartwatches, smartglasses and smart rings. These configurations naturally fit into the flat or curved surfaces of such devices, providing a wide scope of zSense enabled application scenarios. Our evaluations reported over 94.8% gesture recognition accuracy across all configurations.

PINOKY is a wireless ring-like device that can be externally attached to any plush toy as an accessory that animates the toy by moving its limbs. A user is thus able to instantly convert any plush toy into a soft robot. The user can control the toy remotely or input the movement desired by moving the plush toy and having the data recorded and played back. Unlike other methods for animating plush toys, PINOKY is non-intrusive, so alterations to the toy are not required. In a user study, 1) the roles of plush toys in the participants' daily lives were examined, 2) how participants played with plush toys without PINOKY was observed, 3) how they played with plush toys with PINOKY was observed, and their reactions to the device were surveyed. On the basis of the results, potential applications were conceptualized to illustrate the utility of PINOKY.

We present the FuwaFuwa sensor module, a round, hand-size, wireless device for measuring the shape deformations of soft objects such as cushions and plush toys. It can be embedded in typical soft objects in the household without complex installation procedures and without spoiling the softness of the object because it requires no physical connection. Six LEDs in the module emit IR light in six orthogonal directions, and six corresponding photosensors measure the reflected light energy. One can easily convert almost any soft object into a touch-input device that can detect both touch position and surface displacement by embedding multiple FuwaFuwa sensor modules in the object. A variety of example applications illustrate the utility of the FuwaFuwa sensor module. An evaluation of the proposed deformation measurement technique confirms its effectiveness.

We propose a cooking system that operates in an open environment. The system cooks a meal by pouring various ingredients into a boiling pot on an induction heating cooker and adjusts the heating strength according to the user's instructions. We then describe how the system incorporates robotic- and human-specific elements in a shared workspace so as to achieve a cooperative rudimentary cooking capability. First, we use small mobile robots instead of built-in arms to save space, improve flexibility and increase safety. Second, we use detachable visual markers to allow the user to easily configure the real-world environment. Third, we provide a graphical user interface to display detailed cooking instructions to the user. We hope insights obtained in this experiment will be useful for the design of other household systems in the future.