This paper evaluates interaction techniques to increase input accuracy with embodied devices—an emergent type of interactive system where the user’s body serves as both the input and output medium (e.g., gestural input via cameras/IMUs; gestural output via motors/muscle stimulation). A shortcoming of existing embodied devices is their failure to enforce alignment between users’ proprioceptive inputs and interface state. Thus, we present and evaluate interaction techniques that use muscle stimulation to enable embodied devices to: (1) recall previous interface states; (2) provide confirmation cues on state transitions; and (3) constrain inputs to valid ranges. In our study, participants performed pairs of interactions with an embodied slider, separated by a distraction task. The results showed that, compared to the same embodied slider without EMS, the combination of our techniques increased users’: (1) absolute input accuracy; (2) relative input accuracy; and (3) confidence.

Habitual static stretching is recommended in many ergonomics guidelines, but continuing this habit remains challenging to workers. We address two factors causing this difficulty: most workers lack an understanding of the appropriate mechanisms of stretching, and stretching sessions require the interruption of ongoing tasks. We propose Screen-Directed Stretching, a novel technique that supports neck stretching in extended reality (XR) workspaces without interrupting work. Our technique temporarily repositions the virtual screen that the user is focusing on, guiding them to rotate their head in yaw, pitch, and roll directions and to maintain a posture for a specific duration, thus facilitating muscle extension. Through two preliminary user studies, we developed a prototype that seamlessly switches the screen’s coordinate system between world-bound and body-bound frames of reference, balancing practical workability and good stretching guidance. A user study (N = 16) demonstrates that our technique effectively induces stretching movements while minimizing loss of task efficiency.

Providing tactile-feedback when users contact virtual-interfaces has been a seminal advance. However, we posit these advances have been explored in isolation from considerations of users’ physical interactions with surrounding-objects. Most touch-interfaces were designed to optimize virtual interfaces, but rarely consider that users also need to feel physical interfaces (e.g., tools, putting on/off headsets). We argue against this being the sole design-objective driving haptic-interfaces; instead, we propose also to optimize the fidelity of the real-world sensations that users feel while wearing a haptic device. We propose a framework to classify touch-devices by measuring not only their abilities to deliver virtual-feedback but also how much they impair physical-feedback—we argue this balancing act is an urgent mainstream need, given the success of Mixed-Reality. Thus, to accelerate the research in this area, we synthesize existing techniques into new conceptual-categories: feel-through, on-demand, relocated, and remote actuators. Finally, we present their pros/cons and discuss a possible roadmap.

We propose a novel concept for haptics in which one centralized on-body actuator renders haptic effects on multiple body parts by stimulating the brain, i.e., the source of the nervous system—we call this a haptic source-effector, as opposed to the traditional wearables’ approach of attaching one actuator per body part (end-effectors). We implement our concept via transcranial-magnetic-stimulation (TMS)—a non-invasive technique from neuroscience/medicine in which electromagnetic pulses safely stimulate brain areas. Our approach renders ∼15 touch/force-feedback sensations throughout the body (e.g., hands, arms, legs, feet, and jaw—which we found in our first user study), all by stimulating the user's sensorimotor cortex with a single magnetic coil moved mechanically across the scalp. In our second user study, we probed into participants’ experiences while using our haptic display in VR. Finally, as the first implementation of full-body haptics based on non-invasive brain stimulation, we discuss the roadmap to extend its interactive opportunities.

We present a technique for providing remote tactile feedback to the thumb and index finger via a wristband device. This enables haptics for touch and pinch interactions in mixed reality (MR) while keeping the user’s hand entirely free. We achieve this through a novel cross-modal stimulation, which we term visually augmented electro-tactile feedback. This consists of (1) electrically stimulating the nerves that innervate the targeted fingers using our wristband device; and (2) concurrently, visually augmenting the targeted finger in MR to steer the perceived sensation to the desired location. In our psychophysics study, we found that our approach provides tactile perception akin to tapping and, even from the wrist, it is capable of delivering the sensation to the targeted fingers with $\sim$50% of sensation occurring in the thumb and $\sim$40% of sensation occurring in the index finger. These results on localizability are unprecedented compared to electro-tactile feedback alone or any prior work for creating sensations in the hand with devices worn on the wrist/arm. Moreover, unlike conventional electro-tactile techniques, our wristband dispenses with gel electrodes. Instead, it incorporates custommade elastomer-based dry electrodes and a stimulation waveform designed for the electrodes, ensuring the practicality of the device beyond laboratory settings. Lastly, we evaluated the haptic realism of our approach in mixed reality and elicited qualitative feedback from users. Participants preferred our approach to a baseline vibrotactile wrist-worn device.

Smartwatches gained popularity in the mainstream, making them into today’s de-facto wearables. Despite advancements in sensing, haptics on smartwatches is still restricted to tactile feedback (e.g., vibration). Most smartwatch-sized actuators cannot render strong force-feedback. Simultaneously, electrical muscle stimulation (EMS) promises compact force-feedback but, to actuate fingers requires users to wear many electrodes on their forearms. While forearm electrodes provide good accuracy, they detract EMS from being a practical force-feedback interface. To address this, we propose moving the electrodes to the wrist—conveniently packing them in the backside of a smartwatch. In our first study, we found that by cross-sectionally stimulating the wrist in 1,728 trials, we can actuate thumb extension, index extension & flexion, middle flexion, pinky flexion, and wrist flexion. Following, we engineered a compact EMS that integrates directly into a smartwatch’s wristband (with a custom stimulator, electrodes, demultiplexers, and communication). In our second study, we found that participants could calibrate our device by themselves <?TeX $\sim 50 \%$?> Math 1 faster than with conventional EMS. Furthermore, all participants preferred the experience of this device, especially for its social acceptability & practicality. We believe that our approach opens new applications for smartwatch-based interactions, such as haptic assistance during everyday tasks.

Abstract. We engineered LipIO, a novel device enabling the lips to be used simultaneously as an input and output surface. LipIO comprises two overlapping flexible electrode arrays: an outward-facing array for capacitive touch and a lip-facing array for electrotactile stimulation. While wearing LipIO, users feel the interface's state via lip stimulation and respond by touching their lip with their tongue or opposing lip. More importantly, LipIO provides co-located tactile feedback that allows users to feel where in the lip they are touching—this is key to enabling eyes- and hands-free interactions. Our three studies verified participants perceived electrotactile output on their lips and subsequently touched the target location with their tongue with an average accuracy of 93%, while wearing LipIO with five I/O electrodes with co-located feedback. Finally, we demonstrate the potential of LipIO in four exemplary applications that illustrate how it enables new types of eyes- and hands-free micro-interactions.

We present a technique to render tactile feedback to the palmar side of the hand while keeping it unobstructed and, thus, preserving manual dexterity during interactions with physical objects. We implement this by applying electro-tactile stimulation only to the back of the hand and to the wrist. In our approach, there are no electrodes on the palmar side, yet that is where tactile sensations are felt. While we place electrodes outside the user's palm, we do so in strategic locations that conduct the electrical currents to the median/ulnar nerves, causing tactile sensations on the palmar side of the hand. In our user studies, we demonstrated that our approach renders tactile sensations to 11 different locations on the palmar side while keeping users’ palms free for dexterous manipulations. Our approach enables new applications such as tactile notifications during dexterous activities or VR experiences that rely heavily on physical props.

Electrical muscle stimulation (EMS) became a popular method for force-feedback without mechanical-actuators. While much has been written about the advantages of EMS, not much work has investigated circumventing its key limitations: (1) as impulses traverse the skin, they cause an uncomfortable “tingling”; (2) impulses are delivered via gelled-electrodes, which not only require direct skin contact (must be worn under clothes); but, also (3) dry up after a few hours. To tackle these, we explore switching from electrical to magnetic muscle stimulation (MMS), via electromagnetic fields generated by coils. The first advantage is that MMS coils do not require direct skin contact and can actuate up to 5 cm away (Study#1)—this enables applications not possible with EMS, such as stimulation over the clothes and without ever replacing electrodes. Second, and more important, MMS results in ∼50 % less discomfort caused by tingling than EMS (Study#2). We found that reducing this tingling discomfort has two downstream effects for interactive systems: (1) participants rated MMS force-feedback as more realistic than that of EMS (Study#3); and (2) participants could more accurately perceive the pose actuated by the interactive system (Study#4). Finally, we demonstrated applications where our proposed switch from EMS to MMS improves user experience, including for VR feedback, gaming, and pose-control.

We propose a novel interface concept in which interactive systems directly manipulate the user's head orientation. We implement this using electrical-muscle-stimulation (EMS) of the neck muscles, which turns the head around its yaw (left/right) and pitch (up/down) axis. As the first exploration of EMS for head actuation, we characterized which muscles can be robustly actuated. Second, we evaluated the accuracy of our system for actuating participants' head orientation towards static targets and trajectories. Third, we demonstrated how it enables interactions not possible before by building a range of applications, such as (1) synchronizing head orientations of two users, which enables a user to communicate head nods to another user while listening to music, and (2) directly changing the user's head orientation to locate objects in AR. Finally, in our second study, participants felt that our head actuation contributed positively to their experience in four distinct applications.

We engineered DigituSync, a passive-exoskeleton that physically links two hands together, enabling two users to adaptively transmit finger movements in real-time. It uses multiple four-bar linkages to transfer both motion and force, while still preserving congruent haptic feedback. Moreover, we implemented a variable-length linkage that allows adjusting the force transmission ratio between the two users and regulates the amount of intervention, which enables users to customize their learning experience. DigituSync's benefits emerge from its passive design: unlike existing haptic devices (motor-based exoskeletons or electrical muscle stimulation), DigituSync has virtually no latency and does not require batteries/electronics to transmit or adjust movements, making it useful and safe to deploy in many settings, such as between students and teachers in a classroom. We validated DigituSync by means of technical evaluations and a user study, demonstrating that it instantly transfers finger motions and forces with the ability of adaptive force transmission, which allowed participants to feel more control over their own movements and to feel the teacher's intervention was more responsive. We also conducted two exploratory sessions with a music teacher and deaf-blind users, which allowed us to gather experiential insights from the teacher's side and explore DigituSync in applications.

Digital images appearing on displays in everyday activities (e.g., photos on a smartphone) are automatically and instantly rendered without manual intervention such that we can seamlessly appreciate them. In contrast, shape displays require manual designs of outputs upon actuation of input images to render 3D shapes. In this work, we aim to achieve automatic and on-the-spot actuation of digital images so that we can seamlessly see 3D physical images. To this end, we developed BulkScreen, an image projection system that can automatically render 3D shapes of input images on a vertical pin-array screen. Our approach is based on a deep-neural-network saliency estimation coupled with our post-processing algorithm. We believe this spontaneous actuation mechanism facilitates applications with shape displays such as real-time picture browsing and display advertisement, building on the benefit of representing physical shapes; tangibility.

We present DualVib, a compact handheld device that simulates the haptic sensation of manipulating dynamic mass; mass that causes haptic feedback as the user’s hand moves (e.g., shaking a jar and feeling coins rattling inside). Unlike other devices that require actual displacement of weight, DualVib dispenses with heavy and bulky mechanical structures and, instead, uses four vibration actuators. DualVib simulates a dynamic mass by simultaneously delivering two types of haptic feedback to the user’s hand: (1) pseudo-force feedback created by asymmetric vibrations that render the kinesthetic force arising from the moving mass; and (2) texture feedback through acoustic vibrations that render the object’s surface vibrations correlated with mass material properties. By means of our user study, we found out that DualVib allowed users to more effectively distinguish dynamic masses when compared to using either pseudo-force or texture feedback alone. We also report qualitative feedback from users who experienced five virtual reality applications with our device.

To efficiently edit first-person videos, manually highlighting important scenes while recording is helpful. However, little study has been performed on how such annotation contributes to video editing and affects user behavior during recording. To elicit fundamental requirements for designing useful record-time annotation techniques, we conducted a study using a set of prototype wearable camera system and a video editing interface that enables users to annotate scenes during recording. We asked participants to perform video recording and editing tasks with two different interface settings. We observed that the participants edited videos more efficiently with detailed annotation techniques, whereas focussing on annotating scenes affected their record-time behavior. We conclude the paper with the design guidelines developed from the findings.

Deterioration of sewer pipes is one of the very important problems in Japan. Sewer inspections have been carried out mainly by visual check or wired remote robots with a camera. However, such inspection schemes involve high labor and/or monetary cost. Sewer inspection with boat-type video cameras or unwired robots takes a long time to check the result of the inspection because video data are obtained after the equipment is retrieved from the pipe. To realize low cost, safe and quick inspection of sewer pipes, we have proposed a sewer inspection system using drifting wireless sensor nodes. Water, soil, and the narrow space in the pipe make the long-range wireless radio communication difficult. Therefore, we have to identify suitable radio frequency and antenna configuration based on wireless communication characteristics in sewer pipes. If the frequency is high, the Fresnel zone the needed space for the line of sight is small, but the path loss becomes worse. On the other hand, if the frequency is low, the size of the Fresnel zone is large, but the path loss becomes small. We conducted wireless communication experiments using 920MHz, 2.4GHz and 5GHz band equipment in an experimental underground pipe. The measurement results show that the wireless communication distance of 5GHz IEEE802.11a is about 5m in a 200mm-diameter pipe and it is longer than 920MHz (ARIB STD-T108), 2.4GHz (IEEE802.11g, IEEE802.15.4) and with 10dBm.