We present a technique for providing remote tactile feedback to the thumb and index finger via a wristband device. This enables haptics for touch and pinch interactions in mixed reality (MR) while keeping the user’s hand entirely free. We achieve this through a novel cross-modal stimulation, which we term visually augmented electro-tactile feedback. This consists of (1) electrically stimulating the nerves that innervate the targeted fingers using our wristband device; and (2) concurrently, visually augmenting the targeted finger in MR to steer the perceived sensation to the desired location. In our psychophysics study, we found that our approach provides tactile perception akin to tapping and, even from the wrist, it is capable of delivering the sensation to the targeted fingers with $\sim$50% of sensation occurring in the thumb and $\sim$40% of sensation occurring in the index finger. These results on localizability are unprecedented compared to electro-tactile feedback alone or any prior work for creating sensations in the hand with devices worn on the wrist/arm. Moreover, unlike conventional electro-tactile techniques, our wristband dispenses with gel electrodes. Instead, it incorporates custommade elastomer-based dry electrodes and a stimulation waveform designed for the electrodes, ensuring the practicality of the device beyond laboratory settings. Lastly, we evaluated the haptic realism of our approach in mixed reality and elicited qualitative feedback from users. Participants preferred our approach to a baseline vibrotactile wrist-worn device.

In this paper, we investigate postural reinforcement haptics for mid-air typing using squeeze actuation on the wrist. We propose and validate eye-tracking based objective metrics that capture the impact of haptics on the user’s experience, which traditional performance metrics like speed and accuracy are not able to capture. To this end, we design four wrist-based haptic feedback conditions: no haptics, vibrations on keypress, squeeze+vibrations on keypress, and squeeze posture reinforcement + vibrations on keypress. We conduct a text input study with 48 participants to compare the four conditions on typing and gaze metrics. Our results show that for expert qwerty users, posture reinforcement haptics significantly benefit typing by reducing the visual attention on the keyboard by up to 44% relative to no haptics, thus enabling eyes-away behaviors.

Ultrasound mid-air haptic (UMH) devices are a novel tool for haptic feedback, capable of providing localized vibrotactile stimuli to users at a distance. UMH applications largely rely on generating tactile shape outlines on the users’ skin. Here we investigate how to achieve sensations of continuity or gaps within such two-dimensional curves by studying the perception of pairs of amplitude-modulated focused ultrasound stimuli. On the one hand, we aim to investigate perceptual effects that may arise from providing simultaneous UMH stimuli. On the other hand, we wish to provide perception-based rendering guidelines for generating continuous or discontinuous sensations of tactile shapes. Finally, we hope to contribute toward a measure of the perceptually achievable resolution of UMH interfaces. We performed a user study to identify how far apart two focal points need to be to elicit a perceptual experience of two distinct stimuli separated by a gap. Mean gap detection thresholds were found at 32.3-mm spacing between focal points, but a high within- and between-subject variability was observed. Pairs spaced below 15 mm were consistently (>95%) perceived as a single stimulus, while pairs spaced 45 mm apart were consistently (84%) perceived as two separate stimuli. To investigate the observed variability, we resort to acoustic simulations of the resulting pressure fields. These show a non-linear evolution of actual peak pressure spacing as a function of nominal focal point spacing. Beyond an initial threshold in spacing (between 15 and 18 mm), which we believe to be related to the perceived size of a focal point, the probability of detecting a gap between focal points appears to linearly increase with spacing. Our work highlights physical interactions and perceptual effects to consider when designing or investigating the perception of UMH shapes.

Mid-air haptics allow bare-hand tactile stimulation; however, it has a constrained workspace, making it unsuitable for room-scale haptics. We present a novel approach to rendering mid-air haptic sensations in a large rendering volume by turning a static array into a dynamic array following the user's hand. We used a 6DOF robot to drive a haptic ultrasound array over a large 3D space. Our system enables rendering room-scale mid-air experiences while preserving bare-hand interaction, thus, providing tangibility for virtual environments. To evaluate our approach, we performed three evaluations. First, we performed a technical system evaluation, showcasing the feasibility of such a system. Next, we conducted three psychophysical experiments, showing that the motion does not affect the user's perception with high likelihood. Lastly, we explored seven use cases that showcase our system's potential using a user study. We discuss challenges and opportunities in how large-scale mid-air haptics can contribute toward room-scale haptic feedback. Thus, with our system, we contribute to general haptic mid-air feedback on a large scale.

Haptic feedback is known to enhance the realism of an individual’s interactions with objects in virtual environments. Wearable haptic devices, such as vibrotactile sleeves or armbands, can provide haptic feedback in a smaller and more lightweight form factor than haptic gloves that can be bulky and cumbersome to the wearer. In this article, we present tactile and squeeze bracelet interface (Tasbi), a multimodal haptic wristband that can provide radial squeeze forces around the wrist along with vibrotactile feedback at six discrete locations around the band. Tasbi implements a squeezing mechanism that minimizes tangential forces between the band’s points of contact with the skin, instead of focusing the motor actuation to predominantly normal forces. Force sensing capacitors enable closed-loop control of the squeeze force, while vibration is achieved with linear resonant actuators. A detailed description of the design and experimental results demonstrating closed-loop control of squeeze cues provided by Tasbi is presented. Additionally, we present the results of psychophysical experiments that quantify user perception of the vibration and squeeze cues, including vibrotactile identification accuracy in the presence of varying squeeze forces, discrimination thresholds for the squeeze force, and an analysis of user preferences for squeeze actuation magnitudes.

Voice is a rich modality for conveying emotions, however emotional prosody production can be situationally or medically impaired. Since thermal displays have been shown to evoke emotions, we explore how thermal stimulation can augment perception of neutrally-spoken voice messages with affect. We designed ThermalWear, a wearable on-chest thermal display, then tested in a controlled study (N=12) the effects of fabric, thermal intensity, and direction of change. Thereafter, we synthesized 12 neutrally-spoken voice messages, validated (N=7) them, then tested (N=12) if thermal stimuli can augment their perception with affect. We found warm and cool stimuli (a) can be perceived on the chest, and quickly without fabric (4.7-5s) (b) do not incur discomfort (c) generally increase arousal of voice messages and (d) increase / decrease message valence, respectively. We discuss how thermal displays can augment voice perception, which can enhance voice assistants and support individuals with emotional prosody impairments.

In previous vision-to-touch sensory substitution approaches, including most `electronic white canes', typical approaches include mapping space-to-space, space-to-intensity, or space-to-frequency. To our knowledge, however, mapping space to time-delay has not been considered. Yet, because organisms must anticipate impending collisions with obstacles or anticipate being contacted by approaching objects, many organisms have developed computational short-cuts where distance-to-target is assumed to be proportional to a time-span. This short-cut often manifests itself in low-level sensorimotor behaviours and perceptual mechanisms. We studied whether untrained humans would spontaneously employ such a short-cut to estimate distance-to-obstacle in the absence of vision. The observers pressed a push button and a tactile pulse was delivered to the hand with a delay proportional to the distance to an obstacle detected by an optical range finder that they wore. The observers were not informed of the nature of the coding but could freely probe the obstacle while walking toward the target. Upon randomized presentation of obstacle distances, the observers quickly calibrated their judgement of distance-to-obstacle and were able to estimate this distance within a range of four meters for a proportionality factor corresponding to a velocity of one m/s.