Knowing when a user picks up an object plays a vital role in many context-aware applications. For example, tracking water consumption, counting calories consumed, or reminding you to bring your keys are all context-centered scenarios involving picking up objects. In this project, we propose Contextra, a wrist-worn system that uses sensor fusion to recognize when a user grasps objects. Sensor fusion allows all parts of the grasp to be sensed in ways single channels cannot alone. In our wristband, we fuse EMG and IMU data with video captured from three low-power IR cameras. These cameras maintain privacy by using an active-illumination technique to only capture features close to the sensors. Beyond grasps alone, we see Contextra as playing a foundational role in providing continuous awareness of context triggers to extend the functionality of existing AI devices that cannot run continuously due to power and privacy concerns.

We present a technique for providing remote tactile feedback to the thumb and index finger via a wristband device. This enables haptics for touch and pinch interactions in mixed reality (MR) while keeping the user’s hand entirely free. We achieve this through a novel cross-modal stimulation, which we term visually augmented electro-tactile feedback. This consists of (1) electrically stimulating the nerves that innervate the targeted fingers using our wristband device; and (2) concurrently, visually augmenting the targeted finger in MR to steer the perceived sensation to the desired location. In our psychophysics study, we found that our approach provides tactile perception akin to tapping and, even from the wrist, it is capable of delivering the sensation to the targeted fingers with $\sim$50% of sensation occurring in the thumb and $\sim$40% of sensation occurring in the index finger. These results on localizability are unprecedented compared to electro-tactile feedback alone or any prior work for creating sensations in the hand with devices worn on the wrist/arm. Moreover, unlike conventional electro-tactile techniques, our wristband dispenses with gel electrodes. Instead, it incorporates custommade elastomer-based dry electrodes and a stimulation waveform designed for the electrodes, ensuring the practicality of the device beyond laboratory settings. Lastly, we evaluated the haptic realism of our approach in mixed reality and elicited qualitative feedback from users. Participants preferred our approach to a baseline vibrotactile wrist-worn device.

Smart rings for subtle, reliable finger input offer an attractive path for ubiquitous interaction with wearable computing platforms. However, compared to ordinary rings worn for cultural or fashion reasons, smart rings are much bulkier and less comfortable, largely due to the space required for a battery, which also limits the space available for sensors. This paper presents picoRing, a flexible sensing architecture that enables a variety of battery-free smart rings paired with a wristband. By inductively connecting a wristband-based sensitive reader coil with a ring-based fully-passive sensor coil, picoRing enables the wristband to stably detect the passive response from the ring via a weak inductive coupling. We demonstrate four different rings that support thumb-to-finger interactions like pressing, sliding, or scrolling. When users perform these interactions, the corresponding ring converts each input into a unique passive response through a network of passive switches. Combining the coil-based sensitive readout with the fully-passive ring design enables a tiny ring that weighs as little as 1.5 g and achieves a 13 cm stable readout despite finger bending, and proximity to metal.

Text entry is a critical capability for any modern computing experience, with lightweight augmented reality (AR) glasses being no exception. Designed for all-day wearability, a limitation of lightweight AR glass is the restriction to the inclusion of multiple cameras for extensive field of view in hand tracking. This constraint underscores the need for an additional input device. We propose a system to address this gap: a ring-based mid-air gesture typing technique, RingGesture, utilizing electrodes to mark the start and end of gesture trajectories and inertial measurement units (IMU) sensors for hand tracking. This method offers an intuitive experience similar to raycast-based mid-air gesture typing found in VR headsets, allowing for a seamless translation of hand movements into cursor navigation. To enhance both accuracy and input speed, we propose a novel deep-learning word prediction framework, Score Fusion, comprised of three key components: a) a word-gesture decoding model, b) a spatial spelling correction model, and c) a lightweight contextual language model. In contrast, this framework fuses the scores from the three models to predict the most likely words with higher precision. We conduct comparative and longitudinal studies to demonstrate two key findings: firstly, the overall effectiveness of RingGesture, which achieves an average text entry speed of 27.3 words per minute (WPM) and a peak performance of 47.9 WPM. Secondly, we highlight the superior performance of the Score Fusion framework, which offers a 28.2% improvement in uncorrected Character Error Rate over a conventional word prediction framework, Naive Correction, leading to a 55.2% improvement in text entry speed for RingGesture. Additionally, RingGesture received a System Usability Score of 83 signifying its excellent usability.

Tickling is a type of sensation that is associated with laughter, smiling, or other similar reactions. Psychology research has shown that tickling and laughter can significantly relieve stress. Although several tickling artifacts have been suggested in prior work, limited knowledge is available if those artifacts could evoke laughter. In this article, we aim at filling this gap by designing and developing a novel foot-tickling mechanism that can evoke laughter. We first developed an actuator that can create tickling sensations along the sole of the foot utilising magnet-driven brushes. Then, we conducted two studies to identify the most ticklish locations of the foot’s sole and stimulation patterns that can evoke laughter. In a follow-up study with a new set of participants, we confirmed that the identified stimuli could evoke laughter. From the participants’ feedback, we derived several applications that such a simulation could be useful. Finally, we embedded our actuators into a flexible insole, demonstrating the potential of a wearable tickling insole.

We present MAGHair, a novel wearable technique that provides subtle haptic sensation by stimulating the body hair without touching the skin. Our approach builds on previous research in magnetic hair stimulation and magnetic locomotion. We use magnetic cosmetics to augment the body hair, which can then be stimulated by a wearable apparatus that combines electromagnets and permanent magnets. We provide technical insights on the implementation of a fully functional wrist-worn form factor and early adaptations into other form factors. In addition, we provide a workflow for evaluating and characterizing the magnetic cosmetic recipes. Finally, we evaluate MAGHair, which demonstrated that users could detect the sensation of hair movement that they described as gentle and unique.

In this paper, we present M-Hair, a novel method for providing tactile feedback by stimulating only the body hair without touching the skin. It works by applying passive magnetic materials to the body hair, which is actuated by external magnetic fields. Our user study suggested that the value of the M-hair mechanism is in inducing affective sensations such as pleasantness, rather than effectively discriminating features such as shape, size, and direction. This work invites future research to use this method in applications that induce emotional responses or affective states, and as a research tool for investigations of this novel sensation.

In this paper we explore the field of human robot interaction (HRI) by adding a new non-physical player in the interaction rather than only humans and robots. It is a proven fact that physical robots enhance the immersion perception compared to only virtual agents. Thus, we created a shared environment with a Pleo Robot and a Virtual Pleo Robot, which we called Vleo, connected through a server to explore this new paradigm in order to see if the engagement during the interaction is improved in intensity and duration. A straightforward set of interactions between Pleo and Vleo have been designed to create narratives and therefore, tested with a group of 8-12 year old children. Results from the test suggest that virtual social robots are a good way to enhance interaction with physical social robots.