Interactive electrical-muscle-stimulation (EMS) supports motor-skills by actuating the user’s muscles. However, existing EMS-interfaces exclusively focus on demonstrating movements/sequences (e.g., which fingers to actuate to play a piano melody) and have not investigated EMS for skills requiring precise force application (e.g., playing musical instruments, practicing culinary techniques, operating force-sensitive tools). Our user study found that when EMS-interfaces demonstrate a force, participants trying to recall this force, overshoot by a median 19%; with especially larger overshoots at lower target-forces (e.g., produce a ∼ 1.2 kg force, after a 1 kg demonstration). This force mismatch renders EMS-interfaces unable to accurately demonstrate forces—drastically limiting the growing potential of EMS for HCI. To significantly improve on this, we modeled users’ recall of EMS-demonstrated forces. This model allows to adjust EMS-interfaces to render a target force that, when recalled, matches the intended force best—in our study, this improved median force recall by ∼ 35%.

Electrical muscle stimulation (EMS) can support physical-assistance (e.g., shaking a spray-can before painting). However, EMS-assistance is highly-specialized because it is (1) fixed (e.g., one program for shaking spray-cans, another for opening windows); and (2) non-contextual (e.g., a spray-can for cooking dispenses cooking-oil, not paint—shaking it is unnecessary). Instead, we explore a different approach where muscle-stimulation instructions are generated considering the user’s context (e.g., pose, location, surroundings). The resulting system is more general—enabling unprecedented EMS interactions (e.g., opening a pill bottle) yet also replicating existing systems (e.g., Affordance++) without task-specific programming. It uses computer-vision/large-language-models to generate EMS-instructions, constraining these to a muscle-stimulation knowledge-base & joint-limits. In our user-study, we found participants successfully completed physical-tasks while guided by generative-EMS, even when EMS-instructions were (purposely) erroneous. Participants understood generated gestures and, even during forced-errors, understood partial-instructions, identified errors, and re-prompted the system. We believe our concept marks a shift toward more general-purpose EMS-interfaces.

Electrical muscle stimulation (EMS) has been leveraged to assist in learning motor skills by actuating the user's muscles. However, existing systems provide static demonstration—actuating the correct movements, regardless of the user's learning progress. Instead, we contrast two versions of a piano-tutoring system: a conventional EMS setup that moves the participant's fingers to play the sequence of movements correctly, and a novel adaptive-EMS system that changes its guidance strategy based on the participant's performance. The adaptive-EMS dynamically adjusts its guidance: (1) demonstrate by playing the entire sequence when errors are frequent; (2) correct by lifting incorrect fingers and actuating the correct one when errors are moderate; and (3) warn by lifting incorrect fingers when errors are low. We found that adaptive-EMS improved learning outcomes (recall) and was preferred by participants. We believe this approach could inspire new types of physical tutoring systems that promote adaptive over static guidance.

Techniques like electrical muscle stimulation (EMS) offer promise in assisting physical tasks by automating movements, e.g., shaking a spray-can or tapping a button. However, existing actuation systems improve the performance of a task that users are already focusing on (e.g., users are already focused on using the spray-can). Instead, we investigate whether these interactive-actuation systems (e.g., EMS) offer any benefits if they automate a task that happens in the background of the user's focus. Thus, we explored whether automating a repetitive movement via EMS would reduce mental workload while users perform parallel tasks (e.g., focusing on writing an essay while EMS stirs a pot of soup). In our study, participants performed a cognitively-demanding multitask aided by EMS (SplitBody condition) or performed by themselves (baseline). We found that with SplitBody performance increased (35% on both tasks, 18% on the non-EMS-automated task), physical-demand decreased (31%), and mental-workload decreased (26%).

Vertical force-feedback is extremely rare in mainstream interactive experiences. This happens because existing haptic devices capable of sufficiently strong forces that would modify a user's jump require grounding (e.g., motion platforms or pulleys) or cumbersome actuators (e.g., large propellers attached or held by the user). To enable interactive experiences to feature jump-based haptics without sacrificing wearability, we propose JumpMod, an untethered backpack that modifies one's sense of jumping. JumpMod achieves this by moving a weight up/down along the user's back, which modifies perceived jump momentum—creating accelerated & decelerated jump sensations. In our second study, we empirically found that our device can render five effects: jump higher, land harder/softer, pulled higher/lower. Based on these, we designed four jumping experiences for VR & sports. Finally, in our third study, we found that participants preferred wearing our device in an interactive context, such as one of our jump-based VR applications.

The amount of e-waste generated by discarding devices is enormous but options for recycling remain limited. However, inside a discarded device (from consumer devices to one's own prototypes), an electronics designer could find dozens to thousands of reusable components, including microcontrollers, sensors, voltage regulators, etc. Despite this, existing electronic design tools assume users will buy all components anew. To tackle this, we propose ecoEDA, an interactive tool that enables electronics designers to explore recycling electronic components during the design process. We accomplish this via (1) creating suggestions to assist users in identifying and designing with recycled components; and (2) maintaining a library of useful data relevant to reuse (e.g., allowing users to find which devices contain which components). Through example use-cases, we demonstrate how our tool can enable various pathways to recycling e-waste. To evaluate it, we conducted a user study where participants used our tool to create an electronic schematic with components from torn-down e-waste devices. We found that participants’ designs made with ecoEDA featured an average of 66% of recycled components. Last, we reflect on challenges and opportunities for building software that promotes e-waste reuse.

We engineered DigituSync, a passive-exoskeleton that physically links two hands together, enabling two users to adaptively transmit finger movements in real-time. It uses multiple four-bar linkages to transfer both motion and force, while still preserving congruent haptic feedback. Moreover, we implemented a variable-length linkage that allows adjusting the force transmission ratio between the two users and regulates the amount of intervention, which enables users to customize their learning experience. DigituSync's benefits emerge from its passive design: unlike existing haptic devices (motor-based exoskeletons or electrical muscle stimulation), DigituSync has virtually no latency and does not require batteries/electronics to transmit or adjust movements, making it useful and safe to deploy in many settings, such as between students and teachers in a classroom. We validated DigituSync by means of technical evaluations and a user study, demonstrating that it instantly transfers finger motions and forces with the ability of adaptive force transmission, which allowed participants to feel more control over their own movements and to feel the teacher's intervention was more responsive. We also conducted two exploratory sessions with a music teacher and deaf-blind users, which allowed us to gather experiential insights from the teacher's side and explore DigituSync in applications.

We propose a novel type of olfactory device that creates a stereo-smell experience, i.e., directional information about the location of an odor, by rendering the readings of external odor sensors as trigeminal sensations using electrical stimulation of the user's nasal septum. The key is that the sensations from the trigeminal nerve, which arise from nerve-endings in the nose, are perceptually fused with those of the olfactory bulb (the brain region that senses smells). As such, we propose that electrically stimulating the trigeminal nerve is an ideal candidate for stereo-smell augmentation/substitution that, unlike other approaches, does not require implanted electrodes in the olfactory bulb. To realize this, we engineered a self-contained device that users wear across their nasal septum. Our device outputs by stimulating the user's trigeminal nerve using electrical impulses with variable pulse-widths; and it inputs by sensing the user's inhalations using a photoreflector. It measures 10x23 mm and communicates with external gas sensors using Bluetooth. In our user study, we found the key electrical waveform parameters that enable users to feel an odor's intensity (absolute electric charge) and direction (phase order and net charge). In our second study, we demonstrated that participants were able to localize a virtual smell source in the room by using our prototype without any previous training. Using these insights, our device enables expressive trigeminal sensations and could function as an assistive device for people with anosmia, who are unable to smell.

We propose a new type of haptic actuator, which we call MagnetIO, that is comprised of two parts: one battery-powered voice-coil worn on the user's fingernail and any number of interactive soft patches that can be attached onto any surface (everyday objects, user's body, appliances, etc.). When the user's finger wearing our voice-coil contacts any of the interactive patches it detects its magnetic signature via magnetometer and vibrates the patch, adding haptic feedback to otherwise input-only interactions. To allow these passive patches to vibrate, we make them from silicone with regions doped with polarized neodymium powder, resulting in soft and stretchable magnets. This stretchable form-factor allows them to be wrapped to the user's body or everyday objects of various shapes. We demonstrate how these add haptic output to many situations, such as adding haptic buttons to the walls of one's home. In our technical evaluation, we demonstrate that our interactive patches can be excited across a wide range of frequencies (0-500 Hz) and can be tuned to resonate at specific frequencies based on the patch's geometry. Furthermore, we demonstrate that MagnetIO's vibration intensity is as powerful as a typical linear resonant actuator (LRA); yet, unlike these rigid actuators, our passive patches operate as springs with multiple modes of vibration, which enables a wider band around its resonant frequency than an equivalent LRA.

We propose a nail-mounted foldable haptic device that provides tactile feedback to mixed reality (MR) environments by pressing against the user's fingerpad when a user touches a virtual object. What is novel in our device is that it quickly tucks away when the user interacts with real-world objects. Its design allows it to fold back on top of the user's nail when not in use, keeping the user's fingerpad free to, for instance, manipulate handheld tools and other objects while in MR. To achieve this, we engineered a wireless and self-contained haptic device, which measures 24×24×41 mm and weighs 9.5 g. Furthermore, our foldable end-effector also features a linear resonant actuator, allowing it to render not only touch contacts (i.e., pressure) but also textures (i.e., vibrations). We demonstrate how our device renders contacts with MR surfaces, buttons, low- and high-frequency textures. In our first user study, we found that participants perceived our device to be more realistic than a previous haptic device that also leaves the fingerpad free (i.e., fingernail vibration). In our second user study, we investigated the participants’ experience while using our device in a real-world task that involved physical objects. We found that our device allowed participants to use the same finger to manipulate handheld tools, small objects, and even feel textures and liquids, without much hindrance to their dexterity, while feeling haptic feedback when touching MR interfaces.

Electrical muscle stimulation (EMS) is an emergent technique that miniaturizes force feedback, especially popular for untethered haptic devices, such as mobile gaming, VR, or AR. However, the actuation displayed by interactive systems based on EMS is coarse and imprecise. EMS systems mostly focus on inducing movements in large muscle groups such as legs, arms, and wrists; whereas individual finger poses, which would be required, for example, to actuate a user's fingers to fingerspell even the simplest letters in sign language, are not possible. The lack of dexterity in EMS stems from two fundamental limitations: (1) lack of independence: when a particular finger is actuated by EMS, the current runs through nearby muscles, causing unwanted actuation of adjacent fingers; and, (2) unwanted oscillations: while it is relatively easy for EMS to start moving a finger, it is very hard for EMS to stop and hold that finger at a precise angle; because, to stop a finger, virtually all EMS systems contract the opposing muscle, typically achieved via controllers (e.g., PID)—unfortunately, even with the best controller tuning, this often results in unwanted oscillations. To tackle these limitations, we propose dextrEMS, an EMS-based haptic device featuring mechanical brakes attached to each finger joint. The key idea behind dextrEMS is that while the EMS actuates the fingers, it is our mechanical brake that stops the finger in a precise position. Moreover, it is also the brakes that allow dextrEMS to select which fingers are moved by EMS, eliminating unwanted movements by preventing adjacent fingers from moving. We implemented dextrEMS as an untethered haptic device, weighing only 68g, that actuates eight finger joints independently (metacarpophalangeal and proximal interphalangeal joints for four fingers), which we demonstrate in a wide range of haptic applications, such as assisted fingerspelling, a piano tutorial, guitar tutorial, and a VR game. Finally, in our technical evaluation, we found that dextrEMS outperformed EMS alone by doubling its independence and reducing unwanted oscillations.

With the growing popularity of drones, we start to see more wearable drone concepts. For the user to carry around a drone, it must be lightweight as well as have a small portable form factor. However, these constraints affect the battery capacity and therefore decrease the flight time of the vehicle. With Aerial Tethered Companion, a bigger battery is installed in the user's backpack allowing to extend significantly the flight time. Moreover, without an on-board battery, the quadcopter can carry more payload. Such system can be used in various scenarios for example in sports augmentation where the user would see itself through the drone's camera. Furthermore, Aerial Tethered Companion can be applied in telepresence where an external user would be able to see and navigate around the local user.