This paper evaluates interaction techniques to increase input accuracy with embodied devices—an emergent type of interactive system where the user’s body serves as both the input and output medium (e.g., gestural input via cameras/IMUs; gestural output via motors/muscle stimulation). A shortcoming of existing embodied devices is their failure to enforce alignment between users’ proprioceptive inputs and interface state. Thus, we present and evaluate interaction techniques that use muscle stimulation to enable embodied devices to: (1) recall previous interface states; (2) provide confirmation cues on state transitions; and (3) constrain inputs to valid ranges. In our study, participants performed pairs of interactions with an embodied slider, separated by a distraction task. The results showed that, compared to the same embodied slider without EMS, the combination of our techniques increased users’: (1) absolute input accuracy; (2) relative input accuracy; and (3) confidence.

Interactive electrical-muscle-stimulation (EMS) supports motor-skills by actuating the user’s muscles. However, existing EMS-interfaces exclusively focus on demonstrating movements/sequences (e.g., which fingers to actuate to play a piano melody) and have not investigated EMS for skills requiring precise force application (e.g., playing musical instruments, practicing culinary techniques, operating force-sensitive tools). Our user study found that when EMS-interfaces demonstrate a force, participants trying to recall this force, overshoot by a median 19%; with especially larger overshoots at lower target-forces (e.g., produce a ∼ 1.2 kg force, after a 1 kg demonstration). This force mismatch renders EMS-interfaces unable to accurately demonstrate forces—drastically limiting the growing potential of EMS for HCI. To significantly improve on this, we modeled users’ recall of EMS-demonstrated forces. This model allows to adjust EMS-interfaces to render a target force that, when recalled, matches the intended force best—in our study, this improved median force recall by ∼ 35%.

Electrical muscle stimulation (EMS) can support physical-assistance (e.g., shaking a spray-can before painting). However, EMS-assistance is highly-specialized because it is (1) fixed (e.g., one program for shaking spray-cans, another for opening windows); and (2) non-contextual (e.g., a spray-can for cooking dispenses cooking-oil, not paint—shaking it is unnecessary). Instead, we explore a different approach where muscle-stimulation instructions are generated considering the user’s context (e.g., pose, location, surroundings). The resulting system is more general—enabling unprecedented EMS interactions (e.g., opening a pill bottle) yet also replicating existing systems (e.g., Affordance++) without task-specific programming. It uses computer-vision/large-language-models to generate EMS-instructions, constraining these to a muscle-stimulation knowledge-base & joint-limits. In our user-study, we found participants successfully completed physical-tasks while guided by generative-EMS, even when EMS-instructions were (purposely) erroneous. Participants understood generated gestures and, even during forced-errors, understood partial-instructions, identified errors, and re-prompted the system. We believe our concept marks a shift toward more general-purpose EMS-interfaces.

Providing tactile-feedback when users contact virtual-interfaces has been a seminal advance. However, we posit these advances have been explored in isolation from considerations of users’ physical interactions with surrounding-objects. Most touch-interfaces were designed to optimize virtual interfaces, but rarely consider that users also need to feel physical interfaces (e.g., tools, putting on/off headsets). We argue against this being the sole design-objective driving haptic-interfaces; instead, we propose also to optimize the fidelity of the real-world sensations that users feel while wearing a haptic device. We propose a framework to classify touch-devices by measuring not only their abilities to deliver virtual-feedback but also how much they impair physical-feedback—we argue this balancing act is an urgent mainstream need, given the success of Mixed-Reality. Thus, to accelerate the research in this area, we synthesize existing techniques into new conceptual-categories: feel-through, on-demand, relocated, and remote actuators. Finally, we present their pros/cons and discuss a possible roadmap.

Electrical muscle stimulation (EMS) has been leveraged to assist in learning motor skills by actuating the user's muscles. However, existing systems provide static demonstration—actuating the correct movements, regardless of the user's learning progress. Instead, we contrast two versions of a piano-tutoring system: a conventional EMS setup that moves the participant's fingers to play the sequence of movements correctly, and a novel adaptive-EMS system that changes its guidance strategy based on the participant's performance. The adaptive-EMS dynamically adjusts its guidance: (1) demonstrate by playing the entire sequence when errors are frequent; (2) correct by lifting incorrect fingers and actuating the correct one when errors are moderate; and (3) warn by lifting incorrect fingers when errors are low. We found that adaptive-EMS improved learning outcomes (recall) and was preferred by participants. We believe this approach could inspire new types of physical tutoring systems that promote adaptive over static guidance.

We propose an interactive tool that enables reusing printed circuit boards (PCB) as prototyping materials to implement new circuits—this extends the utility of PCBs rather than discards them as e-waste. To enable this, our tool takes a user's desired circuit schematic and analyzes its components and connections to find methods of creating the user's circuit on discarded PCBs (e.g., e-waste, old prototypes). In our technical evaluation, we utilized our tool across a diverse set of PCBs and input circuits to characterize how often circuits could be implemented on a different board, implemented with minor interventions (trace-cutting or bodge-wiring), or implemented on a combination of multiple boards—demonstrating how our tool assists with exhaustive matching tasks that a user would not likely perform manually. We believe our tool offers: (1) a new approach to prototyping with electronics beyond the limitations of breadboards and (2) a new approach to reducing e-waste during electronics prototyping.

We introduce Power-on-Touch, a novel method for powering devices during interaction. Power-on-Touch comprises two main components: (1) a wearable-transmitter attached to the user's body (e.g., fingernail, back of the hand, feet) with wireless power-coils and a battery; and (2) receiver-tags embedded in interactive devices, making them battery-free. Many devices only require power during interaction (e.g., TV remotes, digital calipers). We leverage this interactive opportunity by inductively transferring energy from the user's coil to the device's coil when in close proximity. To achieve this, we engineered receiver-tags and coils, including thin pancake-coils best-suited for wearables and spherical-coils that receive power omnidirectionally. To understand which coils best support a wide range of interactions (e.g., grasping, touching, hovering), we performed technical characterizations, including impedance and 3D efficiency analysis. We believe our technical approach can inspire ubiquitous computing with new ways to scale up the number and diversity of battery-free devices, not just sensors (µWatts) but also actuators (Watts).

Sensory-substitution devices enable perceiving objects by translating one modality (e.g., vision) into another (e.g., tactile). While many explored the placement of the haptic-output (e.g., torso, forehead), the camera's location remains largely unexplored—typically seeing from the eyes’ perspective. Instead, we propose that seeing & feeling information from the hands’ perspective could enhance flexibility & expressivity of sensory-substitution devices to support manual interactions with physical objects. To this end, we engineered a back-of-the-hand electrotactile-display that renders tactile images from a wrist-mounted camera, allowing the user's hand to feel objects while reaching & hovering. We conducted a study with sighted/Blind-or-Low-Vision participants who used our eyes vs. hand tactile-perspectives to manipulate bottles and soldering-irons, etc. We found that while both tactile perspectives provided comparable performance, when offered the opportunity to choose, all participants found value in also using the hands’ perspective. Moreover, we observed behaviors when “seeing with the hands” that suggest a more ergonomic object-manipulation. We believe these insights extend the landscape of sensory-substitution devices.

Designing plant-driven actuators presents an opportunity to create new types of devices that grow, age, and decay, such as robots that embody these qualities in their physical structure. Plant-robot hybrids that grow and decay incorporate unpredictable and gradual transformations inherent across living organisms and suggest an alternative to the design principles of immediacy, responsiveness, control, accuracy, and durability commonly found in robotic design. To explore this, we present a design space of primitives for plant-driven robotic actuators. Proof-of-concept prototypes illustrate how concepts like slow change, slow movement, decay, and destruction can be incorporated into robotic forms. We describe the design considerations required for building plant-driven actuators for robots, including experimental findings regarding the mechanical properties of plant forces. Finally, we speculate on the potential benefits of plant-robot hybrids to interactive domains such as robotics.

Computing devices become waste for a variety of reasons. They breakdown, become obsolete, or are no longer trendy. These events are so common that currently, e-waste has become the largest consumer waste stream in the world. However, taking apart e-waste devices reveals how they often contain many useful parts and components that could be scrapped and creatively integrated into new forms. These include highly expressive materials like sensors, displays, micro-controllers, etc. This workshop will explore processes in creative making with e-waste, examining the unique materiality of e-waste and how its reuse differs and/or converges with other material reuse processes. To do so, our workshop will combine hands-on activities (tear downs, rapid prototyping, and tutorials) with discussion of the challenges and opportunities in this space. We aim to use these activities to explore how HCI can better support the creative acts of making with e-waste across a wide audience.

We present Stick&Slip, a novel approach that alters friction between the fingerpad & surfaces by depositing liquid droplets that coat the fingerpad. The liquid coating modifies the finger's coefficient of friction, allowing users to feel surfaces up to ±60% more slippery or sticky. We selected our fluids to rapidly evaporate so that the surface returns to its original friction. Unlike traditional friction-feedback, such as electroadhesion or vibration, our approach: (1) alters friction on a wide range of surfaces and geometries, making it possible to modulate nearly any non-absorbent surface; (2) scales to many objects without requiring instrumenting the target surfaces (e.g., with conductive electrode coatings or vibromotors); and (3) both in/decreases friction via a single device. We identified nine liquids and characterized their practicality by measuring evaporation rates, etc. To illustrate the applicability of our approach, we demonstrate how it enables friction in virtual/mixed-reality or, even, while using everyday objects/tools.

Techniques like electrical muscle stimulation (EMS) offer promise in assisting physical tasks by automating movements, e.g., shaking a spray-can or tapping a button. However, existing actuation systems improve the performance of a task that users are already focusing on (e.g., users are already focused on using the spray-can). Instead, we investigate whether these interactive-actuation systems (e.g., EMS) offer any benefits if they automate a task that happens in the background of the user's focus. Thus, we explored whether automating a repetitive movement via EMS would reduce mental workload while users perform parallel tasks (e.g., focusing on writing an essay while EMS stirs a pot of soup). In our study, participants performed a cognitively-demanding multitask aided by EMS (SplitBody condition) or performed by themselves (baseline). We found that with SplitBody performance increased (35% on both tasks, 18% on the non-EMS-automated task), physical-demand decreased (31%), and mental-workload decreased (26%).

We propose a novel concept for haptics in which one centralized on-body actuator renders haptic effects on multiple body parts by stimulating the brain, i.e., the source of the nervous system—we call this a haptic source-effector, as opposed to the traditional wearables’ approach of attaching one actuator per body part (end-effectors). We implement our concept via transcranial-magnetic-stimulation (TMS)—a non-invasive technique from neuroscience/medicine in which electromagnetic pulses safely stimulate brain areas. Our approach renders ∼15 touch/force-feedback sensations throughout the body (e.g., hands, arms, legs, feet, and jaw—which we found in our first user study), all by stimulating the user's sensorimotor cortex with a single magnetic coil moved mechanically across the scalp. In our second user study, we probed into participants’ experiences while using our haptic display in VR. Finally, as the first implementation of full-body haptics based on non-invasive brain stimulation, we discuss the roadmap to extend its interactive opportunities.

Feeling haptics with our fingerpads is how we achieve manual tasks (e.g., operate a needle or press buttons). Following this, research started adding actuators atop the users’ fingerpads to render haptic feedback for interactive virtual environments. Recently, many have moved away from thick actuators (e.g., vibration motors) and turned to electrode-films with electrotactile stimulation—allowing users to still feel some sensations through the devices when touching physical objects (e.g., compliance or some macro features). However, we argue & demonstrate that thin devices are not enough to maximize the user's dexterity. We evaluate how adding small holes to electrotactile films can allow direct contact and thus increase haptic permeability, resulting in: (1) improved perception of tactile features; and (2) improved force control in grasping tasks. Finally, we observed participants in interactive experiences and found that holes can preserve dexterity with physical tasks while still benefiting from haptic feedback.

We propose, engineer, and study a novel method to augment the feeling of breathing—enabling interactive applications to let users feel like they are inhaling more/less air (perceived nasal airflow). We achieve this effect by cooling or heating the nose in sync with the user’s inhalation. Our illusion builds on the physiology of breathing: we perceive our breath predominantly through the cooling of our nasal cavities during inhalation. This is why breathing in a “fresh” cold environment feels easier than in a “stuffy” hot environment, even when the inhaled volume is the same. Our psychophysical study confirmed that our in-nose temperature stimulation significantly influenced breathing perception in both directions: making it feel harder & easier to breathe. Further, we found that <?TeX $\sim 90 \,\%$?> Math 1 of the trials were described as a change in perceived airflow/breathing, while only <?TeX $\sim 8 \,\%$?> Math 2 as temperature. Following, we engineered a compact device worn across the septum that uses Peltier elements. We illustrate the potential of this augmented breathing in interactive contexts, such as for virtual reality (e.g., rendering ease of breathing crisp air or difficulty breathing with a deteriorated gas mask) and everyday interactions (e.g., in combination with a relaxation application or to alleviate the perceived breathing resistance when wearing a mask).

Smartwatches gained popularity in the mainstream, making them into today’s de-facto wearables. Despite advancements in sensing, haptics on smartwatches is still restricted to tactile feedback (e.g., vibration). Most smartwatch-sized actuators cannot render strong force-feedback. Simultaneously, electrical muscle stimulation (EMS) promises compact force-feedback but, to actuate fingers requires users to wear many electrodes on their forearms. While forearm electrodes provide good accuracy, they detract EMS from being a practical force-feedback interface. To address this, we propose moving the electrodes to the wrist—conveniently packing them in the backside of a smartwatch. In our first study, we found that by cross-sectionally stimulating the wrist in 1,728 trials, we can actuate thumb extension, index extension & flexion, middle flexion, pinky flexion, and wrist flexion. Following, we engineered a compact EMS that integrates directly into a smartwatch’s wristband (with a custom stimulator, electrodes, demultiplexers, and communication). In our second study, we found that participants could calibrate our device by themselves <?TeX $\sim 50 \%$?> Math 1 faster than with conventional EMS. Furthermore, all participants preferred the experience of this device, especially for its social acceptability & practicality. We believe that our approach opens new applications for smartwatch-based interactions, such as haptic assistance during everyday tasks.

Most thermal interfaces attach Peltier elements and their required cooling systems (heatsinks and fans) directly to the palm or sole, preventing users from grasping or walking. To solve this problem, we present ThermalGrasp, an engineering approach for wearable thermal interfaces that enables users to grab and walk on real objects with minimal obstruction. Our approach moves the therma l device and cooling unit to areas not used in grasping or walking (e.g., dorsal hand/foot). We then use thin, compliant materials to conduct heat to/from the palm or sole. Unlike traditional Peltiers with heatsinks, our thin materials enable grasping and walking on real objects while enjoying thermal feedback. Using our approach, a user can, for example, grasp a passive prop (e.g., a stick that acts as a torch in VR), yet feel its thermal state (e.g., hot due to its flame). In our user studies, ThermalGrasp struck a useful balance between thermal and haptic realism. We believe that ThermalGrasp is a first step towards not forcing users to choose between either feeling thermal feedback or being able to engage with grasping/walking in interactive experiences.

The proliferation of new technologies has led to a proliferation of unwanted electronic devices. E-waste is the largest-growing consumer waste-stream worldwide, but also an issue often ignored. In fact, HCI primarily focuses on designing and understanding device interactions during one segment of their lifecycles—while users use them. Researchers overlook a significant space—when devices are no longer “useful” to the user, such as after breakdown or obsolescence. We argue that HCI can learn from experts who upcycle e-waste and give it second lives in electronics projects, art projects, educational workshops, and more. To acquire and translate this knowledge to HCI, we interviewed experts who unmake e-waste. We explore their practices through the lens of unmaking both when devices are physically unmade and when the perception of e-waste is unmade once waste becomes, once again, useful . Last, we synthesize findings into takeaways for how HCI can engage with the issue of e-waste.

Low-fidelity prototyping is so foundational to Human-Computer Interaction, appearing in most early design phases. So, how do experts prototype olfactory experiences? We interviewed eight experts and found that they do not because no process supports this. Thus, we engineered Smell & Paste, a low-fidelity prototyping toolkit. Designers assemble olfactory proofs-of-concept by pasting scratch-and-sniff stickers onto a paper tape. Then, they test the interaction by advancing the tape in our 3D-printed (or cardboard) cassette, which releases the smells via scratching. Our toolkit uses commodity materials; keeps iterations quick, approachable, and cheap; and circumvents electronics, programming, and chemical handling. We evaluated Smell & Paste in two studies. We found that the toolkit was approachable to people of any technical background and that novices and experts appropriated and extended the toolkit, making it personalized. Novices produced prototypes quickly, and experts were excited about the kit's technical affordances and integrating it into their practice.

Abstract. We engineered LipIO, a novel device enabling the lips to be used simultaneously as an input and output surface. LipIO comprises two overlapping flexible electrode arrays: an outward-facing array for capacitive touch and a lip-facing array for electrotactile stimulation. While wearing LipIO, users feel the interface's state via lip stimulation and respond by touching their lip with their tongue or opposing lip. More importantly, LipIO provides co-located tactile feedback that allows users to feel where in the lip they are touching—this is key to enabling eyes- and hands-free interactions. Our three studies verified participants perceived electrotactile output on their lips and subsequently touched the target location with their tongue with an average accuracy of 93%, while wearing LipIO with five I/O electrodes with co-located feedback. Finally, we demonstrate the potential of LipIO in four exemplary applications that illustrate how it enables new types of eyes- and hands-free micro-interactions.

Vertical force-feedback is extremely rare in mainstream interactive experiences. This happens because existing haptic devices capable of sufficiently strong forces that would modify a user's jump require grounding (e.g., motion platforms or pulleys) or cumbersome actuators (e.g., large propellers attached or held by the user). To enable interactive experiences to feature jump-based haptics without sacrificing wearability, we propose JumpMod, an untethered backpack that modifies one's sense of jumping. JumpMod achieves this by moving a weight up/down along the user's back, which modifies perceived jump momentum—creating accelerated & decelerated jump sensations. In our second study, we empirically found that our device can render five effects: jump higher, land harder/softer, pulled higher/lower. Based on these, we designed four jumping experiences for VR & sports. Finally, in our third study, we found that participants preferred wearing our device in an interactive context, such as one of our jump-based VR applications.

We present a technique to render tactile feedback to the palmar side of the hand while keeping it unobstructed and, thus, preserving manual dexterity during interactions with physical objects. We implement this by applying electro-tactile stimulation only to the back of the hand and to the wrist. In our approach, there are no electrodes on the palmar side, yet that is where tactile sensations are felt. While we place electrodes outside the user's palm, we do so in strategic locations that conduct the electrical currents to the median/ulnar nerves, causing tactile sensations on the palmar side of the hand. In our user studies, we demonstrated that our approach renders tactile sensations to 11 different locations on the palmar side while keeping users’ palms free for dexterous manipulations. Our approach enables new applications such as tactile notifications during dexterous activities or VR experiences that rely heavily on physical props.

Prior research has explored modifying taste through electrical stimulation. While promising, such interfaces often only elicit taste changes while in contact with the user's tongue (e.g., cutlery with electrodes), making them incompatible with eating and swallowing real foods. Moreover, most interfaces cannot selectively alter basic tastes, but only the entire flavor profile (e.g., cannot selectively alter bitterness). To tackle this, we propose taste retargeting, a method of altering taste perception by delivering chemical modulators to the mouth before eating. These modulators temporarily change the response of taste receptors to foods, selectively suppressing or altering basic tastes. Our first study identified six accessible taste modulators that suppress salty, umami, sweet, or bitter and transform sour into sweet. Using these findings, we demonstrated an interactive application of this technique with the example of virtual reality, which we validated in our second study. We found that taste retargeting reduced the flavor mismatch between a food prop and other virtual foods.

The amount of e-waste generated by discarding devices is enormous but options for recycling remain limited. However, inside a discarded device (from consumer devices to one's own prototypes), an electronics designer could find dozens to thousands of reusable components, including microcontrollers, sensors, voltage regulators, etc. Despite this, existing electronic design tools assume users will buy all components anew. To tackle this, we propose ecoEDA, an interactive tool that enables electronics designers to explore recycling electronic components during the design process. We accomplish this via (1) creating suggestions to assist users in identifying and designing with recycled components; and (2) maintaining a library of useful data relevant to reuse (e.g., allowing users to find which devices contain which components). Through example use-cases, we demonstrate how our tool can enable various pathways to recycling e-waste. To evaluate it, we conducted a user study where participants used our tool to create an electronic schematic with components from torn-down e-waste devices. We found that participants’ designs made with ecoEDA featured an average of 66% of recycled components. Last, we reflect on challenges and opportunities for building software that promotes e-waste reuse.

Users often 3D model enclosures that interact with significant heat sources, such as electronics or appliances that generate heat (e.g., CPU, motor, lamps, etc.). While parts made by users might function well aesthetically or structurally, they are rarely thermally-sound. This happens because heat transfer is non-intuitive; thus, engineering thermal solutions is not straightforward. To tackle this, we developed ThermalRouter, a CAD plugin that assists with improving the thermal performance of their models. ThermalRouter automatically converts regions of the model to be made from thermally-conductive materials (such as nylon or metallic-silicone). These regions act as heat channels, branching away from hotspots to dissipate heat. The key is that ThermalRouter automatically simulates the thermal performance of many possible heat channel configurations and presents the user with the most thermally-sound design (e.g., lowest temperature). Furthermore, it allows users to customize by balancing costs, indicating non-modifiable geometry, etc. Most importantly, ThermalRouter achieves this without requiring manual labor to set up or parse the results of complex thermal simulations.

Electrical muscle stimulation (EMS) became a popular method for force-feedback without mechanical-actuators. While much has been written about the advantages of EMS, not much work has investigated circumventing its key limitations: (1) as impulses traverse the skin, they cause an uncomfortable “tingling”; (2) impulses are delivered via gelled-electrodes, which not only require direct skin contact (must be worn under clothes); but, also (3) dry up after a few hours. To tackle these, we explore switching from electrical to magnetic muscle stimulation (MMS), via electromagnetic fields generated by coils. The first advantage is that MMS coils do not require direct skin contact and can actuate up to 5 cm away (Study#1)—this enables applications not possible with EMS, such as stimulation over the clothes and without ever replacing electrodes. Second, and more important, MMS results in ∼50 % less discomfort caused by tingling than EMS (Study#2). We found that reducing this tingling discomfort has two downstream effects for interactive systems: (1) participants rated MMS force-feedback as more realistic than that of EMS (Study#3); and (2) participants could more accurately perceive the pose actuated by the interactive system (Study#4). Finally, we demonstrated applications where our proposed switch from EMS to MMS improves user experience, including for VR feedback, gaming, and pose-control.

Haptic interfaces have been extended to the feet to enhance foot-based activities, such as guidance while walking or stepping on virtual textures. Most feet haptics use mechanical actuators, namely vibration motors. However, we argue that vibration motors are not the ideal actuators for all feet haptics. Instead, we demonstrate that electrotactile stimulation provides qualities that make it a powerful feet-haptic interface: (1) Users wearing electrotactile can not only feel the stimulation but can also better feel the terrain under their feet—this is critical as our feet are also responsible for the balance on uneven terrains and stairs—electrotactile achieves this improved “feel-through” effect because it is thinner than vibrotactile actuators, at 0.1 mm in our prototype; (2) While a single vibrotactile actuator will also vibrate surrounding skin areas, we found improved two-point discrimination thresholds for electrotactile; (3) Electrotactile can be applied directly to soles, insoles or socks, enabling new applications such as barefoot interactive experiences or without requiring users to have custom-shoes with built-in vibration motors. Finally, we demonstrate applications in which electrotactile feet interfaces allow users to feel not only virtual information but also the real terrain under their shoes, such as a VR experience where users walk on ground props and a tactile navigation system that augments the ground with virtual tactile paving to assist pedestrians in low-vision situations.

We propose a novel interface concept in which interactive systems directly manipulate the user's head orientation. We implement this using electrical-muscle-stimulation (EMS) of the neck muscles, which turns the head around its yaw (left/right) and pitch (up/down) axis. As the first exploration of EMS for head actuation, we characterized which muscles can be robustly actuated. Second, we evaluated the accuracy of our system for actuating participants' head orientation towards static targets and trajectories. Third, we demonstrated how it enables interactions not possible before by building a range of applications, such as (1) synchronizing head orientations of two users, which enables a user to communicate head nods to another user while listening to music, and (2) directly changing the user's head orientation to locate objects in AR. Finally, in our second study, participants felt that our head actuation contributed positively to their experience in four distinct applications.

Researchers have been exploring how incorporating care-based interactions can change the user's attitude & relationship towards an interactive device. This is typically achieved through virtual care where users care for digital entities. In this paper, we explore this concept further by investigating how physical care for a living organism, embedded as a functional component of an interactive device, also changes user-device relationships. Living organisms differ as they require an environment conducive to life, which in our concept, the user is responsible for providing by caring for the organism (e.g., feeding it). We instantiated our concept by engineering a smartwatch that includes a slime mold that physically conducts power to a heart rate sensor inside the device, acting as a living wire. In this smartwatch, the availability of heart-rate sensing depends on the health of the slime mold—with the user's care, the slime mold becomes conductive and enables the sensor; conversely, without care, the slime mold dries and disables the sensor (resuming care resuscitates the slime mold). To explore how our living device was perceived by users, we conducted a study where participants wore our slime mold-integrated smartwatch for 9-14 days. We found that participants felt a sense of responsibility, developed a reciprocal relationship, and experienced the organism's growth as a source of affect. Finally, to allow engineers and designers to expand on our work, we abstract our findings into a set of technical and design recommendations when engineering an interactive device that incorporates this type of care-based relationship.

We engineered DigituSync, a passive-exoskeleton that physically links two hands together, enabling two users to adaptively transmit finger movements in real-time. It uses multiple four-bar linkages to transfer both motion and force, while still preserving congruent haptic feedback. Moreover, we implemented a variable-length linkage that allows adjusting the force transmission ratio between the two users and regulates the amount of intervention, which enables users to customize their learning experience. DigituSync's benefits emerge from its passive design: unlike existing haptic devices (motor-based exoskeletons or electrical muscle stimulation), DigituSync has virtually no latency and does not require batteries/electronics to transmit or adjust movements, making it useful and safe to deploy in many settings, such as between students and teachers in a classroom. We validated DigituSync by means of technical evaluations and a user study, demonstrating that it instantly transfers finger motions and forces with the ability of adaptive force transmission, which allowed participants to feel more control over their own movements and to feel the teacher's intervention was more responsive. We also conducted two exploratory sessions with a music teacher and deaf-blind users, which allowed us to gather experiential insights from the teacher's side and explore DigituSync in applications.

With the proliferation of consumer-level virtual reality (VR) devices, users started experiencing VR in less controlled environments, such as in social gatherings and public areas. While the current VR hardware provides an increasingly immersive experience, it ignores stimuli originating from the physical surroundings that distract users from the VR experience. To block distractions from the outside world, many users wear noise-canceling headphones. However, this is insufficient to block loud or transient sounds (e.g., drilling or hammering) and, especially, multi-modal distractions (e.g., air drafts, temperature shifts from an A/C, construction vibrations, or food smells). To tackle this, we explore a new concept, where we directly integrate the distracting stimuli from the user's physical surroundings into their virtual reality experience to enhance presence. Using our approach, an otherwise distracting wind gust can be directly mapped to the sway of trees in a VR experience that already contains trees. Using our novel approach, we demonstrate how to integrate a range of distractive stimuli into the VR experience, such as haptics (temperature, vibrations, touch), sounds, and smells. To validate our approach, we conducted three user studies and a technical evaluation. First, to validate our key principle, we conducted a controlled study where participants were exposed to distractions while playing a VR game. We found that our approach improved users’ sense of presence, compared to wearing noise-canceling headphones. From these results, we engineered a sensing module that detects a set of simple distractive signals (e.g., sounds, winds, and temperature shifts). We validated our hardware in a technical evaluation and in an out-of-lab study where participants played VR games in an uncontrolled environment. Moreover, to gather the perspective of VR content creators that might one day utilize a system inspired by our findings, we invited game designers to use our approach and collected their feedback and VR designs. Finally, we present design considerations for mapping distracting external stimuli and discuss ethical considerations of integrating real-world stimuli into virtual reality.

We propose a new technical approach to implement untethered VR haptic devices that contain no battery, yet can render on-demand haptic feedback. The key is that via our approach, a haptic device charges itself by harvesting the user's kinetic energy (i.e., movement)—even without the user needing to realize this. This is achieved by integrating the energy-harvesting with the virtual experience, in a responsive manner. Whenever our batteryless haptic device is about to lose power, it switches to harvesting mode (by engaging its clutch to a generator) and, simultaneously, the VR headset renders an alternative version of the current experience that depicts resistive forces (e.g., rowing a boat in VR). As a result, the user feels realistic haptics that corresponds to what they should be feeling in VR, while unknowingly charging the device via their movements. Once the haptic device's supercapacitors are charged, they wake up its microcontroller to communicate with the VR headset. The VR experience can now use the recently harvested power for on-demand haptics, including vibration, electrical or mechanical force-feedback; this process can be repeated, ad infinitum. We instantiated a version of our concept by implementing an exoskeleton (with vibration, electrical & mechanical force-feedback) that harvests the user's arm movements. We validated it via a user study, in which participants, even without knowing the device was harvesting, rated its’ VR experience as more realistic & engaging than with a baseline VR setup. Finally, we believe our approach enables haptics for prolonged uses, especially useful in untethered VR setups, since devices capable of haptic feedback are traditionally only reserved for situations with ample power. Instead, with our approach, a user who engages in hours-long VR and grew accustomed to finding a battery-dead haptic device that no longer works, will simply resurrect the haptic device with their movement.

Force-feedback enhances digital touch by enabling users to share non-verbal aspects such as rhythm, poses, and so on. To achieve this, interfaces actuate the user’s to touch involuntarily (using exoskeletons or electrical-muscle-stimulation); we refer to this as computer-driven touch. Unfortunately, forcing users to touch causes a loss of their sense of agency. While researchers found that delaying the timing of computer-driven touch preserves agency, they only considered the naïve case when user-driven touch is aligned with computer-driven touch. We argue this is unlikely as it assumes we can perfectly predict user-touches. But, what about all the remainder situations: when the haptics forces the user into an outcome they did not intend or assists the user in an outcome they would not achieve alone? We unveil, via an experiment, what happens in these novel situations. From our findings, we synthesize a framework that enables researchers of digital-touch systems to trade-off between haptic-assistance vs. sense-of-agency.

We propose a novel type of olfactory device that creates a stereo-smell experience, i.e., directional information about the location of an odor, by rendering the readings of external odor sensors as trigeminal sensations using electrical stimulation of the user's nasal septum. The key is that the sensations from the trigeminal nerve, which arise from nerve-endings in the nose, are perceptually fused with those of the olfactory bulb (the brain region that senses smells). As such, we propose that electrically stimulating the trigeminal nerve is an ideal candidate for stereo-smell augmentation/substitution that, unlike other approaches, does not require implanted electrodes in the olfactory bulb. To realize this, we engineered a self-contained device that users wear across their nasal septum. Our device outputs by stimulating the user's trigeminal nerve using electrical impulses with variable pulse-widths; and it inputs by sensing the user's inhalations using a photoreflector. It measures 10x23 mm and communicates with external gas sensors using Bluetooth. In our user study, we found the key electrical waveform parameters that enable users to feel an odor's intensity (absolute electric charge) and direction (phase order and net charge). In our second study, we demonstrated that participants were able to localize a virtual smell source in the room by using our prototype without any previous training. Using these insights, our device enables expressive trigeminal sensations and could function as an assistive device for people with anosmia, who are unable to smell.

We propose a novel modality for active biometric authentication: electrical muscle stimulation (EMS). To explore this, we engineered an interactive system, which we call ElectricAuth, that stimulates the user’s forearm muscles with a sequence of electrical impulses (i.e., EMS challenge) and measures the user’s involuntary finger movements (i.e., response to the challenge). ElectricAuth leverages EMS’s intersubject variability, where the same electrical stimulation results in different movements in different users because everybody’s physiology is unique (e.g., differences in bone and muscular structure, skin resistance and composition, etc.). As such, ElectricAuth allows users to login without memorizing passwords or PINs.

Current head-mounted displays enable users to explore virtual worlds by simply walking through them (i.e., real-walking VR). This led researchers to create haptic displays that can also simulate different types of elevation shapes. However, existing shape-changing floors are limited by their tabletop scale or the coarse resolution of the terrains they can display due to the limited number of actuators and low vertical resolution. To tackle this challenge, we introduce Elevate, a dynamic and walkable pin-array floor on which users can experience not only large variations in shapes but also the details of the underlying terrain. Our system achieves this by packing 1200 pins arranged on a 1.80 × 0.60m platform, in which each pin can be actuated to one of ten height levels (resolution: 15mm/level). To demonstrate its applicability, we present our haptic floor combined with four walkable applications and a user study that reported increased realism and enjoyment.

Abstract: Force feedback devices, such as motor-based exoskeletons or wearables based on electrical muscle stimulation (EMS), have the unique potential to accelerate users’ own reaction time (RT). However, this speedup has only been explored while the device is attached to the user. In fact, very little is known regarding whether this faster reaction time still occurs after the user removes the device from their bodies–this is precisely what we investigated by means of a simple reaction time (RT) experiment, in which participants were asked to tap as soon as they saw an LED flashing. Participants experienced this in three EMS conditions: (1) fast-EMS, the electrical impulses were synced with the LED; (2) agency-EMS, the electrical impulse was delivered 40ms faster than the participant’s own RT, which prior work has shown to preserve one’s sense of agency over this movement; and, (3) late-EMS: the impulse was delivered after the participant’s own RT. Our results revealed that the participants’ RT was significantly reduced by approximately 8ms (up to 20ms) only after training with the agency-EMS condition. This finding suggests that the prioritizing agency during EMS training is key to motor-adaptation, i.e., it enables a faster motor response even after the user has removed the EMS device from their body.

We propose a new type of haptic actuator, which we call MagnetIO, that is comprised of two parts: one battery-powered voice-coil worn on the user's fingernail and any number of interactive soft patches that can be attached onto any surface (everyday objects, user's body, appliances, etc.). When the user's finger wearing our voice-coil contacts any of the interactive patches it detects its magnetic signature via magnetometer and vibrates the patch, adding haptic feedback to otherwise input-only interactions. To allow these passive patches to vibrate, we make them from silicone with regions doped with polarized neodymium powder, resulting in soft and stretchable magnets. This stretchable form-factor allows them to be wrapped to the user's body or everyday objects of various shapes. We demonstrate how these add haptic output to many situations, such as adding haptic buttons to the walls of one's home. In our technical evaluation, we demonstrate that our interactive patches can be excited across a wide range of frequencies (0-500 Hz) and can be tuned to resonate at specific frequencies based on the patch's geometry. Furthermore, we demonstrate that MagnetIO's vibration intensity is as powerful as a typical linear resonant actuator (LRA); yet, unlike these rigid actuators, our passive patches operate as springs with multiple modes of vibration, which enables a wider band around its resonant frequency than an equivalent LRA.

We propose a technique that allows an unprecedented level of dexterity in electrical muscle stimulation (EMS), i.e., it allows interactive EMS-based devices to flex the user's fingers independently of each other. EMS is a promising technique for force feedback because of its small form factor when compared to mechanical actuators. However, the current EMS approach to flexing the user's fingers (i.e., attaching electrodes to the base of the forearm, where finger muscles anchor) is limited by its inability to flex a target finger's metacarpophalangeal (MCP) joint independently of the other fingers. In other words, current EMS devices cannot flex one finger alone, they always induce unwanted actuation to adjacent fingers. To tackle the lack of dexterity, we propose and validate a new electrode layout that places the electrodes on the back of the hand, where they stimulate the interossei/lumbricals muscles in the palm, which have never received attention with regards to EMS. In our user study, we found that our technique offers four key benefits when compared to existing EMS electrode layouts: our technique (1) flexes all four fingers around the MCP joint more independently; (2) has less unwanted flexion of other joints (such as the proximal interphalangeal joint); (3) is more robust to wrist rotations; and (4) reduces calibration time. Therefore, our EMS technique enables applications for interactive EMS systems that require a level of flexion dexterity not available until now. We demonstrate the improved dexterity with four example applications: three musical instrumental tutorials (piano, drum, and guitar) and a VR application that renders force feedback in individual fingers while manipulating a yo-yo.

We propose a nail-mounted foldable haptic device that provides tactile feedback to mixed reality (MR) environments by pressing against the user's fingerpad when a user touches a virtual object. What is novel in our device is that it quickly tucks away when the user interacts with real-world objects. Its design allows it to fold back on top of the user's nail when not in use, keeping the user's fingerpad free to, for instance, manipulate handheld tools and other objects while in MR. To achieve this, we engineered a wireless and self-contained haptic device, which measures 24×24×41 mm and weighs 9.5 g. Furthermore, our foldable end-effector also features a linear resonant actuator, allowing it to render not only touch contacts (i.e., pressure) but also textures (i.e., vibrations). We demonstrate how our device renders contacts with MR surfaces, buttons, low- and high-frequency textures. In our first user study, we found that participants perceived our device to be more realistic than a previous haptic device that also leaves the fingerpad free (i.e., fingernail vibration). In our second user study, we investigated the participants’ experience while using our device in a real-world task that involved physical objects. We found that our device allowed participants to use the same finger to manipulate handheld tools, small objects, and even feel textures and liquids, without much hindrance to their dexterity, while feeling haptic feedback when touching MR interfaces.

We propose a new class of haptic devices that provide haptic sensations by delivering liquid-stimulants to the user's skin; we call this chemical haptics. Upon absorbing these stimulants, which contain safe and small doses of key active ingredients, receptors in the user's skin are chemically triggered, rendering distinct haptic sensations. We identified five chemicals that can render lasting haptic sensations: tingling (sanshool), numbing (lidocaine), stinging (cinnamaldehyde), warming (capsaicin), and cooling (menthol). To enable the application of our novel approach in a variety of settings (such as VR), we engineered a self-contained wearable that can be worn anywhere on the user's skin (e.g., face, arms, legs). Implemented as a soft silicone patch, our device uses micropumps to push the liquid stimulants through channels that are open to the user's skin, enabling topical stimulants to be absorbed by the skin as they pass through. Our approach presents two unique benefits. First, it enables sensations, such as numbing, not possible with existing haptic devices. Second, our approach offers a new pathway, via the skin's chemical receptors, for achieving multiple haptic sensations using a single actuator, which would otherwise require combining multiple actuators (e.g., Peltier, vibration motors, electro-tactile stimulation). We evaluated our approach by means of two studies. In our first study, we characterized the temporal profiles of sensations elicited by each chemical. Using these insights, we designed five interactive VR experiences utilizing chemical haptics, and in our second user study, participants rated these VR experiences with chemical haptics as more immersive than without. Finally, as the first work exploring the use of chemical haptics on the skin, we offer recommendations to designers for how they may employ our approach for their interactive experiences.

Electrical muscle stimulation (EMS) is an emergent technique that miniaturizes force feedback, especially popular for untethered haptic devices, such as mobile gaming, VR, or AR. However, the actuation displayed by interactive systems based on EMS is coarse and imprecise. EMS systems mostly focus on inducing movements in large muscle groups such as legs, arms, and wrists; whereas individual finger poses, which would be required, for example, to actuate a user's fingers to fingerspell even the simplest letters in sign language, are not possible. The lack of dexterity in EMS stems from two fundamental limitations: (1) lack of independence: when a particular finger is actuated by EMS, the current runs through nearby muscles, causing unwanted actuation of adjacent fingers; and, (2) unwanted oscillations: while it is relatively easy for EMS to start moving a finger, it is very hard for EMS to stop and hold that finger at a precise angle; because, to stop a finger, virtually all EMS systems contract the opposing muscle, typically achieved via controllers (e.g., PID)—unfortunately, even with the best controller tuning, this often results in unwanted oscillations. To tackle these limitations, we propose dextrEMS, an EMS-based haptic device featuring mechanical brakes attached to each finger joint. The key idea behind dextrEMS is that while the EMS actuates the fingers, it is our mechanical brake that stops the finger in a precise position. Moreover, it is also the brakes that allow dextrEMS to select which fingers are moved by EMS, eliminating unwanted movements by preventing adjacent fingers from moving. We implemented dextrEMS as an untethered haptic device, weighing only 68g, that actuates eight finger joints independently (metacarpophalangeal and proximal interphalangeal joints for four fingers), which we demonstrate in a wide range of haptic applications, such as assisted fingerspelling, a piano tutorial, guitar tutorial, and a VR game. Finally, in our technical evaluation, we found that dextrEMS outperformed EMS alone by doubling its independence and reducing unwanted oscillations.

We propose a haptic device that alters the perceived softness of real rigid objects without requiring to instrument the objects. Instead, our haptic device works by restricting the user's fingerpad lateral deformation via a hollow frame that squeezes the sides of the fingerpad. This causes the fingerpad to become bulgier than it originally was—when users touch an object's surface with their now-restricted fingerpad, they feel the object to be softer than it is. To illustrate the extent of softness illusion induced by our device, touching the tip of a wooden chopstick will feel as soft as a rubber eraser. Our haptic device operates by pulling the hollow frame using a motor. Unlike most wearable haptic devices, which cover up the user's fingerpad to create force sensations, our device creates softness while leaving the center of the fingerpad free, which allows the users to feel most of the object they are interacting with. This makes our device a unique contribution to altering the softness of everyday objects, creating “buttons” by softening protrusions of existing appliances or tangibles, or even, altering the softness of handheld props for VR. Finally, we validated our device through two studies: (1) a psychophysics study showed that the device brings down the perceived softness of any object between 50A-90A to around 40A (on Shore A hardness scale); and (2) a user study demonstrated that participants preferred our device for interactive applications that leverage haptic props, such as making a VR prop feel softer or making a rigid 3D printed remote control feel softer on its button.

We explore a temperature illusion that uses low-powered electronics and enables the miniaturization of simple warm and cool sensations. Our illusion relies on the properties of certain scents, such as the coolness of mint or hotness of peppers. These odors trigger not only the olfactory bulb, but also the nose's trigeminal nerve, which has receptors that respond to both temperature and chemicals. To exploit this, we engineered a wearable device based on micropumps and an atomizer that emits up to three custom-made "thermal" scents directly to the user's nose. Breathing in these scents causes the user to feel warmer or cooler. We demonstrate how our device renders warmth and cooling sensations in virtual experiences. In our first study, we evaluated six candidate "thermal" scents. We found two hot-cold pairs, with one pair being less identifiable by odor. In our second study, pParticipants rated VR experiences with our device trigeminal stimulants as significantly warmer or cooler than the baseline conditions. Lastly, we believe this offers an alternative to existing thermal feedback devices, which unfortunately rely on power-hungry heat-lamps or Peltier-elements.

We engineered a wearable microphone jammer that is capable of disabling microphones in its user's surroundings, including hidden microphones. Our device is based on a recent exploit that leverages the fact that when exposed to ultrasonic noise, commodity microphones will leak the noise into the audible range.

Human-Computer Integration (HInt) is an emerging paradigm in which computational and human systems are closely interwoven. Integrating computers with the human body is not new. however, we believe that with rapid technological advancements, increasing real-world deployments, and growing ethical and societal implications, it is critical to identify an agenda for future research. We present a set of challenges for HInt research, formulated over the course of a five-day workshop consisting of 29 experts who have designed, deployed and studied HInt systems. This agenda aims to guide researchers in a structured way towards a more coordinated and conscientious future of human-computer integration.

We engineered an exoskeleton, which we call HandMorph, that approximates the experience of having a smaller grasping range. It uses mechanical links to transmit motion from the wearer's fingers to a smaller hand with five anatomically correct fingers. The result is that HandMorph miniaturizes a wearer's grasping range while transmitting haptic feedback.

Designing immersion is the key challenge in virtual reality; this challenge has driven advancements in displays, rendering and recently, haptics. To increase our sense of physical immersion, for instance, vibrotactile gloves render the sense of touching, while electrical muscle stimulation (EMS) renders forces. Unfortunately, the established metric to assess the effectiveness of haptic devices relies on the user's subjective interpretation of unspecific, yet standardized, questions.

We enable preemptive force-feedback systems to speed up human reaction time without fully compromising the user's sense of agency. Typically these interfaces actuate by means of electrical muscle stimulation (EMS) or mechanical actuators; they preemptively move the user to perform a task, such as to improve movement performance (e.g., EMS-assisted drumming). Unfortunately, when using preemptive force-feedback users do not feel in control and loose their sense of agency. We address this by actuating the user's body, using EMS, within a particular time window (160 ms after visual stimulus), which we found to speed up reaction time by 80 ms in our first study. With this preemptive timing, when the user and system move congruently, the user feels that they initiated the motion, yet their reaction time is faster than usual. As our second study demonstrated, this particular timing significantly increased agency when compared to the current practice in EMS-based devices. We conclude by illustrating, using examples from the HCI literature, how to leverage our findings to provide more agency to automated haptic interfaces.

Force feedback is said to be the next frontier in virtual reality (VR). Recently, with consumers pushing forward with untethered VR, researchers turned away from solutions based on bulky hardware (e.g., exoskeletons and robotic arms) and started exploring smaller portable or wearable devices. However, when it comes to rendering inertial forces, such as when moving a heavy object around or when interacting with objects with unique mass properties, current ungrounded force feedback devices are unable to provide quick weight shifting sensations that can realistically simulate weight changes over 2D surfaces. In this paper we introduce Aero-plane, a force-feedback handheld controller based on two miniature jet propellers that can render shifting weights of up to 14 N within 0.3 seconds. Through two user studies we: (1) characterize the users' ability to perceive and correctly recognize different motion paths on a virtual plane while using our device; and, (2) tested the level of realism and immersion of the controller when used in two VR applications (a rolling ball on a plane, and using kitchen tools of different shapes and sizes). Lastly, we present a set of applications that further explore different usage cases and alternative form-factors for our device.

We present a mobile system that enhances mixed reality experiences and games with force feedback by means of electrical muscle stimulation (EMS). The benefit of our approach is that it adds physical forces while keeping the users' hands free to interact unencumbered-not only with virtual objects, but also with physical objects, such as props and appliances. We demonstrate how this supports three classes of applications along the mixed-reality continuum: (1) entirely virtual objects, such as furniture with EMS friction when pushed or an EMS-based catapult game. (2) Virtual objects augmented via passive props with EMS-constraints, such as a light control panel made tangible by means of a physical cup or a balance-the-marble game with an actuated tray. (3) Augmented appliances with virtual behaviors, such as a physical thermostat dial with EMS-detents or an escape-room that repurposes lamps as levers with detents. We present a user-study in which participants rated the EMS-feedback as significantly more realistic than a no-EMS baseline.

We present metamaterial textures---3D printed surface geometries that can perform a controlled transition between two or more textures. Metamaterial textures are integrated into 3D printed objects and allow designing how the object interacts with the environment and the user's tactile sense. Inspired by foldable paper sheets ("origami") and surface wrinkling, our 3D printed metamaterial textures consist of a grid of cells that fold when compressed by an external global force. Unlike origami, however, metamaterial textures offer full control over the transformation, such as in between states and sequence of actuation. This allows for integrating multiple textures and makes them useful, e.g., for exploring parameters in the rapid prototyping of textures. Metamaterial textures are also robust enough to allow the resulting objects to be grasped, pushed, or stood on. This allows us to make objects, such as a shoe sole that transforms from flat to treaded, a textured door handle that provides tactile feedback to visually impaired users, and a configurable bicycle grip. We present an editor assists users in creating metamaterial textures interactively by arranging cells, applying forces, and previewing their deformation.

We present TrussFormer, an integrated end-to-end system that allows users to 3D print large-scale kinetic structures, i.e., structures that involve motion and deal with dynamic forces. TrussFormer builds on TrussFab, from which it inherits the ability to create static large-scale truss structures from 3D printed connectors and PET bottles. TrussFormer adds movement to these structures by placing linear actuators into them: either manually, wrapped in reusable components called assets, or by demonstrating the intended movement. TrussFormer verifies that the resulting structure is mechanically sound and will withstand the dynamic forces resulting from the motion. To fabricate the design, TrussFormer generates the underlying hinge system that can be printed on standard desktop 3D printers. We demonstrate TrussFormer with several example objects, including a 6 legged walking robot and a 4m tall animatronics dinosaur with 5 degrees of freedom.

We explore how to add haptics to walls and other heavy objects in virtual reality. When a user tries to push such an object, our system actuates the user's shoulder, arm, and wrist muscles by means of electrical muscle stimulation, creating a counter force that pulls the user's arm backwards. Our device accomplishes this in a wearable form factor.

Recently, researchers started to engineer not only the outer shape of objects, but also their internal microstructure. Such objects, typically based on 3D cell grids, are also known as metamaterials. Metamaterials have been used, for example, to create materials with soft and hard regions.

We explore how to create interactive systems based on electrical muscle stimulation that offer expressive output. We present muscle-plotter, a system that provides users with input and output access to a computer system while on the go. Using pen-on-paper interaction, muscle-plotter allows users to engage in cognitively demanding activities, such as writing math. Users write formulas using a pen and the system responds by making the users' hand draw charts and widgets. While Anoto technology in the pen tracks users' input, muscle-plotter uses electrical muscle stimulation (EMS) to steer the user's wrist so as to plot charts, fit lines through data points, find data points of interest, or fill in forms. We demonstrate the system at the example of six simple applications, including a wind tunnel simulator.

We propose a new way of eyes-free interaction for wearables. It is based on the user's proprioceptive sense, i.e., rather than seeing, hearing, or feeling an outside stimulus, users feel the pose of their own body. We have implemented a wearable device called Pose-IO that offers input and output based on proprioception. Users communicate with Pose-IO through the pose of their wrists. Users enter information by performing an input gesture by flexing their wrist, which the device senses using a 3-axis accelerometer. Users receive output from Pose-IO by find-ing their wrist posed in an output gesture, which Pose-IO actuates using electrical muscle stimulation. This mechanism allows users to interact with Pose-IO without visual or auditory senses, but through the proprioceptive sense alone. We developed three simple applications that demonstrate symmetric proprioceptive interaction, where input and output occur through the same limb, as well as asymmetric interaction, where input and output occur through different limbs. In a first user study, participants using a symmetric proprioceptive interface re-entered poses received from Pose-IO with an average accuracy of 5.8° despite the minimal bandwidth offered by the device. In a second, exploratory study, we investigated participants' emotional response to asymmetric proprioceptive interaction and the concept of the user's body serving as interface. Participants reported to enjoy the experience (4.6 out of 5).

We propose extending the affordance of objects by allowing them to communicate dynamic use, such as (1) motion (e.g., spray can shakes when touched), (2) multi-step processes (e.g., spray can sprays only after shaking), and (3) behaviors that change over time (e.g., empty spray can does not allow spraying anymore). Rather than enhancing objects directly, however, we implement this concept by enhancing the user. We call this affordance++. By stimulating the user's arms using electrical muscle stimulation, our prototype allows objects not only to make the user actuate them, but also perform required movements while merely approaching the object, such as not to touch objects that do not "want" to be touched. In our user study, affordance++ helped participants to successfully operate devices of poor natural affordance, such as a multi-functional slicer tool or a magnetic nail sweeper, and to stay away from cups filled with hot liquids.

We present impacto, a device designed to render the haptic sensation of hitting or being hit in virtual reality. The key idea that allows the small and light impacto device to simulate a strong hit is that it decomposes the stimulus: it renders the tactile aspect of being hit by tapping the skin using a solenoid; it adds impact to the hit by thrusting the user's arm backwards using electrical muscle stimulation. The device is self-contained, wireless, and small enough for wearable use, thus leaves the user unencumbered and able to walk around freely in a virtual environment. The device is of generic shape, allowing it to also be worn on legs, so as to enhance the experience of kicking, or merged into props, such as a baseball bat. We demonstrate how to assemble multiple impacto units into a simple haptic suit. Participants of our study rated impact simulated using impacto's combination of solenoid hit and electrical muscle stimulation as more realistic than either technique in isolation.

Motion platforms are used to increase the realism of virtual interaction. Unfortunately, their size and weight is proportional to the size of what they actuate. We present haptic turk, a different approach to motion platforms that is light and mobile. The key idea is to replace motors and mechanical components with humans. All haptic turk setups consist of a player who is supported by one or more turkers. The player enjoys an interactive experience, such as a flight simulation. The motion in the player's experience is generated by the turkers who manually lift, tilt, and push the player's limbs or torso. To get the timing and force right, timed motion instructions in a format familiar from rhythm games are displayed on turkers' mobile devices, which they attach to the player's body. We demonstrate a range of installations based on mobile phones, projectors, and head-mounted displays. In our user study, participants rated not only the experience as player as enjoyable (6.1/7), but also the experience as a turker (4.4/7). The approach of leveraging humans allows us to deploy our approach anytime anywhere, as we demonstrate by experimentally deploying at an art festival in the Nevada desert.

Large-scale touch surfaces have been widely studied in literature and adopted for public installations such as interactive billboards. However, current designs do not take into consideration that touching the interactive surface at different heights is not the same; for body-height displays, the bottom portion of the screen is within easier reach of the foot than the hand. We explore the design space of foot input on vertical surfaces, and propose three distinct interaction modalities: hand, foot tapping, and foot gesturing. Our design exploration pays particular attention to areas of the touch surface that were previously overlooked: out of hand's reach and close to the floor. We instantiate our design space with a working prototype of an interactive surface, in which we are able to distinguish between finger and foot tapping and extend the input area beyond the bottom of the display to support foot gestures.

Force feedback devices resist miniaturization, because they require physical motors and mechanics. We propose mobile force feedback by eliminating motors and instead actuating the user's muscles using electrical stimulation. Without the motors, we obtain substantially smaller and more energy-efficient devices. We present a prototype that fits on the back of a mobile phone. It actuates users' forearm muscles via four electrodes, which causes users' muscles to contract involuntarily, so that they tilt the device sideways. As users resist this motion using their other arm, they perceive force feedback. We demonstrate the interaction at the example of an interactive videogame in which users steer an airplane through winds rendered using force feedback. In a first user study, we found our device to cause users to produce up to 18.7N of force, when used to actuate their palm flexors. In a second study, participants played the video game de-scribed above; all ten participants reported to prefer the experience of muscle-propelled force feedback to vibrotactile feedback.

We propose using spatial gestures not only for input but also for output. Analogous to gesture input, the proposed gesture output moves the user's finger in a gesture, which the user then recognizes. We use our concept in a mobile scenario where a motion path forming a "5" informs users about new emails, or a heart-shaped path serves as a mes- sage from a friend. We built two prototypes: (1) The long- RangeOuija is a stationary prototype that offers a motion range of up to 4cm; (2) The pocketOuija is self-contained mobile device based on an iPhone with up to 1cm motion range. Both devices actuate the user's fingers by means of an actuated transparent foil overlaid onto a touchscreen. We conducted three studies with the longRangeOuija in which participants recognized 2cm marks with 97% accu- racy, Graffiti digits with 98.8%, pairs of Graffiti digits with 90.5%, and Graffiti letters with 93.4%. Participants previ- ously unfamiliar with Graffiti identified 96.2% of digits and 76.4% of letters, suggesting that properly designed gesture output is guessable. After the experiment, the same participants were able to enter 100% of Graffiti digits by heart and 92.2% of letters. This suggests that participants learned gesture input as a side effect of using gesture output on our prototypes.

Personal fabrication tools, such as laser cutters and 3D printers allow users to create precise objects quickly. However, working through a CAD system removes users from the workpiece. Recent interactive fabrication tools reintroduce this directness, but at the expense of precision.