In this paper, we explore the design and development of passive soft 3D-printed structures whose deformation can be sensed accurately without any wired connection. By 3D printing tangible interfaces consisting of flexible TPU (thermoplastic polyurethane), made from lattice structures with bespoke geometries and mechanical properties, and ferromagnetic elements using metal-infused filaments, we enable the detection of structural deformations through inductive sensing. We investigate how different ferromagnetic core configurations within flexible substrates, guided by key design parameters, influence the sensitivity, responsiveness, and deformability of the sensing system. We demonstrate that our 3D-printed inductive sensing approach allows users to switch their fully passive tangible interfaces for specialized tasks without assembly or the need to unplug wires. Our sensing approach can be integrated in portable applications, such as a smart bottle cover that captures subtle deformation to measure liquid intake, or in wearable applications, such as monitoring foot pressure in smart shoes.

What happens when your walls begin to move? This paper explores the design of human-robot interaction for architectural-scale, shape-changing environments. We present findings from two studies: (1) a series of speculative design workshops (N=20) that uncovered aspirational visions for these spaces, and (2) a task-based Wizard-of-Oz elicitation study (N=12) that grounded these visions in the challenges of practical interaction. Our workshop findings reveal a complex landscape of user desires, exposing critical tensions between proactive automation and the preservation of user autonomy, and between personalization and public ownership. Our elicitation study reveals a set of core interaction challenges related to multimodal collaboration; and, most critically: suggests the need for a modality-agnostic model of evolving user intent. We conclude with a set of grounded proposals for creating robotic environments that are collaborative and trusted partners in everyday life.

Walking aids are critical for people with mobility impairments, yet current options remain unsatisfactory. Static knee braces are lightweight and affordable, but their rigid joints force users into unnatural gait patterns, leading to fatigue, reduced safety, and high abandonment rates. Robotic exoskeletons, in contrast, offer dynamic assistance that adapts to gait phases but rely on sensors, motors, and batteries that make them heavy, complex, and prohibitively expensive.

It can be hard to design a physical structure entirely within the confines of a computer monitor. To better capture the interplay between real-world objects and a designer's work-in-progress, practitioners will often go through a sequence of low-fidelity prototypes (paper, clay, foam) before arriving at a form that satisfies both functional and aesthetic concerns. While necessary, this model-making process can be quite time-consuming, particularly at larger scales, and the resulting geometry can be difficult to translate into a CAD environment, where it will be further refined. This paper introduces a user-adjustable, room-scale, "shape-aware" mesh structure for low-fidelity prototyping. A user physically manipulates the mesh by lengthening and shortening the edges, altering the overall curvature and sculpting coarse forms. The edges are equipped with resistive length sensors, and transmit their configuration to a central computer. The structure can later be reproduced in software, connecting this prototyping stage to the larger computational design pipeline.

We propose augmenting initially passive structures built from simple repeated cells, with novel active units to enable dynamic, shape-changing, and robotic applications. Inspired by metamaterials that can employ mechanisms, we build a framework that allows users to configure cells of this passive structure to allow it to perform complex tasks. A key benefit is that our structures can be repeatedly (re)configured by users inserting our configuration units to turn the passive material into, e.g., locomotion robots, integrated motion platforms, or interactive interfaces, as we demonstrate in this paper.

Complex actuators in a small form factor are essential for dynamic interfaces. In this paper, we propose ConeAct, a cone-shaped actuator that can extend, contract, and bend in multiple directions to support rich expression in dynamic materials. A key benefit of our actuator is that it is self-contained and portable as the whole system. We designed our actuator’s structure to be multistable to hold its shape passively, while we control its transition between states using active materials, i.e., shape memory alloys. We present the design space by showcasing our actuator module as part of self-rolling robots, reconfigurable deployable structures, volumetric shape-changing objects and tactile displays. To assist users in designing such structures, we present an interactive editor including simulation to design such interactive capabilities.

Building shape-changing, robotic or deployable interfaces is notoriously difficult, often requiring fabrication skills or specialized hardware. We present a construction kit for novice users to enable immediate hands-on exploration of custom shape-changing or robotic structures through pneumatically actuated origami tubes. Our construction kit consists of generic origami actuators that can be combined with our connectors to result in a variety of shapes and motions. Our novel connectors support simple mechanical connections through snap fits and pneumatic configurations through plugs. To assist users in designing and previewing complex deformations, we provide a design tool that generates control code for user-defined designs. We envision that our construction kit will facilitate creativity support tasks (e.g. product design) or education. We demonstrate the capabilities of our construction kit with three application examples and a series of objects created during our co-creation study with novice users.

Robotic surfaces, whose form and function are under computational control, offer exciting new possibilities for environments that can be customized to fit user-specific needs. When these surfaces can be reprogrammed, a once-static structure can be repurposed to serve multiple different roles over time. In this paper, we introduce such a system. This is an architectural-scale robotic surface, which is able to begin in a neutral state, assume a desired functional shape, and later return to its neutral (flat) position. The surface can then assume a completely different functional shape, all under program control.

Haptic feedback is important for immersive, assistive, or multimodal interfaces, but engineering devices that generalize across applications is notoriously difficult. To address the issue of versatility, we propose Parametric Haptics, geometry-based tactile feedback devices that are customizable to render a variety of tactile sensations. To achieve this, we integrate the actuation mechanism with the tactor geometry into passive 3D printable patches, which are then connected to a generic wearable actuation interface consisting of micro gear motors. The key benefit of our approach is that the 3D-printed patches are modular, can consist of varying numbers and shapes of tactors, and that the tactors can be grouped and moved by our actuation geometry over large areas of the skin. The patches are soft, thin, conformable, and easy to customize to different use cases, thus potentially enabling a large design space of diverse tactile sensations.

We present digital mechanical metamaterials that enable multiple computation loops and reprogrammable logic functions, making a significant step towards passive yet interactive devices. Our materials consist of many cells that transmit signals using an embedded bistable spring. When triggered, the bistable spring displaces and triggers the next cell. We integrate a recharging mechanism to recharge the bistable springs, enabling multiple computation rounds. Between the iterations, we enable reprogramming the logic functions after fabrication. We demonstrate that such materials can trigger a simple controlled actuation anywhere in the material to change the local shape, texture, stiffness, and display. This enables large-scale interactive and functional materials with no or a small number of external actuators. We showcase the capabilities of our system with various examples: a haptic floor with tunable stiffness for different VR scenarios, a display with easy-to-reconfigure messages after fabrication, or a tactile notification integrated into users’ desktops.

Mixed Reality (MR) systems display content freely in space, and present nearly arbitrary amounts of information, enabling ubiquitous access to digital information. This approach, however, introduces clutter and distraction if too much virtual content is shown. We present BlendMR, an optimization-based MR system that blends virtual content onto the physical objects in users’ environments to serve as ambient information displays. Our approach takes existing 2D applications and meshes of physical objects as input. It analyses the geometry of the physical objects and identifies regions that are suitable hosts for virtual elements. Using a novel integer programming formulation, our approach then optimally maps selected contents of the 2D applications onto the object, optimizing for factors such as importance and hierarchy of information, viewing angle, and geometric distortion. We evaluate BlendMR by comparing it to a 2D window baseline. Study results show that BlendMR decreases clutter and distraction, and is preferred by users. We demonstrate the applicability of BlendMR in a series of results and usage scenarios.

We present a novel design for materials that are reconfigurable by end-users. Conceptually, we propose decomposing such reconfigurable materials into (1) a generic, complex material consisting of engineered microstructures (known as metamaterials) designed to be purchased and (2) a simple configuration geometry that can be fabricated by end-users to fit their individual use cases. Specifically, in this paper we investigate reconfiguring our material’s elasticity, such that it can cover existing objects and thereby augment their material properties. Users can configure their materials by generating the configuration geometry using our interactive editor, 3D printing it using commonly available filaments (e. g., PLA), and pressing it onto the generic material for local coupling. We characterize the mechanical properties of our reconfigurable elastic metamaterial and showcase the material’s applicability as, e.g., augmentation for haptic props in virtual reality, a reconfigurable shoe sole for different activities, or a battleship-like ball game.

We propose a novel design of engineered, structured materials that leverages fast fabrication technologies, pushing them towards mass-fabrication. Specifically, our metamaterial is designed to be laser cut, to approximate the volumetric shape and allow for locally varying compliance. Traditional mechanical metamaterials consist of intricate cells arranged on a 3-dimensional grid, limiting them to 3D printing—which is slow. Our metamaterial is designed for laser cutting, which is drastically faster. Our structures are best described as ruffled strips of thin sheet material, such as paper, plastics, metals, etc. Users can interactively define the ruffles’ anisotropic stiffness directions and local density. Our computational design tool assists users by automatically optimizing the ruffle to fill the shape’s volume, and exporting the flat ruffle design ready for cutting. We demonstrate how such ruffled metamaterials can be utilized for, e.g., custom toys with locally varying compliance, custom packaging material, or lightweight formwork for architectural shells.

We present an automatic tool to approximate curved geometries with piece-wise developable surfaces. At the center of our work is an algorithm that wraps a given 3D input surface with multiple developable patches, each modeled as a discrete orthogonal geodesic net. Our algorithm features a global optimization routine for effectively finding the placement of the developable patches. After wrapping the mesh, we use these patches and a non-linear projection step to generate a surface that approximates the original input, but is also amendable to simple and efficient fabrication techniques thanks to being piecewise developable. Our algorithm allows users to steer the trade-off between approximation power and the number of developable patches used. We demonstrate the effectiveness of our approach on a range of 3D shapes. Compared to previous approaches, our results exhibit a smaller or comparable error with fewer patches to fabricate.

In this paper, we establish the underlying foundations of mechanisms that are composed of cell structures---known as metamaterial mechanisms. Such metamaterial mechanisms were previously shown to implement complete mechanisms in the cell structure of a 3D printed material, without the need for assembly. However, their design is highly challenging. A mechanism consists of many cells that are interconnected and impose constraints on each other. This leads to unobvious and non-linear behavior of the mechanism, which impedes user design. In this work, we investigate the underlying topological constraints of such cell structures and their influence on the resulting mechanism. Based on these findings, we contribute a computational design tool that automatically creates a metamaterial mechanism from user-defined motion paths. This tool is only feasible because our novel abstract representation of the global constraints highly reduces the search space of possible cell arrangements.

We present a mobile system that enhances mixed reality experiences and games with force feedback by means of electrical muscle stimulation (EMS). The benefit of our approach is that it adds physical forces while keeping the users' hands free to interact unencumbered-not only with virtual objects, but also with physical objects, such as props and appliances. We demonstrate how this supports three classes of applications along the mixed-reality continuum: (1) entirely virtual objects, such as furniture with EMS friction when pushed or an EMS-based catapult game. (2) Virtual objects augmented via passive props with EMS-constraints, such as a light control panel made tangible by means of a physical cup or a balance-the-marble game with an actuated tray. (3) Augmented appliances with virtual behaviors, such as a physical thermostat dial with EMS-detents or an escape-room that repurposes lamps as levers with detents. We present a user-study in which participants rated the EMS-feedback as significantly more realistic than a no-EMS baseline.

We present metamaterial textures---3D printed surface geometries that can perform a controlled transition between two or more textures. Metamaterial textures are integrated into 3D printed objects and allow designing how the object interacts with the environment and the user's tactile sense. Inspired by foldable paper sheets ("origami") and surface wrinkling, our 3D printed metamaterial textures consist of a grid of cells that fold when compressed by an external global force. Unlike origami, however, metamaterial textures offer full control over the transformation, such as in between states and sequence of actuation. This allows for integrating multiple textures and makes them useful, e.g., for exploring parameters in the rapid prototyping of textures. Metamaterial textures are also robust enough to allow the resulting objects to be grasped, pushed, or stood on. This allows us to make objects, such as a shoe sole that transforms from flat to treaded, a textured door handle that provides tactile feedback to visually impaired users, and a configurable bicycle grip. We present an editor assists users in creating metamaterial textures interactively by arranging cells, applying forces, and previewing their deformation.

We present TrussFormer, an integrated end-to-end system that allows users to 3D print large-scale kinetic structures, i.e., structures that involve motion and deal with dynamic forces. TrussFormer builds on TrussFab, from which it inherits the ability to create static large-scale truss structures from 3D printed connectors and PET bottles. TrussFormer adds movement to these structures by placing linear actuators into them: either manually, wrapped in reusable components called assets, or by demonstrating the intended movement. TrussFormer verifies that the resulting structure is mechanically sound and will withstand the dynamic forces resulting from the motion. To fabricate the design, TrussFormer generates the underlying hinge system that can be printed on standard desktop 3D printers. We demonstrate TrussFormer with several example objects, including a 6 legged walking robot and a 4m tall animatronics dinosaur with 5 degrees of freedom.

In this paper, we explore how to embody mechanical computation into 3D printed objects, i.e., without electronic sensors, actuators, or controllers typically used for this purpose. A key benefit of our approach is that the resulting objects can be 3D printed in one piece and thus do not require assembly. We are building on 3D printed cell structures, also known as metamaterials. We introduce a new type of cell that propagates a digital mechanical signal using an embedded bistable spring. When triggered, the embedded spring discharges and the resulting impulse triggers one or more neighboring cells, resulting in signal propagation. We extend this basic mechanism to implement simple logic functions. We demonstrate interactive objects based on this concept, such as a combination lock. We present a custom editor that allows users to model 3D objects, route signals, simulate signal flow, and synthesize cell patterns.

Recently, researchers started to engineer not only the outer shape of objects, but also their internal microstructure. Such objects, typically based on 3D cell grids, are also known as metamaterials. Metamaterials have been used, for example, to create materials with soft and hard regions.

We propose a new type of tactile displays that drag a physical tactor across the skin in 2D. We call this skin drag. We demonstrate how this allows us to communicate geometric shapes or characters to users. The main benefit of our approach is that it simultaneously produces two types of stimuli, i.e., (1) it moves a tactile stimulus across skin locations and (2) it stretches the user's skin. Skin drag thereby combines the essential stimuli produced by vibrotactile and skin stretch. In our study, skin drag allowed participants to recognize tactile shapes significantly better than a vibrotactile array of comparable size. We present two arm-worn prototype devices that implement our concept.

We propose a new way of eyes-free interaction for wearables. It is based on the user's proprioceptive sense, i.e., rather than seeing, hearing, or feeling an outside stimulus, users feel the pose of their own body. We have implemented a wearable device called Pose-IO that offers input and output based on proprioception. Users communicate with Pose-IO through the pose of their wrists. Users enter information by performing an input gesture by flexing their wrist, which the device senses using a 3-axis accelerometer. Users receive output from Pose-IO by find-ing their wrist posed in an output gesture, which Pose-IO actuates using electrical muscle stimulation. This mechanism allows users to interact with Pose-IO without visual or auditory senses, but through the proprioceptive sense alone. We developed three simple applications that demonstrate symmetric proprioceptive interaction, where input and output occur through the same limb, as well as asymmetric interaction, where input and output occur through different limbs. In a first user study, participants using a symmetric proprioceptive interface re-entered poses received from Pose-IO with an average accuracy of 5.8° despite the minimal bandwidth offered by the device. In a second, exploratory study, we investigated participants' emotional response to asymmetric proprioceptive interaction and the concept of the user's body serving as interface. Participants reported to enjoy the experience (4.6 out of 5).

We present impacto, a device designed to render the haptic sensation of hitting or being hit in virtual reality. The key idea that allows the small and light impacto device to simulate a strong hit is that it decomposes the stimulus: it renders the tactile aspect of being hit by tapping the skin using a solenoid; it adds impact to the hit by thrusting the user's arm backwards using electrical muscle stimulation. The device is self-contained, wireless, and small enough for wearable use, thus leaves the user unencumbered and able to walk around freely in a virtual environment. The device is of generic shape, allowing it to also be worn on legs, so as to enhance the experience of kicking, or merged into props, such as a baseball bat. We demonstrate how to assemble multiple impacto units into a simple haptic suit. Participants of our study rated impact simulated using impacto's combination of solenoid hit and electrical muscle stimulation as more realistic than either technique in isolation.

Overview+Detail interfaces can be used to examine the details of complex data while retaining the data's overall context. Dynamic data introduce challenges for these interfaces, however, as moving objects may exit the detail view, as well as a person's field of view if they are working at a large interactive surface. To address this "off-view" problem, we propose a new information visualization technique, called Canyon. This technique attaches a small view of an off-view object, including some surrounding context, to the external boundary of the detail view. The area between the detail view and the region containing the off-view object is virtually "folded" to conserve space. A comparison study was conducted contrasting the benefits and limitations of Canyon to an established technique, called Wedge. Canyon was more accurate across a number of tasks, especially more complex tasks, and was comparably efficient.