Pull-up objects form 3D shapes by pulling a string routed through a 2D material, offering low-cost 2D fabrication and reversible transformation. However, existing approaches rely on origamic folding, which creates faceted, oftentimes rigid surfaces and requires manual pull-up string routing. We introduce EmbroForm, a digital fabrication pipeline for fully soft pull-up objects with organic, higher-fidelity shapes. Instead of folding, EmbroForm forms 3D shapes by seaming the boundaries of a flexible 2D patch unwrapped from the target. To enable this, we contribute a fabrication technique that automates the routing of sliding strings on flexible sheet materials with machine embroidery, which we extend on to design zig-zag lacings for seaming the boundaries. Then we introduce an end-to-end pipeline that, given a 3D mesh, creates an optimized 2D unwrapped patch and generates pull-up string routing paths for fabrication. We provide a design tool for customization and validate our approach with technical experiments and implemented application cases.

Virtual Reality (VR) depends on haptic feedback to create immersive experiences. Traditional passive proxies align physical props with their virtual counterparts but remain limited in scalability and expressiveness, or require bulky actuators to support reconfiguration. We introduce User-reconfigured Haptics, an approach that utilizes implicit user actions to reconfigure haptic interfaces to extend the gamut of VR haptic experiences. Modular 3D-printed cells are assembled into dynamic interfaces that express diverse haptic properties such as softness and weight. By masking physical reconfigurations with visual (re)mapping, user actions unnoticeably change haptic properties, resulting in user-driven, dynamic haptic experiences. User studies show that our design can provide distinguishable haptic experiences and is perceived as realistic and enjoyable in a VR task. We further showcase four applications: a fishing rod that changes weight and flexibility, a dynamic desktop of pressable buttons, a glove with adjustable squeezing, and a crossbow with variable pulling resistance.

Machine embroidery is a versatile technique for creating custom and entirely fabric-based patterns on thin and conformable textile surfaces. However, existing machine-embroidered surfaces remain static, limiting the interactions they can support. We introduce Embrogami, an approach for fabricating textile structures with versatile shape-changing behaviors. Inspired by origami, we leverage machine embroidery to form finger-tip-scale mountain-and-valley structures on textiles with customized shapes, bistable or elastic behaviors, and modular composition. The structures can be actuated by the user or the system to modify the local textile surface topology, creating interactive elements like toggles and sliders or textile shape displays with an ultra-thin, flexible, and integrated form factor. We provide a dedicated software tool and report results of technical experiments to allow users to flexibly design, fabricate, and deploy customized Embrogami structures. With four application cases, we showcase Embrogami’s potential to create functional and flexible shape-changing textiles with diverse visuo-tactile feedback.

We propose HandAvatar to enable users to embody non-humanoid avatars using their hands. HandAvatar leverages the high dexterity and coordination of users’ hands to control virtual avatars, enabled through our novel approach for automatically-generated joint-to-joint mappings. We contribute an observation study to understand users’ preferences on hand-to-avatar mappings on eight avatars. Leveraging insights from the study, we present an automated approach that generates mappings between users’ hands and arbitrary virtual avatars by jointly optimizing control precision, structural similarity, and comfort. We evaluated HandAvatar on static posing, dynamic animation, and creative exploration tasks. Results indicate that HandAvatar enables more precise control, requires less physical effort, and brings comparable embodiment compared to a state-of-the-art body-to-avatar control method. We demonstrate HandAvatar’s potential with applications including non-humanoid avatar based social interaction in VR, 3D animation composition, and VR scene design with physical proxies. We believe that HandAvatar unlocks new interaction opportunities, especially for usage in Virtual Reality, by letting users become the avatar in applications including virtual social interaction, animation, gaming, or education.

We present digital mechanical metamaterials that enable multiple computation loops and reprogrammable logic functions, making a significant step towards passive yet interactive devices. Our materials consist of many cells that transmit signals using an embedded bistable spring. When triggered, the bistable spring displaces and triggers the next cell. We integrate a recharging mechanism to recharge the bistable springs, enabling multiple computation rounds. Between the iterations, we enable reprogramming the logic functions after fabrication. We demonstrate that such materials can trigger a simple controlled actuation anywhere in the material to change the local shape, texture, stiffness, and display. This enables large-scale interactive and functional materials with no or a small number of external actuators. We showcase the capabilities of our system with various examples: a haptic floor with tunable stiffness for different VR scenarios, a display with easy-to-reconfigure messages after fabrication, or a tactile notification integrated into users’ desktops.

Despite significant improvements to Virtual Reality (VR) technologies, most VR displays are fixed focus and depth perception is still a key issue that limits the user experience and the interaction performance. To supplement humans’ inherent depth cues (e.g., retinal blur, motion parallax), we investigate users’ perceptual mappings of distance to virtual objects’ appearance to generate visual cues aimed to enhance depth perception. As a first step, we explore color-to-depth mappings for virtual objects so that their appearance differs in saturation and value to reflect their distance. Through a series of controlled experiments, we elicit and analyze users’ strategies of mapping a virtual object’s hue, saturation, value and a combination of saturation and value to its depth. Based on the collected data, we implement a computational model that generates color-to-depth mappings fulfilling adjustable requirements on confusion probability, number of depth levels, and consistent saturation/value changing tendency. We demonstrate the effectiveness of color-to-depth mappings in a 3D sketching task, showing that compared to single-colored targets and strokes, with our mappings, the users were more confident in the accuracy without extra cognitive load and reduced the perceived depth error by 60.8%. We also implement four VR applications and demonstrate how our color cues can benefit the user experience and interaction performance in VR.

Emotion recognition is a rising trend in related fields. Existing research mainly aims at the emotion recognition based on single or multi modalities on performance datasets. However, there is little research focusing on gesture-based emotion recognition using non-performance emotional datasets. In this article, we proposed a non-performance emotional data collection scheme using VR to construct a high-quality nonperformance dataset in gesture modality with fear labels. Then we proposed an optimized algorithm for gesture-based fear recognition using BLSTM on this dataset with the accuracy of up to 71.43%. Our work could be applied in many domains such as potentially domestic violence detection and VR game evaluation.