We introduce VR-Doh, an open-source, hands-on 3D modeling system that enables intuitive creation and manipulation of elastoplastic objects in Virtual Reality (VR). By customizing the Material Point Method (MPM) for real-time simulation of hand-induced large deformations and enhancing 3D Gaussian Splatting for seamless rendering, VR-Doh provides an interactive and immersive 3D modeling experience. Users can naturally sculpt, deform, and edit objects through both contact- and gesture-based hand-object interactions. To achieve real-time performance, our system incorporates localized simulation techniques, particle-level collision handling, and the decoupling of physical and appearance representations, ensuring smooth and responsive interactions. VR-Doh supports both object creation and editing, enabling diverse modeling tasks such as designing food items, characters, and interlocking structures, all resulting in simulation-ready assets. User studies with both novice and experienced participants highlight the system's intuitive design, immersive feedback, and creative potential. Compared to existing geometric modeling tools, VR-Doh offers enhanced accessibility and natural interaction, making it a powerful tool for creative exploration in VR.

We propose augmenting initially passive structures built from simple repeated cells, with novel active units to enable dynamic, shape-changing, and robotic applications. Inspired by metamaterials that can employ mechanisms, we build a framework that allows users to configure cells of this passive structure to allow it to perform complex tasks. A key benefit is that our structures can be repeatedly (re)configured by users inserting our configuration units to turn the passive material into, e.g., locomotion robots, integrated motion platforms, or interactive interfaces, as we demonstrate in this paper.

Complex actuators in a small form factor are essential for dynamic interfaces. In this paper, we propose ConeAct, a cone-shaped actuator that can extend, contract, and bend in multiple directions to support rich expression in dynamic materials. A key benefit of our actuator is that it is self-contained and portable as the whole system. We designed our actuator’s structure to be multistable to hold its shape passively, while we control its transition between states using active materials, i.e., shape memory alloys. We present the design space by showcasing our actuator module as part of self-rolling robots, reconfigurable deployable structures, volumetric shape-changing objects and tactile displays. To assist users in designing such structures, we present an interactive editor including simulation to design such interactive capabilities.

Virtual reality (VR) is a revolutionary method of presenting data visualizations, which brings potential possibilities for enhancing analytical activities. However, applying this method to visualize complex data flows remains largely underexplored, especially the Sankey diagrams, which have an advantageous capacity to represent trends in data flows. In this work, we explored a novel design for the immersive Sankey diagram system within VR environments, utilizing a three-dimensional visual design and several interaction techniques that leveraged VR's spatial and immersive capabilities. Through two comparative user studies, we found the effectiveness of the VR Sankey diagram system in improving task performance and engagement and reducing cognitive workload in complex data analysis. We contribute an interactive, immersive Sankey diagram system in VR environments, empirical evidence of its advantages, and design lessons for future immersive visualization tools.

To increase the ubiquity and merging of Olfactory Interfaces (OIs) into daily scenarios seamlessly and aesthetically, we propose OlfacKit, a fully integrated pipeline, which consists of (1) an interactive design tool that enables novice users to quickly model the physical forms of OIs and automatically generate fabrication-ready designs; (2) a construction workflow with a series of assembly kits and a video tutorial; (3) a WebUI for rapidly testing and prototyping smell control through Wi-Fi communication. We tested the atomization performance with different voltages and aroma liquids and further guided the implementation of the WebUI. Six applications have been explored, showing the flexibility and scalability of our toolkit. We conducted a workshop to evaluate the feasibility and usability of OlfacKit, involving 15 participants who proposed 108 creative ideas and implemented 13 working prototypes. We reported the detailed findings and discussed the insights and implications for future design.

Building shape-changing, robotic or deployable interfaces is notoriously difficult, often requiring fabrication skills or specialized hardware. We present a construction kit for novice users to enable immediate hands-on exploration of custom shape-changing or robotic structures through pneumatically actuated origami tubes. Our construction kit consists of generic origami actuators that can be combined with our connectors to result in a variety of shapes and motions. Our novel connectors support simple mechanical connections through snap fits and pneumatic configurations through plugs. To assist users in designing and previewing complex deformations, we provide a design tool that generates control code for user-defined designs. We envision that our construction kit will facilitate creativity support tasks (e.g. product design) or education. We demonstrate the capabilities of our construction kit with three application examples and a series of objects created during our co-creation study with novice users.

Creating personalized 3D printing objects has become increasingly popular with the advance in end-user modeling tools and fabrication techniques. However, it remains challenging for novice makers to design and fabricate functional objects with mechanical motion such as linear or rotational. To empower users to add mechanical movements to their models for particular needs, we first investigate the commonly used 3D printable mechanisms and parameterize four mechanical primitives. We then provide a computational approach to reconfigure the mechanical primitives to user-provided 3D models, with an underlying structural optimization procedure according to a specified bending force. We further showcase a set of design examples and conduct a user study to demonstrate the potential of our approach in creating personalized functional artifacts.

Voice user interfaces (VUI) are becoming indispensable in car for offering drivers the opportunity to make distraction-free inputs and conduct complex tasks. However, the usability and control efficiency of today’s VUI remain to be enhanced due to its sequential nature. In this work, we explored gestural input on the steering wheel to improve the interaction efficiency of VUI. Based on limitations in VUI, we designed novel gestural commands on the steering wheel to augment them. We also elicited corresponding user-defined gestures by exploring drivers’ touch behavior. Then, we implemented a prototype attached to the steering wheel for recognizing gestures. Finally, we evaluated our system’s usability regarding driving performance, interaction efficiency, cognitive workload and user feedback. Results revealed that our system improved the control efficiency of VUI and reduced workload without a significant reduction in driving distraction than just using VUI.