This paper presents HaptiCraft, a modular handheld haptic controller designed to replicate the appearance, mass distribution, and multimodal haptic properties of virtual objects. HaptiCraft consists of many modules that provide distinct functions for assembly and multimodal haptic feedback, supporting five types: vibration, impact, thermal, variable inertia, and variable stiffness. Users can readily assemble these modules to create a controller tailored to their needs. To simplify the design process, especially for novice users, we also provide a graphical authoring tool and assess its usability. Finally, we evaluate the system's effectiveness through various virtual reality (VR) scenarios, demonstrating that HaptiCraft significantly enhances user experience. HaptiCraft makes a substantial contribution to the field of VR handheld controllers by offering comprehensive support for shape-changing, variable inertia, and rich multimodal haptic feedback, coupled with an effective authoring method.

Creating interactive VR content typically requires extensive modeling effort, making scalable production difficult. We present an automated computational pipeline that converts a single image into an interactive 3D scene with plausible kinesthetic feedback. Our approach leverages a Large Language Model (LLM) to detect objects in the image and infer haptic properties from visual and textual cues. These extracted results are then utilized to synthesize 3D models and optimize haptic properties for perceptual distinguishability, while combining them into a complete interactive environment. A user study shows that the generated VR scenes provide compelling visuo-haptic experiences, highlighting the potential of our method for scalable multisensory world generation. To our knowledge, this is the first system to automatically produce VR scenes with force feedback from a single image, pointing toward a practical direction for lowering the barriers to creating haptically enriched VR content.

We present a visuo-haptic crossmodal model of shape perception designed for shape-changing handheld controllers. The model uses the inertia tensor of an object to bridge the two senses. The model was constructed from the results of three perceptual experiments. In the first two experiments, we validate that the primary moment and product of inertia (MOI and POI) in the inertia tensor have critical effects on the haptic perception of object length and asymmetry. Then, we estimate a haptic-to-visual shape matching model using MOI and POI as two link variables from the results of the third experiment for crossmodal magnitude production. Finally, we validate in a summative user study that the inverse of the shape matching model is effective for pairing a perceptually-congruent haptic object from a virtual object—the functionality we need for shape-changing handheld interfaces to afford perceptually-fulfilling sensory experiences in virtual reality.

One can embed a vibration actuator to a physical button and augment the physical button’s original kinesthetic response with a programmable vibration generated by the actuator. Such vibration-augmented buttons inherit the advantages of both physical and virtual buttons. This paper reports the information transmission capacity of vibration-augmented buttons. It was obtained by conducting a series of absolute identification experiments while increasing the number of augmented buttons. The information transmission capacity found was 2.6 bits, and vibration-augmented and physical buttons showed similar abilities in rendering easily recognizable haptic responses. In addition, we showcase a VR text entry application that utilizes vibration-augmented buttons. Our method provides several error messages to the user during text entry using a VR controller that includes an augmented button. We validate that the variable haptic feedback improves task performance, cognitive workload, and user experience for a transcription task.