Creating interactive VR content typically requires extensive modeling effort, making scalable production difficult. We present an automated computational pipeline that converts a single image into an interactive 3D scene with plausible kinesthetic feedback. Our approach leverages a Large Language Model (LLM) to detect objects in the image and infer haptic properties from visual and textual cues. These extracted results are then utilized to synthesize 3D models and optimize haptic properties for perceptual distinguishability, while combining them into a complete interactive environment. A user study shows that the generated VR scenes provide compelling visuo-haptic experiences, highlighting the potential of our method for scalable multisensory world generation. To our knowledge, this is the first system to automatically produce VR scenes with force feedback from a single image, pointing toward a practical direction for lowering the barriers to creating haptically enriched VR content.

Motion effects are indispensable for improving 4D experiences in highly interactive applications, such as amusement parks, 4D theaters, and virtual reality games. Their recent emergence calls for effective algorithms generating motion effects synchronized with audiovisual content. This paper presents an automatic algorithm for synthesizing the object-based motion effects that express the movements of multiple articulated bodies inclusively when the objects’ motion trajectories are available in the 3D camera space. By taking the visual velocities and sizes of all object parts, our method computes a motion proxy that represents the objects’ movements by one point and converts the motion proxy to a motion command through a motion cueing algorithm. The motion proxy is determined by linearly combining the velocities, and its best combination was selected from several candidates by user studies. The results of user studies indicate that our algorithm can produce compelling object-based motion effects that enhance the multisensory experience.

(Haptic) motion effects refer to the vestibular stimuli generated by a motion platform and delivered to the whole body of a user sitting on the platform. Motion effects are an essential tool for creating vivid sensory experiences in various extended reality (XR) applications, ranging from training simulators to recent 4D rides, films, and games for entertainment. For the latter purpose, motion effects emphasize audiovisual events occurring in the scene, such as camera motion, the movement of an object of interest, and special sounds. Recent research developed several algorithms to produce motion effects from the audiovisual stream automatically. However, these algorithms are designed for a single class of motion effects, and extension to multiple motion effect classes remains unexplored. In this paper, we propose an algorithmic framework that merges camera and object motion effects into one motion effect while preserving the perceptual consequences of the two effects. We validate the framework’s perceptual performance through a user study. To our knowledge, this work is one of the first successful reports of merging different kinds of motion effects for improved XR experiences.

In this paper, we report both absolute and differential thresholds for motion in the six cardinal directions as comprehensively as possible. As with general 4D motion effects, we used sinusoidal motions with low intensity and large frequency as stimuli. Hence, we could also compare the effectiveness of motion types in delivering motion effects. We found that the thresholds for the z-axis (up-down) were higher than those for the x-axis (front-back) and y-axis (left-right) in both kinds of thresholds and that the type of motion significantly affected both thresholds. Further, between differential thresholds and reference intensities, we found a strong linear relationship for roll, yaw and, surge. Compared to them, a relatively weak linear relationship was observed for the rest of the motion types. Our results can be useful for generating motion effects for 4D contents while considering the human sensitivity to motion feedback.

Versatile multi-modal robots are advantageous for their wider operational environments. By taking design principles from observation of Pteromyini, commonly known as the flying squirrel, which shows balanced performances in both aerial and terrestrial locomotion, a novel robotic platform with the ability of gliding and walking is designed. The flexible membrane and gliding method of Pteromyini have been applied to the robot design. The legs of the robot were optimized to perform with regulated motor torques in both walking and gliding. The robot glided with an average gliding ratio of 1.88 and controlled its angle-of-attack for slowing down to land safely. The robot was able to walk utilizing different gait patterns. These results demonstrated our robot's balanced multi-modal locomotion of gliding and walking.

A self-contained wall-climbing robot called MRWALLSPECT-II, is presented. The robot has characteristic features in the kinematic design of the leg with closed link mechanism. It reduces the number of actuators compared to the previous ones and makes it possible to have high force-to-weight ratio. Those are enabled by adopting a simple and robust gait pattern mimicking a living creature. Also, as the robot is designed to be a self-contained one, it includes all the components such as air compressors, vacuum generators, an embedded controller, batteries, a CCD camera, sensors, motors and electronics in itself without any tether cable. Its communication with the operating site in the ground is conducted through wireless LAN. In the paper, its principles of design and configuration of the robot are introduced.