Motion effects are vital for enhancing 4D experiences in various applications, e.g., rides, films, and games, by physically moving the user. Recent research introduced algorithms for generating effective motion effects from the audiovisual stream, but these approaches either require object motion trajectories or miss important scene components, such as other objects, particles, and camera motion. This paper proposes a fully automatic algorithm synthesizing motion effects from all scene components. Our method leverages computer vision technologies to capture salient movements among multiple components, enabling the creation of holistic motion effects without the need for human intervention. Then, our method computes a motion proxy representing the movements of the scene components by compressing the captured movements into a singlepoint motion. The motion proxy is converted into a motion command through a motion cueing algorithm and delivered to the user. The results of a user study show that our algorithm can generate compelling motion effects that enhance the 4D experience better than semi-manually authored effects. Our approach can facilitate the production of captivating motion effects for many applications.

Motion effects are indispensable for improving 4D experiences in highly interactive applications, such as amusement parks, 4D theaters, and virtual reality games. Their recent emergence calls for effective algorithms generating motion effects synchronized with audiovisual content. This paper presents an automatic algorithm for synthesizing the object-based motion effects that express the movements of multiple articulated bodies inclusively when the objects’ motion trajectories are available in the 3D camera space. By taking the visual velocities and sizes of all object parts, our method computes a motion proxy that represents the objects’ movements by one point and converts the motion proxy to a motion command through a motion cueing algorithm. The motion proxy is determined by linearly combining the velocities, and its best combination was selected from several candidates by user studies. The results of user studies indicate that our algorithm can produce compelling object-based motion effects that enhance the multisensory experience.

The liquidity of the exchange is the most important factor when crytocurrency traders choose an exchange. However, the amount of liquidity provided by the liquidity providers in decentralized exchanges is insufficient when compared to centralized exchanges. This is because the liquidity providers in decentralized exchanges suffer from the risk of divergence loss inherent to the automated market making system. To this end, we introduce a new concept called margin liquidity and leverage this concept to propose a highly profitable margin liquidity-providing position. Then, we extend this margin liquidity-providing position to a virtual margin liquidity-providing position to alleviate the risk of divergence loss for the liquidity providers and encourage them to provide more liquidity to the pool. We show that our proposed margin liquidity is 8K times more capital efficient than the concentrated liquidity proposed in Uniswap V3.

(Haptic) motion effects refer to the vestibular stimuli generated by a motion platform and delivered to the whole body of a user sitting on the platform. Motion effects are an essential tool for creating vivid sensory experiences in various extended reality (XR) applications, ranging from training simulators to recent 4D rides, films, and games for entertainment. For the latter purpose, motion effects emphasize audiovisual events occurring in the scene, such as camera motion, the movement of an object of interest, and special sounds. Recent research developed several algorithms to produce motion effects from the audiovisual stream automatically. However, these algorithms are designed for a single class of motion effects, and extension to multiple motion effect classes remains unexplored. In this paper, we propose an algorithmic framework that merges camera and object motion effects into one motion effect while preserving the perceptual consequences of the two effects. We validate the framework’s perceptual performance through a user study. To our knowledge, this work is one of the first successful reports of merging different kinds of motion effects for improved XR experiences.

In recent times, millions of people enjoy watching video gameplays at an eSports stadium or home. We seek a method that improves gameplay spectator or viewer experiences by presenting multisensory stimuli. Using a motion chair, we provide the motion effects automatically generated from the audiovisual stream to the viewers watching a first-person shooter (FPS) gameplay. The motion effects express the game character’s movement and gunfire action. We describe algorithms for the computation of such motion effects developed using computer vision techniques and deep learning. By a user study, we demonstrate that our method of providing motion effects significantly improves the viewing experiences of FPS gameplay. The contributions of this paper are with the motion synthesis algorithms integrated for FPS games and the empirical evidence for the benefits of experiencing multisensory gameplays.

Four-dimensional (4D) films, which provide special physical effects to the audience with audiovisual stimuli, are gaining more popularity and acceptance. One of the most frequent 4D effects is the object-based motion effect, which refers to the vestibular stimulus generated by a motion chair to emphasize a moving object of interest, e.g., the flying iron man, displayed on the screen. In this paper, we present an algorithm for synthesizing convincing object-based motion effects automatically from a given object motion trajectory. While previous approaches use the 2D object position on the screen as input, our method takes the 3D position and orientation of the object in the camera space and computes its motion proxy that reflects both the object translation and rotation, as well as its size to the viewers' eyes. The proxy is determined based on the results of a perceptual experiment that presents an optimal additive rule of the translation and rotation information scaled by the object's visual size. The motion proxy is fed to a motion cueing algorithm (MCA) that computes the command using a washout filter or model predictive control. The most appropriate MCA for our purpose is selected from six candidates by a user study. We also consider the effects of visual perception by incorporating two types of motion field equations into the computation of the visually perceived velocity. The results of a user study indicate that our algorithm can generate compelling object-based motion effects that better enhance the 4D film viewing experience than the previous methods.

A motion platform provides the vestibular stimuli that elicit the sensations of self-motion and thereby improves the immersiveness. A representative example is a 4D Ride, which presents a video of POV shots and motion effects synchronized with the camera motion in the video. Previous research efforts resulted in a few automatic motion effect synthesis algorithms for POV shots. Although effective in generating gross motion effects, they do not consider fine features on the ground, such as a rough or bumpy road. In this paper, we propose an algorithm for styling the gross motion effects using a texture image. Our algorithm transforms a texture image into a high-frequency style motion and merges it with the original motion while respecting both perceptual and device constraints. A user study demonstrated that texture styling could increase immersiveness, realism, and harmony.

This paper presents a method that automatically transforms a virtual environment (VE) according to the mood of input music. We use machine learning to extract a mood from the music. We then select images exhibiting the mood and transfer their styles to the textures of objects in the VE photorealistically or artistically. Our user study results indicate that our method is effective in transferring valence-related aspects, but not arousal-related ones. Our method can still provide novel experiences in virtual reality and speed up the production of VEs by automating its procedure.