The pursuit of seamlessness in collaborative VR often creates a paradox: concealing technical failures generates asymmetric awareness, fracturing the shared reality essential for teamwork. We argue instead that disruption timing acts as an information structure. Drawing on the theory of rational rituals, we posit that a simultaneous onset creates a Public, Synchronous, Bounded (PSB) anchor that establishes common knowledge. We tested this framework with 34 triads (N = 102) performing interdependent tasks. Results show that simultaneous disruptions significantly accelerated Time-to-Recovery (TTR) and preserved role stability by enabling a compact A–R–E sequence (affect-check, reorientation, re-entry). Conversely, asynchronous onsets caused epistemic fragmentation and role churn. We contribute the coordination wrapper, a design strategy that transforms inevitable system failures into synthetic PSB cues, shifting the paradigm from error minimization to resilient recovery.

Abstract The field of 3D reconstruction from multi‐view images has advanced rapidly thanks to 3D Gaussian Splatting (3DGS), which enables efficient and photorealistic scene representation. However, optimizing 3DGS requires high‐quality images from various viewpoints with accurate camera poses. The repeated collection of such data demands significant human effort, which poses a major constraint in practical applications. To address this issue, automated capturing systems that uses a turntable and fixed camera are widely employed. In a turntable setup, the background remains stationary while the object rotates. Therefore, preprocessing to remove the backgrond is essential, but the preprocessing reduces the number of reliable feature matches, which destabilizes Structure‐from‐Motion (SfM). This results in inaccurate camera poses, which degrades the quality of 3DGS reconstruction. We propose a novel method to optimize 3DGS in a turntable setup without SfM by leveraging the prior knowledge that objects rotate around a central axis. Unlike previous SfM‐free methods that estimate camera poses for each frame, our approach reduces the complexity of optimization by representing rotations with a single global rotation axis. The estimated rotation is directly applied to the 3D Gaussians, producing motion defined as rotation flow. This rotation flow is then aligned with optical flow to provide strong geometric supervision. Through uncertainty‐to‐detail flow scheduling, our approach remains stable during the initial training stage when the geometry of the Gaussian set is still inaccurate. On the NeRF‐Synthetic dataset and on real‐world datasets captured with a turntable, our method outperforms existing SfM‐free approaches in both reconstruction quality and training speed, and even demonstrates performance comparable to 3DGS optimized with precise camera poses.

Recently, 3D Gaussian Splatting (3DGS) has emerged as a powerful technique for reconstructing high-quality, photorealistic 3D representations of real-world scenes. However, editing 3DGS remains more challenging than mesh-based approaches as it lacks explicit geometry and requires view-consistent updates across Gaussians. Previous approaches have relied primarily on text-driven generative models for 3D Gaussian editing, limiting direct control over specific visual appearances. In this study, we propose a texture editing framework for 3DGS that leverages only a single user-modified image. Our approach maintains coherence across multiple views while accounting for the corresponding lighting adjustments in the edited region. We introduce three key techniques to achieve this: (1) aligned edit propagation, which transfers local edits from the reference view to other viewpoints; (2) mask-based filtering, which restricts modifications to relevant Gaussians and prevents unintended changes; and (3) opacity-based selection, which identifies Gaussians with the most significant visual impact on the edited texture. Through both qualitative and quantitative evaluations on synthetic and real-world datasets, we demonstrate that our method achieves more precise and spatially controllable 3DGS editing than existing techniques. We expect these techniques to pave the way for more intuitive 3D Gaussian Splatting editing pipelines and inspire future research.

Virtual reality (VR) is compelling precisely because it transcends physical realism, allowing reach to be stretched, motion rescaled, and objects relocated. While these distortions expand interaction possibilities, they lack principled limits: How far can they go before agency, ownership, or presence collapse? We present a dual-axis framework that decomposes distortions into metric deviations (continuous remappings akin to stretching) and topological breaks (discontinuous jumps akin to cutting), both applied to the same hand-target relation. To operationalize this framework, we introduce the Single-Interaction Dual-Axis (SIDA) protocol, combining a standardized task with psychophysical estimation of Just-Acceptable Distortion (JAD) and Breaks in Presence (BiP) under stable or jittered mappings. In a study with $N=52$, results show that metric deviations are significantly more tolerable than topological breaks, predictability expands tolerance by approximately 35%, and combined distortions interact contractively, constraining the total budget. Our contributions include (i) a unified model for commensurate comparison, (ii) a reproducible protocol for threshold estimation, and (iii) the comfort envelope, a multi-dimensional budget of acceptable distortion that guides trade-offs between agency and presence in VR design.

Redirected walking (RDW) allows people to explore large virtual environments while walking within a smaller physical space. When physical space is limited, explicit resets such as turn-in-place are unavoidable. Previous studies have reduced the frequency of resets by adjusting user paths. However, resets remain necessary under severe spatial constraints. The frequent use of turn-in-place resets interrupts locomotion and can degrade task performance and user experience. We present Non-Euclidean Portal Reset (NEPR), a reset technique that enables continuous experiences without pausing the user's walking. When a collision risk is detected, NEPR opens a virtual portal leading to a short, non-Euclidean corridor. Traversing the corridor repositions and reorients the user. The exit returns the user near the point of interest target in the primary world, maintaining flow. To evaluate the effectiveness of NEPR, we conducted user experiments comparing (1) the conventional turn-in-place reset, (2) NEPR, and (3) a hybrid method combining both approaches. Our results demonstrate that NEPR and the combined technique significantly improve the user experience and task performance compared to the traditional method. Overall, NEPR reframes resets as seamless transitions rather than interruptions, enhancing the practicality of RDW.

Presence, the felt experience of “being there” in virtual environments, is central to immersive VR, yet its dynamic structure is underexplored. Most prior work treats presence as static or analyzes isolated modalities, limiting both theory and application. We introduce PresenceLens, a computational framework that models presence as a temporally evolving, multimodal phenomenon. Using synchronized visual, auditory, gaze, and interaction data from 120 participants across 20 VR applications, PresenceLens identifies eight recurring patterns linked to high presence. These patterns evolve over time in distinct trajectories and form the basis of Pattern Orchestration Theory, which conceptualizes presence as the temporal coordination of perceptual, cognitive, and interactive processes. Our model achieves high predictive accuracy ($R^{2}=0.64$) and provides interpretable mappings between real-time behavior and subjective presence. This work links theory and temporal modeling, enabling VR systems to dynamically adapt to evolving user states.

Abstract We introduce ClothingTwin, a novel end‐to‐end framework for reconstructing 3D digital twins of clothing that capture both the outer and inner fabric —without the need for manual mannequin removal. Traditional 2D “ghost mannequin” photography techniques remove the mannequin and composite partial inner textures to create images in which the garment appears as if it were worn by a transparent model. However, extending such method to photorealistic 3D Gaussian Splatting (3DGS) is far more challenging. Achieving consistent inner‐layer compositing across the large sets of images used for 3DGS optimization quickly becomes impractical if done manually. To address these issues, ClothingTwin introduces three key innovations. First, a specialized image acquisition protocol captures two sets of images for each garment: one worn normally on the mannequin (outer layer exposed) and one worn inside‐out (inner layer exposed). This eliminates the need to painstakingly edit out mannequins in thousands of images and provides full coverage of all fabric surfaces. Second, we employ a mesh‐guided 3DGS reconstruction for each layer and leverage Non‐Rigid Iterative Closest Point (ICP) to align outer and inner point‐clouds despite distinct geometries. Third, our enhanced rendering pipeline—featuring mesh‐guided back‐face culling, back‐to‐front alpha blending, and recalculated spherical harmonic angles—ensures photorealistic visualization of the combined outer and inner layers without inter‐layer artifacts. Experimental evaluations on various garments show that ClothingTwin outperforms conventional 3DGS‐based methods, and our ablation study validates the effectiveness of each proposed component.

In order to serve better VR experiences to users, existing predictive methods of Redirected Walking (RDW) exploit future information to reduce the number of reset occurrences. However, such methods often impose a precondition during deployment, either in the virtual environment's layout or the user's walking direction, which constrains its universal applications. To tackle this challenge, we propose a mechanism F-RDW that is twofold: (1) forecasts the future information of a user in the virtual space without any assumptions by using the conventional method, and (2) fuse this information while maneuvering existing RDW methods. The backbone of the first step is an LSTM-based model that ingests the user's spatial and eye-tracking data to predict the user's future position in the virtual space, and the following step feeds those predicted values into existing RDW methods (such as MPCRed, S2C, TAPF, and ARC) while respecting their internal mechanism in applicable ways. The results of our simulation test and user study demonstrate the significance of future information when using RDW in small physical spaces or complex environments. We prove that the proposed mechanism significantly reduces the number of resets and increases the traveled distance between resets, hence augmenting the redirection performance of all RDW methods explored in this work.

Eye gaze is regarded as a promising interaction modality in extended reality (XR) environments. However, to address the challenges posed by the Midas touch problem, the determination of selection intention frequently relies on the implementation of additional manual selection techniques, such as explicit gestures (e.g., controller/hand inputs or dwell), which are inherently limited in their functionality. We hereby present a machine learning (ML) model based on the Bayesian framework, which is employed to predict user selection intention in real-time, with the unique distinction that all data used for training and prediction are obtained from gaze data alone. The model utilizes a Bayesian approach to transform gaze data into selection probabilities, which are subsequently fed into an ML model to discern selection intentions. In Study 1, a high-performance model was constructed, enabling real-time inference using solely gaze data. This approach was found to enhance performance, thereby validating the efficacy of the proposed methodology. In Study 2, a user study was conducted to validate a manual-free technique based on the prediction model. The advantages of eliminating explicit gestures and potential applications were also discussed.

In virtual reality (VR), collaborators often experience the same environment at different visual scales, disrupting shared attention and increasing coordination difficulty. While prior work has focused on preventing misalignment, less is known about how teams recover when alignment fails. We examine collaboration under scale asymmetry, a particularly disruptive form of perceptual divergence. In a study with 36 VR teams, we identify behavioral patterns that distinguish adaptive recovery from persistent breakdown. Successful teams flexibly shifted between user-driven and system-supported cues, while others repeated ineffective strategies. Based on these findings, we introduce the Perceptual Asymmetry Coordination Theory (PACT), a dual-pathway model that describes coordination as an evolving process shaped by cue integration and strategic responsiveness. PACT reframes recovery not as a return to alignment, but as a dynamic adaptation to misalignment. These insights inform the design of VR systems that support recovery through multi-channel, adaptive coordination in scale-divergent environments.

The reset technique of Redirected Walking (RDW) forcibly reorients the user's direction overtly to avoid collisions with boundaries, obstacles, or other users in the physical space. However, excessive resetting can decrease the user's sense of immersion and presence. Several RDW studies have been conducted to address this issue. Among them, much research has been done on reset techniques that reduce the number of resets by devising reset direction rules or optimizing them for a given environment. However, existing optimization studies on reset techniques have mainly focused on a single-user environment. In a multi-user environment, the dynamic movement of other users and static obstacles in the physical space increase the possibility of resetting. In this study, we propose Multi-Agent Reinforcement Resetter (MARR), which resets the user taking into account both physical obstacles and multi-user movement to minimize the number of resets. MARR is trained using multi-agent reinforcement learning to determine the optimal reset direction in different environments. This approach allows MARR to effectively account for different environmental contexts, including arbitrary physical obstacles and the dynamic movements of other users in the same physical space. We compared MARR to other reset technologies through simulation tests and user studies, and found that MARR outperformed the existing methods. MARR improved performance by learning the optimal reset direction for each subtle technique used in training. MARR has the potential to be applied to new subtle techniques proposed in the future. Overall, our study confirmed that MARR is an effective reset technique in multi-user environments.

Resetting in redirected walking (RDW) allows users to maintain a continuous, collision-free walking experience in virtual reality (VR), even in a limited physical space. Since frequent resets reduce the user's sense of immersion, extensive research has been conducted to develop resetters that provide optimal reset directions. Various visual reset user interfaces (UIs) have been proposed to help users perform the correct reset direction according to the improved resetter, but their effectiveness has not been sufficiently verified. In addition, expert interviews conducted to identify the problems in the current reset process revealed that users sometimes fail to recognize the visual reset UI in time. Therefore, we propose a novel visual reset UI using Gauge, which is expected to provide users with an effective and high-quality experience. In Study 1, we demonstrate the effectiveness of the Gauge UI by comparing it to existing UIs (Direction, End Point, and Arrow Alignment). Users of various locomotion techniques, including RDW, inevitably need to perform resets, and in this work we propose a novel paradigm: a combined multimodal reset interface.

Recent advances in 3D object reconstruction have been remarkable, and 3D object reconstruction methods capable of real-time rendering are crucial for the creation of real-world content. Most reconstruction research has focused on improving algorithmic performance (e.g., rendering time, and visual quality). However, we need to know which reconstruction method can improve the user experience when used in real-world content, and whether the experience differs across platforms. In this study, we investigate the effects of three different visualization methods, including two real-time reconstruction methods (3DGS and Image-to-3D) and video playback, on user recognition memory and experience on two different platforms (VR and PC). The results show that different visualization methods improve recognition memory and user experience differently and that there are differences in the effects across platforms. In addition, we investigate designers’ views on 3D object visualization techniques and discuss how they can be used in actual content creation and their scalability. The results of this study suggest that it is possible to improve user experience and recognition memory by recommending different methods depending on the visualization perspective and platform used.

Various locomotion techniques are used to navigate and find way through space in virtual environments (VE), and each technique provides different experiences and performances to users. Previous studies have primarily focused on static environments, whereas there is a need for research from a different perspective of dynamic environments because there are many moving objects in VE, such as other users. In this study, we compare the effects of different locomotion techniques on the user's spatial knowledge and experience, depending on whether the virtual objects are moving or not. The investigated locomotion techniques include joystick, teleportation, and redirected walking (RDW), all commonly used for VR navigation. The results showed that the differences in spatial knowledge and user experience provided by different locomotion techniques can vary depending on whether the environment is static or dynamic. Our results also showed that for a given VE, there are different locomotion techniques that induce fewer collisions between the user and other objects, or reduce the time it takes the user to perform a given task. This study suggests that when designing a locomotion interface for a specific VR application, it is possible to improve the user's spatial knowledge and experience by recommending different locomotion techniques depending on the degree of environment dynamism and and type of task.

In this paper, we present the Selective Redirection Controller (SRC), which selects the optimal redirection controller based on the physical and virtual environment in Redirected Walking (RDW). The primary advantage of SRC over existing controllers is its dynamic switching among four different redirection controllers (S2C, TAPF, ARC, and SRL) based on the user's environment, as opposed to using a single fixed controller throughout the experience. By switching between redirection controllers based on the context around the user, SRC aims to optimize the advantages of each redirection strategy. The SRC model is trained using reinforcement learning to dynamically and instantaneously switch redirection controllers based on the user's environment. We evaluated the performance of SRC against traditional redirection controllers through simulations and user studies conducted in various physical and virtual environments. The findings indicate that SRC reduces the number of resets significantly compared to traditional redirection controllers. Heat map visualization was utilized during the development process to analyze which redirection controller SRC chooses based on the different environments around the user. SRC alternates between redirection techniques based on the user's environment, maximizing the advantages of each strategy for a superior RDW experience.

We present ClothCombo, a pipeline to drape arbitrary combinations of clothes on 3D human models with varying body shapes and poses. While existing learning-based approaches for draping clothes have shown promising results, multi-layered clothing remains challenging as it is non-trivial to model inter-cloth interaction. To this end, our method utilizes a GNN-based network to efficiently model the interaction between clothes in different layers, thus enabling multi-layered clothing. Specifically, we first create feature embedding for each cloth using a topology-agnostic network. Then, the draping network deforms all clothes to fit the target body shape and pose without considering inter-cloth interaction. Lastly, the untangling network predicts the per-vertex displacements in a way that resolves interpenetration between clothes. In experiments, the proposed model demonstrates strong performance in complex multi-layered scenarios. Being agnostic to cloth topology, our method can be readily used for layered virtual try-on of real clothes in diverse poses and combinations of clothes.

The growing interest in the self-similarity effect of avatars in virtual reality (VR) has spurred the creation of realistic avatars that closely mirror their users. However, despite extensive research on the self-similarity effect in single-user VR environments, our understanding of its impact in social VR settings remains underdeveloped. This shortfall exists despite the unique socio-psychological phenomena arising from the illusion of embodiment that could potentially alter these effects. To fill this gap, this paper provides an in-depth empirical investigation of how avatars' self-similarity influences social VR experiences. Our research uncovers several notable findings: 1) A high level of avatar self-similarity boosts users' sense of embodiment and social presence but has minimal effects on the overall presence and even slightly hinders immersion. These results are driven by increased self-awareness. 2) Among various factors that contribute to the self-similarity of avatars, voice stands out as a significant influencer of social VR experiences, surpassing other representational factors. 3) The impact of avatar self-similarity shows negligible differences between male and female users. Based on these findings, we discuss the pros and cons of incorporating self-similarity into social VR avatars. Our study serves as a foundation for further research in this field.

A simultaneous walking experience in virtual and real spaces can provide a high sense of presence. However, users may face challenges when walking within a large virtual space while walking in a small and complex real space. Several methods such as Redirected Walking (RDW) and Substitutional Reality (SR) have been proposed as different approaches to this problem. However, the users must “reset” their movement direction at unpredictable moments to avoid collision in a small and complex real space when using subtle RDW that does not maintain the correspondence between virtual and real space. Contrarily, exploration through the SR has a limitation in that the VR scene is restricted to a controlled area. In this paper, we propose Reset at Fixed Positions (RFP), a method that combines RDW with the advantage of the SR and matches walkable real space with walkable virtual space. To utilize RFP, we defined Guaranteed Space Block (GSB), a unit space that constitutes a walkable virtual space. This space is obtained through the point reflection of the GSB utilizing the reset position within the GSB. RFPs can be implemented by two methods: Generating Virtual Space Using RFP (G-RFP) and Implementing Given Virtual Space Using RFP (I-RFP). G-RFP can create an infinitely large virtual space for exploration. On the other hand, I-RFP can conFigure a given virtual environment to make users walk. We observed that G-RFP provides higher presence, immersion and a higher mean distance traveled between resets compared to the existing RDW method in a complex real space through a user study. In addition, exploration through I-RFP provided a higher immersion, a comparable presence, and a similar number of resets.

In multi-user Redirected Walking (RDW), the space subdivision method divides a shared physical space into sub-spaces and allocates a sub-space to each user. While this approach has the advantage of precluding any collisions between users, the conventional space subdivision method suffers from frequent boundary resets due to the reduction of available space per user. To address this challenge, in this study, we propose a space subdivision method called Optimal Space Partitioning (OSP) that dynamically divides the shared physical space in real-time. By exploiting spatial information of the physical and virtual environment, OSP predicts the movement of users and divides the shared physical space into optimal sub-spaces separated with shutters. Our OSP framework is trained using deep reinforcement learning to allocate optimal sub-space to each user and provide optimal steering. Our experiments demonstrate that OSP provides higher sense of immersion to users by minimizing the total number of reset counts, while preserving the advantage of the existing space subdivision strategy: ensuring better safety to users by completely eliminating the possibility of any collisions between users beforehand. Our project is available at https://github.com/AppleParfait/OSP-Archive.

Studies in virtual reality (VR) have introduced numerous multisensory simulation techniques for more immersive VR experiences. However, although they primarily focus on expanding sensory types or increasing individual sensory quality, they lack consensus in designing appropriate interactions between different sensory stimuli. This paper explores how the congruence between auditory and visual (AV) stimuli, which are the sensory stimuli typically provided by VR devices, affects the cognition and experience of VR users as a critical interaction factor in promoting multisensory integration. We defined the types of (in)congruence between AV stimuli, and then designed 12 virtual spaces with different types or degrees of congruence between AV stimuli. We then evaluated the presence, immersion, motion sickness, and cognition changes in each space. We observed the following key findings: 1) there is a limit to the degree of temporal or spatial incongruence that can be tolerated, with few negative effects on user experience until that point is exceeded; 2) users are tolerant of semantic incongruence; 3) a simulation that considers synesthetic congruence contributes to the user's sense of immersion and presence. Based on these insights, we identified the essential considerations for designing sensory simulations in VR and proposed future research directions.

Most virtual reality (VR) experiences are held in limited physical space; therefore, increasing the physical space's spatial efficiency is an essential task for the VR industry. Redirected walking maps a virtual path and a real path with unnoticeable distortion, enabling users to walk through a much bigger virtual space than physical space. To hide the distortion from the user, detection thresholds have been measured, entirely focusing on forward steps. However, it is not uncommon for the user to walk non-forward, that is, sideward and backward in VR. In addition to a forward step, adding options for a non-forward step can expand the VR locomotion in any direction. In this work, we measure the translation and curvature detection thresholds for non-forward steps. The results show similar translation detection thresholds with forward-step and wider detection thresholds for the curvature gain in both backward and sideward step experiments. Having sideward and backward steps in the redirected walking arsenal can add freedom to virtual world design and lead to efficient space usage.

Abstract In this paper, we propose a two‐step temporal interpolation network using forward advection to generate smoke simulation efficiently. By converting a low frame rate smoke simulation computed with a large time step into a high frame rate smoke simulation through inference of temporal interpolation networks, the proposed method can efficiently generate smoke simulation with a high frame rate and low computational costs. The first step of the proposed method is optical flow‐based temporal interpolation using deep neural networks (DNNs) for two given smoke animation frames. In the next step, we compute temporary smoke frames with forward advection, a physical computation with a low computational cost. We then interpolate between the results of the forward advection and those of the first step to generate more accurate and enhanced interpolated results. We performed quantitative analyses of the results generated by the proposed method and previous temporal interpolation methods. Furthermore, we experimentally compared the performance of the proposed method with previous methods using DNNs for smoke simulation. We found that the results generated by the proposed method are more accurate and closer to the ground truth smoke simulation than those generated by the previous temporal interpolation methods. We also confirmed that the proposed method generates smoke simulation results more efficiently with lower computational costs than previous smoke simulation methods using DNNs.

Automatic identification of fictional characters is one of the primary analysis techniques for video content. A common approach to detect characters in live-action movies involves detecting human faces; however, this approach cannot be used in non-realistic domains, such as animated movies. Detection of characters in animated movies presents two major challenges: the same subject of character can be expressed in various unique styles, and there are no stylistic or other restrictions on the nature and design of character objects. To address these challenges, we introduce the “animation adaptive region-based convolutional neural network” model to detect characters in animated movies and determine whether the detected characters are human or non-human types. Our model extends the Faster R-CNN model, which is a two-stage object detector, in the following manner: 1) we add a hierarchical animation adaptation module to learn the variety of unique styles from animated movies using a single model; 2) we incorporate a double-detector architecture to focus on the regions that are visually important in determining the character class. We build a new dataset for the animated character detection task. Experiments on this dataset show that our model outperforms other existing representative object detector models in terms of character detection. Furthermore, our model achieves significant performance improvements compared with previous state-of-the-art methods used for the character dictionary generation task. Our model is robust for a variety of animation styles and can find common visual representations of all types of characters, providing an effective way to detect animated characters.

Redirected Walking (RDW) enables users to walk in both virtual and physical tracking spaces simultaneously, which is an effective method to increase presence in Virtual Reality (VR). Recently, RDW technologies have been developed in a multi-user environment where multiple users share the same physical tracking space and simultaneously explore the same virtual space. Meanwhile, in the Steer-To-Optimal-Target (S2OT) method, user actions are planned in RDW by introducing machine learning models such as reinforcement learning. In this paper, we propose a new predictive RDW algorithm "Multiuser-Steer-to-Optimal-Target (MS2OT)" that extends the S2OT method into an environment with multiple users and various types of tracking space. In addition to the steering actions used in S2OT, MS2OT considers pre-reset actions and uses more steering targets and an improved reward function. The locations of multiple users and tracking space information are treated as visual information to be the state of the reinforcement learning model in MS2OT. Hence, the artificial neural network of a multilayer three-dimensional convolutional neural network with a dueling double deep network architecture is learned through Q-Learning. MS2OT significantly reduces the total number of resets compared to the conventional RDW algorithms such as S2C and APF-RDW in a multi-user environment and improves the total distance and average distance between resets during the same period. Experimental results show that MS2OT can process up to 32 users in real-time.

Various studies have been conducted to realize realistic direct interaction in the virtual environment. In this study, we focus on a situation wherein two users using the same physical space explore the same virtual environment using redirected walking (RDW) technology. For two users to meet each other in a virtual environment to realize realistic direct interaction, they must simultaneously meet each other in physical space. However, if the RDW algorithm is applied to each user independently, the relative positions and orientations of the two users can be significantly different in the virtual and physical spaces. We present a recovery algorithm that adjusts the relative position and orientation such that they become the same in the two spaces. Our recovery algorithm uses either modified subtle RDW techniques or overt recovery techniques in three cases depending on the relative position and orientation of the two users. Once the recovered state is reached, the two users can go forward to meet each other and directly interact in the virtual and physical spaces simultaneously. Based on the experiment results, we can confirm that the application of our recovery technology to the system increases the user’s satisfaction in usability and the presence of coexistence in the virtual environment with other users.

This work presents a novel control algorithm of redirected walking called steer-to-optimal-target (S2OT) for effective real-time planning in redirected walking. S2OT is a method of redirection estimating the optimal steering target that can avoid the collision on the future path based on the user's virtual and physical paths. We design and train the machine learning model for estimating optimal steering target through reinforcement learning, especially, using the technique called Deep Q-Learning. S2OT significantly reduces the number of resets caused by collisions between user and physical space boundaries compared to well-known algorithms such as steer-to-center (S2C) and Model Predictive Control Redirection (MPCred). The results are consistent for any combinations of room-scale and large-scale physical spaces and virtual maps with or without predefined paths. S2OT also has a fast computation time of 0.763 msec per redirection, which is sufficient for redirected walking in real-time environments.

In this paper, we propose a new visual motion delay effect to enhance the presence of a user in a virtual underwater experience. To do this, we simulate the resistance in the underwater environment by delaying the hand and head movements of the user's avatar. The motion delay effect is implemented using two components: a drag force and a recovery force. The experimental results show that the combination of a drag force and a recovery force creates a realistic illusion of an underwater experience and enhances the user's presence, satisfaction, and immersion in the virtual underwater environment.

In this paper, we propose a three-dimensional (3D) convolutional neural network (CNN)-based method for predicting the degree of motion sickness induced by a 360° stereoscopic video. We consider the user's eye movement as a new feature, in addition to the motion velocity and depth features of a video used in previous work. For this purpose, we use saliency, optical flow, and disparity maps of an input video, which represent eye movement, velocity, and depth, respectively, as the input of the 3D CNN. To train our machine-learning model, we extend the dataset established in the previous work using two data augmentation techniques: frame shifting and pixel shifting. Consequently, our model can predict the degree of motion sickness more precisely than the previous method, and the results have a more similar correlation to the distribution of ground-truth sickness.

Fast and reliable physically-based simulation techniques are essential for providing flexible visual effects for computer graphics content. In this paper, we propose a fast and reliable hierarchical cloth simulation method, which combines conventional physically-based simulation with deep neural networks (DNN). Simulations of the coarsest level of the hierarchical model are calculated using conventional physically-based simulations, and more detailed levels are generated by inference using DNN models. We demonstrate that our method generates reliable and fast cloth simulation results through experiments under various conditions.

Redirected walking enables immersive walking experience in a limited-sized room. To apply redirected walking efficiently and minimize the number of resets, an accurate path prediction algorithm is required. We propose a data-driven path prediction model using Long Short-Term Memory(LSTM) network. User path data was collected via path exploration experiment on a maze-like environment and fed into LSTM network. Our algorithm can predict user's future path based on user's past position and facing direction data. We compare our path prediction result with actual user data and show that our model can accurately predict user's future path.

This work examines the differences of the effects between using humanoid and non-humanoid avatars on the user's Illusion of Virtual Body Ownership (IVBO) and experience. We used three kinds of avatars: bipedalism group (human), quadrupedalism group (wolf), and serpentine motion group (snake). The result shows that using non-humanoid avatars feel more sense of change of their body. Users feel more proficient when using the humanoid avatar, but are more pleased with the non-humanoid avatars.

An image is a very effective tool for conveying emotions. Many researchers have investigated emotions in images by using various features extracted from images. In this paper, we focus on two high-level features, the object and the background, and assume that the semantic information in images is a good cue for predicting emotions. An object is one of the most important elements that define an image, and we discover through experiments that there is a high correlation between the objects and emotions in images in most cases. Even with the same object, there may be slight differences in emotion due to different backgrounds, and we use the semantic information of the background to improve the prediction performance. By combining the different levels of features, we build an emotion-based feedforward deep neural network that produces the emotion values of a given image. The output emotion values in our framework are continuous values in two-dimensional space (valence and arousal), which are more effective than using a small number of emotion categories to describe emotions. Experiments confirm the effectiveness of our network in predicting the emotions of images.

The chord progression is a fundamental building block in music which sketches the overall mood of a song. Many composers compose music by first deciding chord progressions as a structure and then adding melody and details. Despite its importance, it is rarely used as an emotional feature in music emotion recognition. Few previous works considered chords or intervals as features but the progression or transition of chords were ignored. In this work, we explore the effect of chord progressions in music emotion recognition. We collected music database and extracted features to form an emotion recognition model. The chord progression is then detected from each song, and its effectiveness is showed using cross-validation. The results show that chord progressions have influence in music emotion, especially valence.

In this paper, we present an automated music video generation framework that utilizes emotion synchronization between video and music. After a user uploads a video or music, the framework segments the video and music, and then predicts the emotion of each of the segments. The preprocessing result is stored on the server's database. The user can select a set of videos and music from the database, and the framework will generate a music video. The system finds the most closely associated video segment with the music segment by comparing certain low level features and the emotion differences. We compare our work to a similar music video generation method by performing a user preference study, and show that our method generates a preferable result.

Abstract We introduce an affective image recoloring method for changing the overall mood in the image in a numerically measurable way. Given a semantically segmented source image and a target emotion, our system finds reference image segments from the collection of images that have been tagged via crowdsourcing with numerically measured emotion labels. We then recolorize the source segments using colors from the selected target segments while preserving the gradient of the source image to generate a seamless and natural result. User study confirms the effectiveness of our method in accomplishing the stated goal of altering the mood of the image to match the target emotion level.

Abstract Many visualization techniques use images containing meaningful color sequences. If such images are converted to grayscale, the sequence is often distorted, compromising the information in the image. We preserve the significance of a color sequence during decolorization by mapping the colors from a source image to a grid in the CIELAB color space. We then identify the most significant hues, and thin the corresponding cells of the grid to approximate a curve in the color space, eliminating outliers using a weighted Laplacian eigenmap. This curve is then mapped to a monotonic sequence of gray levels. The saturation values of the resulting image are combined with the original intensity channels to restore details such as text. Our approach can also be used to recolor images containing color sequences, for instance for viewers with color‐deficient vision, or to interpolate between two images that use the same geometry and color sequence to present different data.

Abstract Color assignment is a complex task of incorporating and balancing area configuration, color harmony, and user's intent. In this paper, we present a novel method for automatic color assignment based on theories of color perception. We define color assignment as an optimization problem with respect to the color relationships as well as the spatial configuration of input segments. We also suggest possible constraints that are suitable for task‐specific purposes and for enhancing visual appeal. Our colorization scheme is useful in many applications such as infographics, computer‐aided design, and visual presentation. The user study shows that our method generates perceptually pleasing results over a variety of data sets.

The squash-and-stretch describes the rigidity of the character. This effect is the most important technique in traditional cartoon animation. In this paper, we introduce a method that applies the squash-and-stretch effect to character motion. Our method exaggerates the motion by sequentially applying the spatial exaggeration technique and the temporal exaggeration technique. The spatial exaggeration technique globally deforms the pose in order to make the squashed or stretched pose by modeling it as a covariance matrix of joint positions. Then, the temporal exaggeration technique computes a time-warping function for each joint, and applies it to the position of the joint allowing the character to stretch its links appropriately. The motion stylized by our method is a sequence of squashed and stretched poses with stretching limbs. By performing a user survey, we prove that the motion created using our method is similar to that used in 2D cartoon animation and is funnier than the original motion for human observers who are familiar with 2D cartoon animation.

We present a method for constructing 3D feature flow from video and its application to video stylization. Our method extracts smoothly aligned 3D vectors that describe the smallest variation of colors within a spatiotemporal video cube, and thus effectively preserves both spatial and temporal coherence in a relatively inexpensive manner. As an application of this flow field we present a particle-based video stylization technique to rerender the video in a feature enhancing, painterly style. Our method consists of per-pixel operations and is suitable for GPU implementation, which enables real-time video stylization.

The expansion of the social computing environment as a new basis for socioeconomic activities could enhance the quality of life of older adults, but also it could make the problem of digital divide more serious. In this study, the research directions and agenda of social computing for an aging society are presented, which have two major directions: basic study and applied study. Regarding human aspects in social computing mainly dealt with in the basic study, existing research and related theories on cognitive characteristics, information usage patterns, and social network site use of older adults were examined. In the applied study, a study about the most effective contents application and interfaces through user needs analysis, usage context analysis, prototype design, and so on, was conducted. Those studies are discussed to develop applications on a PC platform, mobile platform, and IPTV platform targeting the older adults population.

Abstract Motion capture cannot generate cartoon‐style animation directly. We emulate the rubber‐like exaggerations common in traditional character animation as a means of converting motion capture data into cartoon‐like movement. We achieve this using trajectory‐based motion exaggeration while allowing the violation of link‐length constraints. We extend this technique to obtain smooth, rubber‐like motion by dividing the original links into shorter sub‐links and computing the positions of joints using Bézier curve interpolation and a mass‐spring simulation. This method is fast enough to be used in real time.

Abstract A hidden‐picture puzzle contains objects hidden in a background image, in such a way that each object fits closely into a local region of the background. Our system converts image of the background and objects into line drawing, and then finds places in which to hide transformed versions of the objects using rotation‐invariant shape context matching. During the hiding process, each object is subjected to a slight deformation to enhance its similarity to the background. The results were assessed by a panel of puzzle‐solvers.

Motion capture cannot generate cartoon-style animation directly. We emulate the rubber-like exaggerations common in traditional character animation as a means of converting motion capture data into cartoon-like movement. We achieve this using trajectory-based motion exaggeration while allowing the violation of link-length constraints. We extend this technique to obtain smooth, rubber-like motion by dividing the original links into shorter sub-links and computing the positions of joints using B´ezier curve interpolation and a mass-spring simulation. This method is fast enough to be used in real time.

Abstract We make moving caricatures from videos on human faces. Using training images, we created a 3D model of an average face. This allows us to transform the image in each frame of an input video, so that it is seen from the front. Then we apply 2D exaggeration rules to caricature each face. Finally, we rotate the face in each frame back to its original position. A panel of viewers gave positive scores to a series of test videos. Copyright © 2007 John Wiley & Sons, Ltd.

We synchronize background music with an animation by changing the timing of both, an approach which minimizesthe damage to either. Starting from a MIDI file and motion data, feature points are extracted from both sources,paired, and then synchronized using dynamic programming to time-scale the music and to timewarp the motion.We also introduce the music graph, a directed graph which encapsulates connections between many short musicsequences. By traversing a music graph we can generate large amounts of new background music, in which weexpect to find a sequence which matches the motion better than the original music.

We present a method to reconstruct a pipe or a canal surface from a point cloud (a set of unorganized points). A pipe surface is defined by a spine curve and a constant radius of a swept sphere, while a variable radius may be used to define a canal surface. In this paper, by using the shrinking and moving least-squares methods, we reduce a point cloud to a thin curve-like point set which will be approximated to the spine curve of a pipe or canal surface. The distance between a point in the thin point cloud and a corresponding point in the original point set represents the radius of the pipe or canal surface.

Abstract Noise is used to create realistic animations that look like natural phenomena as well as procedural textures and shapes by adding randomness to graphical applications. In this paper, we suggest a method to edit noise values to satisfy the constraints that reflect the user's demands while maintaining the inherent statistical features of the noise function. Noise editing uses optimization to minimize the difference between the statistical characteristics of the ideal and edited versions of a noise source. Using our editing method, detailed control of animation and shape data that include noise is possible. Copyright © 2004 John Wiley & Sons, Ltd.

Abstract We present an efficient and robust algorithm for parameterizing the perspective silhouette of a canal surface and detecting each connected component of the silhouette. A canal surface is the envelope of a moving sphere with varying radius, defined by the trajectory C(t) of its center and a radius function r(t) . This moving sphere, S(t) , touches the canal surface at a characteristic circle K(t) . We decompose the canal surface into a set of characteristic circles, compute the silhouette points on each characteristic circle, and then parameterize the silhouette curve. The perspective silhouette of the sphere S(t) from a given viewpoint consists of a circle Q(t) ; by identifying the values of t at which K(t) and Q(t) touch, we can find all the connected components of the silhouette curve of the canal surface. ACM CSS: I.3.7 Computer Graphics–Three Dimensional Graphics and Realism

Abstract We present a fast visualization scheme of soft objects by introducing an efficient evaluation method for a field function. The evaluation of a field function includes distance computation between a point in space and the defining primitives of the soft object. If the unnecessary distance computations between a point in space and the components that do not influence the point can be avoided, the evaluation can be accelerated. For this purpose, we decompose the space into adaptive‐sized cells according to the bounding volume of the components and build a data structure called an interval tree through which the influencing components for a point are sought. The bounding volume of a component is generated by considering the radius of a component and k‐DOPs. The proposed scheme can be used in many applications for soft objects such as modeling and rendering, especially in the interactive modeling process. Copyright © 2003 John Wiley & Sons, Ltd.

How to compute and represent the Gaussian approximations of planar curved objects is described. Also considered are various applications of the Gaussian approximation to various primitive geometric operations on monotone curve segments. The exact solutions for these problems can be computed by solving simultaneous polynomial equations, however, this required an intensive computation time. Efficient heuristic approximation algorithms using simple binary subdivisions on the original geometric components are suggested. It is shown that simple data structures such as arrays and circular lists can be used to represent the Gaussian approximations of planar curved objects.>