Virtual reality (VR) is found to present significant cognitive challenges due to its immersive nature and frequent sensory conflicts. This study systematically investigates the impact of sensory conflicts induced by VR remapping techniques on cognitive fatigue, and unveils their correlation. We utilized three remapping methods (haptic repositioning, head-turning redirection, and giant resizing) to create different types of sensory conflicts, and measured perceptual thresholds to induce various intensities of the conflicts. Through experiments involving cognitive tasks along with subjective and physiological measures, we found that all three remapping methods influenced the onset and severity of cognitive fatigue, with visual-vestibular conflict having the greatest impact. Interestingly, visual-experiential/memory conflict showed a mitigating effect on cognitive fatigue, emphasizing the role of novel sensory experiences. This study contributes to a deeper understanding of cognitive fatigue under sensory conflicts and provides insights for designing VR experiences that align better with human perceptual and cognitive capabilities.

Using VR in reclining & lying positions is getting common for users, but upward views caused by posture have to be redirected to be parallel to the ground as when users are standing. This affects users’ locomotion performances in VR due to potential physical restrictions, and the visual-vestibular-proprioceptive conflict. This paper is among the first to investigate the suited locomotion methods and how reclining & lying positions and redirection affect them in such conditions. A user-elicitation study was carried out to construct a set of locomotion methods based on users’ preferences when they were in different reclining & lying positions. A second study developed user-preferred ’tapping’ and ’chair rotating’ gestures, by evaluating their performances at various body reclining angles, we measured the general impacts of posture and redirection. The results showed that these methods worked effectively, but exposed some shortcomings, and users performed worst at 45° reclining angles. Finally, four upgraded methods were designed and verified to improve the locomotion performances.

Facial Action Unit (AU) intensity prediction is essential to facial expression analysis and emotion recognition, and thus has attracted much attention from the community. In comparison, AU localization, albeit being important to emotion visualization and tracking, was relatively unexplored. In addition, as most existing AU intensity prediction methods take a cropped face image as input, their actual speed at run-time is often penalized by the pre-processing steps such as face detection and alignment. At the same time, their performance (in terms of inference speed), also does not scale well to multi-face images. To alleviate these problems, we propose a joint AU intensity prediction and localization method that works directly on the whole input image, thus eliminating the need of any pre-processing step and achieving the same inference speed regardless of the number of faces in the image. Based on the observation that different relevancy exists between AU intensities categories, a flexible cost function is proposed. At inference time, we introduce a non-maximum intensity suppression model to refine the prediction result. In order to leverage existing datasets without AU region groundtruth, we also propose an automatic AU region labeling method. Experiments on two benchmark databases, DISFA and FERA2015, show that the proposed approach outperforms the state-of-the-art methods on three metrics, ICC, MAE and F1 for the AU intensity prediction task.

Automatic animation synthesis has attracted much attention from the community. As most existing methods take a small number of discrete expressions rather than continuous expressions, their integrity and reality of the facial expressions is often compromised. In addition, the easy manipulation with simple inputs and unsupervised processing, although being important to the automatic facial expression animation applications, is relatively less concerned. To address these issues, we propose an unsupervised continuous automatic facial expression animation approach through action units (AU) transfer in the latent space of generative adversarial networks. The expression descriptor which is depicted with AU vector is transferred into the input image without the need of labeled pairs of images and even without their expressions and further network training. We also propose a new approach to quickly generate input image's latent code and cluster the boundaries of different AU attributes with their latent codes. Two latent code operators, vector addition and continuous interpolation, are leveraged for facial expression animation simulating align with the boundaries in the latent space. Experiments have shown that the proposed approach is effective on facial expression translation and animation synthesis.

The existing 3D scene layout methods, which mainly focus on indoor scenes, are limited in outdoor applications. In this paper, the example-based modeling method is introduced into the outdoor modeling, and an automatic layout optimization method for outdoor scenes is proposed. As an application platform, the sketch-based 3D model retrieval and assemble system is realized as well. Different from the current methods, firstly, we adopt an improved manifold sorting algorithm in sketch retrieval method, which can get the 3D models rapidly; secondly, according to particular properties of outdoor architectures, specialized energy constraints are proposed, which defines the energy function that meets the functional and aesthetic needs; thirdly, the scene achieves automatic layout by adopting simulated annealing algorithm. By stepped asymptotic optimization strategy and random jumps, we can avoid falling into constraint conflicts and local optimal traps. Experimental results show the robustness and effectiveness of our algorithm in different scenes. Our algorithm has been applied in the actual development of game scenes.