Abstract Extended exposure to virtual reality displays has been linked to the emergence of cybersickness, characterized by symptoms such as nausea, dizziness, fatigue, and disruptions in eye movements. The main objective of our study is to examine the effects of real-time fine-tuning of stereo parameters and blurriness in virtual reality on the discomfort level of users who are experiencing motion sickness triggered by the display. Our hypothesis proposes that by dynamically correcting the rendering settings, the symptoms of motion sickness can be relieved and the overall VR user experience can be improved. Our methodology commences with a prediction model for the comfort level of the viewer based on their gaze parameters, such as pupil diameter, blink count, gaze position, and fixation duration. We then propose a method to dynamically adapt the stereoscopic rendering parameters by considering the predicted comfort level of the viewer.

Abstract While visual saliency has been used for various purposes in virtual reality (VR), the efforts to properly understand the saliency mechanism in VR remain insufficient. In this paper, we present an extensive comparative analysis of learning‐based and heuristic‐based approaches to visual saliency prediction in immersive VR experienced using head‐mounted‐displays with a particular focus on the contribution of the depth cue. To this end, we use three learning‐based RGB‐D image saliency detection methods and two heuristic‐based RGB‐D image saliency detection methods on a VR dataset curated from three distinct virtual environments under two‐dimensional and three‐dimensional viewing conditions. Additionally, we extend the analysis by including a heuristic‐based RGB video saliency detection method and its depth‐infused version. The results acquired using these seven methods reveal the superiority of the learning‐based RGB‐D image saliency prediction methods in VR and validate the importance of the depth cue in the saliency prediction of virtual environments.

Abstract Today, the cutting edge of computer vision research greatly depends on the availability of large datasets, which are critical for effectively training and testing new methods. Manually annotating visual data, however, is not only a labor‐intensive process but also prone to errors. In this study, we present NOVA, a versatile framework to create realistic‐looking 3D rendered worlds containing procedurally generated humans with rich pixel‐level ground truth annotations. NOVA can simulate various environmental factors such as weather conditions or different times of day, and bring an exceptionally diverse set of humans to life, each having a distinct body shape, gender and age. To demonstrate NOVA's capabilities, we generate two synthetic datasets for person tracking. The first one includes 108 sequences, each with different levels of difficulty like tracking in crowded scenes or at nighttime and aims for testing the limits of current state‐of‐the‐art trackers. A second dataset of 97 sequences with normal weather conditions is used to show how our synthetic sequences can be utilized to train and boost the performance of deep‐learning based trackers. Our results indicate that the synthetic data generated by NOVA represents a good proxy of the real‐world and can be exploited for computer vision tasks.

This work explores a set of well-studied visual saliency features through seven saliency prediction methods with the aim of assessing how applicable they are for estimating visual saliency in dynamic virtual reality (VR) environments that are experienced with head-mounted displays. An in-depth analysis of how the saliency methods that make use of depth cues compare to ones that are based on purely image-based (2D) features is presented. To this end, a user study was conducted to collect gaze data from participants as they were shown the same set of three dynamic scenes in 2D desktop viewing and 3D VR viewing using a head-mounted display. The scenes convey varying visual experiences in terms of contents and range of depth-of-field so that an extensive analysis encompassing a comprehensive array of viewing behaviors could be provided. The results indicate that 2D features matter as much as depth for both viewing conditions, yet depth cue is slightly more important for 3D VR viewing. Furthermore, including depth as an additional cue to the 2D saliency methods improves prediction for both viewing conditions, and the benefit margin is greater in 3D VR viewing.

Abstract Previous studies showed that virtual reality (VR) environments can affect emotional state and cause significant changes in physiological responses. Aside from these effects, inadvertently induced cybersickness is a notorious problem faced in VR. In this study, to further investigate the effects of virtual environments (VEs) with different context, three dynamic VEs were created. Each VE had a particular purpose: evoking no emotion in Campfire (CF), unpleasant emotions in Hospital (HH), and cybersickness symptoms in Roller Coaster (RC). We made use of objective measurements of physiological responses such as pupil dilation, blinks, fixations, saccades, and heart rate, as well as subjective self‐assessments via pre‐ and post‐VE session questionnaires. While previous studies investigate different subsets of these measures, our study makes a comprehensive analysis of them jointly in dynamic VEs. The results of the study indicate that cybersickness produced higher saccade mean speed, whereas unpleasant context caused higher fixation count, saccade rate, and pupil dilation. Moreover, CF decreased anxiety, whereas HH and RC increased it and they also decreased comfort. Participants felt cybersickness in all VEs even in CF which is designed to minimize the effects.

Abstract We present a novel method for retargeting human motion to arbitrary 3D mesh models with as little user interaction as possible. Traditional motion‐retargeting systems try to preserve the original motion, while satisfying several motion constraints. Our method uses a few pose‐to‐pose examples provided by the user to extract the desired semantics behind the retargeting process while not limiting the transfer to being only literal. Thus, mesh models with different structures and/or motion semantics from humanoid skeletons become possible targets. Also considering the fact that most publicly available mesh models lack additional structure (e.g. skeleton), our method dispenses with the need for such a structure by means of a built‐in surface‐based deformation system. As deformation for animation purposes may require non‐rigid behaviour, we augment existing rigid deformation approaches to provide volume‐preserving and squash‐and‐stretch deformations. We demonstrate our approach on well‐known mesh models along with several publicly available motion‐capture sequences.

The authors modify the conventional DPCM framework of motion-compensated video coding to improve its error resilience in a packet loss environment. The authors propose a system with two decoding modes based on whether a packet is lost or not, so as to minimize the combined distortion due to quantization and transmission errors.