Virtual Reality (VR) enables immersive and realistic experiences in which complex multisensory interactions often impose elevated cognitive demands on users. Excessive levels of cognitive load (CL) have been shown to degrade performance and user experience, motivating the search for robust and non-intrusive indicators for CL monitoring in immersive environments. While recent approaches have leveraged physiological signals for this purpose, many of these signals are sensitive to motion and require complex sensor setups. Given the widespread integration of eye tracking in modern head-mounted displays, gaze behavior emerges as a promising alternative for objective CL assessment. In this work, we present a comprehensive analysis of gaze behavior and its relationship to cognitive load in immersive, task-oriented VR. Using a publicly available dataset collected in a multisensory visual search experience, we first examine how a range of gaze-derived markers, including fixations, saccades, eye eccentricity, and pupil dilation, vary across cognitive load conditions. We then investigate their relationship with biomarkers derived from complementary physiological signals, including electrocardiogram (ECG), electrodermal activity (EDA), and respiration, to better understand how gaze-based markers relate to broader physiological responses associated with cognitive load. Our results reveal consistent changes in oculomotor behavior under high cognitive load, characterized by reduced saccade amplitudes and eye eccentricity, together with increased pupil dilation. These patterns indicate a more centered and less exploratory visual behavior, consistent with attention tunneling under high task demands. In addition, we identify strong correlations between several gaze markers and the phasic component of electrodermal activity, a well-known indicator of mental effort. Together, these findings highlight the potential of gaze-based measures as lightweight and non-intrusive indicators of cognitive load, supporting the development of adaptive, user-aware VR applications.

Our ability to perceive and navigate the spatial world is a cornerstone of human experience, relying on the integration of visual and auditory cues to form a coherent sense of depth and distance. In stereoscopic 3D vision, depth perception requires fixation of both eyes on a target object, which is achieved through vergence movements, with convergence for near objects and divergence for distant ones. In contrast, auditory cues provide complementary depth information through variations in loudness, interaural differences (IAD), and the frequency spectrum. We investigate the interaction between visual and auditory cues and examine how contradictory auditory information can overdrive visual depth perception in virtual reality (VR). When a new visual target appears, we introduce a spatial discrepancy between the visual and auditory cues: the visual target is shifted closer to the previously fixated object, while the corresponding sound localization is displaced in the opposite direction. By integrating these conflicting cues through multimodal processing, the resulting percept is biased toward the intended depth location. This audiovisual fusion counteracts depth compression, thus reducing the required vergence magnitude and enabling faster gaze retargeting. Such audio-driven depth enhancement may further help mitigate the vergence-accommodation conflict (VAC) in scenarios where physical depth must be compressed. In a series of psychophysical studies, we first assess the efficiency of depth overdriving for various VR-relevant combinations of initial fixations and shifted target locations, considering different scenarios of audio displacements and their loudness and frequency parameters. Next, we quantify the resulting speedup in gaze retargeting for target shifts that can be successfully overdriven by sound manipulations. Finally, we apply our method in a naturalistic VR scenario where user interface interactions with the scene show an extended perceptual depth.

The study of cognitive load (CL) has been an active field of research across disciplines such as psychology, education, and computer graphics and visualization for decades. In the context of Virtual Reality (VR), understanding mental demand becomes particularly relevant, as immersive experiences increasingly integrate multisensory stimuli that require users to distribute their limited cognitive resources. In this work, we investigate the effects of cognitive load during a search task in VR, combining objective and subjective measurements, including physiological signals and validated questionnaires. We designed an experiment in which participants performed a visual search task under two cognitive load conditions (either alone or while responding to a concurrent auditory task) and across two visual search areas (90° and 360°). We collected a rich dataset comprising task performance, eye tracking, electrocardiogram (ECG), electrodermal activity (EDA), photoplethysmography (PPG), and inertial measurements, along with subjective assessments (NASA-TLX questionnaires). Our analysis shows that increased cognitive load hinders visual search performance and affects multiple physiological markers, offering a solid foundation for future research on cognitive load in multisensory virtual environments.

Saliency prediction in 360° video plays an important role in modeling visual attention, and can be leveraged for content creation, compression techniques, or quality assessment methods, among others. Visual attention in immersive environments depends not only on visual input, but also on inputs from other sensory modalities, primarily audio. Despite this, only a minority of saliency prediction models have incorporated auditory inputs, and much remains to be explored about what auditory information is relevant and how to integrate it in the prediction. In this work, we propose an audiovisual saliency model for 360° video content, AViSal360. Our model integrates both spatialized and semantic audio information, together with visual inputs. We perform exhaustive comparisons to demonstrate both the actual relevance of auditory information in saliency prediction, and the superior performance of our model when compared to previous approaches.