Visual field loss, caused by conditions like glaucoma or macular degeneration, affects many people and impacts several life domains. This study contributes to the exploration of how people with visual field loss, such as central and peripheral scotomas, process visual stimuli in digital environments. Current visual attention models are based on experimental data obtained from individuals with normal vision, often overlooking those with limited vision. To address this issue, we compare gaze data from subjects viewing stimuli under two conditions: with and without visual-field masks of varying radii, with the main goal of understanding the role played by different components of vision in the overall grasping of visual information. We use metrics commonly employed for benchmarking saliency models as a means of assessing the similarity between data from each obstruction mask and the control stimuli, which could lead to the conclusion of whether foveal and peripheral vision contribute equally to natural vision or whether one of them stands out in information extraction. Novel saliency models could use this information to predict attention from visually-impaired individuals by possibly balancing these two sources of vision. Our results show a significantly higher similarity between control and central-scotoma saliency maps than between control and peripheral-scotoma data. Another statistical analysis shows no substantial learning effect or familiarity bias when participants revisit the same image under different conditions in the eye-tracking experiment. Finally, a difference-significance study reveals that different radii from central-scotoma conditions demonstrated no meaningful dispersion from each other.

Functional vision assessment is essential for understanding the Quality of Life (QoL) of individuals with visual impairments. The Multi-Luminance Mobility Test (MLMT) is a promising Orientation and Mobility (O&M) method that provides objective functional vision evaluation. However, the current test primarily relies on a statistical factor-based scoring system. Additionally, the scarcity of behavioral data, collection difficulties, and privacy concerns hinder the development of more detailed, behavior-based evaluation metrics. To address these challenges, we propose a data augmentation approach leveraging a Virtual Reality (VR)-based O&M test protocol combined with diffusion policy-based models to generate synthetic behavioral data. In this study, we adapted a transformer-based diffusion policy to generate multi-dimensional motion sequences under varying luminance conditions from VR-based O&M protocols. Quantitative evaluations demonstrate that the synthetic data effectively captures the relationship between luminance and motion for luminance levels seen during training. The zero-shot generalization ability of the policy is also explored. Our findings suggest that diffusion policy-generated synthetic data can enhance functional vision research by addressing data scarcity and supporting the development of behavior-based assessment metrics. The code is available at https://gitlab.univ-nantes.fr/E21A837H/diffusionpolicyvr_motiongeneration.git.

Virtual Reality (VR), as a leading form of immersive media, has rapidly expanded into various application domains, raising the need for standardized methods to evaluate its Quality of Experience (QoE). An ongoing recommendation, ITU-T P.IXC, titled "Interactive test methods for subjective assessment of XR communications", is currently under joint development by the Video Quality Experts Group Immersive Media Group (VQEG-IMG) and ITU-T Study Group 12 (SG12). One major challenge identified in this context is the insufficient attention paid to user behaviors during tasks, despite the increasing availability of behavioral data. Although such data are often collected, they are rarely analyzed in depth. In this paper, we proposed a Hidden Markov Model (HMM)-based behavior analysis approach. Using a VR-based Orientation and Mobility (O&M) test designed for Quality of Life (QoL) assessment–one of the promising applications of VR–we leveraged the collected behavioral data to model behavior patterns. Our results demonstrate that the proposed approach can effectively extract behavior-aware features, highlighting the potential of integrating behavior analysis into QoL assessment frameworks. We believe this work provides valuable insights into incorporating behavioral metrics into immersive media QoE evaluation and contributes to the development of ITU-T P.IXC.

Gaze behavior of virtual characters in video games and virtual reality experiences is a key factor of realism and immersion. Indeed, gaze plays many roles when interacting with the environment; not only does it indicate what characters are looking at, but it also plays an important role in verbal and non-verbal behaviors and in making virtual characters alive. Automated computing of gaze behaviors is however a challenging problem, and to date none of the existing methods are capable of producing close-to-real results in an interactive context. We therefore propose a novel method that leverages recent advances in several distinct areas related to visual saliency, attention mechanisms, saccadic behavior modelling, and head-gaze animation techniques. Our approach articulates these advances to converge on a multi-map saliency-driven model which offers real-time realistic gaze behaviors for non-conversational characters, together with additional user-control over customizable features to compose a wide variety of results. We first evaluate the benefits of our approach through an objective evaluation that confronts our gaze simulation with ground truth data using an eye-tracking dataset specifically acquired for this purpose. We then rely on subjective evaluation to measure the level of realism of gaze animations generated by our method, in comparison with gaze animations captured from real actors. Our results show that our method generates gaze behaviors that cannot be distinguished from captured gaze animations. Overall, we believe that these results will open the way for more natural and intuitive design of realistic and coherent gaze animations for real-time applications.

Understanding the deployment of human gaze on 3D graphical objects is of critical importance in order to propose rich and complex 3D environments without strong latency nor rendering constraints. However, the data needed to study this gaze deployment can be costly and difficult to obtain, especially in the context of the Covid-19 pandemic where in-lab experiments are strongly discouraged. In order to alleviate these issues, we propose to use the BubbleView metaphor as a way of crowdsourcing visual attention data on 3D graphical content. In this paper, we question the adequacy of this method to provide a reliable proxy for visual attention in the context of 3D graphical objects. Moreover, we show how data obtained in this manner can be used to train visual saliency models, with only a slight tradeoff in performances compared to the use of ground-truth eye-tracking data.

Facial expression recognition is an important and challenging task for both the computer vision and affective computing communities, and even more specifically in the context of multimedia applications, where audience understanding is of particular interest. Recent data-oriented approaches have created the need for large-scale annotated datasets. However, most existing datasets present some weaknesses, because of the collecting methods used. In order to further highlight these issues, we investigate in this work how human visual attention is deployed when performing a facial expression recognition task. To do so, we carried out several complementary experiments, using the eye-tracking technology, as well as the BubbleView metaphor, both under laboratory and crowdsourcing settings. We show significant variations in gaze patterns depending on the emotion represented, but also on the difficulty of the task, i.e., whether the emotion is correctly recognised or not. Moreover, we use these results to propose recommendations on the ways to collect label data for facial expression recognition datasets.

According to the literature regarding visual saliency, observers may exhibit considerable variations in their gaze behaviors. These variations are influenced by aspects such as cultural background, age or prior experiences, but also by features in the observed images. The dispersion between the gaze of different observers looking at the same image is commonly referred as inter-observer congruency (IOC). Predicting this congruence can be of great interest when it comes to study the visual perception of an image. In this paper, we introduce a new method based on deep learning techniques to predict the IOC of an image. This is achieved by first extracting features from an image through a deep convolutional network. We then show that using such features to train a model with a shallow network regression technique significantly improves the precision of the prediction over existing approaches.