Visual field loss, caused by conditions like glaucoma or macular degeneration, affects many people and impacts several life domains. This study contributes to the exploration of how people with visual field loss, such as central and peripheral scotomas, process visual stimuli in digital environments. Current visual attention models are based on experimental data obtained from individuals with normal vision, often overlooking those with limited vision. To address this issue, we compare gaze data from subjects viewing stimuli under two conditions: with and without visual-field masks of varying radii, with the main goal of understanding the role played by different components of vision in the overall grasping of visual information. We use metrics commonly employed for benchmarking saliency models as a means of assessing the similarity between data from each obstruction mask and the control stimuli, which could lead to the conclusion of whether foveal and peripheral vision contribute equally to natural vision or whether one of them stands out in information extraction. Novel saliency models could use this information to predict attention from visually-impaired individuals by possibly balancing these two sources of vision. Our results show a significantly higher similarity between control and central-scotoma saliency maps than between control and peripheral-scotoma data. Another statistical analysis shows no substantial learning effect or familiarity bias when participants revisit the same image under different conditions in the eye-tracking experiment. Finally, a difference-significance study reveals that different radii from central-scotoma conditions demonstrated no meaningful dispersion from each other.

Eye tracking has historically been a very popular tool. The data it records allow us to understand how people behave and what they attend to within our visual world; under this perspective the experiments, applications and use-cases are endless. Therefore, it is not surprising to witness a strong rise in the use of eXtended Reality (XR) devices with embedded eye trackers in research. These devices allow for less obtrusive experimenting conditions, and a significantly higher experimental control compared to traditional desktop testing. The use of eye tracking in XR is increasing and so is the need for a toolbox enabling consensus about eye tracking methods in 3D. We present the Salient360! toolbox: it implements functions to identify saccades and fixations and output gaze features (e.g., saccade directions) to generate saliency maps, fixation maps, and scanpath data. It implements comparisons of gaze data with methods adapted to 3D. We plan continuous improvements of the toolbox as the community develops new tools and methods dedicated to 360°gaze tracking. We hope that this toolbox will spark discussions about the methodology of 3D gaze processing, facilitate running experiments, and improve studying gaze in 3D. https://github.com/David-Ef/salient360Toolbox

Eye tracking can serve as a gateway to studying the mind. For this reason it has been adopted by a diverse range of scientific communities. With the improvement of the quality of head-mounted virtual reality devices (HMDs) over the past 10 years, eye tracking has been added to capture gaze in immersive environments. The use of HMDs with eye tracking is increasing significantly and so is the need for a toolbox enabling consensus about eye tracking methods in 3D. We present the Salient360! toolbox: it implements functions to identify saccades and fixations and output gaze characteristics (e.g., fixation duration or saccade directions), to generate saliency maps, fixation maps, and scanpath data. It also implements routines made to compare gaze data that were adapted to 3D. We hope that this toolbox will spark discussions about the methodology of 3D gaze processing, facilitate running experiments, and improve the gaze study in 3D. https://github.com/David-Ef/salient360Toolbox

Understanding the deployment of human gaze on 3D graphical objects is of critical importance in order to propose rich and complex 3D environments without strong latency nor rendering constraints. However, the data needed to study this gaze deployment can be costly and difficult to obtain, especially in the context of the Covid-19 pandemic where in-lab experiments are strongly discouraged. In order to alleviate these issues, we propose to use the BubbleView metaphor as a way of crowdsourcing visual attention data on 3D graphical content. In this paper, we question the adequacy of this method to provide a reliable proxy for visual attention in the context of 3D graphical objects. Moreover, we show how data obtained in this manner can be used to train visual saliency models, with only a slight tradeoff in performances compared to the use of ground-truth eye-tracking data.

In the past years, AI has seen many advances in the field of NLP. This has led to the emergence of LLMs, such as the now famous GPT-3.5, which revolutionise the way humans can access or generate content. Current studies on LLM-based generative tools are mainly interested in the performance of such tools in generating relevant content (code, text or image). However, ethical concerns related to the design and use of generative tools seem to be growing, impacting the public acceptability for specific tasks. This paper presents a questionnaire survey to identify the intention to use generative tools by employees of an IT company in the context of their work. This survey is based on empirical models measuring intention to use (TAM by Davis, 1989, and UTAUT2 by Venkatesh and al., 2008). Our results indicate a rather average acceptability of generative tools, although the more useful the tool is perceived to be, the higher the intention to use seems to be. Furthermore, our analyses suggest that the frequency of use of generative tools is likely to be a key factor in understanding how employees perceive these tools in the context of their work. Following on from this work, we plan to investigate the nature of the requests that may be made to these tools by specific audiences.

Facial expression recognition is an important and challenging task for both the computer vision and affective computing communities, and even more specifically in the context of multimedia applications, where audience understanding is of particular interest. Recent data-oriented approaches have created the need for large-scale annotated datasets. However, most existing datasets present some weaknesses, because of the collecting methods used. In order to further highlight these issues, we investigate in this work how human visual attention is deployed when performing a facial expression recognition task. To do so, we carried out several complementary experiments, using the eye-tracking technology, as well as the BubbleView metaphor, both under laboratory and crowdsourcing settings. We show significant variations in gaze patterns depending on the emotion represented, but also on the difficulty of the task, i.e., whether the emotion is correctly recognised or not. Moreover, we use these results to propose recommendations on the ways to collect label data for facial expression recognition datasets.

Experiments in immersive environments allow the collection of large amounts of data that are closely related to individual behaviour. The recording of such experiments allows for the complex study of under-constrained tasks. That is, tasks that allow for a high degree of contingency in their resolution. This contingency allows for better discrimination of individual behaviour. However, the high complexity of the tasks makes them difficult to analyse.

Many objective quality metrics have been proposed over the years to automate the task of subjective quality assessment. However, few of them are designed for 3D graphical contents with appearance attributes; existing ones are based on geometry and color measures, yet they ignore the visual saliency of the objects. In this paper, we combined an optimal subset of geometry-based and color-based features, provided by a state-of-the-art quality metric for 3D colored meshes, with a visual attention complexity feature adapted to 3D graphics. The performance of our proposed new metric is evaluated on a dataset of 80 meshes with diffuse colors, generated from 5 source models corrupted by commonly used geometry and color distortions. With our proposed metric, we showed that the use of the attentional complexity feature brings a significant gain in performance and better stability.

Understanding human visual attention mechanisms and interaction in immersive scenes are of great importance in perception. In immersive context, users are able to interact with increasingly rich/ complex 3D contents during rendering. Therefore, to avoid latency or rendering issues, there is a critical need for simplifying and filtering the primitives and levels of detail of these high-quality 3D graphics (according to viewing conditions). In order to ensure a high user's quality of experience (QoE) during interactive visualization, these processing operations should take into account perceptual information. To do so, we suggest an approach that uses visual saliency information of the 3D scene to guide simplification and level of details selection. In this paper, we question the efficiency of our novel approach to compute visual saliency on 3D graphics. This approach takes into consideration the viewpoint from which the 3D content was seen/rendered when computing saliency (whichever the considered viewpoint is), by using saliency maps of view-based method computed offline. Such technique could help alleviate rendering constraints during interactive visualization.

The recent development of Virtual Reality (VR) technologies and interactive visual content faces some important challenges in multimedia processing and Quality of Experience (QoE). Considering omnidirectional, also called 360-degree, content, one major research topic is the development of reliable visual attention models. In this paper, we present a new dataset to study the influence of emotions on eye behavior in omnidirectional content. This dataset is based on an eye-tracking experiment where 19 observers have assessed emotional valence and arousal dimensions in 360-degree images. Several analyses are then conducted to compare fixation and saccade features, inter-observer saliency congruency and spatial oculomotor biases in positive, neutral and negative content. Results show a significant impact of negative images on visual attention, with more visual agitation and avoidance behavior from larger, longer and faster saccades. However, no obvious difference of eye behavior is found between positive and neutral stimulis on this dataset.

In order to predict where humans look in a 3D immersive environment, saliency can be computed using either 3D saliency models or view-based approaches (2D projection). In fact, building a 3D complete model is still a challenging task that is not investigated enough in the research field while 2D imaging approaches have been extensively studied and have shown solid performances.As 6 degrees of freedom are allowed in volumetric videos, users are able to navigate through the content in different manners. In this case, 2D saliency models might be less robust if applied naïvely, since advanced parameters such as viewing distance are not considered in such models.The aim of this paper is to investigate the influence of viewpoint on 2D saliency models when applied on volumetric data and this to get a better understanding of how viewpoint information could be integrated into view-based approaches.To do so, a subjective psycho-visual experiment was conducted and a fine analysis was led using the variance analysis statistical method.

Visual attention is a key feature to optimize visual experience of many multimedia applications. 2D visual attention computational modeling is an active research area considering the visualization of natural images on a conventional display. In this paper, we question the ability of such models to be applicable to single computer-generated objects rendered at different sizes (on a conventional display). We benchmark state of art visual attention models and investigate the influence of the viewpoint on those computational models applied on volumetric data and this to get a better understanding of how viewpoint information could be integrated into view-based approaches. To do so, a subjective experiment was conducted and a fine analysis was led using the variance analysis statistical method.

Research on visual attention in 360° content is crucial to understand how people perceive and interact with this immersive type of content and to develop efficient techniques for processing, encoding, delivering and rendering. And also to offer a high quality of experience to end users. The availability of public datasets is essential to support and facilitate research activities of the community. Recently, some studies have been presented analyzing exploration behaviors of people watching 360° videos, and a few datasets have been published. However, the majority of these works only consider head movements as proxy for gaze data, despite the importance of eye movements in the exploration of omnidirectional content. Thus, this paper presents a novel dataset of 360° videos with associated eye and head movement data, which is a follow-up to our previous dataset for still images [14]. Head and eye tracking data was obtained from 57 participants during a free-viewing experiment with 19 videos. In addition, guidelines on how to obtain saliency maps and scanpaths from raw data are provided. Also, some statistics related to exploration behaviors are presented, such as the impact of the longitudinal starting position when watching omnidirectional videos was investigated in this test. This dataset and its associated code are made publicly available to support research on visual attention for 360° content.

Virtual Reality (VR) provides the users with new immersive media experiences, offering the possibility to freely explore 360° content. Understanding these new exploration behaviors is crucial for the development of efficient techniques for processing, coding, delivering and rendering omnidirectional content to offer the highest possible Quality of Experience (QoE). Progress has already been made on visual attention (VA) modeling for 360° content. In this paper we briefly review the current status of research on this topic that led us to propose a benchmarking platform for evaluating and comparing the performance of models for saliency and scanpath prediction for 360° content. This paper introduces the `UN Salient360! benchmark” platform featuring a dataset, a toolbox and a framework for evaluation of different class of models. This online platform can be found in httns://salient360.ls2n.fr/.

In this paper, we investigate the visual attention modeling for stereoscopic video from the following two aspects. First, we build one large-scale eye tracking database as the benchmark of visual attention modeling for stereoscopic video. The database includes 47 video sequences and their corresponding eye fixation data. Second, we propose a novel computational model of visual attention for stereoscopic video based on Gestalt theory. In the proposed model, we extract the low-level features, including luminance, color, texture, and depth, from discrete cosine transform coefficients, which are used to calculate feature contrast for the spatial saliency computation. The temporal saliency is calculated by the motion contrast from the planar and depth motion features in the stereoscopic video sequences. The final saliency is estimated by fusing the spatial and temporal saliency with uncertainty weighting, which is estimated by the laws of proximity, continuity, and common fate in Gestalt theory. Experimental results show that the proposed method outperforms the state-of-the-art stereoscopic video saliency detection models on our built large-scale eye tracking database and one other database (DML-ITRACK-3D).

The emergence of UHD video format induces larger screens and involves a wider stimulated visual angle. Therefore, its effect on visual attention can be questioned since it can impact quality assessment, metrics but also the whole chain of video processing and creation. Moreover, changes in visual attention from different viewing conditions challenge visual attention models. In this paper, we present a comparative study of visual attention and viewing behavior on three video datasets in SD, HD and UHD conditions. Then, we propose and assess an improvement for video visual attention models by applying a stimulated visual angle dependent center model.

Image memorability prediction is a recent topic in computer science. First attempts have shown that it is possible to computationally infer from the intrinsic properties of an image the extent to which it is memorable. In this paper, we introduce a fine-tuned deep learning-based computational model for image memorability prediction. The performance of this model significantly outperforms previous work and obtains a 32.78% relative increase compared to the best-performing model from the state of the art on the same dataset. We also investigate how our model generalizes on a new dataset of 150 images, for which memorability and affective scores were collected from 50 participants. The prediction performance is weaker on this new dataset, which highlights the issue of representativity of the datasets. In particular, the model obtains a higher predictive performance for arousing negative pictures than for neutral or arousing positive ones, recalling how important it is for a memorability dataset to consist of images that are appropriately distributed within the emotional space.

The emergence of UHD video format induces larger screens and involves a wider stimulated visual angle. Therefore, its effect on visual attention can be questioned since it can impact quality assessment, metrics but also the whole chain of video processing and creation. Moreover, changes in visual attention from different viewing conditions challenge visual attention models. In this paper, we present a new HD and UHD video eye tracking dataset composed of 37 high quality videos observed by more than 35 naive observers. This dataset can be used to compare viewing behavior and visual saliency in HD and UHD, as well as for any study on dynamic visual attention in videos. It is available at http://ivc.univ-nantes.fr/en/databases/HD_UHD_Eyetracking_Videos/.

Delivering the same digital image to several users is not necessarily providing them the same experience. In this study, we focused on how different affective experiences impact the memorability of an image. Forty-nine participants took part in an experiment in which they saw a stream of images conveying various emotions. One day later, they had to recognize the images displayed the day before and rate them according to the positivity/ negativity of the emotional experience the images induced. In order to better appreciate the underlying idiosyncratic factors that affect the experience under test, prior to the test session we collected not only personal information but also results of psychological tests to characterize individuals according to their dominant personality in terms of masculinity-femininity (Bem Sex Role Inventory) and to measure their emotional state. The results show that the way an emotional experience is rated depends on personality rather than biological sex, suggesting that personality could be a mediator in the well-established differences in how males and females experience emotional material. From the collected data, we derive a model including individual factors relevant to characterize the memorability of the images, in particular through the emotional experience they induced.

Tone mapping operators (TMOs) employed to visualize high dynamic range (HDR) content on conventional low dynamic range (LDR) devices suffer from two major drawbacks. First, none of them can faithfully reproduce all the contrast present in HDR images. Second, most of them require one or more parameters which are mostly content specific and their optimal values can be set only via subjective testing. To address these issues, this paper proposes that ‘quality driven’ adaptive contrast enhancement is a practical solution. This is achieved by enhancing the contrast adaptively based on the loss of contrast between the HDR and tone mapped image. Experimental results confirm that the proposed adaptive solution always improves upon the contrast achieved from whatever given TMO parameter settings in the tested images. So it helps to achieve the results of a more optimal TMO parameter setting without the human input.

Many computational models of visual attention performing well in predicting salient areas of 2D images have been proposed in the literature. The emerging applications of stereoscopic 3D display bring an additional depth of information affecting the human viewing behavior, and require extensions of the efforts made in 2D visual modeling. In this paper, we propose a new computational model of visual attention for stereoscopic 3D still images. Apart from detecting salient areas based on 2D visual features, the proposed model takes depth as an additional visual dimension. The measure of depth saliency is derived from the eye movement data obtained from an eye-tracking experiment using synthetic stimuli. Two different ways of integrating depth information in the modeling of 3D visual attention are then proposed and examined. For the performance evaluation of 3D visual attention models, we have created an eye-tracking database, which contains stereoscopic images of natural content and is publicly available, along with this paper. The proposed model gives a good performance, compared to that of state-of-the-art 2D models on 2D images. The results also suggest that a better performance is obtained when depth information is taken into account through the creation of a depth saliency map, rather than when it is integrated by a weighting method.

Digital images can be analyzed at wide range of levels going from pixel arrangement to semantics. As a consequence, finding a visual complexity estimator is a difficult task. In this article we propose a definition of attention based perceptual complexity. We study the performance of human eye movements and of different models of computational attention against a ground-truth of image complexity based on the observation time of an image description task. The results obtained show that besides its lack of semantic processing, attentional behavior is a good estimator of image complexity.