Eye gaze is fundamental to common sensorimotor activities. It directly indicates attention and its allocation, making it a critical human factor that reflects cognitive behavior. Data-driven measurements of gaze behavior quantitatively evaluate the allocation of visual attention, indicating the mental and physical activity status of individuals. Situational awareness provides a solid and semantically rich basis for gaze behavior during sensorimotor activities. However, few attempts have been made so far to measure eye gaze data from the perspective of situational awareness. In this paper, we propose four new computational measures of gaze behavior that align with the fundamental concept of situational awareness, effectively measuring the efficiency of visual scanning and sensorimotor activity. Our results have clearly shown that the proposed measures are effective and perform satisfactorily compared to the closely related methods. This work offers a new data-driven approach for biomarker discovery and behavioral biometrics using eye gaze data.

The coordination of eye and head in visual scanning is a fundamental behavior of humans in everyday sensorimotor activities such as walking and navigation. Deep understanding of the spatiotemporal dynamics of this coordination behavior undoubtedly plays an important role for many fields. However, relatively little is known about the computational and high-order interactions between eye and head in visual scanning during sensorimotor tasks. In this paper, based on the utilization of a recent tool from information theory, Partial Information Decomposition (PID), we quantify high-order components, namely uniqueness, redundancy, and synergy, in spatiotemporal interactions between eye movement and head motion time-series data during augmented reality-based navigation experience. To our knowledge, this is the first data-driven approach that leverages an information-theoretic tool to characterize high-order interactions involved in eye-head coordination during sensorimotor activities.

As autonomous driving technology advances, the need for drowsiness detection methods that can identify finer levels of drowsiness becomes crucial to enable more accurate interventions and ensure driving safety. This paper presents an on-road driving experiment designed to assess driver drowsiness levels by collecting eye-tracking data. To effectively utilize eye-tracking data for predicting precise levels of driver drowsiness, we propose a semi-channel-independent convolutional neural network, abbreviated as SCICNN. SCICNN comprises two main components: (i) coordinate to features, which extracts gaze features of varying lengths while preserving the correlations between gaze coordinates; and (ii) a semi-channel-independent backbone, which integrates features of uniform length with eyelid opening size into independent channel groups, employing 1D CNN as the backbone within each group. Extensive experiments demonstrate that SCICNN significantly outperforms existing methods and offers a novel perspective on detecting drowsiness levels.

AR head-mounted displays (HMDs) facilitate pedestrian navigation by integrating AR guidance into users' field of view (FOV). Displaying AR guidance using a body-fixed coordinate system has the potential to further leverage this integration by enabling users to control when the guidance appears in their FOV. However, it remains unclear how to effectively position AR guidance within this coordinate system during pedestrian navigation. Therefore, we explored the effects of three AR guidance positions (top, middle, and bottom) within a body-fixed coordinate system on pedestrian navigation in a virtual environment. Our results showed that AR guidance position significantly influenced eye movements, walking behaviors, and subjective evaluations. The top position resulted in the shortest duration of fixations on the guidance compared to the middle and bottom positions, and lower mental demand than the bottom position. The middle position had the smallest rate of vertical eye movement during gaze shifts between the guidance and the environment, and the smallest relative difference in walking speed between fixations on the guidance and the environment compared to the top and bottom positions. The bottom position led to the shortest duration and smallest amplitude of gaze shifts between the guidance and the environment compared to the top and middle positions, and lower frustration than the top position. Based on these findings, we offer design implications for AR guidance positioning within a body-fixed coordinate system during pedestrian navigation.

Assessing the drowsy state of driver is important for driving safety. In this paper, three kinds of information-theoretic tools are utilized to objectively and quantitatively measure the driver's detailed degrees of drowsiness. That is, fixation cross entropy (FCE) defined as the cross entropy between fixation count and duration, stationary gaze entropies based on fixation count (SGEc) and duration (SGEd) respectively, gaze transition entropy (GTE) based on fixation count are exploited to successfully obtain the drowsiness measurements. These measurements are based on eye-tracking data focusing on a novel division approach to area of interest (AOI), particularly emphasizing the significance of gaze deviation from the center point of screen. Psychophysical results showed that all the proposed measures are strongly correlated with Karolinska Sleepiness Scale (KSS), a widely used subjective measure of drowsiness. Particularly, compared with classical indicators for measuring drowsiness, FCE shows a stronger robustness under different drowsy conditions.

Abstract In recent years, the short video industry is booming. However, there are still many difficulties in the action generation of virtual characters. We observed that on the short video social platform, “hand gesture dance” is a very popular short video form. However, its development is limited by the professionalism of choreography. In order to solve these problems, we propose an intelligent choreography framework, which can generate new gesture sequences for unseen audio based on pairing data in the database. Our framework adopts multimodal method and obtains excellent results. In additional, we collected and produced the first and largest pair labeled hand gesture dance data set. Various experiments showed that our results not only generate smooth and rich action sequences, but also collect some semantic information contained in the audio.

Visual scanning plays an important role in sampling visual information from the surrounding environments for a lot of everyday sensorimotor tasks, such as driving. In this paper, we consider the problem of visual scanning mechanism underpinning sensorimotor tasks in 3D dynamic environments. We exploit the use of eye tracking data as a behaviometric, for indicating the visuo-motor behavioral measure in the context of virtual driving. A new metric of visual scanning efficiency (VSE), which is defined as a mathematical divergence between a fixation distribution and a distribution of optical flows induced by fixations, is proposed by making use of a widely-known information theoretic tool, namely the square root of Jensen-Shannon divergence. Psychophysical eye tracking studies, in virtual reality based driving, are conducted to reveal that the new metric of visual scanning efficiency can be employed very well as a proxy evaluation for driving performance. These results suggest that the exploitation of eye tracking data provides an effective behaviometric for sensorimotor activities.

Abstract Fluid simulation plays an important role in movie special effects, computer games, etc. In recent years, the Smoothed Particle Hydrodynamics (SPH) has become a popular fluid simulation method due to its simpler implementation and better processing capabilities for various complex scenes. Although the computational efficiency of the current SPH method has been significantly improved, there is still much room for improvement in pressure solving. This paper proposes an improved pressure solution method based on DFSPH. Firstly, we eliminate the velocity divergence of the fluid by changing the solution order of the nonpressure force and divergence solvers, thereby reducing the number of iterations required by the density solver. Secondly, the SOR method is applied to the pressure solver, furthering reducing the number of iterations of the density solver. Various experiments demonstrated that our method improves the efficiency of the solver with little decrease in stability compared to state‐of‐the‐art methods. The speedup factor reaches between 1.35 and 1.4 at a time step of 5ms, and the more the number of particles, the faster the performance increases.

We propose a method for multi-exposure image fusion based on information-theoretic channel. In the fusion scheme, conditional entropy, as an information measurement of each pixel in one image to the other image, is calculated through an information channel built between two source images, and then weight maps of the source images are generated. Considering the noise caused by blending source images with weight maps directly, we exploit Laplacian pyramid decomposition to avoid unpleasing artifacts. Based on this pyramid scheme, images at every scale are fused by weight maps, and a final fused image is inversely reconstructed. The proposed method is easy to implement and delivers results which are competitive with state-of-art methods.

The sea has been manifested to cause the emotion of fear to people when it comes to a very depth, especially to those who have thalassophobia. Many people have to work in the sea while nearly no research on influence of fear of the sea to cognition has been carried out. This study explores the impact of fear of the sea induced by immersive virtual reality on working memory which is a cognitive system with a limited capacity. Participants were required to complete n-back working memory task of three difficulty levels in the non-emotional environment and the undersea environment respectively by means of virtual reality. Pupil diameter changes were recorded along with the task performance. In addition to reaction times and accuracy (correctly press a button in response to targets) as two task performance indices used in most researches, the commission errors (incorrectly press a button in response to non-targets) and omission errors (incorrectly do not press a button in response to targets) were also differentiated herein. The results of the study indicated that the virtual undersea environment did induce the emotion of fear. As for the task performance, except that the performance of low-level task did not differ much between the two environments, the fear of the sea increased the accuracy of the medium level n-back task but decreased it of high-level n-back task. Result of omission errors was just the opposite and commission errors were increased in both levels of task. The findings, including the positive role of a moderate level of fear of the sea in the performance of working memory task, make a lot of sense for future cognitive work in the sea.

This paper proposes a novel method for image denoising based on the maximum entropy principle. For an image corrupted by Gaussian and impulse noise, impulse noise with high value is detected first using rank statistics, and then removed by a local Gaussian filter. To remove the noise left, an improved self-adaptive non-local filter, with the weights obtained based on the maximum entropy principle, is performed. The experimental results demonstrate that our approach has significant ability to remove any mix of Gaussian and impulse noise in terms of quantitatively image evaluation and qualitatively visual effect.

Undoubtedly, visualization of the trajectory clustering outputs is very important and some researches have been done on visualization of the clustering results. Still importantly, the research on visualizing the procedure of clustering, which is also of great value, is little touched. In this paper, we propose a novel 3D visualization tool, which comprehensively illustrates the Agglomerative Information Bottleneck (AIB) based clustering scheme, to help users understand the clustering approach vividly and clearly. The point of the proposed metaphor makes use of the visualization, together with rich interactions, to demonstrate the iterative clustering procedure, the corresponding results and the clustering results. The experiment demonstrates the effectiveness of our 3D visualization tool for trajectory analysis.

This paper proposes a novel three-dimensional (3D) visualization tool for the trajectory analysis, helping users understand the trajectory data from different perspectives. The details of a single and a set trajectories are well covered in multiscale views by the four main linked windows, namely Traj View, Color Bar, Multi Property and Track Map. We take advantage of and further improve the color bar and the parallel coordinates to effectively present the important attributes of trajectories and also the relationship between different attributes. In addition, the interactive actions, such as keyboard and mouse operations, provide a rich and wonderful user experience.

In this paper, an innovative 3D visualization tool is proposed to facilitate the quickly browsing and understanding of the video sequence for users. Taking advantage of the major windows, our tool presents the multiscale key frames and the video content clearly and effectively. Namely, KF View provides a wonderful navigation of the video key frames with different levels of details. Frame View presents an interesting view of the whole video content. SIM View allows an expressive exploration of the similarities between key frames and also between key frames and the video frames. Importantly, together with many convenient and attractive interactions, this tool is quite efficient to help users grasp the video information soundly.

This paper proposes a shot-based key frame selection technique, aiming at generating a condensed set of frames representing the essential content of a video sequence. Inspired by the successful utilization of Jensen-Shannon Divergence (JSD) and Jensen-Rényi Divergence (JRD) for key frame selection [1] [2], we investigate several popularly accepted f-divergences for calculating the frame-by-frame distance to segment the video clip and then to obtain the key frames. Based on simulation and real test videos, the performances of the key frame selection method by using different versions of f-divergences are systematically analyzed. Extensive experimentation shows that, compared with the methods using JSD and JRD, the new technique is slightly better and computationally faster.

This paper proposes a new approach for trajectory clustering and visualization. During clustering, speed and direction are both taken into account as the important properties of a trajectory. Additionally, the temporal change of the trajectories is explicitly considered in the procedure of clustering. Importantly, we introduce the uncertainty to represent the homogeneity of the updating trajectories based on their speed and direction attributes, to help observe and understand the happening of the trajectory data. Finally, based on the use of the HSV color space, we devise three patterns for graphically presenting the trajectory data in different perspectives. Namely, 'H+S' visualizes the clustered trajectories emphasizing the speed attribute of them. 'H+S+Update' depicts more information on the "latest" trajectory data, compared with the results by 'H+S'. 'H+V+Update' explicitly presents the uncertainty of the trajectory data. Our approach has shown its potential for the effective trajectory analysis.

Recently a strong research effort has been spent on Vehicular Ad-Hoc Networks to evaluate, also by simulations, the effectiveness of protocol solutions. The difficulties lie not only in the definition of realistic simulation environments (including mobility, propagation effects and large number of nodes), but also in the precise comprehension of the results achieved. In fact the outputs hardly allow an extensive understanding of phenomena, due to several aspects such as mobility, position-dependent performances and transitory effects. A custom visualization tool, like the proposed VisMagna, can represent the right solution towards the understanding, debugging and comparison of protocols. VisMagna is here presented for the first time, adapted and applied to the comparison between IEEE 802.11p and MS-Aloha, a slotted solution based on the same physical layer of IEEE 802.11p.

The paper presents a novel three-stage algorithm for very-low-light video denoising and enhancement. The proposed technique invokes twice, in the first and the third stages, the well-known Non-Local Means (NLM) method for spatial and temporal denoising: it is well adapted to the application, leading to the definition of a novel NLM tool. The intermediate stage performs a custom tone adjustment specifically aimed at enlarging the dynamic range of very dark videos. The overall approach transforms very dark videos into more watchable ones, effectively reduces very high noise, and all in all, produces high quality restored image sequences outperforming the recent state-of-the-art results. Additionally, the first and third stages can be combined as a two-step filtering scheme for normal-light videos: the novel denoising solution achieves a heavy noise removal, while reducing motion blur artifacts and preserving image details.

The key frame selection is aimed to extract a set of typical frames that represent the visual content of a video sequence; the graphical presentation of key frames helps understand the video content. This paper proposes a generic method for extracting key frames in which the Jensen-Shannon divergence (JSD) is employed to measure the difference between neighboring video frames to segment a video clip into shots, and to choose key frames in each shot. The novel method for key frame extraction is computationally inexpensive and yet effective, as shown by experimental results. In addition, an innovative 3D visualization approach, in which video key frames and the useful information related to the process of key frame selection are displayed in different levels of detail, is presented as an exploration tool to contribute to a quick and clear understanding of the video content.

Monte Carlo is the only choice of physically correct method to compute the problem of global illumination in the field of realistic image synthesis. Adaptive sampling is an appealing tool to eliminate noise, which is one of the main problems of Monte Carlo based global illumination algorithms. In this paper, we investigate the use of entropy in the domain of information theory to measure pixel quality and to do adaptive sampling. Especially we explore the nonextensive Tsallis entropy, in which a real number q is introduced as the entropic index that presents the degree of nonextensivity, to evaluate pixel quality. By utilizing the least-squares design, an entropic index q can be obtained systematically to run adaptive sampling effectively. Implementation results show that the Tsallis entropy driven adaptive sampling significantly outperforms the existing methods. To our knowledge, this may be the first try on the systematic choice of an appropriate entropic index to Tsallis entropy in the engineering fields.

This paper presents a new two-stage approach to impulse noise removal for color image. The first step is the impulse detection, in which the so-called rank-ordered absolute differences (ROAD) is used to determine whether a pixel is impulsive to be removed. The second step does noise suppression based on maximizing the proposed pixel similarity measure that considers both magnitude and angular information of pixel vectors' difference. Experimental results show that our method can remove impulse noise effectively, and in the meantime can preserve chromaticity and image details very well.

The classic nonlinear filter performs well in impulse noise suppression and edge preserving. However, the classic nonlinear filtering is not good at reducing the mixture of Gaussian noise and impulse noise. In this paper, we investigate the nonlinear filtering techniques to eliminate the mixture of impulse noise and Gaussian noise. Based on fuzzy theory, we present a weighted average filter by making use of the fuzzy membership functions to optimize the weights of the filter. Computational results, which have been obtained from experiments for noise attenuation, indicate that the new algorithm is promising.