Gaze is inherent in both head pointing and walking. When a person performs these tasks simultaneously, there may be detrimental effects on performance. This study examined whether differences in the number of gaze fixations explain why walking worsens pointing performance based on selection time. An 18-person user study compared the number of fixations with selection time under stationary and walking conditions. These conditions were tested using a Fitts’ law head pointing task in virtual reality. Both selection time and fixations per selection increased significantly with task difficulty, modulated by movement condition and pointing task parameters, resulting in a correlation between task performance and gaze behavior. Decoupling head pointing from gaze movement suggested that gaze is less involved in the physical aspect of head pointing. However, there is a contextual advantage of gaze-controlled interactions as its relative stability is particularly useful for precise pointing while walking.

Depth cues, such as binocular disparity, play a critical role in supporting accurate and precise reaching within near space, facilitating reliable interaction with objects and surfaces [27]. Augmented reality (AR) presents unique challenges for such interactions, as mismatched focal distances between real and virtual objects can distort perceived depth. Prior work has examined the perceptual consequences of focal distance mismatches under controlled viewing conditions, as well as AR task performance in more natural settings; here we consider these factors together in a visually guided reaching task with multiple available depth cues for combinations of real and virtual objects. Participants performed a precision visual motor task in an AR environment, placing a virtual ring around either real or virtual posts at varying distances. Our results show that accuracy was degraded when placing a virtual ring on real posts, with fewer successful placements compared to virtual posts. We demonstrate that this increase in error was due to a systematic underestimation of virtual object distances relative to their physical counterparts. These findings indicate that distortions in perceived depth between real and virtual objects persist during active reaching, despite the availability of multiple depth cues including binocular disparity, proprioception, occlusion, and motion parallax. Measuring and correcting for these distortions is important for optimizing the effectiveness and usability of AR devices.

The widespread use of package managers like Maven has accelerated software development but has also introduced significant security risks due to vulnerabilities in dependencies. In this study, we analyze the prevalence and impact of vulnerabilities within the Maven Central ecosystem, using Common Vulnerabilities and Exposures (CVE) data from OSV.dev and a subsample enriched with aggregated CVE data (CVE_AGGREGATED), which captures both direct and transitive vulnerabilities. In our subsample of around 4 million releases, we found that while only about $1 \%$ of releases have direct vulnerabilities, approximately $46.8 \%$ are affected by transitive vulnerabilities. This highlights how a small number of vulnerable yet influential artifacts can impact a vast portion of the ecosystem. Moreover, our analysis shows that vulnerabilities propagate rapidly through dependency networks and that more central artifacts (those with a high number of dependents) are not necessarily less vulnerable. We also observed that the time taken to patch vulnerabilities, including those of high or critical severity, often spans several years. Additionally, we found that dependents of artifacts tend to prefer presumably non-vulnerable versions; however, some continue to use vulnerable versions, indicating challenges in adopting patched releases. These findings highlight the critical need for improved dependency management practices and timely vulnerability remediation to enhance the security of software ecosystems.

Binocular disparity provides metric depth information, while monocular cues like occlusion offer depth order. In augmented reality (AR), conflicts between these cues can occur when virtual objects fail to be occluded by real-world surfaces, creating a depth cue conflict and subsequently impacting depth perception. The integration of occlusion and binocular disparity was investigated under this cue conflict in AR using distance-matching paradigms. These paradigms were applied within reach space (0.35–0.5 m) and beyond reach space (0.9–1.5 m). Observers matched the distance of a virtual letter ’A,’ superimposed on a physical surface, using a virtual probe. In addition to the probe, manual reach responses were also made with the index finger for the within reach space condition. Results revealed consistent underestimation of the letter’s distance when it was rendered beyond the surface, with errors proportional to distance. These biases persisted even when proprioceptive information was available, highlighting the robust influence of occlusion cue conflicts on perceived depth. Thus, designers must carefully plan and position virtual augmentations to avoid such errors and their impact on user interaction.

Virtual reality (VR) often induces perceptual distortions, for instance, underestimates of perceived distance which may distort perceived size. However, other depth cues are available in these immersive environments that may help users perceive size accurately. Here, we examined how binocular and monocular depth cues, as well as object interaction, influenced the perceived size of 3D shapes in VR. The results showed that binocular cues play an essential role in size judgements, whereas dynamic monocular cues, such as motion parallax, did not fully compensate for their absence, underscoring the importance of rendering high-quality binocular cues for fine size discrimination tasks in virtual environments.

When we move during virtual reality (VR) display lag produces Differences in our Virtual and Physical head pose (DVP). Research suggests that DVP can be used to predict cybersickness during head-mounted display (HMD) based VR. However, these studies always had participants make unusual (continuous oscillatory) head-movements. This study examined whether DVP also predicts cybersickness during more typical VR conditions. After assessing their susceptibility to real-world motion sickness (using the MSSQ-Revised), 67 participants repeatedly moved their heads to “target” objects that appeared inside a virtual room (under different experimentally imposed display lags). We found that cybersickness was more likely and severe when: (1) participants had higher MSSQ scores; (2) the spatial magnitudes and the detrended fluctuation analysis α values of their DVP increased. Based on these findings we believe that real-time estimates of the DVP could be used to warn users about the imminent onset of sickness during consumer HMD VR.

When HMD users move their heads in virtual reality (VR), display lag creates differences between their virtual and physical head pose (DVP). This study examined whether objective estimates of DVP could predict experiences of cybersickness during simulations with three different types of added lag: (1) Constant lag (where the display was always delayed by 250 ms); (2) Predictable time-varying lag (where delays alternated between 0 and 250 ms every 5 s); and (3) Random time-varying lag (where delays alternated between 0 and a randomly determined value, up to 250 ms, every 1–5 s). Constant, Predictable, and Random added lag were found to generate similar levels of cybersickness—with all three conditions producing more severe sickness than the Baseline lag control. Consistent with our DVP hypothesis, the spatial magnitude and temporal dynamics of our participants’ DVP were both found to be reliable predictors of their cybersickness in all display lag conditions tested.

Stereoscopic displays present similar but different images/videos separately to the left and right eyes. Here we report the results of a subjective assessment of computer-generated high dynamic range (HDR) stereoscopic video compression targeting lightweight, low-latency visually lossless performance. The ISO/IEC 29170–2 non-flicker paradigm (Annex A) was adapted for stereoscopic videos and used to evaluate the VESA DSC 1.2a and VDC-M 1.2.2 codecs operating at and beyond their recommended maximum compression levels. Both codecs exhibited visually lossless performance at their target compressions levels. There were differences in compression artifact detection rates with some image sequences between stereoscopic and 2D presentation although the direction of the difference was inconsistent. The results confirm the effectiveness of these codecs for Stereoscopic 3D (S3D) videos and suggest that observers are relatively insensitive to compression artifacts in moving stereoscopic imagery.

Virtual reality (VR) is distinguished by the rich, multimodal, im-mersive sensory information and affordances provided to the user. However, when moving about an immersive virtual world the vi-sual display often conflicts with other sensory cues due to design, the nature of the simulation, or to system limitations (for example impoverished vestibular motion cues during acceleration in racing games). Given that conflicts between sensory cues have been as-sociated with disorientation or discomfort, and theoretically could distort spatial perception, it is important that we understand how and when they are manifested in the user experience. To this end, this set of experiments investigates the impact of mismatch between physical and virtual motion parallax on the per-ception of the depth of an apparently perpendicular dihedral angle (a fold) and its distance. We applied gain distortions between visual and kinesthetic head motion during lateral sway movements and measured the effect of gain on depth, distance and lateral space compression. We found that under monocular viewing, observers made smaller object depth and distance settings especially when the gain was greater than 1. Estimates of target distance declined with increasing gain under monocular viewing. Similarly, mean set depth decreased with increasing gain under monocular viewing, except at 6.0 m. The effect of gain was minimal when observers viewed the stimulus binocularly. Further, binocular viewing (stereopsis) improved the precision but not necessarily the accuracy of gain perception. Overall, the lateral compression of space was similar in the stereoscopic and monocular test conditions. Taken together, our results show that the use of large presentation distances (at 6 m) combined with binocular cues to depth and distance enhanced humans' tolerance to visual and kinesthetic mismatch.

Stereoscopic AR and VR headsets have displays and lenses that are either fixed or adjustable to match a limited range of user inter-pupillary distances (IPDs). Projective geometry predicts a misperception of depth when either the displays or virtual cameras used to render images are misaligned with the eyes. However, misalignment between the eyes and lenses might also affect binocular convergence, which could further distort perceived depth. This possibility has been largely ignored in previous studies. Here, we evaluated this phenomenon in a VR headset in which the inter-lens and inter-axial camera separations are coupled and adjustable. In a baseline condition, both were matched to observers' IPDs. In two other conditions, the inter-lens and inter-axial camera separations were set to the maximum and minimum allowed by the headset. In each condition, observers were instructed to adjust a fold created by two intersecting, textured surfaces until it appeared to have an angle of 90°. The task was performed at three randomly interleaved viewing distances, monocularly and binocularly. In the monocular condition, observers underestimated the fold angle and there was no effect of viewing distance on their settings. In the binocular conditions, we found that when the lens and camera separation were less than the viewer's IPD, they exhibited compression of perceived slant relative to baseline. The reverse pattern was seen when the lens and camera separation were larger than the viewer's IPD. These results were well explained by a geometric model that considers shifts in convergence due to lens and display misalignment with the eyes, as well as the relative contribution of monocular cues.

ICECUBE LED Display [ILDm^3] is a cubic-meter, 1/1000th scale model of the IceCube Neutrino Observatory, a novel telescope that looks for nearly invisible cosmic messengers, neutrinos, using a cubic-kilometer of instrumented ice starting 1450 meters below the surface at the South Pole. The display uses art methodologies as a means for expressing imperceptible astrophysical events as sound, light and colour in the domain of the human sensorium. The experience is as aesthetically critical as it is facilitatory to an intuitive understanding of subatomic astrophysical data, leading to new ways of knowing about our Universe and its processes.

Advances in Virtual Reality technology have enabled physical walking in virtual environments. While most Virtual Reality systems render stereoscopic images to users, the implication of binocular viewing with respect to the performance of human walking in virtual environments remains largely unknown. In the present study, we conducted two walking experiments in virtual environments using a linear treadmill and a novel projected display known as the Wide Immersive Stereo Environment (WISE) to study the role of binocular viewing in virtual locomotion. The first experiment investigated the walking performance of people stepping over obstacles while the second experiment focused on a scenario on stepping over gaps. Both experiments were conducted under both stereoscopic viewing and non-stereoscopic viewing conditions. By analysing the gait parameters, we found that binocular viewing helped people to make more accurate movements to step over obstacles and gaps in virtual locomotion.

Often in 3D games and virtual reality, changes in fixation occur during locomotion or other simulated head movements. We investigated whether a constant camera rotation in a virtual scene modulates saccadic suppression. The users viewed 3D scenes from the vantage point of a virtual camera which was either stationary or rotated at a constant rate about a vertical axis (camera pan) or horizontal axis (camera tilt). During this motion, observers fixated an object that was suddenly displaced horizontally/vertically in the scene, triggering them to produce a saccade. During the saccade an additional sudden movement was applied to the virtual camera. We estimated discrimination thresholds for these transsaccadic camera shifts using a Bayesian adaptive procedure. With an ongoing camera pan, we found higher thresholds (less noticeability) for additional sudden horizontal camera motion. Likewise, during simulated vertical head movements (i.e. a camera tilt), vertical transsaccadic image displacements were better hidden from the users for both horizontal and vertical saccades. Understanding the effect of continuous movement on the visibility of a sudden transsaccadic change can help optimize the visual performance of gaze-contingent displays and improve user experience.

The magnifying optics of virtual reality (VR) head-mounted displays (HMD) often cause undesirable pincushion distortion in the displayed imagery. Eccentrically increasing magnification radially displaces image-points away from the optical axis, causing straight lines to curve outwards. This, in turn, should affect the 3D perception of surface shape by warping binocular and monocular depth cues. Previous research has shown that distortion-induced biases in perceived slant do occur in static images. However, most use cases in VR involve moving images. Here we evaluate the impact of motion on biases in perceived slant. An HMD was used to present flat, textured surfaces that varied in slant and were either stationary, or translated laterally by the observer. In separate studies we varied the degree of distortion and evaluated the impact on perceived slant at several locations along the surface. We found that, irrespective of whether the surface was moving or stationary, distortion introduced significant bias into local slant estimates. The pattern of results is consistent with the surface appearing to be concave (as if viewing the inside surface of a bowl), as predicted from the warping of binocular and monocular cues. Importantly, the intermediate distortion level produced the same, but weaker, pattern of biases seen in the fully-distorted condition. When an appropriate level of pre-warping was applied, slant perception was veridical. Overall, our results highlight the importance of sufficiently correcting for optical distortions in VR HMDs to enable veridical perception of surface attitude.

In stereoscopic displays different images are presented separately to the left and right eyes. This requirement may increase the bandwidth demand as well as increase the occurrence of visible compression-related artefacts. Here we report the results of a large-scale subjective assessment of high dynamic range (HDR) stereoscopic image compression. The ISO/IEC 29170-2 flicker paradigm was adapted for stereoscopic images and used to evaluate two VESA (Video Electronics Standards Association) image compression codecs: DSC 1.2a and VDCM 1.2.2. We compared the performance on stereoscopic images versus 2D images for both codecs.

This paper describes validity testing of the NeuLog NUL-217 GSR measurement device. This was accomplished by comparing the NeuLog device to readings from the Biopac Student Lab Systems EDA system. The results of this research found that measurements from the NeuLog device are correlated with and comparable to the Biopac system. The absolute skin conductance levels typically differed between the two systems. For most psychological and human factors research the dynamic skin conductance responses are more important than absolute levels. The timing and relative magnitude of changes typically tracked well across the two systems indicating that the Neulog system is suitable for the purposes of its intended use in psychological and technological research.

Virtual environments can replicate the visual appearance of terrain conditions, but movements involved in using the interfaces confer their own bodily sensations, which can be incongruent with the visual presentation. Assuming that more natural interfaces produce more natural locomotor behaviors, we propose a framework for assessing the quality of a locomotion interface. Using this framework, we studied the interaction of different locomotion interfaces with visual information on wayfinding decisions in a virtual environment. We compared decisions made using a dual joystick gamepad with a walking-in-place metaphor. Paths presented on a given trial differed visually in one of the following aspects: (a) slope, (b) friction, (c) texture, and (d) width. In this experiment, choices made with the walking-in-place interface more closely matched visual conditions which would minimize energy expenditure or physical risk in the natural world. We provide some observations that would further validate this approach and improve this method in future implementations. This approach provides a way of both studying factors in perceptual decision making and demonstrates the effect of interface on natural behavior.

In gaze-contingent displays, the viewer’s eye movement data are processed in real-time to adjust the graphical content. To provide a high-quality user experience, these graphical updates must occur with minimum delay. Such updates can be used to introduce imperceptible changes in virtual camera pose in applications such as networked gaming, collaborative virtual reality and redirected walking. For such applications, perceptual saccadic suppression can help to hide the graphical artifacts. We investigated whether the visibility of these updates depends on the type of image transformation. Users viewed 3D scenes in which the displacement of a target object triggered them to generate a vertical or horizontal saccade, during which a translation or rotation was applied to the virtual camera used to render the scene. After each trial, users indicated the direction of the scene change in a forced-choice task. Results show that type and size of the image transformation affected change detectability. During horizontal or vertical saccades, rotations along the roll axis were the most detectable, while horizontal and vertical translations were least noticed. We confirm that large 3D adjustments to the scene viewpoint can be introduced unobtrusively and with low latency during saccades, but the allowable extent of the correction varies with the transformation applied.

The saccadic suppression effect, in which visual sensitivity is reduced significantly during saccades, has been suggested as a mechanism for masking graphic updates in a 3D virtual environment. In this study, we investigate whether the degree of saccadic suppression depends on the type of image change, particularly between different natural 3D scene transformations. The user observed 3D scenes and made a horizontal saccade in response to the displacement of a target object in the scene. During this saccade the entire scene translated or rotated. We studied six directions of transformation corresponding to the canonical directions for the six degrees of freedom. Following each trial, the user made a forced-choice indication of direction of the scene change. Results show that during horizontal saccades, the most recognizable changes were rotations along the roll axis.

Recent research has proposed teleoperation of robotic and aerial vehicles using head motion tracked by a head-mounted display (HMD). First-person views of the vehicles are usually captured by onboard cameras and presented to users through the display panels of HMDs. This provides users with a direct, immersive and intuitive interface for viewing and control. However, a typically overlooked factor in such designs is the latency introduced by the vehicle dynamics. As head motion is coupled with visual updates in such applications, visual and control latency always exists between the issue of control commands by head movements and the visual feedback received at the completion of the attitude adjustment. This causes a discrepancy between the intended motion, the vestibular cue and the visual cue and may potentially result in simulator sickness. No research has been conducted on how various levels of visual and control latency introduced by dynamics in robots or aerial vehicles affect users' performance and the degree of simulator sickness elicited. Thus, it is uncertain how much performance is degraded by latency and whether such designs are comfortable from the perspective of users. To address these issues, we studied a prototyped scenario of a head motion controlled quadcopter using an HMD. We present a virtual reality (VR) paradigm to systematically assess the effects of visual and control latency in simulated drone control scenarios.

Head gesture is a natural means of face-to-face communication between people but the recognition of head gestures in the context of virtual reality and use of head gesture as an interface for interacting with virtual avatars and virtual environments have been rarely investigated. In the current study, we present an approach for real-time head gesture recognition on head-mounted displays using Cascaded Hidden Markov Models. We conducted two experiments to evaluate our proposed approach. In experiment 1, we trained the Cascaded Hidden Markov Models and assessed the offline classification performance using collected head motion data. In experiment 2, we characterized the real-time performance of the approach by estimating the latency to recognize a head gesture with recorded real-time classification data. Our results show that the proposed approach is effective in recognizing head gestures. The method can be integrated into a virtual reality system as a head gesture interface for interacting with virtual worlds.

We present a method for estimating the Motion-to-Photon (End-to-End) latency of head mounted displays (HMDs). The specific HMD evaluated in our study was the Oculus Rift DK2, but the procedure is general. We mounted the HMD on a pendulum to introduce damped sinusoidal motion to the HMD during the pendulum swing. The latency was estimated by calculating the phase shift between the captured signals of the physical motion of the HMD and a motion-dependent gradient stimulus rendered on the display. We used the proposed method to estimate both rotational and translational Motion-to-Photon latencies of the Oculus Rift DK2.

Virtual reality simulations of group social interactions are important for many applications, including the virtual treatment of social phobias, crowd and group simulation, collaborative virtual environments (VEs), and entertainment. In such scenarios, when compared to the real world, audio cues are often impoverished. As a result, users cannot rely on subtle spatial audio-visual cues that guide attention and enable effective social interactions in real-world situations. We explored whether gaze-contingent audio enhancement techniques driven by inferring audio-visual attention in virtual displays could be used to enable effective communication in cluttered audio VEs. In all of our experiments, we hypothesized that visual attention could be used as a tool to modulate the quality and intensity of sounds from multiple sources to efficiently and naturally select spatial sound sources. For this purpose, we built a gaze-contingent display (GCD) that allowed tracking of a user’s gaze in real-time and modifying the volume of the speakers’ voices contingent on the current region of overt attention. We compared six different techniques for sound modulation with a base condition providing no attentional modulation of sound. The techniques were compared in terms of source recognition and preference in a set of user studies. Overall, we observed that users liked the ability to control the sounds with their eyes. They felt that a rapid change in attenuation with attention but not the elimination of competing sounds (partial rather than absolute selection) was most natural. In conclusion, audio GCDs offer potential for simulating rich, natural social, and other interactions in VEs. They should be considered for improving both performance and fidelity in applications related to social behaviour scenarios or when the user needs to work with multiple audio sources of information.

There is growing interest in capturing and projecting movies at higher frame rates than the traditional 24 frames per second. Yet there has been little scientific assessment of the impact of higher frame rates (HFR) on the perceived quality of cinema content. Here we investigated the effect of frame rate, and associated variables (shutter angle and camera motion) on viewers' ability to discriminate letters in S3D movie clips captured by a professional film crew. The footage was filmed and projected at varying combinations of frame rate, camera speed and shutter angle. Our results showed that, overall, legibility improved with increased frame rate and reduced camera velocity. However, contrary to expectations, there was little effect of shutter angle on legibility. We also show that specific combinations of camera parameters can lead to dramatic reductions in legibility for localized regions in a scene.

An intriguing aspect of picture perception is the viewer’s tolerance to variation in viewing position, perspective, and display size. These factors are also present in stereoscopic media, where there are additional parameters associated with the camera arrangement (e.g., separation, orientation). The predicted amount of depth from disparity can be obtained trigonometrically; however, perceived depth in complex scenes often differs from geometric predictions based on binocular disparity alone. To evaluate the extent and the cause of deviations from geometric predictions of depth from disparity in naturalistic scenes, we recorded stereoscopic footage of an indoor scene with a range of camera separations (camera interaxial (IA) ranged from 3 to 95 mm) and displayed them on a range of screen sizes. In a series of experiments participants estimated 3D distances in the scene relative to a reference scene, compared depth between shots with different parameters, or reproduced the depth between pairs of objects in the scene using reaching or blind walking. The effects of IA and screen size were consistently and markedly smaller than predicted from the binocular viewing geometry, suggesting that observers are able to compensate for the predicted distortions. We conclude that the presence of multiple realistic monocular depth cues drives normalization of perceived depth from binocular disparity. It is not clear to what extent these differences are due to cognitive as opposed to perceptual factors. However, it is notable that these normalization processes are not task specific; they are evident in both perception- and action-oriented tasks.

High frame-rate (HFR) movie-making refers to the capture and projection of movies at frame rates several times higher than the traditional 24 frames per second. This higher frame rate theoretically improves the quality of motion portrayed in movies, and helps avoid motion blur, judder, and other undesirable artifacts. However, there is considerable debate in the cinema industry regarding the acceptance of HFR content given anecdotal reports of hyper-realistic imagery that reveals too much set and costume detail. Despite the potential theoretical advantages, there has been little empirical investigation of the impact of high frame-rate techniques on the viewer experience. In this study, we use stereoscopic 3D content, filmed and projected at multiple frame rates (24, 48, and 60 fps), with shutter angles ranging from 180° to 358°, to evaluate viewer preferences. In a paired-comparison paradigm, we assessed preferences along a set of five attributes (realism, motion smoothness, blur/clarity, quality of depth, and overall preference). The resulting data show a clear preference for higher frame rates, particularly when contrasting 24 fps with 48 or 60 fps. We found little impact of shutter angle on viewers' choices, with the exception of one measure (motion smoothness) for one clip type. These data are the first empirical evidence of the advantages afforded by high frame-rate capture and presentation in a cinema context.

Computer-generated objects presented on a display typically have the same focal distance regardless of the monocular and binocular depth cues used to portray a 3D scene. This is because they are presented on a flat screen display that has a fixed physical location. In a stereoscopic 3D display, accommodation (focus) of the eyes should always be at the distance of the screen for clear vision regardless of the depth portrayed; this fixed accommodation conflicts with vergence eye movements that the user must make to fuse stimuli located off the screen. This is known as accommodation-vergence conflict and is detrimental for user experience of stereoscopic virtual environments (VE), as it can cause visual discomfort and diplopia during use of a stereoscopic display. It is believed that, by artificially simulating focal blur and natural accommodation, it is possible to compensate for the vergence-accommodation conflict and alleviate these symptoms. We hypothesized that it is possible to compensate for conflict with a fixed accommodation cue by adding simulated focal blur according to instantaneous fixation.

Stereoscopic 3D media has recently increased in appreciation and availability. This popularity has led to concerns over the health effects of habitual viewing of stereoscopic 3D content; concerns that are largely hypothetical. Here we examine the effects of repeated, long-term exposure to stereoscopic 3D in the workplace on several measures of stereoscopic sensitivity (discrimination, depth matching, and fusion limits) along with reported negative symptoms associated with viewing stereoscopic 3D. We recruited a group of adult stereoscopic 3D industry experts and compared their performance with observers who were (i) inexperienced with stereoscopic 3D, (ii) researchers who study stereopsis, and (iii) vision researchers with little or no experimental stereoscopic experience. Unexpectedly, we found very little difference between the four groups on all but the depth discrimination task, and the differences that did occur appear to reflect task-specific training or experience. Thus, we found no positive or negative consequences of repeated and extended exposure to stereoscopic 3D in these populations.

The perception of synchronous, intelligible, speech is fundamental to a high-quality modern cinema experience. Surprisingly, this issue has remained relatively unexplored in stereoscopic 3D (S3D) media, despite its increasing popularity. Instead, visual parameters have been the primary focus of concern for those who create, and those who study the impact of, S3D content. In the work presented here we ask if ability to integrate audio and visual information is influenced by adding the third dimension to film. We also investigate the effects of known visual parameters (horizontal and vertical parallax), on audio-visual integration. To this end, we use an illusion of speech processing known as the McGurk effect as an objective measure of multi-modal integration. In the classic (2D) version of this phenomenon, discrepant auditory (/ba/) and visual (/ga/) information typically results in the perception of a unique 'fusion' syllable (e.g. /da/). We extended this paradigm to measure the McGurk effect in a small theatre. We varied the horizontal (IA: 0, 6, 12, 18, 24 mm) and vertical (0°, 0.5°, 0.75°, 1°) parallax from trial-to-trial and asked observers to report their percept of the phoneme. Our results show a consistently high proportion of the expected fusion responses, with no effect of horizontal or vertical offsets. These data are the first to show that the McGurk effect extends to stereoscopic stimuli and is not a phenomenon isolated to 2D media perception. Furthermore, the results show that audiences can tolerate a high level of both horizontal and vertical disparity and maintain veridical speech perception. We consider these results in terms of current stereoscopic filmmaking recommendations and practices.

Many materials, including water surfaces, jewels, and glassware exhibit transparent refractions. The human visual system can somehow recover 3D shape from refracted images. While previous research has elucidated various visual cues that can facilitate visual perception of transparent objects, most of them focused on monocular material perception. The question of shape perception of transparent objects is much more complex and few studies have been undertaken, particular in terms of binocular vision. In this article, we first design a system for stereoscopic surface orientation estimation with photo-realistic stimuli. It displays pre-rendered stereoscopic images and a real-time S3D (Stereoscopic 3D) shape probe simultaneously. Then we estimate people's perception of the shape of thin transparent objects using a gauge figure task. Our results suggest that people can consistently perceive the surface orientation of thin transparent objects, and stereoscopic viewing improves the precision of estimates. To explain the results, we present an edge-aware orientation map based on image gradients and structure tensors to illustrate the orientation information in images. We also decomposed the normal direction of the surface into azimuth angle and slant angle to explain why additional depth information can improve the accuracy of perceived normal direction.

Superimposed luminance noise is typical of imagery from devices used for low-light vision such as image intensifiers (i.e., night vision devices). In four experiments, we measured the ability to detect and discriminate motion-defined forms as a function of stimulus signal-to-noise ratio at a variety of stimulus speeds. For each trial, observers were shown a pair of image sequences - one containing dots in a central motion-defined target region that moves coherently against the surrounding dots, which moved in the opposite or in random directions, while the other sequence had the same random/uniform motion in both the center and surrounding parts. They indicated which interval contained the target stimulus in a two-interval forced-choice procedure. In the first experiment, simulated night vision images were presented with Poisson-distributed spatiotemporal image noise added to both the target and surrounding regions of the display. As the power of spatiotemporal noise was increased, it became harder for observers to detect the target, particularly at the lowest and highest dot speeds. The second experiment confirmed that these effects also occurred with low illumination in real night vision device imagery, a situation that produces similar image noise. The third experiment demonstrated that these effects generalized to Gaussian noise distributions and noise created by spatiotemporal decorrelation. In the fourth experiment, we found similar speed-dependent effects of luminance noise for the discrimination (as opposed to detection) of the shape of a motion-defined form. The results are discussed in terms of physiological motion processing and for the usability of enhanced vision displays under noisy conditions.

In 1958, JJ Gibson put forward proposals on the visual control of locomotion. Research in the last 50 years has served to clarify the sources of visual and nonvisual information that contribute to successful steering, but has yet to determine how this information is optimally combined under conditions of uncertainty. Here, we test the conditions under which a locomotor robot with a mobile camera can steer effectively using simple visual and extra-retinal parameters to examine how such models cope with the noisy real-world visual and motor estimates that are available to humans. This applied modeling gives us an insight into both the advantages and limitations of using active gaze to sample information when steering.

The visual field is the area of space that can be seen when an observer fixates a given point. Many visual capabilities vary with position in the visual field and many diseases result in changes in the visual field. With current technology, it is possible to build very complex real-time visual field simulations that employ gaze-contingent displays. Nevertheless, there are still no established techniques to evaluate such systems. We have developed a method to evaluate a system's contingency by employing visual blind spot localization as well as foveal fixation. During the experiment, gaze-contingent and static conditions were compared. There was a strong correlation between predicted results and gaze-contingent trials. This evaluation method can also be used with patient populations and for the evaluation of gaze-contingent display systems, when there is need to evaluate a visual field outside of the foveal region.

We examined the eye movements of pilots as they carried out simulated aircraft landings under day and night lighting conditions. Our five students and five certified pilots were instructed to quickly achieve and then maintain a constant 3-degree glideslope relative to the runway. However, both groups of pilots were found to make significant glideslope control errors, especially during simulated night approaches. We found that pilot gaze was directed most often toward the runway and to the ground region located immediately in front of the runway, compared to other visual scene features. In general, their gaze was skewed toward the near half of the runway and tended to follow the runway threshold as it moved on the screen. Contrary to expectations, pilot gaze was not consistently directed at the aircraft's simulated aimpoint (i.e., its predicted future touchdown point based on scene motion). However, pilots did tend to fly the aircraft so that this point was aligned with the runway threshold. We conclude that the supplementary out-of-cockpit visual cues available during day landing conditions facilitated glideslope control performance. The available evidence suggests that these supplementary visual cues are acquired through peripheral vision, without the need for active fixation.

Effective management and treatment of glaucoma and other visual diseases depend on early diagnosis. However, early symptoms of glaucoma often go unnoticed until a significant portion of the visual field is lost. The ability to simulate the visual consequences of the disease offers potential benefits for patients and clinical education as well as for public awareness of its signs and symptoms. Experiments using simulated visual field defects could identify changes in behaviour, for example during driving, that one uses to compensate at the early stages of the disease's development. Furthermore, by understanding how visual field defects affect performance of visual tasks, we can help develop new strategies to cope with other devastating diseases such as macular degeneration. A Gaze-Contingent Display (GCD) system was developed to simulate an arbitrary visual field in a virtual environment. The system can estimate real-time gaze direction and eye position in earth-fixed coordinates during relatively large head movement, and thus it can be used in immersive projection based VE systems like the CAVE™. Arbitrary visual fields are simulated via OpenGL and Shading Language capabilities and techniques that are supported by the GPU, thus enabling fast performance in real time. In order to simulate realistic visual defects, the system performs multiple image processing operations including change in acuity, brightness, color, glare and image distortion. The final component of the system simulates different virtual scenes that the participant can navigate through and explore. As a result, this system creates an experimental environment to study the effects of low vision on everyday tasks such as driving and navigation.

Application designers of collaborative distributed virtual environments must account for the influence of the network connection and its detrimental effects on user performance. Based upon analysis and classification of existing latency compensation techniques, this paper introduces a novel approach to latency amelioration in the form of a two-tier predictor-estimator framework. The technique is variability-aware due to its proactive sender-side prediction of a pose a variable time into the future. The prediction interval required is estimated based on current and past network delay characteristics. This latency estimate is subsequently used by a Kalman filter-based predictor to replace the measurement event with a predicted pose that matches the event's arrival time at the receiving workstation. The compensation technique was evaluated in a simulation through an offline playback of real head motion data and network delay traces collected under a variety of real network conditions. The experimental results indicate that the variability-aware approach significantly outperforms a state-of-the-art one, which assumes a constant system delay

Improving the sense of “presence” is a common goal of three-dimensional (3D) display technology for film, television, and virtual reality. However, there are instances in which 3D presentations may elicit unanticipated negative responses. For example, it is well established that violations of interpersonal space cause discomfort in real-world situations. Here we ask if people respond similarly when viewing life-sized stereoscopic images. Observers rated their level of comfort in response to animate and inanimate objects in live and virtual (stereoscopic projection) viewing conditions. Electrodermal activity was also recorded to monitor their physiological response to these stimuli. Observers exhibited significant negative reactions to violations of interpersonal space in stereoscopic 3D displays, which were equivalent to those experienced in the natural environment. These data have important implications for the creation of 3D media and the use of virtual reality systems.

In interactive graphics it is often necessary to introduce large changes in the image in response to updated information about the state of the system. Updating the local state immediately would lead to a sudden transient change in the image, which could be perceptually disruptive. However, introducing the correction gradually using smoothing operations increases latency and degrades precision. It would be beneficial to be able to introduce graphic updates immediately if they were not perceptible. In the paper the use of saccade-contingent updates is exploited to hide graphic updates during the period of visual suppression that accompanies a rapid, or saccadic, eye movement. Sensitivity to many visual stimuli is known to be reduced during a change in fixation compared to when the eye is still. For example, motion of a small object is harder to detect during a rapid eye movement (saccade) than during a fixation. To evaluate if these findings generalize to large scene changes in a virtual environment, gaze behavior in a 180 degree hemispherical display was recorded and analyzed. This data was used to develop a saccade detection algorithm adapted to virtual environments. The detectability of trans-saccadic scene changes was evaluated using images of high resolution real world scenes. The images were translated by 0.4, 0.8 or 1.2 degrees of visual angle during horizontal saccades. The scene updates were rarely noticeable for saccades with a duration greater than 58 ms. The detection rate for the smallest translation was just 6.25%. Qualitatively, even when trans-saccadic scene changes were detectible, they were much less disturbing than equivalent changes in the absence of a saccade.

Virtual reality displays introduce spatial distortions that are very hard to correct because of the difficulty of precisely modelling the camera from the nodal point of each eye. How significant are these distortions for spatial perception in virtual reality? In this study, we used a helmet-mounted display and a mechanical head tracker to investigate the tolerance to errors between head motions and the resulting visual display. The relationship between the head movement and the associated updating of the visual display was adjusted by subjects until the image was judged as stable relative to the world. Both rotational and translational movements were tested, and the relationship between the movements and the direction of gravity was varied systematically. Typically, for the display to be judged as stable, subjects needed the visual world to be moved in the opposite direction to the head movement by an amount greater than the head movement itself, during both rotational and translational head movements, although a large range of movement was tolerated and judged as appearing stable. These results suggest that it not necessary to model the visual geometry accurately and suggest circumstances when tracker drift can be corrected by jumps in the display which will pass unnoticed by the user.

To enhance presence, facilitate sensory motor performance, and avoid disorientation or nausea, virtual-reality applications require the perception of a stable environment. End-end tracking latency (display lag) degrades this illusion of stability and has been identified as a major fault of existing virtual-environment systems. Oscillopsia refers to the perception that the visual world appears to swim about or oscillate in space and is a manifestation of this loss of perceptual stability of the environment. The effects of end-end latency and head velocity on perceptual stability in a virtual environment were investigated psychophysically. Subjects became significantly more likely to report oscillopsia during head movements when end-end latency or head velocity were increased. It is concluded that perceptual instability of the world arises with increased head motion and increased display lag. Oscillopsia is expected to be more apparent in tasks requiring real locomotion or rapid head movement.

Although technologies such as head-mounted displays and CAVEs can be used to provide large immersive visual displays within small physical spaces, it is difficult to provide virtual environments which are as large physically as they are visually. A fundamental problem is that tracking technologies which work well in a small enclosed environment do not function well over longer distances. In this paper, we describe Trike-a 'rideable' computer system which can be used to generate and explore large virtual spaces both visually and physically. This paper describes the hardware and software components of the system and a set of experiments which have been performed to investigate how the different perceptual cues that can be provided with Trike interact within an immersive environment.