Near-eye volumetric displays, showing multiple focal planes, require knowledge of the accurate position of the nodal point of the eye to correctly render a 3D scene. This is because pixels seen through multiple planes must be accurately aligned with the eye’s visual axis to ensure consistency across focal planes. While most eye-tracking methods focus on determining a gaze position within a designated target space, this work aims to track both the eye position and the corresponding gaze direction expressed in coordinates relative to the physical location of the volumetric display planes. To achieve this, we rely on a near-infra-red (NIR) camera image of the pupil and corneal reflections (glints). The existing eye model is used to establish the relationship between the pupil and glint positions in a NIR image and the eye position and rotation in a 3D space. We address the key challenge of robust tracking of the glints in a system that introduces multiple reflections. We also demonstrate that the system reduces the need for recalibration on subsequent uses. Our experiments on a multiple-focal plane display demonstrate that the method can maintain an accurate projection point for volumetric displays.

The natural accommodation of the human eye to different distances results in focus cues, which contribute to depth perception and appearance. Since focus cues are very difficult to reproduce in an electronic display, it is desirable to know how much they contribute to realistic image appearance. In this work we quantify the potential benefit of focus cues in terms of increased realism compared to regular stereo image presentation. As a secondary goal, we evaluate whether three depth-of-field rendering techniques, which reproduce defocus blur at three different degrees of accuracy, can reintroduce the benefits of focus cues. Our findings confirm the importance of focus cues for realistic image appearance, and also show that they cannot easily be substituted by depth-of-field rendering.

With well-established methods for producing photo-realistic results, the next big challenge of graphics and display technologies is to achieve perceptual realism --- producing imagery indistinguishable from real-world 3D scenes. To deliver all necessary visual cues for perceptual realism, we built a High-Dynamic-Range Multi-Focal Stereo Display that achieves high resolution, accurate color, a wide dynamic range, and most depth cues, including binocular presentation and a range of focal depth. The display and associated imaging system have been designed to capture and reproduce a small near-eye three-dimensional object and to allow for a direct comparison between virtual and real scenes. To assess our reproduction of realism and demonstrate the capability of the display and imaging system, we conducted an experiment in which the participants were asked to discriminate between a virtual object and its physical counterpart. Our results indicate that the participants can only detect the discrepancy with a probability of 0.44. With such a level of perceptual realism, our display apparatus can facilitate a range of visual experiments that require the highest fidelity of reproduction while allowing for the full control of the displayed stimuli.