With the rapid growth of digital textual content, users face increasing challenges in rediscovering information which they have encountered in the past. Traditional search engines lack the ability to prioritize content truly viewed by users over merely visible content. We propose a method that captures both text passages and corresponding gaze data while reading, storing them in a searchable knowledge base. We then explore boosting strategies to improve text retrieval by prioritizing passages that users actually viewed, improving the personalization and relevance of search results. To this end, we repurposed the recent g-Rel-READER dataset to evaluate various gaze-based boosting techniques and address the research gap caused by the lack of combined text, gaze, and relevance data. The evaluation demonstrates the potential of gaze data to serve as a boosting criterion for search, with mean average precision (MAP) increased by over 33% over a purely text-based retrieval.

The inherent freedom of exploration in virtual reality poses a challenge for directing user attention toward relevant points or objects of interest, which are potentially located outside the user's field of view or are temporarily obscured. Accordingly, guidance must sustain users' perceptual focus while preserving the sense of presence. This paper presents RippleVision, a subtle gaze guidance technique using wave-like ripples that modulate brightness with inverted polarity between the eyes and appear only within a cone from the point of interest to the gaze location. A calibration study defined detectability and acceptability thresholds for the cues, followed by a comparative user study against four state-of-the-art techniques. Results show RippleVision achieves similar search times to clearly visible techniques while significantly reducing visual dominance. Moreover, its effectiveness scales with cue visibility, improving search performance with minimal impact on perceived visual obstruction and presence.

With the advent of consumer-targeted, low-cost virtual reality devices and facile authoring technologies, the development and design of experiences in virtual reality are also becoming more accessible to non-expert authors. However, the inherent freedom of exploration in these virtual spaces presents a significant challenge for designers seeking to guide user attention toward points and objects of interest. This paper proposes the new technique WarpVision, which utilizes spatial curvature to subtly guide the user's attention in virtual reality. WarpVision distorts an area around the point of interest, thus changing the size, form, and location of all objects and the space around them. In this way, the user's attention can be guided even when the point of interest is not in the immediate field of vision. WarpVision is evaluated in a user study based on a within-subjects design, comparing it to the state-of-the-art technique Deadeye. Participants completed visual search tasks across two virtual environments being supported with WarpVision at four different intensities. Results show that WarpVision significantly reduces search times compared to Deadeye. While both techniques introduce comparable levels of immersion disruption, WarpVision has a lower reported impact on the user's well-being.

Augmented Reality (AR) applications increasingly rely on eye and head gaze tracking for user interaction, with their efficacy influenced by environmental factors such as spatial arrangements and lighting conditions. This paper presents two studies that examine how these variables affect the performance of eye and head gaze tracking in AR environments. While eye tracking partially delivered faster results, its performance exhibited greater variability, especially under dynamic lighting conditions. Conversely, head gaze tracking, while providing more consistent results, showed a notable reduction in accuracy in environments with fluctuating light levels. Furthermore, the spatial properties of the environment had notable implications on both tracking methods. Our research demonstrates that both spatial properties and lighting conditions are key determinants in the choice of a tracking method, underscoring the need for AR systems that can dynamically adapt to these environmental variables.

No abstract available.

Head-mounted displays (HMDs) with integrated eye trackers have opened up a new realm for gaze-contingent rendering. The accurate estimation of gaze depth is essential when modeling the optical capabilities of the eye. Most recently multifocal displays are gaining importance, requiring focus estimates to control displays or lenses. Deriving the gaze depth solely by sampling the scene's depth at the point-of-regard fails for complex or thin objects as eye tracking is suffering from inaccuracies. Gaze depth measures using the eye's vergence only provide an accurate depth estimate for the first meter. In this work, we combine vergence measures and multiple depth measures into feature sets. This data is used to train a regression model to deliver improved estimates. We present a study showing that using multiple features allows for an accurate estimation of the focused depth (MSE<0.1m) over a wide range (first 6m).

In recent years, a variety of methods have been introduced to exploit the decrease in visual acuity of peripheral vision, known as foveated rendering. As more and more computationally involved shading is requested and display resolutions increase, maintaining low latencies is challenging when rendering in a virtual reality context. Here, foveated rendering is a promising approach for reducing the number of shaded samples. However, besides the reduction of the visual acuity, the eye is an optical system, filtering radiance through lenses. The lenses create depth-of-field (DoF) effects when accommodated to objects at varying distances. The central idea of this article is to exploit these effects as a filtering method to conceal rendering artifacts. To showcase the potential of such filters, we present a foveated rendering system, tightly integrated with a gaze-contingent DoF filter. Besides presenting benchmarks of the DoF and rendering pipeline, we carried out a perceptual study, showing that rendering quality is rated almost on par with full rendering when using DoF in our foveated mode, while shaded samples are reduced by more than 69%.

In order to achieve the highest possible performance, the ray traversal and intersection routines at the core of every high-performance ray tracer are usually hand-coded, heavily optimized, and implemented separately for each hardware platform—even though they share most of their algorithmic core. The results are implementations that heavily mix algorithmic aspects with hardware and implementation details, making the code non-portable and difficult to change and maintain.

Advances in computer graphics enable us to create digital images of astonishing complexity and realism. However, processing resources are still a limiting factor. Hence, many costly but desirable aspects of realism are often not accounted for, including global illumination, accurate depth of field and motion blur, spectral effects, etc. especially in real-time rendering. At the same time, there is a strong trend towards more pixels per display due to larger displays, higher pixel densities or larger fields of view. Further observable trends in current display technology include more bits per pixel (high dynamic range, wider color gamut/fidelity), increasing refresh rates (better motion depiction), and an increasing number of displayed views per pixel (stereo, multi-view, all the way to holographic or lightfield displays). These developments cause significant unsolved technical challenges due to aspects such as limited compute power and bandwidth. Fortunately, the human visual system has certain limitations, which mean that providing the highest possible visual quality is not always necessary. In this report, we present the key research and models that exploit the limitations of perception to tackle visual quality and workload alike. Moreover, we present the open problems and promising future research targeting the question of how we can minimize the effort to compute and display only the necessary pixels while still offering a user full visual experience.

Abstract Head‐mounted displays with dense pixel arrays used for virtual reality applications require high frame rates and low latency rendering. This forms a challenging use case for any rendering approach. In addition to its ability of generating realistic images, ray tracing offers a number of distinct advantages, but has been held back mainly by its performance. In this paper, we present an approach that significantly improves image generation performance of ray tracing. This is done by combining foveated rendering based on eye tracking with reprojection rendering using previous frames in order to drastically reduce the number of new image samples per frame. To reproject samples a coarse geometry is reconstructed from a G‐Buffer. Possible errors introduced by this reprojection as well as parts that are critical to the perception are scheduled for resampling. Additionally, a coarse color buffer is used to provide an initial image, refined smoothly by more samples were needed. Evaluations and user tests show that our method achieves real‐time frame rates, while visual differences compared to fully rendered images are hardly perceivable. As a result, we can ray trace non‐trivial static scenes for the Oculus DK2 HMD at 1182 × 1464 per eye within the the VSync limits without perceived visual differences.