As virtual reality (VR) headsets become more comfortable and accessible, their growing use in high-stakes, time-critical settings raises concerns about fatigue. Fatigue impairs perceptual and cognitive functioning. It reduces oculomotor accuracy and visual focus, and can lead to an early decline in depth estimation performance. By augmenting the visual presentation with depth information cues, adaptive designs can reduce fatigue-related depth perception errors, enhancing safety and task effectiveness. Explicit cues present depth information directly through text or color, while subtle cues adjust scene properties, such as depth-dependent blur, to convey depth information implicitly without drawing overt attention. We examined how fatigue interacts with different cues in a 27-hour within-subject protocol. Across six overnight sessions (20:00– 07:00), twenty-three participants completed a VR depth perception task at varying fatigue levels and under four cue conditions: no cue (baseline), text, color, and blur. Over the night, vigilance declined, sleepiness and mental effort increased, and simulator sickness rose before stabilizing, independent of cue condition. All augmented cues reduced depth estimation error relative to baseline. Text yielded the largest and most consistent accuracy gains, especially for farther targets and later sessions. At peak fatigue, response probability dipped for text but remained stable for blur and color, indicating an accuracy versus responsiveness trade-off. These results support mixed adaptive designs that default to subtle cues to preserve responsiveness at low alertness and introduce explicit overlays when precise metric information is needed.

This research explores various Augmented Reality (AR) annotations for supporting performance and decision-making in a time-critical task. The task was designed in collaboration with defence contacts to mimic a command and control scenario, requiring rapid decisions under time pressure. A set of annotations was derived from previous literature, and modifications were made to suit the tasking requirements. The annotations chosen were based on their level of abstraction from least to most abstract: Animations, StickyNotes, Icons, and Highlights. The study compared five conditions (Animations, StickyNotes, Icons, Highlights, and no annotations) for their effectiveness in supporting performance and decision-making in a time-critical task. StickyNotes was found to be the most effective for supporting performance. The study’s results also identify that annotations for assisting with time-critical tasking should be easy and quick to understand, contain sufficient complexity, and not be too abstract in detail.

This article presents the results from a Spatial Augmented Reality (SAR) study which evaluated the cognitive cost of several predictive cues. Participants performed a validated procedural button pressing task, where the predictive cue annotations guided them to the upcoming task. While existing research has evaluated predictive cues based on their performance and self-rated mental effort, actual cognitive cost has yet to be investigated. To measure the user's brain activity, this study utilized electroencephalogram (EEG) recordings. Cognitive load was evaluated by measuring brain responses for a secondary auditory oddball task, with reduced brain responses to oddball tones expected when cognitive load in the primary task is highest. A simple monitor n-back task and procedural task comparing monitor versus SAR were conducted, followed by a version of the procedural task comparing the SAR predictive cues. Results from the brain responses were able to distinguish between performance enhancing cues with a high and low cognitive load. Electrical brain responses also revealed that having an arc or arrow guide towards the upcoming task required the least amount of mental effort.

This work explores the interaction between Augmented Reality (AR) and eye accommodation for airborne surveillance by simulating AR environments in Virtual Reality (VR). We simulate the AR display as displays with the capabilities needed for airborne surveillance are limited and because it would be hazardous to experiment directly on surveillance aircraft. While there is precedent for simulating AR in a VR environment, our study account for two of the physical and physiological aspects of AR: we factor in the focal plane of the AR technology and simulate the eye accommodation reflex of the user to provide focus. We ran a study with 24 participants examining AR cues to support visual search. We also compare the effects of having secondary tasks (that surveillance operators are normally responsible for) directly on the observation window using AR. Our results show that the effectiveness of the AR cues is dependent on the modality of the secondary task. We also found that, under certain situations, operators’ performances for the search task are improved if the focal plane of the AR display is at the same distance as subsequent search targets.

Spatial Augmented Reality (SAR) is a useful tool for procedural tasks as virtual instructions can be co-located with the physical task. Predictive cue annotations have been utilized to further enhance user performance through SAR. These predictive cues have only been measured under ordinary circumstances. This paper aims to investigate predictive cues under a sub-optimal scenario by depriving users of sleep. Sleep deprivation is a common form of fatigue, which is known to have serious detriments on user performance. Based upon existing sleep literature, we expected predictive cue performance to degrade over time with sharp declines during the early hours of the morning. Despite these drops in performance, we hypothesized that having a predictive cue would benefit user performance in comparison to no cue. Results from a 62-hour sleep deprivation experiment indicated that providing SAR predictive cues was beneficial throughout sleep deprivation. Furthermore, having no predictive cue caused accuracy to decline earlier in the sleep deprivation period. From the predictive cues outlined in this paper, the line cue maintained the fastest response time and was least impacted by early morning performance declines.

Data physicalisations afford people the ability to directly interact with data using their hands, potentially achieving a more comprehensive understanding of a dataset. Due to their complex nature, the representation of graphs and networks could benefit from physicalisation, bringing the dataset from the digital world into the physical one. However, no empirical work exists investigating the effects physicalisations have upon comprehension as they relate to graph representations. In this work, we present initial design considerations for graph physicalisations, as well as an empirical study investigating differences in comprehension between virtual and physical representations. We found that participants perceived themselves as being more accurate via touch and sight (visual-haptic) than the graphical-only modality, and perceived a triangle count task as less difficult in visual-haptic than in the graphical-only modality. Additionally, we found that participants significantly preferred interacting with visual-haptic over other conditions, despite no significant effect on task time or error.

Despite the technological advances in building design, visualizing 3D building layouts can be especially difficult for novice and expert users alike, who must take into account design constraints including line-of-sight and visibility. Using CADwalk, a commercial building design tool that utilizes floor-facing projectors to show 1:1 scale building plans, this work presents and evaluates two floor-based visual cues for assisting with evaluating line-of-sight and visibility. Additionally, we examine the impact of using virtual cameras looking from the inside-out (from user’s location to objects of interest) and outside-in (looking from an object of interest’s location back towards the user). Results show that floor-based cues led to participants more correctly rating visibility, despite taking longer to complete the task. This is an effective tradeoff, given the final outcome (the building design) where accuracy is paramount.

The 2019 VGTC Virtual Reality Best Dissertation Award goes to Qi Sun, a 2018 graduate from Stony Brook University, for his dissertation entitled, “Computational Methods for Immersive Perception.”

Previous research has demonstrated that Augmented Reality can reduce a user's task response time and mental effort when completing a procedural task. This paper investigates techniques to improve user performance and reduce mental effort by providing projector-based Spatial Augmented Reality predictive cues for future responses. The objective of the two experiments conducted in this study was to isolate the performance and mental effort differences from several different annotation cueing techniques for simple (Experiment 1) and complex (Experiment 2) button-pressing tasks. Comporting with existing cognitive neuroscience literature on prediction, attentional orienting, and interference, we hypothesized that for both simple procedural tasks and complex search-based tasks, having a visual cue guiding to the next task's location would positively impact performance relative to a baseline, no-cue condition. Additionally, we predicted that direction-based cues would provide a more significant positive impact than target-based cues. The results indicated that providing a line to the next task was the most effective technique for improving the users' task time and mental effort in both the simple and complex tasks.

This paper presents the results of two cognitive load studies comparing three augmented reality display technologies: spatial augmented reality, the optical see-through Microsoft HoloLens, and the video see-through Samsung Gear VR. In particular, the two experiments focused on isolating the cognitive load cost of receiving instructions for a button-pressing procedural task. The studies employed a self-assessment cognitive load methodology, as well as an additional dual-task cognitive load methodology. The results showed that spatial augmented reality led to increased performance and reduced cognitive load. Additionally, it was discovered that a limited field of view can introduce increased cognitive load requirements. The findings suggest that some of the inherent restrictions of head-mounted displays materialize as increased user cognitive load.

Sleep deprivation is known to have serious deleterious effects on executive functioning and job performance. Augmented reality has an ability to place pertinent information at the fore, guiding visual focus and reducing instructional complexity. This paper presents a study to explore how spatial augmented reality instructions impact procedural task performance on sleep deprived users. The user study was conducted to examine performance on a procedural task at six time points over the course of a night of total sleep deprivation. Tasks were provided either by spatial augmented reality-based projections or on an adjacent monitor. The results indicate that participant errors significantly increased with the monitor condition when sleep deprived. The augmented reality condition exhibited a positive influence with participant errors and completion time having no significant increase when sleep deprived. The results of our study show that spatial augmented reality is an effective sleep deprivation countermeasure under laboratory conditions.