All-day augmented reality (AR) requires contextually-aware artificial intelligence (AI) models that excel across diverse daily contexts. Eye tracking could be a key source of information about user context and intention. However, such models using gaze sometimes struggle to outperform egocentric video-based baseline models. We propose that learning representations of scanpath history in a perceptually-relevant state space may solve this problem. However, scanpaths are often assumed to obey a Markovian assumption, i.e., only the current and previous fixation matter. In a user study (30 participants; 26.2 hours total), we analyzed scanpaths during nine everyday tasks and identified long-term temporal dependencies, with an average timescale of four fixations (2 seconds) into the past (i.e., violating the Markovian assumption). We discovered substantial task-specific variations in these dependencies. This confirms that scanpaths contain stereotyped “motifs” with context-dependent lengths/timescales. We discuss the implications for designing contextual AI models for AR applications.

Outside of self-report surveys, there are no proven, reliable methods to quantify visual discomfort or visually induced motion sickness symptoms when using head-mounted displays. While valuable tools, self-report surveys suffer from potential biases and low sensitivity due to variability in how respondents may assess and report their experience. Consequently, extreme visual-vestibular conflicts are generally used to induce discomfort symptoms large enough to measure reliably with surveys (e.g., stationary participants riding virtual roller coasters). An emerging area of research is the prediction of discomfort survey results from physiological and behavioral markers. However, the signals derived from experimental paradigms that are explicitly designed to be uncomfortable may not generalize to more naturalistic experiences where comfort is prioritized. In this work we introduce a custom VR headset designed to introduce significant near-eye optical distortion (i.e., pupil swim) to induce visual discomfort during more typical VR experiences. We evaluate visual comfort in our headset while users play the popular VR title Job Simulator and show that eye-tracked dynamic distortion correction improves visual comfort in a multi-session, within-subjects user study. We additionally use representational similarity analysis to highlight changes in head and gaze behavior that are potentially more sensitive to visual discomfort than surveys.