Numerous models have been developed for scanpath and saliency prediction, which are typically trained on scanpaths, which model eye movement as a sequence of discrete fixation points connected by saccades, while the rich information contained in the raw trajectories is often discarded. Moreover, most existing approaches fail to capture the variability observed among human subjects viewing the same image. They generally predict a single scanpath of fixed, pre-defined length, which conflicts with the inherent diversity and stochastic nature of real-world visual attention. To address these challenges, we propose DiffEye, a diffusion-based training framework designed to model continuous and diverse eye movement trajectories during free viewing of natural images. Our method builds on a diffusion model conditioned on visual stimuli and introduces a novel component, namely Corresponding Positional Embedding (CPE), which aligns spatial gaze information with the patch-based semantic features of the visual input. By leveraging raw eye-tracking trajectories rather than relying on scanpaths, DiffEye captures the inherent variability in human gaze behavior and generates high-quality, realistic eye movement patterns, despite being trained on a comparatively small dataset. The generated trajectories can also be converted into scanpaths and saliency maps, resulting in outputs that more accurately reflect the distribution of human visual attention. DiffEye is the first method to tackle this task on natural images using a diffusion model while fully leveraging the richness of raw eye-tracking data. Our extensive evaluation shows that DiffEye not only achieves state-of-the-art performance in scanpath generation but also enables, for the first time, the generation of continuous eye movement trajectories. Project webpage: https://diff-eye.github.io/

Objective: In recent years, the training of novice medical professionals with simulated environments such as virtual reality (VR) and augmented reality (AR) has increased dramatically. However, the usability of these technologies is limited due to the complexity involved in creating the clinical content. To be comparable to a clinical environment, the simulation platform should include real-world learning parameters such as patient physiology, emotions, and clinical team behaviors. Incorporating such nondeterministic parameters has historically required faculty to possess advanced programming skills. Lack of effective software for instructors to easily develop VR curriculum content is a hurdle in developing VR based curriculum. Method: We address this challenge through a software platform that simplifies the creation of Interactive Mixed Reality (IMR) scenarios. Three educational components we were able to embed into an IMR scenario includes 1) integrated 360-degree video recording of a clinical encounter to provide a first-person perspective, 2) rich annotated knowledge content, and 3) assessment questionnaire. We developed a sepsis prevention education scenario using the IMR software to demonstrate the potential of enhancing simulated medical training by accelerating clinical exposure for novice students. Result: An IRB approved study was conducted with a group of 28 novice students to evaluate the efficacy of the IMR technology. The participants provided feedback by answering demographics, NASA-TLX and system usability scale questionnaires. Significance: Our software is a step towards improving VR based education content development process. Conclusion: The studies conducted here provide preliminary evidence that the IMR software is a usable technology based on the NASA-TLX and system usability studies conducted. Future work will compare our new educational strategy for medical training with live simulation scenarios inside a hospital room and a simple video-based curriculum.