Virtual reality offers immersive educational environments but presents challenges in accurately detecting user distraction. Prior studies often define distraction based solely on the presence of external interventions, overlooking spontaneous cognitive lapses and assuming that all interventions are effective distractors. In this study, we investigate the feasibility of distraction detection in VR lectures using machine learning models trained on physiological sensor data, including electroencephalography, eye tracking, and heart rate. We evaluate a binary classification model intended for future real-time deployment, incorporating interventioninduced and self-reported cognitive distractions. Our best performing binary model achieved an F1 score of 87.3%. We also explore a multiclass classification approach across five distraction types, which yielded a lower F1 score of 32.8%. Our preprocessing pipeline and model architecture are optimized with real-time compatibility in mind, supporting future applications in adaptive educational VR systems. This work contributes to developing distractionaware VR learning environments by demonstrating the potential for scalable, real-time distraction classification using multimodal physiological data.

Virtual reality (VR) can potentially enhance student engagement and memory retention in the classroom. However, distraction among participants in a VR-based classroom is a significant concern. Several factors, including mind wandering, external noise, stress, etc., can cause students to become internally and/or externally distracted while learning. To detect distractions, single or multi-modal features can be used. A single modality is found to be insufficient to detect both internal and external distractions, mainly because of individual variability. In this work, we investigated multi-modal features: eye tracking and EEG data, to classify the internal and external distractions in an educational VR environment. We set up our educational VR environment and equipped it for multi-modal data collection. We implemented different machine learning (ML) methods, including k-nearest-neighbors (kNN), Random Forest (RF), one-dimensional convolutional neural network - long short-term memory (1 D-CNN-LSTM), and two-dimensional convolutional neural networks (2D-CNN) to classify participants' internal and external distraction states using the multi-modal features. We performed cross-subject, cross-session, and gender-based grouping tests to evaluate our models. We found that the RF classifier achieves the highest accuracy over 83% in the cross-subject test, around 68% to 78% in the cross-session test, and around 90% in the gender-based grouping test compared to other models. SHAP analysis of the extracted features illustrated greater contributions from the occipital and prefrontal regions of the brain, as well as gaze angle, gaze origin, and head rotation features from the eye tracking data.

Virtual reality (VR) makes learning more interesting for students and could help them remember what they have learned better than traditional methods. However, a student could get distracted in a VR environment because of stress, wandering thoughts, unwanted noise, outside sounds, etc. Distractions could be classified as either external (due to the environment) or internal (due to internal thoughts). To identify external distractions, previous researchers have used eye-gaze data. Eye-gaze data cannot, however, detect internal distractions because a user may be looking at the educational material in VR while also thinking about something else. We explored the usage of electroencephalogram (EEG) data to detect internal distractions. We designed an educational VR environment and trained three machine learning models: Random Forest (RF), Support Vector Machine (SVM), and k-nearest-neighbors (kNN), to detect internal distractions of students. For data labeling, we considered two window lengths (20 and 30 seconds) starting at 5 seconds after the distraction task started. We did cross-subject and cross-session tests, and our results show that kNN provides a better accuracy (64%) compared to RF and SVM. We also found that the shorter window length of 20 seconds provided a slightly better accuracy then the 30 second window. Our results are not far from such random guessing. Therefore, our contribution lies more in the fostering of ideas for future work that must employ more advanced and sophisticated techniques.

Educational virtual reality (VR) applications are the most recent addition to the learning management tools in this modern age. Due to health concerns, financial concerns, and convenience, people are looking for alternate ways to teach and learn. An efficient VR-based teaching interface could enhance student engagement, learning outcomes, and overall educational experience. Typically, teachers in a VR classroom do not have a way to know what students are doing since students are not visible. An efficient teaching interface should include some mechanism for a teacher to monitor students and alert the teacher if a student is trying to catch the attention of the teacher. An ideal interface would be one, which helps a teacher effectively monitor students while teaching without increasing the cognitive load of the teacher. In this paper, we present a comparative study of two such student monitoring interfaces. In the first interface, the student activity related information is shown using icons near the student avatar (representing a student in the VR environment). While in the second interface, a set of centrally-arranged emoticon-like visual indicators are present in addition to the student avatar, and the student activity related information is shown near the student emoticon. We present a detailed user experiment comparing the two interfaces in terms of teaching management, student monitoring capability, cognitive load, and user preference. Participants preferred and performed better with Indicator-located interface over avatar-located interface.

We assess the effectiveness of attention guidance cues in an educational platform in local high schools with real students. Three eye-tracked visual cues, previously assessed for their ability to guide and restore attention, are compared against a baseline absence of cue in a VR field trip of a virtual solar energy field. Students experienced four presentations on solar energy production including in-world animations and teacher imagery, in three of which the visual cues guided attention to the relevant object or teacher in the scene. Attention guidance using visual cues is commonly studied using “search and selection” style tasks, but has not been studied in the context of maintaining attention in real-world environments.

Virtual Reality (VR) has been found useful to improve engagement and retention level of students, for some topics, compared to traditional learning tools such as books, and videos. However, a student could still get distracted and disengaged due to a variety of factors including stress, mind-wandering, unwanted noise, and external alerts. Student eye gaze data could be useful for detecting these distracted students. Gaze data-based visualizations have been proposed in the past to help a teacher monitor distracted students. However, it is not practical for a teacher to monitor a large number of student indicators while teaching. To help filter students based on distraction level, we propose an automated system based on machine learning to classify students based on their distraction level. The key aspects are: (1) we created a labeled eye gaze dataset from an educational VR environment, (2) we propose an automatic system to gauge a student’s distraction level from gaze data, and (3) we apply and compare several classifiers for this purpose. Each classifier classifies distraction, per educational activity section, into one of three levels (low, mid or high). Our results show that Random Forest (RF) classifier had the best accuracy (98.88%) compared to the other models we tested. Additionally, a personalized machine learning model using either RF, kNN, or Extreme Gradient Boosting (XGBoost) model was found to improve the classification accuracy significantly.

Virtual Reality (VR) headsets with embedded eye trackers are appearing as consumer devices (e.g. HTC Vive Eye, FOVE). These devices could be used in VR-based education (e.g., a virtual lab, a virtual field trip) in which a live teacher guides a group of students. The eye tracking could enable better insights into students’ activities and behavior patterns. For real-time insight, a teacher’s VR environment can display student eye gaze. These visualizations would help identify students who are confused/distracted, and the teacher could better guide them to focus on important objects. We present six gaze visualization techniques for a VR-embedded teacher’s view, and we present a user study to compare these techniques. The results suggest that a short particle trail representing eye trajectory is promising. In contrast, 3D heatmaps (an adaptation of traditional 2D heatmaps) for visualizing gaze over a short time span are problematic.

Past research has shown that humans exhibit certain eye-head responses to the appearance of visual stimuli, and these natural reactions change during different activities. Our work builds upon these past observations by offering new insight to how humans behave in Virtual Reality (VR) compared to Physical Reality (PR). Using eye- and head- tracking technology, and by conducting a study on two groups of users - participants in VR or PR - we identify how often these natural responses are observed in both environments. We find that users statistically move their heads more often when viewing stimuli in VR than in PR, and VR users also move their heads more in the presence of text. We open a discussion for identifying the HWD factors that cause this difference, as this may not only affect predictive models using eye movements as features, but also VR user experience overall.

Most modern stereoscopic 3D applications use fixed stereoscopic 3D parameters (separation and convergence) to render the scene on a 3D display. But, keeping these parameters fixed during usage does not always provide the best experience since it can reduce the amount of depth perception possible in some applications which have large variability in object distances. We developed two stereoscopic rendering techniques which actively vary the stereo parameters based on the scene content. Our first algorithm calculates a low resolution depth map of the scene and chooses ideal stereo parameters based on that depth map. Our second algorithm uses eye tracking data to get the gaze direction of the user and chooses ideal stereo parameters based on the distance of the gazed object. We evaluated our techniques in an experiment that uses three depth judgment tasks: depth ranking, relative depth judgment and path tracing. Our results indicate that variable stereo parameters provide enhanced depth discrimination compared to static parameters and were preferred by our participants over the traditional fixed parameter approach. We discuss our findings and possible implications on the design of future stereoscopic 3D applications.

We present AnalyticalInk, a novel math learning environment prototype that uses a semantic graph as the knowledge representation of algebraic and geometric word problems. The system solves math problems by reasoning upon the semantic graph and automatically generates conceptual and procedural scaffoldings in sequence. We further introduces a step-wise tutoring framework, which can check students' input steps and provide the adaptive scaffolding feedback. Based on the knowledge representation, AnalyticalInk highlights keywords that allow users to further drag them onto the workspace to gather insight into the problem's initial conditions. The system simulates a pen-and-paper environment to let users input both in algebraic and geometric workspaces. We conducted an usability evaluation to measure the effectiveness of AnalyticalInk. We found that keyword highlighting and dragging is useful and effective toward math problem solving. Answer checking in the tutoring component is useful. In general, our prototype shows the promise in helping users to understand geometrical concepts and master algebraic procedures under the problem solving.

We present the results of a comprehensive video game study which explores how the gaming experience is effected when several 3D user interface technologies are used simultaneously. We custom designed an air-combat game integrating several 3DUI technologies (stereoscopic 3D, head tracking, and finger-count gestures) and studied the combined effect of these technologies on the gaming experience. Our game design was based on existing design principles for optimizing the usage of these technologies in isolation. Additionally, to enhance depth perception and minimize visual discomfort, the game dynamically optimizes stereoscopic 3D parameters (convergence and separation) based on the user's look direction. We conducted a within subjects experiment where we examined performance data and self-reported data on users perception of the game. Our results indicate that participants performed significantly better when all the 3DUI technologies (stereoscopic 3D, head-tracking and finger-count gestures) were available simultaneously with head tracking as a dominant factor. We explore the individual contribution of each of these technologies to the overall gaming experience and discuss the reasons behind our findings.

Counting using one's fingers is a potentially intuitive way to enumerate a list of items and lends itself naturally to gesture-based menu systems. In this paper, we present the results of the first comprehensive study on Finger-Count menus to investigate its usefulness as a viable option for 3D menu selection tasks. Our study compares 3D gesture-based finger counting (Finger Count menus) with two gesture-based menu selection techniques (Hand-n-Hold, Thumbs-Up), derived from existing motion-controlled video game menu selection strategies, as well as 3D Marking menus. We examined selection time, selection accuracy and user preference for all techniques. We also examined the impact of different spatial layouts for menu items and different menu depths. Our results indicate that Finger-Count menus are significantly faster than the other menu techniques we tested and are the most liked by participants. Additionally, we found that while Finger-Count menus and 3D Marking menus have similar selection accuracy, Finger-Count menus are almost twice as fast compared to 3D Marking menus.

We present a study that investigates user performance benefits of playing video games using 3D motion controllers in 3D stereoscopic vision in comparison to monoscopic viewing. Using the PlayStation 3 game console coupled with the PlayStation Move Controller, we explored five different games that combine 3D stereo and 3D spatial interaction. For each game, quantitative and qualitative measures were taken to determine if users performed better and learned faster in the experimental group (3D stereo display) than in the control group (2D display). A game expertise pre-questionnaire was used to classify participants into beginners and expert game player categories to analyze a possible impact on performance differences. The results show two cases where the 3D stereo display did help participants perform significantly better than with a 2D display. For the first time, we can report a positive effect on gaming performance based on stereoscopic vision, although reserved to isolated tasks and depending on game expertise. We discuss the reasons behind these findings and provide recommendations for game designers who want to make use of 3D stereoscopic vision and 3D motion control to enhance game experiences.