Despite the rapid growth of Virtual Reality (VR) technology across diverse applications, cybersickness remains a significant barrier to its widespread adoption. To move toward effective mitigation, current efforts have concentrated on predicting cybersickness, particularly through the integration of multimodal data sources. However, current cybersickness prediction models face several limitations that impede progress toward robust, generalizable systems: most studies are confined to single datasets due to substantial technical challenges in integrating heterogeneous VR sensor data from different hardware platforms, and existing approaches fail to appropriately model the ordinal nature of cybersickness severity scales, treating Fast Motion Sickness (FMS) scores as either continuous variables or arbitrary categorical classifications that discard valuable ordering information. We developed a comprehensive statistical feature abstraction framework combined with dual-head ordinal regression to enable cybersickness prediction across heterogeneous VR datasets. Our approach transforms diverse sensor modalities from VR.net (Meta Quest Pro proprietary systems), SIM21, and VRWalking (HTC VIVE/Tobii) platforms into standardized statistical descriptors that capture cybersickness-relevant behavioral patterns while remaining invariant to study design and data collection methodologies. The dual-head architecture combines ordinal-aware predictions with direct regression outputs through weighted averaging. Using the VR.net and VRWalking datasets, we achieved cross-dataset generalization with an RMSE of 0.8341 ± 0.0731 and MAE of 0.6254 ± 0.0389 through rigorous 10-fold participant-aware cross-validation, along with 97.7% ± 0.87% accuracy in severity classification despite heterogeneous hardware configurations and scale differences. Our statistical feature abstraction approach enables effective cross-platform cybersickness prediction by creating domain-agnostic representations that preserve essential physiological patterns, advancing the development of universal cybersickness monitoring systems capable of operating across diverse VR platforms.

Walking in immersive virtual reality (VR) environments is often disrupted by gait (i.e., walking patterns) instability, a challenge that is further exacerbated for individuals with mobility impairments. This study examines the effectiveness of multimodal feedback by integrating auditory, vibrotactile, and visual cues in improving walking performance within VR. A total of 68 participants, equally divided between those with mobility impairments and those without, completed walking tasks under multiple feedback conditions. Walking velocity was the primary performance metric, supplemented by subjective assessments of mental workload, fatigue, presence, usability, and simulator sickness. Results revealed that multimodal feedback significantly enhanced walking velocity compared to unimodal and bimodal conditions, with statistical analysis confirming strong improvements (p < .001). Participants also reported a favorable user experience under multimodal conditions despite slightly increased cognitive demand. These findings highlight the potential of integrated sensory feedback to mitigate gait disturbances in VR, promoting safer and more accessible locomotion for users with and without mobility impairments.

Cybersickness, a motion sickness like discomfort, is a major barrier to the usability of virtual reality (VR) systems. While prior work has focused mainly on predicting cybersickness severity, practical mitigation requires not only detecting how sick a user feels but also deciding whether a countermeasure is beneficial and determining its appropriate intensity. In this paper, we propose a two phase multitask learning framework that jointly models cybersickness severity, blur effectiveness, and blur intensity. In Phase 1, we pretrain temporal deep learning backbones on two single label datasets with only severity annotations. In Phase 2, we pro-gressively finetune the models on a multi-label dataset containing severity, blur effectiveness, and blur level labels. We evaluate three backbone architectures a Time-Series Transformer, Deep Temporal Convolutional Network, and TS-Mamba under a 10-fold block aware cross validation scheme. Results show that two phase training significantly outperforms single phase baselines, with the Time Series Transformer achieving best performance (FMS MAE = 0.57, R2 = 0.87; Blur Level MAE = 0.49, 2 = 0.95; Blur Preference ACC = 99.5%). Unlike prior rule based reduction frameworks that rely on static heuristics, our approach provides a data-driven "detect-decide-dose" pipeline that adapts blur mitigation dynamically to individual users. This demonstrates that single label pre-training is an effective strategy for developing multitask VR safety models under limited labeled data. To our knowledge, this is the first framework that unifies cybersickness prediction and adaptive reduction in a single model.

Cybersickness, characterized by discomforts such as dizziness, nausea, and eye strain, remains a significant barrier to the widespread adoption of virtual reality (VR). Recent research have proposed supervised machine learning models to predict the onset of cybersickness; however, these approaches depend heavily on labeled datasets. Acquiring labeled datasets typically necessitates time-consuming and resource-intensive user studies, limiting the feasibility of these supervised methods for consumer-level VR applications where obtaining labeled user data during use is impractical. Moreover, due to individual differences, often these datasets are not generalizable in consumer VR use. To address these limitations, we propose a novel semi-supervised learning framework for predicting cybersickness (i.e., Fast Motion Sickness (FMS)) using eye tracking, heart rate, and galvanic skin response data. Our proposed semi-supervised approach uses pseudo-labeling techniques (i.e., Self-training, Label Propagation, and Label Spreading) fused with temporal deep learning models (i.e., DeepTCN, CNN-LSTM, Transformers, LSTM). We evaluated our approach on three public cybersickness datasets (i.e., Bumpy Ride, Simulation 21, Maze) and our proposed semi-supervised approach demonstrates strong cybersickness predictive performance using only$1-5 {\%}$labeled data (i.e.,$95-99 {\%}$data remains unlabeled). Notably, the self-training approach with a DeepTCN model achieved an accuracy of$\mathbf{7 5. 8 6 \%}$in FMS prediction, outperforming the other models and pseudolabeling approaches. Our findings establish the viability of semisupervised learning for cybersickness prediction with minimally labeled datasets, paving the way for more practical and potentially generalizable cybersickness prediction systems in consumer VR applications.

This research aims to examine the effects of various vibrotactile feedback techniques on gait (i.e., walking patterns) in virtual reality (VR). Prior studies have demonstrated that gait disturbances in VR users are significant usability barriers. However, adequate research has not been performed to address this problem. In our study, 39 participants (with mobility impairments: 18, without mobility impairments: 21) performed timed walking tasks in a real-world environment and identical activities in a VR environment with different forms of vibrotactile feedback (spatial, static, and rhythmic). Within-group results revealed that each form of vibrotactile feedback improved gait performance in VR significantly compared to the no vibrotactile condition in VR for individuals with and without mobility impairments. Moreover, spatial vibrotactile feedback increased gait performance significantly in both participant groups compared to other vibrotactile conditions.

Enhancing user experience and performance, including task load in immersive environments, requires accurate prediction of user gaze point, reaction time, and mental and physical load uptake. Current gaze prediction approaches focus primarily on motion-based information, lacking physiological data, which leads to poor prediction accuracy in highly dynamic virtual reality (VR) environments. Traditional cognitive load measurements rely on post-task analysis without proper multimodal data integration and fail to capture the real-time dynamics of user states during interaction. Likewise, reaction time or attention load are often assessed only after the interaction, without using real-time immersive sensor data, which limits adaptive responsiveness. To tackle these limitations, we leveraged a comprehensive multimodal dataset - VRWalking, which recorded timestamped eye-tracking metrics, physiological signals (heart rate and galvanic skin response), and behavioral performance data during real-time engagement in a VR environment. We developed a unified multitask model based on the MultiPatchFormer architecture, which processes multimodal VR signals through dual patch projection branches for gaze and classification inputs. The model employs multiscale patch embeddings, cross-attention between gaze and classification pathways, channel attention, and transformer encoders to jointly predict continuous user gaze and classify reaction time, cognitive load (mental load and physical load). Our methodology achieved excellent predictive performance: 95.64% for reaction time, 98.01% for mental load, and 97.45% for physical load, with a MAPE (Mean Absolute Percentage Error) of 15.24% for gaze prediction. We applied Shapley Additive explanations (SHAP) analysis to interpret the model’s behavior across all features, including eye-tracking, head-tracking, and physiological signals. The analysis revealed which features most influenced the predictions of user gaze, reaction time, mental load, and physical load. Our methods, while based only on the VRWalking dataset, demonstrated strong performance across all tasks, suggesting promising potential for real-world VR applications such as interactive training systems that respond to user attention lapses, educational platforms that adapt to cognitive load, and performance assessments that consider physiological indicators.

Recent combinations of interactive technology, humans, and water have resulted in “WaterHCI”. WaterHCI design seeks to complement the many benefits of engagement with the aquatic domain, by offering, for example, augmented reality systems for snorkelers, virtual reality in floatation tanks, underwater musical instruments for artists, robotic systems for divers, and wearables for swimmers. We conducted a workshop in which WaterHCI experts articulated the field's grand challenges, aiming to contribute towards a systematic WaterHCI research agenda and ultimately advance the field.

Virtual Reality (VR) is quickly establishing itself in various industries, including training, education, medicine, and entertainment, in which users are frequently required to carry out multiple complex cognitive and physical activities. However, the relationship between cognitive activities, physical activities, and familiar feelings of cybersickness is not well understood and thus can be unpredictable for developers. Researchers have previously provided labeled datasets for predicting cybersickness while users are stationary, but there have been few labeled datasets on cybersickness while users are physically walking. Moreover, it is unclear how walking while cybersick will affect cognitive load, even though room-scale interaction is typical in many VR games. Thus, from 39 participants, we collected head orientation, head position, eye tracking, images, physiological readings from external sensors, and the self-reported cybersickness severity, physical load, and mental load in VR. Throughout the data collection, participants navigated mazes via real walking and performed tasks challenging their attention and working memory. To demonstrate the dataset’s utility, we conducted a case study of training classifiers in which we achieved 95% accuracy for cybersickness severity classification. The noteworthy performance of the straightforward classifiers makes this dataset ideal for future researchers to develop cybersickness detection and reduction models. To better understand the features that helped with classification, we performed SHAP(SHapley Additive exPlanations) analysis, highlighting the importance of eye tracking and physiological measures for cybersickness prediction while walking. This open dataset can allow future researchers to study the connection between cybersickness and cognitive loads and develop prediction models. This dataset will empower future VR developers to design efficient and effective Virtual Environments by improving cognitive load management and minimizing cybersickness.

Cybersickness continues to bar many individuals from taking full advantage of virtual reality (VR) technology. Previous work has established that navigating virtual terrain with elevation changes poses a significant risk in this regard. In this paper, we investigate the effectiveness of three cybersickness reduction strategies on users performing a navigation task across virtual elevation-altering terrain. These strategies include static field of view (FOV) reduction, a flat surface approach that disables terrain collision and maintains constant elevation for users, and SmoothRide, a novel technique designed to dampen a user's perception of vertical motion as they travel. To assess the impact of these strategies, we conducted a within-subjects study involving 61 participants. Each strategy was compared against a control condition, where users navigated across terrain without any cybersickness reduction measures in place. Cybersickness data were collected using the Fast Motion Sickness Scale (FMS) and Simulator Sickness Questionnaire (SSQ), along with galvanic skin response (GSR) data. We measured user presence using the IGroup Presence questionnaire (IPQ) and a Single-Item Presence Scale (SIP). Our findings reveal that users experienced significantly lower levels of cybersickness using SmoothRide or FOV reduction. Presence scores reported on the IPQ were statistically similar between SmoothRide and the control condition. Conversely, terrain flattening had adverse effects on user presence scores, and we could not identify a significant effect on cybersickness compared to the control. We demonstrate that SmoothRide is an effective, lightweight, configurable, and easy-to-integrate tool for reducing cybersickness in simulations featuring elevation-altering terrain.

Maintaining balance in immersive virtual reality (VR) environments poses a significant challenge for users, particularly affecting those with pre-existing balance disorders. This study investigates the efficacy of multimodal feedback-comprising auditory, vibrotactile, and visual stimuli-in mitigating balance issues within VR. A sample of 68 participants, divided equally between individuals with balance deficits related to multiple sclerosis and those without, was evaluated. The research explored the impact of various feedback conditions on balance performance. The results demonstrated that the multimodal feedback condition significantly enhanced balance control compared to other conditions, with statistical analysis confirming this improvement (p<.001). These findings underscore the potential of integrated sensory feedback in addressing balance-related difficulties in VR, thereby improving the overall accessibility and user experience for individuals affected by balance impairments. This research contributes valuable insights into optimizing VR environments for enhanced stability and user comfort.

In recent cybersickness research, there has been a growing interest in predicting cybersickness using real-time physiological data such as heart rate, galvanic skin response, eye tracking, postural sway, and electroencephalogram. However, the impact of individual factors such as age and gender, which are pivotal in determining cybersickness susceptibility, remains unknown in predictive models. Our research seeks to address this gap, underscoring the necessity for a more personalized approach to cybersickness prediction to ensure a better, more inclusive virtual reality experience. We hypothesize that a personalized cybersickness prediction model would outperform non-personalized models in predicting cybersickness. Evaluating this, we explored four personalization techniques: 1) data grouping, 2) transfer learning, 3) early shaping, and 4) sample weighing using an open-source cybersickness dataset. Our empirical results indicate that personalized models significantly improve prediction accuracy. For instance, with early shaping, the Deep Temporal Convolutional Neural Network (DeepTCN) model achieved a 69.7% reduction in RMSE compared to its non-personalized version. Our study provides evidence of personalization techniques' benefits in improving cybersickness prediction. These findings have implications for developing personalized cybersickness prediction models tailored to individual differences, which can be used to develop personalized cybersickness reduction techniques in the future.

The use of Head-Mounted Displays (HMDs) in Virtual Reality (VR) can cause gait disturbance problems for users because they are unable to see the real world while in VR. This is particularly challenging for individuals with mobility impairments who rely heavily on visual cues to maintain balance. The limited research that has been conducted on this issue has not focused on ways to solve it. IN this study, we investigated how different visual feedback methods affect walking patterns (i.e., gait) in VR. The study involved 50 participants, including 25 individuals with mobility impairments due to multiple sclerosis and 25 without mobility impairments. The participants completed timed walking tasks in both the real world and in VR environments that included various types of visual feedback, such as spatial, static, and rhythmic. The results showed that static and rhythmic visual feedback significantly improved gait performance in VR for people with mobility impairments compared to no visual feedback in VR. The results will help to make more accessible virtual environments for people with mobility impairments.

Immersive Virtual Reality (VR) users often experience difficulties with maintaining their balance. This issue poses a significant challenge to the widespread usability and accessibility of VR, particularly for individuals with balance impairments. Previous studies have confirmed the existence of balance problems in VR, but little attention has been given to addressing them. To investigate the impact of different feedback modalities (auditory, vibrotactile, and visual) on balance in immersive VR, we conducted a study with 50 participants, consisting of 25 individuals with balance impairments due to type 2 diabetes and 25 without balance impairments. Participants were asked to perform standing reach and grasp tasks. Our findings indicated that auditory and vibrotactile techniques improved balance significantly (p<.001) in immersive VR for participants with and without balance impairments, while visual techniques only improved balance significantly for participants with balance impairments. Also, auditory and vibrotactile feedback techniques improved balance significantly more than visual techniques. Spatial auditory feedback outperformed other conditions significantly for all people. This study presents implementations and comparisons of potential strategies that can be implemented in future VR environments to enhance standing balance and promote the broader adoption of VR.

Virtual reality (VR) technologies are used in a diverse range of applications. Many of these involve an embodied conversational agent (ECA), a virtual human who exchanges information with the user. Unfortunately, VR technologies remain inaccessible to many users due to the phenomenon of cybersickness: a collection of negative symptoms such as nausea and headache that can appear when immersed in a simulation. Many factors are believed to affect a user's level of cybersickness, but little is known regarding how these factors may influence a user's opinion of an ECA. In this study, we examined the effects of virtual stairs, a factor associated with increased levels of cybersickness. We recruited 39 participants to complete a simulated airport experience. This involved a simple navigation task followed by a brief conversation with a virtual airport customs agent in Spanish. Participants completed the experience twice, once walking across flat hallways, and once traversing a series of staircases. We collected self-reported ratings of cybersickness, presence, and perception of the ECA. We additionally collected physiological data on heart rate and galvanic skin response. Results indicate that the virtual staircases increased user level's of cybersickness and reduced their perceived realism of the ECA, but increased levels of presence.

We are pleased to present the proceedings of the 30th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2023), held March 25–29, 2023, in Shanghai, China, in hybrid format. These proceedings contain 69 of the 130 papers presented at IEEE VR 2023, with the other 61 papers being published in the IEEE VR 2023 special issue of the Transactions on Visualization and Computer Graphics. IEEE VR 2023 had 612 submissions for an acceptance rate of 21%.

Cybersickness is a common ailment associated with virtual reality (VR) user experiences. Several automated methods exist based on machine learning (ML) and deep learning (DL) to detect cyber-sickness. However, most of these cybersickness detection methods are perceived as computationally intensive and black-box methods. Thus, those techniques are neither trustworthy nor practical for deploying on standalone energy-constrained VR head-mounted devices (HMDs). In this work, we present an explainable artificial intelligence (XAI)-based framework Lite VR for cybersickness detection, explaining the model's outcome, reducing the feature dimensions, and overall computational costs. First, we develop three cybersick-ness DL models based on long-term short-term memory (LSTM), gated recurrent unit (GRU), and multilayer perceptron (MLP). Then, we employed a post-hoc explanation, such as SHapley Additive Explanations (SHAP), to explain the results and extract the most dominant features of cybersickness. Finally, we retrain the DL models with the reduced number of features. Our results show that eye-tracking features are the most dominant for cybersickness detection. Furthermore, based on the XAI-based feature ranking and dimensionality reduction, we significantly reduce the model's size by up to 4.3×, training time by up to 5.6×, and its inference time by up to 3.8×, with higher cybersickness detection accuracy and low regression error (i.e., on Fast Motion Scale (FMS)). Our proposed lite LSTM model obtained an accuracy of 94% in classifying cyber-sickness and regressing (i.e., FMS 1–10) with a Root Mean Square Error (RMSE) of 0.30, which outperforms the state-of-the-art. Our proposed Lite VR framework can help researchers and practitioners analyze, detect, and deploy their DL-based cybersickness detection models in standalone VR HMDs.

In this special issue of IEEE Transactions on Visualization and Computer Graphics (TVCG), we are pleased to present the top papers from the 30th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2023), held March 25–29, 2023, in Shanghai, China, in hybrid format.

Users of head-mounted displays (HMDs) for virtual reality (VR) sometimes have balance issues since HMDs impede their view of the outside world. This has a greater impact on people with balance impairments since many rely more heavily on their visual cues to keep their balance. This is a significant obstacle to the universal usability and accessibility of VR. Although previous studies have verified the imbalance issue, not much work has been done to diminish it. In this study, we investigated how to increase VR balance by utilizing additional visual cues. To examine how different visual approaches (static, rhythmic, spatial, and center of pressure (CoP) based feedback) affect balance in VR, we recruited 100 people (50 with balance impairments due to multiple sclerosis and 50 without balance impairments) across two different geographic locations (United States and Bangladesh). All people completed both standing visual exploration as well as standing reach and grasp tasks. Results demonstrated that static, rhythmic, and CoP visual feedback approaches enhanced balance significantly ( ) in VR for people with balance impairments. The methods described in this study could be applied to design more accessible virtual environments for people with balance impairments.

In this paper, we propose a novel method for monocular depth estimation in dynamic scenes. We first explore the arbitrariness of object’s movement trajectory in dynamic scenes theoretically. To overcome the arbitrariness, we use assume that points move along a straight line over short distances and then summarize it as a triangular constraint loss in two dimensional Euclidean space. This triangular loss function is used as part of our proposed pixel movement prediction network, PMPNet, to estimate a dense depth map from a single input image. To overcome the depth inconsistency problem around the edges, we propose a deformable support window module that learns features from different shapes of objects, making depth value more accurate around edge area. The proposed model is trained and tested on two outdoor datasets - KITTI and Make3D, as well as an indoor dataset - NYU Depth V2. The quantitative and qualitative results reported on these datasets demonstrate the success of our proposed model when compared against other approaches. Ablation study results on the KITTI dataset also validate the effectiveness of the proposed pixel movement prediction module as well as the deformable support window module.

A plethora of studies has been conducted to detect and reduce cybersickness in real-time. However, prior attempts to detect and minimize cybersickness after its onset may be ineffective as the onset tends to persist beyond its first occurrence. By forecasting the onset of cybersickness, it may be possible to mitigate the severity of cybersickness through earlier interventions. This research proposed a multimodal deep fusion approach to forecast cybersickness from the user’s physiological, head-tracking, and eye-tracking data. We proposed several hybrid multimodal deep fusion neural networks with Long short-term memory (LSTMs), Neural basis expansion analysis for interpretable time series forecasting(NBEATs) and Deep Temporal Convolutional Networks(DeepTCN) neural models to forecast cybersickness 30-60s in advance to its onset. To validate our proposed approach, we recruited 30 participants who were immersed in five virtual reality simulations. We collected eye-tracking, head-tracking, heart rate, and galvanic skin response data and used the fast-motion scale as ground truth. Our results suggest that the DeepTCN model with our proposed multimodal fusion network can forecast cybersickness onset 60 seconds in advance with a root-mean-square error of 0.49 (on a scale from 0-10). Furthermore, our results demonstrated that fusing eye tracking, heart rate, and galvanic skin response data outperformed other data fusion approaches. This research clarifies how early cybersickness can be forecast, paving the way for future research on early cybersickness mitigation approaches.

The objective of this study is to investigate the impact of several auditory feedback modalities on gait (i.e., walking patterns) in virtual reality (VR). Prior research has substantiated gait disturbances in VR users as one of the primary obstacles to VR usability. However, minimal research has been done to mitigate this issue. We recruited 39 participants (with mobility impairments: 18, without mobility impairments: 21) who completed timed walking tasks in a real-world environment and the same tasks in a VR environment with various types of auditory feedback. Within-subject results showed that each auditory condition significantly improved gait performance while in VR $(p \lt$.001) compared to the no auditory condition in VR for both groups of participants with and without mobility impairments. Moreover, spatial audio improved gait performance significantly $(p \lt$.001) compared to other auditory conditions for both groups of participants. This research could help to make walking in VR more accessible for people with and without mobility impairments.

Cybersickness (i.e., visually induced motion sickness) serves as a significant obstacle to the usage and broader adoption of virtual reality (VR) technologies. This collection of symptoms akin to motion sickness can be impacted by different characteristics of a virtual experience, such as visual realism and optical flow. However, relatively little is known regarding how cybersickness is influenced by traversing uneven virtual terrain. In this study, we aim to better understand the impacts of different virtual terrain types on cybersickness in VR. We recruited 38 participants to navigate a virtual forest environment with three terrain variants: flat surface, terrain with regular bumps, and irregular terrain generated from Perlin noise. We collected cybersickness data using the Fast Motion Sickness Scale (FMSS) and Simulator Sickness Questionnaire (SSQ) in addition to galvanic skin response data. Our results indicate that users felt greater levels of cybersickness in the presence of regular bumps and irregular terrain than they did when traversing flat geometry. We recommend that designers exercise caution when incorporating uneven terrain into their virtual experiences, and maintain awareness of the risks carried by these design decisions.

Virtual Reality (VR) users often experience postural instability, i.e., balance problems, which could be a major barrier to universal usability and accessibility for all, especially for persons with balance impairments. Prior research has confirmed the imbalance effect, but minimal research has been conducted to reduce this effect. We recruited 42 participants (with balance impairments: 21, without balance impairments: 21) to investigate the impact of several auditory techniques on balance in VR, specifically spatial audio, static rest frame audio, rhythmic audio, and audio mapped to the center of pressure (CoP). Participants performed two types of tasks - standing visual exploration and standing reach and grasp. Within-subject results showed that each auditory technique improved balance in VR for both persons with and without balance impairments. Spatial and CoP audio improved balance significantly more than other auditory conditions. The techniques presented in this research could be used in future virtual environments to improve standing balance and help push VR closer to universal usability.

Virtual Reality (VR) users often encounter postural instability, i.e., balance issues, which can be a significant impediment to universal usability and accessibility, particularly for those with balance impairments. Prior research has validated imbalance issues, but little effort has been made to mitigate them. We recruited 39 participants (with balance impairments: 18, without balance impairments: 21) to examine the effect of various vibrotactile feedback techniques on balance in virtual reality, specifically spatial vibrotactile, static vibrotactile, rhythmic vibrotactile, and vibrotactile feedback mapped to the center of pressure (CoP). Participants completed standing visual exploration and standing reach and grasp tasks. According to within-subject results, each vibrotactile feedback enhanced balance in VR significantly (p <.001) for those with and without balance impairments. Spatial and CoP vibrotactile feedback enhanced balance significantly more (p <.001) than other vibrotactile feedback. This study presents strategies that might be used in future virtual environments to enhance standing balance and bring VR closer to universal usage.

This research investigates how experiencing virtual embodiment in a wheelchair affects implicit bias towards people who use wheelchairs. We also investigate how receiving information from a virtual instructor who uses a wheelchair affects implicit bias towards people who use wheelchairs. Implicit biases are actions or judgments of people towards various concepts or stereotypes (e.g., races). We hypothesized that experiencing a Disability Simulation (DS) through an avatar in a wheelchair and receiving information from an instructor with a disability will have a significant effect on participants' ability to recall disability-related information and will reduce implicit biases towards people who use wheelchairs. To investigate this hypothesis, a 2x2 between-subjects user study was conducted where participants experienced an immersive VR DS that presents information about Multiple Sclerosis (MS) with factors of instructor (i.e., instructor with a disability versus instructor without a disability) and locomotion interface (i.e., without a disability - locomotion through in-place-walking, with a disability - locomotion in a wheelchair). Participants took a disability-focused Implicit Association Test two times, once before and once after experiencing the DS. They also took a test of knowledge retention about MS. The primary result is: experiencing the DS through locomotion in a wheelchair was better for both the disability-related information recall task and reducing implicit bias towards people who use wheelchairs.

Cybersickness prediction is one of the significant research challenges for real-time cybersickness reduction. Researchers have proposed different approaches for predicting cybersickness from bio-physiological data (e.g., heart rate, breathing rate, electroencephalogram). However, collecting bio-physiological data often requires external sensors, limiting locomotion and 3D-object manipulation during the virtual reality (VR) experience. Limited research has been done to predict cybersickness from the data readily available from the integrated sensors in head-mounted displays (HMDs) (e.g., head-tracking, eye-tracking, motion features), allowing free locomotion and 3D-object manipulation. This research proposes a novel deep fusion network to predict cybersickness severity from heterogeneous data readily available from the integrated HMD sensors. We extracted 1755 stereoscopic videos, eye-tracking, and head-tracking data along with the corresponding self-reported cybersickness severity collected from 30 participants during their VR gameplay. We applied several deep fusion approaches with the heterogeneous data collected from the participants. Our results suggest that cybersickness can be predicted with an accuracy of 87.77% and a root-mean-square error of 0.51 when using only eye-tracking and head-tracking data. We concluded that eye-tracking and head-tracking data are well suited for a standalone cybersickness prediction framework.

This article investigates how experiencing Virtual Reality (VR) Disability Simulation (DS) affects information recall and participants' implicit association towards people with disabilities (PwD). Implicit attitudes are our actions or judgments towards various concepts or stereotypes (e.g., race) which we may or may not be aware of. Previous research has shown that experiencing ownership over a dark-skinned body reduces implicit racial bias. We hypothesized that a DS with a tracked Head Mounted Display (HMD) and a wheelchair interface would have a significantly larger effect on participants' information recall and their implicit association towards PwD than a desktop monitor and gamepad. We conducted a 2 x 2 between-subjects experiment in which participants experienced a VR DS that teaches them facts about Multiple Sclerosis (MS) with factors of display (HMD, a desktop monitor) and interface (gamepad, wheelchair). Participants took two Implicit Association Tests before and after experiencing the DS. Our study results show that the participants in an immersive HMD condition performed better than the participants in the non-immersive Desktop condition in their information recall task. Moreover, a tracked HMD and a wheelchair interface had significantly larger effects on participants' implicit association towards PwD than a desktop monitor and a gamepad.

High-fidelity visual simulation-based environments and advanced learning algorithms can be used to train robots to carry out specific tasks. Behavior cloning is a fast and easy way to train robots to learn from experience by modeling their actions according to human actions. As we make use of these agents in our day-to-day life, the robustness of such system-of-systems trained on simulation environments are of great concern. In this paper, we explore adversarial attacks in simulation environments, specifically for behavioral cloning models that cause the adversary to be able to take control of the steering mechanism of an autonomous agent. We focus our attention on improving latency and noticeability, two fundamental issues with adversarial attacks, by reducing the number of iterations to a single step during a white-box adversarial attack within a noticeability threshold. More specifically, the gradients at the image input layer and the output layer of the neural network are utilized in the adversarial attack. We implement a hybridized version of the fast gradient sign and basic iterative methods to attack the input image and fool the agent. We've shown that our method reduces the attack time per frame to within 3 milliseconds.

Cybersickness is one of the primary challenges to the usability and acceptability of virtual reality (VR). Cybersickness can cause motion sickness-like discomforts, including disorientation, headache, nausea, and fatigue, both during and after the VR immersion. Prior research suggested a significant correlation between physiological signals and cybersickness severity, as measured by the simulator sickness questionnaire (SSQ). However, SSQ may not be suitable for automatic detection of cybersickness severity during immersion, as it is usually reported before and after the immersion. In this study, we introduced an automated approach for the detection and prediction of cybersickness severity from the user's physiological signals. We collected heart rate, breathing rate, heart rate variability, and galvanic skin response data from 31 healthy participants while immersed in a VR roller coaster simulation. We found a significant difference in the participants' physiological signals during their cybersickness state compared to their resting baseline. We compared a support vector machine classifier and three deep neural classifiers for cybersickness severity detection and prediction in two minutes' future, given the previous two minutes of physiological signals. Our proposed simplified convolutional long short-term memory classifier achieved an accuracy of 97.44% for detecting current cybersickness severity and 87.38% for predicting future cybersickness severity from the physiological signals.

Predicting driver's cognitive states using deep learning from electroencephalography (EEG) signals is considered this paper. To address the challenge posed by limited labeled training samples, a semi-supervised Wasserstein Generative Adversarial Network with gradient penalty (sWGAN-GP) is proposed. The proposed sWGAN-GP includes a classifier with the shared architecture with the discriminator in GAN and its loss function enables the augmentation of limited training samples with generated EEG samples during training, thus resulting in improved classification performance. The several modeling challenges including frequency artifacts and training instability, are also considered. The test results on predicting the alert and drowsy states from a simulated driving experiment demonstrate improved prediction performance and training stability over the baseline semi-supervised GAN and a convolutional neural network model.

Electroencephalography (EEG) data is difficult to obtain due to complex experimental setups and reduced comfort due to prolonged wearing. This poses challenges to train powerful deep learning model due to the limited EEG data. Hence, being able to generate EEG data computationally is highly desirable. We propose a novel Conditional Wasserstein Generative Adversarial Network with gradient penalty (cWGAN-GP) that can be trained to synthesize EEG data for different cognitive events. This network addresses several modeling challenges, including frequency artifacts and training instability. The proposed GAN model is tested to generate one channel EEG data for the rapid serial visual presentation. We demonstrated the validity of the generated samples using several evaluation metrics and show that the synthesized EEG data can augment the real EEG data to achieve improved event classification performance.

The objective of this paper is to explore three different interaction methods in a confirmation task on a head-mounted Augmented Reality (AR) device with a population of children aged 9-11 years. The three interaction methods we look at are voice recognition, gesture recognition, and controller. We conducted a within-subjects study using a Fitts’ Law confirmation task performed by children with a Microsoft HoloLens. We measured elapsed time during the completion of the tasks. Also, we collected usability and fatigue measures using the System Usability Scale and the OMNI RPE (Ratings of Perceived Exertion) scale. We found significant differences between voice and controller for time, fatigue and usability. We also found significant differences between gesture and controller for time, fatigue and usability. We hope to apply the results of this study to improve augmented reality educational tools for children in the future.

This study investigates and compares brain signals between persons with and without Multiple Sclerosis (MS) when exposed to cybersickness-provoking Virtual Reality (VR). Cybersickness is a set of discomforts and commonly triggered by VR exposure. It has symptoms similar to motion sickness, such as dizziness, nausea, and disorientation etc. Although cybersickness has been studied for decades, populations with neurological disabilities, such as MS, have remained minimally studied. Cybersickness could have negative impact on effectiveness of VR-based rehabilitation systems and limit the accessibility of VR for persons with disabilities. MS can disrupt communication between neurons (signal carrying nerve cells) from different areas of the brain. Cybersickness also can affect brain signals, for example, frequency powers may change due to cybersickness. This study investigates the combination of MS and cybersickness in terms of brain signals. To investigate the effect of cybersickness on participants' brain signals, electroencephalogram (EEG) data were recorded before, during and after exposure to a cybersickness-provoking VR driving simulation. The EEG data suggests that in response to cybersickness-provoking VR exposure, participants with MS have mostly shown similar changes in brain activity with different magnitudes than participants without MS. Also, for at least one scalp location we have found completely opposite brain signals in MS-Group when compared to Non-MS-Group. Difference in magnitude or completely different trend in brain signals can imply that cybersickness affects persons with MS differently than persons without MS and may be different cybersickness reduction techniques are required for different populations.

Disability Simulation (DS) is an approach used to modify attitudes regarding people with disabilities. DS places people without disabilities in situations that are designed for the users to experience a disability. In this research we investigate reverse disability simulation (RDS) in a virtual reality environment. In a RDS people with disabilities perform tasks that are made easier in the virtual environment compared to the real world. We hypothesized that putting people with disabilities in a RDS will increase confidence and enable efficient task completion. To investigate this hypothesis, we conducted a within-subjects experiment in which participants performed a virtual “kicking a ball” task in two different conditions: a normal condition without RDS (i.e., same difficulty as in the real world) and an easy condition with RDS (i.e., physically easier than the real world but visually the same). The results from our study suggest that RDS increased participants' confidence.

The objective of this study is to investigate how different visual components of Virtual Reality (VR), such as field of view, frame rate, and display resolution affect postural stability in VR. Although previous studies identified these visual components as some of the primary factors that differ significantly in VR from reality, the effect of each component on postural stability in VR is yet unknown. While most people experience postural instability in VR, it is worse for people with balance impairments (BIs). This is likely because they depend more on their visual cues to maintain postural stability. Therefore, we conducted a within-subject study with ten people with balance impairments due to Multiple Sclerosis (MS). In each condition, we varied one component and kept all other components fixed. Each participant explored the virtual environment (VE) in a controlled fashion to make sure that the effect of the visual components was consistent for all participants. Results from our study suggest that decreased field of view and frame rate have significant effects on postural stability, but the effect of display resolution is inconclusive. Therefore, VR systems targeting people with balance impairments should focus on improving field of view and frame rate rather than display resolution.

Joint attention is critical to the education and development of a child. Deficits in joint attention are considered by many researchers to be an early predictor of children with Autism Spectrum Disorder (ASD). Training of joint attention have been a significant topic in ASD intervention education research. We propose a novel joint attention training approach using a Customizable Virtual Human (CVH) and a Virtual Reality (VR) game to assist with joint attention training. Previous work has shown that CVHs potentially help the users with ASD to increase their performance in hand-eye coordination, motivate the users to play longer, as well as improve user experience in a training game. Based upon these discovered CVH benefits, we hypothesize that CVHs may also be beneficial in training joint attention for users with ASD. To test our hypothesis, we developed a CVH with customizable facial features in an educational game - Imagination Drums - and conducted a user study on adolescents with high functioning ASD to investigate the effects of CVHs. We collected users' eye-gaze data and task performance during the game to evaluate the users' joint attention with CVHs and the effectiveness of CVHs compared with Non-Customizable Virtual Humans (NCVHs). The study results showed that the CVH make the participants gaze less at the irrelevant area of the game's storyline (i.e. background), but surprisingly, also provided evidence that participants react slower to the CVH's joint attention bids, compared with NCVH. Overall, the study reveals insights of how users with ASD interact with CVHs and how these interactions affect joint attention.

The purpose of this paper is to investigate a concept called virtual ability simulation (VAS) for people with disability in a virtual reality (VR) environment. In a VAS people with disabilities perform tasks that are made easier in the virtual environment (VE) compared to the real world. We hypothesized that putting people with disabilities in a VAS will increase confidence and enable more efficient task completion than without a VAS. To investigate this hypothesis, we conducted a within-subjects experiment in which participants performed a virtual task called "kick the ball" in two different conditions: a no gain condition (i.e., same difficulty as in the real world) and a rotational gain condition (i.e., physically easier than the real world but visually the same). The results from our study suggest that VAS increased participants' confidence which in turn enables them to perceive the difficulty of the same task easier.

The objective of this study is to investigate how different visual components of Virtual Reality (VR), such as field of view, frame rate, and display resolution affect postural stability in VR. Although previous studies identified these visual components as some of the primary factors that differ significantly in VR from reality, the effect of each component on postural stability is yet unknown. While most people experience postural instability in VR, it is worse for people with balance impairments (BIs). This may be because they depend more on their visual cues to maintain postural stability. We conducted a study with ten people with balance impairments due to Multiple Sclerosis (MS) and seven people without balance impairments to investigate the effect of different visual components on postural stability. In each condition, we varied one of the visual components and kept all other components fixed. Each participant explored the virtual environment (VE) in a controlled fashion to make sure that the effect of the visual components was consistent for all participants. Results from our study suggest that for people with BIs, decreased field of view and decreased frame rate have significant negative effects on postural stability, but the effect of display resolution is inconclusive. However, for people without BIs, there were no significant differences for any of the visual components. Therefore, VR systems targeting people with balance impairments should focus on improving field of view and frame rate before improving display resolution.

The objective of this research is to compare the effectiveness of different tracking devices underwater. There have been few works in aquatic virtual reality (VR) - i.e., VR systems that can be used in a real underwater environment. Moreover, the works that have been done have noted limitations on tracking accuracy. Our initial test results suggest that inertial measurement units work well underwater for orientation tracking but a different approach is needed for position tracking. Towards this goal, we have waterproofed and evaluated several consumer tracking systems intended for gaming to determine the most effective approaches. First, we informally tested infrared systems and fiducial marker based systems, which demonstrated significant limitations of optical approaches. Next, we quantitatively compared inertial measurement units (IMU) and a magnetic tracking system both above water (as a baseline) and underwater. By comparing the devices rotation data, we have discovered that the magnetic tracking system implemented by the Razer Hydra is more accurate underwater as compared to a phone-based IMU. This suggests that magnetic tracking systems should be further explored for underwater VR applications.

The purpose of this paper is to investigate the effect of the sense of presence on one aspect of learning, information recall, in an immersive virtual reality (VR) disability simulation. Previous research has shown that the use of VR technology in education may facilitate improved learning outcomes, however, it is still an active research topic as the learning outcomes can vary widely. We hypothesized that a higher level of immersion and involvement in a VR disability simulation that leads to a high sense of presence will help the user improve information recall. To investigate this hypothesis, we conducted a between subjects experiment in which participants were presented information about multiple sclerosis in different immersive conditions and afterwards they attempted to recall the information. We also looked into whether there is any adverse effect of cybersickness on the information recall task in our disability simulation. The results from our study suggest that participants who were in immersive conditions were able to recall the information more effectively than the participants who experienced a non-immersive condition.

The objective of this research is to improve the accessibility of Head-Mounted Displays (HMDs) for users with balance impairments while they are in immersive Virtual Environments (VEs). Previous research has shown that most users experience some imbalance in a fully immersive VE. However, this imbalance is significantly worse in users with balance deficits. Thus, this research aims to determine an effective visual feedback technique to improve balance of persons while using VEs to improve the accessibility of HMDs. In order to do that, we conducted a study with seven users without impairment and seven users with balance impairments due to Multiple Sclerosis (MS). We investigated how a static reference frame (SRF) (e.g., a cross-hair always rendered in the same position on the user's display screen) impacts the participants' balances in VR. Results indicate that a SRF significantly improves balance in VR for users with MS. Based on these results, we propose guidelines for designing more accessible VEs for persons with balance impairments.

Cybersickness is commonly experienced by the users in immersive Virtual Environments (VE). It has symptoms similar to Motion Sickness, such as dizziness, nausea etc. Although there have been many cybersickness experiments conducted with persons without disabilities, persons with disabilities, such as Multiple Sclerosis (MS), have been minimally studied. This is an important area of research because cybersickness could have negative effects on virtual rehabilitation effectiveness and the accessibility of VEs. For this experiment, we recruited 16 participants - 8 persons with MS and 8 persons without MS from similar demographics (e.g. age, race). Two participants from population without MS could not complete the experiment due to severe cybersickness. We asked each participant to experience a VE. We collected Galvanic Skin response (GSR) data before and during VR exposure; GSR is commonly used as an objective measure of cybersickness. Also, Simulator Sickness Questionnaire (SSQ) feedback was recorded before and after the experiment. SSQ results show that the VE induced cybersickness in the participants. The GSR data suggests that the cybersickness may have induced similar physiological changes in participants with MS as participants without MS, albeit with greater variability in participants without MS. However, participants with MS had significantly lower GSR during VR exposure. In this paper, we compare the effects of cybersickness between the people with MS and the people without MS with respect to SSQ score and GSR data.

Latency (i.e., time delay) in a virtual environment is known to disrupt user performance, presence and induce simulator sickness. Thus, with emerging use of virtual rehabilitation, the target populations' latency perception thresholds need to be considered to fully understand and possibly control the implications of latency in a Virtual Rehabilitation environment. We present a study that quantifies the latency discrimination thresholds of a yet untested population-a specific subset of mobility impaired participants where participants suffer from Multiple Sclerosis-and compare the results to a control group of healthy participants. The study was modeled after previous latency discrimination research and shows significant differences in latency perception between the two populations with MS participants showing lower sensitivity to latency than healthy participants.

Training games have many potential benefits for autism spectrum disorder (ASD) intervention, such as increasing motivation and improving the abilities of performing daily living activities, due to their ability to simulate real world scenarios. A more motivating game may stimulate users to play the game more, and it may also result in users performing better in the game. Incorporating users' interests into the game could be a good way to build a motivating game, especially for users with ASD. We propose a Customizable Virtual Human (CVH) which enables users with ASD to easily customize a virtual human and then interact with the CVH in a 3D interaction task. Previous work has shown that users with ASD may have less efficient hand-eye coordination in performing 3D interaction tasks than users without ASD. We developed a hand-eye coordination training game - Imagination Soccer - and presented a user study on adolescents with high functioning ASD to investigate the effects of CVHs. We compare the differences of participants' 3D interaction performances, game performances and user experiences (i.e. presence, involvement, and flow) under CVH and Non-customizable Virtual Human (with randomly generated appearances) conditions. As expected, the results indicated that for users with ASD, CVHs could effectively motivate them to play the game more, and offer a better user experience. Surprisingly, results also showed that the CVHs improved performance in the hand-eye-coordination task users had higher success rate and blocked more soccer balls with the CVH than with a non-customizable virtual human.

Virtual Reality (VR) training games have many potential benefits for autism spectrum disorder (ASD) therapy, such as increasing motivation and improving the abilities of performing daily living activities. Persons with ASD often have deficits in hand-eye coordination, which makes many activities of daily living difficult. A VR game that trains hand-eye coordination could help users with ASD improve their quality of life. Moreover, incorporating users' interests into the game could be a good way to build a motivating game for users with ASD. We propose a Customizable Virtual Human (CVH) which enables users with ASD to easily customize a virtual human and then interact with the CVH in a 3D task. Specifically, we investigated the effects of CVHs with a VR hand-eye coordination training game — Imagination Soccer — and conducted a user study on adolescents with high functioning ASD. We compared the differences of participants' 3D interaction performances, game performances and user experiences (i.e. presence, involvement, and flow) under CVH and Non-customizable Virtual Human (NCVH) conditions. The results indicate that CVHs could effectively improve performance in 3D interaction tasks (i.e., blocking a soccer ball) for users with ASD, motivate them to play the game more, and offer a better user experience.

We present a novel underwater VR game - Shark Punch - in which the user must fend off a virtual Great White shark with real punches in a real underwater environment. This poster presents our underwater VR system and our iterative design process through field tests with a user with disabilities. We conclude with proposed usability, accessibility, and system design guidelines for future underwater VR rehabilitation games.

The long term objective of this research is to enable persons with disabilities to play VR games during aquatic rehabilitation [2] sessions. Unfortunately, there have been only a few research projects that have explored VR underwater use [3][1][4], and none of these have addressed rehabilitation. This research focuses on understanding and mitigating the associated usability, accessibility, and system design issues for underwater VR rehabilitation games. Specifically, we present a novel underwater VR game — Shark Punch — in which the user must fend off a virtual Great White shark with real punches in a real underwater environment. This video presents our underwater VR system and our iterative design process through field tests with a disabled user. We conclude with proposed usability, accessibility, and system design guidelines for future underwater VR rehabilitation games.

Latency (i.e., time delay) in a Virtual Environment is known to disrupt user performance, presence and induce simulator sickness. However, can we utilize the effects caused by experiencing latency to benefit virtual rehabilitation technologies? We investigate this question by conducting an experiment that is aimed at altering gait by introducing latency applied to one side of a self-avatar with a front-facing mirror. This work was motivated by previous findings where participants altered their gait with increasing latency, even when participants failed to notice considerably high latencies as 150ms or 225ms. In this paper, we present the results of a study that applies this novel technique to average healthy persons (i.e., to demonstrate the feasibility of the approach before applying it to persons with disabilities). The results indicate a tendency to create asymmetric gait in persons with symmetric gait when latency is applied to one side of their self-avatar. Thus, the study shows the potential of applying one-sided latency in a self-avatar, which could be used to develop asymmetric gait rehabilitation techniques.

Abstract Virtual environments (VEs) have been shown to be beneficial in physical rehabilitation, increasing motivation and the range of exercises that can be safely performed. However, little is known about how disabilities may impact a user's responses to a VE, which could affect rehabilitation motivation. Thus, the primary objective of this research is to understand how VEs affect users with mobility impairments (MI). Specifically, we investigate the influence of full body avatars that have canes. To begin investigating this, we designed a VE that included a range of multimodal feedback to induce a strong sense of presence and was novel to the participants. Using this VE, we conducted a study with two different populations: eight persons with MI and eight healthy persons as a control. The healthy participants were of similar demographics (e.g., age, weight, height, and previous VE experience) to the participants with MI who walked with a cane (i.e., on the basis of strict selection criteria to maintain homogeneity). This is one of the first studies to investigate how a VE can affect the gait of the users with MI, physiological response, presence, behavior, and the influence of avatars. Results of the study suggest generalizable guidelines for the design of VEs for users with MI. Copyright © 2014 John Wiley & Sons, Ltd.

Most literature on the usability of 3D user interfaces (3DUI) in ASD therapy consists of a series of case studies based on games for rehabilitation. These games have been largely successful. However, it is difficult to generalize the results of these specific case studies. The usability of atomic 3DUI interactions (e.g., rotation, translation) with respect to adolescents with ASD has not yet been evaluated. Adolescents with ASD often have enhanced spatial cognitive abilities and less efficient hand-eye coordination. Our main research question is "Do adolescents with ASD perform 3DUI tasks differently than typically developed adolescents and if so, why?" To address this question, we present a matched pair user study including adolescents without ASD (i.e. as controls) paired with adolescents who had a high ASD severity score, but were still considered high functioning. Our results give insight into the usability of 3DUI for adolescents with ASD and provide generalizable guidelines for future 3DUI applications for children with autism.

Previous research on healthy subjects showed that a higher sense of presence can be elicited through full body avatars versus no avatar. However, minimal avatar research has been conducted with persons with mobility impairments. For these users, Virtual Environments (VEs) and avatars are becoming more common as tools for rehabilitation. If we can maximize presence in these VEs, users may be more effectively distracted from the pain and repetitiveness of rehabilitation, thereby increasing users' motivation. To investigate this we replicated the classic virtual pit experiment and included a responsive full body avatar (or lack thereof) as a 3D user interface. We recruited from two different populations: mobility impaired persons and healthy persons as a control. Results give insight into many other differences between healthy and mobility impaired users' experience of presence in VEs.

Background motion and visual clutter are present in almost all augmented reality applications. However, there is minimal prior work that has investigated the effects that background motion and clutter (e.g., a busy city street) can have on the perception of virtual object motion in augmented reality. To investigate these issues, we conducted an experiment in which participants' perceptions of changes in overlaid virtual object velocity were evaluated on a black background and a high clutter/motion background. Results offer insights into the impact that background clutter and motion has on perception in augmented reality.

Latency and avatars in Virtual Environments have been extensively studied over the years. However, there has been minimal research conducted on the effects of latency and avatars for mobility impaired users. To address this, we have conducted a study involving both healthy and mobility impaired participants with the simple task of walking across a simulated room under various latency and avatar conditions. We investigated the impact of latency and avatars on perceived latency and gait parameters. The results suggest that mobility impaired persons react to latency and the presence of an avatar differently than healthy users.

We are investigating if/how Mobility Impaired (MI) persons and healthy persons respond differently to Virtual Environments (VE). Previous research on healthy users has investigated a VE's effects on presence, gait (i.e., walking patterns), and physiological responses (e.g., heart rate). However, almost all of the previous research studies have been conducted only with healthy persons. Thus, it very little is known about how MI persons respond to a VE physiologically, how a VE will affect their gait (i.e., walking patterns), or how their sense of presence may differ from healthy persons. To begin investigating this, we designed a VE that included a range of multimodal feedback to induce a strong sense of presence and was novel to the participants. Using this VE, we conducted a study with two different populations: 8 MI persons and 8 healthy persons. The healthy participants were of similar demographics (e.g., age, weight, height) to the MI participants. The MI population was symptomatically homogeneous (e.g., they all walked with canes) and no participants had cognitive impairment. This is one of the first studies to investigate how a VE can affect MI users' gait, physiological response, and presence.

Using Microsoft Kinect as a whole body motion tracking system and 3D user interface, we developed two exercise-based 3D interaction techniques for a car racing game - a genre of game traditionally unrelated to physical exercise. Interaction (i.e., control of car acceleration) is enabled through the use of two real exercises 1) a half crouching exercise, and 2) a crouching and rising exercise. In two within subjects user studies, with 27 participants in half crouching and another 30 participants in crouching and rising respectively, we compared exercises in-game to the same exercises without the game, focusing on the physiological and motivational impact of mapping exercises as interfaces to games.

There are many guidelines for composing programs, but few methodologies take into account the expectations readers have when reading an unfamiliar program. As a result, code that seems well-written and clear to the developer might not be read and interpreted by the reader in the way the programmer expected. We conducted a survey of Java programmers to determine how a program's structure may signal subjective cues to the reader. We found that the use of meaning-preserving program refactorings had a measurable effect on what readers believed the programmer's main intention was.

Provides an abstract for each of the tutorial presentations and a brief professional biography of each presenter. The complete presentations were not made available for publication as part of the conference proceedings.

This paper proposes collocated After Action Review (AAR) of training experiences. Through Mixed Reality (MR), collocated AAR allows users to review past training experiences in situ with the user’s current, real-world experience. MR enables a user-controlled egocentric viewpoint, a visual overlay of virtual information, and playback of recorded training experiences collocated with the user’s current experience. Collocated AAR presents novel challenges for MR, such as collocating time, interactions, and visualizations of previous and current experiences. We created a collocated AAR system for anesthesia education, the Augmented Anesthesia Machine Visualization and Interactive Debriefing system (AAMVID). The system was evaluated in two studies by students (n=19) and educators (n=3). The results demonstrate how collocated AAR systems such as AAMVID can: (1) effectively direct student attention and interaction during AAR and (2) provide novel visualizations of aggregate student performance and insight into student understanding for educators.

Mixed reality's (MR) ability to merge real and virtual spaces is applied to merging different knowledge types, such as abstract and concrete knowledge. To evaluate whether the merging of knowledge types can benefit learning, MR was applied to an interesting problem in anesthesia machine education. The virtual anesthesia machine (VAM) is an interactive, abstract 2D transparent reality simulation of the internal components and invisible gas flows of an anesthesia machine. It is widely used in anesthesia education. However when presented with an anesthesia machine, some students have difficulty transferring abstract VAM knowledge to the concrete real device. This paper presents the augmented anesthesia machine (AAM). The AAM applies a magic-lens approach to combine the VAM simulation and a real anesthesia machine. The AAM allows students to interact with the real anesthesia machine while visualizing how these interactions affect the internal components and invisible gas flows in the real world context. To evaluate the AAM's learning benefits, a user study was conducted. Twenty participants were divided into either the VAM (abstract only) or AAM (concrete+abstract) conditions. The results of the study show that MR can help users bridge their abstract and concrete knowledge, thereby improving their knowledge transfer into real world domains.

For simulating hands-on tasks, the ease of enabling two-handed interaction with virtual objects gives Mixed Reality (MR) an expected advantage over Virtual Reality (VR). A user study examined whether two-handed interaction is critical for simulating hands-on tasks in MR. The study explored the effect of one- and two-handed interaction on task performance in a MR assembly task. When presented with a MR system, most users chose to interact with two hands. This choice was not affected by a user's past VR experience or the quantity and complexity of the real objects with which users interacted. Although two-handed interaction did not yield a significant performance improvement, two hands allowed subjects to perform the virtual assembly task similarly to the real-world task. Subjects using only one hand performed the task fundamentally differently, showing that affording two-handed interaction is critical for training systems.

A method is presented to rapidly incorporate real objects into virtual environments using laser scanned 3D models with color-based marker tracking. Both the real objects and their geometric models are put into a mixed environment (ME). In the ME, users can manipulate the scanned, articulated real objects, such as tools, parts, and physical correlates to complex computer-aided design (CAD) models. Our aim is to allow engineering teams to effectively conduct hands-on assembly design verification. This task would be simulated at a high degree of fidelity, and would benefit from the natural interaction afforded by a ME with many specific real objects.