Numerous studies have investigated the effects of system fidelity as a whole on one’s total sense of presence in virtual reality (VR). The Fidelity-based Presence Scale (FPS), a recently introduced presence questionnaire, provides a method for investigating the effects of different system fidelities (interaction, scenario, and display) on different aspects of one’s sense of presence. In this paper, we present one of the first studies to investigate those effects for a locomotion task by conducting a 2 × 2 × 2 within-subjects experiment that reveals insight on how the components of system fidelity affect sense of presence. Like recent research, our results indicate that interaction fidelity and display fidelity significantly affect one’s interaction presence and display presence, respectively. However, unlike prior work, we did not find that changes in scenario fidelity significantly affected one’s scenario presence. We discuss other results and the possible implications of this research.

Within the virtual reality (VR) research community, there have been several efforts to develop questionnaires with the aim of better understanding the sense of presence. Despite having numerous surveys, the community does not have a questionnaire that informs which components of a VR application contributed to the sense of presence. Furthermore, previous literature notes the absence of consensus on which questionnaire or questions should be used. Therefore, we conducted a Delphi study, engaging presence experts to establish a consensus on the most important presence questions and their respective verbiage. We then conducted a validation study with an exploratory factor analysis (EFA). The efforts between our two studies led to the creation of the Fidelity-based Presence Scale (FPS). With our consensus-driven approach and fidelity-based factoring, we hope the FPS will enable better communication within the research community and yield important future results regarding the relationship between VR system fidelity and presence.

Augmented reality (AR) and virtual reality (VR) can enhance task guidance by overlaying visual information to improve efficiency and reduce errors. However, challenges remain in designing the appropriate presentation format and amount of information for real-time assistance. Prior research has shown benefits of visual cues in procedural tasks, but these findings are limited to simplified scenarios, highlighting a gap in understanding their effectiveness for complex, real-world applications. Therefore, we study visual design and cue effectiveness in the context of compound procedures encompassing subtasks and heterogeneous instructions. We present an experiment assessing different visual cues in VR to test a user’s ability to harness distinct information streams for different tasks, separating cues for object search and object placement for multi-step procedures. The results show that even for compound tasks requiring processing of multiple types of information, the addition of simple interaction cues for individual subtasks did significantly improved task performance for both time and errors. However, in contrast to prior studies showing successful precueing of future steps in more simplistic tasks, the study did not find evidence of precueing with the more complex tasks.

Augmented reality (AR) and virtual reality (VR) applications commonly employ interaction cues that denote to the user what interaction to take. In this paper, we present a Method for Analyzing the Grammar of Incomplete Cues (MAGIC), which provides an approach for evaluating the design of interaction cues based on the completeness or incompleteness of the functional grammar that they convey through perceptual stimuli. To demonstrate the importance of complete cues, we also present a user study investigating the effects of complete and incomplete cues on which interactions participants choose. The results indicate that incomplete cues do not afford sufficient information, so users make assumptions about the intended interactions. Furthermore, the results indicate that users are more likely to choose intended interactions when the cues are complete. Hence, we present MAGIC as a potentially useful tool for helping interaction designers avoid usability issues with incomplete interaction cues.

Pilots operating modern cockpits often face high cognitive demands due to complex interfaces and multitasking requirements, which can lead to overload and decreased performance. This study introduces AdaptiveCoPilot, a neuroadaptive guidance system that adapts visual, auditory, and textual cues in real time based on the pilot’s cognitive workload, measured via functional Near-Infrared Spectroscopy (fNIRS). A formative study with expert pilots (N=3) identified adaptive rules for modality switching and information load adjustments during preflight tasks. These insights informed the design of AdaptiveCoPilot, which integrates cognitive state assessments, behavioral data, and adaptive strategies within a context-aware Large Language Model (LLM). The system was evaluated in a virtual reality (VR) simulated cockpit with licensed pilots (N=8), comparing its performance against baseline and random feedback conditions. The results indicate that the pilots using AdaptiveCoPilot exhibited higher rates of optimal cognitive load states on the facets of working memory and perception, along with reduced task completion times. Based on the formative study, experimental findings, qualitative interviews, we propose a set of strategies for future development of neuroadaptive pilot guidance systems and highlight the potential of neuroadaptive systems to enhance pilot performance and safety in aviation environments.

Visually searching for objects is an everyday task. In many contexts, people must visually search for multiple objects at the same time while avoiding distractor objects, such as triage during a mass casualty incident. While many prior augmented reality (AR) and virtual reality (VR) studies have investigated cues to aid in visual search tasks, few have investigated cues in contexts involving multiple targets and distractors with a full 360° effective field of regard (EFOR). Individually, multiple targets, distractors, and a full 360° EFOR each add complexity to visual search; when combined, they compound the difficulty even further. In this paper, we present such a study that compares three common types of visual cues (2D Wedge, 3D Arrow, and Gaze Line) to a baseline condition with no cueing for a 360° visual search task. Our results reinforce the importance of providing some type of cue, with the Gaze Line design being particularly beneficial. We discuss the potential implications of these findings for designing cues specifically for such complex visual search tasks.

Fidelity describes how closely a replication resembles the original. It can be helpful to analyze how faithful interactions in virtual reality (VR) are to a reference interaction. In prior research, fidelity has been restricted to the simulation of reality—also called realism. Our definition includes other reference interactions, such as superpowers or fiction. Interaction fidelity is a multilayered concept. Unfortunately, different aspects of fidelity have either not been distinguished in scientific discourse or referred to with inconsistent terminology. Therefore, we present the Interaction Fidelity Model (IntFi Model). Based on the human-computer interaction loop, it systematically covers all stages of VR interactions. The conceptual model establishes a clear structure and precise definitions of eight distinct components. As a communication tool, it helps teams to understand and discuss fidelity in VR. It was reviewed through workshops with fourteen VR experts. We provide guidelines, diverse examples, and educational material to apply the IntFi Model universally to any VR experience and propose foundational research opportunities.

Communication in collaboration, especially synchronous, remote communication, is crucial to the success of task-specific goals. Insufficient or excessive forms of communication may lead to detrimental effects on task performance while increasing mental fatigue. However, identifying which combinations of communication modalities provide the most efficient transfer of information in collaborative settings will greatly improve collaboration. To investigate this, we developed a remote, synchronous, asymmetric VR collaborative assembly task application, where users play the role of either mentor or mentee, and were exposed to different combinations of three communication modalities: voice, gestures, and gaze. Through task-based experiments with 25 pairs of participants (50 individuals), we evaluated quantitative and qualitative data and found that gaze did not differ significantly from multiple combinations of communication modalities. Our qualitative results indicate that mentees experienced more difficulty and frustration in completing tasks than mentors, with both types of users preferring all three modalities to be present.

Recently, much work has been done to research personal identifiability of extended reality (XR) users. Many of these prior studies are task-specific and involve identifying users completing a specific XR task. On the other hand, some studies have been domainspecific and focus on identifying users completing different XR tasks from the same domain, such as watching 360° videos or assembling structures. In this paper, we present one of the few studies to investigate cross-domain identification (i.e., identifying users completing XR tasks from different domains). To facilitate our investigation, we used open-source datasets from two different virtual reality (VR) studies-one from an assembly domain and one from a gaming domain-to investigate the feasibility of cross-domain gender identification, as personal identification is not possible between these datasets. The results of our machine learning experiments clearly demonstrate that cross-domain gender identification is more difficult than domain-specific gender identification. Furthermore, our results indicate that head position is important for gender identification and demonstrate that the k-nearest neighbors (kNN) algorithm is not suitable for cross-domain gender identification, which future researchers should be aware of.

Matching avatar characteristics to a user can impact sense of embodiment (SoE) in YR. However, few studies have examined how participant demographics may interact with these matching effects. We recruited a diverse and racially balanced sample of 78 participants to investigate the differences among participant groups when embodying both demographically matched and unmatched avatars. We found that participant ethnicity emerged as a significant factor, with Asian and Black participants reporting lower total SoE compared to Hispanic participants. Furthermore, we found that user ethnicity significantly influences ownership (a subscale of SoE), with Asian and Black participants exhibiting stronger effects of matched avatar ethnicity compared to White participants. Additionally, Hispanic participants showed no significant differences, suggesting complex dynamics in ethnic-racial identity. Our results also reveal significant main effects of matched avatar ethnicity and gender on SoE, indicating the importance of considering these factors in VR experiences. These findings contribute valuable insights into understanding the complex dynamics shaping VR experiences across different demographic groups.

In many consumer virtual reality (VR) applications, users embody predefined characters that offer minimal customization options, frequently emphasizing storytelling over user choice. We explore whether matching a user's physical characteristics, specifically ethnicity and gender, with their virtual self-avatar affects their sense of embodiment in VR. We conducted a $2\times 2$ within-subjects experiment ($\mathrm{n}=32$) with a diverse user population to explore the impact of matching or not matching a user's self-avatar to their ethnicity and gender on their sense of embodiment. Our results indicate that matching the ethnicity of the user and their self-avatar significantly enhances sense of embodiment regardless of gender, extending across various aspects, including appearance, response, and ownership. We also found that matching gender significantly enhanced ownership, suggesting that this aspect is influenced by matching both ethnicity and gender. Interestingly, we found that matching ethnicity specifically affects self-location while matching gender specifically affects one's body ownership.

Recently, there has been much interest in using virtual reality (VR) tracking data to authenticate or identify users. Most prior research has relied on task-specific characteristics but newer studies have begun investigating task-agnostic, domain-specific approaches. In this paper, we present one of the first systematic investigations of how different combinations of VR tracked devices (i.e., the headset, dominant hand controller, and non-dominant hand controller) and their spatial representations (i.e., position and/or rotation as Euler angles, quaternions, or 6D) affect identification accuracy for domain-specific approaches. We conducted a user study $( n =45)$ involving participants learning how to assemble two distinct full-scale constructions. Our results indicate that more tracked devices improve identification accuracies for the same assembly task, but only headset features afford the best accuracies across the domain-specific tasks. Our results also indicate that spatial features involving position and any rotation yield better accuracies than either alone.

Virtual humans can be used to deliver persuasive arguments; yet, those with synthetic text-to-speech (TTS) have been perceived less favorably than those with recorded human speech. In this paper, we investigate standard concatenative TTS and more advanced neural TTS. We conducted a 3x2 between-subjects experiment (n=79) to evaluate the effect of a virtual human’s speech fidelity at three levels (Standard TTS, Neural TTS, and Human speech) and the listener’s gender (male or female) on perceptions and persuasion. We found that the virtual human was perceived as significantly less trustworthy by both genders, if they used neural TTS compared to human speech, while male listeners (but not females) also perceived standard TTS as less trustworthy than human speech. Our findings indicate that neural TTS may not be an effective choice for persuasive virtual humans and that gender of the listener plays a role in how virtual humans are perceived.

In this paper, we compare the performance of a canonical genetic algorithm (CGA), the Self Adaptive Genetic Algorithm (SAGA), and a feed-forward neural network (FFNN) on a predictive modeling problem with incomplete data. Predictive modeling involves learning relationships between the features and labels of the data points in a dataset. Datasets with missing input values may cause problems for some learning algorithms by biasing the learned models. Imputation refers to techniques for replacing missing data through methods such as statistical probabilities, multivariate analysis, machine learning, or K-nearest neighbors.

Modern text-to-speech engines can be an effective speech choice for embodied virtual pedagogical agents. However, it is not known how synthesized accents influence learning outcomes and perceptions of the agent. In this paper, we conducted a between-subjects experiment (n=60) to determine the effect of a pedagogical agent’s machine synthesized text-to-speech accent (United States English or Indian English) on learning outcomes and perceptions of the agent for students in the United States. Our results indicate that learner gender interacts with synthesized speech accent to significantly affect learning outcomes and perceptions of the agent. Our results reveal that a foreign synthetic speech accent may affect the learning outcomes of female university students (n=30), but not male university students (n=30). Finally, our results indicate that learner gender interacts with synthesized speech accent to affect perceptions of the pedagogical agent’s human-likeness. We provide novel insights on the differences between male and female learners for interactions with pedagogical agents with synthetic TTS accents.

Photorealistic media aim to faithfully represent the world, creating an experience that is perceptually indistinguishable from a real world experience. In the past few years, this area has grown significantly, with new multimedia areas emerging, such as light fields, point clouds, ultra-high definition, high frame rate, high dynamic range imaging, and novel 3D audio and sound field technologies. In spite of all the advances done so far, there are several technological challenges to overcome. In particular, research in this field typically requires the use of big datasets, software tools, and powerful infrastructures. Among these, the availability of meaningful datasets, with a diverse and high-quality content, is of significant importance and, to date, most available datasets are limited and do not provide researchers adequate tools to advance the area of photorealistic applications. To help advancing research efforts in this area, this workshop aims to engage experts and researchers on the synthesis of photorealistic images and/or virtual environments, particularly in the form of public datasets, software tools, or infrastructures, for not only multimedia systems research, but also for other fields, such as machine learning, robotics, computer vision, mixed reality, and virtual reality.

Virtual reality (VR) researchers have had a long-standing interest in studying locomotion for developing new techniques, improving upon prior ones, and analyzing their effects on users. To help organize prior work, several researchers have presented taxonomies for categorizing locomotion techniques in general. More recently, researchers have begun to conduct systematic reviews to better understand what locomotion techniques have been investigated. In this paper, we present our own systematic review of locomotion techniques based on a well-established taxonomy, and we use k-means clustering to identify to what extent locomotion techniques have been explored. Our results indicate that selection-based, walking-based, and steering-based locomotion techniques have been moderately to highly explored while manipulation-based and automated locomotion techniques have been less explored. We also present results on what types of metrics have been used to evaluate locomotion techniques. While usability, discomfort, and travel performance metrics have been moderately to highly explored, other metrics, such as biometrics, user experience, and emotions, have been less explored.

Training simulations in virtual reality (VR) have become a focal point of both research and development due to allowing users to familiarize themselves with procedures and tasks without needing physical objects to interact with or needing to be physically present. However, the increasing popularity of VR training paradigms raises the question: Are VR-based training assessments accurate? Many VR training programs, particularly those focused on inspection tasks, employ simple pass or fail assessments. However, these types of assessments do not necessarily reflect the user’s knowledge.

League of Legends (LoL) is the most widely played multiplayer online battle arena (MOBA) game in the world. An important aspect of LoL is competitive ranked play, which utilizes a skill-based matchmaking system to form fair teams. However, players’ skill levels vary widely depending on which champion, or hero, that they choose to play as. In this paper, we propose a method for predicting game outcomes in ranked LoL games based on players’ experience with their selected champion. Using a deep neural network, we found that game outcomes can be predicted with 75.1% accuracy after all players have selected champions, which occurs before gameplay begins. Our results have important implications for playing LoL and matchmaking. Firstly, individual champion skill plays a significant role in the outcome of a match, regardless of team composition. Secondly, even after the skill-based matchmaking, there is still a wide variance in team skill before gameplay begins. Finally, players should only play champions that they have mastered, if they want to win games.

Recent research indicates that user tracking data from virtual reality (VR) experiences can be used to personally identify users with degrees of accuracy as high as 95%. However, these results indicating that VR tracking data should be understood as personally identifying data were based on observing 360° videos. In this paper, we present results based on sessions of user tracking data from an ecologically valid VR training application, which indicate that the prior claims may not be as applicable for identifying users beyond the context of observing 360° videos. Our results indicate that the degree of identification accuracy notably decreases between VR sessions. Furthermore, we present results indicating that user tracking data can be obfuscated by encoding positional data as velocity data, which has been successfully used to predict other user experience outcomes like simulator sickness and knowledge acquisition. These results, which show identification accuracies were reduced by more than half, indicate that velocity-based encoding can be used to reduce identifiability and help protect personal identifying data.

Target and steering-based techniques are two common approaches to travel in consumer VR applications. In this paper, we present two within-subject studies that employ a prior dual-task methodology to evaluate and compare the cognitive loads, travel performances, and simulator sickness of three common target-based travel techniques and three common steering-based travel techniques. We also present visual meta-analyses comparing our results to prior results using the same dual-task methodology. Based on our results and meta-analyses, we present several design suggestions for travel techniques based on various aspects of user experiences.

Depth perception of objects can greatly affect a user's experience of an augmented reality (AR) application. Many AR applications require depth matching of real and virtual objects and have the possibility to be influenced by depth cues. Color and luminance are depth cues that have been traditionally studied in two-dimensional (2D) objects. However, there is little research investigating how the properties of three-dimensional (3D) virtual objects interact with color and luminance to affect depth perception, despite the substantial use of 3D objects in visual applications. In this paper, we present the results of a paired comparison experiment that investigates the effects of object shape, fidelity, color, and luminance on depth perception of 3D objects in handheld mobile AR. The results of our study indicate that bright colors are perceived as nearer than dark colors for a high-fidelity, simple 3D object, regardless of hue. Additionally, bright red is perceived as nearer than any other color. These effects were not observed for a low-fidelity version of the simple object or for a more-complex 3D object. High-fidelity objects had more perceptual differences than low-fidelity objects, indicating that fidelity interacts with color and luminance to affect depth perception. These findings reveal how the properties of 3D models influence the effects of color and luminance on depth perception in handheld mobile AR and can help developers select colors for their applications.

Many research studies have investigated avatar embodiment and its effects on self-location, agency, and body ownership. Researchers have also investigated the effects of various external stimuli and avatar appearances during embodiment. However, the effects of body tracking fidelity while embodying an inverse-kinematic avatar are relatively unexplored. In this paper, we present two studies using a set of six trackers that investigate four levels of body tracking fidelity during avatar embodiment for male participants only: Complete (head, hands, feet, and pelvis trackers), Head-and-Extremities (head, hands, and feet trackers), Head-and-Hands (head and hands trackers), and No-Avatar (head and hands trackers; only controllers visible). Our results indicate that tracking the head, hands, and feet significantly increases the sense of embodiment and the sense of spatial presence when embodying an inverse-kinematic avatar for male participants.

Walking metaphors have been extensively researched for travel in virtual reality (VR) applications. However, only a few walking metaphors are feasible for most consumer VR systems. In this paper, we present a study that compares three of these suitable metaphors: Scaled Walking, Human Joystick, and Walking-In-Place. Our study empirically assesses the cognitive loads and travel performances of these three walking metaphors by employing a novel dual-task methodology. We also evaluated their effects on simulator sickness, presence, and perceived usability. The results of our study indicate that Scaled Walking afforded significantly better travel performances and perceived usability than Human Joystick and Walking-In-Place. Our results also indicate that Walking-In-Place required the worst cognitive loads and that Human Joystick induced the worst simulator sickness. Given these results, we discuss the implications of using a high-fidelity, full-gait walking metaphor.

Virtual Reality (VR) for training and education of real-world tasks has been researched extensively and has growing use in industry. The data generated by trainees in VR could be leveraged to improve the ability to evaluate learning beyond that which is possible in traditional training scenarios. In this paper, we present a machine learning approach that is able to classify users into participants with low-learning (LL) and high-learning (HL) gains, based on a knowledge test, using only the linear and angular velocities of the head-mounted display (HMD) and handheld controllers. To collect this data, we conduct a VR training user study. We demonstrate that even with a limited data set, it is possible to train a machine learning classifier to predict a trainee's learning performance for a given task with high degrees of accuracy and confidence. We investigate three different sets of velocity-based input features and two feature representations in a machine learning experiment. Our results indicate that all feature combinations resulted in high degrees of accuracy and confidence for predicting learning gains in our testing data. By employing a novel visualization technique, we were able to determine that participants with HL gains moved with greater velocities and fewer changes in direction than those with LL gains. These results indicate that it may be feasible to create VR training applications that can predict a user's learning gains and dynamically adapt the training to better support the user's learning, based on commonly available tracking data.

Authoring a collaborative, interactive Mixed Reality (MR) tour requires flexible design and development of various software modules for tasks such as managing geographically distributed participants, adaptable travel and virtual camera techniques, data logging for assessment of the incorporated techniques, as well as for evaluating the Quality of Experiences (QoE). In most cases, authors might have to develop all these software modules, instead of focusing only on the virtual environment design. In this article, we propose SCeVE, a component-based framework that supports flexible design and authoring of interactive MR tours by offering ease of access to four major design choices: (i) S ynchronization, (ii) C ollaborative e xploration, (iii) V isualization, and (iv) E valuation. Based on tour requirements, an author can access one or more components (or software libraries) of design choices via SCeVE’s API (Application Programming Interface) services , as demonstrated by the two case studies on group travel in a plant walk MR tour. SCeVE framework is innovative in the sense that it facilitates group travel in virtual environments involving “live” models of participants from geographically distributed sites. SCeVE empowers authors to focus only on the design of the required virtual environments. They can quickly build a diverse set of collaborative MR tours by utilizing the flexibility of SCeVE in terms of the various available options for traveling, rendering on multiple devices, and virtual camera viewpoint computation strategies. By providing data logs of various components, SCeVE facilitates performance evaluation of the various strategies used as well as the user experience in collaborative MR tours. SCeVE is designed in an extensible manner, allowing authors to add devices and software services as additional components.

Some of the most widely used selection techniques for extended reality (XR) are based on virtual hand interactions. Many existing XR frameworks provide this functionality by default; however, their implementation can differ in slight, but important ways. When preparing to make a selection with a virtual hand technique, a user's desired selection can potentially be ambiguous due to multiple intersections. Systems with varying underlying virtual hand implementations may yield contrasting selections due to resolving multiple intersections differently. This is particularly an issue when objects are smaller in size than the virtual hand representation and in dense environments. To demonstrate the importance of these differences, we present a virtual hand selection study comparing three methods that are currently used in popular XR frameworks for disambiguating selections: Closest Intersected, First Intersected, and Last Intersected. The results of our study show that the Closest Intersected method affords significantly faster selections, significantly fewer incorrect and missed selections, and yields significantly better effective throughput than the other two methods. These results show that using a framework's built-in selection technique can significantly affect an XR application's usability.

In this paper, we propose a taxonomy for 360° videos that categorizes videos based on moving objects and camera motion. We gathered and produced 28 videos based on the taxonomy, and recorded viewport traces from 60 participants watching the videos. In addition to the viewport traces, we provide the viewers' feedback on their experience watching the videos, and we also analyze viewport patterns on each category.

The 9th annual IEEE 3DUI Contest focuses on the development of 3D User Interfaces (3DUIs) for three different tasks in fully immersive Virtual Environments (VEs): (1) Ladder Climbing, (2) First-Person View Flying, and (3) Tower Stacking. The 3DUI Contest is part of the 2018 IEEE Conference on Virtual Reality and 3D User Interfaces held in Reutlingen, Germany. The contest is open to anyone interested in 3DUIs, from researchers to students, enthusiasts, and professionals. The purpose of the contest is to stimulate innovative and creative solutions to challenging 3DUI problems.

With the increasing accessibility of the mobile head-mounted displays (HMDs), mobile virtual reality (VR) systems are finding applications in various areas. However, mobile HMDs are highly constrained with limited graphics processing units (GPUs) and low processing power and onboard memory. Hence, VR developers must be cognizant of the number of polygons contained within their virtual environments to avoid rendering at low frame rates and inducing simulator sickness. The most robust and rapid approach to keeping the overall number of polygons low is to use mesh simplification algorithms to create low-poly versions of pre-existing, high-poly models. Unfortunately, most existing mesh simplification algorithms cannot adequately handle meshes with lots of boundaries or nonmanifold meshes, which are common attributes of many 3D models. In this article, we present QEM 4VR , a high-fidelity mesh simplification algorithm specifically designed for VR. This algorithm addresses the deficiencies of prior quadric error metric (QEM) approaches by leveraging the insight that the most relevant boundary edges lie along curvatures while linear boundary edges can be collapsed. Additionally, our algorithm preserves key surface properties, such as normals, texture coordinates, colors, and materials, as it preprocesses 3D models and generates their low-poly approximations offline. We evaluated the effectiveness of our QEM 4VR algorithm by comparing its simplified-mesh results to those of prior QEM variations in terms of geometric approximation error, texture error, progressive approximation errors, frame rate impact, and perceptual quality measures. We found that QEM 4VR consistently yielded simplified meshes with less geometric approximation error and texture error than the prior QEM variations. It afforded better frame rates than QEM variations with boundary preservation constraints that create unnecessary lower bounds on overall polygon count reduction. Our evaluation revealed that QEM 4VR did not fair well in terms of existing perceptual distance measurements, but human-based inspections demonstrate that these algorithmic measurements are not suitable substitutes for actual human perception. In turn, we present a user-based methodology for evaluating the perceptual qualities of mesh simplification algorithms.

With the increasing accessibility of the mobile head-mounted displays (HMDs), mobile virtual reality (VR) systems are finding applications in various areas. However, mobile HMDs are highly constrained with limited graphics processing units (GPUs), low processing power and onboard memory. Hence, VR developers must be cognizant of the number of polygons contained within their virtual environments to avoid rendering at low frame rates and inducing simulator sickness. The most robust and rapid approach to keeping the overall number of polygons low is to use mesh simplification algorithms to create low-poly versions of preexisting, high-poly models. Unfortunately, most existing mesh simplification algorithms cannot adequately handle meshes with lots of boundaries or non-manifold meshes, which are common attributes of 3D models made with computer-aided design tools.; [email protected] this paper, we present a high-fidelity mesh simplification algorithm specifically designed for VR. This new algorithm, QEM4VR, addresses the deficiencies of prior quadric error metric (QEM) approaches by leveraging the insight that the most relevant boundary edges lie along curvatures while linear boundary edges can be collapsed. Additionally, our QEM4VR algorithm preserves key surface properties, such as normals, texture coordinates, colors, and materials. It pre-processes the 3D models and generate their low-poly approximations offline. We used six publicly available, high-poly models, with and without textures to compare the accuracy and fidelity of our QEM4VR algorithm to previous QEM variations. We also performed a frame rate analysis with original high-poly models and low-poly models obtained using QEM4VR and previous QEM variations. Our results indicate that QEM4VR creates low-poly, high-fidelity virtual environments for VR applications on devices that are constrained by the low number of polygons they can work with.

The emotional response a person has to a living space is predominantly affected by light, color and texture as space-making elements. In order to verify whether this phenomenon could be replicated in a simulated environment, we conducted a user study in a six-sided projected immersive display that utilized equivalent design attributes of brightness, color and texture in order to assess to which extent the emotional response in a simulated environment is affected by the same parameters affecting real environments. Since emotional response depends upon the context, we evaluated the emotional responses of two groups of users: inactive (passive) and active (performing a typical daily activity). The results from the perceptual study generated data from which design principles for a virtual living space are articulated. Such a space, as an alternative to expensive built dwellings, could potentially support new, minimalist lifestyles of occupants, defined as the neo-nomads, aligned with their work experience in the digital domain through the generation of emotional experiences of spaces. Data from the experiments confirmed the hypothesis that perceivable emotional aspects of real-world spaces could be successfully generated through simulation of design attributes in the virtual space. The subjective response to the virtual space was consistent with corresponding responses from real-world color and brightness emotional perception. Our data could serve the virtual reality (VR) community in its attempt to conceive of further applications of virtual spaces for well-defined activities.

Context-dependent memory studies have indicated that olfaction, the sense of smell, has a special odor memory that can significantly improve recall in some cases. Virtual reality (VR), which has been investigated as a training tool, could feasibly benefit from odor memory by incorporating olfactory stimuli. There have been a few studies on this concept for semantic learning, but not for procedural training. To address this gap in knowledge, we investigated the effects of olfaction on the transfer of knowledge from training to next-day execution for building a complex LEGO jet-plane model. Our results indicate that the pleasantness of an odor significantly affects training transfer more than whether the encoding and recall contexts match.

Several researchers have investigated phantom tactile sensation (i.e., the perception of a nonexistent actuator between two real actuators) and apparent tactile motion (i.e., the perception of a moving actuator due to time delays between onsets of multiple actuations). Prior work has focused primarily on determining appropriate Durations of Stimulation (DOS) and Stimulus Onset Asynchronies (SOA) for simple touch gestures, such as a single finger stroke. To expand upon this knowledge, we investigated complex touch gestures involving multiple, simultaneous points of contact, such as a whole hand touching the arm. To implement complex touch gestures, we modified the Tactile Brush algorithm to support rectangular areas of tactile stimulation.

Virtual reality training systems are commonly used in a variety of domains, and it is important to understand how the realism of a training simulation influences training effectiveness. We conducted a controlled experiment to test the effects of display and scenario properties on training effectiveness for a visual scanning task in a simulated urban environment. The experiment varied the levels of field of view and visual complexity during a training phase and then evaluated scanning performance with the simulator's highest levels of fidelity and scene complexity. To assess scanning performance, we measured target detection and adherence to a prescribed strategy. The results show that both field of view and visual complexity significantly affected target detection during training; higher field of view led to better performance and higher visual complexity worsened performance. Additionally, adherence to the prescribed visual scanning strategy during assessment was best when the level of visual complexity during training matched that of the assessment conditions, providing evidence that similar visual complexity was important for learning the technique. The results also demonstrate that task performance during training was not always a sufficient measure of mastery of an instructed technique. That is, if learning a prescribed strategy or skill is the goal of a training exercise, performance in a simulation may not be an appropriate indicator of effectiveness outside of training-evaluation in a more realistic setting may be necessary.

Located under Japan's Mount Ikenoyama, the Super-Kamiokande (or “Super-K”) neutrino detector is used to study neutrino particle physics. The Super-K detector consists of a cylindrical stainless steel tank (41.4m tall and 39.3m in diameter) holding 50,000 tons of water and 13,031 photomultiplier tubes (PMTs). To view the data captured by these sensors, many physicists use 2D visualization tools which present the data color-coded on a deconstructed representation of the cylinder. Unfortunately, this deconstructed visualization makes it difficult for physicists to fully visualize patterns of neutrino interactions. To address this, we have developed a novel virtual reality (VR) application called “Super-KAVE”, which uses a CAVE to immerse users in a lifesize representation of the Super-K detector. Super-KAVE displays the collocation of photon sensors and their color-coded data, provides a new visualization technique for neutrino-interaction patterns, and supports transitioning between data events. In this paper, we describe the Super-K detector and its data, discuss the design and implementation of our Super-KAVE application, and report on its expected uses.

MATLAB is a popular computational system and programming environment that is used in numerous engineering and science programs in the United States. One feature of MATLAB is the capability to generate 3D visualizations, which can be used to visualize scientific data or even to simulate engineering models and processes. Unfortunately, MATLAB provides only limited interactivity for these visualizations. As a solution to this problem, we have developed a software system that easily integrates with MATLAB scripts to provide the capability to view visualizations and interact with them in virtual reality (VR) systems. We call this system “ML2VR” and expect it will introduce more users to VR by enabling a large population of MATLAB programmers to easily transition to immersive systems. We will describe the system architecture of ML2VR and report on a successful case study involving the use of ML2VR.

In some virtual reality (VR) systems, OpenGL intercept methods are used to capture and render a desktop application's OpenGL calls within an immersive display. These systems often suffer from lower frame rates due to network bandwidth limitations, implementation of the intercept routine, and in some cases, the intercepted application's frame rate. To mitigate these issues and to enhance intercept-based systems in other ways, we present intercept tags, which are OpenGL geometries that are interpreted instead of rendered. We have identified and developed several uses for intercept tags, including hand-off interactions, display techniques, and visual enhancements. To demonstrate the value of intercept tags, we conducted a user study to compare a simple virtual hand technique implemented with and without intercept tags. Our results show that intercept tags significantly improve user performance and experience.

In recent years, consumers have witnessed a technological revolution that has delivered more-realistic experiences in their own homes through high-definition, stereoscopic televisions and natural, gesture-based video game consoles. Although these experiences are more realistic, offering higher levels of fidelity, it is not clear how the increased display and interaction aspects of fidelity impact the user experience. Since immersive virtual reality (VR) allows us to achieve very high levels of fidelity, we designed and conducted a study that used a six-sided CAVE to evaluate display fidelity and interaction fidelity independently, at extremely high and low levels, for a VR first-person shooter (FPS) game. Our goal was to gain a better understanding of the effects of fidelity on the user in a complex, performance-intensive context. The results of our study indicate that both display and interaction fidelity significantly affect strategy and performance, as well as subjective judgments of presence, engagement, and usability. In particular, performance results were strongly in favor of two conditions: low-display, low-interaction fidelity (representative of traditional FPS games) and high-display, high-interaction fidelity (similar to the real world).

When viewed from below, a user's feet cast shadows onto the floor screen of an under-floor projection system, such as a six-sided CAVE. Tracking those shadows with a camera provides enough information for calculating a user's ground-plane location, foot orientation, and footstep events. We present Shadow Walking, an unencumbered locomotion technique that uses shadow tracking to sense a user's walking direction and step speed. Shadow Walking affords virtual locomotion by detecting if a user is walking in place. In addition, Shadow Walking supports a sidestep gesture, similar to the iPhone's pinch gesture. In this paper, we describe how we implemented Shadow Walking and present a preliminary assessment of our new locomotion technique. We have found Shadow Walking provides advantages of being unencumbered, inexpensive, and easy to implement compared to other walking-in-place approaches. It also has potential for extended gestures and multi-user locomotion.

Empirical evidence of the benefits of immersion is an important goal for the virtual environment (VE) community. Direct comparison of immersive systems and non-immersive systems is insufficient because differences between such systems may be due not only to the level of immersion, but also to other factors, such as the input devices and interaction techniques used. In this paper, a study is presented that separates the effects of level of immersion and 3D interaction technique for a six-degree-of-freedom manipulation task. In the study, two components of immersion -- stereoscopy and field of regard -- were varied and three 3D interaction techniques -- HOMER, Go-Go, and DO-IT (a new keyboard- and mouse-based technique) -- were tested. The results of the experiment show that the interaction technique had a significant effect on object manipulation time, while the two components of immersion did not. The implications of these results are discussed for VE application developers.