Immersive virtual reality (VR) systems offer flexible control of an interactive environment, along with precise position and orientation tracking of realistic movements. Immersive VR can also be used in conjunction with neurophysiological monitoring techniques, such as electroencephalography (EEG), to record neural activity as users perform complex tasks. As such, the fusion of VR, kinematic tracking, and EEG offers a powerful testbed for naturalistic neuroscience research. In this study, we combine these elements to investigate the cognitive and neural mechanisms that underlie motor skill learning during a multi-day simulated marksmanship training regimen conducted with 20 participants. On each of 3 days, participants performed 8 blocks of 60 trials in which a simulated clay pigeon was launched from behind a trap house. Participants attempted to shoot the moving target with a firearm game controller, receiving immediate positional feedback and running scores after each shot. Over the course of the 3 days that individuals practiced this protocol, shot accuracy and precision improved significantly while reaction times got significantly faster. Furthermore, results demonstrate that more negative EEG amplitudes produced over the visual cortices correlate with better shooting performance measured by accuracy, reaction times, and response times, indicating that early visual system plasticity underlies behavioral learning in this task. These findings point towards a naturalistic neuroscience approach that can be used to identify neural markers of marksmanship performance.

We propose Specimen Box, an interaction technique that allows world-fixed display (such as CAVEs) users to naturally hold a plausible physical object while manipulating virtual content inside it. This virtual content is rendered based on the tracked position of the box. Specimen Box provides the weight and tactile feel of an actual object and does not occlude rendered objects in the scene. The end result is that the user sees the virtual content as if it exists inside the clear physical box. We conducted a user study which involved a cognitively loaded inspection task requiring extensive manipulation of the box. We compared Specimen Box to Grab-and-Twirl, a naturalistic bimanual manipulation technique that closely mimics the mechanics of our proposed technique. Results show that performance was significantly faster with Specimen Box. Further, performance of the control technique was positively affected by experience with Specimen Box.

Here we explore a visual display technique for low frame rate virtual environments called low persistence (LP). This involves displaying the rendered frame for a single display frame and blanking the screen while waiting for the next frame to be generated. To gain greater knowledge about the LP technique, we have conducted a user study to evaluate user performance and learning during a dynamic target acquisition task. The task involved the acquisition of targets moving along several different trajectories, modeled after a shotgun trap shooting task. The results of our study indicate the LP condition approaches high frame rate performance within certain classes of target trajectories. Interestingly we also see that learning is consistent across conditions, indicating that it may not always be necessary to train under a visually high frame rate system.

In virtual reality applications, there is an aim to provide real time graphics which run at high refresh rates. However, there are many situations in which this is not possible due to simulation or rendering issues. When running at low frame rates, several aspects of the user experience are affected. For example, each frame is displayed for an extended period of time, causing a high persistence image artifact. The effect of this artifact is that movement may lose continuity, and the image jumps from one frame to another. In this paper, we discuss our initial exploration of the effects of high persistence frames caused by low refresh rates and compare it to high frame rates and to a technique we developed to mitigate the effects of low frame rates. In this technique, the low frame rate simulation images are displayed with low persistence by blanking out the display during the extra time such image would be displayed. In order to isolate the visual effects, we constructed a simulator for low and high persistence displays that does not affect input latency. A controlled user study comparing the three conditions for the tasks of 3D selection and navigation was conducted. Results indicate that the low persistence display technique may not negatively impact user experience or performance as compared to the high persistence case. Directions for future work on the use of low persistence displays for low frame rate situations are discussed.

It has been shown that virtual reality (VR) can be used to train mine workers for safety in critical situations [4]. The National Institute for Occupational Safety and Health (NIOSH) has a virtual reality (VR) laboratory on its Pittsburgh campus. Currently, input devices for the system are an Xbox 360 game pad and an air mouse. Due to the high cost and added complexity of most 3D tracking systems, we wanted to first test to see if the mine safety application could benefit from an upgrade to a 6-DOF tracking system. Thus, we conducted a pilot study at Duke University's six-sided CAVE-type system, and collected performance and questionnaire data for three tasks (selection, navigation, and maneuvering) and three devices (gamepad, air mouse, 6-DOF wand). Results indicate that the wand allows users to complete tasks faster and is preferred by users. However, in certain situations its use led to more errors.

MATLAB is a popular computational system and programming environment that is used in numerous engineering and science programs in the United States. One feature of MATLAB is the capability to generate 3D visualizations, which can be used to visualize scientific data or even to simulate engineering models and processes. Unfortunately, MATLAB provides only limited interactivity for these visualizations. As a solution to this problem, we have developed a software system that easily integrates with MATLAB scripts to provide the capability to view visualizations and interact with them in virtual reality (VR) systems. We call this system “ML2VR” and expect it will introduce more users to VR by enabling a large population of MATLAB programmers to easily transition to immersive systems. We will describe the system architecture of ML2VR and report on a successful case study involving the use of ML2VR.

In some virtual reality (VR) systems, OpenGL intercept methods are used to capture and render a desktop application's OpenGL calls within an immersive display. These systems often suffer from lower frame rates due to network bandwidth limitations, implementation of the intercept routine, and in some cases, the intercepted application's frame rate. To mitigate these issues and to enhance intercept-based systems in other ways, we present intercept tags, which are OpenGL geometries that are interpreted instead of rendered. We have identified and developed several uses for intercept tags, including hand-off interactions, display techniques, and visual enhancements. To demonstrate the value of intercept tags, we conducted a user study to compare a simple virtual hand technique implemented with and without intercept tags. Our results show that intercept tags significantly improve user performance and experience.

The goal of the current study was to investigate the effects of different virtual environment (VE) technologies on emotional arousal and task performance in a modified Stroop task presented under low- and high-stress conditions. Fifty-three participants were recruited from a college population. Reactivity to stressful VEs was examined in three representative VE systems from desktop to high-end fully immersive systems. The experiment was a 3 (desktop system, head mounted display (HMD), and six wall system) × 2 (high- and low-stressful VE) within subject design, with self-reported emotional arousal and valence, skin conductance, task performance, presence, and simulator sickness. Replicating previous studies, the fully immersive system induced the highest sense of presence and the HMD system elicited the highest amount of simulator sickness. Extending previous studies, the results showed that different technological platforms evoked unique response patterns. Our findings suggest that different VE systems may be appropriate for different scientific purposes when studying stress reactivity using emotionally evocative tasks.

In recent years, consumers have witnessed a technological revolution that has delivered more-realistic experiences in their own homes through high-definition, stereoscopic televisions and natural, gesture-based video game consoles. Although these experiences are more realistic, offering higher levels of fidelity, it is not clear how the increased display and interaction aspects of fidelity impact the user experience. Since immersive virtual reality (VR) allows us to achieve very high levels of fidelity, we designed and conducted a study that used a six-sided CAVE to evaluate display fidelity and interaction fidelity independently, at extremely high and low levels, for a VR first-person shooter (FPS) game. Our goal was to gain a better understanding of the effects of fidelity on the user in a complex, performance-intensive context. The results of our study indicate that both display and interaction fidelity significantly affect strategy and performance, as well as subjective judgments of presence, engagement, and usability. In particular, performance results were strongly in favor of two conditions: low-display, low-interaction fidelity (representative of traditional FPS games) and high-display, high-interaction fidelity (similar to the real world).

When viewed from below, a user's feet cast shadows onto the floor screen of an under-floor projection system, such as a six-sided CAVE. Tracking those shadows with a camera provides enough information for calculating a user's ground-plane location, foot orientation, and footstep events. We present Shadow Walking, an unencumbered locomotion technique that uses shadow tracking to sense a user's walking direction and step speed. Shadow Walking affords virtual locomotion by detecting if a user is walking in place. In addition, Shadow Walking supports a sidestep gesture, similar to the iPhone's pinch gesture. In this paper, we describe how we implemented Shadow Walking and present a preliminary assessment of our new locomotion technique. We have found Shadow Walking provides advantages of being unencumbered, inexpensive, and easy to implement compared to other walking-in-place approaches. It also has potential for extended gestures and multi-user locomotion.

We present Redgraph, a generic virtual reality (VR) visualization program for network data, based on resource description framework (RDF), the primary standard of data underlying the semantic Web. Redgraph bypasses a number of problems in 3D graph visualization by relying on users to interactively "extrude" a 2D network into the third dimension. This pilot study applies Redgraph to data from the U.S. Patent and Trademark Office to explore innovations in the history of computer science. Comparison of subjects' response times utilizing 3D pull-out vs. 2D strategies on tasks involving fine grained connectivity or broad network observation found that subjects were faster correctly answering questions involving fine-grained connectivity using 3D strategies, particularly when data was densely clustered. Subjects' qualitative feedback suggests that the most valuable application of this 3D technique lies in untimed exploration to discover relationships in the underlying data structure.