Designers of virtual reality (VR) systems are aware of the need to minimize delays between the user's tracked physical actions and the consequent displayed actions in the virtual environment. Such delays, also referred to as end-to-end latency, are known to degrade user performance and even cause simulator sickness. Though a wide variety of hardware and software design strategies have been used to reduce delays, techniques for measuring and minimizing latency continue to be needed since transmission and switching delays are likely to continue to introduce new sources of latency, especially in wireless mobile environments. This article describes a convenient low-cost technique for measuring end-to-end latencies using a human evaluator and an ordinary consumer camera (e.g., cell phone camera). Since the technique does not depend upon the use of specialized hardware and software, it differs from other methods in that it can easily be used to measure latencies of systems in the specific hardware and software configuration and the relevant performance environments. The achievable measurement accuracy was assessed in an experimental trial. Results indicate a measurement uncertainty below 10 ms. Some refinements to the technique are discussed, which may further reduce the measurement uncertainty to approximately 1 ms.

Many augmented reality (AR) applications operate within near-field reaching distances, and require matching the depth of a virtual object with a real object. The accuracy of this matching was measured in three experiments, which examined the effect of focal distance, age, and brightness, within distances of 33.3 to 50 cm, using a custom-built AR haploscope. Experiment I examined the effect of focal demand, at the levels of collimated (infinite focal distance), consistent with other depth cues, and at the midpoint of reaching distance. Observers were too young to exhibit age-related reductions in accommodative ability. The depth matches of collimated targets were increasingly overestimated with increasing distance, consistent targets were slightly underestimated, and midpoint targets were accurately estimated. Experiment II replicated Experiment I, with older observers. Results were similar to Experiment I. Experiment III replicated Experiment I with dimmer targets, using young observers. Results were again consistent with Experiment I, except that both consistent and midpoint targets were accurately estimated. In all cases, collimated results were explained by a model, where the collimation biases the eyes' vergence angle outwards by a constant amount. Focal demand and brightness affect near-field AR depth matching, while age-related reductions in accommodative ability have no effect.

Many compelling augmented reality (AR) applications require users to correctly perceive the location of virtual objects, some with accuracies as tight as 1 mm. However, measuring the perceived depth of AR objects at these accuracies has not yet been demonstrated. In this paper, we address this challenge by employing two different depth judgment methods, perceptual matching and blind reaching, in a series of three experiments, where observers judged the depth of real and AR target objects presented at reaching distances. Our experiments found that observers can accurately match the distance of a real target, but when viewing an AR target through collimating optics, their matches systematically overestimate the distance by 0.5 to 4.0 cm. However, these results can be explained by a model where the collimation causes the eyes' vergence angle to rotate outward by a constant angular amount. These findings give error bounds for using collimating AR displays at reaching distances, and suggest that for these applications, AR displays need to provide an adjustable focus. Our experiments further found that observers initially reach ∼4 cm too short, but reaching accuracy improves with both consistent proprioception and corrective visual feedback, and eventually becomes nearly as accurate as matching.

In many medical Augmented Reality (AR) applications, doctors want the ability to place a physical object, such as a needle or other medical device, at the depth indicated by a virtual object. Often, this object will be located behind an occluding surface, such as the patient's skin. In this poster we describe efforts to determine how accurately this can be done. In particular, we used a table-mounted AR haploscope to conduct two depth-matching experiments.

A previous study indicated that peripheral visual information strongly affects the judgment of egocentric distances for users of immersive virtual environments. The experiment described in this document aimed to investigate if these effects could be explained in terms of changes in gait caused by visual information in the extreme periphery. Three conditions with varying degrees of peripheral occlusion were tested and participants' walking characteristics measured. The results indicate that the improvements in distance judgments, as peripheral information increases, can only partially be explained in terms of gait modification, but likely involve both changes in the characteristics of gait and other spatial or movement parameters.

This tutorial is for researchers and engineers, working in the field of Virtual Reality (VR) and Augmented Reality (AR), who wish to conduct user-based experiments with a specific aim of promoting both traditional quantitative human-subject experiments and qualitative methods for assessing usability. This tutorial is for a full-day tutorial presenting both quantitative and qualitative approaches to conducting human-subject experiments. It covers (1) the basic principles of experimental design and analysis, with an emphasis on human-subject experiments in VR/AR; (2) qualitative studies (e.g., formative evaluation methods) for assessing and improving VR/AR user interfaces and user interaction along with lessons learned from conducting many user-based studies; and (3) a “journalistic approach” to measuring human performance that organizes the activity around questions such as “Who? What? When? Where? How? and Why?”.

In this abstract we describe an experiment that measured depth judgments in optical see-through augmented reality (AR) at near-field reaching distances of ~ 24 to ~ 56 cm. The 2×2 experiment crossed two depth judgment tasks, perceptual matching and blind reaching, with two different environments, a real-world environment and an augmented reality environment. We designed a task that used a direct reaching gesture at constant percentages of each participant's maximum reach; our task was inspired by previous work by Tresilian and Mon-Williams [6] that found very accurate blind reaching results in a real-world environment.

The parameter estimation variance of the Single Point Active Alignment Method (SPAAM) is studied through an experiment where 11 subjects are instructed to create alignments using an Optical See-Through Head Mounted Display (OSTHMD) such that three separate correspondence point distributions are acquired. Modeling the OSTHMD and the subject's dominant eye as a pinhole camera, findings show that a correspondence point distribution well distributed along the user's line of sight yields less variant parameter estimates. The estimated eye point location is studied in particular detail. The findings of the experiment are complemented with simulated data which show that image plane orientation is sensitive to the number of correspondence points. The simulated data also illustrates some interesting properties on the numerical stability of the calibration problem as a function of alignment noise, number of correspondence points, and correspondence point distribution.

A frequently observed problem in virtual environments is the underestimation of egocentric depth. This problem has been described numerous times and with widely varying degrees of severity. Though there has been considerable progress made in modifying observer behavior to compensate for these misperceptions, the question of why these errors exist is still an open issue. The study detailed in this document presents the preliminary findings of a large, between-subjects experiment (N=98) that attempts to identify and quantify the source of a pattern of adaptation and improved accuracy in the absence of explicit feedback found in Jones et al. [1].

In this poster abstract we describe an experiment that measured depth judgments in optical see-through augmented reality at near-field distances of 34 to 50 centimeters. The experiment compared two depth judgment tasks: perceptual matching, a closed-loop task, and blind reaching, a visually open-loop task. The experiment tested each of these tasks in both a real-world environment and an augmented reality environment, and used a between-subjects design that included 40 participants. The experiment found that matching judgments were very accurate in the real world, with errors on the order of millimeters and very little variance. In contrast, matching judgments in augmented reality showed a linear trend of increasing overestimation with increasing distance, with a mean overestimation of ~1 cm. With reaching judgments participants underestimated ~4.5 cm in both augmented reality and the real world. We also discovered and solved a calibration problem that arises at near-field distances.

While label placement algorithms are generally successful in managing visual clutter by preventing label overlap, they can also cause significant label movement in dynamic displays. This study investigates motion detection thresholds for various types of label movement in realistic and complex virtual environments, which can be helpful for designing less salient and disturbing algorithms. Our results show that label movement in stereoscopic depth is shown to be less noticeable than similar lateral monoscopic movement, inherent to 2D label placement algorithms. Furthermore, label movement can be introduced more readily into the visual periphery (over 15° eccentricity) because of reduced sensitivity in this region. Moreover, under the realistic viewing conditions that we used, motion of isolated labels is more easily detected than that of overlapping labels. This perhaps counterintuitive finding may be explained by visual masking due to the visual clutter arising from the label overlap. The quantitative description of the findings presented in this paper should be useful not only for label placement applications, but also for any cluttered AR or VR application in which designers wish to control the users' visual attention, either making text labels more or less noticeable as needed.

System rendering latency has been recognized by senior researchers, such as Professor Fredrick Brooks of UNC (Turing Award 1999), as a major factor limiting the realism and utility of head-referenced display systems. Latency has been shown to reduce the user's sense of immersion within a virtual environment, to disturb user interaction with virtual objects, and to contribute to motion sickness during some simulation tasks. Latency, however, is not just an issue for external display systems since finite nerve conduction rates and variation in transduction times in the human body's sensors also pose problems for latency management within the nervous system. Some of the phenomena arising from the brain's handling of sensory asynchrony due to latency will be discussed as a prelude to consideration of the effects of latency in interactive displays. The causes and consequences of the erroneous movement that appears in displays due to latency will be illustrated with examples of the user performance impact provided by several experiments. These experiments will review the generality of user sensitivity to latency when users judge either object or environment stability. Hardware and signal processing countermeasures will also be discussed. In particular the tuning of a simple extrapolative predictive filter not using a dynamic movement model will be presented. Results show that it is possible to adjust this filter so that the appearance of some latencies may be hidden without the introduction of perceptual artifacts such as overshoot. Several examples of the effects of user performance will be illustrated by three-dimensional tracking and tracing tasks executed in virtual environments. These experiments demonstrate classic phenomena known from work on manual control and show the need for very responsive systems if they are intended to support precise manipulation. The practical benefits of removing interfering latencies from interactive systems will be emphasized with some classic final examples from surgical telerobotics and human-computer interaction.

We examined human perception of head-referenced image motion during concurrent head movement. The visual stimulus was a checkerboard image in a head mounted display that moved from side-to-side. Observers rated the magnitude of the checkerboard motion while either rotating their head about a vertical axis (yaw), about a horizontal axis (pitch), or holding it still. In Experiment 1, we tested four image oscillation frequencies (0.25, 0.5, 1, and 2 Hz) while holding the head motion frequency constant at 0.5 Hz. In Experiment 2, we tested three head motion frequencies (0.25, 0.5, and 1 Hz) while holding the image oscillation frequency constant at 1 Hz. Across all image and head motion frequencies, perceptual sensitivity to image motion was reduced by about 45% during horizontal head movement. During vertical head movement, perceptual sensitivity was reduced by about 25% when head and image motion were of the same frequency. Compared with when the head was still, horizontal and vertical head movements produced a downward shift of about 10% in overall motion magnitude estimation response. Findings from this study provide virtual environment developers with a quantitative description of the influence of concurrent head movement on the perception of frontoparallel image motion.

This paper describes a novel technique for segregating overlapping labels in stereoscopic see-through displays. The present study investigates the labeling of far-field objects, with distances ranging 100-120 m. At these distances the stereoscopic disparity difference between objects is below 1 arcmin, so labels rendered at the same distance as their corresponding objects appear as if on a flat layer in the display. This flattening is due to limitations of both display and human visual resolution. By remapping labels to pre-determined depth layers on the optical path between the observer and the labeled object, an interlayer disparity ranging from 5 to 20 arcmin can be achieved for 5 overlapping labels. The present study evaluates the impact of such depth separation of superimposed layers, and found that a 5 arcmin interlayer disparity yields a significantly lower response time, over 20% on average, in a visual search task compared to correctly registering labels and objects in depth. Notably the performance does not improve when doubling the interlayer disparity to 10 arcmin and, surprisingly, the performance degrades significantly when again doubling the interlayer disparity to 20 arcmin, approximating the performance in situations with no interlayer disparity. These results confirm that our technique can be used to segregate overlapping labels in the far visual field, without the cost associated with traditional label placement algorithms.

The precision with which users can maintain boresight alignment between visual targets at different depths is recorded for 24 subjects using two different boresight targets. Subjects' normal head stability is established using their Romberg coefficients. Weibull distributions are used to describe the probabilities of the magnitude of head positional errors and the three dimensional cloud of errors is displayed by orthogonal two dimensional density plots. These data will lead to an understanding of the limits of user introduced calibration error in augmented reality systems.

We present a new technique for managing visual clutter caused by overlapping labels in complex information displays. This technique, "label layering", utilizes stereoscopic disparity as a means to segregate labels in depth for increased legibility and clarity. By distributing overlapping labels in depth, we have found that selection time during a visual search task in situations with high levels of overlap is reduced by four seconds or 24%. Our data show that the depth order of the labels must be correlated with the distance order of their corresponding objects. Since a random distribution of stereoscopic disparity in contrast impairs performance, the benefit is not solely due to the disparity-based image segregation. An algorithm using our label layering technique accordingly could be an alternative to traditional label placement algorithms that avoid label overlap at the cost of distracting motion, symbology dimming or label size reduction.

The postural sway in 24 subjects performing a boresight calibration task on a large format head-up display is studied to estimate the impact of human limits on boresight calibration precision and ultimately on static registration errors. The dependent variables, accumulated sway path and omni-directional standard deviation, are analyzed for the calibration exercise and compared against control cases where subjects are quietly standing with eyes open and eyes closed. Findings show that postural stability significantly deteriorates during boresight calibration compared to when the subject is not occupied with a visual task. Analysis over time shows that the calibration error can be reduced by 39% if calibration measurements are recorded in a three second interval at approximately 15 seconds into the calibration session as opposed to an initial reading. Furthermore parameter optimization on experiment data suggests a Weibull distribution as a possible error description and estimation for omni-directional calibration precision. This paper extends previously published preliminary analyses and the conclusions are verified with experiment data that has been corrected for subject inverted pendulum compensatory head rotation by providing a better estimate of the position of the eye. With correction the statistical findings are reinforced.

Recent work has shown that overlapping labels in far-field AR environments can be successfully segregated by remapping them to predefined stereoscopic depth layers. User performance was found to be optimal when setting the interlayer disparity to 5--10 arcmin. The current paper investigates to what extent this label segregation technique, label layering, is affected by important perceptual defects in AR such as registration errors and mismatches in accommodation, visual resolution and contrast. A virtual environment matched to a corresponding AR condition but lacking these problems showed a reduction in average response time by 10%. However, the performance pattern for different label layering parameters was not significantly different in the AR and VR environments, showing robustness of this label segregation technique against such perceptual issues.

The experience of motion sickness in a virtual environment may be measured through pre and postexperiment self-reported questionnaires such as the Simulator Sickness Questionnaire (SSQ). Although research provides converging evidence that users of virtual environments can experience motion sickness, there have been no controlled studies to determine to what extent the user's subjective response is a demand characteristic resulting from pre and posttest measures. In this study, subjects were given either SSQ's both pre and postvirtual environment immersion, or only postimmersion. This technique tested for contrast effects due to demand characteristics in which administration of the questionnaire itself suggested to the participant that the virtual environment may produce motion sickness. Results indicate that reports of motion sickness after immersion in a virtual environment are much greater when both pre and postquestionnaires are given than when only a posttest questionnaire is used. The implications for assessments of motion sickness in virtual environments are discussed.

Virtual reality (VR) and augmented reality (AR) systems are reaching sufficient maturity that some systems are being used on a regular basis by users and others are close to implementation. There are limitations, however, that have prevented many systems from being truly useful for users. An important limitation is the lack of understanding of human factors issues, such as human perceptual and cognitive limitations, that affect the potential utility of VR or AR systems.

Stereo and bi-ocular head-mounted displays (HMDs) require the user to fuse two images into a coherent picture of the threedimensional world. The human visual system performs this task constantly, but when the input images contain both real and graphical depictions, the problem becomes more difficult. A vertical disparity in the graphics causes diplopia for users trying to fuse the real and virtual objects simultaneously. We implement three methods to measure and correct this disparity and assess them with a collection of a single model of optical see-through HMD.

Summary form only for tutorial. As virtual and augmented reality hardware and software have grown more mature over the past decade, the focus of the field is shifting away from the basic engineering technology, and towards the science and applications of VR and AR techniques. Increasingly, virtual and augmented reality researchers are conducting human-subject experiments, both to understand the way humans perceive, manipulate, and cognate with VR and AR information, and to quantify the utility of VR and AR in different application contexts.

The experience of motion sickness in a virtual environment may be measured through pre- and post-experiment self-reported questionnaires such as the Simulator Sickness Questionnaire (SSQ). Although research provides converging evidence that users of virtual environments can experience motion sickness, there have been no controlled studies to determine to what extent the user’s subjective response is a demand characteristic resulting from pre- and post-test measures. In this study, subjects were given either SSQ’s both pre and post virtual environment immersion, or only post immersion. This technique was used to test for contrast effects due to demand characteristics in which administration of the questionnaire itself suggests to the participant that the virtual environment may produce motion sickness. Results indicate that reports of motion sickness after immersion in a virtual environment are much greater when both pre and post questionnaires are given than when only a post test questionnaire is used. The implications for assessments of motion sickness in virtual environments are discussed.

No abstract available.

The research community is challenged to investigate the factors that make virtual reality technologies effective and productive. Realising the goals of virtual reality systems and harnessing them to successful applications can be accomplished only by employing robust metrics and formal experimentation. Subsequently, the quality of interfaces and platforms implemented could be assessed.

The contribution to spatial awareness of adding a roll degree-of-freedom (DOF) to telepresence camera platform yaw and pitch was examined in an experiment where subjects judged direction and rotation of stationary target markers in a remote scene. Subjects viewed the scene via head-slaved camera images in a head-mounted display. Elimination of the roll DOF affected rotation judgment, but only at extreme yaw and pitch combinations, and did not affect azimuth and elevation judgement. Systematic azimuth overshoot occurred regardless of roll condition. Observed rotation misjudgments are explained by kinematic models for eye-head direction of gaze.

We examined the effects of human 3D tracking performance of several common defects of immersing virtual environments: spatial sensor distortion, visual latency and low update rates. Results show: removal of relatively small static distortion had minor effects on tracking accuracy; an adapted Cooper-Harper controllability scale proved the most sensitive subjective indicator of simulation degradation; and RMS tracking error and subjective impressions were more influenced by changing visual latency than by update rate.

Errors in the localization of nearby virtual objects presented via see-through, helmet mounted displays are examined as a function of viewing conditions and scene content. Monocular, biocular or stereoscopic presentation of the virtual objects, accommodation (required focus), subjects' age, and the position of physical surfaces are examined. Nearby physical surfaces are found to introduce localization errors that differ depending upon the other experimental factors. The apparent physical size and transparency of the virtual objects and physical surfaces respectively are also influenced by their relative position when superimposed. Design implications are discussed.

This paper describes a testbed and method for characterizing the dynamic response of the type of spatial displacement transducers commonly used in virtual environment (VE) applications. The testbed consists of a motorized rotary swing arm that imparts known displacement inputs to the VE sensor. The experimental method involves a series of tests in which the sensor is displaced back and forth at a number of controlled frequencies that span the bandwidth of volitional human movement. During the tests, actual swing arm angle and reported VE sensor displacements are collected and time stamped. Because of the time stamping technique, the response time of the sensor can be measured directly, independent of latencies in data transmission from the sensor unit and any processing by the interface applications running on the host computer. Analysis of these experimental results allows sensor time delay and gain characteristics to be determined as a function of input frequency. Results from tests of several differnt VE spatial sensors (Ascension, Logitech, and Polhemus) are presented here to demonstrate use of the testbed and method.

A previously formulated mathematical model, describing how observers reconstruct three-dimensional spatial layouts from perspective projections, has been extended to more complex situations. The model assumes that the observer has a priori knowledge of certain characteristics of the viewed objects, like size, shape, or parallelism or perpendicularity of lines or planes. These assumptions are used in a three-dimensional process to reconstruct a spatial layout that 'best matches' the perceived lines of sight to the object coordinates. Sources of errors and biases in this process are specified and their effects on model outputs are discussed. An experiment, in which eight subjects judged the relative direction of one object with respect to another, has been conducted to validate the model. The model has been found to generally reproduce the systematic trends of the experimental results and also has provided an analytical explanation for them. The mathematical model is expected to be a useful tool in analyzing and developing pictorial perspective flight displays.>

Position and rate control are the two common manual control modes in teleoperations. Human operator performance using the two modes is evaluated and compared. Simulated three-axis pick-and-place operations are used as the primary task for evaluation. First, ideal position and rate control are compared by considering several factors, such as joystick gain, joystick type, display mode, task, and manipulator work space size. Then the effects of the manipulator system dynamics are investigated by varying the natural frequency and speed limit. Experimental results show that ideal position control is superior to ideal rate control, regardless of joystick type or display mode, when the manipulation work space is small or comparable to the human operator's control space. Results also show that when the manipulator system is slow, the superiority of position control disappears. Position control is recommended for small-work-space telemanipulation tasks, while rate control is recommended for slow wide-work-space telemanipulation tasks.

An experimental telerobotics (TR) simulation is described suitable for studying human operator (HO) performance. Simple manipulator pick-and-place and tracking tasks allowed quantitative comparison of a number of calligraphic display viewing conditions. An enhanced perspective display was effective with a reference line from target to base, with or without a complex three-dimensional grid framing the view. This was true especially if geometrical display parameters such as azimuth (AZ) and elevation (EL) were arranged to be near optimal. Quantitative comparisons were made possible utilizing control performance measures such as root mean square error (rmse). There was a distinct preference for controlling the manipulator in end-effector Cartesian space for our primitive pick-and-place task, rather than controlling joint angles and then, via direct kinematics, the end-effector position. An introduced communication delay was found to produce decrease in performance. In considerable part, this difficulty could be compensated for by preview control information. That neurological control of normal human movement contains a sampled data period of 0.2 s may relate to this robustness of HO control to delay.

Optimal presentation of three-dimensional information on a two-dimensional display screen requires careful design of the projection to the display surface. Monoscopic perspective projection alone is usually not sufficient to represent three-dimensional spatial information. It can, however, be improved by the adjustment of perspective parameters and by geometric visual enhancements such as reference lines and a background grid. Stereoscopic display is another method of providing three-dimensional information to the human operator. Two experiments are performed with three-axis manual tracking tasks. The first experiment investigates the effects of perspective parameters on tracking performance. The second experiment investigates the effects of visual enhancements for both monoscopic and stereoscopic displays. Results indicate that, though stereoscopic displays do generally permit superior tracking performance, monoscopic displays can allow equivalent performance when they are defined with optimal perspective parameters and provided with adequate visual enhancements.

The distribution of commands issued by experienced users of a computer operating system allowing command customization tends to conform to Zipf's law. This result documents the emergence of a statistical property of natural language as users master an artificial language. Analysis of Zipf's law by Mandelbrot and Cherry shows that its emergence in the computer interaction of experienced users may be interpreted as evidence that these users optimize their encoding of commands. Accordingly, the extent to which users of a command language exhibit Zipf's law can provide a metric of the naturalness and efficiency with which that language is used.