Today's Virtual Reality (VR) displays are dramatically better than the head-worn displays offered 30 years ago, but today's displays remain nearly as bulky as their predecessors in the 1980's. Also, almost all consumer VR displays today provide 90-110 degrees field of view (FOV), which is much smaller than the human visual system's FOV which extends beyond 180 degrees horizontally. In this paper, we propose ThinVR as a new approach to simultaneously address the bulk and limited FOV of head-worn VR displays. ThinVR enables a head-worn VR display to provide 180 degrees horizontal FOV in a thin, compact form factor. Our approach is to replace traditional large optics with a curved microlens array of custom-designed heterogeneous lenslets and place these in front of a curved display. We found that heterogeneous optics were crucial to make this approach work, since over a wide FOV, many lenslets are viewed off the central axis. We developed a custom optimizer for designing custom heterogeneous lenslets to ensure a sufficient eyebox while reducing distortions. The contribution includes an analysis of the design space for curved microlens arrays, implementation of physical prototypes, and an assessment of the image quality, eyebox, FOV, reduction in volume and pupil swim distortion. To our knowledge, this is the first work to demonstrate and analyze the potential for curved, heterogeneous microlens arrays to enable compact, wide FOV head-worn VR displays.

This paper explores whether it is effective to use real-time on-screen guidance to help users take better photos with mobile devices. Using a three-camera array, we developed a photo-taking interface that provides real-time feedback on how to position the subject-of-interest according to a photography composition rule: rule-of-thirds. We conduct a user study to compare the aesthetic quality of photos taken with our real-time guidance interface against a static gridline interface common to existing digital cameras. Expert photographers and Mechanical Turk workers rate the aesthetic quality of these pairs of photos. Results indicate the photos taken with our real-time guidance interface have significantly higher aesthetics scores. This study shows the potential in using camera array, computational photography, and real-time guidance interface to help non-expert users take better photos.

There are currently two primary ways of viewing location specific information in-situ on hand-held mobile device screens: using a see-through augmented reality interface and using a touch-based interface with panoramas. The two approaches use fundamentally different interaction metaphors: an AR-style of interacting where the user holds up the device and physically moves it to change views of the world, and a touch-based technique where panorama navigation is independent of the physical world. We have investigated how this difference in interaction technique impacts a user's spatial understanding of the mixed reality world. Our study found that AR-style interaction provided better spatial understanding overall, while touch-based interaction changed the experience to have more similar characteristics to interaction in a separate virtual environment.

This work introduces techniques to facilitate large-scale Augmented Reality (AR) experiences in unprepared outdoor environments. We develop a shape-based object detection framework that works with limited texture and can robustly handle extreme illumination and occlusion issues. The contribution of this work is a purely geometric approach for detecting marker-like objects under difficult and realistic outdoor conditions. We demonstrate these techniques for mobile AR experiences by detecting and tracking star-shaped pentagrams embedded in the Hollywood Walk of Fame at 30Hz on a Nokia N900 phone.

The three papers in this special section are extended versions of papers presented at the International Symposium on Mixed and Augmented Reality (ISMAR).

We describe and evaluate a new tracking concept for outdoor Augmented Reality. A few mobile beacons added to the environment correct errors in head-worn inertial and GPS sensors. We evaluate the accuracy through detailed simulation of many error sources. The most important parameters are the errors in measuring the beacon and user's head positions, and the geometric configuration of the beacons around the point to augment. Using Monte Carlo simulations, we identify combinations of beacon configurations and error parameters that meet a specified goal of 1 m net error at 100 m range.

View management, a relatively new area of research in Augmented Reality (AR) applications, is about the spatial layout of 2D virtual annotations in the view plane. This paper represents the first study in an actual AR application of a specific view management task: evaluating the placement of 2D virtual labels that identify information about real counterparts. Here, we objectively evaluated four different placement algorithms, including a novel algorithm for placement based on identifying existing clusters. The evaluation included both a statistical analysis of traditional metrics (e.g. counting overlaps) and an empirical user study guided by principles from human cognition. The numerical analysis of the three real-time algorithms revealed that our new cluster-based method recorded the best average placement accuracy while requiring only relatively moderate computation time. Measures of objective readability from the user study demonstrated that in practice, human subjects were able to read labels fastest with the algorithms that most quickly prevented overlap, even if placement wasn't ideal.

One unique feature of mixed and augmented reality (MR/AR) systems is that hidden and occluded objects an be readily visualized. We call this specialized use of MR/AR, obscured information visualization (OIV). In this paper, we describe the beginning of a research program designed to develop such visualizations through the use of principles derived from perceptual psychology and cognitive science. In this paper we surveyed the cognitive science literature as it applies to such visualization tasks, described experimental questions derived from these cognitive principles, and generated general guidelines that can be used in designing future OIV systems (as well improving AR displays more generally). We also report the results from an experiment that utilized a functioning AR-OIV system: we found that in relative depth judgment, subjects reported rendered objects as being in front of real-world objects, except when additional occlusion and motion cues were presented together.

"Free Flight" will change today's air traffic control system by giving pilots increased flexibility to choose and modify their routes in real time, reducing costs and increasing system capacity. This increased flexibility comes at the price of increased complexity. If Free Flight is to become a reality, future air traffic controllers, pilots, and airline managers will require new conflict detection, resolution and visualization decision support tools. The paper describes a testbed system for building and evaluating such tools, including its current capabilities, lessons we learned and feedback received from expert users. The visualization system provides an overall plan view supplemented with a detailed perspective view, allowing a user to examine highlighted conflicts and select from a list of proposed solutions, as the scenario runs in real time. Future steps needed to improve this system are described.

Almost all previous Augmented Reality (AR) systems work indoors. Outdoor AR systems offer the potential for new application areas. However, building an outdoor AR system is difficult due to portability constraints, the inability to modify the environment, and the greater range of operating conditions. We demonstrate a hybrid tracker that stabilizes an outdoor AR display with respect to user motion, achieving more accurate registration than previously shown in an outdoor AR system. The hybrid tracker combines rate gyros with a compass and tilt orientation sensor in a near real-time system. Sensor distortions and delays required compensation to achieve good results. The measurements from the two sensors are fused together to compensate for each other's limitations. From static locations with moderate head rotation rates, peak registration errors are /spl sim/2 degrees, with typical errors under 1 degree, although errors can become larger over long time periods due to compass drift. Without our stabilization, even small motions make the display nearly unreadable.

The biggest single obstacle to building effective augmented reality (AR) systems is the lack of accurate wide-area sensors for trackers that report the locations and orientations of objects in an environment. Active (sensor-emitter) tracking technologies require powered-device installation. Limiting their use to prepared areas that are relatively free of natural or man-made interference sources. Vision-based systems can use passive landmarks, but they are more computationally demanding and often exhibit erroneous behavior due to occlusion or numerical instability. Inertial sensors are completely passive, requiring no external devices or targets, however, the drift rates in portable strapdown configurations are too great for practical use. In this paper, we present a hybrid approach to AR tracking that integrates inertial and vision-based technologies. We exploit the complementary nature of the two technologies to compensate for the weaknesses in each component. Analysis and experimental results demonstrate this system's effectiveness.

The use of prediction to eliminate or reduce the effects of system delays in Head-Mounted Display systems has been the subject of several recent papers.A variety of methods have been proposed but almost all the analysis has been empirical, making comparisons of results difficult and providing little direction to the designer of new systems.In this paper, we characterize the performance of two classes of head-motion predictors by analyzing them in the frequency domain.The first predictor is a polynomial extrapolation and the other is based on the Kalman filter.Our analysis shows that even with perfect, noise-free inputs, the error in predicted position grows rapidly with increasing prediction intervals and input signal frequencies.Given the spectra of the original head motion, this analysis estimates the spectra of the predicted motion, quantifying a predictor's performance on different systems and applications.Acceleration sensors are shown to be more useful to a predictor than velocity sensors.The methods described will enable designers to determine maximum acceptable system delay based on maximum tolerable error and the characteristics of user motions in the application.

In Augmented Reality, see-through HMDs superimpose virtual 3D objects on the real world. This technology has the potential to enhance a user's perception and interaction with the real world. However, many Augmented Reality applications will not be accepted until we can accurately register virtual objects with their real counterparts. In previous systems, such registration was achieved only from a limited range of viewpoints, when the user kept his head still. This paper offers improved registration in two areas. First, our system demonstrates accurate static registration across a wide variety of viewing angles and positions. An optoelectronic tracker provides the required range and accuracy. Three calibration steps determine the viewing parameters. Second, dynamic errors that occur when the user moves his head are reduced by predicting future head locations. Inertial sensors mounted on the HMD aid head-motion prediction. Accurate determination of prediction distances requires low-overhead operating systems and eliminating unpredictable sources of latency. On average, prediction with inertial sensors produces errors 2-3 times lower than prediction without inertial sensors and 5-10 times lower than using no prediction at all. Future steps that may further improve registration are outlined.

An optoelectronic head-tracking system for head-mounted displays is described. The system features a scalable work area that currently measures 10' x 12', a measurement update rate of 20-100 Hz with 20-60 ms of delay, and a resolution specification of 2 mm and 0.2 degrees. The sensors consist of four head-mounted imaging devices that view infrared lightemitting diodes (LEDs) mounted in a 10' x 12' grid of modular 2' x 2' suspended ceiling panels. Photogrammetric techniques allow the head's location to be expressed as a function of the known LED positions and their projected images on the sensors. The work area is scaled by simply adding panels to the ceiling's grid. Discontinuities that occurred when changing working sets of LEDs were reduced by carefully managing all error sources, including LED placement tolerances, and by adopting an overdetermined mathematical model for the computation of head position: space resecfion by collinearity. The working system was demonstrated in the Tomorrow's Realities gallery at the ACM SIGGRAPH '91 conference.