While there is significant recent progress in technologies supporting augmented reality for small indoor environments, there is still much work to be done for large outdoor environments. This workshop focuses primarily on research that enables high-quality outdoor Mixed Reality (MR) and Augmented Reality (AR) applications. These research topics include, but are not restricted to: — 3D geo-referenced data (images, point clouds, and models) — Algorithms for object recognition from large databases of geo-referenced data — Algorithms for object tracking in outdoor environment — Multi-cue fusion to achieve improved performance of object detection and tracking — Novel representation schemes to facilitate large-scale content distribution — 3D reasoning to support intelligent augmentation — Novel and improved mobile capabilities for data capture (device sensors), processing, and display — Applications, experiences, and user interface techniques. The workshop will also showcase existing prototypes of applications enabled by these technologies: mirror worlds, high-fidelity virtual environments, applications of panoramic imagery, and user studies relating to these media types. This workshop aims to bring together academic and industrial researchers and to foster discussion amongst participants on the current state of the art and future directions for technologies that enable large-scale outdoor MR and AR applications. The workshop will start with a session in which position statements and overviews of the state of the art are presented. In the afternoon, we will follow up with discussion sessions and a short closing session.

Detailed 3D scans of urban environments are increasingly being collected with the goal of bringing more location-aware content to mobile users. This work converts large collections of LIDAR scans and street-view panoramas into a representation that extracts semantically meaningful components of the scene. Compressing this data by an order of magnitude or more enables rich user interactions with mobile applications that have a very good knowledge of the scene around them. These representations are suitable for integrating into physics engines and transmission over mobile networks — key components of modern AR entertainment solutions.

This work introduces techniques to facilitate large-scale Augmented Reality (AR) experiences in unprepared outdoor environments. We develop a shape-based object detection framework that works with limited texture and can robustly handle extreme illumination and occlusion issues. The contribution of this work is a purely geometric approach for detecting marker-like objects under difficult and realistic outdoor conditions. We demonstrate these techniques for mobile AR experiences by detecting and tracking star-shaped pentagrams embedded in the Hollywood Walk of Fame at 30Hz on a Nokia N900 phone.

In this paper we investigate the problem of automatically detecting 2D grid structures such as windows on building facades from images taken in urban settings. The key assumption that the background is strongly structured allows searching for near-regular textures in the image. We describe a probabilistic framework using Markov Random Field modeling and Markov Chain Monte Carlo (MCMC) optimization to explicitly recognize and group rectangular structures that appear in a grid-like pattern. Results on a variety of images of building facades are shown.

We present a technique for constructing a "clean" texture map of a partially occluded building facade from a series of images taken from a moving camera. Building regions blocked by trees, signs, people, and other foreground objects in a minority of views can be recovered via temporal median filtering on a registered image mosaic of the planar facade. However, when such areas are occluded in the majority of camera views, appearance information from other visible portions of the facade provides a critical cue to correctly complete the mosaic. In this paper, we apply a robust measure of spread to infer whether a particular mosaic pixel is occluded in a majority of views, and introduce a novel spatiotemporal timeline-based inpainting algorithm that uses appearance and motion cues in order to fill the texture map in majority-occluded regions. We describe methods for automatically training appearance-based classifiers from a coarse motion-based segmentation to efficiently recognize foreground and background patches in static imagery. Results of recovered building facades are shown for various sequences.

This paper studies the problem of aligning images from multiple cameras with minimally- or non-overlapping fields of view using frame-to-frame transformations calculated for sequences from each camera. We examine implementation issues for the algorithm of Caspi and Irani that performs the alignment for two cameras which are fixed relative to each other and have approximately the same center of projection. Furthermore, we extend it to compute the camera-to-camera homographies for an N/spl ges/2 multi-camera network by simultaneously solving for all parameters of the unknown transformations. This enforces tighter constraints on the solution than performing the alignment for each pair independently. We show the efficacy of the approach on both synthetic as well as real sequences captured using a polycamera built in our lab. The aligned images can be mosaiced together to obtain a wider field of view virtual camera for subsequent processing.

We present a technique for constructing a "clean" texture map of a partially occluded building facade from a series of images taken from a moving camera. Building regions blocked by trees, signs, people, and other foreground objects in a minority of views can be recovered via temporal median filtering on a registered image mosaic of the planar facade. However, when such areas are occluded in the majority of camera views, appearance information from other visible portions of the facade provides a critical cue to correctly complete the mosaic. In this paper we apply a robust measure of spread to infer whether a particular mosaic pixel is occluded in a majority of views, and introduce a novel spatiotemporal timeline-based inpainting algorithm that uses an additional motion cue in order to fill the texture map in majority-occluded regions.

We present two key parts of an ongoing robotic, vision-based architectural modeling project. The first component is a randomized approach to view planning for a single ground robot scanning a building perimeter to recover a series of texture map mosaics. This algorithm generates paths that simultaneously address coverage and quality (i.e., real-valued distance and foreshortening factors). The second part is a technique for "cleaning" the captured texture maps in the presence of occlusions caused by trees, signs, people, and other foreground objects. When such occlusions comprise a minority of views a background feature can be recovered via temporal median filtering, but when they are in the majority, appearance information from other visible portions of the facade provides a critical cue to correctly complete the mosaic. We describe a novel spatiotemporal timeline-based inpainting algorithm that identifies and corrects such areas.