Many augmented reality (AR) applications use marker-based vision tracking systems to recover camera pose by detecting one or more planar landmarks. However, most of these systems do not interactively quantify the accuracy of the pose they calculate. Instead, the accuracy of these systems is either ignored, assumed to be a fixed value, or determined using error tables (constructed in an off-line ground-truthed process) along with a run-time interpolation scheme. The validity of these approaches are questionable as errors are strongly dependent on the intrinsic and extrinsic camera parameters and scene geometry. In this paper we present an algorithm for predicting the statistics of marker tracker error in real-time. Based on the scaled spherical simplex unscented transform (SSSUT), the algorithm is applied to the augmented reality toolkit plus (ARToolKitPlus). The results are validated using precision off-line photogrammetric techniques.

Some tasks vital to many mixed and augmented reality systems are either too time consuming or complex to be carried out whilst the system is active. 3D scene modelling and labelling are two such tasks commonly performed by skilled operators in an off-line initialisation phase. Because this phase sometimes needs specialist software and/or expertise it can be a considerable limiting factor for new mixed reality system developers. If a mixed reality system is to operate in real-time, where artificial graphics are woven into real world live images, the way in which these off-line processes are tackled is critical. In this paper we propose a flexible new approach that reduces the time spent during the off-line initialisation phase by adopting an on-line interactive primitive modelling technique. Our solution combines two existing and freely available packages, the Augmented Reality Toolkit Plus (ARToolKitPlus) and the Mixed Reality Toolkit (MRT), to enable rapid interactive modelling over live video using a freely moving camera. As a demonstration we show how these can be used to rapidly seed an object appearance-based tracking algorithm.

Many mixed-reality systems require real-time composition of virtual objects with real video. Such composition requires some description of the virtual and real scene geometries and calibration information for the real camera. Once these descriptions are available, they can be used to perform many types of visual simulation including virtual object placement, occlusion culling, texture extraction, collision detection and reverse and reillumination methods. In this paper we present a demonstration where we rapidly register prefabricated virtual models to a videoed scene. Using this registration information we were able to augment animated virtual avatars to create a novel mixed reality system. Rather than build a single monolithic system, we briefly introduce our lightweight modelling tool, the Mixed-Reality Toolkit (MRT) which enables rapid reconfiguration of scene objects without performing a full reconstruction. We also generalise to outline some initial requirements for a Mixed Reality Modelling Language (MRML). Copyright 2005 ACM.