A cost-effective approach to improve tracking system accuracy is to employ two or more inexpensive sensors. For example, a radar that exhibits good range, but relatively poor cross-range accuracy may be operated in concert with a camera possessing inverted properties. For each separate sensor, we assume that the target location error is represented by a bivariate Gaussian distribution with an elliptical constant probability contour. The problem posed is that of fusing data from these sensors to produce single-frame location coordinate estimates for a target. One of the methods proposed in the past is covariance intersection which assumes that all sensor measurements are uncorrelated. In this paper, we present a generalized method that provides localization coordinate estimation when sensor estimate distributions are skewed. We show that for a variety of practically important cases, our method offers substantially improved performance over conventional approaches. Based on our results, it is possible to segment the tracking volume into sub-volumes.

Since introducing a hybrid vision-inertial tracker using passive fiducial markers, we have encountered several applications in which the use of encoded LEDs would be preferable to paper fiducials. This includes small-scale applications requiring higher precision, and applications where the size of the target, or the range of viewing angles, or operation in the dark is important. We present a novel technique to encode LEDs without any need for synchronization between the LEDs and cameras. By using amplitude modulation codes instead of blinking binary codes, the LED is always on, and can therefore be tracked after it has been decoded at an arbitrary or even non-periodic frame rate with no missed data.

One of the earliest fielded augmented reality applications was enhanced vision for pilots, in which a display projected on the pilot's visor provides geo-spatially registered information to help the pilot navigate, avoid obstacles, maintain situational awareness in reduced visibility, and interact with avionics instruments without looking down. This requires exceptionally robust and accurate head-tracking, for which there is not a sufficient solution yet available. In this paper, we apply miniature MEMS sensors to cockpit helmet-tracking for enhanced/synthetic vision by implementing algorithms for differential inertial tracking between helmet-mounted and aircraft-mounted inertial sensors, and novel optical drift correction techniques. By fusing low-rate inside-out and outside-in optical measurements with high-rate inertial data, we achieve millimeter position accuracy and milliradian angular accuracy, low-latency and high robustness using small and inexpensive sensors.

We have previously presented a prototype of a novel vision/inertial hybrid self-tracker intended for AR, wearable computing and mobile robotics applications. In this paper we describe a new prototype of the system which has been greatly reduced in size, weight, power consumption and cost, while simultaneously improved in performance through careful calibration. We describe the calibration approach in detail and present results to show the high accuracy levels achieved for the camera calibration and for the integrated tracking system.

We present a demonstrated and commercially viable self-tracker, using robust software that fuses data from inertial and vision sensors. Compared to infrastructure-based trackers, self-trackers have the advantage that objects can be tracked over an extremely wide area, without the prohibitive cost of an extensive network of sensors or emitters to track them. So far, most AR research has focused on the long-term goal of a purely vision-based tracker that can operate in arbitrary unprepared environments, even outdoors. We instead chose to start with artificial fiducials, in order to quickly develop the first self-tracker which is small enough to wear on a belt, low cost, easy to install and self-calibrate, and low enough latency to achieve AR registration. We also present a roadmap for how we plan to migrate from artificial fiducials to natural ones. By designing to the requirements of AR, our system can easily handle the less challenging applications of wearable VR systems and robot navigation.

A wearable low-power hybrid vision-inertial tracker has been demonstrated based on a flexible sensor fusion core architecture, which allows easy reconfiguration by plugging-in different kinds of sensors. A particular prototype implementation consists of one inertial measurement unit and one out-ward-looking wide-angle smart camera, with a built-in DSP to run all required image-processing tasks. The smart camera operates on newly designed 2D bar-coded fiducials printed on a standard black-and-white printer. The fiducial design allows having thousands of different codes, thus enabling uninterrupted tracking throughout a large building or even a campus at very reasonable cost. The system operates in various real-world lighting conditions without user intervention due to homomorphic image processing algorithms for extracting fiducials in the presence of very non-uniform lighting.