In this paper we present a novel model-based hybrid technique for user localization and drift-free tracking in urban environments. In outdoor augmented reality, instantaneous 6-DoF user localization is achieved with position and orientation sensors such as GPS and gyroscopes. Initial pose obtained with these sensors is dominated by large positional errors due to coarse granularity of GPS data. Subsequent tracking is also erroneous as gyroscopes are prone to drifts and often need recalibration. We propose to use model-to-image registration technique to refine initial rough estimate for accurate user localization. Large positional errors in user localization are mitigated by aligning silhouettes of the model with that of the camera image using shape context descriptors as they are invariant to translation, scale and rotational errors. Once initialized, drift-free tracking is achieved by combining frame-to-frame and model-to-frame feature tracking. Frame-to-frame tracking is done by matching corners whereas edges are used for model-to-frame silhouette tracking. Final camera pose is obtained with M-estimators.

This paper presents a novel approach for user positioning, robust tracking and online 3D mapping for outdoor augmented reality applications. As coarse user pose obtained from GPS and orientation sensors is not sufficient for augmented reality applications, sub-meter accurate user pose is then estimated by a one-step silhouette matching approach. Silhouette matching of the rendered 3D model and camera data is carried out with shape context descriptors as they are invariant to translation, scale and rotational errors, giving rise to a non-iterative registration approach. Once the user is correctly positioned, further tracking is carried out with camera data alone. Drifts associated with vision based approaches are minimized by combining different feature modalities. Robust visual tracking is maintained by fusing frame-to-frame and model-to-frame feature matches. Frame-to-frame tracking is accomplished with corner matching while edges are used for model-to-frame registration. Results from individual feature tracker are fused using a pose estimate obtained from an extended Kalman filter (EKF) and a weighted M-estimator. In scenarios where dense 3D models of the environment are not available, online 3D incremental mapping and tracking is proposed to track the user in unprepared environments. Incremental mapping prepares the 3D point cloud of the outdoor environment for tracking.