This paper shares the challenges facing today's augmented reality (AR) smartphone applications, particularly in the realm of localization and tracking failure. Our research identifies limitations in current vision-based landmarks such as QR codes and AprilTags, commonly used to aid in localization, and the drawbacks of LiDAR integration in variable lighting conditions, compromising AR's accuracy and functionality. We also examine the constraints of Inertial Measurement Units (IMU) on movement speed, highlighting its impact on the dynamic performance of AR applications. Based on our extensive 316 experimental cases for 113 hours, including 34 case studies with 17 subjects in 2 sites, this paper presents the field with a nuanced analysis of the failure modes inherent in smartphone-based AR localization. We further explore a prototype solution which fuses ultra-wideband (UWB)-based sensing with the vision-based systems to alleviate these failure modes. Our approach addresses the immediate challenges of AR localization and opens avenues for future research and development in creating more spatially aware and interactive digital worlds. All of our demonstration videos, code, and datasets are available here1.

The accurate location of objects and people is central to providing contextual information for various AR/VR applications. We developed XRLoc [2], a compact localization module, sized less than 1 m, which can be integrated with television screens, soundbars or independently deployed in rooms to provide accurate locations of these assets. In this demo, we showcase the capability of XRLoc to localize and track UWB tags with cm-level accuracy in realistic room-level scenarios. We will additionally compare the location accuracy of XRLoc with visual-based HTC Vive trackers.