The smart buildings of the future need to understand the movement and occupancy of the people in the environment. Using cameras to provide this context can be privacy-invasive. Alternatively, installing dedicated hardware to sense the environment can be cost-prohibitive and limit ubiquitous adoption. WiFi-based sensing has hence been championed to provide this building-scale sensing, as it allows for both privacy and is ubiquitously deployed in most buildings. However, industry-translatable research in this space has been challenging as no building-scale systems can provide WiFi sensing data. Consequently, many real-world challenges of deploying these sensing systems remain a mystery. To overcome this veil of mystery, we develop and open-source WiSenseHub, a building-scale WiFi-sensing system. We build our system on commercially available WiFi radios, deploy our backend services to collect data on infinitely scalable AWS cloud or a local server desktop, and build a front-end phone-based interface to collect diverse WiFi sensing data. We deployed multiple WiFi radios in our building and collected data for user devices for over 38 hours.

This paper shares the challenges facing today's augmented reality (AR) smartphone applications, particularly in the realm of localization and tracking failure. Our research identifies limitations in current vision-based landmarks such as QR codes and AprilTags, commonly used to aid in localization, and the drawbacks of LiDAR integration in variable lighting conditions, compromising AR's accuracy and functionality. We also examine the constraints of Inertial Measurement Units (IMU) on movement speed, highlighting its impact on the dynamic performance of AR applications. Based on our extensive 316 experimental cases for 113 hours, including 34 case studies with 17 subjects in 2 sites, this paper presents the field with a nuanced analysis of the failure modes inherent in smartphone-based AR localization. We further explore a prototype solution which fuses ultra-wideband (UWB)-based sensing with the vision-based systems to alleviate these failure modes. Our approach addresses the immediate challenges of AR localization and opens avenues for future research and development in creating more spatially aware and interactive digital worlds. All of our demonstration videos, code, and datasets are available here1.

Interest in autonomous navigation and exploration for indoor applications has spurred research into indoor Simultaneous Localization and Mapping (SLAM) robot systems. While most of these SLAM systems use camera and LiDAR sensors in tandem with an odometry sensor, these odometry sensors drift over time. Visual (LiDAR/camera-based) SLAM systems deploy compute and memory-intensive search algorithms to detect 'Loop Closures' to combat this drift, making the trajectory estimate globally consistent. Instead, WAIS (WiFi Assisted Indoor SLAM) demonstrates using WiFi-based sensing can reduce this resource intensiveness drastically. By covering over 1500 m in realistic indoor environments and WiFi deployments, we showcase 4.3× and 4× reduction in compute and memory consumption compared to state-of-the-art Visual and Lidar SLAM systems. Incorporating WiFi into the sensor stack improves the resiliency of the Visual-SLAM system. We find the 90th percentile translation errors improve by ~ 40% and orientation errors by ~ 60% compared with purely camera-based systems. Additionally, we open-source a toolbox, WiROS, to furnish online and compute efficient WiFi measurements.

The accurate location of objects and people is central to providing contextual information for various AR/VR applications. We developed XRLoc [2], a compact localization module, sized less than 1 m, which can be integrated with television screens, soundbars or independently deployed in rooms to provide accurate locations of these assets. In this demo, we showcase the capability of XRLoc to localize and track UWB tags with cm-level accuracy in realistic room-level scenarios. We will additionally compare the location accuracy of XRLoc with visual-based HTC Vive trackers.

RF signals can be leveraged for many sensing and monitoring tasks in industrial, home, or robot applications. Despite the advantages of leveraging WiFi sensing modality, no versatile WiFi sensors are available. We develop WiROS to address this immediate need. We leverage the robot operating system (ROS) framework to expose real-time WiFi sensing information to an end-user. Specifically, we demonstrate a plug-and-play toolbox providing access to coarse-grained WiFi signal strength (RSSI), fine-grained WiFi channel state information (CSI), and other MAC-layer information (device address, packet id’s or frequency-channel information). Additionally, we opensource state-of-art algorithms to calibrate and process WiFi measurements to intuitively visualize/debug measurements and measure signal path parameters like the signal’s angles of arrival or departure.

Understanding the location of ultra-wideband (UWB) tag-attached objects and people in the real world is vital to enabling a smooth cyber-physical transition. However, most UWB localization systems today require multiple anchors in the environment, which can be very cumbersome to set up. In this work, we develop XRLoc, providing an accuracy of a few centimeters in many real-world scenarios. This paper will delineate the key ideas that allow us to overcome the fundamental restrictions that plague a single anchor point from localization of a device to within an error of a few centimeters. We deploy a VR chess game using everyday objects as a demo and find that our system achieves 2.4 cm median accuracy and 5.3 cm 90th percentile accuracy in dynamic scenarios, performing at least 8× better than state-of-art localization systems. Additionally, we implement a MAC protocol to furnish these locations for over 10 tags at update rates of 100 Hz, with a localization latency of ~1 ms. We have additionally open-sourced our system's codebase at https://github.com/ucsdwcsng/xrloc.git

Wide-scale adoption of VR/AR technologies in gaming, video conferencing, and for other remote telepresence applications demands limb tracking for a more immersive experience. In an attempt to bolster limb tracking, we present UWBTrac, a UWB + IMU based fusion tracker for VR applications. In this demo, and accompanying video1, we showcase this UWB tracker in comparison with HTC Vive VR trackers.

Location services, fundamentally, rely on two components: a mapping system and a positioning system. The mapping system provides the physical map of the space, and the positioning system identifies the position within the map. Outdoor location services have thrived over the last couple of decades because of well-established platforms for both these components (e.g. Google Maps for mapping, and GPS for positioning). In contrast, indoor location services haven't caught up because of the lack of reliable mapping and positioning frameworks. Wi-Fi positioning lacks maps and is also prone to environmental errors. In this paper, we present DLoc, a Deep Learning based wireless localization algorithm that can overcome traditional limitations of RF-based localization approaches (like multipath, occlusions, etc.). We augment DLoc with an automated mapping platform, MapFind. MapFind constructs location-tagged maps of the environment and generates training data for DLoc. Together, they allow off-the-shelf Wi-Fi devices like smartphones to access a map of the environment and to estimate their position with respect to that map. During our evaluation, MapFind has collected location estimates of over 105 thousand points under 8 different scenarios with varying furniture positions and people motion across two different spaces covering 2000 sq. Ft. DLoc outperforms state-of-the-art methods in Wi-Fi-based localization by 80% (median & 90th percentile) across the two different spaces.

Social distancing has been the key factor which has helped control the COVID-19 pandemic spread. We present BluBLE, which utilizes Bluetooth Low Energy (BLE) based mobile sensing to help monitor these social distancing protocols. Specifically, we formulate the problem in two parts - spatial and temporal social distancing. The spatial distancing formulation aims to enforce the 6 feet distance recommended by various public health organization around the world. The temporal distancing formulation aims to inform and prevent users from entering high-occupancy regions (hotspots) in buildings. BluBLE achieved more than 80 % classification accuracy in both the tasks, that is, predicting if a user is within '6' feet of another user as well as characterizing the user's location within a particular hotspot.