Radars are an ideal complement to cameras: both are inexpensive, solid-state sensors, with cameras offering fine angular resolution, while radars provide metric depth and robustness under adverse weather. However, radar data is more difficult to interpret than camera images and varies significantly between sensors, necessitating increased reliance on simulation for prototyping sensors and processing pipelines. Recent work treating radar reconstruction as a novel view synthesis problem has shown great promise in reconstructing radar-relevant geometry and simulating lowlevel radar data. However, such methods are constrained by the low spatial resolution of the underlying radar. To address this, we propose a unified differentiable renderer, RadarSim, which leverages the high angular resolution of RGB cameras to generate Doppler radar range images from a camera-initialized neural field. Using a novel data set of calibrated radar camera recordings from a custom handheld rig, we demonstrate that RadarSim produces sharper geometry and Doppler range frames than radar-only reconstructions.

mmWave radars are compact, inexpensive, and durable sensors that are robust to occlusions and work regardless of environmental conditions, such as weather and darkness. However, this comes at the cost of poor angular resolution, especially for inexpensive single-chip radars, which are typically used in automotive and indoor sensing applications. Although many have proposed learning-based methods to mitigate this weakness, no standardized foundational models or large datasets for the mmWave radar have emerged, and practitioners have largely trained task-specific models from scratch using relatively small datasets. In this paper, we collect (to our knowledge) the largest available raw radar dataset with 1M samples (29 hours) and train a foundational model for 4D single-chip radar, which can predict 3D occupancy and semantic segmentation with quality that is typically only possible with much higher resolution sensors. We demonstrate that our Generalizable Radar Transformer (GRT) generalizes across diverse settings, can be fine-tuned for different tasks, and shows logarithmic data scaling of 20\% per $10\times$ data. We also run extensive ablations on common design decisions, and find that using raw radar data significantly outperforms widely-used lossy representations, equivalent to a $10\times$ increase in training data. Finally, we roughly estimate that $\approx$100M samples (3000 hours) of data are required to fully exploit the potential of GRT.

With the increasing use of wireless technologies in robotics for communication, sensing, and localization, the potential benefits of how robotics can complement and enhance wireless systems remain underexplored. This paper explores a novel application of the existing inflatable robots for wireless communication systems by forming a shape-programming, reflective waveguide that enhances the received signal quality for wireless devices. Our primary target is enhancing Low-Power Wide-Area Networks (LP-WANs) - where 10-year batterypowered client devices (e.g. energy meters or smart home sensors) connect to cellular-like base stations to deliver data. Devices in these networks often experience significant seasonal variability in battery life - even simple obstructions between the device and base station (e.g. due to construction) can shave off years of battery life. We propose MetaMorph, a programmable robotic reflector attached to base stations that enhances signal quality from client devices by enhancing received signal energy with controlled reflections. We investigate the design of the reflector, and our experiments show the ability to improve the signal quality for LP-WAN (LoRa) communication systems demonstrating signal quality and battery-benefits. To our best knowledge, MetaMorph is the first paper to explore how flexible robotics can serve as virtuous reflectors for wireless communication systems.

Simulation is an invaluable tool for radio-frequency system designers that enables rapid prototyping of various algorithms for imaging, target detection, classification, and tracking. However, simulating realistic radar scans is a challenging task that requires an accurate model of the scene, radio frequency material properties, and a corresponding radar synthesis function. Rather than specifying these models explicitly, we propose DART - Doppler Aided Radar Tomography, a Neural Radiance Field-inspired method which uses radar-specific physics to create a reflectance and transmittance-based rendering pipeline for range-Doppler images. We then evaluate DART by constructing a custom data collection platform and collecting a novel radar dataset together with accurate position and instantaneous velocity measurements from lidar-based localization. In comparison to state-of-the-art baselines, DART synthesizes superior radar range-Doppler images from novel views across all datasets and additionally can be used to generate high quality tomographic images.11Our implementation, data collection platform, and collected datasets can be found via our project site: https://wiselabcmu.github.io/dart/.

In this paper, we ask, "Can millimeter-wave (mmWave) radars sense objects not directly illuminated by the radar - for instance, objects located outside the transmit beamwidth, behind occlusions, or placed fully behind the radar?" Traditionally, mmWave radars are limited to sense objects that are directly illuminated by the radar and scatter its signals directly back. In practice, however, radar signals scatter to other intermediate objects in the environment and undergo multiple bounces before being received back at the radar. In this paper, we present Hydra, a framework to explicitly model and exploit multi-bounce paths for sensing. Hydra enables standalone mmWave radars to sense beyond-field-of-view objects without prior knowledge of the environment. We extensively evaluate the localization performance of Hydra with an off-the-shelf mmWave radar in five different environments with everyday objects. Exploiting multi-bounce via Hydra provides 2×-10× improvement in the median beyond-field-of-view localization error over baselines.

This paper explores a machine learning approach on data from a single-chip mmWave radar for generating high resolution point clouds – a key sensing primitive for robotic applications such as mapping, odometry and localization. Unlike lidar and vision-based systems, mmWave radar can operate in harsh environments and see through occlusions like smoke, fog, and dust. Unfortunately, current mmWave processing techniques offer poor spatial resolution compared to lidar point clouds. This paper presents RadarHD, an end-to-end neural network that constructs lidar-like point clouds from low resolution radar input. Enhancing radar images is challenging due to the presence of specular and spurious reflections. Radar data also doesn't map well to traditional image processing techniques due to the signal's sinc-like spreading pattern. We overcome these challenges by training RadarHD on a large volume of raw I/Q radar data paired with lidar point clouds across diverse indoor settings. Our experiments show the ability to generate rich point clouds even in scenes unobserved during training and in the presence of heavy smoke occlusion. Further, RadarHD's point clouds are high-quality enough to work with existing lidar odometry and mapping workflows.

We demonstrate Platypus, a sub-millimeter micro-displacement sensing system presented in [3]. Micro-displacement measurement is a crucial task in industrial systems such as structural health monitoring, where millimeter-level displacement of specific points on the structure or machinery parts can jeopardize the integrity of the structure and potentially leading to catastrophic damage or collapse. Platypus enables sub-millimeter level sensing accuracy by using mmWave backscatter tags and their reflection as phase carriers to shift the phase changes due to tiny displacements to clean frequency bins for precise tracking. It then reconstructs the tag phase changes with sub-millimeter level accuracy even from extended ranges (over 100m) or in non-line-of-sight (NLoS) situations where the tag is blocked by other objects. Here, we demonstrate Platypus’s performance by attaching a Platypus tag to a stepper motor-driven motion-stage and demonstrating the micro-displacement detection in real time, and the system robustness against multipath and occlusions.

Micro-displacement measurement is a crucial task in industrial systems such as structural health monitoring, where millimeter-level displacement of specific points on the structure or machinery displace can jeopardize the integrity of the structure and potentially leading to catastrophic damage or collapse. Traditionally, such displacements on large structures are measured using visual sensing platforms or advanced surveying equipment. However, they either fall short in varying weather and lighting conditions or require installation and maintenance of high-power sensing platforms that are expensive to deploy at scale, especially if continuous measurements are desired.

No abstract available.

Millimeter wave radars can perceive through occlusions like dust, fog, smoke and clothes. But compared to cameras and lidars, their perception quality is orders of magnitude poorer. RadarHD [3] tackles this problem of poor quality by creating a machine learning super resolution pipeline trained against high quality lidar scans to mimic lidar. RadarHD ingests low resolution radar and generates high quality lidar-like point clouds even in occluded settings. RadarHD can also make use of the high quality output for typical robotics tasks like odometry, mapping and classification using conventional lidar workflows. Here, we demonstrate the effectiveness of RadarHD's point clouds against lidar in occluded settings.

Robotic geo-fencing and surveillance systems require accurate monitoring of objects if/when they violate perimeter restrictions. In this paper, we seek a solution for depth imaging of such objects of interest at high accuracy (few tens of cm) over extended ranges (up to 300 meters) from a single vantage point, such as a pole mounted platform. Unfortunately, the rich literature in depth imaging using camera, lidar and radar in isolation struggles to meet these tight requirements in real-world conditions. This paper proposes Metamoran, a solution that explores long-range depth imaging of objects of interest by fusing the strengths of two complementary technologies: mmWave radar and camera. Unlike cameras, mmWave radars offer excellent cm-scale depth resolution even at very long ranges. However, their angular resolution is at least 10x worse than camera systems. Fusing these two modalities is natural, but in scenes with high clutter and at long ranges, radar reflections are weak and experience spurious artifacts. Metamoran's core contribution is to leverage image segmentation and monocular depth estimation on camera images to help declutter radar and discover true object reflections. We perform a detailed evaluation of Metamoran's depth imaging capabilities in 400 diverse scenarios. Our evaluation shows that Metamoran estimates the depth of static objects up to 90 m away and moving objects up to 305 m away and with a median error of 28 cm, an improvement of 13 x over a naive radar+camera baseline and 23 x compared to monocular depth estimation.

In this paper, we demonstrate Adaptive Millimetro as an extension of Millimetro, an ultra-low power millimeter-wave (mmWave) retro-reflector presented in [1], for high mobility scenarios. Adaptive Millimetro makes use of automotive radars and enables communication with and accurate localization of roadside infrastructure overextended distances (i.e. >100m). Millimetro achieves this by designing ultra-low-power retro-reflective tags that operate in the mmWave frequency band and can be embedded in road signs, pavements, bi-cycles, or even the clothing of pedestrians. Millimetro addresses the severe path loss problem of mmWave signals by combining coding gain and retro-reflective antenna front-end to achieve long-range operation. However, highly mobile scenarios may still experience unreliable performance due to the Doppler effect changing the received signals. In this paper, we demonstrate a simple solution for robust localization in high mobility by implementing a Moving Target Indication (MTI) filter and an adaptive Kalman filter. We also present an augmented reality app, as an in-car AR platform, that uses Adaptive Millimetro’s algorithms to estimate the tag positions and overlay a virtual box at the estimated locations.

Should you decide to launch a nano-satellite today in Low-Earth Orbit (LEO), the cost of renting ground station communication infrastructure is likely to significantly exceed your launch costs. While space launch costs have lowered significantly with innovative launch vehicles, private players, and smaller payloads, access to ground infrastructure remains a luxury. This is especially true for smaller LEO satellites that are only visible at any location for a few tens of minutes a day and whose signals are extremely weak, necessitating bulky and expensive ground station infrastructure.

This paper presents Millimetro, an ultra-low-power tag that can be localized at high accuracy over extended distances. We develop Millimetro in the context of autonomous driving to efficiently localize roadside infrastructure such as lane markers and road signs, even if obscured from view, where visual sensing fails. While RF-based localization offers a natural solution, current ultra-low-power localization systems struggle to operate accurately at extended ranges under strict latency requirements. Millimetro addresses this challenge by re-using existing automotive radars that operate at mmWave frequency where plentiful bandwidth is available to ensure high accuracy and low latency. We address the crucial free space path loss problem experienced by signals from the tag at mmWave bands by building upon Van Atta Arrays that retro-reflect incident energy back towards the transmitting radar with minimal loss and low power consumption. Our experimental results indoors and outdoors demonstrate a scalable system that operates at a desirable range (over 100 m), accuracy (centimeter-level), and ultra-low-power (< 3 uW).

Tire wear is a leading cause of automobile accidents globally. Beyond safety, tire wear affects performance and is an important metric that decides tire replacement, one of the biggest maintenance expense of the global trucking industry. We believe that it is important to measure and monitor tire wear in all automobiles. Current approach to measure tire wear is manual and extremely tedious. Embedding sensor electronics in tires to measure tire wear is challenging, given the inhospitable temperature, pressure and dynamics of the tire. Further, off-tire sensors placed in the well such as laser range-finders are vulnerable to road debris that may settle in tire grooves.

In this paper, we demonstrate Osprey, a tire wear sensor presented in [4]. Osprey makes use of commodity automotive, mmWave RADAR, places it in the tire well of automobiles to image the tire and then measures the tire wear. Osprey measures accurate tire wear continuously while being resilient to road debris and without embedding any electronics in tires. Osprey achieves this by building a super resolution algorithm based on Inverse Synthetic Aperture RADAR imaging and by embedding thin metallic strips along coded patterns in the grooves to combat debris. Here, we implement Osprey on a tire rotation rig and demonstrate the ability to measure tire wear (with and without debris) accurately and detect potentially harmful foreign objects.