The perception of autonomous vehicles using radars has attracted increased research interest due its ability to operate in fog and bad weather. However, training radar models is hindered by the cost and difficulty of annotating largescale radar data. To overcome this bottleneck, we propose a self-supervised learning framework to leverage the large amount of unlabeled radar data to pre-train radar only embeddings for self-driving perception tasks. The proposed method combines radar-to-radar and radar-to-vision contrastive losses to learn a general representation from unlabeled radar heatmaps paired with their corresponding camera images. When used for downstream object detection, we demonstrate that the proposed self-supervision framework can improve the accuracy of state-of-the-art supervised baselines by 5.8% in mAP. Code is available at https://github.com/yiduohao/Radical.

Next generation cellular networks will implement radio sensing functions alongside customary communications, thereby enabling unprecedented worldwide sensing coverage outdoors. Deep learning has revolutionised computer vision but has had limited application to radio perception tasks, in part due to lack of systematic datasets and benchmarks dedicated to the study of the performance and promise of radio sensing. To address this gap, we present MaxRay: a synthetic radio-visual dataset and benchmark that facilitate precise target localisation in radio. We further propose to learn to localise targets in radio without supervision by extracting self-coordinates from radio-visual correspondence. We use such self-supervised coordinates to train a radio localiser network. We characterise our performance against a number of state-of-the-art baselines. Our results indicate that accurate radio target localisation can be automatically learned from paired radio-visual data without labels, which is important for empirical data. This opens the door for vast data scalability and may prove key to realising the promise of robust radio sensing atop a unified communication-perception cellular infrastructure. Dataset will be hosted on IEEE DataPort.

WiFi channel state information (CSI) has emerged as a plausible modality for sensing different human vital signs, i.e., respiration and body motion, as a function of modulated wireless signals that travel between WiFi devices. Although a remarkable proposition, most of the existing research in this space struggles to withstand robust performance beyond experimental conditions. To this end, we take a careful look at the dynamics of WiFi signals under human respiration and body motions in the wild. We first characterize the WiFi signal components—multipath and signal subspace—that are modulated by human respiration and body motions. We extrapolate on a set of transformations, including first-order differentiation, max-min normalization and component projections, that faithfully explains and quantifies the dynamics of respiration and body motions on WiFi signals. Grounded in this characterization, we propose two methods: 1) a respiration tracking technique that models the peak dynamics observed in the time-varying signal subspace and 2) a body-motion tracking technique built with a multi-dimensional clustering of evolving signal subspace. Finally, we reflect on the manifestation of these techniques in a practical sleep monitoring application. Our systematic evaluation with over 550 hours of data from 5 users covering both line-of-sight (LOS) and non-line-of-sight (NLOS) settings shows that the proposed techniques can achieve comparable performance to purpose-built pulse-Doppler radar.

In future 6G cellular networks, a joint communication and sensing protocol will allow the network to perceive the environment, opening the door for many new applications atop a unified communication-perception infrastructure. However, interpreting the sparse radio representation of sensing scenes is challenging, which hinders the potential of these emergent systems.We propose to combine radio and vision to automatically learn a radio-only sensing model with minimal human intervention. We want to build a radio sensing model that can feed on millions of uncurated data points. To this end, we leverage recent advances in self-supervised learning and formulate a new label-free radio-visual co-learning scheme, whereby vision trains radio via cross-modal mutual information. We implement and evaluate our scheme according to the common linear classification benchmark, and report qualitative and quantitative performance metrics. In our evaluation, the representation learnt by radio-visual self-supervision works well for a downstream sensing demonstrator, and outperforms its fully-supervised counterpart when less labelled data is used. This indicates that self-supervised learning could be an important enabler for future scalable radio sensing systems.

For long, we have studied tiny energy harvesters to liberate sensors from batteries. With remarkable progress in embedded deep learning, we are now re-imagining these sensors as intelligent compute nodes. Naturally, we are approaching a crossroad where sensor intelligence is meeting energy autonomy enabling maintenance-free swarm intelligence and unleashing a plethora of applications ranging from precision agriculture to ubiquitous asset tracking to infrastructure monitoring. One of the critical challenges, however, is to adapt intelligence fidelity in response to available energy to maximise the overall system availability. To this end, we present the design and implementation of ePerceptive: a novel framework for best-effort embedded intelligence, i.e., inference fidelity varies in proportion to the instantaneous energy supplied. ePerceptive operates on two core principles. First, it enables training a single deep neural network (DNN) to operate on multiple input resolutions without compromising accuracy or incurring memory overhead. Second, it modifies a DNN architecture by injecting multiple exits to guarantee valid, albeit lower-fidelity inferences in the event of energy interruption. The combination of these techniques offers a smooth adaptation between inference latency and recognition accuracy while matching the computational load to the available power budget. We report the manifestation of ePerceptive in designing batteryless cameras and microphones built with TI MSP430 MCU and off-the-shelf RF and solar energy harvesters. Our evaluation of these batteryless sensors with multiple vision and acoustic workloads suggest that the dynamic adaptation of ePerceptive can increase the inference throughput by up to 80% compared to a static baseline while ensuring a maximum accuracy drop of less than 6%.

We propose a system for enabling auxiliary communication channels in which a node transmits a millimeter (mm) wave signal which is reflected off a deliberately vibrating surface of a second node and then received by the first node. Data sequences can be encoded in the modulation of the surface, and radar sensing techniques can be used to demodulate the reflected signal. Hence our system enables not only conventional sensing in terms of range, velocity, and orientation estimation but also allows for information to be conveyed by the sensed device. We introduce the design of a metasurface driven by an energy-efficient programmable piezo-electric actuator, detail suitable radar processing, and characterize the link performance of the kinetically induced channel at distances up to five meters. As this metasurface could be used for both mobile devices and infrastructure devices, we describe opportunities for enabling novel capabilities including secure device authentication and extended-range wireless sensing across multiple devices.

We present an efficient CDMA detection core suited for multiuser indoor acoustic positioning. An ultrasonic multi-code despreader is proposed, allowing simultaneous broadband acoustic ranging signals to be processed in real time by embedded sensors. The ranging performance is characterised using a dataset gathered from a real deployment of ultrasonic devices and is shown to be favourable. The proposed core can be used as a basis for more sophisticated receivers, such as those capable of detecting heavily Doppler-shifted signals.