Imagine placing your smartphone on a table in a noisy restaurant and clearly capturing the voices of friends seated around you, or recording a lecturer’s voice with clarity in a reverberant auditorium. We introduce SonicSieve, the first intelligent directional speech extraction system for smartphones using a bio-inspired acoustic microstructure. Our passive design embeds directional cues onto incoming speech without any additional electronics. It attaches to the in-line mic of low-cost wired earphones which can be attached to smartphones. We present an end-to-end neural network that processes the raw audio mixtures in real-time on mobile devices. Our results show that SonicSieve achieves a signal quality improvement of 5.0 dB when focusing on a 30° angular region. Additionally, the performance of our system based on only two microphones exceeds that of conventional 5-microphone arrays.

Wireless sensing literature has long aspired to achieve X-ray-like vision at radio frequencies. Yet, state-of-the-art wireless sensing literature has yet to generate the archetypal X-ray image: one of the bones beneath flesh. In this paper, we explore OssiSense, a penetration-based RF-imaging system for imaging bones at mm-resolution, one that significantly exceeds prior penetration-based RF imaging literature. Indeed the long wavelength, significant attenuation and complex diffraction that occur as RF propagates through flesh, have long limited imaging resolution (to several centimeters at best). We address these concerns through a novel penetration-based synthetic aperture algorithm, coupled with a learning-based pipeline to correct for diffraction-induced artifacts. A detailed evaluation of meat models demonstrates a resolution improvement from sub-decimeter to sub-centimeter over prior art in RF penetrative imaging.

Despite researchers having extensively studied various ways to track body pose on-the-go, most prior work does not take into account wheelchair users, leading to poor tracking performance. Wheelchair users could greatly benefit from this pose information to prevent injuries, monitor their health, identify environmental accessibility barriers, and interact with gaming and VR experiences. In this work, we present WheelPoser, a real-time pose estimation system specifically designed for wheelchair users. Our system uses only four strategically placed IMUs on the user’s body and wheelchair, making it far more practical than prior systems using cameras and dense IMU arrays. WheelPoser is able to track a wheelchair user’s pose with a mean joint angle error of 14.30° and a mean joint position error of 6.74 cm, more than three times better than similar systems using sparse IMUs. To train our system, we collect a novel WheelPoser-IMU dataset, consisting of 167 minutes of paired IMU sensor and motion capture data of people in wheelchairs, including wheelchair-specific motions such as propulsion and pressure relief. Finally, we explore the potential application space enabled by our system and discuss future opportunities. Open-source code, models, and dataset can be found here: https://github.com/axle-lab/WheelPoser.

The quality of audio recordings in outdoor environments is often degraded by the presence of wind. Mitigating the impact of wind noise on the perceptual quality of single-channel speech remains a significant challenge due to its non-stationary characteristics. Prior work in noise suppression treats wind noise as a general background noise without explicit modeling of its characteristics. In this paper, we leverage ultrasound as an auxiliary modality to explicitly sense the airflow and characterize the wind noise. We propose a multi-modal deep-learning framework to fuse the ultrasonic Doppler features and speech signals for wind noise reduction. Our results show that DeWinder can significantly improve the noise reduction capabilities of state-of-the-art speech enhancement models.

While localization has gained a tremendous amount of attention from both academia and industry, much less attention has been paid to equally important orientation estimation. Traditional orientation estimation systems relying on gyroscopes suffer from cumulative errors. In this paper, we propose UWBOrient, the first fine-grained orientation estimation system utilizing ultra-wideband (UWB) modules embedded in smartphones. The proposed system presents an alternative solution that is more accurate than gyroscope estimates and free of error accumulation. We propose to fuse UWB estimates with gyroscope estimates to address the challenge associated with UWB estimation alone and further improve the estimation accuracy. UWBOrient decreases the estimation error from the state-of-the-art 7.6° to 2.7° while maintaining a low latency (20 ms) and low energy consumption (40 mWh). Comprehensive experiments with both iPhone and Android smartphones demonstrate the effectiveness of the proposed system under various conditions including natural motion, dynamic multipath and NLoS. Two real-world applications, i.e., head orientation tracking and 3D reconstruction are employed to showcase the practicality of UWBOrient.

Extending the range of RF energy harvesting can revolutionize battery-free/low-power sensing and networking. This paper explores the design space for RF infrastructure to charge battery-free devices (e.g. RFID) or devices with coin-cell batteries (e.g. water and security sensors) over much longer range than the state-of-the-art. Rather than rely completely on ambient RF (e.g. TV towers) or dedicated infrastructure (e.g. RFID readers), we explore a middle path - combine RF energy from (nearly) all available major wireless frequency bands and then supplement this with low-cost specially designed RF charging infrastructure to fill in any gaps.

This paper introduces RFIMap, a system that aims to inexpensively characterize the spatial and temporal distribution of RF spectrum occupancy of any indoor space at fine granularity (tens of centimeters). RFIMap builds rich wide-band indoor spectrum occupancy maps using low-cost and battery-free commodity RFID tags. RFIMap's spectrum maps have wide-ranging applications such as monitoring ambient interference in smart manufacturing, and smart hospitals. RFIMap relies on the observation that commodity RFID tags naturally reflect ambient transmission at other frequency bands, without any modification. RFIMap uses these reflections to estimate the ambient signal power originally received at these tags. RFIMap further performs a careful modeling of indoor multipath to build a dense spectrum map with fine spatial granularity. Our experiments demonstrate spatial spectrum measurement with 2.15 dB of median error at 2.4 GHz, 4.45 dB of median error at 470-700 MHz TV whitespace band, 2.1 dB of median error at 1.8-1.9 GHz in diverse industrial and university settings.

With increasing numbers of nano-satellites (CubeSats) being launched into space in recent years, monitoring their health and debugging become crucial problems. In traditional systems such as big satellites, time-series telemetry is widely used by users to monitor the state of the satellite from the ground stations. However, today smaller CubeSats do not enjoy the benefits of live telemetry due to low throughput and lack of coverage from ground station infrastructure. In this poster, we conduct a motivation study based on data collected from public satellites in low-earth orbit to demonstrate the potential bottlenecks in obtaining live telemetry data from CubeSats. We then describe the design space of possible solutions and opportunities for researchers to improve time-series telemetry for CubeSats.

Acoustic motion tracking is an exciting new research area with promising progress in the last few years. Due to the inherent low propagation speed in the air, acoustic signals have the unique advantage of fine sensing granularity compared to RF signals. Speakers and microphones nowadays are pervasively available in devices surrounding us, such as smartphones and voice-controlled smart speakers. Though promising, one fundamental issue hindering the adoption of acoustic-based motion tracking is that the positions of microphones and speakers inside a device are fixed, which greatly limits the flexibility of acoustic motion tracking. In this work, we propose a new modality of acoustic motion tracking using earphones. Earphone-based tracking mitigates the constraints associated with traditional smartphone-based tracking. With novel designs and comprehensive experiments, we show earphone-based motion tracking can achieve a great flexibility and a high accuracy at the same time. We believe this is an important step towards "earable" sensing.