Covert radio-frequency exfiltration using short, low-duty-cycle transmissions highlights the need for continuous, low-latency spectrum monitoring capable of capturing fleeting emissions. Modern RF environments increasingly feature bursty, opportunistic signals that appear only for milliseconds, making them difficult for traditional receivers to detect. Conventional wideband sensing architectures depend on high-speed ADCs, local oscillators (LO), and mixer chains that draw significant power, while swept-LO and sequential scanning approaches suffer from limited real-time visibility. Parallel filter-bank designs improve instantaneous coverage but remain bulky and energy-intensive. We present SpecSentry, a low-power spectrum-monitoring architecture that maintains wideband visibility while reducing analog complexity by more than an order of magnitude. SpecSentry implements a "mark-fold-capture-detect" workflow in which the RF front end passively imprints frequency-dependent codes onto any active signal and folds the result into a compact representation sampled at rates more than 20× below the Nyquist rate. A two-stage neural regression pipeline analyzes this compressed snapshot to infer key signal characteristics, achieving median detection errors of 0.3 MHz for the bandwidth and 7.89 MHz for the center-frequency of the transmission over a 400 MHz band. This hybrid analog-ML design enables persistent, wideband, ultra-low-power monitoring suited for mobile, unattended, and maintenance-free deployments where detecting ephemeral transmissions is essential.

Immersive telepresence has the potential to transform real-time communication through highly interactive and engaging experiences. Despite recent advances in reducing communication and computation costs, existing systems largely overlook packet loss, which can severely degrade the quality of experience (QoE). Recovering lost immersive content is considerably more challenging than in 2D video due to the complexity of dense 3D representations. Recovery must be both accurate and timely while minimizing the communication and computation overhead it incurs. To address these challenges, we present LITE, the first loss-resilient immersive telepresence system. LITE incorporates three key design principles: (1) leveraging semantic communication to transmit compact motion and audio semantics, which can be reconstructed into the remote user's immersive representation and voice, enabling fast semantic-level recovery and remaining robust to congestion-control-induced rate reductions under loss; (2) fusing audio and motion semantics via a lightweight multimodal model to achieve accurate, real-time recovery of motion semantics; and (3) encoding audio semantics from multiple past frames into succinct neural redundancy to enable robust recovery. We prototype LITE using a well-known parametric facial motion representation and extensively evaluate its performance across diverse networks. Our results demonstrate that LITE improves QoE by up to 109% compared with existing schemes, while sustaining real-time streaming at 30 frames per second and preserving high visual fidelity (structural similarity index measure above 0.9, where 1 indicates perfect similarity).

Integrating spatial context into large language models (LLMs) has the potential to revolutionize human-computer interaction, particularly in wearable devices. In this work, we present a novel system architecture that incorporates spatial speech understanding into LLMs, enabling contextually aware and adaptive applications for wearable technologies. Our approach leverages microstructure-based spatial sensing to extract precise Direction of Arrival (DoA) information using a monaural microphone. To address the lack of existing dataset for microstructure-assisted speech recordings, we synthetically create a dataset by using the LibriSpeech dataset. This spatial information is fused with linguistic embeddings from OpenAI’s Whisper model, allowing each modality to learn complementary contextual representations. The fused embeddings are aligned with the input space of LLaMA-3.2 3B model and fine-tuned with lightweight adaptation technique LoRA to optimize for on-device processing. SING supports spatially-aware automatic speech recognition (ASR), achieving a mean error of 25.72°—a substantial improvement compared to the 88.52° median error in existing work—with a word error rate (WER) of 5.3. SING also supports soundscaping, for example, inference how many people were talking and their directions, with up to 5 people and a median DoA error of 16°. Our system demonstrates superior performance in spatial speech understanding while addressing the challenges of power efficiency, privacy, and hardware constraints, paving the way for advanced applications in augmented reality, accessibility, and immersive experiences.

Estimation of a speaker’s direction and head orientation with binaural recordings can be a critical piece of information in many real-world applications with emerging ‘earable’ devices, including smart headphones and AR/VR headsets. However, it requires predicting the mutual head orientations of both the speaker and the listener, which is challenging in practice. This paper presents a system for jointly predicting speaker-listener head orientations by leveraging inherent human voice directivity and listener’s head-related transfer function (HRTF) as perceived by the ear-mounted microphones on the listener. We propose a convolution neural network model that, given binaural speech recording, can predict the orientation of both speaker and listener with respect to the line joining the two. The system builds on the core observation that the recordings from the left and right ears are differentially affected by the voice directivity as well as the HRTF. We also incorporate the fact that voice is more directional at higher frequencies compared to lower frequencies. Our proposed system achieves 2.5° 90th percentile error in the listener’s head orientation and 12.5° 90th percentile error for that of the speaker.

This demonstration presents LiTEfoot, an ultra-low power localization system leveraging ambient cellular signals. To address the limitations of traditional GPS-based tracking systems in terms of power consumption and latency, LiTEfoot employs a non-linear transformation of the cellular spectrum to achieve efficient self-localization. Our design uses a simple envelope detector to realize spectrum folding, enabling the identification of multiple active base stations. The LiTEfoot prototype shows a median localization error of 22 meters in urban areas and 50 meters in rural areas, consuming only 40 μJoules of energy per localization update.

In this paper, we introduce a low-power wide-area cellular localization system, called LiTEfoot. The core architecture of the radio carefully applies non-linear transform of the entire cellular spectrum to obtain a systematic superimposition of the synchronization signals at the baseband. The system develops methods to simultaneously identify all the base stations that are active at any cellular band from the transformed signal. The radio front end uses a simple envelop detector to realize the non-linear transformation. We build on this low-power radio to implement a self-localization system leveraging ambient 4G-LTE signals. We show that the core system can also be extended to other cellular technologies like 5G-NR and NB-IoT. The prototype achieves a median localization error of 22 meters in urban areas and 50 meters in rural areas. It can sense a 3GHz wideband LTE spectrum in 10ms using non-linear intermodulation while consuming 0.9 mJ of energy for a PCB-based implementation and 40 μJ for CMOS simulation. In other words, LiTEfoot tags can last for 11 years on a coin cell while continuously estimating location every 5 seconds. We believe that LiTEfoot will have widespread implications in city-scale asset tracking and other location-based services. The radio architecture can be useful beyond low-power self-localization and can find application in synchronization and communication on battery-less platforms.

Low-power sensor networks are transforming large-scale sensing in precision farming, livestock tracking, climate-monitoring and surveying. Accurate and robust localization in such low-power sensor nodes has never been as crucial as it is today. This paper presents, Sirius, a self-localization system using a single receiver for low-power IoT nodes. Traditionally, systems have relied on antenna arrays and tight synchronization to estimate angle-of-arrival (AoA) and time-of-flight with known access points. While these techniques work well for regular mobile systems, low-power IoT nodes lack the resources to support these complex systems. Sirius explores the use of gain-pattern reconfigurable antennas with passive envelope detector-based radios to perform AoA estimation without requiring any kind of synchronization. It shows a technique to embed direction specific codes to the received signals which are transparent to regular communication channel but carry AoA information with them. Sirius embeds these direction-specific codes by using reconfigurable antennas and fluctuating the gain pattern of the antenna. Our prototype demonstrates a median error of 7 degrees in AoA estimation and 2.5 meters in localization, which is similar to state-of-the-art antenna array-based systems. Sirius opens up new possibilities for low-power IoT nodes.

In this poster, we present a new approach to low-power self-localization for IoT nodes called Sirius. With the rise of low-power sensor networks in precision farming, climate monitoring, and surveying, it has become increasingly critical to accurately and robustly localize low-power sensor nodes. However, traditional systems that rely on antenna arrays and time synchronization are too complex for low-power IoT nodes. To overcome this limitation, Sirius utilizes gain-pattern reconfigurable antennas with passive envelope detector-based radios to estimate angle-of-arrival. This is achieved by embedding direction-specific codes in the received signals, which carry angle-of-arrival information. Our prototype has demonstrated a median error of 7 degrees in AoA estimation and 2.5 meters in localization, comparable to state-of-the-art antenna array-based systems. This new approach opens up exciting possibilities for low-power IoT nodes in various fields.

Malicious editing of audiovisual content has emerged as a popular tool for targeted defamation, spreading disinformation, and triggering political unrest. Public speeches and statements of political leaders, public figures, or celebrities are particularly at target due to their effectiveness in influencing the masses. Ubiquitous audiovisual recording of live speeches with smart devices and unrestricted content sharing and redistributing on social media make it difficult to address this threat using existing authentication techniques. Given public recordings of live events lack source control over the media, standard solutions falter. This paper presents TalkLock, a speech integrity verification system that can enable live speakers to protect their speeches from malicious alterations even when the speech is recorded by any member of the audience. The core idea is to generate meta-information from the speech signal in real-time and disseminate it through a secure QR code-based screen-camera communication. The QR code when recorded along with the speech embeds the meta-information in the content and it can be used later for independent verification in stand-alone applications or online platforms. A user study with live speech and real-world experiments with different types of voices, languages, environments, and distances show that TalkLock can verify fake content with 94.4% accuracy.

Malicious editing to alter audiovisual content has become increasingly prevalent in recent years, as it allows for targeted defamation, the dissemination of disinformation, and the incitement of political unrest. Public speeches and statements made by political leaders, public figures, and celebrities are particularly vulnerable to such attacks, as they have the power to sway public opinion. The widespread use of smart devices to record live speeches, combined with unrestricted content sharing and redistribution on social media platforms, makes it difficult to prevent the spread of manipulated media. Existing solutions, which rely on source control over the media, are not effective for live events. This paper presents TalkLock, a speech integrity verification system that can enable live speakers to protect their speeches from malicious alterations even when the speech is recorded by any member of the audience. The core idea is to generate meta-information from the speech signal in real-time and disseminate it through a secure QR code-based screen-camera communication. The QR code when recorded along with the speech embeds the meta-information in the content and it can be used later for independent verification in stand-alone applications or online platforms. A user study with live speech and real-world experiments with different types of voices, languages, environments, and distances show that TalkLock can verify fake content with 94.4% accuracy.

This paper presents the design and implementation of SPiDR, an ultra-low-power spatial sensing system for miniature mobile robots. This acoustic sensor produces a cross-sectional map of the field-of-view using only one speaker/microphone pair. While it is challenging to have enough spatial diversity of signal with a single omnidirectional source, we leverage sound's interaction with small structures to create a 3D-printed passive filter, called a stencil, that can project spatially coded signals on a region at a fine granularity. The system receives a linear combination of the reflections from nearby objects and applies a novel power-aware depth-map reconstruction algorithm. The algorithm first estimates the approximate locations of the objects in the scene and then iteratively applies fractional multi-resolution inversion. SPiDR consumes only 10mW of power to generate a depth-map in real-world scenario with over 80% structural similarity score with the scene.

This poster presents the design and implementation of SPiDR, an ultra-low-power acoustic imaging system. This imaging system produces a cross-sectional map of the field-of-view using only one speaker/microphone pair. It leverages the fact that sound's interaction with small structures can project spatially coded signals on a region at a fine granularity. We create a 3D-printed passive filter, called a stencil, that can image the scene with a single omnidirectional source and sensor. With spatially coded signal, the system receives a linear combination of the reflections from nearby objects and applies a novel power-aware depth-map reconstruction algorithm. SPiDR consumes only 10mW of power to generate a depth-map in real-world scenario with over 80% structural similarity score with the scene.

This paper presents a low-power and miniaturized design for acoustic direction-of-arrival (DoA) estimation and source localization, called Owlet. The required aperture, power consumption, and hardware complexity of the traditional array-based spatial sensing techniques make them unsuitable for small and power-constrained IoT devices. Aiming to overcome these fundamental limitations, Owlet explores acoustic microstructures for extracting spatial information. It uses a carefully designed 3D-printed metamaterial structure that covers the microphone. The structure embeds a direction-specific signature in the recorded sounds. Owlet system learns the directional signatures through a one-time in-lab calibration. The system uses an additional microphone as a reference channel and develops techniques that eliminate environmental variation, making the design robust to noises and multipaths in arbitrary locations of operations. Owlet prototype shows 3.6° median error in DoA estimation and 10cm median error in source localization while using a 1.5cm × 1.3cm acoustic structure for sensing. The prototype consumes less than 100th of the energy required by a traditional microphone array to achieve similar DoA estimation accuracy. Owlet opens up possibilities of low-power sensing through 3D-printed passive structures.

This demonstration presents a working prototype of Owlet, an alternative design for spatial sensing of acoustic signals. To overcome the fundamental limitations in form-factor, power consumption, and hardware requirements with array-based techniques, Owlet explores wave's interaction with acoustic structures for sensing. By combining passive acoustic microstructures with microphones, we envision achieving the same functionalities as microphone and speaker arrays with less power consumption and in a smaller form factor. Our design uses a 3D-printed metamaterial structure over a microphone to introduce a carefully designed spatial signature to the recorded signal. Owlet prototype shows 3.6° median error in Direction-of-Arrival (DoA) estimation and 10 cm median error in source localization while using a 1.5cm × 1.3cm acoustic structure for sensing.

Eavesdropping on private conversations is one of the most common yet detrimental threats to privacy. A number of recent works have explored side-channels on smart devices for recording sounds without permission. This paper presents LidarPhone, a novel acoustic side-channel attack through the lidar sensors equipped in popular commodity robot vacuum cleaners. The core idea is to repurpose the lidar to a laser-based microphone that can sense sounds from subtle vibrations induced on nearby objects. LidarPhone carefully processes and extracts traces of sound signals from inherently noisy laser reflections to capture privacy sensitive information (such as speech emitted by a victim's computer speaker as the victim is engaged in a teleconferencing meeting; or known music clips from television shows emitted by a victim's TV set, potentially leaking the victim's political orientation or viewing preferences). We implement LidarPhone on a Xiaomi Roborock vacuum cleaning robot and evaluate the feasibility of the attack through comprehensive real-world experiments. We use the prototype to collect both spoken digits and music played by a computer speaker and a TV soundbar, of more than 30k utterances totaling over 19 hours of recorded audio. LidarPhone achieves approximately 91% and 90% average accuracies of digit and music classifications, respectively.

Private conversations are an attractive target for malicious actors intending to conduct audio eavesdropping attacks. Previous works discovered unexpected vectors for these attacks, such as analyzing high-speed video of objects adjacent to sound sources, or using WiFi signal information. We propose LidarPhone, a novel side-channel attack that exploits the lidar sensors in commodity robot vacuum cleaners to perform acoustic eavesdropping attacks. LidarPhone is able to detect the minute vibrations induced on objects that are near audio sources, and extract meaningful signals from inherently noisy raw lidar returns. We evaluate a realistic scenario for potential victims: recovering privacy-sensitive digits (e.g., credit card numbers, social security numbers) emitted by computer speakers during teleconferencing calls. We implement LidarPhone on a Xiaomi Roborock vacuum cleaning robot and perform a comprehensive series of real-world experiments to determine its performance. LidarPhone achieves up to 91% accuracy for digit classification.

Ear devices, such as noise-canceling headphones and hearing aids, have dramatically changed the way we listen to the outside world. We re-envision this area by combining wireless communication with acoustics. The core idea is to scatter IoT devices in the environment that listen to ambient sound and forward it over their wireless radio. Since wireless signals travel much faster than sound, the ear-device receives the sound much earlier than its actual arrival. This "glimpse" into the future allows sufficient time for acoustic digital processing, serving as a valuable opportunity for various signal processing and machine learning applications. We believe this will enable a digital app store around human ears.

Active Noise Cancellation (ANC) is a classical area where noise in the environment is canceled by producing anti-noise signals near the human ears (e.g., in Bose's noise cancellation headphones). This paper brings IoT to active noise cancellation by combining wireless communication with acoustics. The core idea is to place an IoT device in the environment that listens to ambient sounds and forwards the sound over its wireless radio. Since wireless signals travel much faster than sound, our ear-device receives the sound in advance of its actual arrival. This serves as a glimpse into the future, that we call lookahead, and proves crucial for real-time noise cancellation, especially for unpredictable, wide-band sounds like music and speech. Using custom IoT hardware, as well as lookahead-aware cancellation algorithms, we demonstrate MUTE, a fully functional noise cancellation prototype that outperforms Bose's latest ANC headphone. Importantly, our design does not need to block the ear - the ear canal remains open, making it comfortable (and healthier) for continuous use.

Consider sounds, say at 40kHz, that are completely outside the human's audible range (20kHz), as well as a microphone's recordable range (24kHz). We show that these high frequency sounds can be designed to become recordable by unmodified microphones, while remaining inaudible to humans. The core idea lies in exploiting non-linearities in microphone hardware. Briefly, we design the sound and play it on a speaker such that, after passing through the microphone's non-linear diaphragm and power-amplifier, the signal creates a "shadow" in the audible frequency range. The shadow can be regulated to carry data bits, thereby enabling an acoustic (but inaudible) communication channel to today's microphones. Other applications include jamming spy microphones in the environment, live watermarking of music in a concert, and even acoustic denial-of-service (DoS) attacks. This paper presents BackDoor, a system that develops the technical building blocks for harnessing this opportunity. Reported results achieve upwards of 4kbps for proximate data communication, as well as room-level privacy protection against electronic eavesdropping.

We demonstrate that high frequency ultrasonic sounds can be designed to become recordable by unmodified smartphone microphones, while remaining inaudible to humans. The core idea lies in exploiting nonlinearities in microphone hardware with a combination of ultrasound frequencies. These frequencies can be regulated to carry data bits, thereby enabling an acoustic (but inaudible) communication channel to today's microphones. Other applications include jamming spy microphones in the environment, live watermarking of music in a concert, and even acoustic Denial-of-Service (DoS) attacks.

This paper studies the feasibility of minimizing convergence delay and forwarding disruption without carrying any additional bits in the IP header, to provide high availability despite link failures in traditional IP networks. Previously proposed mechanisms achieve two of these three objectives by trading off the other objective. For instance, the ordered forwarding information base updates approach may prolong the convergence delay, whereas the SafeGuard scheme requires carrying the path cost in the IP header. As a better alternative, we propose a scheme called fast convergence with fast reroute (FCFR), which combines the features of IP fast rerouting and interface-specific forwarding. We show that FCFR can achieve minimal convergence delay, while ensuring loop-free delivery during convergence, after a single non-partitioning failure in an IP network, without altering the IP header format, making it amenable for immediate deployment.

Randomized backoff is a well-established approach for avoiding collisions in CSMA networks. Today's backoff operation, such as in WiFi, attempts to create a total ordering among all the nodes contending for the channel. Total ordering requires assigning a unique backoff to each node, which is achieved by having nodes choose their back-offs from a large range, ultimately leading to channel wastage. This paper observes that total ordering can be achieved more efficiently. We propose “hierarchical backoff” in which nodes pick random numbers from a smaller range, resulting in groups of nodes picking the same number (i.e., partial order). Now, the group of nodes that picks the smallest number is advanced to a second round, where they again perform the same operation. This results in more efficient backoff because the time for partially ordering all nodes plus totally ordering each small groups is actually less than the time needed to totally order all nodes. Realizing the above intuition requires addressing new protocol challenges in group signaling, the feasibility of which is demonstrated on a USRP/GNUradio prototype. Large scale simulations also show consistent throughput gains by incorporating the proposed backoff approach into two CSMA protocols - WiFi and oCSMA. We also show that the proposed approach can be complementary to and even outperform existing backoff optimization schemes.

Due to the popularity of soccer and the purposeful use of the head during play, traumatic brain injury to soccer players has been a concern for decades. However, there is a sense of urgency now in understanding and preventing concussions better, due to raising public awareness. Towards that end, intra-oral devices such as Vector MouthGuards are being studied [3] to measure the athlete's head's linear and rotational accelerations from impacts experienced in practices and games. But given the players' natural distaste for such intra-oral devices, more palatable alternatives for head impact monitoring are being developed [2]. X2 Biosystems xPatchis an electronic skin patch thatis worn behind the ear. Reebok Checklight embeds the impact sensor in the back of a skullcap which can be worn with or without a helmet. Triax SIM-P is placed inside a headband for non-helmeted sports and a skullcap for helmeted sports. While all these devices are much more convenient to wear than intra-oral devices, it is yet to be seen whether they gain wider acceptance, particularly by the millions of amateur soccer players all over the world.

This paper demonstrates the feasibility of using the vibration motor in mobile devices as a sound sensor, almost like a microphone. We show that the vibrating mass inside the motor -- designed to oscillate to changing magnetic fields -- also responds to air vibrations from nearby sounds. With appropriate processing, the responses become intelligible, to the extent that humans can understand the vibra-motor recorded words with greater than 80% average accuracy. Even off-the-shelf speech recognition softwares are able to decode at 60% accuracy, without any training or machine learning. We present our overall techniques and results through a system called VibraPhone, and discuss implications to both sensing and security.

Mobile computing has traditionally implied mobile clients connected to a static infrastructure. This paper breaks away from this point of view and envisions the possibility of injecting mobility into infrastructure. We envision a WiFi access point on wheels, that moves to optimize desired performance metrics. Movements need not necessarily be all around the floor of a home or office, neither do they have to operate on batteries, or connect wirelessly to the Internet. At homes, they could remain tethered to power and Ethernet outlets while moving in small areas (perhaps under the study table). In offices of the future, perhaps APs could move on tracks installed on top of false ceilings.

This paper describes WalkCompass, a system that exploits smartphone sensors to estimate the direction in which a user is walking. We find that several smartphone localization systems in the recent past, including our own, make a simplifying assumption that the user's walking direction is known. In trying to relax this assumption, we were not able to find a generic solution from past work. While intuition suggests that the walking direction should be detectable through the accelerometer, in reality this direction gets blended into various other motion patterns during the act of walking, including up and down bounce, side-to-side sway, swing of arms or legs, etc. Moreover, the walking direction is in the phone's local coordinate system (e.g., along Y axis), and translation to global directions, such as 45 degree North, can be challenging when the compass is itself erroneous. WalkCompass copes with these challenges and develops a stable technique to estimate the user's walking direction within a few steps. Results drawn from 15 different environments demonstrate median error of less than 8 degrees, across 6 different users, 3 surfaces, and 3 holding positions. While there is room for improvement, we believe our current system can be immediately useful to various applications centered around localization and human activity recognition.

We present a demonstration of WalkCompass, a system to appear in the MobiSys 2014 main conference. WalkCompass exploits smartphone sensors to estimate the direction in which a user is walking. We find that several smartphone localization systems in the recent past, including our own, make a simplifying assumption that the user's walking direction is known. In trying to relax this assumption, we were not able to find a generic solution from past work. While intuition suggests that the walking direction should be detectable through the accelerometer, in reality this direction gets blended into various other motion patterns during the act of walking, including up and down bounce, side-to-side sway, swing of arms or legs, etc. WalkCompass analyzes the human walking dynamics to estimate the dominating forces and uses this knowledge to find the heading direction of the pedestrian. In the demonstration we will show the performance of this system when the user holds the smartphone on the palm. A collection of YouTube videos of the demo is posted at http://synrg.csl.illinois.edu/projects/ localization/walkcompass.