Cardiovascular diseases remain the leading cause of death worldwide, and Doppler blood flow indices play a crucial role in their early detection and diagnosis. Existing ultrasound devices, though effective, are costly, bulky, and unsuitable for daily or at-home use. In this work, we introduce Phonotonos, a system that transforms commodity smartphones into portable and accessible tools for through-skin ultrasonic sensing of blood flow velocity. We demonstrate that despite smartphones' inherent limitations—including low ultrasound power, coarse spatial resolution, and lack of beamforming—blood flow velocity waveforms can still be recovered using novel phase-based signal processing, including nonlinearity cancellation and baseline drift removal. We further propose a triple-modality framework that fuses Doppler ultrasound, arterial sound, and IMU for robust artery localization and interfering motion (e.g., involuntary hand motion) rejection. Extensive simulations, phantom experiments, and IRB-approved human studies validate that our proposed system can measure four key Doppler indices (AT, S/D ratio, RI, PI) with accuracy comparable to dedicated medical devices. These results highlight the potential of smartphones to democratize vascular health monitoring and enable continuous cardiovascular screening in everyday environments.

Acoustic signal has been used to sense important contextual information of targets such as location and movement. This paper explores its potential for measuring the geometric information of moving targets, i.e., shape and size. We propose SONDAR, a novel shape and size measurement system using Inverse Synthetic Aperture Radar (ISAR) imaging on commodity devices. We first design a Doppler-free echo alignment method to accurately align target reflections even if there exists a severe Doppler effect. Then we down-convert the received signals reflected from multiple scatter points on the target and construct a modulated downchirp signal to generate the image. We further develop a lightweight approach to extract the geometric information from a 2-D frequency image. We implement and evaluate a proof-of-concept system on both a Bela platform and a smartphone. Extensive experiments show that we can correctly estimate the target shape and achieve a millimeterlevel size measurement accuracy. Our system can achieve high accuracies even when the target moves along a deviated trajectory, at a relatively high speed, and under obstruction.

Acoustic sensing on smartphones has gained extensive attention from both industry and research communities. Prior studies suffer from one fundamental limit, i.e., audio sampling rates on smartphones are constrained at 48 kHz. In this work, we present PowerPhone, a software reconfiguration to support higher sampling rates on both microphones and speakers of smartphones. We reverse-engineered more than 100 smartphones and found that their sampling rates can be reconfigured to 192 kHz. We conducted benchmark experiments and showcased field studies to demonstrate the unleashed sensing capability using our reconfigured smart-phones. First, we improve the sensing resolution from 7 cm to 1cm and enable multi-finger gesture recognition on smart-phones. Second, we push the sensing granularity of subtle movements to 2 μm and show the feasibility of turning the smartphone into a micrometer-level machine vibration meter. Third, we increase the sensing range to 6 m and showcase room-scale human presence detection using a smartphone. Finally, we demonstrate that PowerPhone can enable new applications that were previously infeasible. Specifically, we can detect the home appliance status by analyzing ultrasonic leakages above 24 kHz from the wireless charger while charging a smartphone. Our open-source artifacts can be found at: https://powerphone.github.io.

Acoustic sensing is a new sensing modality that senses the contexts of human targets and our surroundings using acoustic signals. It becomes a hot topic in both academia and industry owing to its finer sensing granularity and the wide availability of microphone and speaker on commodity devices. While prior studies focused on addressing well-known challenges such as increasing the limited sensing range and enabling multi-target sensing, we propose a novel scheme to leverage the non-linearity distortion of microphones to further boost the sensing granularity. Specifically, we observe the existence of the non-linear signal generated by the direct path signal and target reflection signal. We mathematically show that the non-linear chirp signal amplifies the phase variations and this property can be utilized to improve the granularity of acoustic sensing. Experiment results show that, by properly leveraging the hardware non-linearity, the amplitude estimation error for sub-millimeter-level vibration can be reduced from 0.137 mm to 0.029 mm.

While a tremendous amount of effort has been devoted to localization, the orientation of a device, especially in 3D space, is seldom explored. Although many sensor-based methods utilizing gyro-scope, accelerometer, and magnetometer have been proposed to measure 3D orientation, these methods generally suffer from high cumulative errors and performance degradation when the device is moving. In this paper, we present MOM, the first microphone-based system that estimates the 3D orientation of a device. The key idea of MOM is to employ free sound sources in our surrounding environment as anchors. The prior knowledge of these sound sources, including the signal waveform and the locations of the sound sources, is not required to be known. In particular, we propose an angle-of-arrival (AoA) extraction algorithm that compares fine-grained time delays over microphones at a low computational cost. We implement our system on three platforms including a 6-microphone array Seeed Studio ReSpeaker, a commodity earphone Sennheiser AMBEO smart headset and a commodity smartphone Google Pixel 4. Extensive experiments show that MOM can achieve significantly higher accuracy compared with status quo approaches and is robust against cumulative errors. We apply MOM to two real-life applications, i.e., head tracking and 3D reconstruction, to demonstrate the applicability and generality of MOM in practice.

Acoustic sensing shows great potential to transform billions of consumer-grade electronic devices that people interact with on a daily basis into ubiquitous sensing platforms. In this paper, we share our experience and findings during the process of developing and deploying acoustic sensing systems for real-world usage. We identify multiple practical problems that were not paid attention to in the research community, and propose the corresponding solutions. The challenges include: (i) there exists annoying audible sound leakage caused by acoustic sensing; (ii) acoustic sensing actually affects music play and voice call; (iii) acoustic sensing consumes a significant amount of power, degrading the battery life; (iv) real-world device mobility can fail acoustic sensing. We hope the shared experience can benefit not only the future development of sensing algorithms but also the hardware design, pushing acoustic sensing one step further towards real-life adoption.

Acoustic signal has been recently adopted for contact-free hand gesture recognition due to its fine-grained sensing granularity and wide availability of microphone and speaker in consumer-grade electronic devices such as smartphones. However, a very limited sensing range constrains acoustic sensing to application scenarios where users interact with devices in close proximity. In this paper, we improve the range of acoustic sensing and demonstrate the feasibility of enabling room-scale hand gesture recognition using commodity smart speakers. We develop a series of novel signal processing techniques and implement our system on two commodity smart speaker prototypes with different numbers of microphones. Extensive evaluations are performed in three different environments with 1440 gestures collected from 16 participants. Experiment results show that our system can significantly increase the sensing range from 1 m to 4--5 m. In the challenging scenario where the user is 4 m away from the smart speaker and there is strong interference, the achieved gesture recognition accuracy is still higher than 90%.

Contactless acoustic motion tracking enables new opportunities to interact with smart devices, such as smartphones and voice-controlled smart assistants. The speakers and microphones integrated in these devices provide unique opportunities to simultaneously track multiple targets in a fine-grained manner. To this end, we propose a system, namely FM-Track, that enables contactless multi-target tracking using acoustic signals. We first introduce a signal model to characterize the location and motion status of targets by fusing the information from multiple dimensions (i.e., range, velocity, and angle of targets). Then we develop a series of techniques to separate signals reflected from multiple targets and accurately track each individual target. We implement and evaluate FM-Track on both research-purpose hardware platform (i.e., Bela) and commercial devices (i.e., smartphones and smart speakers). Extensive experiments show that FM-Track can successfully differentiate two targets with a spacing as small as 1 cm, and achieve a median tracking accuracy of 0.86 cm and 0.11 cm for absolute range and displacement estimates respectively. For multi-target tracking, FM-Track can accurately track four targets and the tracking range can be up to 3 m.

2018 IEEE International Conference on Pervasive Computing and Communications (PerCom), Athens, Greece, March 19-23, 2018

Gesture recognition is an innovative technology which is fundamentally reshaping the way people live, entertain and work. However, most gesture recognition systems focus on the recognition of simple gestures and ignore the full potential of sequential gestures involving a series of temporally-related simple actions in order. This paper presents SGRS, a battery-free, scalable and non-specific sequential gesture recognition system based on COTS RFID. The key insight is that finegrained phase information extracted from RF signals is capable of perceiving various gestures. In SGRS, we meticulously devise gesture recognition mechanism by incorporating the k-means based vector quantizer and string matching algorithm to enable precise and real-time sequential gesture identification. Moreover, an improved edit distance algorithm is proposed for suppressing individual diversity. We implement SGRS and comprehensively evaluate the performance by recognizing traffic command gestures of Chinese traffic police. Experimental result shows that SGRS achieves an average recognition accuracy of 96.2% with eight sequential gestures and is highly robust to both individual diversity and multipath effect.

Innovative Human Machine Interface technologies are fundamentally reshaping the way people live, entertain and work. Passive RFID tags, benefiting from its wireless, inexpensive and battery-free sensing ability, are gradually being applied in new-style interaction interfaces, ranging from virtual touch screen to 3D mouse. This paper presents TagController, a universal wireless and battery-free remote controller with two types of interactive actions. The key insight is that the fine-grained phase information extracted from RF signals is capable of perceiving various actions. TagController can recognize 10 actions without any training or prestored profiles by executing a sequence of functional components, i.e. preprocessor, action detector and action recognizer. We have implemented TagController with COTS RFID devices and conducted substantial experiments in different scenarios. The results demonstrate that TagController can achieve an average recognition accuracy of 95.8% and 94.3% in the scenarios of one and two remote controllers, respectively, which promises its feasibility and robustness.

Workflow recognition is a key technique in the field of activity recognition with benefits of monitoring the step being performed in the workflow, detecting the missing step, and providing assistance to the performer of the workflow, among others. In this paper, we present an unobtrusive workflow recognition system called RFlow-ID, which is the first device-free, battery-free and privacy-preserving workflow recognition system based on RFID technique. RFlow-ID perceives the use and movement of associated objects in the workflow using fine-grained phase information extracted from low-level RF signal, and infers the most likely sequence of workflow activities via a VQ-HMM model. We implement RFlow-ID on COTS RFID devices and evaluate it through a common biomedical experiment. The results validate the high recognition accuracy and robustness of our system.

Internet of Things (IoT) is rather prevalent in many manufacturing and smart city applications, while localization is a premise for many other processes, varying from ordering objects in manufacturing lines to locating books on bookshelves. Radio Frequency Identification (RFID) based localization is of great interest in many IoT applications. Synthetic aperture RFID, due to its anti-noise capability and robustness against multipath distortion, is becoming a rising star in the field of localization. Existing systems achieve finer lateral resolution, whereas their radial accuracy is limited by the narrow bandwidth of RFID signal. In this paper, we present a novel synthetic aperture RFID localization method which combines RFID phase based ranging with synthetic aperture technology, to achieve a higher radial accuracy than the existing systems. With only one reader antenna and one 1-dimensional (1D) trajectory, a synthetic array is constructed to get an accurate localization result both in lateral and radial direction. Its core idea is to make full use of the coherence of all multi-frequency phase data and merge them into a unique ranging based likelihood function. To improve the accuracy, the relative phase is leveraged to eliminate phase offsets caused by the reader antenna, and the phase deviation from the angle-of-arrival response is calibrated by pre-processing. Then a weighted enhancement is fully exploited to further improve the localization performance. We evaluate its performance with commercial-off-the-shelf (COTS) RFID devices and the results show that it achieves median accuracy of 3cm in both lateral and radial direction. This novel promising method is suitable for locating tags placed densely in many IoT applications, such as test tubes in hospitals.