With the rapid developments of Internet of Thing (IoT) technologies, side-channel sound sensing has become a key research area. Unlike traditional sound-sensing methods that rely on direct acoustic signal recording, side-channel sound-sensing methods utilize various indirect emissions from electronic devices, such as mechanical vibrations, electromagnetic leakage, etc., to infer sound content, making it preferred in many application scenarios. Recently, many works related to side-channel sound sensing have been proposed and drawn much attention from not only academia but also industry. In this article, we present a comprehensive survey of various side-channel sound-sensing methods. Specifically, we first review the commonly sensed object-state characteristics in side-channel sound-sensing methods. Then, we introduce the widely exploited hardware infrastructure, and signal processing techniques, which are the basis of side-channel sound sensing. After that, we provide a categorization introduction of emerging side-channel sound-sensing methods, and review the common application scenarios. Finally, we provide a thoughtful analysis of the countermeasures, challenges, and future directions in this field. With this survey, we aim to summarize the latest research results in the field of side-channel sound sensing and provide insights and inspiration for future research works.

As one of the most essential accessories, headsets have been widely used in common online conversations. The metal coil vibration patterns of headset speakers/microphones have been proven to be highly correlated with the speaker-produced/microphone-received sound. This paper presents an online conversation eavesdropping system,RFSpy, which uses only one RFID tag attached on a headset to alternately sense metal coil vibrations of headset speaker and microphone for eavesdropping on speaker-produced and microphone-received sound. In some accessible scenarios, assuming attackers secretly attach a small, battery-free RFID tag under one ear cushion of an eavesdropped user’s headset without being noticed. Meanwhile, RFID readers are camouflaged as decorations placed in/out of rooms to transmit and receive RF signals. When the eavesdropped user talks with other users online through the headset,RFSpyfirst activates the RFID tag to capture the metal coil vibration patterns of headset speaker and microphone upon RF signals. Then,RFSpyreconstructs sound spectrograms from the RF signal-based vibration patterns for not only trained words but also untrained (i.e., out-of-vocabulary) words utilizing designed SSR network. Finally,RFSpyconverts the sound spectrograms to conversation content through sound recognition API. Extensive experiments demonstrate thatRFSpycan eavesdrop on online conversations with out-of-vocabulary words effectively.

Eavesdropping on human sound is one of the most common but harmful ways to threaten personal privacy. As one of the most essential accessories, headsets have been widely used in common online conversations, such as online calls, video meetings, etc. The metal coil vibration patterns of headset speakers/microphones have been proven to be highly correlated with the speaker-produced/microphone-received sound content. This paper presents an online conversation eavesdropping system, RFSpy, which uses only one RFID tag attached on a headset to alternately sense the metal coil vibrations of headset speaker and microphone for eavesdropping on speaker-produced and microphone-received sound. In some accessible scenarios, such as meeting rooms, offices, etc., assuming attackers secretly attach a small, battery-free RFID tag under one ear cushion of an eavesdropped user's headset without being noticed. Meanwhile, RFID readers are camouflaged as decorations placed in/out of rooms to transmit and receive RF signals. When the eavesdropped user talks with other users online by using the headset, RFSpy first activates the RFID tag attached on the headset to capture the metal coil vibration patterns of headset speaker and microphone upon RF signals. Then, RFSpy reconstructs sound spectrograms from the RF signal-based vibration patterns for not only trained words but also untrained (i.e., out-of-vocabulary) words by utilizing a designed Sound Spectrogram Reconstruction (SSR) network. Finally, RFSpy converts the sound spectrograms to conversation content through a sound recognition API. Extensive experiments in real environments demonstrate that RFSpy can eavesdrop on online conversations with out-of-vocabulary (OOV) words effectively.

Gait-based user authentication schemes have been widely explored because of their ability of non-invasive sensing and avoid replay attacks. However, existing gait-based user authentication methods are environment-dependent. In this paper, we present an environment-independent gait-based user authentication system,RFPass, which can identify different individuals leveraging RFID signals. Specifically, we find that Doppler shift of RF signals can describe environment-independent gait features for different individuals. InRFPass, when a user walks through theRFPasssystem, RF signals are first collected by a deployed RFID tag array. Then,RFPassremoves environmental interference from the collected RF signals through a proposed Multipath Direction of arrival (DoA) Signal Select (MDSS) algorithm, and further constructs the environment-independent gait profile. Afterwards, environment-independent gait features are extracted from the constructed gait profile by a proposed CNN-RNN model. Based on the extracted gait features, a hierarchical classifier is trained for user authentication and spoofer detection. Extensive experiments in different real environments demonstrate thatRFPasscan achieve environment-independent gait-based user authentication.

Gait-based user authentication schemes have been widely explored because of their ability of non-invasive sensing and avoid replay attacks. However, existing gait-based user au-thentication methods are environment-dependent. In this paper, we present an environment-independent gait-based user authen-tication system, RFPass, which can identify different individuals leveraging RFID signals. Specifically, we find that Doppler shift of RF signals can describe environment-independent gait features for different individuals. In RFPass, when a user walks through the RFPass system, RF signals are first collected by a deployed RFID tag array. Then, RFPass removes environmental interfer-ence from the collected RF signals through a proposed Multipath Direction of arrival (DoA) Signal Select (MDSS) algorithm. Next, we construct an environment-independent gait profile to describe the user's walking movements. Afterward, environment-independent gait features are extracted by a proposed CNN-RNN model. Based on the extracted gait features, a trained model is constructed for user authentication and spoofer detection. Extensive experiments in different real environments demonstrate that RFPass can achieve environment-independent gait-based user authentication.