Sound waves from speech can potentially induce vibrations, proportional to the speech signal, on nearby objects. Each of these objects introduces the risk for a malicious attacker to exploit the induced vibrations to eavesdrop on the speech. Such an eavesdropping attack is critical when we consider the potential for induced vibrations in standard magnetic hard disk drives (HDDs). As an instance of this threat, prior research has demonstrated that speech in certain scenarios can induce vibrations on the read/write head of an HDD in order to eavesdrop on the speech (Kwong et al.; Oakland'19). In this paper, we revisit this line of research and aim to provide a closer investigation into whether HDDs can in fact be used as a source for eavesdropping on speech vibrations. As a foundation for our study, we utilize an effective, and robust methodology using laser vibrometry to measure the subtle speech vibrations induced on the read/write head. The prior study tested only a single HDD and only machine-rendered speech in a single setting with very loud speech. Our work broadens the scope of this research in many significant ways. First, we test multiple popular HDDs of different models and sizes to evaluate the generalizability of the overall threat. Second, we evaluate the threat from live human speech spoken near an HDD, expanding the scope of the attack to include most real-world speech settings involving normal human conversations. Third, we define machine-rendered speech scenarios to explore different propagation media and degrees of speech loudness. Our findings are two-fold. First, we observed that live human speech traveling through the air is not generally strong enough to impact HDDs such that intelligible speech information is leaked. Second, most tested HDDs did not seem capable of eavesdropping on machine-rendered speech unless the speech is loud enough, or the HDD shares a surface or is in direct contact with the speaker device. This implies HDDs cannot eavesdrop live human speech.

Recent advances in speaker verification and speech processing technology have seen voice authentication being adopted on a wide scale in commercial applications like online banking and customer care support and on devices such as smartphones and IoT voice assistant systems. However, it has been shown that the current voice authentication systems can be ineffective against voice synthesis attacks that mimic a user's voice to high precision. In this work, we suggest a paradigm shift from the traditional voice authentication systems operating in the audio domain but susceptible to speech synthesis attacks (in the same audio domain). We leverage a motion sensor's capability to pick up phonatory vibrations, that can help to uniquely identify a user via voice signatures in the vibration domain. The user's speech is played/echoed back by a device's speaker for a short duration (hence our method is termed EchoVib) and the resulting non-linear phonatory vibrations are picked up by the motion sensor for speaker recognition. The uniqueness of the device's speaker and its accelerometer results in a device-specific fingerprint in response to the echoed speech. The use of the vibration domain and its non-linear relationship with audio allows EchoVib to resist the state-of-the-art voice synthesis attacks, shown to be successful in the audio domain.

In this paper, we build a speech privacy attack that exploits speech reverberations from a smartphone's inbuilt loudspeaker captured via a zero-permission motion sensor (accelerometer). We design our attack Spearphone, and demonstrate that speech reverberations from inbuilt loudspeakers, at an appropriate loudness, can impact the accelerometer, leaking sensitive information about the speech. In particular, we show that by exploiting the affected accelerometer readings and carefully selecting feature sets along with off-the-shelf machine learning techniques, Spearphone can perform gender classification (accuracy over 90%) and speaker identification (accuracy over 80%) for the audio/video playback on the smartphone for our recorded dataset. We use lightweight classifiers and an off-the-shelf machine learning tool so that the attacking effort is minimized, making our attack practical. Our results with testing the attack on a voice call and voice assistant response were also encouraging, showcasing the impact of the proposed attack. In addition, we perform speech recognition and speech reconstruction to extract more information about the eavesdropped speech to an extent. Our work brings to light a fundamental design vulnerability in many currently-deployed smartphones, which may put people's speech privacy at risk while using the smartphone in the loudspeaker mode during phone calls, media playback or voice assistant interactions.

In this paper, we propose Voicefox1, a defense against the threat of automated voice synthesis attacks in machine-based and human-based speaker verification applications. Voicefox is based on a hitherto undiscovered potential of speech-to-text transcription, already built into these applications. Voicefox relies on the premise that while the synthesized samples might be falsely accepted by the speaker verification systems and human listeners, they cannot be transcribed as accurately as a natural human voice by transcribers. Voicefox is not a speaker verification system, but rather an independent module that can be integrated with any speaker verification system to enhance its security against voice synthesis attacks.

Voice access technologies are widely adopted in mobile devices and voice assistant systems as a convenient way of user interaction. Recent studies have demonstrated a potentially serious vulnerability of the existing voice interfaces on these systems to "hidden voice commands". This attack uses synthetically rendered adversarial sounds embedded within a voice command to trick the speech recognition process into executing malicious commands, without being noticed by legitimate users.

This paper explores the effectiveness of common sound masking solutions deployed for preserving speech privacy in workplace environment such as hospitals, financial institutions, lawyers offices, nursing homes and government buildings. With the increased awareness about personal privacy among the general population, we set out to examine the effectiveness of current speech privacy preserving tools. We seek to determine if the general approach used by the current masking mechanisms is adequate to provide the level of privacy desired from these solutions. In addition, we also seek to investigate preservation of speech privacy in the face of ubiquitous and less conspicuous devices like smartphones that possess the capability of sound recording with inbuilt noise cancellation technology. Our approach in this paper is to expose the vulnerability in sound masking technology in scenarios that require preserving privacy in personal spaces. We use human listeners to attack speech privacy under sound masking where we aim to identify spoken words eavesdropped under different scenarios. We also test currently available speech recognition tools to assess their performance at decoding speech in noisy environment. Our results indicate that pink noise, the commonly used technology to provide speech privacy for use in personal space, is ineffective against a dedicated eavesdropping adversary that uses commonplace devices such as smartphones to record the speech and noise reduction tools to counteract sound masking.

Internet of Things (IoT) is embodied by smart network-enabled devices that utilize computing power, networking, and miniaturization to enable richer and improved user experience. Due to their interconnectedness, ubiquitous nature and low computational power, trustworthy and secure communication between IoT devices has become a security concern. To authenticate the devices, “pairing” may be secured by the use of an auxiliary channel such as audio, visual and vibrations for sharing the key or keying material between the IoT devices. In this paper, we evaluate the security of vibration channel, susceptible to an acoustic eavesdropper, that can capture audio leakage from the vibrations of the transmitting IoT device. We propose a noisy vibration scheme for cloaking vibration sounds during pairing against such attacks. The scheme only requires a speaker for emitting the masking sound during key transmission. We evaluate the scheme in proximity, co-located and remote settings with an eavesdropping attacker. We also study motion sensor exploits against this scheme and compliment it with additional measures to mask vibration effects on motion sensors. Our scheme is user transparent and requires only a speaker (that may already be present on the device), so it can be readily implemented in the IoT setting, smart wearables, and other commodity gadgets.

Keyboard acoustic side channel attacks to date have been mostly studied in the context of an adversary eavesdropping on keystrokes by placing a listening device near the intended victim creating a local eavesdropping scenario. However, being in close physical proximity of the victim significantly limits the applicability of the attack.

According to recent research, motion sensors available on current smartphone platforms may be sensitive to speech signals. From a security and privacy perspective, this raises a serious concern regarding sensitive speech reconstruction, and speaker or gender identification by a malicious application having unrestricted access to motion sensor readings, without using the microphone. In this paper, we revisit this important line of research and closely inspect the effect of speech on smartphone motion sensors, in particular, gyroscope and accelerometer. First, we revisit the previously studied scenario (Michalevsky et al.; USENIX Security 2014), where the smartphone shares a common surface with a loudspeaker (with subwoofer) generating speech signals. We observe some effect on the motion sensor signals, which may indeed allow speaker and gender recognition to an extent. However, we also argue that the recorded effect on the sensor readings is possibly from conductive vibrations through the shared surface instead of direct acoustic vibrations due to speech as perceived in previous work. Second, we further extend the previous work by analyzing the effect of speech produced by (1) other less powerful speakers like the in-built laptop and smartphone speakers, and (2) live humans. Our experiments show that in-built laptop speakers were only able to affect the accelerometer when the laptop and the motion sensor shared a surface. Smartphone speakers were not found to be powerful enough to invoke a response in the motion sensors through aerial vibrations. We also report that in the presence of live human speech, we did not notice any effect on the motion sensor readings. Our results have two-fold implications. First, human-rendered speech seems potentially incapacitated to trigger smartphone motion sensors within the limited sampling rates imposed by the smartphone operating systems. Second, it seems that even machine-rendered speech may not be powerful enough to affect smartphone motion sensors through the aerial medium, although it may induce vibrations through a conductive surface that these sensors, especially accelerometer, could pick up if a relatively powerful speaker is used. Overall, our results suggest that smartphone motion sensors may pose a threat to speech privacy only in some limited scenarios.

An interesting approach to pairing devices involves the use of a vibrational channel, over which the keying material (e.g., a short PIN) is sent. This approach is efficient (only a unidirectional transfer of PIN is needed) and simple (the sending device requires a vibration motor and receiving device requires an accelerometer). However, it has been shown to be susceptible to acoustic emanations usually produced by the vibration motor. Recent research introduced a mechanism to defeat these attacks by attempting to mask the acoustic leakage with deliberate acoustic noises. In this paper, we pursue a systematic investigation of the security of such a "noisy vibrational pairing" mechanism in a strong yet realistic adversarial model where the eavesdropper is co-located with the victim device(s).

Deauthentication is an important component of any computing system that promises to offer legitimate access to restricted services residing on the system. As computing devices are ubiquitous, it has underscored the need to design zero-effort deauthentication systems from a usability perspective. While the design of such deauthentication systems is geared towards making them more usable, often the security implication of these deigns overlook the physical security of the system resulting in various side channel vulnerabilities in the system. This issue highlights the need to design a defense mechanism that is capable of minimizing the threat posed by such side channel attacks while having minimal impact on the design of the system.

Pairing between wireless devices may be secured by the use of an auxiliary channel such as audio, visuals or vibrations. A simple approach to pairing involves one of the devices initiating the transmission of a key, or keying material like a short password, over the auxiliary channel to the other device. A successful pairing is achieved when the receiving device is able to decode the key without any errors while the attacker is unable to eavesdrop the key.