This paper introduces mmWave-Whisper, a system that demonstrates the feasibility of full-corpus automated speech recognition (ASR) on phone calls eavesdropped remotely using off-the-shelf frequency modulated continuous wave (FMCW) millimeter-wave radars. Operating in the 77-81 GHz range, mmWave-Whisper captures earpiece vibrations from smart-phones, converts them into audio, and processes the audio to produce speech transcriptions automatically. Unlike previous work that focused on loudspeakers or a limited vocabulary, this is the first to perform this kind of speech recognition by handling a large vocabulary and full sentences on earpiece vibrations from smartphones. This approach expands the potential for radar-audio eavesdropping. mmWave-Whisper addresses challenges such as the lack of large-scale training datasets, low SNR, and limited frequency information in radar data through a systematic data pipeline designed to leverage synthetic training data, domain adaptation, and inference by incorporating OpenAI’s Whisper automatic speech recognition model. The system achieves a word accuracy rate of 44.74% and a character accuracy rate of 62.52% over a range of 25 cm to 125 cm. The paper highlights emerging misuse modalities of AI as the technology evolves rapidly.

This paper presents WirelessTap, a system that demonstrates the potential for automated speech recognition (ASR) on phone call audio eavesdropped remotely using commercially available frequency modulated continuous wave millimeter-wave (mmWave) radars operating in the 77-81 GHz range. WirelessTap detects minute vibrations from smartphone earpieces, converts them into audio, and processes this audio for speech transcription. This work presents the first full-sentence ASR using mmWave radars on earpiece vibrations using a 10,000-word vocabulary, achieving a 300 cm attack range across multiple smartphone models. It surpasses prior radar-based eavesdropping studies limited to loudspeakers, small vocabularies, or constrained evaluations. To address challenges like the absence of large mmWave radar-based audio datasets, low signal-to-noise ratio, and limited voice frequency ranges extractable from radar data, WirelessTap incorporates synthetic data generation, domain adaptation, and inference using OpenAI's Whisper ASR model. Our experiments systematically show how word accuracy rate gradually decreases with distance, from as high as 59.25% at 50 cm to 2% at 300 cm; additionally, we deploy this attack to a real-world setting with a user study targeting a victim holding a smartphone to their ear. This paper highlights the evolving risks of artificial intelligence and sensor systems being misused as technology advances.

This paper presents a system mmSpy that shows the feasibility of eavesdropping phone calls remotely. Towards this end, mmSpy performs sensing of earpiece vibrations using an off-the-shelf radar device that operates in the mmWave spectrum (77GHz, and 60GHz). Given that mmWave radars are becoming popular in a number of autonomous driving, remote sensing, and other IoT applications, we believe this is a critical privacy concern. In contrast to prior works that show the feasibility of detecting loudspeaker vibrations with larger amplitudes, mmSpy exploits smaller wavelengths of mmWave radar signals to detect subtle vibrations in the earpiece devices used in phonecalls. Towards designing this attack, mmSpy solves a number of challenges related to non-availability of large scale radar datasets, systematic correction of various sources of noises, as well as domain adaptation problems in harvesting training data. Extensive measurement-based validation achieves an endto-end accuracy of 83-44% in classifying digits and keywords over a range of 1-6ft, thereby compromising the privacy in applications such as exchange of credit card information. In addition, mmSpy shows the feasibility of reconstruction of the audio signals from the radar data, using which more sensitive information can be potentially leaked.

We overview the ensmallen numerical optimization library, which provides a flexible C++ framework for mathematical optimization of user-supplied objective functions. Many types of objective functions are supported, including general, differentiable, separable, constrained, and categorical. A diverse set of pre-built optimizers is provided, including Quasi-Newton optimizers and many variants of Stochastic Gradient Descent. The underlying framework facilitates the implementation of new optimizers. Optimization of an objective function typically requires supplying only one or two C++ functions. Custom behavior can be easily specified via callback functions. Empirical comparisons show that ensmallen outperforms other frameworks while providing more functionality. The library is available at https://ensmallen.org and is distributed under the permissive BSD license.

K-Nearest Neighbors (KNN) has remained one of the most popular methods for supervised machine learning tasks. However, its performance often depends on the characteristics of the dataset and on appropriate feature scaling. In this paper, we explore characteristics of a dataset that make it suitable for being used within KNN. As part of this, two new measures for dataset dispersion, called mean neighborhood target standard deviation (MNTSD), and mean neighborhood target entropy (MNTE) are formulated to determine the expeced performance while using KNN regressors and classifiers, respectively. It is empirically demonstrated that these measures of dispersion can be indicative of the performance of KNN regression and classification. This idea is further used to learn feature weights that help improve the accuracy of KNN classification and regression. For this, it is argued that the MNTSD and MNTE, when used to learn feature weights, cannot be optimized using gradient-based optimization methods and we develop optimization strategies based on metaheuristic methods, namely genetic algorithms and particle swarm optimization. The feature-weighting method is tried in both regression and classification contexts on publicly available datasets, and the performance is compared to KNN without feature weighting. The results indicate that the performance of KNN with appropriate feature weighting leads to better performance.