Recent speech-based services such as voice assistants and cloud computing services have brought security concerns, since those services constantly send user's speech data to the server. Speech data contain user's sensitive biometric data and spoken words and can be misused if it is leaked into wrong hands. In this context, Signal Processing in Encrypted Domain (SPED) can be a solution by mixing up Homomorphic Encryption (HE) with signal processing. Using HE enables computing on user's encrypted data without decrypting it, thus providing security and privacy. In this paper, we present a simple, but fast homomorphic Quantized Fourier Transform (QFT) with efficient packing of speech signals. Our work is based on the Fully Homomor-phic Encryption (FHE) scheme TFHE, which was proposed by Chillotti et al. We then present a thorough noise analysis of our QFT that helps to keep a reasonable noise level. Also, considering the TFHE's bootstrapping manner, we statistically analyze the boundary of the QFT coefficients, and present a simple criterion for scaling up the coefficients. Our criteria help keep the message precision as high as possible during the TFHE bootstrapping. We use our criteria to evaluate the magnitude of QFT with low latency, but with reasonable precision. Finally, we provide a proof-of-concept implementation of our QFT. With a ring dimension of 1024 and TFHE parameters that achieve 106 bits of security, we show that the QFT can be evaluated in 35 milliseconds for a single ciphertext of length 1024. This result is 74.6 times faster than in the previous work. We also built a homomorphic end-to-end speech processing framework that processes and classifies gender (resp. vowel) of encrypted speech data from the VoxCeleb (resp. PCVC) dataset. Our implementation classifies the gender (resp. vowel) with more than 83% (resp. 79%) accuracy with a minimum of 0.05 (resp. 0.56) seconds with multithreading.

Smartphone sensors potentially threaten the privacy of individuals, placing society at risk. Previous studies have demonstrated that smartphone sensors are susceptible to privacy intrusion. Inspired by this finding, we designed a mechanism of invasion that targets the location privacy of subway passengers. Specifically, we recovered the travel trajectories of subway passengers using sensor data and matched them with railway data collected from OpenStreetMap. This study primarily exploits an accelerometer and gyroscope, which are suitable for subway tracking because they operate appropriately in underground and indoor conditions. Although these sensors are easily influenced by passenger activity, we devised a method for recovering clean trajectories of subway passengers by utilizing gravitational acceleration and event detection methods. Subsequently, we conducted several experiments to prove the threat and feasibility of our proposals, even in the presence of human-generated noise (e.g., texting, watching videos, playing games, device rotation, and changing positions) influencing the sensor data. Specifically, we applied dynamic time warping (DTW) to obtain the costs between the reference data and reconstructed trace. Finally, a cost combination mechanism aggregated the DTW costs and predicted the best matches.

Using electrical network frequency (ENF) for video forensics has been intensely studied in recent years. The ENF signal found in videos has twice the electrical frequency (100 Hz or 120 Hz), whereas frame rates of common videos are relatively low (around 30 Hz). To extract ENF signal from video, state-of-the-art works exploit the rolling shutter effect. However, this method has a constraint that the region affected by the flickering light has to be large enough to contain all the information which light leaves at the pixels. As these regions are only part of the scene in many cases, it is hard to take advantage of the rolling shutter effect. In this paper, we propose a novel method to extract ENF signals by topological approach without utilizing the rolling shutter effect. Based on the fact that the topological representation of the possible outcomes is in the form of a closed-loop, we obtain the phase angles of each frame using manifold learning. We convert the phase angles into the frequency values based on the prior knowledge about the nominal frequency of ENF and the frame rate of the video. We tested two different manifold learning algorithms (i.e., UMAP and t-SNE) and compared the result with the state-of-the-art works, and t-SNE shows the best performance achieving root-mean-square error (RMSE) of 0.00036 Hz.

Electric Network Frequency (ENF) analysis has been an intriguing tool for multimedia forensics as former studies have paved the way for estimating ENF signals from digital audio, video, or even image files. However, for ENF signals to be widely used in extensive applications, supplementary research is needed so that ENF signals can be stably extracted without restrictions. In this letter, we propose a new phase-based approach for extracting ENF signals from CMOS sensor recordings. It uses phase differences between row signals from two consecutive frames, such that problems due to missing sample points during the idle periods are circumvented. The proposed method has substantial advantages in that it is applicable without a predefined read-out time and when the length of given videos is too short. Extensive experiments conducted with numerous devices demonstrate that the proposed method can take precedence over state-of-the-art methods because it robustly produces accurate ENF estimates in terms of alias frequency on the frame-level. The coding framework used for this letter is available at:https://github.com/hyekyunghan/Phase-based-ENF-extraction-method.

The “2019 Automatic Speaker Verification Spoofing And Countermeasures Challenge” (ASVspoof) competition aimed to facilitate the design of highly accurate voice spoofing attack detection systems. the competition did not emphasize model complexity and latency requirements; such constraints are strict and integral in real-world deployment. Hence, most of the top performing solutions from the competition all used an ensemble approach, and combined multiple complex deep learning models to maximize detection accuracy - this kind of approach would sit uneasily with real-world deployment constraints. To design a lightweight system, we combined the notions of skip connection (from ResNet) and max feature map (from Light CNN), and evaluated the accuracy of the system using the ASVspoof 2019 dataset. With an optimized constant Q transform (CQT) feature, our single model achieved a replay attack detection equal error rate (EER) of 0.37% on the evaluation set, surpassing the top ensemble system from the competition that achieved an EER of 0.39%.

Electrical network frequency (ENF) signals have common patterns that can be used as signatures for identifying recorded time and location of videos and sound. To enable cost-efficient, reliable and scalable location inference, we created a reference map of ENF signals representing hundreds of locations world wide -- extracting real-world ENF signals from online multimedia streaming services (e.g., YouTube and Explore). Based on this reference map of ENF signals, we propose a novel side-channel attack that can identify the physical location of where a target video or sound was recorded or streamed from. Our attack does not require any expensive ENF signal receiver nor any software to be installed on a victim»s device -- all we need is the recorded video or sound files to perform the attack and they are collected from world wide web. The evaluation results show that our attack can infer the intra-grid location of the recorded audio files with an accuracy of $76$% when those files are $5$ minutes or longer. We also showed that our proposed attack works well even when video and audio data are processed within a certain distortion range with audio codecs used in real VoIP applications.

The frequency of power distribution networks in a power grid is called electrical network frequency (ENF). Because it provides the spatio-temporal changes of the power grid in a particular location, ENF is used in many application domains including the prediction of grid instability and blackouts, detection of system breakup, and even digital forensics. In order to build high performing applications and systems, it is necessary to capture a large-scale nationwide or worldwide ENF map. Consequently, many studies have been conducted on the distribution of specialized physical devices that capture the ENF signals. However, this approach is not practical because it requires significant effort from design to setup, moreover, it has a limitation in its efficiency to monitor and stably retain the collection equipment distributed throughout the world. Furthermore, this approach requires a significant budget.