Toxic speech detection has become a crucial challenge in maintaining safe online communication environments. However, existing approaches to toxic speech detection often neglect the contribution of paralinguistic cues, such as emotion, intonation, and speech rate, which are key to detecting speech toxicity. Moreover, current toxic speech datasets are predominantly text-based, limiting the development of models that can capture paralinguistic cues. To address these challenges, we present ToxiAlert-Bench, a large-scale audio dataset comprising over 30,000 audio clips annotated with seven major toxic categories and twenty fine-grained toxic labels. Uniquely, our dataset annotates toxicity sources—distinguishing between textual content and paralinguistic origins—for comprehensive toxic speech analysis. Furthermore, we propose a dual-head neural network with a multi-stage training strategy tailored for toxic speech detection. This architecture features two task-specific classification headers: one for identifying the source of sensitivity (textual or paralinguistic), and the other for categorizing the specific toxic type. The training process involves independent head training followed by joint fine-tuning to reduce task interference. To mitigate data class imbalance, we incorporate class-balanced sampling and weighted loss functions. Our experimental results show that leveraging paralinguistic features significantly improves detection performance. Our method consistently outperforms existing baselines across multiple evaluation metrics, with a 21.1% relative improvement in Macro-F1 score and a 13.0% relative gain in accuracy over the strongest baseline, highlighting its enhanced effectiveness and practical applicability.

The integration of microphones into a broad array of devices, from consumer electronics to industrial sensors, introduces vulnerabilities to out-of-band injection attacks, including ultrasound, laser, electromagnetic, and magnetic field attacks. These attacks enable adversaries to inject inaudible or imperceptible commands, compromising systems without direct physical access. This paper presents a robust, passive detection framework designed to address the full spectrum of out-of-band attacks on microphone-equipped devices. Unlike prior approaches, our system leverages advanced speech disentanglement to separate semantic and acoustic features from recorded audio, enabling a refined analysis of injection artifacts within each feature domain. By quantifying entropy-based chaos within the disen tangled representations, we detect subtle spectral and structural irregularities indicative of injected signals. The system further incorporates a preliminary stage to identify carrier traces where applicable, expediting detection in cases such as ultrasound and laser attacks. Extensive evaluations across various device types, including smartphones, tablets, and microphones, demonstrate the system's high accuracy and stability, achieving an AUC of 98% under diverse conditions and attack configurations.

Recently, reconstruction-based methods have gained attention for AIGC image detection. These methods leverage pre-trained diffusion models to reconstruct inputs and measure residuals for distinguishing real from fake images. Their key advantage lies in reducing reliance on dataset-specific artifacts and improving generalization under distribution shifts. However, they are limited by significant inefficiency due to multi-step inversion and reconstruction, and their reliance on diffusion backbones further limits generalization to other generative paradigms such as GANs. In this paper, we propose a novel fake image detection framework, called R$^{2}$BD, built upon two key designs: (1) G-LDM, a unified reconstruction model that simulates the generation behaviors of VAEs, GANs, and diffusion models, thereby broadening the detection scope beyond prior diffusion-only approaches; and (2) a residual bias calculation module that distinguishes real and fake images in a single inference step, which is a significant efficiency improvement over existing methods that typically require 20$+$steps. Extensive experiments on the benchmark from 10 public datasets demonstrate that R$^{2}$BD is over 22× faster than existing reconstruction-based methods while achieving superior detection accuracy. In cross-dataset evaluations, it outperforms state-of-the-art methods by an average of 13.87%, showing strong efficiency and generalization across diverse generative methods. The code and dataset used for evaluation are available athttps://github.com/QingyuLiu/RRBD.

Recently, Large Language Models (LLMs) have been in tegrated into many real-world applications like Code Copilot. These applications have significantly expanded the attack surface of LLMs, ex posing them to complex real-world jailbreak threats. Despite the promising advances in actively finding jailbreak vulnerabilities of LLMs (i.e., red teaming) in general contexts, identifying these threats in complex domain-specific contexts (e.g, mathematical LLMs) remains underexplored. In this paper, we study whether the context these real-world LLM applications work in, including different system prompts, tools, and scenarios of tasks, give rise to context-specific jailbreak threats. Particularly, we adapt general jailbreak prompts to the context of the target application via LLM rewriting to generate context-specific attacks. By measuring the differences in jailbreak responses between general attacks and context-specific attacks, we reveal that customized domain specific LLMs are more vulnerable in their specific context. Motivated by this observation, we propose a context-aware red teaming approach, RedAgent, to generate context-specific jailbreak attacks towards customized LLM applications. Through effectively retrieving and updating structured knowledge in an agent system, RedAgent efficiently perceive and utilize contextual information to adapt the jailbreak prompts to the target contexts. Extensive experiments show that our system can jailbreak most black-box LLMs within just five queries, improving the efficiency of existing red teaming methods by two times. Further, RedAgent can effectively jailbreak customized LLM applications. By generating context-specific jailbreak prompts towards 60 trending applications on the marketplace of OpenAI, we discover 600 vulnerabilities of these real world applications with only two queries per vulnerability.

The Polymorphic Network (PINet) is a future network architecture that can dynamically load multiple network modalities (NM), solving the rigidity and single IP bearer problem. However, due to the different design concepts and protocol stacks of each NM, it is challenging to provide compatible data access control for different NMs. In this paper, we propose a unified attribute-based data access control framework within PINet. This framework ensures consistent access control across NMs by managing user attribute identifiers and seamlessly integrating access policies. We design Span-hiding Cipher-policy Attribute-based Encryption (SHCP-ABE). The design of span attribute cuckoo filter and two-round encryption in SHCP-ABE implements access control that supports comparing attribute values and policy hiding. We further modified the linear secret sharing scheme in SHCP-ABE to achieve isolation of access policy information among NMs. The proposed SHCP-ABE scheme is unified, fine-grained, privacy-preserving, and can effectively adapt to heterogeneous future network architectures such as PINet. Security analysis shows that SHCP-ABE is secure against the adaptive chosen plaintext attack under the generic group model. Experimental results show that SHCP-ABE reduces the time overhead of key generation and decryption and the key storage overhead by about 50% over the existing comparable CP-ABE scheme, providing excellent efficiency.

Deep neural network (DNN) services have been widely deployed for efficient and accurate inferences in many different domains. In practice, a client may send its private data (e.g., images, text messages and videos) to the service to get the inferences with the proprietary DNN models. Significant privacy and security concerns would emerge in such scenarios. Cryptographic inference systems have been proposed to address such privacy and security concerns. However, existing systems are tailored for DNNs on image inferences, but not directly applicable to video inference tasks that operate on the spatio-temporal (3D) features. To address such critical deficiencies, we design and implement the first cryptographic inference system, Crypto3D, which privately and efficiently infers videos with compressed 3D DNNs while ensuring rigorous privacy guarantees. We also update most cryptographic inference systems (designed for images) to support video understanding on 3D features with non-trivial extensions, treating them as baselines. We evaluate Crypt3D and benchmark with baselines utilizing the widely adopted C3D and I3D models on the UCF-101 and HMDB-51 datasets. Our results demonstrate that Crypto3D significantly outperforms existing systems on execution time: 554.68× vs. CryptoDL (3D), 189.21× vs. HEANN (3D), 182.61× vs. MP-SPDZ (3D), 133.56× vs. E2DM (3D), 11.09× vs. Intel SGX (3D), 8.90× vs. Gazelle (3D), 3.71 × vs. Delphi (3D), 12.97 × vs. CryptFlow2 (3D), 1.49× vs. Cheetah (3D); accuracy: 82.4% vs. <80% for all of them. Code is available at https://github.com/datasec-lab/crypto3D

Diffusion models have achieved remarkable success in novel view synthesis, but their reliance on large, diverse, and often untraceable Web datasets has raised pressing concerns about image copyright protection. Current methods fall short in reliably identifying unauthorized image use, as they struggle to generalize across varied generation tasks and fail when the training dataset includes images from multiple sources with few identifiable (watermarked or poisoned) samples. In this paper, we present novel evidence that diffusion-generated images faithfully preserve the statistical properties of their training data, particularly reflected in their spectral features. Leveraging this insight, we introduce CoprGuard, a robust frequency domain watermarking framework to safeguard against unauthorized image usage in diffusion model training and fine-tuning. CoprGuard demonstrates remarkable effectiveness against a wide range of models, from naive diffusion models to sophisticated text-to-image models, and is robust even when watermarked images comprise a mere 1% of the training dataset. This robust and versatile approach empowers content owners to protect their intellectual property in the era of AI-driven image generation.

This work asks: with abundant, unlabeled real faces, how to learn a robust and transferable facial representation that boosts various face security tasks with respect to generalization performance? We make the first attempt and propose a self-supervised pretraining framework to learn fundamental representations of real face images, FSFM, that leverages the synergy between masked image modeling (MIM) and instance discrimination (ID). We explore various facial masking strategies for MIM and present a simple yet powerful CRFR-P masking, which explicitly forces the model to capture meaningful intra-region Consistency and challenging inter-region Coherency. Furthermore, we devise an ID network that naturally couples with MIM to establish underlying local-to-global Correspondence through tailored self-distillation. These three learning objectives, namely 3C, empower encoding both local features and global semantics of real faces. After pretraining, a vanilla ViT serves as a universal vision Foundation Model for downstream Face Security tasks: cross-dataset deepfake detection, cross-domain face anti-spoofing, and unseen diffusion facial forgery detection. Extensive experiments on 10 public datasets demonstrate that our model transfers better than supervised pretraining, visual and facial self-supervised learning arts, and even outperforms task-specialized SOTA methods.

Robustness is essential for deep neural networks, especially in security-sensitive applications. To this end, randomized smoothing provides theoretical guarantees for certifying robustness against adversarial perturbations. Recently, diffusion models have been successfully employed for randomized smoothing to purify noise-perturbed samples before making predictions with a standard classifier. While these methods excel at small perturbation radii, they struggle with larger perturbations and incur a significant computational overhead during inference compared to classical methods. To address this, we reformulate the generative modeling task along the diffusion trajectories in pixel space as a discriminative task in the latent space. Specifically, we use instance discrimination to achieve consistent representations along the trajectories by aligning temporally adjacent points. After fine-tuning based on the learned representations, our model enables implicit denoising-then-classification via a single prediction, substantially reducing inference costs. We conduct extensive experiments on various datasets and achieve state-of-the-art performance with minimal computation budget during inference. For example, our method outperforms the certified accuracy of diffusion-based methods on ImageNet across all perturbation radii by 5.3\% on average, with up to 11.6\% at larger radii, while reducing inference costs by 85x on average.

Advances in image generation enable hyper-realistic synthetic faces but also pose risks, thus making synthetic face detection crucial. Previous research focuses on the general differences between generated images and real images, often overlooking the discrepancies among various generative techniques. In this paper, we explore the intrinsic relationship between synthetic images and their corresponding generation technologies. We find that specific images exhibit significant reconstruction discrepancies across different generative methods and that matching generation techniques provide more accurate reconstructions. Based on this insight, we propose a Multi-Reconstruction-based detector. By reversing and reconstructing images using multiple generative models, we analyze the reconstruction differences among real, GAN-generated, and DM-generated images to facilitate effective differentiation. Additionally, we introduce the Asian Synthetic Face Dataset (ASFD), containing synthetic Asian faces generated with various GANs and DMs. This dataset complements existing synthetic face datasets. Experimental results demonstrate that our detector achieves exceptional performance, with strong generalization and robustness.

Voice communication is convenient while also poses risks of privacy leakage, due to potential interception or eavesdropping during voice transmission. Current protections of voice privacy are almost entirely controlled by communication service providers (CSPs), which operate as a black-box to users thus hard to fully trust. To take back the control of user privacy, in this paper, we introduce SecHeadset, an end-to-end solution for secure voice communication based on voice obfuscation, which is plug-and-play and compatible with various CSPs. Our solution involves two parts. First, we design a voice-like noise masking scheme for voice obfuscation. The noise, mimicking voice characteristics, could effectively obscure users' voices while demonstrating resilience against noise reduction methods. Second, we develop a protocol that enables efficient channel state estimation and secure information exchange between two communication entities. Based on this information, we propose a lightweight algorithm for voice retrieval during communication. We develop a prototype of SecHeadset and evaluate its performance with 8 widely-used applications, including Telegram and Skype. It reduces the voice recognition accuracy of various adversaries to below 15% while maintaining communication quality. We also integrate SecHeadset with off-the-shelf portable devices and verify its real-world effectiveness.

Invisible Image Watermarking is crucial for ensuring content provenance and accountability in generative AI. While Gen-AI providers are increasingly integrating invisible watermarking systems, the robustness of these schemes against forgery attacks remains poorly characterized. This is critical, as forging traceable watermarks onto illicit content leads to false attribution, potentially harming the reputation and legal standing of Gen-AI service providers who are not responsible for the content. In this work, we propose WMCopier, an effective watermark forgery attack that operates without requiring any prior knowledge of or access to the target watermarking algorithm. Our approach first models the target watermark distribution using an unconditional diffusion model, and then seamlessly embeds the target watermark into a non-watermarked image via a shallow inversion process. We also incorporate an iterative optimization procedure that refines the reconstructed image to further trade off the fidelity and forgery efficiency. Experimental results demonstrate that WMCopier effectively deceives both open-source and closed-source watermark systems (e.g., Amazon’s system), achieving a significantly higher success rate than existing methods. Additionally, we evaluate the robustness of forged samples and discuss the potential defense against our attack. Code is available at: https://github.com/holdrain/WMCopier.

The advent of voice cloning has fundamentally threatened the role of voice as a unique biometric. Many criminal incidents have already been reported to demonstrate its significant risks of identity forgery. Previous works explored the risks of voice cloning in constrained settings, which require victim speakers to either be already seen in the training data of voice cloning models, or leak dozens of minutes of their speech samples to adversaries. However, with the rapid progress of voice cloning in AIGC (Artificial Intelligence Generated Content) era, these requirements have largely been released, leaving the exact risks of state-of-the-art (SOTA) voice cloning techniques shrouded in a dense fog. To uncover it, this paper conducts a large-scale study in real-world scenarios to assess the risks of advanced voice cloning techniques. This study involves 5 SOTA voice cloning techniques (open-source and commercial), across 8 SOTA voice authentication systems (open-source and real-world) and 30 human listeners, using voice data of over 7,000 speakers (public and custom). By experimental and theoretical analysis, this study reveals that 1) state-of-the-art voice cloning techniques pose severe threats in spoofing voice authentication systems and human listeners; 2) demographic factors such as age and gender of victim speakers have a subtle impact on voice cloning attacks; 3) human listeners' subjective opinions and background about voice cloning play an important role in their susceptibility to attacks; 4) advanced detection methods still fail to identify voice cloning samples as expected.

The widespread smart devices raise people’s concerns of being eavesdropped on. To enhance voice privacy, recent studies exploit the nonlinearity in microphone to jam audio recorders with inaudible ultrasound. However, existing solutions solely rely on energetic masking. Their simple-form noise leads to several problems, such as high energy requirements and being easily removed by speech enhancement techniques. Besides, most of these solutions do not support authorized recording, which restricts their usage scenarios. In this paper, we design an efficient yet robust system that can jam microphones while preserving authorized recording. Specifically, we propose a novel phoneme-based noise with the idea of informational masking, which can distract both machines and humans and is resistant to denoising techniques. Besides, we optimize the noise transmission strategy for broader coverage and implement a hardware prototype of our system. Experimental results show that our system can reduce the recognition accuracy of recordings to below 50% under all tested speech recognition systems, which is much better than existing solutions.

Pseudonyms used in Online social networks (OSNs) post a great challenge to fighting against cyber crimes. To build a strong case, law enforcement may want to link multiple user accounts with pseudonyms to a physical suspect. Images based camera fingerprinting has been used for account linking when a suspect takes pictures and videos with his camera and posts them online. However, image post-processing software may introduce noise into images. This noise is hard to eliminate by conventional strategies, is partly resident in the estimated photo-response non-uniformity (PRNU) fingerprints, and interferes with matching fingerprints. We define this noise as software noise, which pollutes PRNU fingerprints and affects accounts linking in online social networks. In this article, we propose new approaches for camera fingerprint matching given software noise. The key idea is to determine the PRNU hardware noise correlation component with our new test statistic–fingerprinttosoftware noise ratio (FITS). We performed extensive experiments and 10,000+ images taken by 90+ smartphones were used to validate our robust camera fingerprint matching system. The experiment results show FITS outperforms the state-of-the-art approaches for polluted fingerprints. This is the first work studying camera fingerprint matching with the presence of software noise.

Physical adversarial examples (AEs) have become an increasing threat to deploying deep neural network (DNN) models in the real world. Popular approaches adopt sticking-based or projecting-based strategies that stick the printed adversarial patches to objects or directly project the AE onto objects. Although effective, these methods require access to target objects and generate visible artifacts, which reduces the attack's stealthiness. In this article, we propose MagShadow, a new attack vector that leverages imperceptible electromagnetic (EM) signals to realize physical AEs. MagShadow utilizes the CCD camera sensor's susceptibility to EM injection attacks and induces fine-grained adversarial perturbations on the camera's captured image by injecting carefully-crafted signals with a low-cost portable device. As MagShadow directly manipulates the image sensor's output with EM signals, the attack requires no access to the target object and can keep stealthy. We study the feasibility of MagShadow in two typical DNN application scenarios (image classification and object detection) and design a framework to implement four different types of attacks, i.e., untargeted, targeted, hiding, and appearing attacks. Extensive real-world experiments on five different cameras are conducted, which demonstrate MagShadow's effectiveness against different popular DNN models (Inception v3, ResNet101, YOLO v3/v4).

Photoplethysmography (PPG) has been extensively employed in commercial and medical products to assess human cardiac activities. However, despite PPG’s active role in improving people’s daily lives, research on the vulnerabilities of PPG systems is still in its infancy. This paper investigates the feasibility of deceiving PPG sensors in the physical domain. We propose FakePPG, which utilizes a low-cost Liquid Crystal Modulator (LCM) device to mimic the PPG signals of a legitimate user, thus deceiving both the PPG-based health assessment and potential authentication applications. To implement FakePPG in practical scenarios, we build the attack prototype using commercial off-the-shelf electronic components and further design an automated optimization and attack framework. By leveraging the modified multi-Gaussian model for parameterization, the evolutionary strategy for optimization, and the reference heart rate model for heartbeat variability alignment, FakePPG can achieve efficient, flexible, and automated PPG forgery against arbitrary users and heart states. Extensive experimental results show that FakePPG can achieve a success rate of 96.7% for Atrial Fibrillation (AFib) spoofing and 91.2% for identity spoofing, respectively, revealing a realistic threat to PPG systems.

Security issues in the Internet of Things (IoT) are inevitable. Uncertain threats, such as known vulnerabilities and backdoors exist within IoT, and traditional passive network security technologies are ineffective against uncertain threats. To address the above issues, we propose an active defense adjudication method based on adaptive anomaly sensing for mimic IoT. The method constructs a mimic IoT active defense architecture, improving system security and reliability despite prevailing security threats. In addition, an intelligent anomaly sensing algorithm is integrated into the adjudication module of the mimic IoT active defense architecture to support arbitration. An adaptive anomaly sensing model based on multi-feature selection is used to determine the anomaly score of the IoT device outputs, and this model fully considers the reliability of the adjudication data and improves the accuracy of the adjudication. Finally, we conduct a comparative analysis of the proposed adjudication algorithm against three others via a mimic power communication IoT system as an application scenario. The experimental results show that our algorithm can improve security and reduce the failure rate of the mimic IoT system.

Deepfake technology has given rise to a spectrum of novel and compelling applications. Unfortunately, the widespread proliferation of high-fidelity fake videos has led to pervasive confusion and deception, shattering our faith that seeing is believing. One aspect that has been overlooked so far is that current deepfake detection approaches may easily fall into the trap of overfitting, focusing only on forgery clues within one or a few local regions. Moreover, existing works heavily rely on neural networks to extract forgery features, lacking theoretical constraints guaranteeing that sufficient forgery clues are extracted and superfluous features are eliminated. These deficiencies culminate in unsatisfactory accuracy and limited generalizability in real-life scenarios. In this paper, we try to tackle these challenges through three designs: (1) We present a novel framework to capture broader forgery clues by extracting multiple non-overlapping local representations and fusing them into a global semantic-rich feature. (2) Based on the information bottleneck theory, we derive Local Information Loss to guarantee the orthogonality of local representations while preserving comprehensive task-relevant information. (3) Further, to fuse the local representations and remove task-irrelevant information, we arrive at a Global Information Loss through the theoretical analysis of mutual information. Empirically, our method achieves state-of-the-art performance on five benchmark datasets. Our code is available at https://github.com/QingyuLiu/Exposing-the-Deception, hoping to inspire researchers.

Black-box adversarial attacks have demonstrated strong potential to compromise machine learning models by iteratively querying the target model or leveraging transferability from a local surrogate model.Recently, such attacks can be effectively mitigated by state-of-the-art (SOTA) defenses, e.g., detection via the pattern of sequential queries, or injecting noise into the model. To our best knowledge, we take the first step to study a new paradigm of black-box attacks with provable guarantees -- certifiable black-box attacks that can guarantee the attack success probability (ASP) of adversarial examples before querying over the target model. This new black-box attack unveils significant vulnerabilities of machine learning models, compared to traditional empirical black-box attacks, e.g., breaking strong SOTA defenses with provable confidence, constructing a space of (infinite) adversarial examples with high ASP, and the ASP of the generated adversarial examples is theoretically guaranteed without verification/queries over the target model. Specifically, we establish a novel theoretical foundation for ensuring the ASP of the black-box attack with randomized adversarial examples (AEs). Then, we propose several novel techniques to craft the randomized AEs while reducing the perturbation size for better imperceptibility. Finally, we have comprehensively evaluated the certifiable black-box attacks on the CIFAR10/100, ImageNet, and LibriSpeech datasets, while benchmarking with 16 SOTA black-box attacks, against various SOTA defenses in the domains of computer vision and speech recognition. Both theoretical and experimental results have validated the significance of the proposed attack.

Despite the promising advances in deepfake detection on current datasets, detecting visual deepfakes in real-world scenarios (e.g., deepfake videos and live streaming on YouTube) remains a challenge due to the inherent quality degradation such as unpredictable compression employed by social media platforms. Such degradation perturbs discernible forgery clues and diminishes the effectiveness of deepfake detection methods, raising a critical safety concern to the misuse of forgery faces in real-world scenarios. In this paper, we aim to understand the impacts of real-world degradation on the robustness of deepfake detection. Particularly, we investigate the risk of degraded deepfakes towards their detection on two real-world scenarios (i.e., deepfake videos and deepfake live streaming on social media platforms). By measuring the effects of real-world degradations on the performance and representation capabilities of detection models, we reveal that real-world deepfakes can be simulated via common degradation operations (e.g., JPEG compression) as they are perceptually similar to deepfake detectors. By analyzing the training dynamics under different sequences of training samples, we observe that the training order of deepfakes progressing from non-degraded (easy) to heavily degraded (hard) enhances the adaptability of detection models to various degradation in real-world scenarios. Drawing from these observations, we present a novel deepfake detection method ProFake to enhance the robustness of deepfake detection against real-world quality degradations. ProFake enables quality-adaptive learning via progressively degrade, detect and assign weights for the training samples driven by the feedback of model performance and image quality, which ensures that our model gradually focuses on more challenging samples to achieve quality-adaptive deepfake detection. Extensive experiments show that compared with existing methods, ProFake improves deepfake detection accuracy by an average of over 10 % in real-world scenarios and by an average of over 30 % in heavily degraded scenarios, while maintaining comparable performance in detecting high-quality deepfakes.

Recent years have witnessed a dramatic growing trend of Virtual YouTubers (VTubers) as a new business on social media, such as YouTube, Twitch, and TikTok. However, a significant challenge arises when VTuber voice actors face health issues or retire, jeopardizing the continuity of their avatar’s recognizable voices. A potential solution reminiscent of Conan’s Bow Tie voice changer in the popular animation Case Closed (i.e., Detective Conan) has inspired our work. To make this a reality, we introduce VTuberBowTie, a user-friendly streaming voice conversion system for real-time VTuber livestreaming. We propose an innovative streaming voice conversion approach that tackles the challenges of limited context modeling and bidirectional context dependence inherent to conventional real-time voice conversion. Rather than individually processing the voice stream in data chunks, our approach adopts a fully sequential structure that leverages contextual information preceding the input chunk, thereby expanding the perceptual range and enabling seamless concatenation. Moreover, we developed a ready-to-use interaction interface for VTuberBowTie and deployed it on various computing platforms. The experimental results show that VTuberBowTie can achieve high-quality voice conversion in a streaming manner with a latency of 179.1ms on CPU and 70.8ms on GPU while providing users a friendly interactive experience.

Extensive research has revealed that adversarial examples (AE) pose a significant threat to voice-controllable smart devices. Recent studies have proposed black-box adversarial attacks that require only the final transcription from an automatic speech recognition (ASR) system. However, these attacks typically involve many queries to the ASR, resulting in substantial costs. Moreover, AE-based adversarial audio samples are susceptible to ASR updates. In this paper, we identify the root cause of these limitations, namely the inability to construct AE attack samples directly around the decision boundary of deep learning (DL) models. Building on this observation, we propose ALIF, the first black-box adversarial linguistic feature-based attack pipeline. We leverage the reciprocal process of text-to-speech (TTS) and ASR models to generate perturbations in the linguistic embedding space where the decision boundary resides. Based on the ALIF pipeline, we present the ALIF-OTL and ALIF-OTA schemes for launching attacks in both the digital domain and the physical playback environment on four commercial ASRs and voice assistants. Extensive evaluations demonstrate that ALIF-OTL and -OTA significantly improve query efficiency by 97.7% and 73.3%, respectively, while achieving competitive performance compared to existing methods. Notably, ALIF-OTL can generate an attack sample with only one query. Furthermore, our test-of-time experiment validates the robustness of our approach against ASR updates.

The language models, especially the basic text classification models, have been shown to be susceptible to textual adversarial attacks such as synonym substitution and word insertion attacks. To defend against such attacks, a growing body of research has been devoted to improving the model’s robustness. However, providing provable robustness guarantees instead of empirical robustness is still widely unexplored. In this paper, we propose Text-CRS, a generalized certified robustness framework for natural language processing (NLP) based on randomized smoothing. To our best knowledge, existing certified schemes for NLP can only certify the robustness against ℓ0perturbations in synonym substitution attacks. Representing each word-level adversarial operation (i.e., synonym substitution, word reordering, insertion, and deletion) as a combination of permutation and embedding transformation, we propose novel smoothing theorems to derive robustness bounds in both permutation and embedding space against such adversarial operations. To further improve certified accuracy and radius, we consider the numerical relationships between discrete words and select proper noise distributions for the randomized smoothing. Finally, we conduct substantial experiments on multiple language models and datasets. Text-CRS can address all four different word-level adversarial operations and achieve a significant accuracy improvement. We also provide the first benchmark on certified accuracy and radius of four word-level operations, besides outperforming the state-of-the-art certification against synonym substitution attacks.1

In recent years, the field of audio deepfake detection has witnessed significant advancements. Nonetheless, the majority of solutions have concentrated on high-quality audio, largely overlooking the challenge of low-quality compressed audio in real-world scenarios. Low-quality compressed audio typically suffers from a loss of high-frequency details and time-domain information, which significantly undermines the performance of advanced deepfake detection systems when confronted with such data. In this paper, we introduce a deepfake detection model that employs knowledge distillation across the frequency and time domains. Our approach aims to train a teacher model with high-quality data and a student model with low-quality compressed data. Subsequently, we implement frequency-domain and time-domain distillation to facilitate the student model's learning of high-frequency information and time-domain details from the teacher model. Experimental evaluations on the ASVspoof 2019 LA and ASVspoof 2021 DF datasets illustrate the effectiveness of our methodology. On the ASVspoof 2021 DF dataset, which consists of low-quality compressed audio, we achieved an Equal Error Rate (EER) of 2.82%. To our knowledge, this performance is the best among all deepfake voice detection systems tested on the ASVspoof 2021 DF dataset. Additionally, our method proves to be versatile, showing notable performance on high-quality data with an EER of 0.30% on the ASVspoof 2019 LA dataset, closely approaching state-of-the-art results.

Ubiquitous microphones on smart devices considerably raise users’ concerns about speech privacy. Since the microphones are primarily controlled by hardware/software developers, profit-driven organizations can easily collect and analyze individuals’ daily conversations on a large scale with deep learning models, and users have no means to stop such privacy-violating behavior. In this article, we propose UniAP to empower users with the capability of protecting their speech privacy from the large-scale analysis without affecting their routine voice activities. Based on our observation of the recognition model, we utilize adversarial learning to generate quasi-imperceptible perturbations to disturb speech signals captured by nearby microphones, thus obfuscating the recognition results of recordings into meaningless contents. As validated in experiments, our perturbations can protect user privacy regardless of what users speak and when they speak. The jamming performance stability is further improved by training optimization. Additionally, the perturbations are robust against noise removal techniques. Extensive evaluations show that our perturbations achieve successful jamming rates of more than 87% in the digital domain and at least 90% and 70% for common and challenging settings, respectively, in the real-life chatting scenario. Moreover, our perturbations, solely trained on DeepSpeech, exhibit good transferability over other models based on similar architecture.

Online voice communications are widely used nowadays. To protect speech from leakage, people tend to initiate the talk in sound-isolated environments. In this paper, we reveal a novel attack that recovers high-quality speech from outside soundproof zones. The rationale of the attack is to leverage sound-sensitive characteristics of piezoelectric materials, i.e., a piezo film that can change the phase of reflected mmWaves when placed in a sound field. If the attacker transmits mmWaves and analyzes reflected signals from the piezo film, the speech information can be compromised. More importantly, the piezo film is paper-like and works without a power supply. We propose a new speech recovery methodology to transform sound waves into wireless signals and build an end-to-end eavesdropping system working as a through-wall “microphone” to recover high-quality speech stealthily. To combat signal attenuation and improve speech quality, we develop a speech-enhancement scheme based on generative adversarial networks and propose to use multi-antenna information for intelligible speech reconstruction. We conduct extensive experiments to evaluate the system. The results indicate that the system achieves over 98% accuracy for digit recognition and works well over 5m away through the wall. We also test the system under complex scenarios and give countermeasures.

Device fingerprints can satisfy the high-security requirement of modern mobile applications (e.g., mobile payments) by guaranteeing the operation is performed on a trusted device. However, existing works on device fingerprints are weak to leakage, which leads to an irreversible failure of the device fingerprint authentication system after suffering from fingerprint theft attacks. The vulnerability drives us to propose a reconfigurable device fingerprint, i.e.,MotoPrint, that can recover the system after suffering from such attacks.MotoPrintstems from the motor vibration that can represent in both signals of the accelerometer and the gyroscope (i.e., they are homologous motion signals). Therefore, we designed a two-path feature extracting network and a sensor-independent training strategy to eliminate sensor noise that can decline authentication performance. In addition,MotoPrinthas a complete reconfiguration mechanism to cope with fingerprint leakage, which brings the damaged authentication system back to health. The evaluation of 80 stand-alone vibration motors and 20 in-built ones shows thatMotoPrintcan achieve high authentication accuracy of 98.5%. Meanwhile, we also demonstrate the reconfiguredMotoPrint, which can also effectively indicate the device's uniqueness with over 98% accuracy, is independent ofMotoPrints under other stimulating codes.

Human motion copy is an intriguing yet challenging task in artificial intelligence and computer vision, which strives to generate a fake video of a target person performing the motion of a source person. The problem is inherently challenging due to the subtle human-body texture details to be generated and the temporal consistency to be considered. Existing approaches typically adopt a conventional GAN with an L1 or L2 loss to produce the target fake video, which intrinsically necessitates a large number of training samples that are challenging to acquire. Meanwhile, current methods still have difficulties in attaining realistic image details and temporal consistency, which unfortunately can be easily perceived by human observers. Motivated by this, we try to tackle the issues from three aspects: (1) We constrain pose-to-appearance generation with a perceptual loss and a theoretically motivated Gromov-Wasserstein loss to bridge the gap between pose and appearance. (2) We present an episodic memory module in the pose-to-appearance generation to propel continuous learning that helps the model learn from its past poor generations. We also utilize geometrical cues of the face to optimize facial details and refine each key body part with a dedicated local GAN. (3) We advocate generating the foreground in a sequence-to-sequence manner rather than a single-frame manner, explicitly enforcing temporal inconsistency. Empirical results on five datasets,iPER, ComplexMotion, SoloDance, Fish, and Mouse datasets, demonstrate that our method is capable of generating realistic target videos while precisely copying motion from a source video. Our method significantly outperforms state-of-the-art approaches and gains 7.2% and 12.4% improvements in PSNR and FID respectively.

Swift advancements in large language model (LLM) technologies lead to widespread research and applications, particularly in integrating LLMs with auxiliary tools, known as tool-using LLM agents. However, amid user interactions, the transmission of private information to both LLMs and tools poses considerable privacy risks to users. In this paper, we delve into current privacy-preserving solutions for LLMs and outline three pivotal challenges for tool-using LLM agents: generalization to both open-source and closed-source LLMs and tools, compliance with privacy requirements, and applicability to unrestricted tasks. To tackle these challenges, we present PrivacyAsst, the first privacy-preserving framework tailored for tool-using LLM agents, encompassing two solutions for different application scenarios. First, we incorporate a homomorphic encryption scheme to ensure computational security guarantees for users as a safeguard against both open-source and closed-source LLMs and tools. Moreover, we propose a shuffling-based solution to broaden the framework's applicability to unrestricted tasks. This solution employs an attribute-based forgery generative model and an attribute shuffling mechanism to craft privacy-preserving requests, effectively concealing individual inputs. Additionally, we introduce an innovative privacy concept,$t$-closeness in image data, for privacy compliance within this solution. Finally, we implement PrivacyAsst, accompanied by two case studies, demonstrating its effectiveness in advancing privacy-preserving artificial intelligence.

Inaudible command injection transmits inaudible ultrasounds to inject adversarial speech commands into a voice assistant, therefore manipulating voice control systems (e.g., a garage door or a security camera) for illegitimate purposes. Although the attack is inaudible, we find it does leave visible “footprints”. Such attack “footprints” are the side product due to the interaction between the attack signal (i.e., input) and the acoustic components (i.e., transfer function), so they reflect the hardware characteristics of the sound capture system, including the microphone diaphragm, the low-pass filter, and the analog-to-digital converter. Moreover, unlike the non-linearity distortion that is erasable with signal-shaping techniques, the “footprints” are indelible because they are unrelated to the content of injected commands. We discover two types of indelible “footprints” embedded in the recording spectrogram, namely abnormal interfering noise and abnormal demodulation. A software-based detection method and a portable detector, DolphinTag, are further designed to identify these “footprints”. The software-based method achieves a detection accuracy of 99.8% on the phone models exhibiting abnormal interfering noise, and our DolphinTag achieves 100% detection accuracy which detects the ultrasound attack by actively facilitating the abnormal demodulation.

As one of the most representative applications built on deep learning, audio systems, including keyword spotting, automatic speech recognition, and speaker identification, have recently been demonstrated to be vulnerable to adversarial examples, which have already raised general concerns in both academia and industry. Existing attacks follow the same adversarial example generation paradigm from computer vision, i.e., overlaying the optimized additive perturbations on original voices. However, due to the additive perturbations’ nature on human audibility, balancing the stealthiness and attack capability remains a challenging problem. In this paper, we rethink the stealthiness of audio adversarial examples and turn to introduce another kind of audio distortion, i.e., reverberation, as a new perturbation format for stealthy adversarial example generation. Such convolutional adversarial perturbations are crafted as real-world impulse responses and behave as a natural reverberation for deceiving humans. Based on this idea, we propose AdvReverb to construct, optimize, and deliver phoneme-level convolutional adversarial perturbations on both speech and music carriers with a well-designed objective. Experimental results demonstrate that AdvReverb could realize high attack success rates over 95% on three audio-domain tasks while achieving superior perceptual quality and keeping stealthy from human perception in over-the-air and over-the-line delivery scenarios.

Despite their prevalence and indispensability in the perception modules of autonomous vehicles, cameras have shown susceptibility to numerous attacks. Among them, the phantom spoofing attack is of significant concern. In such attacks, malefactors employ electronic display devices like projectors and display monitors to generate deceptive objects, thereby duping the object detectors in autonomous vehicles. However, existing detection methodologies are narrowly focused on a single device category, ignoring the multitude of devices that could be leveraged for attacks. Furthermore, the artificial modality-based solution presently in use lacks efficacious fusion mechanisms. In response to these limitations, we propose PhaDe, a practical deep learning-based system adept at detecting phantom spoofing attacks from a variety of and even unfamiliar attack devices. Our approach introduces two image processing techniques to construct artificial modalities and further advances a multi-head self-attention MSA-based fusion module for more versatile integration of disparate modalities. To boost the generalization capacity of our system against novel, unseen attacks, we incorporate two representation-level losses to align feature distributions from various domains. Evaluations conducted on our own dataset, encompassing fake objects from several device types, attest to the efficacy of our system. Our results indicate an accuracy of 98.80% on familiar domains and a detection success rate of 94.03% on unfamiliar domains. Additionally, PhaDe demonstrates a swift response time, fulfilling the practicality requisites.

Federated Learning (FL) is a promising distributed learning approach that enables multiple clients to collaboratively train a shared global model. However, recent studies show that FL is vulnerable to various poisoning attacks, which can degrade the performance of global models or introduce backdoors into them. In this paper, we first conduct a comprehensive study on prior FL attacks and detection methods. The results show that all existing detection methods are only effective against limited and specific attacks. Most detection methods suffer from high false positives, which lead to significant performance degradation, especially in not independent and identically distributed (non-IID) settings. To address these issues, we propose FLTracer, the first FL attack provenance framework to accurately detect various attacks and trace the attack time, objective, type, and poisoned location of updates. Different from existing methodologies that rely solely on cross-client anomaly detection, we propose a Kalman filter-based cross-round detection to identify adversaries by seeking the behavior changes before and after the attack. Thus, this makes it resilient to data heterogeneity and is effective even in non-IID settings. To further improve the accuracy of our detection method, we employ four novel features and capture their anomalies with the joint decisions. Extensive evaluations show that FLTracer achieves an average true positive rate of over 96.88% at an average false positive rate of less than 2.67%, significantly outperforming SOTA detection methods (https://github.com/Eyr3/FLTracer).

Recent years have witnessed enormous research efforts on WiFi sensing to enable intelligent services of Internet of Things. However, due to the omni-directional broadcasting manner of WiFi signals, the activity semantic underlying the signals can be leaked to adversaries for surveillance, as demonstrated by our previous work. In this paper, we further extend the attack capability ofActListenerto impersonation attack, which could eavesdrop on users’ behavioral uniqueness imperceptibly using a WiFi infrastructure in any location of user sensing area. In particular,ActListener detects each human activityand converts the eavesdropped signals to that by legitimate devices based on our proposed signal propagation models. To extract noise-resilient individual behavioral uniqueness from converted CSI of WiFi signals, we further add user identification models into the substitute model set for training the signal pattern calibration generative model. Experimental results demonstrate thatActListenercould achieve over 80% accuracy in activity semantics retrieval and impersonation by using the converted signals.

Physically unclonable hardware fingerprints can be used for device authentication. The photo-response non-uniformity (PRNU) is the most reliable hardware fingerprint of digital cameras and can be conveniently extracted from images. However, we find image post-processing software may introduce extra noise into images. Part of this noise remains in the extracted PRNU fingerprints and is hard to be eliminated by traditional approaches, such as denoising filters. We define this noise as software noise, which pollutes PRNU fingerprints and interferes with authenticating a camera armed device. In this paper, we propose novel approaches for fingerprint matching, a critical step in device authentication, in the presence of software noise. We calculate the cross correlation between PRNU fingerprints of different cameras using a test statistic such as the Peak to Correlation Energy (PCE) so as to estimate software noise correlation. During fingerprint matching, we derive the ratio of the test statistic on two PRNU fingerprints of interest over the estimated software noise correlation. We denote this ratio as the fingerprint to software noise ratio (FITS), which allows us to detect the PRNU hardware noise correlation component in the test statistic for fingerprint matching. Extensive experiments over 10,000 images taken by more than 90 smartphones are conducted to validate our approaches, which outperform the state-of-the-art approaches significantly for polluted fingerprints. We are the first to study fingerprint matching with the existence of software noise.

This paper proposes a novel Deconvolution-enhanced data Augmentation method, DeAug, for ultrasonic-based speaker verification anti-spoofing systems to detect the liveness of voice sources in physical access, which aims to improve the performance of liveness detection on unseen devices where no data is collected yet. Specifically, DeAug first employs a wiener deconvolution pre-processing on available collected data to generate enhanced clean signal samples. Then, the generated samples are convolved with different device impulse responses, to enable the signal with the unseen devices' channel characteristics. Experiments on cross-domain datasets show that our proposed augmentation method can improve the performance of ultrasonic-based anti-spoofing systems by 97.8% relatively, and a further improvement of up to 43.4% can be obtained after applying domain adversarial training on multi-device augmented data.

We introduce DualMind, a generalist agent designed to tackle various decision-making tasks that addresses challenges posed by current methods, such as overfitting behaviors and dependence on task-specific fine-tuning. DualMind uses a novel "Dual-phase" training strategy that emulates how humans learn to act in the world. The model first learns fundamental common knowledge through a self-supervised objective tailored for control tasks and then learns how to make decisions based on different contexts through imitating behaviors conditioned on given prompts. DualMind can handle tasks across domains, scenes, and embodiments using just a single set of model weights and can execute zero-shot prompting without requiring task-specific finetuning. We evaluate DualMind on MetaWorld [40] and Habitat [31] through extensive experiments and demonstrate its superior generalizability compared to previous techniques, outperforming other generalist agents by over 50% and 70% on Habitat and MetaWorld, respectively. On the 45 tasks in MetaWorld, DualMind achieves over 30 tasks at a 90% success rate. Our source code is available at https://github.com/yunyikristy/DualMind.

The malicious use and widespread dissemination of deepfake pose a significant crisis of trust. Current deepfake detection models can generally recognize forgery images by training on a large dataset. However, the accuracy of detection models degrades significantly on images generated by new deepfake methods due to the difference in data distribution. To tackle this issue, we present a novel incremental learning framework that improves the generalization of deepfake detection models by continual learning from a small number of new samples. To cope with different data distributions, we propose to learn a domain-invariant representation based on supervised contrastive learning, preventing overfit to the insufficient new data. To mitigate catastrophic forgetting, we regularize our model in both feature-level and label-level based on a multi-perspective knowledge distillation approach. Finally, we propose to select both central and hard representative samples to update the replay set, which is beneficial for both domain-invariant representation learning and rehearsal-based knowledge preserving. We conduct extensive experiments on four benchmark datasets, obtaining the new state-of-the-art average forgetting rate of 7.01 and average accuracy of 85.49 on FF++, DFDC-P, DFD, and CDF2. Our code is released at \textcolorblue https://github.com/DeepFakeIL/DFIL.

Deepfake has taken the world by storm, triggering a trust crisis. Current deepfake detection methods are typically inadequate in generalizability, with a tendency to overfit to image contents such as the background, which are frequently occurring but relatively unimportant in the training dataset. Furthermore, current methods heavily rely on a few dominant forgery regions and may ignore other equally important regions, leading to inadequate uncovering of forgery cues.

With the broad integration of deep learning in Speaker Recognition (SR) systems, adversarial example attacks have been a significant threat raising user security concerns. Nevertheless, recent studies demonstrate that using input transformations (e.g., re-quantization, resampling, bandpass filtering) as a low-cost prefilter can efficiently mitigate such adversarial example attacks. These prefilters constrain the injection space of adversarial perturbations in both time and frequency domains, leading to either degraded attack performance or amplified perturbation noise. This paper proposes a new adversarial example attack, BypTalker, which could bypass these prefilter-enabled SR systems while remaining imperceptible to human listeners. BypTalker employs ensemble learning with diverse substitute pre-filters in the training phase to enhance the adversarial example’s adaptiveness to different prefilters. Furthermore, it incorporates an Acoustic Masker to cloak adversarial perturbations based on psychoacoustics effectively. This masker is well selected from a proposed metric M-Sup for minimizing the perturbation’s auditory to human perception. Experimental results show that BypTalker can achieve an Attack Success Rate of 99.1% and a Perceptual Evaluation of Speech Quality of 4.32, respectively.

Fingerprint recognition has been a vital security guard for various applications whose vulnerability has been explored by different works. However, previous works on spoofing fingerprint recognition rely on prior knowledge (e.g., photos and minutiae) of the target fingerprint, which fails to implement in practical scenarios. In this paper, we design a fingerprint spoofing attack, namely FingerFaker, to explore the vulnerability of fingerprint recognition, which can spoof automated fingerprint recognition systems (AFRSs) without prior knowledge of target fingerprints. Specifically, we propose a novel concept of "pseudo-minutiae-set" as an effective optimization object and design a two-stage scheme to optimize "pseudo-minutiaeset" leveraging a two-factor evolutionary strategy. In addition, we use a GAN-based training strategy with a minutiae loss function to pre-train a fingerprint generator to map a "pseudo-minutiae-set" into a fingerprint. We use 6342 fingerprint images to verify the performance of FingerFaker on spoofing the open-source AFRS, which shows a high attack success rate (ASR) of 97.78%. Meanwhile, we conduct a realistic case study on commercial off-the-shelf (COTS) AFRS, where FingerFaker also shows 94.22% ASR. Finally, we explore the impact of different conditions to guide the attack and propose countermeasures to mitigate the harm.

An audio system containing loudspeakers and microphones is the fundamental hardware for voice-enabled devices, enabling voice interaction with mobile applications and smart homes. This paper presents MagBackdoor, the first magnetic field attack that injects malicious commands via a loudspeaker-based backdoor of the audio system, compromising the linked voice interaction system. MagBackdoor focuses on the magnetic threat on loudspeakers and manipulates their sound production stealthily. Consequently, the microphone will inevitably pick up malicious sound generated by the attacked speaker, due to the closely packed arrangement of internal audio systems. To prove the feasibility of MagBackdoor, we conduct comprehensive simulations and experiments. This study further models the mechanism by which an external magnetic field excites the sound production of loudspeakers, giving theoretical guidance to MagBackdoor. Aiming at stealthy magnetic attacks in real-world scenarios, we self-design a prototype that can emit magnetic fields modulated by voice commands. We implement MagBackdoor and evaluate it across a wide range of smart devices involving 16 smartphones, four laptops, two tablets, and three smart speakers, achieving an average 95% injection success rate with high-quality injected acoustic signals.

The proliferation of deepfake content has motivated a surge of detection studies. However, existing detection methods in the audio area exclusively work in English, and there is a lack of data resources in other languages. Cross-lingual deepfake detection, a critical but rarely explored area, urges more study. This paper conducts the first comprehensive study on the cross-lingual perspective of deepfake detection. We observe that English data enriched in deepfake algorithms can teach a detector the knowledge of various spoofing artifacts, contributing to performing detection across language domains. Based on the observation, we first construct a first-of-its-kind cross-lingual evaluation dataset including heterogeneous spoofed speech uttered in the two most widely spoken languages, then explored domain adaptation (DA) techniques to transfer the artifacts detection capability and propose effective and practical DA strategies fitting the cross-lingual scenario. Our adversarial-based DA paradigm teaches the model to learn real/fake knowledge while losing language dependency. Extensive experiments over 137-hour audio clips validate the adapted models can detect fake audio generated by unseen algorithms in the new domain.

Eavesdropping via inertial measurement units (IMUs) has brought growing concerns over smartphone users’ privacy. In such attacks, adversaries utilize IMUs, including accelerometers and gyroscopes, which require zero permissions for access to acquire speeches. A common countermeasure is to limit sampling rates (within 200 Hz) to reduce overlap of vocal fundamental bands (85$\sim$255 Hz) and inertial measurements (0$\sim$100 Hz). Nevertheless, we observe that IMUs sampling below 200 Hz still record adequate speech-related information because of aliasing distortions. Accordingly, we propose a practical side-channel attack, namelyInertiEAR, to break the defense of sampling rate restriction on the zero-permission eavesdropping. It leverages accelerometers and gyroscopes jointly to eavesdrop on both top and bottom speakers in smartphones. We exploit coherence between responses of the built-in accelerometer and gyroscope using a mathematical model. The coherence allows precise segmentation without manual assistance. We also mitigate the impact of hardware diversity and achieve better device-independent performance than existing approaches that have to massively increase training data from different smartphones for a scalable network model. These two advantages re-enable zero-permission attacks but also extend the attacking surface and endangering degree to off-the-shelf smartphones.InertiEARachieves the recognition accuracy of 78.8% with the cross-device accuracy of up to 60.9% among 12 smartphones.

Recently, deepfakes have raised severe concerns about the authenticity of online media. Prior works for deepfake detection have made many efforts to capture the intra-modal artifacts. However, deepfake videos in real-world scenarios often consist of a combination of audio and visual. In this paper, we propose an Audio-Visual Joint Learning for Detecting Deepfake (AVoiD-DF), which exploits audio-visual inconsistency for multi-modal forgery detection. Specifically, AVoiD-DF begins by embedding temporal-spatial information in Temporal-Spatial Encoder. A Multi-Modal Joint-Decoder is then designed to fuse multi-modal features and jointly learn inherent relationships. Afterward, a Cross-Modal Classifier is devised to detect manipulation with inter-modal and intra-modal disharmony. Since existing datasets for deepfake detection mainly focus on one modality and only cover a few forgery methods, we build a novel benchmark DefakeAVMiT for multi-modal deepfake detection. DefakeAVMiT contains sufficient visuals with corresponding audios, where any one of the modalities may be maliciously modified by multiple deepfake methods. The experimental results on DefakeAVMiT, FakeAVCeleb, and DFDC demonstrate that the AVoiD-DF outperforms many state-of-the-arts in deepfake detection. Our proposed method also yields superior generalization on various forgery techniques.

Deep neural network (DNN) services have been widely deployed in many different domains. For instance, a client may send its private input data (e.g., images, texts and videos) to the cloud for accurate inferences with pre-trained DNN models. However, significant privacy concerns would emerge in such applications due to the potential data or model sharing. Secure inferences with cryptographic techniques have been proposed to address such issues, and the system can perform secure two-party inferences between each client and cloud. However, most of existing cryptographic systems only focus on DNNs for extracting 2D features for image inferences, which have major limitations on latency and scalability for extracting spatio-temporal (3D) features from videos for accurate inferences. To address such critical deficiencies, we design and implement the first cryptographic inference system, Crypto3D, which privately infers videos on 3D features with rigorous privacy guarantees. We evaluate Crypto3D and benchmark with the state-of-the-art systems on privately inferring videos in the UCF-101 and HMDB-51 datasets with C3D and I3D models. Our results demonstrate that Crypto3D significantly outperforms existing systems (substantially extended to inferences with 3D features): execution time: 186.89x vs. CryptoDL (3D), 63.75x vs. HEANN (3D), 61.52x vs. MP-SPDZ (3D), 45x vs. E2DM (3D), 3.74x vs. Intel SGX (3D), and 3x vs. Gazelle (3D); accuracy: 82.3% vs. below 70% for all of them.

Recent years have witnessed enormous research efforts on WiFi sensing to enable intelligent services of Internet of Things. However, due to the omni-directional broadcasting manner of WiFi signals, the activity semantic underlying the signals is leaked to adversaries for surveillance in all probability. To reveal the threat, this paper demonstrates ActListener, which could eavesdrop on user activities imperceptibly using a WiFi infrastructure in any location of user sensing area. The proposed attack requires no direct physical access to the victim user’s devices and prior knowledge of activity recognition model details and device locations. In particular, ActListener first detects the signal segment induced by each human activity, and estimates the locations of legitimate devices and the victim users relative to the adversary’s device for further signal modeling. Then, ActListener models propagating WiFi signals to construct the relationship between physical locations and received signals, and converts the eavesdropped signals to that by legitimate devices based on the models. Furthermore, a neural network-based generative model is designed to calibrate the converted signals for resisting noises in over-the-air WiFi signals. Experiments show ActListener achieves 88.4% average α-similarity on recovering originally signals from eavesdropped ones, and over 90% accuracy in activity recognition.

Voice has become a fundamental method for human-computer interactions and person identification these days. Benefit from the rapid development of deep learning, speaker recognition exploiting voice biometrics has achieved great success in various applications. However, the shadow of adversarial example attacks on deep neural network-based speaker recognition recently raised extensive public concerns and enormous research interests. Although existing studies propose to generate adversarial examples by iterative optimization to deceive speaker recognition, these methods require multiple iterations to construct specific perturbations for a single voice, which is input-specific, time-consuming, and non-transferable, hindering the deployment and application for non-professional adversaries. In this paper, we propose PhoneyTalker, an out-of-the-box toolkit for any adversary to generate universal and transferable adversarial examples with low complexity, releasing the requirement for professional background and specialized equipment. PhoneyTalker decomposes an arbitrary voice into phone combinations and generates phone-level perturbations using a generative model, which are reusable for voices from different persons with various texts. Experiments on mainstream speaker recognition systems with large-scale corpus show that PhoneyTalker outperforms state-of-the-art methods with overall attack success rates of 99.9% and 84.0% under white-box and black-box settings respectively.

IMU-based eavesdropping has brought growing concerns over smartphone users’ privacy. In such attacks, adversaries utilize IMUs that require zero permissions for access to acquire speeches. A common countermeasure is to limit sampling rates (within 200 Hz) to reduce overlap of vocal fundamental bands (85-255 Hz) and inertial measurements (0-100 Hz). Nevertheless, we experimentally observe that IMUs sampling below 200 Hz still record adequate speech-related information because of aliasing distortions. Accordingly, we propose a practical side-channel attack, InertiEAR, to break the defense of sampling rate restriction on the zero-permission eavesdropping. It leverages IMUs to eavesdrop on both top and bottom speakers in smartphones. In the InertiEAR design, we exploit coherence between responses of the built-in accelerometer and gyroscope and their hardware diversity using a mathematical model. The coherence allows precise segmentation without manual assistance. We also mitigate the impact of hardware diversity and achieve better device-independent performance than existing approaches that have to massively increase training data from different smartphones for a scalable network model. These two advantages re-enable zero-permission attacks but also extend the attacking surface and endangering degree to off-the-shelf smartphones. InertiEAR achieves a recognition accuracy of 78.8% with a cross-device accuracy of up to 49.8% among 12 smartphones.

More and more people turn to online voice communication with loudspeaker-equipped devices due to its convenience. To prevent speech leakage, soundproof rooms are often adopted. This paper presents mmPhone, a novel acoustic eavesdropping system that recovers loudspeaker speech protected by soundproof environments. The key idea is that properties of piezoelectric films in mmWave band can change with sound pressure due to the piezoelectric effect. If the property changes are acquired by an adversary (i.e., characterizing the piezoelectric effect with mmWaves), speech leakage can happen. More importantly, the piezoelectric film can work without a power supply. Base on this, we proposed a methodology using mmWaves to sense the film and decoding the speech from mmWaves, which turns the film into a passive "microphone". To recover intelligible speech, we further develop an enhancement scheme based on a denoising neural network, multi-channel augmentation, and speech synthesis, to compensate for the propagation and penetration loss of mmWaves. We perform extensive experiments to evaluate mmPhone and conduct digit recognition with over 93% accuracy. The results indicate mmPhone can recover high-quality and intelligible speech from a distance over 5m and is resilient to incident angles of sound waves (within 55 degrees) and different types of loudspeakers.

Faced with the threat of identity leakage during voice data publishing, users are engaged in a privacy-utility dilemma while enjoying convenient voice services. Existing studies employ direct modification or text-based re-synthesis to de-identify users' voices, but resulting in inconsistent audibility for human participants and not adaptive to informed attacks. In this poster, we propose a non-intrusive and adaptive speaker de-identification scheme to balance the privacy and utility of voice services. We generate adversarial examples to conceal user identity from exposure by Automatic Speaker Identification (ASI). By learning a compact distribution with a conditional variational auto-encoder, our system enables on-demand target sampling and diverse identity transformation. We also introduce the acoustic masking effect to construct inaudible perturbations, thus preserving the speech content and perceptual quality. Experiments on 50 speakers show our system could achieve 98.2% successful de-identification on 4 mainstream ASIs with an objective perceptual quality of 4.38 and a subjective mean opinion score of 4.56.

The integration of deep learning on Speaker Recognition (SR) advances its development and wide deployment, but also introduces the emerging threat of adversarial examples. However, only a few existing studies investigate its practical threat in physical domain, which either evaluate its feasibility only by directly replaying generated adversarial examples, or explore the partial channel interference for robustness improvement. In this paper, we propose a physical adversarial example attack, PhyTalker, which could generate and inject perturbations on voices in a live-streaming manner on attacking various SR models in different physical channels. Compared with the typical adversarial example for digital attacks, PhyTalker generates a subphoneme-level perturbation dictionary to decouple the perturbation optimization and injection. Moreover, we introduce the channel augmentation to compensate both device and environmental distortions, as well as model ensemble to improve the perturbation transferability. Finally, PhyTalker recognizes and localizes the latest recorded phoneme to determine the corresponding perturbations for real-time broadcasting. Extensive experiments are conducted with a large-scale corpus in real physical scenarios, and results show that PhyTalker achieves an overall Attack Success Rate (ASR) of 85.5% in attacking mainstream SR systems and Mel Cepstral Distortion (MCD) of 2.45dB in human audibility.

Facial recognition has become the surge on mobile authentication scenarios and makes up a huge market share for various apps, such as MasterCard, Google Wallet, and AliPay. However, existing solutions suffer from various impersonation attacks, including photo-spoofing attack, video-replay attack, and 3D facial mask attack. State-of-the-art countermeasures either require additional user intervention or introduce specialized high-end sensors. Even introducing these extra efforts, these approaches still hardly defend the latest 3D facial mask attacks, which gradually become accessible due to the prevalence of low-cost 3D printing. In this paper, we propose an anti-spoofing facial recognition system, PassFace, which verifies the smartphone for authentication as the second factor merely using raw facial videos without any user intervention, to defeat impersonation attacks. In particular, when receiving a user’s selfie video, PassFace identifies the user’s face from the video, and meanwhile extracts the highly unique and physically irreproducible camera fingerprint, i.e., Photo Response Non-Uniformity (PRNU), built in the smartphone from key frames of the video. After that, the system compares the Peak to Correlation Energy (PCE) calculated by the estimated PRNU and the reference profile with a threshold for authentication. Experiment results demonstrate PassFace can achieve satisfactory performance in authentication and attack resistance.

With the advance in automatic speech recognition, voice user interface has gained popularity recently. Since the COVID-19 pandemic, VUI is increasingly preferred in online communication due to its non-contact. Additionally, various ambient noise impedes the public applications of voice user interfaces due to the requirement of audio-only speech recognition methods for a high signal-to-noise ratio. In this paper, we present Wavoice, the first noise-resistant multi-modal speech recognition system that fuses two distinct voice sensing modalities, i.e., millimeter-wave (mmWave) signals and audio signals from a microphone, together. One key contribution is that we model the inherent correlation between mmWave and audio signals. Based on it, Wavoice facilitates the real-time noise-resistant voice activity detection and user targeting from multiple speakers. Furthermore, we elaborate on two novel modules into the neural attention mechanism for multi-modal signals fusion, and result in accurate speech recognition. Extensive experiments verify Wavoice's effectiveness under various conditions with the character recognition error rate below 1% in a range of 7 meters. Wavoice outperforms existing audio-only speech recognition methods with lower character error rate and word error rate. The evaluation in complex scenes validates the robustness of Wavoice.

Adversarial attacks, which attract explosive rese- arch attention in recent years, have achieved fantastic success in fooling neural networks, especially for image-classification tasks. While for automatic speech recognition (ASR) tasks, the state-of-the-arts mainly focus on white-box attacks where the adversary is assumed to get full access to the details inside the system, e.g., network architecture, weights, etc. However, this assumption does not hold in practice. The construction of real-world adversarial examples against ASR systems is still a very challenging problem. In this paper, we, for the first time, present a novel and effective attack on ASR systems, named Selective Gradient Estimation Attack (SGEA). Compared with prior literatures, SGEA only needs limited access to the output probabilities of neural networks, and achieves extremely high efficiency and success rates. We attacked the DeepSpeech system on Mozilla Common Voice and LibriSpeech datasets in our experiments. The results demonstrate that SGEA improves the attack success rate from 35% to 98%, while reducing the number of queries by 66%.

In this demonstration, we show that audio signals emitted by a smartphone speaker can be captured by the accelerometer on the same smartphone, and accelerometers on recently released smartphones can cover most of the fundamental frequency band of adult speech. Based on these pivotal observations, we present AccelEve, a new side channel attack that allows smartphone applications to eavesdrop on the smartphone speaker without the requirement of sensitive system permissions. Through analyzing the accelerometer measurements of a smartphone, AccelEve is able to: 1) recognize the speech information (text) carried by the acceleration signal; 2) reconstruct the audio signal played by the smartphone speaker. This demo will present experimental validations for our observations and the proposed system.

It has been discovered that every photo carries an unique hardware fingerprint of the photographing digital camera. This camera fingerprint is remarkably effective in image-to-camera matching and has been applied in a wide variety of beneficial forensic tasks such as copyright protection and integrity validation. However, the fingerprints carried by the images can also be utilized for malicious purposes. An adversary can launch identity linking attack, which re-identifies the anonymous social network accounts, through exploiting the digital cameras' fingerprints that are carried by the posted images. Moreover, the adversary can easily frame an innocent victim or bypass camera-based smartphone authentication systems by launching identity forgery attacks, i.e., fabricating unoriginal fingerprints onto images. Currently, no effective counter measures against such attacks have been proposed yet. To solve this problem, in this paper, we first evaluate the effectiveness of the attacks in the current image sharing practices. We then propose CFP, an intermediary between smartphone users and image sharing platforms that conceals the camera fingerprint of the photographing device. Instead of removing the camera fingerprint from the image of interest, our system protects user privacy through obfuscating the camera fingerprint with a specially designed random perturbation component. With such design, the proposed system is enabled to prevent malicious utilizations of camera fingerprint while preserving the beneficial applications. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of the CFP system on various social platforms. Using the CFP obfuscated images, the True Positive Rate of identity linking attacks is reduced by around 85%. For identity forgery attacks, our system enables an effective detection mechanism that could achieve 100% detection rate.

Indoor localization has been an active research area given the popularity of Location-Based Services. The CSI fingerprinting based approach is one of the most practical and effective approaches since it can provide adequate accuracy with low overhead for users. The key drawback that limits its wide application is the huge amount of human effort required to build the fingerprint map. This paper is the first to explore addressing this limitation by automating CSI map construction using an Unmanned Aerial Vehicle (UAV). Given the limited battery capacity of commodity UAVs, it is extremely important yet challenging to optimize energy efficiency for the UAV during the CSI measurement task. To address this challenge, we formulate an energy optimization problem based on a novel graph model that includes the cost of possible actions for UAVs. We then transform the formulated problem to the classic Generalized Traveling Salesman Problem (GTSP), which can be solved efficiently. We implement the system on an off-the-shelf programmable drone equipped with a CSI measurement module. We achieve great energy efficiency improvement over the conventional coverage path planning algorithm. Meanwhile, accurate indoor localization can be achieved using the CSI data collected by our UAV system.

The photo response non-uniformity of a smartphone camera is a reliable hardware fingerprint that can be used to authenticate the smartphone owner. This type of camera-based authentication is convenient and of low cost since it requires only pictures taken by a smartphone. However, as shown in this paper, existing camera-based authentication systems are either impractical or subject to fingerprint forgery attacks, in which an adversary intentionally submits forged pictures with fake fingerprints. We propose Camera in Motion (CIM), a practical and reliable camera-based smartphone authentication system. In CIM, a user is asked to move his/her smartphone along a specific route, take pictures of QR codes displayed on the verifier's interface in burst mode, and submit particular burst pictures to the verifier for authentication. We find that, because burst images are captured in rapid succession, the random noise components of a captured image can be partially preserved across multiple images that are captured in a row. The preserved noise forms a forgery-sensitive noisechain embedded in burst images. We also find that there exists various correlations between the movement of the camera and the noise components of the captured images. The noisechain and these correlations are then explored for forgery detection. We performed extensive experiments with 22 smartphones of 5 different models. Our experiment results show that CIM can achieve 100% true acceptance rate at 0% false acceptance rate in both fingerprint matching and forgery detection.

Multi-factor authentication is a well-recognized access control method that enhances the security of users' sensitive data and identities. A successful authentication attempt requires a user to correctly present two or more authentication factors such as knowledge factors, possession factors and inherence factors. For smartphone-based multi-factor authentication, a promising way to authenticate a user is to verify his possession of a legitimate smartphone, which calls for secure and usable device authentication schemes. In this article, we propose to authenticate a device through tracking the hardware fingerprint of its built-in sensor. We first review the existing hardware-rooted identification methods and discuss the merits of applying a hardware fingerprint as a smartphone's unique identity. Then, we analyze the security issues underlying these methods and identify two security requirements for the identification methods to be used in an authentication scheme: Fingerprint Leakage Resilience and Fingerprint Forgery Resilience. Finally, we look into a specific hardware fingerprint originally used for digital cameras. We analyze the feasibility of applying this fingerprint to differentiate off-the-shelf smartphones and list several challenging practical issues underlying this method.

Additive manufacturing, also known as 3D printing, has been increasingly applied to fabricate highly intellectual property (IP) sensitive products. However, the related IP protection issues in 3D printers are still largely underexplored. On the other hand, smartphones are equipped with rich onboard sensors and have been applied to pervasive mobile surveillance in many applications. These facts raise one critical question: is it possible that smartphones access the side-channel signals of 3D printer and then hack the IP information? To answer this, we perform an end-to-end study on exploring smartphone-based side-channel attacks against 3D printers. Specifically, we formulate the problem of the IP side-channel attack in 3D printing. Then, we investigate the possible acoustic and magnetic side-channel attacks using the smartphone built-in sensors. Moreover, we explore a magnetic-enhanced side-channel attack model to accurately deduce the vital directional operations of 3D printer. Experimental results show that by exploiting the side-channel signals collected by smartphones, we can successfully reconstruct the physical prints and their G-code with Mean Tendency Error of 5.87% on regular designs and 9.67% on complex designs, respectively. Our study demonstrates this new and practical smartphone-based side channel attack on compromising IP information during 3D printing.

Abstract Cloud computing offers a cheap and efficient solution for the deployment of web applications. It results in a big increase of the number of service provider. Users hold multiple identities for using services from different domains. The openness of public clouds requires the authentication system to accept user identities from various domains and to support hybrid authentication protocols. This work proposes a cross‐domain single sign‐on mechanism to address the preceding issues and makes a formal mathematical model to analyze the security issues of the proposed mechanism's authentication architecture; furthermore, an algorithm is proposed to detect the authentication architecture's weak vertex whose failure would lead to a partial failure in the architecture. The proposed mechanism allows service providers to verify user identities in a decentralized way and allows users to unify their identities from various domains in a safe way. The verification process used in this mechanism is able to support hybrid authentication protocols as well as to accelerate the verification of credentials by eliminating single point of failure and single‐point bottleneck. Copyright © 2015 John Wiley & Sons, Ltd.