Voice data is crucial for modern artificial intelligence (AI) systems, powering various applications from speaker recognition (SR) to AI-generated content. The reliance on massive data raises serious concerns about privacy and property rights due to potential unauthorized misuse. Unfortunately, no effective method has been proposed to protect the privacy and copyright of voice data while meeting perceptual audio quality requirements. To bridge this gap, we introduce Volto, a unified dual-function framework for generating unlearnable yet traceable voice examples. Volto jointly integrates unlearnable perturbations and learnable watermarks, exploiting weaknesses in both human auditory perception and deep neural network (DNN) representations. This synergistic design enables Volto to corrupt critical model-sensitive features, hindering unauthorized model learning, while encoding watermark signals that remain recoverable for ownership verification. The unlearnable examples can confuse DNNs while preserving the perceptual quality, reducing the accuracy of unauthorized models by more than 21.44% in black-box scenarios. Volto also introduces a verifiable watermark, providing robust evidence of data ownership. Our work effectively balances protection effectiveness and usability, offering an effective defense for voice privacy.

Deep Neural Networks are vulnerable to data poisoning attacks, which inject a backdoor by poisoning the training data set with a predefined trigger pattern. However, most existing studies design trigger patterns as exogenous features introduced to clean samples (such as a checkerboard patch), whereas the endogenous features inherited from sample origins (such as deep generative models) have not been investigated yet. In this study, we investigate the efficacy of utilizing Generative Adversarial Network fingerprints to design trigger patterns by examining three attack patterns: the all-label attack, label-specific attack, and semantic-specific attack. Specifically, we select training data that satisfies the requirements of various attack patterns, train a GAN model, and employ samples embedded with GAN fingerprints to generate poisoned data sets. Our evaluations on three data sets (CIFAR-10, GTSRB, and LSUN) demonstrate that employing GAN fingerprints as trigger patterns 1) can achieve average attack success rate results of 39.40% in all-label attack, 89.84% in label-specific attack, and 91.06% in semantic-specific attack, 2) is stealthy by reducing the probability of being exposed, and 3) can resist six existing backdoor detection techniques, three backdoor erasing techniques, and two deepfake detection techniques. Furthermore, it exhibits practical applicability in federated learning scenario.

Deep learning-based automatic speech recognition (ASR) systems are capable of transcribing input audio of arbitrary duration into character sequences, which are widely used in daily life. However, recent research has found that deep learning models are vulnerable to backdoor attacks. A malicious adversary can embed a backdoor functionality into the model during the training phase and manipulate the output of the backdoored model by adding a specific trigger to the input during the inference phase. Unfortunately, TrojanModel, the existing state-of-the-art backdoor attack against ASR systems (Zong et al. S&P’23), relies on an overly strong assumption that requires the adversary to modify the model structure beyond data poisoning, which significantly limits its practicability. In this paper, we propose CuckooAttack, a more practical backdoor attack against ASR systems that only requires poisoning a small portion of the training data. We first construct a phoneme-level auxiliary dataset to generate effective, robust, and unnoticeable triggers, while substantially lowering computational expenses. Considering the real-world ASR application scenarios, we propose an adaptive trigger injection mechanism to ensure that the backdoor can be activated on variable-duration input audio under asynchronous temporal conditions. To further enhance the efficacy of CuckooAttack, we design a character-filling strategy tailored for ASR to construct poisoned samples, which facilitates the model in establishing backdoor connections. Extensive experiments show that CuckooAttack achieves comparable performance with TrojanModel under a weaker assumption. Specifically, CuckooAttack achieves an attack success rate of about 99% in the digital domain and over 90% in the physical domain, with a poison rate of only 1%.

In recent years, extensive research has been conducted on the vulnerability of ASR systems, revealing that black-box adversarial example attacks pose significant threats to real-world ASR systems. However, most existing black-box attacks rely on queries to the target ASRs, which is impractical when queries are not permitted. In this paper, we propose ZQ-Attack, a transfer-based adversarial attack on ASR systems in the zero-query black-box setting. Through a comprehensive review and categorization of modern ASR technologies, we first meticulously select surrogate ASRs of diverse types to generate adversarial examples. Following this, ZQ-Attack initializes the adversarial perturbation with a scaled target command audio, rendering it relatively imperceptible while maintaining effectiveness. Subsequently, to achieve high transferability of adversarial perturbations, we propose a sequential ensemble optimization algorithm, which iteratively optimizes the adversarial perturbation on each surrogate model, leveraging collaborative information from other models. We conduct extensive experiments to evaluate ZQ-Attack. In the over-the-line setting, ZQ-Attack achieves a 100% success rate of attack (SRoA) with an average signal-to-noise ratio (SNR) of 21.91dB on 4 online speech recognition services, and attains an average SRoA of 100% and SNR of 19.67dB on 16 open-source ASRs. In the over-the-air setting, ZQ-Attack also achieves a 100% SRoA with an average SNR of 15.77dB on 2 commercial intelligent voice control devices.

Transfer-based adversarial attacks highlight a critical security concern in the vulnerability of deep neural networks (DNNs). By generating deceptive inputs on a surrogate model, these attacks efficiently transfer the malicious examples to target models, even those with different architectures. However, current transfer-based adversarial attacks face a significant challenge. Existing strategies, including gradient optimization, input transformation, and model ensemble methods, struggle to strike an effective balance between computational cost and transferability. To alleviate this issue, we introduce a novel method, dubbed Noise Injection Augmentation (NIA). NIA enhances the transferability of the generated adversarial examples by introducing randomness into the surrogate models. The key idea of NIA is to explore the regularization properties of noise injection. Furthermore, we achieve stronger transferability by combining NIA with the idea of model self-ensemble. Extensive experiments show that NIA significantly enhances the attack performance of various potent adversarial attacks such as MI-FGSM, MDTI-FGSM, and S2I-FGSM by 28%, 20.4%, and 18.6%. On average, combined with the state-of-the-art transfer-based attack, NIA further improves transferability to 93.8% on normally trained models and 72% on robust models.

Adversarial transferability presents an intriguing phenomenon, where adversarial examples designed for one model can effectively deceive other models. By exploiting this property, various transfer-based methods are proposed to conduct adversarial attacks without knowledge of target models, posing significant threats to practical black-box applications. However, these methods either have limited transferability or require high resource consumption. To bridge the gap, we investigate adversarial transferability from the optimization perspective and propose the ghost sample attack (GSA). GSA improves adversarial transferability by alleviating the overfitting issue of adversarial examples on the surrogate model. Based on the insight that a slight shift of the adversarial example is similar to a minor change in the decision boundary, we aggregate gradients of perturbed adversarial copies (named ghost samples) to efficiently achieve a similar effect to calculating gradients of multiple ensemble surrogate models. Extensive experiments demonstrate that GSA achieves state-of-the-art adversarial transferability with restricted resources. On average, GSA improves the attack success rate by 4.8% on normally trained models compared to state-of-the-art attacks. Additionally, GSA reduces the computational cost by 62% compared with TAIG-R. When combined with other methods, GSA further improves transferability to 96.9% on normally trained models and 82.7% on robust models.

Network pruning has been extensively studied in model compression to reduce neural networks’ memory, latency, and computation cost. However, the pruned networks still suffer from the threat posed by adversarial examples, limiting the broader application of the pruned networks in safety-critical applications. Previous studies maintain the robustness of the pruned networks by combining adversarial training and network pruning but ignore preserving the robustness at a high sparsity ratio in structured pruning. To address such a problem, we propose an effective filter importance criterion, Filter Robustness Estimation (FRE), to evaluate the importance of filters by estimating their contribution to the adversarial training loss. Empirical results show that our FRE-based Robustness-aware Filter Pruning (FRFP) outperforms the state-of-the-art methods by 12.19%∼37.01% of empirical robust accuracy on the CIFAR10 dataset with the VGG16 network at an extreme pruning ratio of 90%.

Automatic speech recognition (ASR) has been widely and commercially employed in health care, autonomous vehicles, and finance. Yet, recent studies have shown that universal adversarial perturbations (UAPs) pose a serious threat to white-box ASR systems, when the adversary has access to the target model. Until now, the impacts of such a threat on commercial systems are still open since their models are not publicly available. To understand the security weakness in the practical black-box setting, this paper introduces the firstzero-queryUAP attacks, called AdvDDoS, with black-box access to ASR systems: we do not need to pay any query expense to estimate UAPs. Specifically, we craft targeted UAPs under a popular feature extractor and a local ASR model by reversing the robust target-category features, in which adversarial perturbations containing robust features are believed to have better transferability. Compared with vanilla UAPs, our UAPs incorporated with target-category features lead to better attacks against commercial ASR systems. We validate the efficacy of our AdvDDoS by launching attacks against a range of commercial ASR systems,i.e., three API services (Alibaba, Tencent, and Baidu), and three personal assistants (Apple Siri, iFlytek, and Google). Extensive experimental results demonstrate the superiority of AdvDDoS. For example, AdvDDoS achieves 83.26% word error rate (WER) and 53.25% success rates of attacks (SRoA) for the universal attack against Tencent ASR API, which outperforms the vanilla UAPs by up to 61.56% on WER and 11.6% on SRoA. The success of our attack sheds light on zero-query UAP attacks against Commercial ASR systems.

Previous researches have demonstrated deep learning models' vulnerabilities to decision-based adversarial attacks, which craft adversarial examples based solely on information from output decisions (top-1 labels). However, existing decision-based attacks have two major limitations, i.e., expensive query cost and being easy to detect. To bridge the gap and enlarge real threats to commercial applications, we propose a novel and efficient decision-based attack against black-box models, dubbed FastDrop, which only requires a few queries and work well under strong defenses. The crux of the innovation is that, unlike existing adversarial attacks that rely on gradient estimation and additive noise, FastDrop generates adversarial examples by dropping information in the frequency domain. Extensive experiments on three datasets demonstrate that FastDrop can escape the detection of the state-of-the-art (SOTA) black-box defenses and reduce the number of queries by 13~133× under the same level of perturbations compared with the SOTA attacks. FastDrop only needs 10~20 queries to conduct an attack against various black-box models within 1s. Besides, on commercial vision APIs provided by Baidu and Tencent, FastDrop achieves an attack success rate (ASR) of 100% with 10 queries on average, which poses a real and severe threat to real-world applications.

Adversarial attacks against commercial black-box speech platforms, including cloud speech APIs and voice control devices, have received little attention until recent years. Constructing such attacks is difficult mainly due to the unique characteristics of time-domain speech signals and the much more complex architecture of acoustic systems. The current "black-box" attacks all heavily rely on the knowledge of prediction/confidence scores or other probability information to craft effective adversarial examples (AEs), which can be intuitively defended by service providers without returning these messages. In this paper, we take one more step forward and propose two novel adversarial attacks in more practical and rigorous scenarios. For commercial cloud speech APIs, we propose Occam, a decision-only black-box adversarial attack, where only final decisions are available to the adversary. In Occam, we formulate the decision-only AE generation as a discontinuous large-scale global optimization problem, and solve it by adaptively decomposing this complicated problem into a set of sub-problems and cooperatively optimizing each one. Our Occam is a one-size-fits-all approach, which achieves 100% success rates of attacks (SRoA) with an average SNR of 14.23dB, on a wide range of popular speech and speaker recognition APIs, including Google, Alibaba, Microsoft, Tencent, iFlytek, and Jingdong, outperforming the state-of-the-art black-box attacks. For commercial voice control devices, we propose NI-Occam, the first non-interactive physical adversarial attack, where the adversary does not need to query the oracle and has no access to its internal information and training data. We, for the first time, combine adversarial attacks with model inversion attacks, and thus generate the physically-effective audio AEs with high transferability without any interaction with target devices. Our experimental results show that NI-Occam can successfully fool Apple Siri, Microsoft Cortana, Google Assistant, iFlytek and Amazon Echo with an average SRoA of 52% and SNR of 9.65dB, shedding light on non-interactive physical attacks against voice control devices.

COVID-19 is a severe global epidemic in human history. Even though there are particular medications and vaccines to curb the epidemic, tracing and isolating the infection source is the best option to slow the virus spread and reduce infection and death rates. There are three disadvantages to the existing contact tracing system: 1. User data is stored in a centralized database that could be stolen and tampered with, 2. User’s confidential personal identity may be revealed to a third party or organization, 3. Existing contact tracing systems [1][2] only focus on information sharing from one dimension, such as location-based tracing, which significantly limits the effectiveness of such systems.We propose a global COVID-19 information sharing and risk notification system that utilizes the Blockchain, Smart Contract, and Bluetooth. To protect user privacy, we design a novel Blockchain-based platform that can share consistent and non-tampered contact tracing information from multiple dimensions, such as location-based for indirect contact and Bluetooth-based for direct contact. Hierarchical smart contract architecture is also designed to achieve global agreements from users about how to process and utilize user data, thereby enhancing the data usage transparency. Furthermore, we propose a mechanism to protect user identity privacy from multiple aspects. More importantly, our system can notify the users about the exposure risk via smart contracts. We implement a prototype system to conduct extensive measurements to demonstrate the feasibility and effectiveness of our system.

Motivated by resource-limited scenarios, knowledge distillation (KD) has received growing attention, effectively and quickly producing lightweight yet high-performance student models by transferring the dark knowledge from large teacher models. However, many pre-trained teacher models are downloaded from public platforms that lack necessary vetting, posing a possible threat to knowledge distillation tasks. Unfortunately, thus far, there has been little research to consider the backdoor attack from the teacher model into student models in KD, which may pose a severe threat to its wide use. In this paper, we, for the first time, propose a novel Anti-Distillation Backdoor Attack (ADBA), in which the backdoor embedded in the public teacher model can survive the knowledge distillation process and thus be transferred to secret distilled student models. We first introduce a shadow to imitate the distillation process and adopt an optimizable trigger to transfer information to help craft the desired teacher model. Our attack is powerful and effective, which achieves 95.92%, 94.79%, and 90.19% average success rates of attacks (SRoAs) against several different structure student models on MNIST, CIFAR-10, and GTSRB, respectively. Our ADBA also performs robustly under different user distillation environments with 91.72% and 92.37% average SRoAs on MNIST and CIFAR-10, respectively. Finally, we show that the ADBA has a low overhead in the injecting process, which converges on 50 and 70 epochs on CIFAR-10 and GTSRB, respectively, while the normal training epochs of these datasets are almost 200.

In recent years, there is an emerging trend that some computing services are moving from cloud to the edge of the networks. Compared to cloud computing, edge computing can provide services with faster response, lower expense, and more security. The massive idle computing resources closing to the edge also enhance the deployment of edge services. Instead of using cloud services from some primary providers, edge computing provides people a great chance to join the market of computing resources actively. However, edge computing also has some critical impediments that we have to overcome.In this paper, we design an edge computing service platform that can receive and distribute the computing resources from the end-users in a decentralized way. Without the centralized trade control, we propose a novel Blockchain-enabled decentralized technique to establish the trade trust among users and implement it with using embedded immutable intermediary smart contract. Our system also considers and resolves a variety of security and privacy challenges when utilizing the Blockchain technique. We implement our system and conduct extensive experiments to show the feasibility and effectiveness of our proposed system.