Deep neural networks (DNNs) underpin critical applications yet remain vulnerable to backdoor attacks, typically reliant on heuristic brute-force methods. Despite significant empirical advancements in backdoor research, the lack of rigorous theoretical analysis limits understanding of underlying mechanisms, constraining attack predictability and adaptability. Therefore, we provide a theoretical analysis targeting backdoor attacks, focusing on how sparse decision boundaries enable disproportionate model manipulation. Based on this finding, we derive a closed-form ''ambiguous boundary region'' wherein negligible relabeled samples induce substantial misclassification. Influence function analysis further quantifies significant parameter shifts caused by these margin samples, with minimal impact on clean accuracy, formally grounding why such low poison rates suffice for efficacious attacks. Leveraging these insights, we propose Eminence, an explainable and robust black-box backdoor framework with provable theoretical guarantees and inherent stealth properties. Eminence optimizes a universal, visually subtle trigger that strategically exploits vulnerable decision boundaries and effectively achieves robust misclassification with exceptionally low poison rates (≤ 0.01%, compared to SOTA methods typically requiring ≥ 1 %). Comprehensive experiments validate our theoretical discussions and demonstrate the effectiveness of Eminence, confirming an exponential relationship between margin poisoning and adversarial boundary manipulation. Eminence maintains ≥ 90% attack success rate, exhibits negligible clean-accuracy loss, and demonstrates high transferability across diverse models, datasets and scenarios. Our code is available at https://github.com/NESA-Lab/Eminence

Deep learning models heavily depend on training data quality. While online datasets offer cost-effective solutions for diversity and scale, they introduce security risks. Malicious actors can inject hidden triggers, enabling backdoor attacks that compromise model integrity. Existing defenses remain limited—often demanding large clean datasets, showing inconsistent robustness across attacks, and struggling against adaptive adversaries. Therefore, in this paper, we endeavor to address the challenges of backdoor attack countermeasures in real-world scenarios, thereby fortifying the security of the training paradigm under the data-collection manner. Concretely, we first explore the inherent relationship between the robustness of the poisoned samples, demonstrating the poisoned samples are more robust to perturbation than the clean ones through the theoretical analysis and experiments. Then, we propose a robust and clean-data-free backdoor defense framework, namely Mellivora Capensis (MeCa), which enables training a clean model on the poisoned dataset.MeCadetects poisoned samples and trains clean models without needing clean data or prior knowledge of the poisoning (e.g., poison ratio). We conduct extensive experiments in defending against 8 SOTA attacks (including 3 adaptive attacks) on 4 datasets. The experimental results reveal thatMeCacan achieve an average attack success rate with almost 0.00% to defend against SOTA backdoor attacks while maintaining model availability, which outperforms 7 SOTA backdoor defense methods. Furthermore, the excellent performance on 3 different model architectures and poison ratios also highlights the remarkable generalization capability ofMeCa.

Vertical Federated Learning (VFL) is an emerging paradigm that enables collaborators to build machine learning models together in a distributed fashion. However, the security of the VFL model remains underexplored, particularly regarding the Byzantine Generals Problem (BGP), which is a well-known issue in distributed systems. This paper focuses on revealing the threat of BGP in VFL systems. Specifically, we propose two attacks, the replay attack and the generation attack, to evaluate the vulnerability of VFL when there is only one malicious party. The goal of the adversary is to hijack the VFL model to give desired predictions. Moreover, considering the uneven distribution of importance among parties, we combine data poisoning with the aforementioned attacks to explore whether they can bypass the situation where the adversary has few features. The evaluation results demonstrate the effectiveness of our attacks. For instance, the adversary holding only 10 90 capability is limited and usually at the cost of performance loss of the VFL task. Our work highlights the need for advanced defenses to protect the prediction results of a VFL model and calls for more exploration of VFL's security issues.

Deep neural networks (DNNs) are vulnerable to adversarial examples (AEs) that mislead the model while appearing benign to human observers. A critical concern is the transferability of AEs, which enables black-box attacks without direct access to the target model. However, many previous attacks have failed to explain the intrinsic mechanism of adversarial transferability, lacking a unified and representative metric for transferability as well. In this paper, we rethink the property of transferable AEs and develop a novel metric to measure transferability from the perspective of generalization. Building on insights from this metric, we analyze the generalization of AEs across models with different architectures and prove that we can find a local perturbation to mitigate the gap between surrogate and target models. We further establish the inner connections between model smoothness and flat local maxima, both of which contribute to the transferability of AEs. Further, we propose a new adversarial attack algorithm, Adversarial Weight Tuning (AWT), which adaptively adjusts the parameters of the surrogate model using generated AEs to optimize the flat local maxima and model smoothness simultaneously, without the need for extra data. AWT is a data-free tuning method that combines gradient-based and model-related attack methods to enhance the transferability of AEs. Extensive experiments on a variety of models with different architectures on ImageNet demonstrate that AWT yields superior performance over other attacks, with an average increase of nearly 5% and 10% attack success rates on CNN-based and Transformer-based models, respectively, compared to state-of-the-art attacks.

In the burgeoning domain of machine learning, the reliance on third-party services for model training and the adoption of pre-trained models have surged. However, this reliance introduces vulnerabilities to model hijacking attacks, where adversaries manipulate models to perform unintended tasks, leading to significant security and ethical concerns, like turning an ordinary image classifier into a tool for detecting faces in pornographic content, all without the model owner’s knowledge. This paper introduces Category-Agnostic Model Hijacking (CAMH), a novel model hijacking attack method capable of addressing the challenges of class number mismatch, data distribution divergence, and performance balance between the original and hijacking tasks. CAMH incorporates synchronized training layers, random noise optimization, and a dual-loop optimization approach to ensure minimal impact on the original task’s performance while effectively executing the hijacking task. We evaluate CAMH across multiple benchmark datasets and network architectures, demonstrating its potent attack effectiveness while ensuring minimal degradation in the performance of the original task.

Backdoor attacks embed malicious triggers into training data, enabling attackers to manipulate neural network behavior during inference while maintaining high accuracy on benign inputs. However, existing backdoor attacks face limitations manifesting in excessive reliance on training data, poor stealth, and instability, which hinder their effectiveness in real-world applications. Therefore, this paper introduces ShadowPrint, a versatile backdoor attack that targets feature embeddings within neural networks to achieve high ASRs and stealthiness. Unlike traditional approaches, ShadowPrint reduces reliance on training data access and operates effectively with exceedingly low poison rates (as low as 0.01%). It leverages a clustering-based optimization strategy to align feature embeddings, ensuring robust performance across diverse scenarios while maintaining stability and stealth. Extensive evaluations demonstrate that ShadowPrint achieves superior ASR (up to 100%), steady CA (with decay no more than 1% in most cases), and low DDR (averaging below 5%) across both clean-label and dirty-label settings, and with poison rates ranging from as low as 0.01% to 0.05%, setting a new standard for backdoor attack capabilities and emphasizing the need for advanced defense strategies focused on feature space manipulations.

The rise of advanced voice deepfake technologies has raised serious concerns over user audio privacy, as malicious actors increasingly exploit publicly available voice data to generate convincing fake audio for malicious purposes such as identity theft, financial fraud and misinformation campaigns. While existing defense methods offer partial protection, they suffer from critical limitations, including weak adaptability to unseen user data, poor scalability to long audio, regid reliance on white-box knowledge and high computational and temporal costs to encryption process. Therefore, to defend against personalized voice deepfake threats, we propose Enkidu, a novel user-oriented privacy-preserving framework that leverages universal frequential perturbations generated through black-box knowledge and few-shot training on a small amount of user samples. These high-malleablity frequency-domain noise patches enable real-time, lightweight protection with strong generalization across variable-length audio and robust resistance against voice deepfake attacks-all while preserving high perceptual and intelligible audio quality. Notably, Enkidu achieves over 50-200× processing memory efficiency (requiring only 0.004 GB) and over 3-7000× runtime efficiency (real-time coefficient as low as 0.004) compared to six SOTA countermeasures. Extensive experiments across six mainstream Text-to-Speech (TTS) models and five cutting-edge Automated Speaker Verification (ASV) models demonstrate the effectiveness, transferability, and practicality of Enkidu in defending against voice deepfakes and adaptive attacks.

Face recognition service has been widely adopted across various domains, offering significant convenience and enhancing efficiency in numerous applications. However, once a user's facial data is transmitted to a service provider, the user will lose control over his/her biometric data. In recent years, there have been various security and privacy issues due to the leakage of facial data. Although many privacy enhancement methods have been proposed, they usually fail when they are not accessible to adversaries' strategies or the complete face recognition model. Therefore, in this work, we propose a Privacy Minimization Transformation (PMT) method, designed to address two common scenarios in practical face recognition systems: the uploading of facial images and facial features. This method can process the private facial data based on the shallow network of the face recognition model to obtain the obfuscated data. The obfuscated data cannot only maintain satisfactory performance on the authorized models (i.e., the models specified by the user) and restrict the performance on other unauthorized models (i.e., the models not specified by the user) but also prevent privacy data from leaking by AI methods and human visual theft. Additionally, since a service provider may execute preprocessing operations on the received data, we propose an enhanced perturbation method to improve the robustness of PMT. Besides, to authorize one facial image to multiple service models simultaneously, a multiple-restriction mechanism is proposed to improve the scalability of PMT. Finally, we conduct extensive experiments and evaluate the effectiveness of the proposed PMT against face reconstruction, function creep, and face attribute estimation attacks. Experimental results demonstrate that PMT performs well in preventing facial function creep and privacy leakage while maintaining high face recognition accuracy

Natural language processing (NLP) models are widely used in various scenarios, yet they are vulnerable to adversarial attacks. Existing works aim to mitigate this vulnerability, but each work targets a specific attack category or has computational overhead limitations, making them vulnerable to adaptive attacks. In this paper, we exhaustively investigate the adversarial attack algorithms in NLP and discover that existing attack algorithms mainly disrupt the importance distribution of words in a text. A well-trained model can distinguish subtle importance distribution differences between clean and adversarial texts. Based on this intuition, we propose TextDefense, a new adversarial example detection framework that utilizes the target model’s capability to defend against adversarial attacks, requiring no prior knowledge. Unlike previous approaches, TextDefense is attack-type agnostic and outperforms existing methods in experiments with different architectures, datasets, and attack methods. We also discover that the target model’s generalizability is a leading factor influencing the performance of TextDefense. Finally, we provide insights into the adversarial attacks in NLP and the principles of our defense method by analyzing the properties of the target model and the adversarial example.

To tackle the scarcity and privacy issues associated with domain-specific datasets, the integration of federated learning in conjunction with fine-tuning (FT) has emerged as a practical solution. However, our findings reveal that federated learning has the risk of skewing FT features and compromising the out-of-distribution (OOD) robustness of pretrained models. By introducing three robustness indicators and conducting experiments across diverse robust datasets, we elucidate these phenomena by scrutinizing the ability of data representations, transferability, and deviations within the model. To mitigate the negative impact of practical federated learning on model robustness, we introduce a general noisy projection (GNP)-based robust algorithm, ensuring no deterioration of accuracy on the target distribution. Specifically, the key strategy for enhancing model robustness entails the transfer of robustness from the pretrained model to the fine-tuned model, coupled with adding a small amount of Gaussian noise to augment the representative capacity of the model. The comprehensive experimental results demonstrate that our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient FT (PEFT) methods and confronting different levels of label distribution skew and quantity distribution skew.

Large Language Models (LLMs) are powerful but also raise significant security concerns, particularly regarding the harm they can cause, such as generating fake news that manipulates public opinion on social media and providing responses to unethical activities. Traditional red teaming approaches for identifying AI vulnerabilities rely on manual prompt construction and expertise. This paper introduces AdversaFlow, a novel visual analytics system designed to enhance LLM security against adversarial attacks through human-AI collaboration. AdversaFlow involves adversarial training between a target model and a red model, featuring unique multi-level adversarial flow and fluctuation path visualizations. These features provide insights into adversarial dynamics and LLM robustness, enabling experts to identify and mitigate vulnerabilities effectively. We present quantitative evaluations and case studies validating our system's utility and offering insights for future AI security solutions. Our method can enhance LLM security, supporting downstream scenarios like social media regulation by enabling more effective detection, monitoring, and mitigation of harmful content and behaviors.

Vertical Federated Learning (VFL) is a solution increasingly used by companies with the same user group but differing features, enabling them to collaboratively train a machine learning model. VFL ensures that clients exchange intermediate results extracted by their local models, without sharing raw data. However, in practice, VFL encounters several challenges, such as computational and communication overhead, privacy leakage risk, and adversarial attack. Our study reveals that the usage of floating-point (FP) numbers is a common factor causing these issues, as they can be redundant and contain too much information. To address this, we propose a new architecture called rounding layer, which converts intermediate results to integers. Our theoretical analysis and empirical results demonstrate the benefits of the rounding layer in reducing computation and memory overhead, providing privacy protection, preserving model performance, and mitigating adversarial attacks. We hope this paper inspires further research into novel architectures to address practical issues in VFL.

Recent advancements in multi-agent reinforcement learning (MARL) have opened up vast application prospects, such as swarm control of drones, collaborative manipulation by robotic arms, and multi-target encirclement. However, potential security threats during the MARL deployment need more attention and thorough investigation. Recent research reveals that attackers can rapidly exploit the victim's vulnerabilities, generating adversarial policies that result in the failure of specific tasks. For instance, reducing the winning rate of a superhuman-level Go AI to around 20%. Existing studies predominantly focus on two-player competitive environments, assuming attackers possess complete global state observation.

The rich semantic information in Control Flow Graphs (CFGs) of executable programs has made Graph Neural Networks (GNNs) a key focus for malware detection. However, existing CFG-based detection techniques face limitations in node feature extraction, such as information loss, neglect of execution sequence information, and redundancy in representation vectors. These limitations compromise the balance between high efficiency and precision when training detectors. Addressing this, we introduce an innovative Malware CFG Node Embedding (MalGNE) method. This approach utilizes a novel instruction encoding rule to address the Out-Of-Vocabulary(OOV) problem, generates high-quality initial vectors. Then, it employs aggregation layer and sequence layer to extract node aggregation feature and execution sequence feature, in conjunction with GNNs to develop a pre-trained node embedding model. The model maps the semantic information of node assembly instruction sequences into a compact, low-dimensional continuous space, ensuring high-quality feature extraction, and enhancing the performance and efficiency of the detector. We trained the MalGNE model using the BIG 2015 dataset and validated MalGNE-enhanced detector on the SOREL-20M and BODMAS datasets. MalGNE-enhanced detector demonstrates outstanding performance and efficiency in low-dimensional spaces, especially when the dimensionality of the node feature vector is reduced to 16. MalGNE-enhanced detector not only maintains a high detection accuracy of 95.49%. sacrificing only about 1.7% of accuracy to save approximately 73% of training time compared to 128 dimensions.

Vertical federated learning (VFL) is an emerging privacy-preserving paradigm that enables collaboration between companies. These companies have the same set of users but different features. One of them is interested in expanding new business or improving its current service with others’ features. For instance, an e-commerce company, who wants to improve its recommendation performance, can incorporate users’ preferences from another corporation such as a social media company through VFL. On the other hand, graph data is a powerful and sensitive type of data widely used in industry. Their leakage, e.g., the node leakage and/or the relation leakage, can cause severe privacy issues and financial loss. Therefore, protecting the security of graph data is important in practice. Though a line of work has studied how to learn with graph data in VFL, the privacy risks remain underexplored. In this paper, we perform the first systematic study onrelation inference attacksto reveal VFL's risk of leaking samples’ relations. Specifically, we assume the adversary to be a semi-honest participant. Then, according to the adversary's knowledge level, we formulate three kinds of attacks based on different intermediate representations. Particularly, we design a novel numerical approximation method to handle VFL's encryption mechanism on the participant's representations. Extensive evaluations with four real-world datasets demonstrate the effectiveness of our attacks. For instance, the area under curve of relation inference can reach more than 90%, implying an impressive relation inference capability. Furthermore, we evaluate possible defenses to examine our attacks’ robustness. The results show that their impacts are limited. Our work highlights the need for advanced defenses to protect private relations and calls for more exploration of VFL's privacy and security issues.

Neural networks have become increasingly popular. Nevertheless, understanding their decision process turns out to be complicated. One vital method to explain a models' behavior is feature attribution, i.e., attributing its decision to pivotal features. Although many algorithms are proposed, most of them aim to improve the faithfulness (fidelity) to the model. However, the real environment contains many random noises, which may cause the feature attribution maps to be greatly perturbed for similar images. More seriously, recent works show that explanation algorithms are vulnerable to adversarial attacks, generating the same explanation for a maliciously perturbed input. All of these make the explanation hard to trust in real scenarios, especially in security-critical applications.

Due to the abundant marine resources, smart ocean has attracted much attention of the government, industry, and academy. The Internet-of-Things (IoT) architectures for smart ocean have been proposed to collect various of data from the ocean, thereby assisting environmental protection, military reconnaissance, and so on. However, few researchers have paid attention to the security and privacy issues of data collection and transmission. In this article, for the unreliable underwater environment, we present a secure, efficient, and complete data collection, and transmission and storage scheme for IoT in smart ocean. Especially, to prolong the lifetime of the underwater node, two novel data compression algorithms [lossy data compression algorithm (LCA) and lossless data compression algorithm (NLCA)] are also proposed. Moreover, due to the vulnerability of underwater nodes, we also propose a corresponding IoT framework and data collection pattern to resist the single point failure attack. Besides, to guarantee the confidentiality, reliability, and integrity of transmitting data, Elliptic Curve-ElGamal (EC-ElGamal) and elliptic curve digital signature algorithm (ECDSA) are employed. The consensus algorithm and blacklisting mechanism are also employed to detect and address failure or malicious nodes. Finally, the security analysis demonstrates that our scheme is able to resist many typical attacks for underwater nodes, such as manipulation attacks, Distributed Denial-of-Service (DDoS) attacks, malicious node injection attacks, and so on. Additionally, relevant experimental results show that the scheme is feasibility and efficiency.

Edge servers (ESs) are utilized to achieve the storage and sharing of IoT data. However, even if ES brings us much benefit, it also leads to many serious privacy leakage issues because users' data in ESs are out of control. Moreover, ES providers may also disclose user's private-sensitive data. Hence, in this article, we present a privacy-preserving, recoverable, and revocable edge data sharing scheme. In this scheme, we propose a novel attribute revocation chain based on the blockchain technology to achieve attribute revocation in ciphertext-policy attribute-based encryption (CP-ABE). Meanwhile, a secret sharing scheme (SSS) is introduced to assist the data recovery. Especially, for the situation that a single ES is hijacked, we also propose a corresponding efficient detection mechanism and key updating policy to promise the subsequent security of the whole system. Moreover, this scheme also resists Economic Denial-of-Sustainability (EDoS) attacks which are launched by some malicious users. The analysis shows that the proposed scheme can protect user's privacy and resist many attacks. Additionally, relevant experimental results demonstrate that our scheme has low computational overhead on the user side.

As the next generation of information and communication infrastructure, Internet of Things (IoT) enables many advanced applications such as smart healthcare, smart grid, smart home, and so on, which provide the most flexibility and convenience in our daily life. However, pervasive security and privacy issues are also increasing in IoT. For instance, an attacker can get health condition of a patient via analyzing real-time records in a smart healthcare application. Therefore, it is very important for users to protect their private data. In this paper, we present two efficient data aggregation schemes to preserve private data of customers. In the first scheme, each IoT device slices its actual data randomly, keeps one piece to itself, and sends the remaining pieces to other devices which are in the same group via symmetric encryption. Then, each IoT device adds the received pieces and the held piece together to get an immediate result, which is sent to the aggregator after the computation. Moreover, homomorphic encryption and AES encryption are employed to guarantee secure communication. In the second scheme, the slicing strategy is also employed. Noise data are introduced to prevent the exchanged actual data of devices from disclosure when the devices blend data each other. AES encryption is also employed to guarantee secure communication between devices and aggregator, compared to homomorphic encryption, which has significantly less computational cost. Analysis shows that integrity and confidentiality of IoT devices’ data can be guaranteed in our schemes. Both schemes can resist external attack, internal attack, colluding attack, and so on.