Adversarial perturbations (APs) have become a great concern in image classification tasks. The most challenging branch, universal adversarial perturbations (UAPs), are exploited to fool most of the unseen samples. Such one-to-all perturbations have the merit of transferability, which has strong practical significance. In this paper, we firstly define the transferability gap and the algorithm stability of the UAP algorithm, and prove the relationship between them. In analyzing the UAP algorithm stability, we prove that the convergence domain of existing UAP algorithms with dynamic constraints is excessively small, which degrades the capacity of UAPs. Thus, we further propose a new expected constraint and prove that UAPs in the expected constraint suit any sample in a high probability. Besides, we propose a Stochastic Universal Adversarial Perturbation (SUAP) that involves additive noise and the expected constraint. Finally, by treating the proposed algorithm as a stochastic differential equation, we prove an upper bound of the UAP algorithm stability of SUAP, which decreases exponentially at the beginning and then increases with a sublinear rate to at most a fixed constant. Experimental results show that SUAP is aligned with our analysis.

The core challenge of high-dimensional and expensive black-box optimization (BBO) is how to obtain better performance faster with little function evaluation cost. The essence of the problem is how to design an efficient optimization strategy tailored to the target task. This paper designs a powerful optimization framework to automatically learn the optimization strategies from the target or cheap surrogate task without human intervention. However, current methods are weak for this due to poor representation of optimization strategy. To achieve this, 1) drawing on the mechanism of genetic algorithm, we propose a deep neural network framework called B2Opt, which has a stronger representation of optimization strategies based on survival of the fittest; 2) B2Opt can utilize the cheap surrogate functions of the target task to guide the design of the efficient optimization strategies. Compared to the state-of-the-art BBO baselines, B2Opt can achieve multiple orders of magnitude performance improvement with less function evaluation cost.

With the urgent need to enhance the safety of LLM applications, there has been a growing focus on alignment training algorithms designed to keep large language models (LLMs) behaving in alignment with human values. Alignment training algorithms rely heavily on preference datasets, which are essential for finetuning LLMs to follow human preferences. However, generating and annotating these datasets is often costly and labor-intensive, making it critical to protect their copyright against unauthorized use. In this paper, we propose PreferCare, the first framework tailor-made for preference dataset copyright protection via watermark injection and verification. PreferCare comprises two consecutive stages: injection and verification. In the injection stage, a style transfer-based watermark signal and a bi-level watermark optimization process are designed to embed the watermark into the preference dataset. In the verification stage, we employ statistical tests to determine whether a suspect LLM has used the watermarked preference dataset without authorization. Extensive experiments on multiple popular LLMs have demonstrated that PreferCare achieves effectiveness, harmlessness, transferability, and robustness across diverse settings, and can successfully verify the watermark within 20 queries.

Vision-Language Retrieval-Augmented Generation (VLRAG) systems have been widely applied to Large Vision-Language Models (LVLMs) to enhance their generation ability. However, the reliance on external multimodal knowledge databases renders VLRAG systems vulnerable to malicious poisoning attacks. In this paper, we introduce PoisonedEye, the first knowledge poisoning attack designed for VLRAG systems. Our attack successfully manipulates the response of the VLRAG system for the target query by injecting only one poison sample into the knowledge database. To construct the poison sample, we follow two key properties for the retrieval and generation process, and identify the solution by satisfying these properties. Besides, we also introduce a class query targeted poisoning attack, a more generalized strategy that extends the poisoning effect to an entire class of target queries. Extensive experiments on multiple query datasets, retrievers, and LVLMs demonstrate that our attack is highly effective in compromising VLRAG systems.

Zero-shot optimization aims to achieve both generalization and performance gains on solving previously unseen black-box optimization problems over SOTA methods without task-specific tuning. Pre-trained optimization models (POMs) address this challenge by learning a general mapping from task features to optimization strategies, enabling direct deployment on new tasks. In this paper, we identify three essential components that determine the effectiveness of POMs: (1) task feature modeling, which captures structural properties of optimization problems; (2) optimization strategy representation, which defines how new candidate solutions are generated; and (3) the feature-to-strategy mapping mechanism learned during pre-training. However, existing POMs often suffer from weak feature representations, rigid strategy modeling, and unstable training. To address these limitations, we propose EPOM, an enhanced framework for pre-trained optimization. EPOM enriches task representations using a cross-attention-based tokenizer, improves strategy diversity through deformable attention, and stabilizes training by replacing non-differentiable operations with a differentiable crossover mechanism. Together, these enhancements yield better generalization, faster convergence, and more reliable performance in zero-shot black-box optimization.

Foundation models for time series analysis (TSA) have attracted significant attention. However, challenges such as training data scarcity and imbalance continue to hinder their development. Inspired by complex dynamic system theories, we design a series-symbol data generation mechanism, enabling the unrestricted creation of high-quality time series data paired with corresponding symbolic expressions. To leverage series-symbol data pairs with strong correlations, we develop SymTime, a pre-trained foundation model for enhancing time series representation using symbolic information. SymTime demonstrates competitive performance across five major TSA tasks when fine-tunes with downstream tasks, rivaling foundation models pre-trained on real-world datasets. This approach underscores the potential of series-symbol data generation and pretraining mechanisms in overcoming data scarcity and enhancing task performance. The code is available at https://github.com/wwhenxuan/SymTime.

Graph neural networks (GNNs) can retain the structural information of graph data when processing graph data via message passing mechanism. Recently, GNNs have been widely used in recommendation systems, social networks, finance, etc. However, GNNs are also vulnerable when encountering severe security and data privacy challenges. Membership inference attacks (MIA) on graph data can make the trained GNN model leak the training data, which causes serious privacy problems. However, all the studies reviewed so far suffer from the fact that there are few types of research on GNNs to defend against MIA. In this paper, we proposeMPGStack: a framework that utilizes ensemble learning of GNNs to safeguard against MIA. More concretely, we propose three ensemble strategies inMPGStack: KFold, GP-KFold, and LPA-KFold, which partition the graph at different degrees of granularity based on random uniform partition, class information, and label propagation, respectively.MPGStackcan enhance the target model's generalization performance, mitigate MIA, and boost classification accuracy. The experimental results conducted on seven benchmark datasets demonstrate thatMPGStackcan effectively defend against MIA and significantly reduce the attack model's success rate to the level of random guessing.MPGStackalso improves the classification accuracy of the target model up to 99% or more. Extensive experimental results demonstrate thatMPGStackis more effective, stable, and applicable to different datasets and scenarios than other methods, achieving random guessing attack models while improving the model's classification accuracy on different datasets.

Deep learning models have become ubiquitous in myriad application areas due to their remarkable performance. This success would not be possible without the high-quality datasets for model training that are contributed by numerous data owners. Datasets have not only become valuable assets for data owners, but also contain sensitive information that raises concerns about privacy leakage. This gives rise to urgent needs for data owners to verify that model developers have stopped using their datasets immediately upon receiving data deletion requests, as mandated by the right to be forgotten regulation. In this paper, we provide an affirmative answer by proposingDuplexGuard: a novel framework for deletion right verification via a duplex watermarking approach. During watermark injection, for each owner's dataset,DuplexGuardgenerates duplex subsets of watermarked samples, i.e., the ambush subset and the surfacing subset. This duplex design is capable of offering a combination of watermark behaviors before and after data deletion, therefore allowing it to signify all potential dataset usage statuses.DuplexGuardalso proposes a new two-way handshake protocol for issuing data deletion requests to provide more robust and decisive verification for the deletion right. Extensive experiments on multiple benchmark datasets demonstrate thatDuplexGuardis effective and reliable in verification.

Pre-trained models are extensively embraced in deep learning, facilitating efficient fine-tuning for downstream user-specific tasks and yielding substantial computational savings. However, backdoor attacks present a significant security threat to downstream models constructed on corrupted pre-trained models, necessitating the implementation of effective countermeasures to mitigate this threat prior to deploying the models in safety-critical applications. This paper introducesPurifierand its advanced versionPurifier$^{+}$, the former of which mitigates backdoors in pre-trained models by aligning anomaly activation to normal activation, and the latter builds on this by making importance rating about activation patterns, boosting important activation patterns and suppressing unimportant activation patterns.PurifierandPurifier$^{+}$draw inspiration from the observation that anomaly activation patterns for backdoor triggers manifest across various perspectives such as channel-wise, cube-wise, and feature-wise, each exhibiting distinct levels of granularity. Crucially, the choice of alignment granularity plays a pivotal role in ensuring robustness and accuracy. In addressing this challenge,PurifierandPurifier$^{+}$demonstrate the ability to effectively thwart various categories of backdoor triggers devoid of requiring prior information about the specific backdoor attacks. Additionally, it offers a convenient and flexible deployment feature, namely, plug-and-play capability. The comprehensive experimental results demonstrate thatPurifierandPurifier$^{+}$outperform current methodologies regarding defense efficacy and accuracy in model inference with uncontaminated samples when subjected to a series of State-of-the-Art mainstream attacks.

Class-imbalanced node classification tasks are prevalent in real-world scenarios. Due to the uneven distribution of nodes across different classes, learning high-quality node representations remains a challenging endeavor. The engineering of loss functions has shown promising potential in addressing this issue. It involves the meticulous design of loss functions, utilizing information about the quantities of nodes in different categories and the network’s topology to learn unbiased node representations. However, the design of these loss functions heavily relies on human expert knowledge and exhibits limited adaptability to specific target tasks. In this paper, we introduce a high-performance, flexible, and generalizable automated loss function search framework to tackle this challenge. Across 15 combinations of graph neural networks and datasets, our framework achieves a significant improvement in performance compared to state-of-the-art methods. Additionally, we observe that homophily in graph-structured data significantly contributes to the transferability of the proposed framework.

As concerns over privacy protection grow and relevant laws come into effect, machine unlearning (MU) has emerged as a pivotal research area. Due to the complexity of the forgetting data distribution, the sample-wise MU is still open challenges. Gradient ascent, as the inverse of gradient descent, is naturally applied to machine unlearning, which is also the inverse process of machine learning. However, the straightforward gradient ascent MU method suffers from the trade-off between effectiveness, fidelity, and efficiency. In this work, we analyze the gradient ascent MU process from a multi-task learning (MTL) view. This perspective reveals two problems that cause the trade-off, i.e., the gradient direction problem and the gradient dominant problem. To address these problems, we propose a novel MU method, namely GDR-GMA, consisting of Gradient Direction Rectification (GDR) and Gradient Magnitude Adjustment (GMA). For the gradient direction problem, GDR rectifies the direction between the conflicting gradients by projecting a gradient onto the orthonormal plane of the conflicting gradient. For the gradient dominant problem, GMA dynamically adjusts the magnitude of the update gradients by assigning the dynamic magnitude weight parameter to the update gradients. Furthermore, we evaluate GDR-GMA against several baseline methods in three sample-wise MU scenarios: random data forgetting, sub-class forgetting, and class forgetting. Extensive experimental results demonstrate the superior performance of GDR-GMA in effectiveness, fidelity, and efficiency

Zero-shot optimization involves optimizing a target task that was not seen during training, aiming to provide the optimal solution without or with minimal adjustments to the optimizer. It is crucial to ensure reliable and robust performance in various applications. Current optimizers often struggle with zero-shot optimization and require intricate hyperparameter tuning to adapt to new tasks. To address this, we propose a Pretrained Optimization Model (POM) that leverages knowledge gained from optimizing diverse tasks, offering efficient solutions to zero-shot optimization through direct application or fine-tuning with few-shot samples. Evaluation on the BBOB benchmark and two robot control tasks demonstrates that POM outperforms state-of-the-art black-box optimization methods, especially for high-dimensional tasks. Fine-tuning POM with a small number of samples and budget yields significant performance improvements. Moreover, POM demonstrates robust generalization across diverse task distributions, dimensions, population sizes, and optimization horizons. For code implementation, see https://github.com/ninja-wm/POM/.

In the realm of daily services, the deployment of deep neural networks underscores the paramount importance of their reliability. However, the vulnerability of these networks to adversarial attacks, primarily evasion-based, poses a concerning threat to their functionality. Common methods for enhancing robustness involve heavy adversarial training or leveraging learned knowledge from clean data, both necessitating substantial computational resources. This inherent time-intensive nature severely limits the agility of large foundational models to swiftly counter adversarial perturbations. To address this challenge, this paper focuses on the \textbf{Ra}pid \textbf{P}lug-\textbf{i}n \textbf{D}efender (\textbf{RaPiD}) problem, aiming to rapidly counter adversarial perturbations without altering the deployed model. Drawing inspiration from the generalization and the universal computation ability of pre-trained transformer models, we propose a novel method termed \textbf{CeTaD} (\textbf{C}onsidering Pr\textbf{e}-trained \textbf{T}ransformers \textbf{a}s \textbf{D}efenders) for RaPiD, optimized for efficient computation. \textbf{CeTaD} strategically fine-tunes the normalization layer parameters within the defender using a limited set of clean and adversarial examples. Our evaluation centers on assessing \textbf{CeTaD}'s effectiveness, transferability, and the impact of different components in scenarios involving one-shot adversarial examples. The proposed method is capable of rapidly adapting to various attacks and different application scenarios without altering the target model and clean training data. We also explore the influence of varying training data conditions on \textbf{CeTaD}'s performance. Notably, \textbf{CeTaD} exhibits adaptability across differentiable service models and proves the potential of continuous learning.

Training a deep learning model from scratch requires a great deal of available labeled data, computation resources, and expert knowledge. Thus, the time-consuming and complicated learning procedure catapulted the trained model to valuable intellectual property (IP), spurring interest from attackers in model copyright infringement and stealing. Recently, a new defense approach leverages watermarking techniques to inject watermarks into the training procedure and verify model ownership when necessary. To our best knowledge, there is no research work on model ownership stealing attacks in federated learning, and the existing defense or mitigation methods can not be directly used for federated learning scenarios. In this paper, we introduce watermarking neural networks in asynchronous federated learning and propose a novel model privacy attack, dubbed model ownership deprivation attack (MODA). MODA is launched by an inside adversarial participant, targeting occupying and depriving the remaining participants' (victims) copyright to achieve his maximum profit. The extensive experimental results on five benchmark datasets (MNIST, Fashion-MNIST, GTSRB, SVHN, CIFAR10) show that MODA is highly effective in a two-participant learning scenario with a minor impact on model's performance. When extending MODA into multiple participants scenario, MODA still maintains high attack success rate and classification accuracy. Compared to the state-of-the-art works, MODA has a higher attack success rate than the black-box solution and comparable efficacy with the approach in the white-box scenario.

Matrix factorization (MF) is a fundamental model in data mining and machine learning, which finds wide applications in diverse application areas, including recommendation systems with user-item rating matrices, phenotype extraction from electronic health records, and spatial-temporal data analysis for check-in records. The "right to be forgotten" has become an indispensable privacy consideration due to the widely enforced data protection regulations, which allow personal users having contributed their data for model training to revoke their data through a data deletion request. Consequently, it gives rise to the emerging task of machine unlearning for the MF model, which removes the influence of the matrix rows/columns from the trained MF factors upon receiving the deletion requests from the data owners of these rows/columns. The central goal is to effectively remove the influence of the rows/columns to be forgotten, while avoiding the computationally prohibitive baseline approach of retraining from scratch. Existing machine unlearning methods are either designed for single-variable models and not compatible with MF that has two factors as coupled model variables, or require alternative updates that are not efficient enough. In this paper, we propose a closed-form machine unlearning method. In particular, we explicitly capture the implicit dependency between the two factors, which yields the total Hessian-based Newton step as the closed-form unlearning update. In addition, we further introduce a series of efficiency-enhancement strategies by exploiting the structural properties of the total Hessian. Extensive experiments on five real-world datasets from three application areas as well as synthetic datasets validate the efficiency, effectiveness, and utility of the proposed method.

Machine unlearning can fortify the privacy and security of machine learning applications. Unfortunately, the exact unlearning approaches are inefficient, and the approximate unlearning approaches are unsuitable for complicated CNNs. Moreover, the approximate approaches have serious security flaws because even unlearning completely different data points can produce the same contribution estimation as unlearning the target data points. To address the above problems, we try to define machine unlearning from the knowledge perspective, and we propose a knowledge-level machine unlearning method, namely ERM-KTP. Specifically, we propose an entanglement-reduced mask (ERM) structure to reduce the knowledge entanglement among classes during the training phase. When receiving the un-learning requests, we transfer the knowledge of the non-target data points from the original model to the unlearned model and meanwhile prohibit the knowledge of the target data points via our proposed knowledge transfer and prohibition (KTP) method. Finally, we will get the un-learned model as the result and delete the original model to accomplish the unlearning process. Especially, our proposed ERM-KTP is an interpretable unlearning method because the ERM structure and the crafted masks in KTP can explicitly explain the operation and the effect of un-learning data points. Extensive experiments demonstrate the effectiveness, efficiency, high fidelity, and scalability of the ERM-KTP unlearning method. Code is available at https://github.com/RUIYUN-ML/ERM-KTP

Adversarial training (AT) is the most commonly used mechanism to improve the robustness of deep neural networks. Recently, a novel adversarial attack against intermediate layers exploits the extra fragility of adversarially trained networks to output incorrect predictions. The result implies the insufficiency in the searching space of the adversarial perturbation in adversarial training. To straighten out the reason for the effectiveness of the intermediate-layer attack, we interpret the forward propagation as the Clustering Effect, characterizing that the intermediate-layer representations of neural networks for samples i.i.d. to the training set with the same label are similar, and we theoretically prove the existence of Clustering Effect by corresponding Information Bottleneck Theory. We afterward observe that the intermediate-layer attack disobeys the clustering effect of the AT-trained model. Inspired by these significant observations, we propose a regularization method to extend the perturbation searching space during training, named sufficient adversarial training (SAT). We give a proven robustness bound of neural networks through rigorous mathematical proof. The experimental evaluations manifest the superiority of SAT over other state-of-the-art AT mechanisms in defending against adversarial attacks against both output and intermediate layers. Our code and Appendix can be found at https://github.com/clustering-effect/SAT.

Machine unlearning is an emerging task of removing the influence of selected training datapoints from a trained model upon data deletion requests, which echoes the widely enforced data regulations mandating the Right to be Forgotten. Many unlearning methods have been proposed recently, achieving significant efficiency gains over the naive baseline of retraining from scratch. However, existing methods focus exclusively on unlearning from standard training models and do not apply to adversarial training models (ATMs) despite their popularity as effective defenses against adversarial examples. During adversarial training, the training data are involved in not only an outer loop for minimizing the training loss, but also an inner loop for generating the adversarial perturbation. Such bi-level optimization greatly complicates the influence measure for the data to be deleted and renders the unlearning more challenging than standard model training with single-level optimization. This paper proposes a new approach called MUter for unlearning from ATMs. We derive a closed-form unlearning step underpinned by a total Hessian-related data influence measure, while existing methods can mis-capture the data influence associated with the indirect Hessian part. We further alleviate the computational cost by introducing a series of approximations and conversions to avoid the most computationally demanding parts of Hessian inversions. The efficiency and effectiveness of MUter have been validated through experiments on four datasets using both linear and neural network models.

In recent years, deep neural networks(DNNs) have relied on an increasing amount of training samples as the premise of the deployment for real-world scenarios. This gives rise to backdoor attacks, where a small fraction of poisoned data is inserted into the training dataset to manipulate the predictions of DNNs when presented with backdoor inputs. Backdoor attacks pose serious security threats during the prediction stage of DNNs. As a result, there is growing research attention to defend against backdoor attacks. This paper proposes Activation Clipping and Quantizing (ACQ), a novel backdoor elimination module via transforming the intermediate-layer output of DNNs during forward propagation by embedding Clipper and Quantizer into the backdoored DNNs. ACQ is motivated by the observation that the backdoored DNNs always output abnormally large or small intermediate-layer activations when presented with backdoored samples, eventually leading to the malicious prediction of backdoored DNNs. ACQ modifies backdoored DNNs to keep the intermediate-layer activations in a proper domain and align the forward propagation of backdoored samples with that of clean samples. Besides, we highlight that ACQ has the ability to eliminate the backdoor of DNNs in few-shot even zero-shot scenarios, which requires much fewer or even no clean samples for the backdoor elimination stage than existing approaches. Experiments demonstrate the effectiveness and robustness of ACQ against various attacks and tasks compared to existing methods. Our code and Appendix can be found in https://github.com/Backdoor-defense/ACQ

Pre-trained models have been widely adopted in deep learning development, benefiting the fine-tuning of downstream user-specific tasks with enormous computation saving. However, backdoor attacks pose severe security threat to the subsequent models built upon compromised pre-trained models, which call for effective countermeasures to mitigate the backdoor threat before deploying the victim models to safety-critical applications. This paper proposesPurifier : a novel backdoor mitigation framework for pre-trained models via suppressing anomaly activation.Purifier is motivated by the observation that, for backdoor triggers, anomaly activation patterns exist across different perspectives (e.g., channel-wise, cube-wise, and feature-wise), featuring different degrees of granularity. More importantly, choosing to suppress at the right granularity is vital to robustness and accuracy. To this end,Purifier is capable of defending against diverse types of backdoor triggers without any prior knowledge of the backdoor attacks, meanwhile featuring a convenient and flexible characteristic during deployment, i.e., plug-and-play-able. The extensive experimental results show, against a series of state-of-the-art mainstream attacks, thatPurifier performs better in terms of both defense effectiveness and model inference accuracy on clean examples than the state-of-the-art methods. Our code and Appendix can be found in \urlgithub.com/RUIYUN-ML/Purifier.

Industrial Internet of Things (IIoT) is significant of building powerful industrial systems and applications. Deep learning has provided a promising opportunity to extract useful knowledge by utilizing vast amounts of data in IIoT. However, lacking of massive public datasets will lead to low performance and overfitting of the learned model. Therefore, the federated deep learning over distributed datasets has been proposed. Whereas, it inevitably introduces some new security challenges, i.e., disclosing participant's data privacy. However, existing methods cannot guarantee each participant's data privacy in a learning group. In this article, we propose two privacy-preserving asynchronous deep learning schemes [privacy-preserving and asynchronous deep learning via re-encryption (DeepPAR) and dynamic privacy-preserving and asynchronous deep learning (DeepDPA)]. Compared to the state-of-the-art work, DeepPAR protects each participant's input privacy while preserving dynamic update secrecy inherently. Meanwhile, DeepDPA enables to guarantee backward secrecy of group participants in a lightweight manner. Security analysis and performance evaluations on real dataset show that our proposed schemes are secure, efficient and effective.

With the advent of artificial intelligence, machine learning has been well explored and extensively applied into numerous fields, such as pattern recognition, image processing and cloud computing. Very recently, machine learning hosted in a cloud service has gained more attentions due to the benefits from the outsourcing paradigm. Based on cloud-aided computation techniques, the heavy computation tasks involved in machine learning process can be off-loaded into the cloud server in a pay-per-use manner, whereas outsourcing large-scale collection of sensitive data risks privacy leakage since the cloud server is semi-honest. Therefore, privacy preservation for the client and verification for the returned results become two challenges to be dealt with. In this paper, we focus on designing a novel privacy-preserving single-layer perceptron training scheme which supports batch patterns training and verification for the training results on the client side. In addition, adopting classical secure two-party computation method, we design a novel lightweight privacy-preserving predictive algorithm. Both two participants learns nothing about other’s inputs, and the calculation result is only known by one party. Detailed security analysis shows that the proposed scheme can achieve the desired security properties. We also demonstrate the efficiency of our scheme by providing the experimental evaluation on two different real datasets.