Jialiang Guo, Fucheng Xiong, Xu He, Haodong Zhao, Xingyang li, Ke Zeng, Xunliang Cai. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Implicit discourse relation recognition (IDRR) is faced with a domain dilemma. Recent studies have achieved breakthroughs in standard datasets, while they are not appropriate in domains with insufficient data, such as bio-medicine. In this paper, we treat this problem as a cross-domain IDRR task, which transfers knowledge from the source domain to improve the understanding of the target domain. However, cross-domain IDRR is supervised and suffers from striking domain gaps, and general domain adaptation methods are not applicable. Therefore, we propose a Conditional-Balanced Adversarial Delta Tuning (CBADT) framework, which 1) leverages delta tuning to mine the dense domain-specific knowledge in low-resource scenarios; 2) builds a conditional adversarial verbalizer to inject domain-invariant knowledge into soft prompts; 3) proposes a domain label fusion method to balance domain-specific label words and fit the hybrid features from two domains. Experiments on both Chinese and English corpora demonstrate the transferability of our model.

Multi-Modal Large Language Models (MLLMs) have exhibited remarkable performance on various vision-language tasks such as Visual Question Answering (VQA). Despite accumulating evidence of privacy concerns associated with task-relevant content, it remains unclear whether MLLMs inadvertently memorize private content that is entirely irrelevant to the training tasks. In this paper, we investigate how randomly generated task-irrelevant private content can become spuriously correlated with downstream objectives due to partial mini-batch training dynamics, thus causing inadvertent memorization. Concretely, we randomly generate task-irrelevant watermarks into VQA fine-tuning images at varying probabilities and propose a novel probing framework to determine whether MLLMs have inadvertently encoded such content. Our experiments reveal that MLLMs exhibit notably different training behaviors in partial mini-batch settings with task-irrelevant watermarks embedded. Furthermore, through layer-wise probing, we demonstrate that MLLMs trigger distinct representational patterns when encountering previously seen task-irrelevant knowledge, even if this knowledge does not influence their output during prompting. Our code is available at https://github.com/illusionhi/ProbingPrivacy.

ABSTRACT Deep Neural Networks (DNNs) have recently achieved remarkable success in the field of network security, and deep neural network‐based Intrusion Detection Systems (IDSs) are able to automatically learn potential patterns from network traffic and detect malicious traffic. However, IDSs are vulnerable to adversarial network traffic, which weakens their defense capabilities. Therefore, IDSs need high‐quality adversarial traffic for adversarial training to improve robustness. Existing adversarial sample generation algorithms are mainly focused on image‐based applications, overlooking the textual features of network traffic and inadequately considering perturbation magnitude. In response to these challenges, this paper proposes a Loss‐Guided Dynamic Step Adversarial Attack for Network Intrusion Detection Systems (LGDA) and a Discrete Adversarial Rounding (DAR) scheme. The LGDA analyzes model loss trend and dynamically adjusts perturbation step size to balance attack effectiveness and perturbation magnitude. The DAR targets discrete text features in network traffic, using integer encoding and rounding to allow partial preservation of perturbation direction, ensuring compatibility between existing adversarial attack algorithms and network traffic. Compared with state‐of‐the‐art adversarial attacks, experimental results on the NSL‐KDD, UNSW‐NB15, and CIC‐IDS2017 datasets using five models (LR, MLP, CNN, LSTM, and Transformer) show that the LGDA improves the attack success rate by 12.9% and reduces the perturbation magnitude by 19.1%. Additionally, the DAR scheme can be incorporated into any adversarial attack algorithm in the field of network security, increasing adversarial sample's attack success rate by 6.9%.

Language models (LMs) are becoming increasingly popular in real-world applications. Outsourcing model training and data hosting to third-party platforms has become a standard method for reducing costs. In such a situation, the attacker can manipulate the training process or data to inject a backdoor into models. Backdoor attacks are a serious threat where malicious behavior is activated when triggers are present; otherwise, the model operates normally. However, there is still no systematic and comprehensive review of LMs from the attacker's capabilities and purposes on different backdoor attack surfaces. Moreover, there is a shortage of analysis and comparison of the diverse emerging backdoor countermeasures. Therefore, this work aims to provide the natural language processing (NLP) community with a timely review of backdoor attacks and countermeasures. According to the attackers' capability and affected stage of the LMs, the attack surfaces are formalized into four categorizations: attacking the pretrained model with fine-tuning (APMF) or parameter-efficient fine-tuning (PEFT), attacking the final model with training (AFMT), and attacking large language model (ALLM). Thus, attacks under each categorization are combed. The countermeasures are categorized into two general classes: sample inspection and model inspection. Thus, we review countermeasures and analyze their advantages and disadvantages. Also, we summarize the benchmark datasets and provide comparable evaluations for representative attacks and defenses. Drawing the insights from the review, we point out the crucial areas for future research on the backdoor, especially soliciting more efficient and practical countermeasures.

To trace the copyright of deep neural networks, an owner can embed its identity information into its model as a watermark. The capacity of the watermark quantify the maximal volume of information that can be verified from the watermarked model. Current studies on capacity focus on the ownership verification accuracy under ordinary removal attacks and fail to capture the relationship between robustness and fidelity. This paper studies the capacity of deep neural network watermarks from an information theoretical perspective. We propose a new definition of deep neural network watermark capacity analogous to channel capacity, analyze its properties, and design an algorithm that yields a tight estimation of its upper bound under adversarial overwriting. We also propose a universal non-invasive method to secure the transmission of the identity message beyond capacity by multiple rounds of ownership verification. Our observations provide evidence for neural network owners and defenders that are curious about the tradeoff between the integrity of their ownership and the performance degradation of their products.

Backdoor attacks pose a serious security threat for natural language processing (NLP). Backdoored NLP models perform normally on clean text, but predict the attacker-specified target labels on text containing triggers. Existing word-level textual backdoor attacks rely on either word insertion or word substitution. Word-insertion backdoor attacks can be easily detected by simple backdoor defenses. Meanwhile, word-substitution backdoor attacks tend to substantially degrade the fluency and semantic consistency of the poisoned text. In this paper, we propose a more natural word substitution method to implement covert textual backdoor attacks. Specifically, we combine three different ways to construct a diverse synonym thesaurus for clean text. We then train a learnable word selector for producing poisoned text using a composite loss function of poison and fidelity terms. This enables automated selection of minimal critical word substitutions necessary to induce the backdoor. Experiments demonstrate our method achieves high attack performance with less impact on fluency and semantics. We hope this work can raise awareness regarding the threat of subtle, fluent word substitution attacks.

Abstract Transfer learning has made significant advancements, however, the issue of overfitting continues to pose a major challenge. Data augmentation has emerged as a highly promising technique to counteract this challenge. Current data augmentation methods are fixed in nature, requiring manual determination of the appropriate intensity prior to the training process. However, this entails substantial computational costs. Additionally, as the model approaches convergence, static data augmentation strategies can become suboptimal. In this paper, we introduce the concept of Dynamic Data Augmentation (DynamicAug), a method that autonomously adjusts the intensity of data augmentation, taking into account the convergence state of the model. During each iteration of the model’s forward pass, we utilize a Gaussian distribution based sampler to stochastically sample the current intensity of data augmentation. To ensure that the sampled intensity is aligned with the convergence state of the model, we introduce a learnable expectation to the sampler and update the expectation iteratively. In order to assess the convergence status of the model, we introduce a novel loss function called the convergence loss. Through extensive experiments conducted over 27 vision datasets, we have demonstrated that DynamicAug can significantly enhance the performance of existing transfer learning methods.

The huge training overhead, considerable commercial value, and various potential security risks make it urgent to protect the intellectual property (IP) of Deep Neural Networks (DNNs). DNN watermarking has become a plausible method to meet this need. However, most of the existing watermarking schemes focus on image classification tasks. The schemes designed for the textual domain lack security and reliability. Moreover, how to protect the IP of widely-used pre-trained language models (PLMs) remains a blank. To fill these gaps, we propose PLMmark, the first secure and robust black-box watermarking framework for PLMs. It consists of three phases: (1) In order to generate watermarks that contain owners’ identity information, we propose a novel encoding method to establish a strong link between a digital signature and trigger words by leveraging the original vocabulary tables of PLMs. Combining this with public key cryptography ensures the security of our scheme. (2) To embed robust, task-agnostic, and highly transferable watermarks in PLMs, we introduce a supervised contrastive loss to deviate the output representations of trigger sets from that of clean samples. In this way, the watermarked models will respond to the trigger sets anomaly and thus can identify the ownership. (3) To make the model ownership verification results reliable, we perform double verification, which guarantees the unforgeability of ownership. Extensive experiments on text classification tasks demonstrate that the embedded watermark can transfer to all the downstream tasks and can be effectively extracted and verified. The watermarking scheme is robust to watermark removing attacks (fine-pruning and re-initializing) and is secure enough to resist forgery attacks.

Multi-level implicit discourse relation recognition (MIDRR) aims at identifying hierarchical discourse relations among arguments.Previous methods achieve the promotion through finetuning PLMs.However, due to the data scarcity and the task gap, the pre-trained feature space cannot be accurately tuned to the task-specific space, which even aggravates the collapse of the vanilla space.Besides, the comprehension of hierarchical semantics for MIDRR makes the conversion much harder.In this paper, we propose a prompt-based Parameter-Efficient Multilevel IDRR (PEMI) framework to solve the above problems.First, we leverage parameterefficient prompt tuning to drive the inputted arguments to match the pre-trained space and realize the approximation with few parameters.Furthermore, we propose a hierarchical label refining (HLR) method for the prompt verbalizer to deeply integrate hierarchical guidance into the prompt tuning.Finally, our model achieves comparable results on PDTB 2.0 and 3.0 using about 0.1% trainable parameters compared with baselines and the visualization demonstrates the effectiveness of our HLR method.

Federated learning (FL) has enabled global model training on decentralized data in a privacy-preserving way. However, for tasks that utilize pre-trained language models (PLMs) with massive parameters, there are considerable communication costs. Prompt tuning, which tunes soft prompts without modifying PLMs, has achieved excellent performance as a new learning paradigm. In this paper, we want to combine these methods and explore the effect of prompt tuning under FL. We propose "FedPrompt" studying prompt tuning in a model split aggregation way using FL, and prove that split aggregation greatly reduces the communication cost, only 0.01% of the PLMs’ parameters, with little decrease on accuracy both on IID and Non-IID data distribution. We further conduct backdoor attacks by data poisoning on FedPrompt. Experiments show that attack achieve a quite low attack success rate and can not inject backdoor effectively, proving the robustness of FedPrompt.

One-shot Neural architecture search (One-shot NAS) has been proposed as a time-efficient approach to obtain optimal subnet architectures and weights under different complexity cases by training only once. However, the subnet performance obtained by weight sharing is often inferior to the performance achieved by retraining. In this paper, we investigate the performance gap and attribute it to the use of uniform sampling, which is a common approach in supernet training. Uniform sampling concentrates training resources on subnets with intermediate computational resources, which are sampled with high probability. However, subnets with different complexity regions require different optimal training strategies for optimal performance.To address the problem of uniform sampling, we propose ShiftNAS, a method that can adjust the sampling probability based on the complexity of subnets. We achieve this by evaluating the performance variation of subnets with different complexity and designing an architecture generator that can accurately and efficiently provide subnets with the desired complexity. Both the sampling probability and the architecture generator can be trained end-to-end in a gradient-based manner. With ShiftNAS, we can directly obtain the optimal model architecture and parameters for a given computational complexity. We evaluate our approach on multiple visual network models, including convolutional neural networks (CNNs) and vision transformers (ViTs), and demonstrate that ShiftNAS is model-agnostic. Experimental results on ImageNet show that ShiftNAS can improve the performance of one-shot NAS without additional consumption. Source codes are available at GitHub.

Implicit discourse relation recognition aims to understand and infer the relations between discourse arguments, which is a classification task. Existing models mostly model the interaction of discourse arguments from the perspective of a single relation, which may fail to learn the complicated interaction patterns between two arguments under different discourse relations. In this paper, we propose a novel neural approach based on capsule networks with dynamic interactive routing mechanism, named Interactive Capsule Networks (ICN) model to address the issues. It dynamically maps argument feature capsules to discourse relation capsules in an iterative refinement manner, which can mine the discourse relation-indicative semantic clues of argument features. Then, the designed dynamic interactive routing mechanism captures the more comprehensive argument interaction from the perspective of multiple relations. The experimental results on the Penn Discourse TreeBank (PDTB) demonstrate the effectiveness of our proposed ICN model.

Speaker verification has been widely used in many authentication scenarios.However, training models for speaker verification requires large amounts of data and computing power, so users often use untrustworthy third-party data or deploy thirdparty models directly, which may create security risks.In this paper, we propose a backdoor attack for the above scenario.Specifically, for the Siamese network in the speaker verification system, we try to implant a universal identity in the model that can simulate any enrolled speaker and pass the verification.So the attacker does not need to know the victim, which makes the attack more flexible and stealthy.In addition, we design and compare three ways of selecting attacker utterances and two ways of poisoned training for the GE2E loss function in different scenarios.The results on the TIMIT and Voxceleb1 datasets show that our approach can achieve a high attack success rate while guaranteeing the normal verification accuracy.Our work reveals the vulnerability of the speaker verification system and provides a new perspective to further improve the robustness of the system.

In recent years, knowledge distillation has been widely used in the field of deep learning in order to reduce the model size and save time and space. The student-teacher paradigm is a framework for knowledge distillation, and knowledge distillation proposed to minimize the KL divergence between the probabilistic outputs of a teacher and student network. However, apart from the probabilistic outputs, there are much valuable information contained in the middle layers of the teacher network. As for NLP tasks, the hidden vectors from different layers of a model have different semantic information, but the vectors' dimension of the student network is different from that of the teacher network in many cases, which makes hidden layer distillation hard to be performed directly. We propose to simply use a transition matrix to project the student's vector to a space of the same dimension as the teacher's vector, and we theoretically prove the effectiveness of this method. Our analysis shows how the transition matrix preserve important semantic information, which is closely related to the vector's characteristic in Euclidean space. We provide a geometric method for the interpretability of shared knowledge space for student-teacher architectures. Our experiments show that this method can significantly improve the performance of a small model in different tasks with different models.

This article proposes an uncertainty analysis method for evaluating the reliability of a system and calculating the relative importance of the system inputs when the probabilities of basic events are characterized by intervals. The reliability requirement or the safety criterion is first considered in the system reliability assessment. A nonprobabilistic reliability index, called R-index, is defined to evaluate the reliability of the system. Two importance measure indices, called location effect and size effect, are introduced for estimating the influence of the probability uncertainties of basic events on the R-index, thus identifying influential and noninfluential basic events and comparing their relative importance. Then, the proposed method is applied to an illustrative fault tree example and the fault tree analysis of unilateral asymmetric movement of an aircraft flap mechanism, thereby explaining the significance of this method. The results show that the proposed method provides a viable tool for system reliability and importance measure analysis when probability intervals are involved.