Yuetai Li, Zhangchen Xu, Fengqing Jiang, Bhaskar Ramasubramanian, Luyao Niu, Bill Yuchen Lin, Xiang Yue, Radha Poovendran. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

The Controller Area Network (CAN) protocol is the most widely used standard in the automotive industry for in-vehicle networks. However, the CAN protocol lacks essential security features such as encryption and message authentication. Absence of such security features has been shown to make the vehicle network vulnerable to exploits by an adversary. Although multiple types of intrusion detection systems (IDS) have been developed for CAN, it can be difficult to deploy them in real-time with low latency. Further, many of these IDSs are unable to isolate specific CAN frames on which an attack has been mounted, which makes it challenging to design defense mechanisms. In this paper, we develop CANLP, a Natural Language Processing (NLP)-based intrusion detection system to determine whether each transmitted message originated from a legitimate ECU or an adversary. CANLP uses Term Frequency-Inverse Document Frequency (TF-IDF), a NLP technique to discern complex features associated with CAN data and trains machine learning models to identify three types of attacks- fuzzing, spoofing, and masquerade. When an attack is detected, CANLP identifies the malicious CAN frame, which is important for developing resilient systems. Extensive experiments on 4 publicly available vehicle network datasets (which represent data collected from over three vehicle makes and four models) show that CANLP performs attack classification with high F1-scores of 0.9974. We also show that CANLP can be deployed for attack detection on resource-constrained hardware through implementation using RaspberryPi and experiments on a testbed with latency as low as$\lt \text{0.05}~ms$, making it suitable for real-world automotive applications such as fleet monitoring within the same vehicle class.

Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, Radha Poovendran. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Transfer learning (TL) has been demonstrated to improve DNN model performance when faced with a scarcity of training samples. However, the suitability of TL as a solution to reduce vulnerability of overfitted DNNs to privacy attacks is unexplored. A class of privacy attacks called membership inference attacks (MIAs) aim to determine whether a given sample belongs to the training dataset (member) or not (nonmember). We introduce Double-Dip to investigate the use of TL (Stage-1) combined with randomization (Stage-2) to thwart MIAs on overfitted DNNs without degrading classification accuracy. Our study examines roles of shared feature space and parameter values between source and target models, number of frozen layers, and complexity of pretrained models. Our preliminary evaluations of Double-Dip demonstrate that Stage-1 reduces adversary success while also significantly increasing classification accuracy of nonmembers against an adversary attempting to carry out SOTA label-only MIAs. After Stage-2, success of an adversary carrying out a label-only MIA is further reduced to near 50%, bringing it closer to a random guess and showing the effectiveness of Double-Dip. Stage-2 of Double-Dip also achieves lower ASR and higher classification accuracy than regularization and differential privacy-based methods.

Deep Neural Network (DNN) models are vulnerable to Trojan attacks, wherein a Trojaned DNN will mispredict trigger-embedded inputs as malicious targets, while outputs for clean inputs remain unaffected. Output-based Trojaned model detectors, which analyze outputs of DNNs to perturbed inputs have emerged as a promising approach for identifying Trojaned DNN models. At present, these SOTA detectors assume that the adversary is (i) static and (ii) does not have prior knowledge about deployed detection mechanisms.

Yuetai Li, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Dinuka Sahabandu, Bhaskar Ramasubramanian, Radha Poovendran. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

Large language models (LLMs) are shown to benefit from chain-of-thought (COT) prompting, particularly when tackling tasks that require systematic reasoning processes. On the other hand, COT prompting also poses new vulnerabilities in the form of backdoor attacks, wherein the model will output unintended malicious content under specific backdoor-triggered conditions during inference. Traditional methods for launching backdoor attacks involve either contaminating the training dataset with backdoored instances or directly manipulating the model parameters during deployment. However, these approaches are not practical for commercial LLMs that typically operate via API access. In this paper, we propose BadChain, the first backdoor attack against LLMs employing COT prompting, which does not require access to the training dataset or model parameters and imposes low computational overhead. BadChain leverages the inherent reasoning capabilities of LLMs by inserting a backdoor reasoning step into the sequence of reasoning steps of the model output, thereby altering the final response when a backdoor trigger is embedded in the query prompt. In particular, a subset of demonstrations will be manipulated to incorporate a backdoor reasoning step in COT prompting. Consequently, given any query prompt containing the backdoor trigger, the LLM will be misled to output unintended content. Empirically, we show the effectiveness of BadChain for two COT strategies across four LLMs (Llama2, GPT-3.5, PaLM2, and GPT-4) and six complex benchmark tasks encompassing arithmetic, commonsense, and symbolic reasoning. We show that the baseline backdoor attacks designed for simpler tasks such as semantic classification will fail on these complicated tasks. In addition, our findings reveal that LLMs endowed with stronger reasoning capabilities exhibit higher susceptibility to BadChain, exemplified by a high average attack success rate of 97.0\% across the six benchmark tasks on GPT-4. We also demonstrate the interpretability of BadChain by showing that the relationship between the trigger and the backdoor reasoning step can be well-explained based on the output of the backdoored model. Finally, we propose two defenses based on shuffling and demonstrate their overall ineffectiveness against BadChain. Therefore, BadChain remains a severe threat to LLMs, underscoring the urgency for the development of robust and effective future defenses.

Reinforcement learning (RL) is a well-studied framework to solve complex decision-making problems in unknown environments. The actor-critic model in RL facilitates autonomy transfer by allowing agents to iteratively update their policies using ongoing dynamic evaluations of value functions via a critic. In this paper, we examine the impact of using different pretrained critics on the performance of actor-critic algorithms. First, in any single given environment, we show that a pretrained critic can be effective in reducing the duration of an initial training phase, thereby accelerating convergence by a factor of up to 2×. In this setting, we identify the critical range of the number of episodes for which a critic will need to be trained in order for it to be an effective pretrained critic. Second, we show that a critic trained in one environment enables transfer of autonomy by aiding learning of behaviors in a different, yet related environment. We carry out extensive experiments on a bipedal locomotion task in the MuJoCo physics engine to verify our hypotheses. Our results in this paper mark the first step towards demonstrating the role and impact of pretrained critics to achieve rapid autonomy transfer for complex reinforcement learning tasks while minimizing costs associated with retraining in new environments.

Qifan Lu, Bhaskar Ramasubramanian, Radha Poovendran. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

The data used to train deep neural network (DNN) models in applications such as healthcare and finance typically contain sensitive information. A DNN model may suffer from overfitting– it will perform very well on samples seen during training, and poorly on samples not seen during training. Overfitted models have been shown to be susceptible to query-based attacks such as membership inference attacks (MIAs). MIAs aim to determine whether a sample belongs to the dataset used to train a classifier (members) or not (nonmembers). Recently, a new class of label-based MIAs (LAB MIAs) was proposed, where an adversary was only required to have knowledge of predicted labels of samples. LAB MIAs used the insight that member samples were typically located farther away from a classification decision boundary than nonmembers, and were shown to be highly effective across multiple datasets. Developing a defense against an adversary carrying out a LAB MIA on DNN models that cannot be retrained remains an open problem.

Machine learning models that use deep neural networks (DNNs) are vulnerable to backdoor attacks. An adversary carrying out a backdoor attack embeds a predefined perturbation called a trigger into a small subset of input samples and trains the DNN such that the presence of the trigger in the input results in an adversary-desired output class. Such adversarial retraining however needs to ensure that outputs for inputs without the trigger remain unaffected and provide high classification accuracy on clean samples. Existing defenses against backdoor attacks are computationally expensive, and their success has been demonstrated primarily on image-based inputs. The increasing popularity of deploying pretrained DNNs to reduce costs of re/training large models makes defense mechanisms that aim to detect 'suspicious' input samples preferable.

The opinions of members of a population are influenced by opinions of their peers, their own predispositions, and information from external sources via one or more information channels (e.g., news, social media). Due to individual cognitive biases, the perceptual impact of and importance assigned by agents to information on each channel can be different. In this paper, we propose a model of opinion evolution that uses prospect theory to represent perception of information from the external source along each channel. Our prospect-theoretic model reflects traits observed in humans such as loss aversion, assigning inflated (deflated) values to low (high) probability events, and evaluating outcomes relative to an individually known reference point. We consider the problem of determining information dissemination strategies for the external source to adopt in order to drive opinions of individuals towards a desired value. However, computing a strategy faces a challenge that agents' initial predispositions and functions characterizing their perceptions of information disseminated might be unknown. We overcome this challenge by using Gaussian process learning to estimate these unknown parameters. When the external source sends information over multiple channels, the problem of jointly selecting optimal dissemination strategies is in general, combinatorial. We prove that this problem is submodular, and design near-optimal dissemination algorithms. We evaluate our model on three different widely used large graphs that represent real-world social interactions. Our results indicate that the external source can effectively drive opinions towards a desired value when using prospect-theory based dissemination strategies.

Federated learning (FL) provides a distributed training paradigm where multiple clients can jointly train a global model without sharing their local data. However, recent studies have shown that FL offers an additional surface for backdoor attacks. For instance, an attacker can compromise a subset of clients and thus corrupt the global model to misclassify an input with a backdoor trigger as the adversarial target. Existing defenses for FL against backdoor attacks usually detect and exclude the corrupted information from the compromised clients based on a static attacker model. However, such defenses are inadequate against dynamic attackers who strategically adapt their attack strategies. To bridge this gap, we model the strategic interactions between the defender and dynamic attackers as a minimax game. Based on the analysis of the game, we design an interactive defense mechanism FedGame. We prove that under mild assumptions, the global model trained with FedGame under backdoor attacks is close to that trained without attacks. Empirically, we compare FedGame with multiple state-of-the-art baselines on several benchmark datasets under various attacks. We show that FedGame can effectively defend against strategic attackers and achieves significantly higher robustness than baselines. Our code is available at: https://github.com/AI-secure/FedGame.

Fifth-generation (5G) cellular communication networks are being deployed on applications beyond mobile devices, including vehicular networks and industry automation. Despite their increasing popularity, 5G networks, as defined by the Third Generation Partnership Project (3GPP), have been shown to be vulnerable against fake base station (FBS) attacks. An adversary carrying out an FBS attack emulates a legitimate base station by setting up a rogue base station. This enables the adversary to control the connection of any user equipment that (inadvertently) connects with the rogue base station. Such an adversary can gather sensitive information belonging to the user. While there is a large body of work focused on the development of tools to detect FBSs, the user equipment will continue to remain vulnerable to an FBS attack. In this paper, we propose BARON, a defense methodology to enable user equipment to determine whether a target base station that it is connecting to is legitimate or rogue. BARON accomplishes this by ensuring that the user receives an authentication token from the target base station which can be computed only by a legitimate and trusted entity. As a consequence, receiving such an authentication token from a base station ensures legitimacy of the base station. We evaluate BARON through extensive experiments on the handover process between base stations in 5G networks. Our experimental results show that BARON introduces an overhead of less than 1% during handover completion, which is 10000× lower than the overhead reported by a state-of-the-art method. BARON is also effective in thwarting an FBS attack and quickly recovering connection to a legitimate base station.