Tianyu Du

dblp:128/2982 · DBLP profile ↗
← Back
59ranked-venue papers
8as first author
48since 2021 · last 2026
—ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 29 · 2 first-author · 29 since 2021Graphics, computer vision, multimedia, augmented reality and games · 14 · 14 since 2021Security and privacy · 13 · 2 first-author · 8 since 2021Databases, data management, data science and information retrieval · 6 · 1 first-author · 5 since 2021Computer networks · 4 · 2 first-author · 2 since 2021Theory of computation · 2 · 1 since 2021Software engineering, systems software and programming languages · 1 · 1 since 2021Applied, interdisciplinary, general and emerging computing · 1 · 1 since 2021
YearPublicationVenuePosition
2026 LSHFed: Robust and Communication-Efficient Federated Learning with Locally-Sensitive Hashing Gradient Mapping
abstract
Federated learning (FL) enables collaborative model training across distributed nodes without exposing raw data, but its decentralized nature makes it vulnerable in trust-deficient environments. Inference attacks may recover sensitive information from gradient updates, while poisoning attacks can degrade model performance or induce malicious behaviors. Existing defenses often suffer from high communication and computation costs, or limited detection precision. To address these issues, we propose LSHFed, a robust and communication-efficient FL framework that simultaneously enhances aggregation robustness and privacy preservation. At its core, LSHFed incorporates LSHGM, a novel gradient verification mechanism that projects high-dimensional gradients into compact binary representations via multi-hyperplane locality-sensitive hashing. This enables accurate detection and filtering of malicious gradients using only their irreversible hash forms, thus mitigating privacy leakage risks and substantially reducing transmission overhead. Extensive experiments demonstrate that LSHFed maintains high model performance even when up to 50% of participants are collusive adversaries, while achieving up to a 1000× reduction in gradient verification communication compared to full-gradient methods.
Guanjie Cheng, Mengzhen Yang, Xinkui Zhao, Shuyi Yu, Tianyu Du, Mengying Zhu, Shuiguang Deng
AAAI5
2026 FedAU2: Attribute Unlearning for User-Level Federated Recommender Systems with Adaptive and Robust Adversarial Training
abstract
Federated Recommender Systems (FedRecs) leverage federated learning to protect user privacy by retaining data locally. However, user embeddings in FedRecs often encode sensitive attribute information, rendering them vulnerable to attribute inference attacks. Attribute unlearning has emerged as a promising approach to mitigate this issue. In this paper, we focus on user-level FedRecs, which is a more practical yet challenging setting compared to group-level FedRecs. Adversarial training emerges as the most feasible approach within this context. We identify two key challenges in implementing adversarial training-based attribute unlearning for user-level FedRecs: i) mitigating training instability caused by user data heterogeneity, and ii) preventing attribute information leakage through gradients. To address these challenges, we propose FedAU2, an attribute unlearning method for user-level FedRecs. For CH1, we propose a adaptive adversarial training strategy, where the training dynamics are adjusted in response to local optimization behavior. For CH2, we propose a dual-stochastic variational autoencoder to perturb the adversarial model, effectively preventing gradient-based information leakage. Extensive experiments on three real-world datasets demonstrate that our proposed FedAU2 achieves superior performance in unlearning effectiveness and recommendation performance compared to existing baselines.
Yuyuan Li 0001, Junjie Fang, Fengyuan Yu 0001, Xichun Sheng, Tianyu Du, Xuyang Teng, Shaowei Jiang, Linbo Jiang, Jianan Lin 0003, Chaochao Chen 0001
AAAI5
2026 DP-GenG: Differentially Private Dataset Distillation Guided by DP-Generated Data
abstract
Dataset distillation (DD) compresses large datasets into smaller ones while preserving the performance of models trained on them. Although DD is often assumed to enhance data privacy by aggregating over individual examples, recent studies reveal that standard DD can still leak sensitive information from the original dataset due to the lack of formal privacy guarantees. Existing differentially private (DP)-DD methods attempt to mitigate this risk by injecting noise into the distillation process. However, they often fail to fully leverage the original dataset, resulting in degraded realism and utility. This paper introduces DP-GENG, a novel framework that addresses the key limitations of current DP-DD by leveraging DP-generated data. Specifically, DP-GENG initializes the distilled dataset with DP-generated data to enhance realism. Then, generated data refines the DP-feature matching technique to distill the original dataset under a small privacy budget, and trains an expert model to align the distilled examples with their class distribution. Furthermore, we design a privacy budget allocation strategy to determine budget consumption across DP components and provide a theoretical analysis of the overall privacy guarantees. Extensive experiments show that DP-GENG significantly outperforms state-of-the-art DP-DD methods in terms of both dataset utility and robustness against membership inference attacks, establishing a new paradigm for privacy-preserving dataset distillation.
Jinghuai Zhang, Shijie Jiang, Chunyi Zhou 0001, Yuyuan Li 0001, Mengying Zhu, Tianyu Du
AAAI8
2026 Bridging the Copyright Gap: Do Large Vision-Language Models Recognize and Respect Copyrighted Content?
abstract
Large vision-language models (LVLMs) have achieved remarkable advancements in multimodal reasoning tasks. However, their widespread accessibility raises critical concerns about potential copyright infringement. Will LVLMs accurately recognize and comply with copyright regulations when encountering copyrighted content (i.e., user input, retrieved documents) in the context? Failure to comply with copyright regulations may lead to serious legal and ethical consequences, particularly when LVLMs generate responses based on copyrighted materials (e.g., retrieved book experts, news reports). In this paper, we present a comprehensive evaluation of various LVLMs, examining how they handle copyrighted content – such as book excerpts, news articles, music lyrics, and code documentation when they are presented as visual inputs. To systematically measure copyright compliance, we introduce a large-scale benchmark dataset comprising 50,000 multimodal query-content pairs designed to evaluate how effectively LVLMs handle queries that could lead to copyright infringement. Given that real-world copyrighted content may or may not include a copyright notice, the dataset includes query-content pairs in two distinct scenarios: with and without a copyright notice. For the former, we extensively cover four types of copyright notices to account for different cases. Our evaluation reveals that even state-of-the-art closed-source LVLMs exhibit significant deficiencies in recognizing and respecting the copyrighted content, even when presented with the copyright notice. To solve this limitation, we introduce a novel tool-augmented defense framework for copyright compliance, which reduces infringement risks in all scenarios. Our findings underscore the importance of developing copyright-aware LVLMs to ensure the responsible and lawful use of copyrighted content.
Naen Xu, Jinghuai Zhang, Changjiang Li, Hengyu An, Chunyi Zhou 0001, Jun Wang 0001, Yuyuan Li 0001, Tianyu Du, Shouling Ji
AAAI9
2026 HogVul: Black-box Adversarial Code Generation Framework Against LM-based Vulnerability Detectors
abstract
Recent advances in software vulnerability detection have been driven by Language Model (LM)-based approaches. However, these models remain vulnerable to adversarial attacks that exploit lexical and syntax perturbations, allowing critical flaws to evade detection. Existing black-box attacks on LM-based vulnerability detectors primarily rely on isolated perturbation strategies, limiting their ability to efficiently explore the adversarial code space for optimal perturbations. To bridge this gap, we propose HogVul, a black-box adversarial code generation framework that integrates both lexical and syntax perturbations under a unified dual-channel optimization strategy driven by Particle Swarm Optimization (PSO). By systematically coordinating two-level perturbations, HogVul effectively expands the search space for adversarial examples, enhancing the attack efficacy. Extensive experiments on four benchmark datasets demonstrate that HogVul achieves an average attack success rate improvement of 26.05% over state-of-the-art baseline methods. These findings highlight the potential of hybrid optimization strategies in exposing model vulnerabilities.
Jingxiao Yang, Tianyu Du, Sun Bing, Xuhong Zhang 0002
AAAI3
2026 ACIArena: Toward Unified Evaluation for Agent Cascading Injection
abstract
Hengyu An, Minxi Li, Jinghuai Zhang, Naen Xu, Chunyi Zhou, Changjiang Li, Xiaogang Xu, Tianyu Du, Shouling Ji. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.
Hengyu An, Minxi Li, Jinghuai Zhang, Naen Xu, Chunyi Zhou 0001, Changjiang Li, Xiaogang Xu 0002, Tianyu Du, Shouling Ji
ACL (1)8
2026 "I See What You Did There": Can Large Vision-Language Models Understand Multimodal Puns?
abstract
Naen Xu, Jiayi Sheng, Changjiang Li, Chunyi Zhou, Yuyuan Li, Tianyu Du, Jun Wang, Zhihui Fu, Jinbao Li, Shouling Ji. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.
Naen Xu, Jiayi Sheng, Changjiang Li, Chunyi Zhou 0001, Yuyuan Li 0001, Tianyu Du, Zhihui Fu, Shouling Ji
ACL (1)6
2026 Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors
abstract
Rui Yin, Tianxu Han, Naen Xu, Changjiang Li, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Jinbao Li, Shouling Ji. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.
Tianxu Han, Naen Xu, Changjiang Li, Chunyi Zhou 0001, Jun Wang 0020, Zhihui Fu, Tianyu Du, Shouling Ji
ACL (1)9
2026 The Eminence in Shadow: Exploiting Feature Boundary Ambiguity for Robust Backdoor Attacks
abstract
Deep neural networks (DNNs) underpin critical applications yet remain vulnerable to backdoor attacks, typically reliant on heuristic brute-force methods. Despite significant empirical advancements in backdoor research, the lack of rigorous theoretical analysis limits understanding of underlying mechanisms, constraining attack predictability and adaptability. Therefore, we provide a theoretical analysis targeting backdoor attacks, focusing on how sparse decision boundaries enable disproportionate model manipulation. Based on this finding, we derive a closed-form ''ambiguous boundary region'' wherein negligible relabeled samples induce substantial misclassification. Influence function analysis further quantifies significant parameter shifts caused by these margin samples, with minimal impact on clean accuracy, formally grounding why such low poison rates suffice for efficacious attacks. Leveraging these insights, we propose Eminence, an explainable and robust black-box backdoor framework with provable theoretical guarantees and inherent stealth properties. Eminence optimizes a universal, visually subtle trigger that strategically exploits vulnerable decision boundaries and effectively achieves robust misclassification with exceptionally low poison rates (≤ 0.01%, compared to SOTA methods typically requiring ≥ 1 %). Comprehensive experiments validate our theoretical discussions and demonstrate the effectiveness of Eminence, confirming an exponential relationship between margin poisoning and adversarial boundary manipulation. Eminence maintains ≥ 90% attack success rate, exhibits negligible clean-accuracy loss, and demonstrates high transferability across diverse models, datasets and scenarios. Our code is available at https://github.com/NESA-Lab/Eminence
Zhou Feng, Chunyi Zhou 0001, Yuwen Pu, Tianyu Du, Jianhai Chen, Shouling Ji
KDD (1)5
2026 Content-Adaptive Implicit Neural Representations for Resolution-Agnostic Remote Sensing Watermarking
abstract
High-resolution remote sensing (RS) imagery frequently undergoes multi-scale tiling and resampling, introducing geometric and resolution variances that cause synchronization failure in conventional deep watermarking models reliant on fixed-grid assumptions. We propose an Implicit Neural Watermarking framework that reconceptualizes watermark embedding as a continuous, coordinate-conditioned mapping, inherently achieving resolution-agnostic extraction. Our framework introduces three core innovations: (i) Decentralized Coordinate Encoding (DCE), which ensures translation-robustness across arbitrary tiles via boundary-consistent periodic mapping; (ii) Content-Adaptive Modulation (CAM), which couples watermark synthesis with image semantics through a HyperNetwork to thwart template-estimation attacks; and (iii) Texture-Aware Optimization (TAO), which adaptively modulates embedding intensity based on local structural complexity to balance imperceptibility and decodability. Quantitative evaluations on Sentinel-2 datasets demonstrate superior visual fidelity and state-of-the-art robustness against typical RS-specific distortions, including geometric warping, scaling, and re-tiling, providing a theoretically grounded solution for practical RS copyright protection.
Minxi Li, Naen Xu, Hengyu An, Tianyu Du
ICMR4
2026 FraudShield: Knowledge Graph Empowered Defense for LLMs against Fraud Attacks
Naen Xu, Jinghuai Zhang, Chunyi Zhou 0001, Jun Wang 0020, Zhihui Fu, Tianyu Du, Zhaoxiang Wang, Shouling Ji
WWW7
2026 Hyper-network curvature: A new representation method for high-order brain network analysis
Tianyu Du, Qi Zhu 0001, Xuyun Wen, Jiashuang Huang, Xibei Yang, Daoqiang Zhang
Pattern Recognit.2
2026 Automatic Red Teaming LLM-Based Agents With Model Context Protocol Tools
abstract
The remarkable capability of large language models (LLMs) has led to the wide application of LLM-based agents in various domains. To standardize interactions between LLM-based agents and external resources, model context protocol (MCP) tools have become the de facto standard and are now widely integrated into these agents. However, the incorporation of MCP tools introduces the risk of tool poisoning attacks, in which malicious MCP tools can steer the behavior of LLM-based agents toward unintended outcomes. Although previous studies have identified such vulnerabilities, their red teaming approaches have largely remained at the proof-of-concept stage, leaving the automatic red teaming of LLM-based agents under the MCP tool poisoning paradigm an open question. To bridge this gap, we propose AutoMalTool, an automated red teaming framework for LLM-based agents by generating malicious MCP tools. Our extensive evaluation shows that AutoMalTool effectively generates malicious MCP tools capable of manipulating the behavior of mainstream LLM-based agents while evading current detection mechanisms, thereby revealing new security risks in these agents.
Changjiang Li, Tianyu Du, Shouling Ji
IEEE Trans. Inf. Forensics Secur.4
2026 G2uardFL: Safeguarding Federated Learning Against Backdoor Attacks via Attributed Client Graph Clustering
abstract
Federated Learning (FL) offers collaborative model training across multiple decentralized devices without the need to share data directly, enhancing privacy and data security. However, FL systems are susceptible to backdoor attacks, where malicious clients inject poisoned weights during training. Existing defenses, primarily based on anomaly detection, are prone to erroneous rejections of normal weights while accepting poisoned ones, largely due to shortcomings in quantifying similarities among client models. Furthermore, other defenses demonstrate effectiveness only when dealing with a limited number of malicious clients, typically fewer than 10%. To alleviate these vulnerabilities, we present G2uardFL, a protective framework that translates the detection of malicious clients into an attributed graph clustering problem, thus safeguarding FL systems. Specifically, this framework employs a client graph clustering approach to identify malicious clients and integrates an adaptive mechanism to amplify the discrepancy between the aggregated model and the poisoned ones, effectively eliminating embedded backdoors. Through empirical evaluation, comparing G2uardFL with cutting-edge defenses, such as FLAME (USENIX Security 2022) [37] and DeepSight (NDSS 2022) [43], against various backdoor attacks, including 3DFed (SP 2023) [26], our results demonstrate its significant effectiveness in mitigating backdoor attacks while having a negligible impact on the aggregated model’s performance on benign samples (i.e., the primary task performance). For instance, in an FL system with 25% malicious clients, G2uardFL reduces the attack success rate to 10.61%, while maintaining a primary task performance of 80.98% on the CIFAR-10 dataset. This surpasses the performance of the best-performing baseline, which merely achieves the attack success rate of 19.54%.
Hao Yu 0017, Chuan Ma 0001, Meng Liu 0014, Tianyu Du, Ming Ding 0001, Tao Xiang 0001, Shouling Ji, Xinwang Liu 0002
IEEE Trans. Inf. Forensics Secur.4
2025 DP-MemArc: Differential Privacy Transfer Learning for Memory Efficient Language Models
abstract
Large language models have repeatedly shown outstanding performance across diverse applications. However, deploying these models can inadvertently risk user privacy. The significant memory demands during training pose a major challenge in terms of resource consumption. This substantial size places a heavy load on memory resources, raising considerable practical concerns. In this paper, we introduce DP-MemArc, a novel training framework aimed at reducing the memory costs of large language models while emphasizing the protection of user data privacy. DP-MemArc incorporates side network or reversible network designs to support a variety of differential privacy memory-efficient fine-tuning schemes. Our approach not only achieves about 2.5 times in memory optimization but also ensures robust privacy protection, keeping user data secure and confidential. Extensive experiments have demonstrated that DP-MemArc effectively provides differential privacy-efficient fine-tuning across different task scenarios.
Yanming Liu 0003, Xinyue Peng, Xiaolan Ke, Songhang Deng, Jiannan Cao, Mengchen Fu, Xuhong Zhang 0002, Jianwei Yin, Tianyu Du
AAAI13
2025 DROWN: Towards Tighter LiRPA-based Robustness Certification
abstract
The susceptibility of deep neural networks to adversarial attacks is a well-established concern. To address this problem, robustness certification is proposed, which, unfortunately, suffers from precision or scalability issues. In this paper, we present DROWN (Dual CROWN), a novel method for certifying the robustness of DNNs. The advantage of DROWN is that it tightens classic LiRPA-based methods yet maintains similar scalability, which comes from refining pre-activation bounds of ReLU relaxations using two pairs of linear bounds derived from different relaxations of ReLU units in previous layers. The extensive evaluations show that DROWN achieves up to 83.39% higher certified robust accuracy than the baseline on CNNs and up to 4.68 times larger certified radii than the baseline on Transformers. Meanwhile, the running time of DROWN is about twice that of the baseline.
Yunruo Zhang, Tianyu Du, Shouling Ji, Shanqing Guo
COLING2
2025 IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents
abstract
Large language model (LLM) agents are widely deployed in real-world applications, where they leverage tools to retrieve and manipulate external data for complex tasks.However, when interacting with untrusted data sources (e.g., fetching information from public websites), tool responses may contain injected instructions that covertly influence agent behaviors and lead to malicious outcomes, a threat referred to as Indirect Prompt Injection (IPI).Existing defenses typically rely on advanced prompting strategies or auxiliary detection models.While these methods have demonstrated some effectiveness, they fundamentally rely on assumptions about the model's inherent security, which lacks structural constraints on agent behaviors.As a result, agents still retain unrestricted access to tool invocations, leaving them vulnerable to stronger attack vectors that can bypass the security guardrails of the model.To prevent malicious tool invocations at the source, we propose a novel defensive task execution paradigm, called IPIGUARD 1 , which models the agents' task execution process as a traversal over a planned Tool Dependency Graph (TDG).By explicitly decoupling action planning from interaction with external data, IPIGUARD significantly reduces unintended tool invocations triggered by injected instructions, thereby enhancing robustness against IPI attacks.Experiments on the AgentDojo benchmark show that IPIGUARD achieves a superior balance between effectiveness and robustness, paving the way for the development of safer agentic systems in dynamic environments.
Hengyu An, Jinghuai Zhang, Tianyu Du, Chunyi Zhou 0001, Qingming Li, Tao Lin 0004, Shouling Ji
EMNLP3
2025 VideoEraser: Concept Erasure in Text-to-Video Diffusion Models
abstract
The rapid growth of text-to-video (T2V) diffusion models has raised concerns about privacy, copyright, and safety due to their potential misuse in generating harmful or misleading content. These models are often trained on numerous datasets, including unauthorized personal identities, artistic creations, and harmful materials, which can lead to uncontrolled production and distribution of such content. To address this, we propose VideoEraser, a training-free framework that prevents T2V diffusion models from generating videos with undesirable concepts, even when explicitly prompted with those concepts. Designed as a plug-and-play module, VideoEraser can seamlessly integrate with representative T2V diffusion models via a two-stage process: Selective Prompt Embedding Adjustment (SPEA) and Adversarial-Resilient Noise Guidance (ARNG). We conduct extensive evaluations across four tasks, including object erasure, artistic style erasure, celebrity erasure, and explicit content erasure. Experimental results show that VideoEraser consistently outperforms prior methods regarding efficacy, integrity, fidelity, robustness, and generalizability. Notably, VideoEraser achieves state-of-the-art performance in suppressing undesirable content during T2V generation, reducing it by 46% on average across four tasks compared to baselines.
Naen Xu, Jinghuai Zhang, Changjiang Li, Chunyi Zhou 0001, Qingming Li, Tianyu Du, Shouling Ji
EMNLP7
2025 CLMTracing: Black-box User-level Watermarking for Code Language Model Tracing
abstract
With the widespread adoption of open-source code language models (code LMs), intellectual property (IP) protection has become an increasingly critical concern.While current watermarking techniques have the potential to identify the code LM to protect its IP, they have limitations when facing the more practical and complex demand, i.e., offering the individual user-level tracing in the black-box setting.This work presents CLMTracing, a black-box code LM watermarking framework employing the rule-based watermarks and utility-preserving injection method for user-level model tracing.CLMTracing further incorporates a parameter selection algorithm sensitive to the robust watermark and adversarial training to enhance the robustness against watermark removal attacks.Comprehensive evaluations demonstrate CLM-Tracing is effective across multiple state-ofthe-art (SOTA) code LMs, showing significant harmless improvements compared to existing SOTA baselines and strong robustness against various removal attacks.
Tianyu Du, Xuhong Zhang 0002, Lei Yun, Kingsum Chow, Jianwei Yin
EMNLP3
2025 An Inversion-Based Measure of Memorization for Diffusion Models
Zhe Ma 0002, Qingming Li, Xuhong Zhang 0002, Tianyu Du, Ruixiao Lin, Zonghui Wang, Shouling Ji, Wenzhi Chen
ICCV4
2025 Bridging Context Gaps: Leveraging Coreference Resolution for Long Contextual Understanding
abstract
Large language models (LLMs) have shown remarkable capabilities in natural language processing; however, they still face difficulties when tasked with understanding lengthy contexts and executing effective question answering. These challenges often arise due to the complexity and ambiguity present in longer texts. To enhance the performance of LLMs in such scenarios, we introduce the Long Question Coreference Adaptation (LQCA) method. This innovative framework focuses on coreference resolution tailored to long contexts, allowing the model to identify and manage references effectively. The LQCA method encompasses four key steps: resolving coreferences within sub-documents, computing the distances between mentions, defining a representative mention for coreference, and answering questions through mention replacement. By processing information systematically, the framework provides easier-to-handle partitions for LLMs, promoting better understanding. Experimental evaluations on a range of LLMs and datasets have yielded positive results, with a notable improvements on OpenAI-o1-mini and GPT-4o models, highlighting the effectiveness of leveraging coreference resolution to bridge context gaps in question answering. Our code is public at https://github.com/OceannTwT/LQCA.
Yanming Liu 0003, Xinyue Peng, Jiannan Cao, Shi Bo, Yanxin Shen, Tianyu Du, Jianwei Yin, Xuhong Zhang 0002
ICLR6
2025 Tool-Planner: Task Planning with Clusters across Multiple Tools
abstract
Large language models (LLMs) have demonstrated exceptional reasoning capabilities, enabling them to solve various complex problems. Recently, this ability has been applied to the paradigm of tool learning. Tool learning involves providing examples of tool usage and their corresponding functions, allowing LLMs to formulate plans and demonstrate the process of invoking and executing each tool. LLMs can address tasks that they cannot complete independently, thereby enhancing their potential across different tasks. However, this approach faces two key challenges. First, redundant error correction leads to unstable planning and long execution time. Additionally, designing a correct plan among multiple tools is also a challenge in tool learning. To address these issues, we propose Tool-Planner, a task-processing framework based on toolkits. Tool-Planner groups tools based on the API functions with the same function into a toolkit and allows LLMs to implement planning across the various toolkits. When a tool error occurs, the language model can reselect and adjust tools based on the toolkit. Experiments show that our approach demonstrates a high pass and win rate across different datasets and optimizes the planning scheme for tool learning in models such as GPT-4 and Claude 3, showcasing the potential of our method. Our code is public at https://github.com/OceannTwT/Tool-Planner.
Yanming Liu 0003, Xinyue Peng, Jiannan Cao, Shi Bo, Xuhong Zhang 0002, Jianwei Yin, Tianyu Du
ICLR10
2025 CollabEdit: Towards Non-destructive Collaborative Knowledge Editing
abstract
Collaborative learning of large language models (LLMs) has emerged as a new paradigm for utilizing private data from different parties to guarantee efficiency and privacy. Meanwhile, Knowledge Editing (KE) for LLMs has also garnered increased attention due to its ability to manipulate the behaviors of LLMs explicitly, yet leaves the collaborative KE case—in which knowledge edits of multiple parties are aggregated in a privacy-preserving and continual manner—unexamined. To this end, this manuscript dives into the first investigation of collaborative KE, in which we start by carefully identifying the unique three challenges therein, including knowledge overlap, knowledge conflict, and knowledge forgetting. We then propose a non-destructive collaborative KE framework, COLLABEDIT, which employs a novel model merging mechanism to mimic the global KE behavior while preventing the severe performance drop. Extensive experiments on two canonical datasets demonstrate the superiority of COLLABEDIT compared to other destructive baselines, and results shed light on addressing three collaborative KE challenges and future applications. Our code is available at [https://github.com/LINs-lab/CollabEdit](https://github.com/LINs-lab/CollabEdit).
Jiamu Zheng, Jinghuai Zhang, Tianyu Du, Xuhong Zhang 0002, Jianwei Yin
ICLR3
2025 Scalable Multi-Stage Influence Function for Large Language Models via Eigenvalue-Corrected Kronecker-Factored Parameterization
abstract
Pre-trained large language models (LLMs) are commonly fine-tuned to adapt to downstream tasks. Since the majority of knowledge is acquired during pre-training, attributing the predictions of fine-tuned LLMs to their pre-training data may provide valuable insights. Influence functions have been proposed as a means to explain model predictions based on training data. However, existing approaches often fail to compute "multi-stage" influence and lack scalability to billion-scale LLMs. In this paper, we propose multi-stage influence functions to attribute the downstream predictions of fine-tuned LLMs to pre-training data under the full-parameter fine-tuning paradigm. To enhance the efficiency and practicality of our multi-stage influence function, we leverage Eigenvalue-corrected Kronecker-Factored (EK-FAC) parameterization for efficient approximation. Empirical results validate the superior scalability of EK-FAC approximation and the effectiveness of our multi-stage influence function. Additionally, case studies on a real-world LLM, dolly-v2-3b, demonstrate its interpretive power, with exemplars illustrating insights provided by multi-stage influence estimates.
Yuntai Bao, Xuhong Zhang 0002, Tianyu Du, Xinkui Zhao, Jiang Zong, Hao Peng 0002, Jianwei Yin
IJCAI3
2025 Enkidu: Universal Frequential Perturbation for Real-Time Audio Privacy Protection against Voice Deepfakes
abstract
The rise of advanced voice deepfake technologies has raised serious concerns over user audio privacy, as malicious actors increasingly exploit publicly available voice data to generate convincing fake audio for malicious purposes such as identity theft, financial fraud and misinformation campaigns. While existing defense methods offer partial protection, they suffer from critical limitations, including weak adaptability to unseen user data, poor scalability to long audio, regid reliance on white-box knowledge and high computational and temporal costs to encryption process. Therefore, to defend against personalized voice deepfake threats, we propose Enkidu, a novel user-oriented privacy-preserving framework that leverages universal frequential perturbations generated through black-box knowledge and few-shot training on a small amount of user samples. These high-malleablity frequency-domain noise patches enable real-time, lightweight protection with strong generalization across variable-length audio and robust resistance against voice deepfake attacks-all while preserving high perceptual and intelligible audio quality. Notably, Enkidu achieves over 50-200× processing memory efficiency (requiring only 0.004 GB) and over 3-7000× runtime efficiency (real-time coefficient as low as 0.004) compared to six SOTA countermeasures. Extensive experiments across six mainstream Text-to-Speech (TTS) models and five cutting-edge Automated Speaker Verification (ASV) models demonstrate the effectiveness, transferability, and practicality of Enkidu in defending against voice deepfakes and adaptive attacks.
Zhou Feng, Chunyi Zhou 0001, Yuwen Pu, Qingming Li, Tianyu Du, Shouling Ji
ACM Multimedia6
2025 CLIBE: Detecting Dynamic Backdoors in Transformer-based NLP Models
Yuwen Pu, Xuhong Zhang 0002, Tianyu Du, Shouling Ji
NDSS5
2024 ERA-CoT: Improving Chain-of-Thought through Entity Relationship Analysis
abstract
Large language models (LLMs) have achieved commendable accomplishments in various natural language processing tasks.However, LLMs still encounter significant challenges when dealing with complex scenarios involving multiple entities.These challenges arise from the presence of implicit relationships that demand multi-step reasoning.In this paper, we propose a novel approach ERA-CoT, which aids LLMs in understanding context by capturing relationships between entities and supports the reasoning of diverse tasks through Chainof-Thoughts (CoT).Experimental results show that ERA-CoT demonstrates the superior performance of our proposed method compared to current CoT prompting methods, achieving a significant improvement of an average of 5.1% on GPT3.5 compared to previous SOTA baselines.Our analysis indicates that ERA-CoT increases the LLM's understanding of entity relationships, significantly improves the accuracy of question answering, and enhances the reasoning ability of LLMs. 1
Yanming Liu 0003, Xinyue Peng, Tianyu Du, Jianwei Yin, Xuhong Zhang 0002
ACL (1)3
2024 Unveiling the Vulnerability of Private Fine-Tuning in Split-Based Frameworks for Large Language Models: A Bidirectionally Enhanced Attack
abstract
Recent advancements in pre-trained large language models (LLMs) have significantly influenced various domains. Adapting these models for specific tasks often involves fine-tuning (FT) with private, domain-specific data. However, privacy concerns keep this data undisclosed, and the computational demands for deploying LLMs pose challenges for resource-limited data holders. This has sparked interest in split learning (SL), a Model-as-a-Service (MaaS) paradigm that divides LLMs into smaller segments for distributed training and deployment, transmitting only intermediate activations instead of raw data. SL has garnered substantial interest in both industry and academia as it aims to balance user data privacy, model ownership, and resource challenges in the private fine-tuning of LLMs. Despite its privacy claims, this paper reveals significant vulnerabilities arising from the combination of SL and LLM-FT: the Not-too-far property of fine-tuning and the auto-regressive nature of LLMs. Exploiting these vulnerabilities, we propose Bidirectional Semi-white-box Reconstruction (BiSR), the first data reconstruction attack (DRA) designed to target both the forward and backward propagation processes of SL. BiSR utilizes pre-trained weights as prior knowledge, combining a learning-based attack with a bidirectional optimization-based approach for highly effective data reconstruction. Additionally, it incorporates a Noise-adaptive Mixture of Experts (NaMoE) model to enhance reconstruction performance under perturbation. We conducted systematic experiments on various mainstream LLMs and different setups, empirically demonstrating BiSR's state-of-the-art performance. Furthermore, we thoroughly examined three representative defense mechanisms, showcasing our method's capability to reconstruct private data even in the presence of these defenses.
Zhenghan Qin, Mingxin Yang, Tao Fan 0002, Tianyu Du, Zenglin Xu
CCS6
2024 SecCoder: Towards Generalizable and Robust Secure Code Generation
abstract
After large models (LMs) have gained widespread acceptance in code-related tasks, their superior generative capacity has greatly promoted the application of the code LM.Nevertheless, the security of the generated code has raised attention to its potential damage.Existing secure code generation methods have limited generalizability to unseen test cases and poor robustness against the attacked model, leading to safety failures in code generation.In this paper, we propose a generalizable and robust secure code generation method SecCoder by using in-context learning (ICL) and the safe demonstration.The dense retriever is also used to select the most helpful demonstration to maximize the improvement of the generated code's security.Experimental results show the superior generalizability of the proposed model Sec-Coder compared to the current secure code generation method, achieving a significant security improvement of an average of 7.20% on unseen test cases.The results also show the better robustness of SecCoder compared to the current attacked code LM, achieving a significant security improvement of an average of 7.74%.Our analysis indicates that SecCoder enhances the security of LMs in generating code, and it is more generalizable and robust.
Tianyu Du, Junkai Tong 0001, Xuhong Zhang 0002, Kingsum Chow, Jianwei Yin
EMNLP2
2024 ReMasker: Imputing Tabular Data with Masked Autoencoding
abstract
We present ReMasker, a new method of imputing missing values in tabular data by extending the masked autoencoding framework. Compared with prior work, ReMasker is extremely simple -- besides the missing values (i.e., naturally masked), we randomly "re-mask" another set of values, optimize the autoencoder by reconstructing this re-masked set, and apply the trained model to predict the missing values; and yet highly effective -- with extensive evaluation on benchmark datasets, we show that ReMasker performs on par with or outperforms state-of-the-art methods in terms of both imputation fidelity and utility under various missingness settings, while its performance advantage often increases with the ratio of missing data. We further explore theoretical justification for its effectiveness, showing that ReMasker tends to learn missingness-invariant representations of tabular data. Our findings indicate that masked modeling represents a promising direction for further research on tabular data imputation. The code is publicly available.
Tianyu Du, Luca Melis, Ting Wang 0006
ICLR1
2024 Cons2Plan: Vector Floorplan Generation from Various Conditions via a Learning Framework based on Conditional Diffusion Models
abstract
The field of floorplan generation has attracted significant interest from the community. Remarkably, recent advances in generative models have markedly enhanced the development of this field. However, generating floorplans that satisfy various conditions remains a challenging task. This paper proposes a learning framework, named Cons2Plan, for automatically and high-quality generating vector floorplans from various conditions. The input conditions can be graphs, boundaries, or a combination of both. The conditional diffusion model is the core component of our Cons2Plan. The denoising network uses a conditional embedding module to incorporate the conditions during the reverse process. Additionally, Cons2Plan incorporates a two-stage approach that generates graph conditions based on boundaries. It uses three networks for node prediction and a novel conditional edge generation diffusion model, named CEDM, for edge generation. We conduct qualitative evaluations, quantitative comparisons, and ablation studies to show that our method produces better floorplans than state-of-the-art methods.
Shibo Hong, Xuhong Zhang 0002, Tianyu Du, Jianwei Yin
ACM Multimedia3
2024 TransLinkGuard: Safeguarding Transformer Models Against Model Stealing in Edge Deployment
abstract
Proprietary large language models (LLMs) have been widely applied in various scenarios. Additionally, deploying LLMs on edge devices is trending for efficiency and privacy reasons. However, edge deployment of proprietary LLMs introduces new security challenges: edge-deployed models are exposed as white-box accessible to users, enabling adversaries to conduct model stealing (MS) attacks. Unfortunately, existing defense mechanisms fail to provide effective protection. Specifically, we identify four critical protection properties that existing methods fail to simultaneously satisfy: (1) maintaining protection after a model is physically copied; (2) authorizing model access at request level; (3) safeguarding runtime reverse engineering; (4) achieving high security with negligible runtime overhead. To address the above issues, we propose TransLinkGuard, a plug-and-play model protection approach against model stealing on edge devices. The core part of TransLinkGuard is a lightweight authorization module residing in a secure environment, e.g., TEE, which can freshly authorize each request based on its input. Extensive experiments show that TransLinkGuard achieves the same security as the black-box guarantees with negligible overhead.
Qinfeng Li, Zhenghan Qin, Yangfan Xie, Xuhong Zhang 0002, Tianyu Du, Jianwei Yin
ACM Multimedia6
2024 A truthful near-optimal mechanism for online linear packing-covering problem in the random order model
Jinshan Zhang 0001, Xiaoye Miao, Meng Xi 0002, Tianyu Du, Jianwei Yin
Inf. Comput.4
2023 An Embarrassingly Simple Backdoor Attack on Self-supervised Learning
abstract
As a new paradigm in machine learning, self-supervised learning (SSL) is capable of learning high-quality representations of complex data without relying on labels. In addition to eliminating the need for labeled data, research has found that SSL improves the adversarial robustness over supervised learning since lacking labels makes it more challenging for adversaries to manipulate model predictions. However, the extent to which this robustness superiority generalizes to other types of attacks remains an open question.We explore this question in the context of backdoor attacks. Specifically, we design and evaluate Ctrl, an embarrassingly simple yet highly effective self-supervised backdoor attack. By only polluting a tiny fraction of training data (≤ 1%) with indistinguishable poisoning samples, Ctrl causes any trigger-embedded input to be misclassified to the adversary's designated class with a high probability (≥ 99%) at inference time. Our findings suggest that SSL and supervised learning are comparably vulnerable to backdoor attacks. More importantly, through the lens of Ctrl, we study the inherent vulnerability of SSL to backdoor attacks. With both empirical and analytical evidence, we reveal that the representation invariance property of SSL, which benefits adversarial robustness, may also be the very reason making SSL highly susceptible to backdoor attacks. Our findings also imply that the existing defenses against supervised backdoor attacks are not easily retrofitted to the unique vulnerability of SSL. Code is available at: https://github.com/meet-cjli/CTRL
Changjiang Li, Ren Pang, Zhaohan Xi, Tianyu Du, Shouling Ji, Yuan Yao 0001, Ting Wang 0006
ICCV4
2023 A Deep Learning based Multi-edge-type decoding algorithm for 5G NR LDPC codes
abstract
Low-density parity-check(LDPC) code has been selected as the channel coding method by 5G NR because of its excellent error-correcting performance. To further improve the performance of LDPC decoding, this paper proposes a neural normalized min-sum(NNMS) algorithm based on multi-edge-type(MET). Based on the LLR convergence analysis of the protograph matrix of 5G NR, the base matrix is divided into several independent regions. Each part is assigned a unique scaling factor at different iterations. To verify the effectiveness of the proposed algorithm, We use two parity-check matrixes(PCM) derived from different base graphs in simulations. The results show that the proposed algorithm performs at most 0.45dB better than BP, 0.37dB better than NMS, and 0.25dB better than OMS, respectively, when the frame error rate (FER) is at 1$0^{-5}$ level over additive white Gaussian noise (AWGN) channels using BPSK modulation.
Tianyu Du, Hao Ju 0002, Yin Xu 0001, Dazhi He, Wenjun Zhang 0001
IWCMC1
2023 VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models
abstract
Vision-Language (VL) pre-trained models have shown their superiority on many multimodal tasks. However, the adversarial robustness of such models has not been fully explored. Existing approaches mainly focus on exploring the adversarial robustness under the white-box setting, which is unrealistic. In this paper, we aim to investigate a new yet practical task to craft image and text perturbations using pre-trained VL models to attack black-box fine-tuned models on different downstream tasks. Towards this end, we propose VLATTACK to generate adversarial samples by fusing perturbations of images and texts from both single-modal and multi-modal levels. At the single-modal level, we propose a new block-wise similarity attack (BSA) strategy to learn image perturbations for disrupting universal representations. Besides, we adopt an existing text attack strategy to generate text perturbations independent of the image-modal attack. At the multi-modal level, we design a novel iterative cross-search attack (ICSA) method to update adversarial image-text pairs periodically, starting with the outputs from the single-modal level. We conduct extensive experiments to attack three widely-used VL pretrained models for six tasks on eight datasets. Experimental results show that the proposed VLATTACK framework achieves the highest attack success rates on all tasks compared with state-of-the-art baselines, which reveals a significant blind spot in the deployment of pre-trained VL models.
Ziyi Yin 0003, Muchao Ye, Tianrong Zhang, Tianyu Du, Jinguo Zhu, Han Liu 0008, Ting Wang 0006, Fenglong Ma
NeurIPS4
2023 Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attacks
abstract
Pre-trained language models (PLMs) have demonstrated remarkable performance as few-shot learners. However, their security risks under such settings are largely unexplored. In this work, we conduct a pilot study showing that PLMs as few-shot learners are highly vulnerable to backdoor attacks while existing defenses are inadequate due to the unique challenges of few-shot scenarios. To address such challenges, we advocate MDP, a novel lightweight, pluggable, and effective defense for PLMs as few-shot learners. Specifically, MDP leverages the gap between the masking-sensitivity of poisoned and clean samples: with reference to the limited few-shot data as distributional anchors, it compares the representations of given samples under varying masking and identifies poisoned samples as ones with significant variations. We show analytically that MDP creates an interesting dilemma for the attacker to choose between attack effectiveness and detection evasiveness. The empirical evaluation using benchmark datasets and representative attacks validates the efficacy of MDP. The code of MDP is publicly available.
Zhaohan Xi, Tianyu Du, Changjiang Li, Ren Pang, Shouling Ji, Fenglong Ma, Ting Wang 0006
NeurIPS2
2023 UniT: A Unified Look at Certified Robust Training against Text Adversarial Perturbation
abstract
Recent years have witnessed a surge of certified robust training pipelines against text adversarial perturbation constructed by synonym substitutions. Given a base model, existing pipelines provide prediction certificates either in the discrete word space or the continuous latent space. However, they are isolated from each other with a structural gap. We observe that existing training frameworks need unification to provide stronger certified robustness. Additionally, they mainly focus on building the certification process but neglect to improve the robustness of the base model. To mitigate the aforementioned limitations, we propose a unified framework named UniT that enables us to train flexibly in either fashion by working in the word embedding space. It can provide a stronger robustness guarantee obtained directly from the word embedding space without extra modules. In addition, we introduce the decoupled regularization (DR) loss to improve the robustness of the base model, which includes two separate robustness regularization terms for the feature extraction and classifier modules. Experimental results on widely used text classification datasets further demonstrate the effectiveness of the designed unified framework and the proposed DR loss for improving the certified robust accuracy.
Muchao Ye, Ziyi Yin 0003, Tianrong Zhang, Tianyu Du, Ting Wang 0006, Fenglong Ma
NeurIPS4
2023 On the Security Risks of Knowledge Graph Reasoning
Zhaohan Xi, Tianyu Du, Changjiang Li, Ren Pang, Shouling Ji, Xiapu Luo, Xusheng Xiao, Fenglong Ma, Ting Wang 0006
USENIX Security Symposium2
2023 Your Labels are Selling You Out: Relation Leaks in Vertical Federated Learning
abstract
Vertical federated learning (VFL) is an emerging privacy-preserving paradigm that enables collaboration between companies. These companies have the same set of users but different features. One of them is interested in expanding new business or improving its current service with others’ features. For instance, an e-commerce company, who wants to improve its recommendation performance, can incorporate users’ preferences from another corporation such as a social media company through VFL. On the other hand, graph data is a powerful and sensitive type of data widely used in industry. Their leakage, e.g., the node leakage and/or the relation leakage, can cause severe privacy issues and financial loss. Therefore, protecting the security of graph data is important in practice. Though a line of work has studied how to learn with graph data in VFL, the privacy risks remain underexplored. In this paper, we perform the first systematic study onrelation inference attacksto reveal VFL's risk of leaking samples’ relations. Specifically, we assume the adversary to be a semi-honest participant. Then, according to the adversary's knowledge level, we formulate three kinds of attacks based on different intermediate representations. Particularly, we design a novel numerical approximation method to handle VFL's encryption mechanism on the participant's representations. Extensive evaluations with four real-world datasets demonstrate the effectiveness of our attacks. For instance, the area under curve of relation inference can reach more than 90%, implying an impressive relation inference capability. Furthermore, we evaluate possible defenses to examine our attacks’ robustness. The results show that their impacts are limited. Our work highlights the need for advanced defenses to protect private relations and calls for more exploration of VFL's privacy and security issues.
Pengyu Qiu, Xuhong Zhang 0002, Shouling Ji, Tianyu Du, Yuwen Pu, Jun Zhou 0011, Ting Wang 0006
IEEE Trans. Dependable Secur. Comput.4
2023 Multi-level caching and data verification based on ethereum blockchain
Qingzhe Zhang, Chunlin Li 0001, Tianyu Du, Youlong Luo
Wirel. Networks3
2022 NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identification
abstract
Deep neural networks (DNNs) have demonstrated their outperformance in various domains. However, it raises a social concern whether DNNs can produce reliable and fair decisions especially when they are applied to sensitive domains involving valuable resource allocation, such as education, loan, and employment. It is crucial to conduct fairness testing before DNNs are reliably deployed to such sensitive domains, i.e., generating as many instances as possible to uncover fairness violations. However, the existing testing methods are still limited from three aspects: interpretability, performance, and generalizability. To overcome the challenges, we propose NeuronFair, a new DNN fairness testing framework that differs from previous work in several key aspects: (1) interpretable - it quantitatively interprets DNNs' fairness violations for the biased decision; (2) effective - it uses the interpretation results to guide the generation of more diverse instances in less time; (3) generic - it can handle both structured and unstructured data. Extensive evaluations across 7 datasets and the corresponding DNNs demonstrate NeuronFair's superior performance. For instance, on structured datasets, it generates much more instances (~ ×5.84) and saves more time (with an average speedup of 534.56%) compared with the state-of-the-art methods. Besides, the instances of NeuronFair can also be leveraged to improve the fairness of the biased DNNs, which helps build more fair and trustworthy deep learning systems. The code of NeuronFair is open-sourced at https://github.com/haibinzheng/NeuronFair.
Haibin Zheng, Zhiqing Chen, Tianyu Du, Xuhong Zhang 0002, Yao Cheng 0002, Shouling Ji, Jingyi Wang 0004, Yue Yu 0001, Jinyin Chen
ICSE3
2022 ROLAND: Graph Learning Framework for Dynamic Graphs
abstract
Graph Neural Networks (GNNs) have been successfully applied to many real-world static graphs. However, the success of static graphs has not fully translated to dynamic graphs due to the limitations in model design, evaluation settings, and training strategies. Concretely, existing dynamic GNNs do not incorporate state-of-the-art designs from static GNNs, which limits their performance. Current evaluation settings for dynamic GNNs do not fully reflect the evolving nature of dynamic graphs. Finally, commonly used training methods for dynamic GNNs are not scalable. Here we propose ROLAND, an effective graph representation learning framework for real-world dynamic graphs. At its core, the ROLAND framework can help researchers easily repurpose any static GNN to dynamic graphs. Our insight is to view the node embeddings at different GNN layers as hierarchical node states and then recurrently update them over time. We then introduce a live-update evaluation setting for dynamic graphs that mimics real-world use cases, where GNNs are making predictions and being updated on a rolling basis. Finally, we propose a scalable and efficient training approach for dynamic GNNs via incremental training and meta-learning. We conduct experiments over eight different dynamic graph datasets on future link prediction tasks. Models built using the ROLAND framework achieve on average 62.7% relative mean reciprocal rank (MRR) improvement over state-of-the-art baselines under the standard evaluation settings on three datasets. We find state-of-the-art baselines experience out-of-memory errors for larger datasets, while ROLAND can easily scale to dynamic graphs with 56 million edges. After re-implementing these baselines using the ROLAND training strategy, ROLAND models still achieve on average 15.5% relative MRR improvement over the baselines.
Jiaxuan You, Tianyu Du, Jure Leskovec
KDD2
2022 Latency-aware computation offloading and DQN-based resource allocation approaches in SDN-enabled MEC
Tianyu Du, Chunlin Li 0001, Youlong Luo
Ad Hoc Networks1
2022 DetectS ec: Evaluating the robustness of object detection models to adversarial attacks
abstract
Despite their tremendous success in various machine learning tasks, deep neural networks (DNNs) are inherently vulnerable to adversarial examples, which are maliciously crafted inputs to cause DNNs to misbehave. Intensive research has been conducted on this phenomenon in simple tasks (e.g., image classification). However, little is known about this adversarial vulnerability for object detection, a much more complicated task, which often requires specialized DNNs and multiple additional components. In this paper, we present DetectSec, a uniform platform for robustness analysis of object detection models. Currently, DetectSec implements 13 representative adversarial attacks with 7 utility metrics and 13 defenses on 18 standard object detection models. Leveraging DetectSec, we conduct the first rigorous evaluation of adversarial attacks on the state-of-the-art object detection models. We analyze the impact of the factors including DNN architecture and capacity on the model robustness. We show that many conclusions about adversarial attacks and defenses in image classification tasks do not transfer to object detection tasks, for example, the targeted attack is stronger than the untargeted attack for two-stage detectors. Our findings will aid future efforts in understanding and defending against adversarial attacks in complicated tasks. In addition, we compare the robustness of different detection models and discuss their relative strengths and weaknesses. The platform DetectSec will be open source as a unique facility for further research on adversarial attacks and defenses in object detection tasks.
Tianyu Du, Shouling Ji, Bo Li 0026, Tao Wei 0002, Yunhan Jia, Raheem A. Beyah, Ting Wang 0006
Int. J. Intell. Syst.1
2021 Cert-RNN: Towards Certifying the Robustness of Recurrent Neural Networks
abstract
Certifiable robustness, the functionality of verifying whether the given region surrounding a data point admits any adversarial example, provides guaranteed security for neural networks deployed in adversarial environments. A plethora of work has been proposed to certify the robustness of feed-forward networks, e.g., FCNs and CNNs. Yet, most existing methods cannot be directly applied to recurrent neural networks (RNNs), due to their sequential inputs and unique operations.
Tianyu Du, Shouling Ji, Lujia Shen, Yao Zhang 0019, Chengfang Fang, Jianwei Yin, Raheem A. Beyah, Ting Wang 0006
CCS1
2021 Enhancing Model Robustness by Incorporating Adversarial Knowledge into Semantic Representation
abstract
Despite that deep neural networks (DNNs) have achieved enormous success in many domains like natural language processing (NLP), they have also been proven to be vulnerable to maliciously generated adversarial examples. Such inherent vulnerability has threatened various real-world deployed DNNs-based applications. To strength the model robustness, several countermeasures have been proposed in the English NLP domain and obtained satisfactory performance. However, due to the unique language properties of Chinese, it is not trivial to extend existing defenses to the Chinese domain. Therefore, we propose AdvGraph, a novel defense which enhances the robustness of Chinese-based NLP models by incorporating adversarial knowledge into the semantic representation of the input. Extensive experiments on two real-world tasks show that AdvGraph exhibits better performance compared with previous work: (i) effective – it significantly strengthens the model robustness even under the adaptive attacks setting without negative impact on model performance over legitimate input; (ii) generic – its key component, i.e., the representation of connotative adversarial knowledge is task-agnostic, which can be reused in any Chinese-based NLP models without retraining; and (iii) efficient – it is a light-weight defense with sub-linear computational complexity, which can guarantee the efficiency required in practical scenarios.
Tianyu Du, Rong Zhang 0006, Hui Xue 0001, Shouling Ji
ICASSP2
2021 FineFool: A novel DNN object contour attack on image recognition based on the attention perturbation adversarial technique
Jinyin Chen, Haibin Zheng, Hui Xiong 0005, Ruoxi Chen, Tianyu Du, Zhen Hong, Shouling Ji
Comput. Secur.5
2020 SirenAttack: Generating Adversarial Audio for End-to-End Acoustic Systems
abstract
Despite their immense popularity, deep learning-based acoustic systems are inherently vulnerable to adversarial attacks, wherein maliciously crafted audios trigger target systems to misbehave. In this paper, we present SirenAttack, a new class of attacks to generate adversarial audios. Compared with existing attacks, SirenAttack highlights with a set of significant features: (i) versatile -- it is able to deceive a range of end-to-end acoustic systems under both white-box and black-box settings; (ii) effective -- it is able to generate adversarial audios that can be recognized as specific phrases by target acoustic systems; and (iii) stealthy -- it is able to generate adversarial audios indistinguishable from their benign counterparts to human perception. We empirically evaluate SirenAttack on a set of state-of-the-art deep learning-based acoustic systems (including speech command recognition, speaker recognition and sound event classification), with results showing the versatility, effectiveness, and stealthiness of SirenAttack. For instance, it achieves 99.45% attack success rate on the IEMOCAP dataset against the ResNet18 model, while the generated adversarial audios are also misinterpreted by multiple popular ASR platforms, including Google Cloud Speech, Microsoft Bing Voice, and IBM Speech-to-Text. We further evaluate three potential defense methods to mitigate such attacks, including adversarial training, audio downsampling, and moving average filtering, which leads to promising directions for further research.
Tianyu Du, Shouling Ji, Qinchen Gu, Ting Wang 0006, Raheem A. Beyah
AsiaCCS1
2020 TextShield: Robust Text Classification Based on Multimodal Embedding and Neural Machine Translation
Tianyu Du, Shouling Ji, Rong Zhang 0006, Min Yang 0002, Ting Wang 0006
USENIX Security Symposium2
2019 Symmetric Frame Cracking: A Powerful Dynamic Textual CAPTCHAs Cracking Policy
Yueyao Chen, Qianjun Liu, Tianyu Du, Shouling Ji
Inscrypt3
2019 Invisible Poisoning: Highly Stealthy Targeted Poisoning Attack
Jinyin Chen, Haibin Zheng, Mengmeng Su, Tianyu Du, Chang-Ting Lin, Shouling Ji
Inscrypt4
2019 TextBugger: Generating Adversarial Text Against Real-world Applications
Shouling Ji, Tianyu Du, Bo Li 0026, Ting Wang 0006
NDSS3
2018 Online E-Commerce Fraud: A Large-Scale Detection and Analysis
abstract
Nowadays, e-commerce has become prevalent world-wide. With the big success of e-commerce, many malicious promotion services also rise: with the goal of increasing sales, malicious merchants attempt to promote their target items by illegally optimizing the search results using fake visits, purchases, etc. In this paper, we study the fraud detection problem on large-scale e-commerce platforms. First, we develop an efficient and scalable AnTi-Fraud system (ATF) to detect e-commerce frauds for large-scale e-commerce platforms, and implement it in parallel on a large-scale computing platform, called Open Data Processing Service (ODPS). Then, we evaluate ATF using two real large-scale e-commerce datasets (with tens of millions users and items). The results demonstrate that both the precision and the recall of ATF can achieve 0.97+, which suggests that ATF is very effective. More importantly, we deploy ATF on the Taobao platform of Alibaba, which is one of the world's largest e-commerce platforms. The evaluation results show that ATF can also achieve an accuracy of 98.16% on Taobao, which again suggests that ATF is very effective and deployable in practice. Our study in this paper is expected to shed light on defending against online frauds for practical e-commerce platforms.
Haiqin Weng, Zhao Li 0007, Shouling Ji, Chen Chu, Haifeng Lu, Tianyu Du, Qinming He
ICDE6
2018 Quantifying Graph Anonymity, Utility, and De-anonymity
abstract
In this paper, we study the correlation of graph da-ta's anonymity, utility, and de-anonymity. Our main contributions include four perspectives. First, to the best of our knowledge, we conduct the first Anonymity-Utility-De-anonymity (AUD) correlation quantification for graph data and obtain close-forms for such correlation under both a preliminary mathematical model and a general data model. Second, we integrate our AUD quantification to SecGraph [31], a recently published Secure Graph data sharing/publishing system, and extend it to Sec-Graph+. Compared to SecGraph, SecGraph+ is an improved and enhanced uniform and open-source system for comprehensively studying graph anonymization, de-anonymization, and utility evaluation. Third, based on our AUD quantification, we evaluate the anonymity, utility, and de-anonymity of 12 real world graph datasets which are generated from various computer systems and services. The results show that the achievable anonymity/de-anonymity depends on multiple factors, e.g., the preserved data utility, the quality of the employed auxiliary data. Finally, we apply our AUD quantification to evaluate the performance of state-of-the-art anonymization and de-anonymization techniques. Interestingly, we find that there is still significant space to improve state-of-the-art de-anonymization attacks. We also explicitly and quantitatively demonstrate such possible improvement space.
Shouling Ji, Tianyu Du, Zhen Hong, Ting Wang 0006, Raheem A. Beyah
INFOCOM2
2017 Influence Spread in Social Networks with both Positive and Negative Influences
Selena He, Ying Xie 0001, Tianyu Du, Shouling Ji, Zhao Li 0007
COCOON3
2015 Adaptive Preamble Padding with Retransmission Control for ZigBee network under Wi-Fi interference
abstract
The low-power, low-rate ZigBee wireless sensor networks are vulnerable to the interference of collocated Wi-Fi wireless local area networks (WLAN). The results acquired through our extensive experimental studies on ZigBee-WLAN coexistence indicate that ZigBee connections may experience severe packet losses, caused by combined effect of interference and relatively long receive to transmit (RX-TX) turnaround time in ZigBee devices, even when the Wi-Fi and ZigBee devices are able to detect each other's signal and apply CSMA/CA algorithm accordingly. This paper proposes a novel Adaptive Preamble Padding with Retransmission Control (APPRC) technique for ZigBee devices to address this issue, meet certain packet loss rate (PLR) requirement, and improve packet transmission efficiency when they are suffering time varying interference from the collocated WLAN. The experimental performance evaluation results showed that the proposed APPRC technique can achieve significantly higher transmission efficiency while satisfying PLR requirements of sensing applications than packet retransmission. The technique described in this paper is patent pending.
Tianyu Du, Zhipeng Wang 0007, Dimitrios Makrakis, Hussein T. Mouftah
IWCMC1
2015 Protective Dummy-byte Preamble Padding for improving ZigBee packet transmission under Wi-Fi interference
abstract
Recent studies have shown that the low-power ZigBee based wireless sensor networks (WSN) are vulnerable to the interference generated by nodes of Wi-Fi wireless local area networks (WLAN). Mutual interference can be mitigated at nodes of either technology when energy detection (ED) is enabled in clear channel assessment (CCA). From our experimental studies on ZigBee and Wi-Fi coexistence issue, it is determined that a significant amount of ZigBee packet losses occur due to the Wi-Fi interference induced corruption of the physical layer header of ZigBee packets, which could happen even when the ED mechanisms of the Wi-Fi and ZigBee devices are able to detect each other's signal and CSMA/CA algorithms are applied accordingly. To study this phenomenon, a series of experiments were carried out, followed by thorough analysis of the recorded data. The study led to the design of a simple but effective technique named Protective Dummy-byte Preamble Padding (PDBPP) that improves the performance of ZigBee packet transmission in terms of packet loss rate (PLR) and transmission efficiency. The experimental performance evaluation results confirmed the effectiveness of PDBPP in improving PLR and transmission efficiency of a ZigBee network exposed to interference generated by collocated WLAN. Some material in this paper is part of a pending patent.
Tianyu Du, Zhipeng Wang 0007, Dimitrios Makrakis, Hussein T. Mouftah
WCNC1
2013 Study of clear channel assessment mechanism for ZigBee packet transmission under Wi-Fi interference
abstract
Recent studies have shown that low-power ZigBee based wireless sensor networks (WSN) are vulnerable to the interference generated by Wi-Fi nodes. In this study, the effects of energy detection (ED) mechanism in clear channel assessment (CCA) of ZigBee transmitter under 802.11g Wi-Fi interference have been evaluated through experimentation. To improve the performance of ZigBee packet transmission, a preliminary adaptive mechanism is implemented and evaluated in our testbed.
Zhipeng Wang 0007, Tianyu Du, Dimitrios Makrakis, Hussein T. Mouftah
CCNC3