Red team testing, an effective proactive method for evaluating the security of multimodal large language models (MLLMs), requires an expanding toolkit alongside the development of MLLM safeguards.We propose the Reference Attack, a powerful tool for red team testing against MLLMs.The Reference Attack is a reference-guided cross-modal jailbreak method that enhances existing prompt-to-image injection attacks by exploiting MLLMs' semantic reconstruction capabilities.Our method embeds malicious prompts in non-text modalities (e.g., images, spreadsheets) and constructs recursive symbolic references in text, enabling MLLMs to gradually recover and generate harmful content through layered reference resolution.The attack introduces a new vector that circumvents conventional content moderation by exploiting MLLMs' lack of security checks during crossmodal reference resolution.We evaluate the Reference Attack on leading MLLMs, including ChatGPT, Gemini, Claude, and the widely used open-source LLaMA model, and achieved an attack success rate of over 93% across all tested models.Compared to state-of-the-art attacks, Reference Attack achieves higher success rates than all baselines under identical evaluation, with a maximum gain of 70.8%.Our study reveals a critical gap in MLLM security and highlights the need for strict security auditing of cross-modal interactions in future content moderation.

Yizhou Ying, Geng Zhang, Cui Danxin, Chengyu Du, Guanglei Yue, Sihang Jiang, Jiaqing Liang, Yifei Fu, Hailin Hu, Yanghua Xiao. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Adversarial patches pose a significant threat to deep neural networks (DNNs). Unlike conventional adversarial attacks that are digital and less effective in the real world, adversarial patches can disrupt DNNs in real-world scenarios with potentially catastrophic outcomes. Understanding the characteristics of these patches is crucial to comprehending this new form of adversarial attack. While prior research has primarily aimed at enhancing the success rate of adversarial patches on specific DNN models, the rise of federated learning (FL) introduces a novel attack vector. In this context, attackers could manipulate a DNN model’s learnable parameters by contributing models trained on poisoned data. To assess the feasibility and danger of adversarial patches in this context, we introduce a novel attack method named PoisonPatch. This method merges FL poisoning attacks with adversarial patch search, first poisoning a DNN-based image classifier through FL, and then employing an adversarial patch search algorithm to create patches that increase the success rate of the attacks. This dual approach, combining poisoning attacks with adversarial patches, results in patches that are more challenging for machines to detect than traditional poisoning attacks and less noticeable to the human eye than typical adversarial patches due to their natural appearance. Our extensive experimental results demonstrate that PoisonPatch surpasses current state-of-the-art methods, producing natural-looking patches while achieving a 100% attack success rate.