Jiujiang Guo, Zhengliang Guo, Kai Wang, Meiyang Wang, Dehua Peng, Shaozu Yuan, Chengyin Hu, Shuan Ai, Yiwei Wei. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Despite strong performance on open-source large vision-language models (LVLMs), transfer-based targeted adversarial attacks often fail against closed-source commercial LVLMs. Existing methods typically rely on static perturbation allocation and uniformly weighted surrogate ensembles, limiting their ability to capture model-agnostic semantic vulnerabilities essential for cross-model transferability. Consequently, perturbations are frequently allocated to semantically irrelevant regions, reducing effectiveness under realistic black-box settings. To address this limitation, we propose the Gated Multi-signal Adaptive Ensemble (GMAE) framework, a novel transfer-based targeted attack that formulates adversarial generation as a joint optimization over a gated perturbation field and an adaptive ensemble objective. A learnable spatial gating mask dynamically reallocates perturbation budgets toward semantically vulnerable regions, while surrogate models are adaptively reweighted according to optimization difficulty and historical consistency, encouraging alignment on shared semantic weaknesses across heterogeneous architectures. To further enhance robustness, GMAE performs global semantic alignment in a symbolic discrete space, reducing the influence of non-essential feature variations and improving cross-model generalization. In addition, we introduce a local attention-guided multi-scale consistency mechanism that transfers fine-grained target textures into attention-identified sensitive regions, ensuring both visual imperceptibility and semantic coherence at feature and concept levels. Extensive experiments demonstrate that GMAE achieves strong transferability to commercial LVLMs, and reasoning-oriented models. Notably, GMAE achieves an attack success rate of 82.90% on GPT-4o, outperforming the strongest baseline by 7.8%, establishing a new state of the art for targeted transfer attacks on large-scale black-box LVLMs.

Abstract Recent studies have demonstrated that finely tuned deep neural networks (DNNs) are susceptible to adversarial attacks. Conventional physical attacks employ stickers as perturbations, achieving robust adversarial effects but compromising stealthiness. Recent innovations utilise light beams, such as lasers and projectors, for perturbation generation, allowing for stealthy physical attacks at the expense of robustness. In pursuit of implementing both stealthy and robust physical attacks, the authors present an adversarial catoptric light (AdvCL). This method leverages the natural phenomenon of catoptric light to generate perturbations that are both natural and stealthy. AdvCL first formalises the physical parameters of catoptric light and then optimises these parameters using a genetic algorithm to derive the most adversarial perturbation. Finally, the perturbations are deployed in the physical scene to execute stealthy and robust attacks. The proposed method is evaluated across three dimensions: effectiveness, stealthiness, and robustness. Quantitative results obtained in simulated environments demonstrate the efficacy of the proposed method, achieving an attack success rate of 83.5%, surpassing the baseline. The authors utilise common catoptric light as a perturbation to enhance the method's stealthiness, rendering physical samples more natural in appearance. Robustness is affirmed by successfully attacking advanced DNNs with a success rate exceeding 80% in all cases. Additionally, the authors discuss defence strategies against AdvCL and introduce some light‐based physical attacks.