For an extensive period, Vision Transformers (ViTs) have been deemed unsuitable for attaining robust performance on small-scale datasets, with WideResNet models maintaining dominance in this domain. While WideResNet models have persistently set the state-of-the-art (SOTA) benchmarks for robust accuracy on datasets such as CIFAR-10 and CIFAR-100, this paper challenges the prevailing belief that only WideResNet can excel in this context. We pose the critical question of whether ViTs can surpass the robust accuracy of WideResNet models. Our results provide a resounding affirmative answer. By employing ViT, enhanced with data generated by a diffusion model for adversarial training, we demonstrate that ViTs can indeed outshine WideResNet in terms of robust accuracy. Specifically, under the Infty-norm threat model with epsilon = 8/255, our approach achieves robust accuracies of 74.97% on CIFAR-10 and 44.07% on CIFAR-100, representing improvements of +3.9% and +1.4%, respectively, over the previous SOTA models. Notably, our ViT-B/2 model, with 3 times fewer parameters, surpasses the previously best-performing WRN-70-16. Our achievement opens a new avenue, suggesting that future models employing ViTs or other novel efficient architectures could eventually replace the long-dominant WRN models.

Transformer models are increasingly deployed in critical applications, raising concerns about their security and robustness. Like convolutional neural networks (CNNs), transformers are vulnerable to adversarial attacks, including Universal Adversarial Perturbations (UAP), which mislead models with a single perturbation across multiple inputs. However, existing UAP methods are primarily designed for CNNs and struggle with Transformer architectures due to their self-attention mechanisms, leaving potential vulnerabilities underexplored. To address these limitations, we propose Attention-Shift Universal Adversarial Perturbation (AS-UAP), a novel UAP method specifically designed for transformer-based models. AS-UAP generates perturbations that mislead the model’s attention toward irrelevant regions by leveraging attention patterns, without requiring label information. It introduces a new way to identify distracted attention heads based on their contribution to the [CLS] token during the forward pass, rather than relying on gradients. By leveraging the redundancy and operational patterns of attention heads, AS-UAP effectively misdirects the model’s attention, resulting in misclassification. Experimental results demonstrate that AS-UAP improves attack effectiveness, achieving a 11.23% higher fooling rate than the best existing UAP method. This study systematically reveals universal adversarial vulnerabilities in Transformer architectures, highlighting exploitable security gaps. These insights contribute to the advancement of AI security, ultimately improving the dependability of AI systems in mission-critical applications.