Although adversarial training (AT) is currently one of the most promising methods to make deep neural networks adversarially robust, it suffers from the issue of robust overfitting and thus aggravates the robust generalization gap between the training and testing dataset. At the same time, data augmentations (DAs) are considered to be powerful tools for improving model generalization in standard training; however, they have been observed by many previous studies to be ineffective when applied in AT. In this paper, we try to break this prejudice and focus on improving the robust generalization ability of AT by DAs alone. We first take a close look at the effect of DAs in the adversarial training process and find that compared to common DAs, mixing-based augmentations (i.e.,MixUpandCutMix) can effectively prevent robust overfitting in AT. Then, after revisiting these two mixing-based DAs we found that they can be complementary and we can subtly stimulate the effectiveness ofMixUpandCutMixin improving the robust generalization of AT by a joint mixing manner. To this end, we propose a joint mixing-based augmentation scheme, namedMix2Aug, for improving robust generalization of AT and ultimately improving model robustness. Experimental results show that ourMix2Augcan significantly increase the upper limit ofMixUpandCutMixwithout the need of additional ensemble techniques, achieving state-of-the-art accuracy and robustness on extensive datasets.

Transfer-based adversarial attacks have endowed adversarial examples with the ability to transfer from a source model to an unknown target model, which poses a more realistic threat to security-critical applications. Existing transferable adversarial attacks generally suffer from overfitting to the source model, i.e., the perturbations are locally optimal in the source model and focus on the model-specific information. We demand the adversarial perturbation to contain more generalized knowledge, which reveals the intrinsic general properties and can introduce model-general optimum into adversarial examples, for improving transferability. To this end, we devise a Bi-level Bias Mitigated Attack (BBMA), which empowers the transferability of adversarial examples by exploring generalization in two levels: 1) Progressive filtering of high-frequency sample components. We first propose to remove the sample-specific high-frequency components of samples to explore model-level generation. To simulate how a model evaluates feature importance at different stages, we devise a stride-wise step-tuning strategy to progressively produce multiple samples for aggregating the gradients. 2) Accumulated gradient-guided model attention shift. To facilitate the sample-level bias mitigation, we employ an accumulated gradient-guided attention map to distort the more generalized features during perturbation generation. Comprehensive experiments on several benchmarks demonstrate the superiority of our method in attack transferability over state-of-the-art attacks.

Self-ensemble adversarial training methods improve model robustness by ensembling models at different training epochs, such as model weight averaging (WA). However, previous research has shown that self-ensemble defense methods in adversarial training (AT) still suffer from robust overfitting, which severely affects the generalization performance. Empirically, in the late phases of training, the AT becomes more overfitting to the extent that the individuals for weight averaging also suffer from overfitting and produce anomalous weight values, which causes the self-ensemble model to continue to undergo robust overfitting due to the failure in removing the weight anomalies. To solve this problem, we aim to tackle the influence of outliers in the weight space in this work and propose an easy-to-operate and effective Median-Ensemble Adversarial Training (MEAT) method to solve the robust overfitting phenomenon existing in self-ensemble defense from the source by searching for the median of the historical model weights. Experimental results show that MEAT achieves the best robustness against the powerful AutoAttack and can effectively allievate the robust overfitting. We further demonstrate that most defense methods can improve robust generalization and robustness by combining with MEAT.

While the transferability property of adversarial examples allows the adversary to perform black-box attacks (i.e., the attacker has no knowledge about the target model), the transfer-based adversarial attacks have gained great attention. Previous works mostly study gradient variation or image transformations to amplify the distortion on critical parts of inputs. These methods can work on transferring across models with limited differences, i.e., from CNNs to CNNs, but always fail in transferring across models with wide differences, such as from CNNs to ViTs. Alternatively, model ensemble adversarial attacks are proposed to fuse outputs from surrogate models with diverse architectures to get an ensemble loss, making the generated adversarial example more likely to transfer to other models as it can fool multiple models concurrently. However, existing ensemble attacks simply fuse the outputs of the surrogate models evenly, thus are not efficacious to capture and amplify the intrinsic transfer information of adversarial examples. In this paper, we propose an adaptive ensemble attack, dubbed AdaEA, to adaptively control the fusion of the outputs from each model, via monitoring the discrepancy ratio of their contributions towards the adversarial objective. Furthermore, an extra disparity-reduced filter is introduced to further synchronize the update direction. As a result, we achieve considerable improvement over the existing ensemble attacks on various datasets, and the proposed AdaEA can also boost existing transfer-based attacks, which further demonstrates its efficacy and versatility. The source code: https://github.com/CHENBIN99/AdaEA

In response to the threat of adversarial examples, adversarial training provides an attractive option for improving robustness by training models on online-augmented adversarial examples. However, most existing adversarial training methods focus on improving the model’s robust accuracy by strengthening the adversarial examples but neglecting the increasing shift between natural data and adversarial examples, leading to a decrease in natural accuracy. To maintain the trade-off between natural and robust accuracy, we alleviate the shift from the perspective of feature adaption and propose a Feature Adaptive Adversarial Training (FAAT) optimizing the class-conditional feature adaption across natural data and adversarial examples. Specifically, we propose to incorporate a class-conditional discriminator to encourage the features to become(1)class-discriminative and(2)invariant to the change of adversarial attacks. The novel FAAT framework enables the trade-off between natural and robust accuracy by generating features with similar distribution across natural and adversarial data within the same class and achieves higher overall robustness benefiting from the class-discriminative feature characteristics. Experiments on various datasets demonstrate that FAAT produces more discriminative features and performs favorably against state-of-the-art methods.