Neural architecture search (NAS) has been widely adopted to design high-accuracy architectures, which are often vulnerable against adversarial attacks. To address this problem, existing robust NAS methods mainly focus on optimizing both natural accuracy and adversarial robustness in a fixed supernet, which is designed for natural accuracy. As a result, the derived architectures have the same construction scheme as the supernet, suffering from limited adversarial robustness and flexibility. In this article, we present the ARNAS++ method to search for accurate and robust neural architectures via a supernet with flexible parameter budgets and width. Specifically, we propose a parameter budget controlling loss to make architectures contain less parameters in the rear cells, based on which the adversarial robustness can be guaranteed. Moreover, we also propose a learnable filter number reduction ratio to control the filter numbers in the supernet, which can find more robust architectures beyond the fixed supernet, and make the supernet more flexible at the same time. We conduct experiments on six widely used benchmark datasets against the state of the art. The experimental results demonstrate that the proposed ARNAS++ method outperforms the competitors in terms of both natural accuracy and adversarial robustness under various popular adversarial attacks. In addition, the ablation studies show the effectiveness of the designed components and their positive contributions to the overall performance. The source code is available at: https://github.com/fyqsama/ARNASpp.

Deep neural networks (DNNs) easily cause security issues due to the lack of adversarial robustness. An emerging research topic for this problem is to design adversarially robust architectures via neural architecture search (NAS), i.e., robust NAS. However, robust NAS needs to train numerous DNNs for robustness estimation, making the search process prohibitively expensive. In this paper, we propose a zero-cost proxy to evaluate the adversarial robustness without training. Specifically, the proposed zero-cost proxy formulates the upper bound of adversarial loss, which can directly reflect the adversarial robustness. The formulation involves only the initialized weights of DNNs, thus the training process is no longer needed. Moreover, we theoretically justify the validity of the proposed proxy based on the theory of neural tangent kernel and input loss landscape. Experimental results show that the proposed zero-cost proxy can bring more than $20\times$ speedup compared with the state-of-the-art robust NAS methods, while the searched architecture has superior robustness and transferability under white-box and black-box attacks. Furthermore, compared with the state-of-the-art zero-cost proxies, the calculation of the proposed method has the strongest correlation with adversarial robustness. Our source code is available at https://github.com/fyqsama/Robust_ZCP.

To defend deep neural networks from adversarial attacks, adversarial training has been drawing increasing attention for its effectiveness. However, the accuracy and robustness resulting from the adversarial training are limited by the architecture, because adversarial training improves accuracy and robustness by adjusting the weight connection affiliated to the architecture. In this work, we propose ARNAS to search for accurate and robust architectures for adversarial training. First we design an accurate and robust search space, in which the placement of the cells and the proportional relationship of the filter numbers are care-fully determined. With the design, the architectures can obtain both accuracy and robustness by deploying accurate and robust structures to their sensitive positions, re-spectively. Then we propose a differentiable multi-objective search strategy, performing gradient descent towards directions that are beneficial for both natural loss and adversar-ial loss, thus the accuracy and robustness can be guaran-teed at the same time. We conduct comprehensive experiments in terms of white-box attacks, black-box attacks, and transferability. Experimental results show that the searched architecture has the strongest robustness with the compet-itive accuracy, and breaks the traditional idea that NAS-based architectures cannot transfer well to complex tasks in robustness scenarios. By analyzing outstanding architectures searched, we also conclude that accurate and robust neural architectures tend to deploy different structures near the input and output, which has great practical significance on both hand-crafting and automatically designing of accurate and robust architectures.