Remarkable advancements have been made in the field of object detection, and given its widespread application, it is of paramount importance to investigate the robustness of detection models. However, previous methods have primarily focused on models based on Convolutional Neural Networks (CNNs), seriously neglecting the Transformer-based models that develop rapidly but exhibit obvious differences in terms of information processing. Therefore, this paper aims to address this gap by exploring potential attacks arising from the self-attention mechanism inhered in Transformer. Specifically, we propose a novel adversarial attack scenario targeting Transformer-based object detection models, where only objects of specific class fail to be detected, while irrelevant objects remain undisturbed. Therefore, human perception is hard to find errors even with the detector fail. To achieve this goal, we introduce an adversarial patch generation method, termed Class-Specific Adversarial (CSAdv) patches, which simultaneously leverages class probability to attack specific objects and utilizes the output from Transformer decoder structures, Query Output, to protect irrelevant objects. Due to the long-range interactions of Transformer, the adversarial patch does not need to directly cover or closely surround the specific objects. Instead, it achieves remote targeted attacks simply by being placed in the corner of image, which greatly enhances the concealment of patches. Extensive experiments are conducted on various benchmark datasets and Transformer-based baselines, and the experimental results show that CSAdv can effectively mask certain class while keeping other classes as unaffected as far as possible.

Deep-learning-based (DL-based) face recognition has become an important application in the Internet of Things (IoT) environment. However, recent studies demonstrate that elaborate adversarial examples can mislead the results of DL-based face recognition on mobile and edge devices. Such vulnerability threats the robustness of face recognition systems and causes security issues. Generative adversarial defense methods can reform adversarial examples before input into the face recognition model to improve the accuracy under adversarial attacks. Unfortunately, the existing generative adversarial defense methods cannot completely remove the misleading features of adversarial examples due to the lack of robust encoding ability. In this article, we propose a local consistency generative adversarial network (LC-GAN) framework by adding the constraint of local consistency to force the encoder to mine consistent features in each local area, achieving robust encoding ability consequently. The framework includes three main novel designs. First, we present a patch-wise contrastive learning-based refinement stage with local consistency loss to encode robust identity features from nonsalient areas that are undamaged by adversarial attacks. Second, we use a powerful expert network to guide the training of LC-GAN for eliminating adversarial identity features. Third, we design a multilevel identity loss to enhance the identity preservation ability by unifying the local and global identity features. Experimental results on four widely used face data sets show that LC-GAN outperforms other generative adversarial defense methods.

Recent on-manifold adversarial attacks can mislead gait recognition by generating adversarial walking postures (AWP) with image generation techniques. However, existing defense methods only eliminate adversarial perturbations on each frame isolatedly but ignore the temporal correlation of gait sequence, which leads to vulnerability of robust gait recognition. In this paper, we propose GaitReload, a post-processing adversarial defense method to defend against AWP for the gait recognition model with sequenced inputs. First, GaitReload utilizes sequenced entity recognition (SER) module to detect the adversarial frames by the temporal constraints of gait sequence. Then, we apply bayesian uncertainty filtering-based (BUF-based) gait interpolation to reform adversarial gait examples. After that, we reload the reformed gait sequence and rectify the recognition results with the guidance of reloading strategy. Specifically, SER has a bi-directional frame difference attention and a temporal feature aggregation to boost the detection performance. For training SER, we apply hidden posture selective attack (HPSA) to generate training samples. The extensive experimental results on CASIA-A, CASIA-B, and OU-ISIR demonstrate that GaitReload can defend against adversarial gait by large margins in both RGB and silhouette modes.

Adversarial training can defend against universal adversarial perturbation (UAP) by injecting corresponding adversarial samples during training. However, adversarial samples used by existing methods, such as UAP, inevitably include excessive perturbations related to other categories due to its inherent goal of universality. Training with them will cause more erroneous predictions with larger local positive curvature. In this paper, we propose a curvature-aware category adversarial training method to avoid excessive perturbations. We introduce the category-oriented adversarial masks that are synthesized with class distinctive momentum. Besides, we split the min-max optimization loops of adversarial training into two parallel processes to reduce the training cost. Experimental results on CIFAR-10 and ImageNet show that our method achieves better defense accuracy under UAP with less training cost than state-of-the-art baselines.

Adversarial training can boost the robustness of the model by aligning discriminative features between natural and generated adversarial samples. However, the generated adversarial samples tend to have more features derived from changed patterns in other categories along with the training process, which prevents better feature alignment between natural and adversarial samples. Unfortunately, existing adversarial training methods ignore such dynamicity of generated adversarial samples. In this paper, we propose Adversarial Contrastive Decoupling (ACD) to filter the features derived from changed patterns. Specificity, we decouple the changed patterns from adversarial samples and then extract robust representations from remaining features. First, we introduce a decoupling module with a dynamic labeling strategy to explore the dynamicity of generated adversarial samples. Then, we propose a siamese network with contrastive learning mechanism to align remaining robust representations between adversarial and natural samples. Extensive experimental results demonstrate the superior performance of ACD over baselines.

Recent studies show that image retrieval systems are vulnerable to adversarial attacks that adds imperceptible noise to the images. But the imperceptibility of noise limits the attack performance. As a commonly acceptable interference, watermark often appears in images, which can be treated as ‘‘imperceptible’’ but is unexploited. In this paper, we propose a Watermark Attack method on Image Retrieval systems (WAIR). Attacking retrieval systems is challenging due to black-box model, the absence of confidence guide, and attack failures and low efficiency caused by the randomness of traditional evolutionary algorithms. To solve these challenges, we propose a new evolutionary algorithm called Gene Joint Selecting Algorithm (GJSA) that jointly optimize the watermark parameters. We also design the Fitness Record Table (FRT), a new data structure that records the historical attack effect to guide the following evolution and avoid local optimal solutions. Thanks to FRT, WAIR can also reduce the duplicate searching caused by algorithm randomness. The extensive experiments show that WAIR can attack the black-box image retrieval system with a successful rate of 0.806, which is 12.4% higher than the traditional evolutionary algorithm. Moreover, for attacking commercial image retrieval system, we achieve 2$\times$ higher attack success rate on Baidu Image Retrieval API than the existing methods.

Deep learning-based person re-identification (Re-ID) methods have achieved the significant performance of matching person images across camera views, which plays an important role in the construction of the smart city. Recent works of adversarial attacks have explored the serious vulnerability of deep Re-ID systems. However, existing attacks are performed with idealized conditions and ignore the real-world environments, such as occlusion caused by walking habits. In this paper, we propose a two-stage method to perform an occlusion resilient adversarial attack for better evaluation of deep Re-ID systems. Specifically, we construct the occlusion template from the observation and statics of pedestrian walking habits. Then, we design a partition training strategy for a better combination of occluded and exposed adversarial patches. During the training, we introduce contextual loss to penalize the semantic distance of attacked images with the same identity. The extensive experiments on Market1501 demonstrate the performance of our method.

Deep neural networks are easily affected by adversarial patches, causing prediction errors. However, existing adversarial patches are trained with specific model-dataset pairs and only effective for the images with the predetermined size in the dataset. The semantic information of the patch will be distorted when scaling the image, which is a common preprocessing process in practical applications. In this paper, we propose SRA Patch (Scaling-Resilient Adversarial Patch), a new adversarial patch resilient to image scaling. Specifically, we generate the patch in a block-wise way and utilize the superpixel method to resist the loss of semantic information during scaling. Further, we introduce the ensemble model as a Black-Box indicator to address the noise space shrinking issue, which is caused by the small size and the block operation of SRA patch. Finally, we leverage Class Activation Mapping to extract the region with salient features as the final patch to improve the ratio of effective semantic features on the patch to remain during scaling. Extensive experiments have demonstrated that our SRA patch has much stronger attack capability and scaling robustness than existing methods.

Weakly Supervised Object Detection (WSOD) is a challenging visual understanding task due to the absence of expensive human annotations like bounding boxes and segmentation. Recent WSOD methods usually generate a bounding box for the most contrastive part of the object rather than the entire object. Some methods alleviate the incompleteness problem of detection object with segmentation supplement or classifier refinement. However, segmentation requires high model cost and classifier refinement highly relies on the quality of initial candidate boxes at the beginning. In this paper, we propose a WSOD classifier refinement approach to overcome candidate boxes initialization and high segmentation model cost problems. Our approach can get high-quality class-specific activation maps for the objects and generate nail boxes at the maximum response point of the activation map to suppress incorrect refinement direction. Compared with previous methods, our WSOD classifier refinement approach can achieve 42.1% mAP on PASCAL VOC 2007 benchmarks without high segmentation model cost.

In the field of video surveillance, the demands of intelligent video analysis services based on Deep Neural Networks (DNNs) have grown rapidly. Although most existing studies focus on the performance of DNNs pre-deployed at remote clouds, the network delay caused by computation offloading from network cameras to remote clouds is usually long and sometimes unbearable. Edge computing can enable rich services and applications in close proximity to the network cameras. However, owing to the limited computing resources of distributed edge clouds, it is challenging to satisfy low latency and high accuracy requirements for all users, especially when the number of users surges. To address this challenge, we first formulate the intelligent video surveillance task scheduling problem that minimizes the average response time while meeting the performance requirements of tasks and prove that it is NP-hard. Second, we present an adaptive DNN model selection method to identify the most effective DNN model for each task by comparing the feature similarity between the input video segment and pre-stored training videos. Third, we propose a two-stage delay-aware graph searching approach that presents a beneficial trade-off between network delay and computing delay. Experimental results demonstrate the efficiency of our approach.