While Convolutional Neural Networks (CNNs) have been the de-facto architecture for computer vision tasks, recent works have shown that self-attention-based models, such as the Vision Transformer (ViT), have achieved competitive results on various visual benchmarks compared to CNNs. More recently, MLP-Mixer replaces the self-attention layer with a spatial MLP block as a token-mixing module. It attains good scores on image classification tasks when training on large datasets with strong regularization. In this paper, we propose a Local Context MLP-like architecture (LocoMixer). Unlike MLP-Mixer, where global spatial information is captured for information exchange through token-mixing MLP, we primarily focus on the local dependencies between the patch embeddings. Our LocoMixer independently encodes the spatial features along the rows and columns of 2D image patches with fully-connected layers. Meanwhile, we apply a scale-transformation operation to learn multi-scale feature representations in our proposed multiscale token-MLP. Furthermore, we further model local cross-channel interaction by absorbing a context block into the proposed dynamic channel-MLP. When only trained on the ImageNet-1K dataset, the proposed LocoMixer achieves state-of-the-art performace compared with other MLP-like models. Moreover, the proposed LocoMixer achieves 82.3% top-1 accuracy with only 26M parameters, which is much better than most CNNs and vision Transformers under the same model size constraint. When scaling up to 90M parameters, LocoMixer achieves 84.2% top1 accuracy, which outperforms the state-of-the-art Deformable Attention Transformer (DAT).

Recently, vision Transformers (ViTs) have achieved competitive performance on many computer vision tasks. However, vision Transformers show impaired performance on small datasets when training from scratch compared with Convolutional Neural Networks (CNNs), which is interpreted as the lack of locality inductive bias. This impedes the application of vision Transformers for small-size datasets. In this work, we propose Adaptive Hybrid Vision Transformer (AHVT) as the solution to boost the performance of vision Transformers on small-scale datasets. Specifically, on spatial dimension, we exploit a Convolutional Overlapping Patch Embedding (COPE) layer to inject desirable inductive bias in model, forcing the model to learn the local token features. On channel dimension, we insert a adaptive channel features aggregation block into vanilla feed forward network to calibrate channel responses. Meanwhile, we add several extra learnable “cardinality tokens” to patch token sequences to capture cross-channel interaction. We present extensive experiments to validate the effectiveness of our method on five small/medium datasets including CFAR101100, SVHN, Tiny-ImageNet and ImageNet-lk. Our approach attains state-of-the-art performance on above four small datasets when training from scratch.

Blackbox transfer attacks for image classifiers have been extensively studied in recent years. In contrast, little progress has been made on transfer attacks for object detectors. Object detectors take a holistic view of the image and the detection of one object (or lack thereof) often depends on other objects in the scene. This makes such detectors inherently context-aware and adversarial attacks in this space are more challenging than those targeting image classifiers. In this paper, we present a new approach to generate context-aware attacks for object detectors. We show that by using co-occurrence of objects and their relative locations and sizes as context information, we can successfully generate targeted mis-categorization attacks that achieve higher transfer success rates on blackbox object detectors than the state-of-the-art. We test our approach on a variety of object detectors with images from PASCAL VOC and MS COCO datasets and demonstrate up to 20 percentage points improvement in performance compared to the other state-of-the-art methods.

Deep Neural Networks (DNNs) have been shown to be vulnerable to adversarial examples, which are slightly perturbed input images which lead DNNs to make wrong predictions. To protect from such examples, various defense strategies have been proposed. A very recent defense strategy for detecting adversarial examples, that has been shown to be robust to current attacks, is to check for intrinsic context consistencies in the input data, where context refers to various relationships (e.g., object-to-object co-occurrence relationships) in images. In this paper, we show that even context consistency checks can be brittle to properly crafted adversarial examples and to the best of our knowledge, we are the first to do so. Specifically, we propose an adaptive framework to generate examples that subvert such defenses, namely, Adversarial attacks against object Detection that evade Context consistency checks (ADC). In ADC, we formulate a joint optimization problem which has two attack goals, viz., (i) fooling the object detector and (ii) evading the context consistency check system, at the same time. Experiments on both PASCAL VOC and MS COCO datasets show that examples generated with ADC fool the object detector with a success rate of over 85% in most cases, and at the same time evade the recently proposed context consistency checks, with a "bypassing" rate of over 80% in most cases. Our results suggest that "how to robustly model con- text and check its consistency," is still an open problem.

Vision systems that deploy Deep Neural Networks (DNNs) are known to be vulnerable to adversarial examples. Recent research has shown that checking the intrinsic consistencies in the input data is a promising way to detect adversarial attacks (e.g., by checking the object co-occurrence relationships in complex scenes). However, existing approaches are tied to specific models and do not offer generalizability. Motivated by the observation that language descriptions of natural scene images have already captured the object co-occurrence relationships that can be learned by a language model, we develop a novel approach to perform context consistency checks using such language models. The distinguishing aspect of our approach is that it is independent of the deployed object detector and yet offers very high accuracy in terms of detecting adversarial examples in practical scenes with multiple objects. Experiments on the PASCAL VOC and MS COCO datasets show that our method can outperform state-of-the-art methods in detecting adversarial attacks.

People's dependence on search engines allows various computer viruses to spread faster and stronger. Most scholars have neglected the influence of search engines on virus propagation and immunity. It is impossible to immunize all users at the same time with a huge system like social networks. So the main problem is how to pick a fixed-scale node cluster as the source of immunity in the network, which can make other individuals immune and continue to spread (called immune seeds). The immune seeds are scattered on some web pages of search engines to reduce the network virus infection rate. We establish two models, one is the model of computer virus early propagation based on the search engine, and the other is the model of the virus propagation and immunization model. Then we propose an improved immunization strategy: Community Partition immunization strategy based on the target immunization strategy. And we use four real datasets and two simulated datasets to do the simulation experiments, which shows that search engine can promote the propagation of the virus and the immune seeds, and the efficiency of the Community Partition immunization strategy is slightly higher than the target immunization strategy based on degree under the same conditions.