Current adversarial attacks pose a serious threat to the robustness of visual-language models (VLMs), including vision-language pre-trained models (VLPMs) and multimodal large language models (MLLMs). Traditional adversarial attacks are example-specific and rely on specific datasets. This practice suffers from low transferability and additional computation cost, while universal adversarial perturbations (UAPs) offer example-agnostic solutions by generalizing across inputs. However, current UAP methods mainly target VLPMs, demonstrating limited transferability and effectiveness in MLLMs. To bridge this gap, we propose the Recursive Perturbation Attack (RPA), a novel black-box UAP method for both VLPMs and MLLMs. RPA employs a recursive perturbations strategy, utilizing token filtering and polynomial sampling methods to generate perturbations, thereby achieving incremental disruption and enhancing the transferability of the attack. To further enhance the effectiveness of the attack, RPA integrates a three-tier modality decoupling strategy, disentangling intra-modal, cross-modal, and fusion-modal features to effectively disrupt feature alignment and interactions. Extensive experiments validate that RPA achieves superior attack performance compared to existing UAP approaches. This work highlights new security concerns in multimodal AI systems and provides insights into the design of more robust models. Code is available at https://github.com/chilljudaoren/RPAttack.

Although adversarial training (AT) is currently one of the most promising methods to make deep neural networks adversarially robust, it suffers from the issue of robust overfitting and thus aggravates the robust generalization gap between the training and testing dataset. At the same time, data augmentations (DAs) are considered to be powerful tools for improving model generalization in standard training; however, they have been observed by many previous studies to be ineffective when applied in AT. In this paper, we try to break this prejudice and focus on improving the robust generalization ability of AT by DAs alone. We first take a close look at the effect of DAs in the adversarial training process and find that compared to common DAs, mixing-based augmentations (i.e.,MixUpandCutMix) can effectively prevent robust overfitting in AT. Then, after revisiting these two mixing-based DAs we found that they can be complementary and we can subtly stimulate the effectiveness ofMixUpandCutMixin improving the robust generalization of AT by a joint mixing manner. To this end, we propose a joint mixing-based augmentation scheme, namedMix2Aug, for improving robust generalization of AT and ultimately improving model robustness. Experimental results show that ourMix2Augcan significantly increase the upper limit ofMixUpandCutMixwithout the need of additional ensemble techniques, achieving state-of-the-art accuracy and robustness on extensive datasets.

With the rapid development of Large Language Models (LLMs), an increasing number of Large Visual-Language Models (LVLMs) have achieved unprecedented performance in response generation. Recent work shows that LVLMs are vulnerable to adversarial attacks. However, many existing methods tend to overfit to the source model by overemphasizing specific features, which compromises their transferability. Other approaches suffer from reduced attack effectiveness due to insufficient differentiation between features. In this paper, we propose a novel transfer-based black-box untargeted attack—Shared Adversarial Feature (SAF) dynamic attack. By exploring the feature extraction patterns of LVLMs, we identify the features shared among various models that are most susceptible to adversarial attacks and disrupt them. Moreover, due to the powerful attention mechanisms of LVLMs, they are still able to extract similar semantics from perturbed images, even when primary features are disrupted. We design a dynamic update strategy to address this challenge. Finally, from the perspective of SAF, we conduct an in-depth analysis of vulnerabilities in the vision encoder and projector within LVLMs and find that attacking the projector exhibits stronger transferability across heterogeneous model architectures. Extensive experiments show that our method exhibits superior attack performance compared to existing methods across different models, datasets, and tasks. The code will be publicly available after publication.

Deep-neural-network-based autonomous driving perception systems in the Industrial Internet of Things remain vulnerable to adversarial attacks despite their critical role in Industry 4.0. While numerous studies have investigated adversarial attacks on individual perception tasks such as monocular depth estimation or object detection in static situations, real-world complex industrial scenarios introduce two understudied challenges: 1) dynamic scenarios with moving objects and changing viewpoints and 2) simultaneous attacks across multiple perception tasks. In this article, we make three key contributions to address these challenges. First, we introduce a systematic study of environment-aware adversarial attacks in dynamic scenarios by introducing three adversarial attack tasks: object away attack, object close attack, and object creation attack. Second, we propose a novel dynamic scene-oriented adversarial road patch generation framework that accounts for real-world environmental variations. Third, we develop a comprehensive technical framework featuring: improved adversarial loss functions, a dynamic optimization architecture, and an integrated approach combining expectation over transformation with advanced physical augmentation optimization. Extensive experimental results demonstrate the robustness of our proposed method in digital, simulation, and real-world physical domains, as well as under different weather conditions.

Vision-Language Pretrained (VLP) models exhibit strong multimodal understanding and reasoning capabilities, finding wide application in tasks such as image-text retrieval and visual grounding. However, they remain highly vulnerable to adversarial attacks, posing serious reliability concerns in safety-critical scenarios. We observe that existing adversarial examples optimization methods typically rely on individual features from the other modality as guidance, causing the crafted adversarial examples to overfit that modality's learning preferences and thus limiting their transferability. In order to further enhance the transferability of adversarial examples, we propose a novel adversarial attack framework, I&CA (Individual & Common feature Attack), which simultaneously considers individual features within each modality and common features cross-modal interactions. Concretely, I&CA first drives divergence among individual features within each modality to disrupt single-modality learning, and then suppresses the expression of common features during cross-modal interactions, thereby undermining the robustness of the fusion mechanism. In addition, to prevent adversarial perturbations from overfitting to the learning bias of the other modality, which may distort the representation of common features, we simultaneously introduce augmentation strategies to both modalities. Across various experimental settings and widely recognized multimodal benchmarks, the I&CA framework achieves an average transferability improvement of 6.15% over the state-of-the-art DRA method, delivering significant performance gains in both cross-model and cross-task attack scenarios.

Abstract Deep Neural Networks (DNNs) have demonstrated outstanding performance in various medical image processing tasks. However, recent studies have revealed a heightened vulnerability of medical DNNs to adversarial attacks compared to their natural counterparts. In this work, we present a novel perspective by analyzing the disparities between medical datasets and natural datasets, specifically focusing on the dataset collection process. Our analysis uncovers unique differences in the data distribution across different image classes in medical datasets, a phenomenon absent in natural datasets. To gain deeper insights into medical datasets, we employ Fourier analysis tools to investigate medical DNNs. Intriguingly, we discover that high-frequency components in medical images exhibit stronger associations with corresponding labels compared to those in natural datasets. These high-frequency components distract the attention of medical DNNs, rendering them more susceptible to adversarial images. To mitigate this vulnerability, we propose a preprocessing technique called Removing High-frequency Components (RH) training. Our experimental results demonstrate that the application of RH training significantly enhances the robustness of medical DNNs against adversarial attacks. Notably, in certain scenarios, RH training even outperforms traditional adversarial training methods, particularly when subjected to black-box attacks.

Deep Neural Networks (DNNs) are notoriously susceptible to adversarial examples. To mitigate the impact of well-designed adversarial attacks on network models, researchers have developed various defense mechanisms, among which adversarial training has emerged as one of the most effective strategies to date. Adversarial training aims to augment training data with adversarial examples, thus giving DNNs a certain degree of robustness to defend against adversarial attacks. However, while obtaining adversarial robustness, this method comes at the cost of reducing the generalization performance, manifested in the reduced classification effect of clean test datasets. Researchers have been actively seeking to counter the balance between adversarial robustness and model generalization. We believe that the key to balancing these two aspects lies in identifying appropriate adversarial examples. Overly potent examples can lead to a decline in clean accuracy, whereas weaker examples may offer limited robustness. Based on our analysis, a new adversarial example generation algorithm called Denoising Projection Gradient Descent (DPGD) was proposed. DPGD adds a purification module and a constraint in generating adversarial examples, the former is used to limit the influence of too strong adversarial examples on model training and the latter is used to ensure the necessary attack intensity. Combining DPGD with the framework of traditional adversarial training, we obtain the Diffusion Adversarial Training (DifAT) approach. To verify the effectiveness of our proposed method, we conducted extensive experiments on benchmark datasets, including CIFAR-10, CIFAR-100, and Tiny-Imagenet. Our results demonstrate the effectiveness of DifAT in improving the robustness of DNNs while maintaining or even improving their generalization performance.

Deep neural networks (DNNs) find extensive applications, including object detection in various security domains. However, these DNN models are susceptible to backdoor attacks. While significant research has been conducted on backdoor attacks in classified models, limited attention has been given to object detection models. Previous studies have predominantly focused on backdoor attacks in digital environments, overlooking real-world implications. Notably, the efficacy of backdoor attacks in real-world scenarios can be significantly influenced by physical factors such as distance and illumination. In this article, we introduce a variable-size backdoor trigger designed to accommodate objects of different sizes, mitigating disruptions arising from varying distances between the viewing point and the targeted object. Additionally, we propose malicious adversarial training for backdoor training, enabling the backdoor object detector to learn trigger features amidst physical noise. Experimental results demonstrate that our robust backdoor attack (RBA) enhances the success rate of attacks in real-world settings.

Pruning offers an efficient approach to compressing models deployed on resource-constrained devices. In this paper, we introduce a novel method called Dual-Coupledness Object Detection Pruning (DCODP), specifically designed for object detection models. Taking into account the complexity of model coupling, our algorithm utilizes a depth-first search approach to identify interlayer coupling within the model. It then groups sublayers with the same parent layer together. Filters corresponding to feature maps with strong coupling are pruned within the layer, and the same pruning operation is applied to the corresponding indices in other coupled layers. In order to prove the validity of our method, extensive experiments are conducted on PASCAL VOC2007, PASCAL VOC2012 and MS COCO2017. The results show that our DCODP achieves a significant reduction of 50% in parameters and an average of more than 70% impressive score.

Vision-language pretraining (VLP) models have demonstrated outstanding performance in image-text understanding tasks but remain highly susceptible to transferable adversarial attacks. While ensemble-based guided attacks improve adversarial transferability by increasing the diversity of image-text pairs, they primarily rely on spatial-domain data augmentation, which can lead to model overfitting to image details and limit the generalization capability of attacks. To address this limitation, this study proposes a frequency-domain adjustment-based adversarial attack method that modifies specific frequency components of input images to reduce detail interference and enhance the stability of adversarial examples. Additionally, a fine-grained feature extraction technique is introduced to optimize image-text alignment, further improving the transferability of cross-modal attacks. Experimental results demonstrate that the proposed method achieves superior attack transferability and generalization performance across two major VLP architectures, fusion models and alignment models, as well as multiple tasks on the Flickr30 K and MSCOCO datasets.

Deep neural networks (DNNs) are vulnerable to adversarial examples crafted by well-designed perturbations. This could lead to disastrous results on critical applications such as self-driving cars, surveillance security, and medical diagnosis. At present, adversarial training is one of the most effective defenses against adversarial examples. However, in traditional adversarial training, it is still difficult to achieve a good trade-off between clean accuracy and robustness since DNNs still learn spurious features. The intrinsic reason is that traditional adversarial training makes it difficult to fully learn core features from adversarial examples when noise and examples cannot be disentangled. In this paper, we disentangle the adversarial examples into natural and perturbed patterns by bit-plane slicing. We assume the higher bit-planes represent natural patterns and the lower bit-planes represent perturbed patterns, respectively. We propose Feature-Focusing Adversarial Training (F$^{2}$AT), which differs from previous work in that it enforces the model to focus on the core features from natural patterns and reduce the impact of spurious features from perturbed patterns. The experimental results demonstrated that the clean accuracy and adversarial robustness with our F$^{2}$AT can be significantly improved.

Deep neural networks are vulnerable to adversarial examples, where decision-based attacks can generate adversarial examples based solely on the predicted labels. However, these attacks typically require excessive queries to attack one example. Considering this challenge, we propose FBA (Frequency based Boundary Attack), a decision-based attack against the limitation of query efficiency. FBA incorporates a novel search process, utilizing high-frequency based importance sampling for efficient gradient estimation. Empirical results confirm the superior query efficiency of our method. Specifically, FBA surpasses SOTA attacks by achieving a 54% average improvement in query efficiency, quantified by the reduction in perturbation size within the same number of queries.

In today’s power transmission grids, Internet-of-Things networks employ long-haul optical wires for regular sensor data dissemination to a server. Ensuring resilience against link faults is paramount to observe the grid states accurately via a process known as state estimation (SE). The accuracy is achieved by minimizing the end-to-end failure rate in packet delivery (EEFR). Current approaches focus on hop-by-hop retransmission control with in-path caching. Notably, the disruption-resilient transport protocol (DRTP) stands out for achieving the lowest EEFR. DRTP employs robust hop-by-hop retransmission and a recursive collaboration process guided by arrival timeouts. However, challenges arise in maintaining recursiveness with timeouts, leading to increased EEFR due to cache mismatch. These intensify when a hop triggers arrival timeouts, spawning retransmission instances in an unexpected sequence, which can experience an unprotected parallel race condition. To address this, we propose RSDD, a resilient mechanism for sensor data dissemination for implementing DRTP in the correct and fully verified manner. RSDD orchestrates concurrent retransmission instances, ensuring exclusive execution for the same lost packet, precisely scheduled based on timeouts. We evaluated the performance of RSDD in a simulated network that combines SE and a grid, using ndnSIM, MATPOWER, and RTDS. The results validate RSDD as a correct DRTP implementation, highlighting its exclusiveness and quality-of-service performance. RSDD achieves an EEFR of 2.44% and an average end-to-end packet delivery time (EEDT) of 2.7 ms during full path disruption with a 20% link loss rate in packets. Moreover, RSDD excels in enabling SE to maintain the grid observability and accuracy.

Recent studies have shown that Deep Neural Networks (DNNs) are easily deceived by adversarial examples, revealing their serious vulnerability. Due to the transferability, adversarial examples can attack across multiple models with different architectures, called transfer-based black-box attacks. Input transformation is one of the most effective methods to improve adversarial transferability. In particular, the attacks fusing other categories of image information reveal the potential direction of adversarial attacks. However, the current techniques rely on input transformations in the spatial domain, which ignore the frequency information of the image and limit its transferability. To tackle this issue, we propose Mixed-Frequency Inputs (MFI) based on a frequency domain perspective. MFI alleviates the overfitting of adversarial examples to the source model by considering high-frequency components from various kinds of images in the process of calculating the gradient. By accumulating these high-frequency components, MFI acquires a more steady gradient direction in each iteration, leading to the discovery of better local maxima and enhancing transferability. Extensive experimental results on the ImageNet-compatible datasets demonstrate that MFI outperforms existing transform-based attacks with a clear margin on both Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), which proves MFI is more suitable for realistic black-box scenarios.

Dynamic link prediction aims to predict future connections among unconnected nodes in a network. It can be applied for friend recommendations, link completion, and other tasks. Network representation learning algorithms have demonstrated considerable effectiveness in various prediction tasks. However, most network representation learning algorithms are based on homogeneous networks and static networks for link prediction that do not consider rich semantic and dynamic information. Additionally, existing dynamic network representation learning methods neglect the neighborhood interaction structure of the node. In this work, we design a neighbor-enhanced dynamic heterogeneous attributed network embedding method (NeiDyHNE) for link prediction. In light of the impressive achievements of the heuristic methods, we learn the information of common neighbors and neighbors’ interaction in heterogeneous networks to preserve the neighbors proximity and common neighbors proximity. NeiDyHNE encodes the attributes and neighborhood structure of nodes as well as the evolutionary features of the dynamic network. More specifically, NeiDyHNE consists of the hierarchical structure attention module and the convolutional temporal attention module. The hierarchical structure attention module captures the rich features and semantic structure of nodes. The convolutional temporal attention module captures the evolutionary features of the network over time in dynamic heterogeneous networks. We evaluate our method and various baseline methods on the dynamic link prediction task. Experimental results demonstrate that our method is superior to baseline methods in terms of accuracy.

Convolutional neural networks (CNNs) have been successfully applied to various fields. However, CNNs' overparameterization requires more memory and training time, making it unsuitable for some resource-constrained devices. To address this issue, filter pruning as one of the most efficient ways was proposed. In this article, we propose a feature-discrimination-based filter importance criterion, uniform response criterion (URC), as a key component of filter pruning. It converts the maximum activation responses into probabilities and then measures the importance of the filter through the distribution of these probabilities over classes. However, applying URC directly to global threshold pruning may cause some problems. The first problem is that some layers will be completely pruned under global pruning settings. The second problem is that global threshold pruning neglects that filters in different layers have different importance. To address these issues, we propose hierarchical threshold pruning (HTP) with URC. It performs a pruning step limited in a relatively redundant layer rather than comparing the filters' importance across all layers, which can avoid some important filters being pruned. The effectiveness of our method benefits from three techniques: 1) measuring filter importance by URC; 2) normalizing filter scores; and 3) conducting prune in relatively redundant layers. Extensive experiments on CIFAR-10/100 and ImageNet show that our method achieves the state-of-the-art performance on multiple benchmarks.

Recent work shows that well-designed adversarial examples can fool deep neural networks (DNNs). Due to their transferability, adversarial examples can also attack target models without extra information, called black-box attacks. However, most existing ensemble attacks depend on numerous substitute models to cover the vulnerable subspace of a target model. In this work, we find three types of models with non-overlapping vulnerable frequency regions, which can cover a large enough vulnerable subspace. Based on this finding, we propose a lightweight ensemble adversarial attack named LEA2, integrated by standard, weakly robust, and robust models. Moreover, we analyze Gaussian noise from the perspective of frequency and find that Gaussian noise is located in the vulnerable frequency regions of standard models. Therefore, we substitute standard models with Gaussian noise to ensure the use of high-frequency vulnerable regions while reducing attack time consumption. Experiments on several image datasets indicate that LEA2achieves better transferability under different defended models compared with extensive baselines and state-of-the-art attacks.

RGB–thermal scene parsing has recently attracted increasing research interest in the field of computer vision. However, most existing methods fail to perform good boundary extraction for prediction maps and cannot fully use high-level features. In addition, these methods simply fuse the features from RGB and thermal modalities but are unable to obtain comprehensive fused features. To address these problems, we propose an edge-aware guidance fusion network (EGFNet) for RGB–thermal scene parsing. First, we introduce a prior edge map generated using the RGB and thermal images to capture detailed information in the prediction map and then embed the prior edge information in the feature maps. To effectively fuse the RGB and thermal information, we propose a multimodal fusion module that guarantees adequate cross-modal fusion. Considering the importance of high-level semantic information, we propose a global information module and a semantic information module to extract rich semantic information from the high-level features. For decoding, we use simple elementwise addition for cascaded feature fusion. Finally, to improve the parsing accuracy, we apply multitask deep supervision to the semantic and boundary maps. Extensive experiments were performed on benchmark datasets to demonstrate the effectiveness of the proposed EGFNet and its superior performance compared with state-of-the-art methods. The code and results can be found at https://github.com/ShaohuaDong2021/EGFNet.

Adversarial training is by far one of the most effective methods to improve the robustness of deep neural networks against adversarial examples. However, the trade-off between robustness and accuracy is still a challenge in adversarial training. Previous methods used adversarial examples with a fixed perturbation budget or specific perturbation budgets for each example, which is inefficient in improving the trade-off and lacks the ability to control the trade-off flexibly. In this paper, we show that the largest element of logit, [Formula: see text], can roughly represent the minimum distance between an example and its neighboring decision boundary. Thus, we propose group adaptive adversarial training (GAAT) that divides the training dataset into several groups based on [Formula: see text] and develops a binary search algorithm to determine the group perturbation budgets for each group. Using the group perturbation budgets to perform adversarial training can fine-tune the trade-off between robustness and accuracy. Extensive experiments conducted on CIFAR-10 and ImageNet-30 show that our GAAT can achieve a more perfect trade-off than TRADES, MMA, and MART.

Edge intelligence has played an important role in constructing smart cities, but the vulnerability of edge nodes to adversarial attacks becomes an urgent problem. A so-called adversarial example can fool a deep learning model on an edge node for misclassification. Due to the transferability property of adversarial examples, an adversary can easily fool a black-box model by a local substitute model. Edge nodes in general have limited resources, which cannot afford a complicated defense mechanism like that on a cloud data center. To address the challenge, we propose a dynamic defense mechanism, namely EI-MTD. The mechanism first obtains robust member models of small size through differential knowledge distillation from a complicated teacher model on a cloud data center. Then, a dynamic scheduling policy, which builds on a Bayesian Stackelberg game, is applied to the choice of a target model for service. This dynamic defense mechanism can prohibit the adversary from selecting an optimal substitute model for black-box attacks. We also conduct extensive experiments to evaluate the proposed mechanism, and results show that EI-MTD could protect edge intelligence effectively against adversarial attacks in black-box settings.

RGB‐infrared (RGB‐IR) person reidentification is a challenge problem in computer vision due to the large crossmodality difference between RGB and IR images. Most traditional methods only carry out feature alignment, which ignores the uniqueness of modality differences and is difficult to eliminate the huge differences between RGB and IR. In this paper, a novel AGF network is proposed for RGB‐IR re‐ID task, which is based on the idea of global and local alignment. The AGF network distinguishes pedestrians in different modalities globally by combining pixel alignment and feature alignment and highlights more structure information of person locally by weighting channels with SE‐ResNet‐50, which has achieved ideal results. It consists of three modules, including alignGAN module (A), crossmodality paired‐images generation module (G), and feature alignment module (F). First, at pixel level, the RGB images are converted into IR images through the pixel alignment strategy to directly reduce the crossmodality difference between RGB and IR images. Second, at feature level, crossmodality paired images are generated by exchanging the modality‐specific features of RGB and IR images to perform global set‐level and fine‐grained instance‐level alignment. Finally, the SE‐ResNet‐50 network is used to replace the commonly used ResNet‐50 network. By automatically learning the importance of different channel features, it strengthens the ability of the network to extract more fine‐grained structural information of person crossmodalities. Extensive experimental results conducted on SYSU‐MM01 dataset demonstrate that the proposed method favorably outperforms state‐of‐the‐art methods. In addition, we evaluate the performance of the proposed method on a stronger baseline, and the evaluation results show that a RGB‐IR re‐ID method will show better performance on a stronger baseline.

Many studies have revealed the vulnerability of deep neural networks (DNNs) in the face of adversarial attacks. By adding a small perturbation to the input, adversarial attacks could fool many advanced models for computer vision, speech recognition and natural language processing tasks, posing severe security threats to DNNs. In this paper, we proposed a gradient-based word-level attack method in the embedding space to attack text classification models. This method computes the significance of each word and chooses the optimal word for substitution after word embedding; the generated adversarial texts have little semantic changes but could successfully fool classification DNNs. Through extensive experiments, we confirmed that the generated adversarial texts could achieve a success rate approaching 100% with a very low word substitution rate in attacking the WordCNN and LSTM models on three datasets. By human evaluation, the adversarial texts evaded human notice which implied little semantic changes were made. Experiments on different models also confirmed the transferability of the adversarial texts. Finally, we adopted adversarial training, and this improved the models' generalization capacity and robustness.