Collaborative attack, an adversarial attack method that could attack multiple modalities simultaneously, has attracted significant interest in the research community. Vision-Language Pre-training (VLP) models have shown their vulnerability to collaborative attacks, as they perform tasks through the interaction of images and texts. However, current collaborative attacks against VLP models are mainly focused on white-box and transfer-based, which require full or partial prior knowledge of victim VLP models. Besides that, VLP collaborative attacks iteratively optimize the adversarial example on one set of image-text pairs, which may ignore other potential attack targets and cause local overfitting. In this paper, we propose a reinforcement and complementary space search attack (RCSS-Attack) as a new decision-based attack for multimodal models. Based on Markov Decision Process (MDP), we adaptatively model the adversarial attack process as a policy search process, which could enlarge the search space. We also utilize step-level constraint optimization to design a complementary space search policy. Benefiting from the wide-range search capability of reinforcement learning, the RCSS-Attack uses the reward mechanism to select the optimal direction between different image-text pairs, which could relieve the overfitting problem. In the experiments, we evaluate the RCSS-Attack on 4 V+L tasks with 3 datasets and 4 metrics.

Autonomous driving systems (ADSs) have attracted wide attention in the machine learning communities. With the help of deep neural networks (DNNs), ADSs have shown both satisfactory performance under significant uncertainties in the environment and the ability to compensate for system failures without external intervention. However, the vulnerability of ADSs has raised concerns since DNNs have been proven vulnerable to adversarial attacks. In this paper, we present a comprehensive survey of current physical adversarial vulnerabilities in ADSs. We first divide the physical adversarial attack methods and defense methods by their restrictions of deployment into three scenarios: the real-world, simulator-based, and digital-world scenarios. Then, we consider the adversarial vulnerabilities that focus on various sensors in ADSs and separate them as camera-based, light detection and ranging (LiDAR) based, and multifusion-based attacks. Subsequently, we divide the attack tasks by traffic elements. For the physical defenses, we establish the taxonomy with reference to input image preprocessing, adversarial example detection, and model enhancement for the DNN models to achieve full coverage of the adversarial defenses. Based on the above survey, we finally discuss the challenges in this research field and provide further outlook on future directions.

Adversarial attacks on deep neural networks (DNNs) have raised significant concerns, particularly in safety-critical applications such as autonomous driving. Autonomous vehicles rely on both vision and LiDAR sensors to provide accurate 3D visual perception of their surroundings. However, adversarial vulnerabilities in these models pose several risks, as they can lead to misinterpretation of sensor data, ultimately endangering safety. While substantial research has been devoted to image-level adversarial attacks, these efforts are predominantly confined to 2D-pixel spaces, lacking physical realism and applicability in the 3D world. To address these limitations, we introduce AdvNeRF, a groundbreaking approach for generating 3D adversarial meshes that effectively target both vision and LiDAR models simultaneously. AdvNeRF is a Transferable Target Adversarial Attack that leverages Neural Radiance Fields (NeRF) to achieve its objectives. NeRF ensures the creation of high-quality adversarial objects and enhances attack performance by maintaining consistency across unseen viewpoints, making the adversarial examples robust from multiple angles. By integrating NeRF, our method represents a leap forward in improving the robustness and effectiveness of 3D adversarial attacks. Experimental results validate the superior performance of AdvNeRF, demonstrating its ability to degrade the accuracy of 3D object detectors under various conditions. These findings highlight the critical implications of AdvNeRF, emphasizing its potential to consistently undermine the perception systems of autonomous vehicles across different perspectives, thus marking an advancement in the field of adversarial attacks and 3D perception security.

The adversarial examples have demonstrated the vulnerability of machine learning models. While the data augmentation strategy has been a cornerstone in circumventing overfitting within the realm of standard training paradigms, the prior research has proved the limited efficiency of data augmentation in ameliorating overfitting concerns, specifically within the scope of adversarial training. In this work, we have shown that a data augmentation strategy could enhance the generalization of adversarial training. Our framework, Adversarial Denoising-based Training(ADT), first utilizes the adversarial denoising strategy to improve the generalization of adversarial training. Specifically, we use patch-wise adversarial denoising as a data augmentation strategy to boost the robustness of adversarial training. We have evaluated our method on the MNIST, CIFAR-10, Tiny-ImageNet, and GTSRB datasets. The result shows that our framework could successfully improve the adversarial robustness of models against both digital and physical adversarial examples.

Adversarial attacks deceive deep neural networks by adding subtle perturbations to benign examples, threatening various applications. However, traditional methods lack effective transferability to unknown black-box networks. Addressing this, we introduce a new method, Patch-based Transfer Attack (PTA), comprising Sensitive Area Localization (SAL) and Robust Perturbations Generation (RPG). SAL identifies highly activated regions, while RPG employs adversarial fine-tuning (AFT) and adversarial feature mixing (AFM) to target robust features, enhancing transferability. AFT adapts the surrogate model to current adversarial examples, and AFM combines adversarial and clean example features, compelling the focus on robust features. Our comprehensive tests demonstrate PTA’s superior transferability, outperforming existing methods, especially when integrated with other techniques.

Oracle Bone Inscriptions (OBI) are ancient hieroglyphs originated in China and are considered one of the most famous writing systems in the world. Up to now, thousands of OBIs have been discovered, which require deciphering by experts to understand their contents. Experts typically need to restore, classify, and compare each character with previous inscriptions. Although existing research can assist with one of these operations, their performance falls short of practical requirements. In this work, we propose the OraclePoints framework, which represents OBI images as hybrid neural representations comprising features of images and point sets. The image representation provides inscription appearance and character structure, while the point representation makes it easy and effective to distinguish characters and noises. In addition, we demonstrate that OraclePoints can be easily integrated with existing models in a plug-and-play manner. Comprehensive experiments demonstrate that the proposed hybrid neural representation framework supports a range of OBI tasks, including character image retrieval, recognition, and denoising. It is also demonstrated that OraclePoints is helpful for deciphering OBIs by linking ancient characters to modern Chinese characters. Our codes are available at https://ddghjikle.github.io/.