Pre-trained vision-language models (VLMs) exhibit significant vulnerability to imperceptible adversarial perturbations. Current advanced defense strategies typically employ adversarial prompt tuning to improve the adversarial robustness of VLMs, which struggle to simultaneously maintain generalization across both natural and adversarial examples under different benchmarks and downstream tasks. We propose a collaborative adversarial prompt tuning (CoAPT) approach from pre-trained VLMs to target robust VLMs. Inspired by the image mask modeling, we adopt an improved real-time total variation algorithm to suppress and eliminate high-frequency details from images while preserving edge structures, thereby disrupting the adversarial perturbation space. Subsequently, guided by the high-level image and text representations in the latent space of the pre-trained VLMs, the corrupted natural features are restored while inheriting the superior generalization capability. Experiments on four benchmarks demonstrate that CoAPT achieves an excellent trade-off among natural generalization, adversarial robustness, and task-specific adaptation compared to state-of-the-art methods.

A reliable driving trajectory prediction of surrounding vehicles is an essential reference for decision-making and safe driving of an autonomous vehicle. Although predicting short-term trajectories can be well achieved, it is still very challenging for long-term prediction of trajectories since the prediction space grows exponentially. In this paper, we propose a novel architecture for trajectory prediction from factored intention estimation (I2T), which decouples the trajectory prediction space into a high-level space for intention estimation and a low-level space for motion prediction. The long-term dependencies between intention cues and future motions during driving are naturally extended to the internal sharing mechanism of I2T, leading to improved performance. Furthermore, we design a Prioriformer model to serve as the backbone network for I2T so that it can accurately capture the long-term dependency couplings related to the task of intention estimation or motion prediction. Prioriformer model adopts a personalized normalization method, which facilitates learning latent representations of long-term features and avoids getting stuck on local optimum. A designed multi-scale fusion encoder extracts features from various receptive fields and then learns richer information from the representation subspaces. An efficient non-autoregressive decoder reduces the pressure in long-term prediction of trajectories while avoiding cumulative errors. Experiments on three real-world motion datasets show that I2T can significantly outperform the state-of-the-art.

Accurately forecasting the future motions of road participants is essential for proactive hazard avoidance and safety planning of autonomous vehicles. Existing methods for motion prediction based on probabilistic generative models are limited to low-accuracy likelihood calculations and relatively finite mode distributions. Recent studies show that score-based models can naturally overcome these limitations. In this work, we present a novel paradigm of conditional score-based models for vehicle motion prediction, called Motion-CSM. First, we model scene contextual representations of interaction regions at the feature level via graph convolutional networks. We then interpolate these representations as conditions into the solution process of the continuous-time reverse stochastic differential equation (SDE) to guide trajectory generation, which progressively converts the known prior distributions into multimodal trajectories including the ground truth modes. The designed stacked Transformer structure with dual control conditions is adopted to learn the score function approximation of the Gaussian perturbation kernel. Finally, we develop multiple consistency constraints to align the inference results of Motion-CSM in reverse SDE solving to improve the self-consistency and stability of multimodal trajectory generation. Experimental results on the real-world motion dataset demonstrate that the multimodal forecasting accuracy of Motion-CSM outperforms state-of-the-art methods.