Pedestrian Trajectory Prediction (PTP) aims to predict the future trajectory of pedestrians based on a historical trajectory. Transformer-based approaches have demonstrated unparalleled performance for PTP tasks, encoding long-term temporal dependencies and heterogeneous spatial interactions of pedestrians. However, Transformer often involves redundant information and noisy interactions from irrelevant regions by considering all available trajectory features. Recently, the structured state space model, Mamba has been proposed, which captures long-range dependency in sequences with a selective mechanism to filter out redundant information. To further tap into the potential of the novel Mamba architecture for the PTP task, in this paper, we presentMambaPTP, which predicts future trajectories based purely on Mamba mechanisms, to mitigate the noisy interactions of irrelevant trajectory features and avoid repetitive trajectory modeling, while maintaining high-performance trajectory prediction. Specifically, we propose a new Bidirectional Gating Mamba (BGM) module with bidirectional state space models, which leverages the sparse gate mechanism to select informative temporal patterns and spatial interactions. Moreover, we design a Bidirectional Trajectory Alignment (BTA) module towards aligning the predicted trajectory to the ground truth, ensuring that the model to learn the effective sparse feature representation of trajectories. We conduct extensive experiments on several mainstream pedestrian trajectory prediction datasets. The results demonstrate that the proposed MambaPTP achieves competitive performance compared to advanced Transformer-based models. We hope this paper can further inspire research in Mamba for the PTP task, leading to a tighter integration of the Mamba and PTP communities.

Adversarial Training (AT) has been shown to significantly enhance adversarial robustness via a min-max optimization approach. However, its effectiveness in video recognition tasks is hampered by two main challenges. First, fast adversarial training for video models remains largely unexplored, which severely impedes its practical applications. Specifically, most video adversarial training methods are computationally costly, with long training times and high expenses. Second, existing methods struggle with the trade-off between clean accuracy and adversarial robustness. To address these challenges, we introduce Video Fast Adversarial Training with Weak-to-Strong consistency (VFAT-WS), the first fast adversarial training method for video data. Specifically, VFAT-WS incorporates the following key designs: First, it integrates a straightforward yet effective temporal frequency augmentation (TF-AUG), and its spatial-temporal enhanced form STF-AUG, along with Fast Gradient Sign Method (FGSM) to boost training efficiency and robustness. Second, it devises a weak-to-strong spatial-temporal consistency regularization, which seamlessly integrates the simple TF-AUG and the more complex STF-AUG. Leveraging the consistency regularization, it steers the learning process from simple to complex augmentations. Both of them work together to achieve a better trade-off between clean accuracy and robustness. Extensive experiments on UCF-101 and HMDB-51 with both CNN and Transformer-based models demonstrate that VFAT-WS achieves great improvements in adversarial robustness and corruption robustness, while accelerating training by nearly 490%.

In recent years, adversarial attacks on video recognition models have attracted increasing attention. However, most existing strategies are extensions of image-based methods, where adversarial perturbations are computed independently and embedded into individual frames. This independent per-frame perturbation process wastes computational resources and leads to excessive query consumption. To address this problem, we introduce Frequency Domain Distributed Perturbations (FDP), a straightforward yet effective black-box video attack method using temporal correlations between video frames. Specifically, FDP first converts the input video into the frequency domain and calculates globally coordinated adversarial perturbations in the spectral space. By conducting global optimization in the frequency domain, FDP improves the effectiveness of each query, significantly decreasing the total number of queries needed. The resulting perturbations are temporally distributed across frames to preserve the spatiotemporal structure. Furthermore, we introduce a frequency-sensitive mask to identify the spectral regions most critical to the model's predictions. By applying perturbations only to these key frequency bands, FDP further reduces the perturbation search space and improves query efficiency. Extensive experiments demonstrate that our method significantly reduces query consumption while achieving higher attack success rates than state-of-the-art approaches.

Transfer-based attacks depend on commonalities between models. However, unique analysis patterns of different models cause overfitting to source models and limit transferability to the target model. In this paper, we argue that the public domain is the spatial-temporal commonality that contributes most to the recognition for most video models. For that, we propose a novel Public-Domain Locator (PDL) Attack method, which calculates adversarial perturbations on the public domain for different source models, reducing overfitting caused by the model-specific spatial-temporal analysis pattern to enhance the transferability of video attacks on the target model. To accurately localize these public domains between different models, a locator based on Multi-Agent Reinforcement Learning (MARL) is designed in our methods. The locator is trained using elaborately designed rewards received from multiple sources to adjust the public-domain selection strategy. Extensive experiments on nine mainstream video recognition models and the widely-used action recognition dataset Kinetics400 demonstrate that the proposed PDL attack outperforms state-of-the-art methods.

Adversarial robustness assessment for video recognition models has raised concerns owing to their wide applications on safety-critical tasks. Compared with images, videos have much high dimension, which brings huge computational costs when generating adversarial videos. This is especially serious for the query-based black-box attacks where gradient estimation for the threat models is usually utilized, and high dimensions will lead to a large number of queries. To mitigate this issue, we propose to simultaneously eliminate the temporal and spatial redundancy within the video to achieve an effective and efficient gradient estimation on the reduced searching space, and thus query number could decrease. To implement this idea, we design the novel Adversarial spatial-temporal Focus (AstFocus) attack on videos, which performs attacks on the simultaneously focused key frames and key regions from the inter-frames and intra-frames in the video. AstFocus attack is based on the cooperative Multi-Agent Reinforcement Learning (MARL) framework. One agent is responsible for selecting key frames, and another agent is responsible for selecting key regions. These two agents are jointly trained by the common rewards received from the black-box threat models to perform a cooperative prediction. By continuously querying, the reduced searching space composed of key frames and key regions is becoming precise, and the whole query number becomes less than that on the original video. Extensive experiments on four mainstream video recognition models and three widely used action recognition datasets demonstrate that the proposed AstFocus attack outperforms the SOTA methods, which is prevenient in fooling rate, query number, time, and perturbation magnitude at the same time.