Autonomous vehicles depend on accurate trajectory prediction to navigate safely in complex traffic. Yet current models are vulnerable to stealthy backdoor attacks: an adversary embeds subtle, physically plausible triggers during training that remain latent until activated. To address this risk, we introduce a structured framework categorizing four trigger types—spatial, kinetic (braking), coordinated, and composite—and demonstrate on two benchmarks (nuScenes and Argoverse 2) and two state-of-the-art architectures (Autobot and Wayformer) that poisoning as little as 5% of training samples can reliably hijack future predictions. We further propose a real-time defense leveraging social attention: by encoding agent histories, computing cross-attention to the target vehicle, and filtering out agents with anomalously high weights, our method neutralizes backdoor triggers without degrading clean-data accuracy. Comprehensive experiments show our defense reduces attack success rates across diverse urban scenarios—intersections, roundabouts, multi-lane roads—highlighting both the severity of backdoor threats and a promising pathway to secure trajectory predictors in autonomous driving systems.

In contrast to vast academic efforts to study AI security, few real-world reports of AI security incidents exist. Released incidents prevent a thorough investigation of the attackers' motives, as crucial information about the company and AI application is missing. As a consequence, it often remains unknown how to avoid incidents. We tackle this gap and combine previous reports with freshly collected incidents to a small database of 32 AI security incidents. We analyze the attackers' target and goal, influencing factors, causes, and mitigations. Many incidents stem from non-compliance with best practices in security and privacy-enhancing technologies. In the case of direct AI attacks, access control may provide some mitigation, but there is little scientific work on best practices. Our paper is thus a call for action to address these gaps.

Abstract Protecting ML classifiers from adversarial examples is crucial. We propose that the main threat is an attacker perturbing a confidently classified input to produce a confident misclassification. We consider in this paper the $$L_0$$ L 0 attack in which a small number of inputs can be perturbed by the attacker at test-time. To quantify the risk of this form of attack we have devised a formal guarantee in the form of an adversarial bound (AB) for a binary, Gaussian process classifier using the EQ kernel. This bound holds for the entire input domain, bounding the potential of any future adversarial attack to cause a confident misclassification. We explore how to extend to other kernels and investigate how to maximise the bound by altering the classifier (for example by using sparse approximations). We test the bound using a variety of datasets and show that it produces relevant and practical bounds for many of them.

Despite the large body of academic work on machine learning security, little is known about the occurrence of attacks on machine learning systems in the wild. In this paper, we report on a quantitative study with 139 industrial practitioners. We analyze attack occurrence and concern and evaluate statistical hypotheses on factors influencing threat perception and exposure. Our results shed light on real-world attacks on deployed machine learning. On the organizational level, while we find no predictors for threat exposure in our sample, the amount of implement defenses depends on exposure to threats or expected likelihood to become a target. We also provide a detailed analysis of practitioners’ replies on the relevance of individual machine learning attacks, unveiling complex concerns like unreliable decision making, business information leakage, and bias introduction into models. Finally, we find that on the individual level, prior knowledge about machine learning security influences threat perception. Our work paves the way for more research about adversarial machine learning in practice, but yields also insights for regulation and auditing.

The recent lottery ticket hypothesis proposes that there is at least one sub-network that matches the accuracy of the original network when trained in isolation.Recent work shows that under SGD noise, several such tickets emerge.We build on these works and study how winning tickets derived from one fixed network differ in structural and functional terms under varying levels of stochasticity.Structurally, we show that the Hamming distance of winning tickets' shapes follow the hypergeometric distribution.Functionally, our experiments validate that different emerging winning tickets are not disguised variants of each other, but diverge also concerning their classification outputs.Last but not least, different regimes of stochasticity affect winning tickets.Decreasing randomness during training also decreases the tickets' functional and structural distance.

In machine learning (ML) security, attacks like evasion, model stealing or membership inference are generally studied in individually. Previous work has also shown a relationship between some attacks and decision function curvature of the targeted model. Consequently, we study an ML model allowing direct control over the decision surface curvature: Gaussian Process Classifiers (GPCs). For evasion, we find that changing GPC's curvature to be robust against one attack algorithm boils down to enabling a different norm or attack algorithm to succeed. This is backed up by our formal analysis showing that static security guarantees are opposed to learning. Concerning intellectual property, we show formally that lazy learning does not necessarily leak all information when applied. In practice, often a seemingly secure curvature can be found. For example, we are able to secure GPC against empirical membership inference by proper configuration. In this configuration, however, the GPC's hyper-parameters are leaked, e.g. model reverse engineering succeeds. We conclude that attacks on classification should not be studied in isolation, but in relation to each other.