Accurate trajectory prediction is vital for various applications, including autonomous vehicles. However, the complexity and limited transparency of many prediction algorithms often result in black-box models, making it challenging to understand their limitations and anticipate potential failures. This further raises potential risks for systems based on these prediction models. This study introduces the performance monitor for black-box trajectory prediction model (PMBP) to address this challenge. The PMBP estimates the performance of black-box trajectory prediction models online, enabling informed decision-making. The study explores various methods’ applicability to the PMBP, including anomaly detection, machine learning, deep learning, and ensemble, with specific monitors designed for each method to provide online output representing prediction performance. Comprehensive experiments validate the PMBP’s effectiveness, comparing different monitoring methods. Results show that the PMBP effectively achieves promising monitoring performance, particularly excelling in deep learning-based monitoring. It achieves improvement scores of 0.81 and 0.79 for average prediction error and final prediction error monitoring, respectively, outperforming previous white-box and gray-box methods. Furthermore, the PMBP’s applicability is validated on different datasets and prediction models, while ablation studies confirm the effectiveness of the proposed mechanism. Hybrid prediction and autonomous driving planning experiments further show the PMBP’s value from an application perspective. Project page: https://swb19.github.io/PMBP/.Note to Practitioners—This research presents PMBP, a valuable tool for practitioners in the automation industry. The PMBP enables online monitoring of black-box trajectory prediction models, enhancing system reliability and facilitating informed decision-making. The practical application of PMBP lies in improving safety and reliability in critical domains, especially in the context of autonomous vehicles. Black-box trajectory prediction models commonly used in these domains may exhibit unexpected deficiencies, potentially leading to risks. By monitoring the prediction performance online, systems can proactively identify potential insufficiencies and make informed decisions to ensure safer and more reliable operations. The PMBP offers practitioners different monitoring solutions based on various approaches, addressing their specific needs effectively. While the PMBP has shown promising outcomes, further exploration and testing are necessary to fully harness and apply its monitoring results in automated systems. Practitioners are encouraged to adopt the PMBP as an essential monitoring mechanism to enhance the reliability of their trajectory prediction models and achieve safer and more efficient automation in their domains.

Remote sensing image change detection has important applications in many fields. However, current studies mostly focus on identifying pixel-level changes. Although these methods can achieve better performance, this paradigm fails to determine changes in specific object instances due to the definition of the task itself. For this reason, we conduct preliminary exploration and propose a method named OBJ-CD, which can detect the changes of object instance. Specifically, OBJ-CD initially employs a lightweight Siamese object detector to detect objects within two temporal images. Subsequently, OBJ-CD calculates the metric matrix for the detected objects in these images. Finally, the conditions of the object instance change are limited by a certain threshold, and the final object-level change detection results can be obtained. We conducted several experiments on our constructed dataset, and the experimental results indicate that the proposed method can achieve object-level change detection with good performance.

In the rapidly evolving field of autonomous driving, reliable prediction is pivotal for vehicular safety. However, trajectory predictions often deviate from actual paths, particularly in complex and challenging environments, leading to significant errors. To address this issue, our study introduces a novel method for Dynamic Occupancy Set (DOS) prediction, it effectively combines advanced trajectory prediction networks with a DOS prediction module, overcoming the shortcomings of existing models. It provides a comprehensive and adaptable framework for predicting the potential occupancy sets of traffic participants. The innovative contributions of this study include the development of a novel DOS prediction model specifically tailored for navigating complex scenarios, the introduction of precise DOS mathematical representations, and the formulation of optimized loss functions that collectively advance the safety and efficiency of autonomous systems. Through rigorous validation, our method demonstrates marked improvements over traditional models, establishing a new benchmark for safety and operational efficiency in intelligent transportation systems.

Autonomous driving confronts great challenges in complex traffic scenarios, where the SOTIF risk can be triggered by the dynamic operational environment and system insufficiencies. The SOTIF risk is reflected not only intuitively in the collision risk with objects outside the autonomous vehicles, but also inherently in the performance limitation risk of the implemented algorithms. How to minimize the SOTIF risk for autonomous driving is currently a critical, difficult, and unresolved issue. Therefore, this paper proposes the “Self-Surveillance and Self-Adaption System” as a systematic approach to online minimize the SOTIF risk, which aims to provide a systematic solution for monitoring, quantification, and mitigation of inherent and external risks. As a demonstration of the system, the risk monitoring of the perception algorithm is highlighted. Moreover, the inherent perception algorithm risk and external collision risk are jointly quantified via SOTIF entropy, which is then propagated downstream to the decision-making module and mitigated. Finally, Hardware-in-the-Loop experiments are conducted to verify the efficiency and effectiveness of the system. The results demonstrate that the system enables dependable online monitoring, quantification, and mitigation of SOTIF risk in real-time critical traffic environments.

Motion prediction is essential for safe and efficient autonomous driving. However, the inexplicability and uncertainty of complex artificial intelligence models may lead to unpredictable failures of the motion prediction module, which may mislead the system to make unsafe decisions. Therefore, it is necessary to develop methods to guarantee reliable autonomous driving, where failure detection is a potential direction. Uncertainty estimates can be used to quantify the degree of confidence a model has in its predictions and may be valuable for failure detection. We propose a framework of failure detection for motion prediction from the uncertainty perspective, considering both motion uncertainty and model uncertainty, and formulate various uncertainty scores according to different prediction stages. The proposed approach is evaluated based on different motion prediction algorithms, uncertainty estimation methods, uncertainty scores, etc., and the results show that uncertainty is promising for failure detection for motion prediction but should be used with caution.

Perception algorithms in autonomous driving systems confront great challenges in long-tail traffic scenarios, where the problems of Safety of the Intended Functionality (SOTIF) could be triggered by the algorithm performance insufficiency and dynamic operational environment. However, such scenarios are not systematically included in current open-source datasets, and this paper fills the gap accordingly. Based on the analysis and enumeration of trigger conditions, a high-quality diverse dataset is released, including various long-tail traffic scenarios collected from multiple resources. Considering the development of probabilistic object detection (POD), this dataset marks trigger sources that may cause perception SOTIF problems in the scenarios as key objects. In addition, an evaluation protocol is suggested to verify the effectiveness of POD algorithms in identifying the key objects via uncertainty. The dataset never stops expanding, and the first batch of open-source data includes 1126 frames with an average of 2.27 key objects and 2.47 normal objects in each frame. To demonstrate how to use this dataset for SOTIF research, this paper further quantifies the perception SOTIF entropy to confirm whether a scenario is unknown and unsafe for a perception system. The experimental results show that the quantified entropy can effectively and efficiently reflect the failure of the perception algorithm.

Trajectory prediction is one of the key components of the autonomous driving software stack. Accurate prediction for the future movement of surrounding traffic participants is an important prerequisite for ensuring the driving efficiency and safety of intelligent vehicles. Trajectory prediction algorithms based on artificial intelligence have been widely studied and applied in recent years and have achieved remarkable results. However, complex artificial intelligence models are uncertain and difficult to explain, so they may face unintended failures when applied in the real world. In this paper, a self-aware trajectory prediction method is proposed. By introducing a self-awareness module and a two-stage training process, the original trajectory prediction module's performance is estimated online, to facilitate the system to deal with the possible scenario of insufficient prediction function in time, and create conditions for the realization of safe and reliable autonomous driving. Comprehensive experiments and analysis are performed, and the proposed method performed well in terms of self-awareness, memory footprint, and real-time performance, showing that it may serve as a promising paradigm for safe autonomous driving.

Accurate trajectory prediction is essential for safe and efficient autonomous driving in complex traffic environments. While artificial intelligence has shown great promise in improving prediction accuracy, its inherent uncertainty and lack of explainability may lead to unpredictable failures, creating challenges for safety-critical decision-making. This study aims to address these challenges by exploring the impact of traffic environment on prediction algorithms. The study proposes a trajectory prediction framework with epistemic uncertainty estimation ability to output high uncertainty when facing unforeseeable or unknown scenarios. The framework analyzes the environmental effect on the trajectory prediction by considering scenario features and shifts. Features are divided into kinematic features of a target agent, features of surrounding traffic participants, and other scenario features. Feature correlation and importance analyses are performed to study their influence on prediction error and epistemic uncertainty. The impact of unavoidable distributional shifts in the real world on trajectory predictions is investigated using multiple intersection datasets. The results indicate that deep ensemble-based methods have advantages in improving robustness while estimating epistemic uncertainty. Consistent conclusions were obtained from the correlation and importance analyses, indicating that kinematic features of the target agent have relatively strong effects on both prediction error and epistemic uncertainty. Finally, the study analyzes the accuracy deterioration caused by distributional shifts and the potential of the deep ensemble-based method. Through deep ensemble, the errors of the prediction methods based on GRIP++ and Trajectron++ have been improved by 6.4% and 10.8% in the same-dataset test, and 6.3% and 10.8% in the cross-dataset test.

Motion prediction modules are crucial for autonomous vehicles to forecast the future behavior of surrounding road users. Failures in prediction modules can mislead a downstream planner to make unsafe decisions. Currently, deep learning technology has been widely used to design prediction models due to its impressive performance. However, such models may fail in long-tail driving scenarios where the training data are insufficient or unavailable, which represents the so-called epistemic uncertainty of prediction models. This paper proposes a risk-aware decision-making (RADM) framework to handle the epistemic uncertainty arising from training the prediction model on insufficient data. First, a multi-agent prediction network with epistemic uncertainty quantification is proposed. This network uses the historical states of nearby road users, map information, and traffic lights as inputs. Then, the RADM utilizes model predictive control technique to not only process the multi-agent prediction results but also to consider the epistemic uncertainty of the prediction model. In addition, the accuracy of the established prediction model is verified on real-world driving datasets. Furthermore, the proposed RADM is evaluated on the log-replay data obtained from real-world driving logs and using the SUMO simulator, considering multiple challenging cases where pedestrians and non-motorized vehicles cross the intersection illegally. The experimental results demonstrate that RADM can reduce the driving risk and improve driving safety and supplementary videos are provided athttps://github.com/SOTIF-AVLab/RADM.

As the development direction of intelligent driving in the future, the research of key technologies has made significant progress. However, due to the recent unmanned accidents, there are concerns about safety performance. To solve the safety problem, an intended safety systems for intelligent driving was proposed. This system provides security analysis and monitoring services in real time for intended problems with smart car perception, decision and control modules. Based on the concept of safety of the intended functionality, the driving scene and system safety are analyzed and evaluated to improve the safety of intelligent driving, which may help the development of intelligent driving.