The well-established modular autonomous driving system is decoupled into different standalone tasks, e.g. perception, prediction and planning, suffering from information loss and error accumulation across modules. In contrast, end-to-end paradigms unify multi-tasks into a fully differentiable framework, allowing for optimization in a planning-oriented spirit. Despite the great potential of end-to-end paradigms, both the performance and efficiency of existing methods are not satisfactory, particularly in terms of planning safety. We attribute this to the computationally expensive BEV (bird's eye view) features and the straightforward design for prediction and planning. To this end, we explore the sparse representation and review the task design for end-to-end autonomous driving, proposing a new paradigm named SparseDrive. Concretely, SparseDrive consists of a symmetric sparse perception module and a parallel motion planner. The sparse perception module unifies detection, tracking and online mapping with a symmetric model architecture, learning a fully sparse representation of the driving scene. For motion prediction and planning, we review the great similarity between these two tasks, leading to a parallel design for motion planner. Based on this parallel design, which models planning as a multi-modal problem, we propose a hierarchical planning selection strategy, which incorporates a collision-aware rescore module, to select a rational and safe trajectory as the final planning output. With such effective designs, SparseDrive surpasses previous state-of-the-arts by a large margin in performance of all tasks, while achieving much higher training and inference efficiency.

Effective, reliable, and efficient measurement of autonomous driving safety performance is essential for demonstrating its trustworthiness. Criticality metrics offer an objective assessment of autonomous driving safety. However, the wide variety of criticality metrics, each with distinct characteristics, lacks a unified standard for evaluation and selection. We contend that a criticality metric should accurately reflect the true danger level of vehicle pairs at risk. This paper focuses on scene-level criticality metrics and proposes a necessary criterion: a robust criticality metric should accurately distinguish between ‘collision-unavoidable’ and ‘collision-avoidable’ states. To achieve this, we employ Monte Carlo sampling to systematically explore the state space of two-vehicle conflict scenes (>106samples) and use intention-sharing Distributed Model Predictive Control (DMPC) to determine the ground truth of collision states. We analyze failure cases of three classical and two state-of-the-art scene-level criticality metrics and quantify their performance using the Receiver Operating Characteristic (ROC) method. Our approach has the potential to establish a necessary standard for evaluating criticality metrics, facilitating accurate assessment, analysis, and enhancement of autonomous vehicle safety.

Evolutionary algorithms (EAs) have shown promise in solving sequential decision problems (SDPs) by simplifying them to static optimization problems and searching for the optimal policy parameters in a zeroth-order way. While these methods are highly versatile, they often suffer from high sample complexity due to their ignorance of the underlying temporal structures. In contrast, reinforcement learning (RL) methods typically formulate SDPs as Markov Decision Process (MDP). Although more sample efficient than EAs, RL methods are restricted to differentiable policies and prone to getting stuck in local optima. To address these issues, we propose a novel evolutionary framework Zeroth-Order Actor-Critic (ZOAC). We propose to use step-wise exploration in parameter space and theoretically derive the zeroth-order policy gradient. We further utilize the actor-critic architecture to effectively leverage the Markov property of SDPs and reduce the variance of gradient estimators. In each iteration, ZOAC employs samplers to collect trajectories with parameter space exploration, and alternates between first-order policy evaluation (PEV) and zeroth-order policy improvement (PIM). To evaluate the effectiveness of ZOAC, we apply it to a challenging multi-lane driving task, optimizing the parameters in a rule-based, non-differentiable driving policy that consists of three sub-modules: behavior selection, path planning, and trajectory tracking. We also compare it with gradient-based RL methods on three Gymnasium tasks, optimizing neural network policies with thousands of parameters. Experimental results demonstrate the strong capability of ZOAC in solving SDPs. ZOAC significantly outperforms EAs that treat the problem as static optimization and matches the performance of gradient-based RL methods even without first-order information, in terms of total average return across all tasks.

Model Predictive Control (MPC) method is widely used in autonomous vehicle control technology. The adjustment of MPC weights is crucial for optimizing its control performance, ensuring precise and reliable operation. Traditionally, these weights are adjusted manually, which is inefficient. This study introduces a novel Butterfly Optimization Algorithm (BOA) learning-based method to determine the optimal MPC weights in an efficient way. By adopting the data-driven idea in machine learning, the trajectory data of field experiment human drivers is used to train the controller weights. A simulation-based training platform that enables the automatic training of the MPC controller with varying weights is also developed. Simulation results demonstrate the superior control accuracy and stability performance of BOA learning-based method compared to Linear Quadratic Regulator (LQR) and pure pursuit strategies. The findings suggest that the control method proposed in this research can significantly improve autonomous vehicle control performance and their reliability, thereby contributing to the advancement of autonomous driving technology.

We focus on learning the zero-constraint-violation safe policy in model-free reinforcement learning (RL). Existing model-free RL studies mostly use the posterior penalty to penalize dangerous actions, which means they must experience the danger to learn from the danger. Therefore, they cannot learn a zero-violation safe policy even after convergence. To handle this problem, we leverage the safety-oriented energy functions to learn zero-constraint-violation safe policies and propose the safe set actor-critic (SSAC) algorithm. The energy function is designed to increase rapidly for potentially dangerous actions, locating the safe set on the action space. Therefore, we can identify the dangerous actions prior to taking them and achieve zero-constraint violation. Our major contributions are twofold. First, we use the data-driven methods to learn the energy function, which releases the requirement of known dynamics. Second, we formulate a constrained RL problem to solve the zero-violation policies. We prove that our Lagrangian-based constrained RL solutions converge to the constrained optimal zero-violation policies theoretically. The proposed algorithm is evaluated on the complex simulation environments and a hardware-in-loop (HIL) experiment with a real autonomous vehicle controller. Experimental results suggest that the converged policies in all environments achieve zero-constraint violation and comparable performance with model-based baseline.

Safety certificates based on energy functions can provide demonstrable safety for complex robotic systems. However, all recent studies on learning-based energy function synthesis only consider the feasibility of the control policy, which might cause over-conservativeness and even fail to achieve the control goal. To solve the problem of over-conservative controllers, we proposed the magnitude regularization technique to improve the controller performance of safe controllers by reducing the conservativeness inside the energy function, while keeping the promising provable safety guarantees. Specifically, we quantify the conservativeness by the magnitude of the energy function, and we reduce the conservativeness by adding a magnitude regularization term to the synthesis loss. We propose an algorithm using reinforcement learning (RL) for synthesis to unify the learning process of safe controllers and energy functions. We conducted simulation experiments on Safety Gym and real-robot experiments using small quadrotors. Simulation results show that the proposed algorithm does reduce the conservativeness of the energy function and outperforms baselines in terms of controller performance while maintaining safety. Real-robot experiments have shown that the proposed algorithm indeed reduce conservativeness on the small quadrotors.

In real traffic environment, a single control mode of traditional autonomous vehicles cannot meet various driving requirements for different drivers, which will decrease the acceptance of autonomous vehicles, and even may further cause traffic risks. This paper studies the cooperative strategies between ego vehicle and surrounding vehicles with the naturalistic experiment data, and then designs an autonomous vehicle control method based on the distributed Model Predictive Control (MPC) in order to consider the interaction relationship of ego vehicles and surrounding vehicles. Finally, the proposed method is verified by software simulation and Hardware in the Loop (HIL) simulation experiments, and the experiment results demonstrate that the control method proposed in this paper not only can control the vehicle to complete the typical driving tasks smoothly, in terms of car-following and lane-changing, but also can reflect the different cooperative strategies among different driving behavior characteristics, which can improve safety and acceptance of autonomous vehicles to promote the practical application of autonomous vehicle technology.

Multiple object tracking (MOT) is becoming increasingly significant for autonomous driving and intelligent transportation systems. However, traditional MOT methods cannot track the objects accurately and robustly due to the lack of effective feature extraction and data association in complex traffic scenarios. In this paper, we propose a novel joint detection and tracking method AttentionTrack by introducing multiple features attention. Firstly, we design a self-motivated feature extraction attention network (FEAN) to adaptively produce effective decoupled features for detection and tracking tasks in different scenarios. Secondly, we build a spatial-temporal data association (STDA) framework to achieve more accurate and robust tracking by considering the historical features of trajectory through different times. Moreover, we conduct comprehensive experiments on the KITTI, UA-DETRAC and MOT17 benchmarks, and the results show that our approach achieves competitive performance compared with the state-of-the-art (SOTA) trackers.

How to fully test the safety and functionality under different driving scenarios is a key issue for the development and application of autonomous vehicles. In this study, aimed at the test scenarios of autonomous vehicle, we propose a lidar-camera fusion approach for traffic environment sensing. Based on the successful Lift-Splat-Shoot (LSS) model, we propose a unique data enhancement strategy to develop the fusion accuracy. Through building a test dataset with the highprecision acquisition vehicle, the proposed method is verified that the new fusion authorism proposed in this paper can accurately distinguish the translation, scale, orientation and velocity of the target. This study can promote test scenario generation methods.

Trajectory prediction plays a vital role in the performance of autonomous driving systems, and prediction accuracy, such as average displacement error (ADE) or final displacement error (FDE), is widely used as a performance metric. However, a significant disparity exists between the accuracy of predictors on fixed datasets and driving performance when the predictors are used downstream for vehicle control, because of a dynamics gap. In the real world, the prediction algorithm influences the behavior of the ego vehicle, which, in turn, influences the behaviors of other vehicles nearby. This interaction results in predictor-specific dynamics that directly impacts prediction results. In fixed datasets, since other vehicles' responses are predetermined, this interaction effect is lost, leading to a significant dynamics gap. This paper studies the overlooked significance of this dynamics gap. We also examine several other factors contributing to the disparity between prediction performance and driving performance. The findings highlight the trade-off between the predictor's computational efficiency and prediction accuracy in determining real-world driving performance. In summary, an interactive, task-driven evaluation protocol for trajectory prediction is crucial to capture its effectiveness for autonomous driving. Source code along with experimental settings is available online (https://whatmatters23.github.io/).

Current 3-D object detection technology for autonomous driving usually cannot efficiently utilize local sensitive points. Meanwhile, contextual feature extracted from a object is not sufficient, which easily leads to deteriorated detection accuracy of the final object estimation. For the problems, a point–voxel-based 3-D dynamic object detection algorithm is proposed. First, local points are grouped with a camera frustum. Then, the global feature extracted by the submanifold 3-D voxel CNNs is aggregated into frustum key points. Second, a module of vector pool with feature aggregation is used to aggregate multiscale features of the point cloud. Moreover, the frustum raw feature and BEV feature are used for feature extension. Subsequently, the fine multiscale feature extracted from the point cloud is used as input to a subsequent fully convolutional network for final classification and continuous estimation of oriented 3-D boxes. The proposed method was compared with other state-of-the-art algorithms on the KITTI, Waymo, and nuScenes data sets. Experimental results showed that the proposed algorithm was better in accuracy, robustness, and generalization capabilities in 3-D dynamic object detection. Experiments on a real scenario and extensive ablation studies also demonstrated that the proposed algorithm not only effectively controls computational cost but also achieved more efficient results in 3-D object detection.

Reinforcement learning (RL) has shown excellent performance in the sequential decision-making problem, where safety in the form of state constraints is of great significance in the design and application of RL. Simple constrained end-to-end RL methods might lead to significant failure in a complex system like autonomous vehicles. In contrast, some hierarchical RL (HRL) methods generate driving goals directly, which could be closely combined with motion planning. With safety requirements, some safe-enhanced RL methods add post-processing modules to avoid unsafe goals or achieve expectation-based safety, which accepts the existence of unsafe states and allows some violations of safe constraints. However, ensuring state safety is vital for autonomous vehicles. Therefore, this paper proposes a state-based safety enhancement method for autonomous driving via direct hierarchical reinforcement learning. Finally, we design a constrained reinforcement learner based on the State-based Constrained Markov Decision Process (SCMDP), where a learnable safety module could adjust the constraint strength adaptively. We integrate a dynamic module in the policy training and generate future goals considering safety, temporal-spatial continuity, and dynamic feasibility, which could eliminate dependence on the prior model. Simulations in the typical highway scenes with uncertainties show that the proposed method has better training performance, higher driving safety in interactive scenes, more decision intelligence in traffic congestions, and better economic driving ability on roads with changing slopes.

Reinforcement learning (RL) has shown promising performance in autonomous driving applications in recent years. The early end-to-end RL method is usually unexplainable and fails to generate stable actions, while the hierarchical RL (HRL) method can tackle the above issues by dividing complex problems into multiple sub-tasks. Prior HRL works either select discrete driving behaviors with continuous control commands, or generate expected goals for the low-level controller. However, they typically have strong scenario dependence or fail to generate goals with good quality. To address the above challenges, we propose a Continuous-Lattice Hierarchical RL (Cola-HRL) method for autonomous driving tasks to make high-quality decisions in various scenarios. We utilize the continuous-lattice module to generate reasonable goals, ensuring temporal and spatial reachability. Then, we train and evaluate our method under different traffic scenarios based on real-world High Definition maps. Experimental results show our method can handle multiple scenarios. In addition, our method also demonstrates better performance and driving behaviors compared to existing RL methods.

Model information can be used to predict future trajectories, so it has huge potential to avoid dangerous regions when applying reinforcement learning (RL) on real-world tasks, like autonomous driving. However, existing studies mostly use model-free constrained RL, which causes inevitable constraint violations. This paper proposes a model-based feasibility enhancement technique of constrained RL, which enhances the feasibility of policy using generalized control barrier function (GCBF) defined on the distance to constraint boundary. By using the model information, the policy can be optimized safely without violating actual safety constraints, and the sample efficiency is increased. The infeasibility in solving the constrained policy gradient is handled by an adaptive coefficient mechanism. We evaluate the proposed method in both simulations and real vehicle experiments in a complex autonomous driving collision avoidance task. The proposed method achieves up to four times fewer constraint violations and converges 3.36 times faster than baseline constrained RL approaches.

Intelligent vehicles should be capable to understand the intention of other traffic participants when driving on urban roads. Yet, current approaches mostly emphasize the importance of the crossing/not-crossing (C/NC) problem and neglect the intention estimation task. To this end, we propose a pedestrian intention estimation method based on the extended theory of planned behavior (TPB). In contrast to previous qualitative modeling based on surveys and questionnaires, neural networks and hand-crafted rules are designed to quantitatively model the components of the extended TPB in the proposed architecture. Besides, the interaction between the components is simulated by a mixed classification strategy. Our pedestrian intention estimation model achieves 82% accuracy and outperforms the baseline method by 3% on the pedestrian intention estimation (PIE) dataset.

A multimode hybrid automaton is proposed for setting vehicle platoon modes with velocity, distance, length, lane position, and other state information. Based on a vehicle platoon shift movement under different modes, decisions are made based on key conditional actions, such as sudden acceleration changes because of vehicle distance changes, emergency braking to avoid collisions and free-lane changing choices adapted to various traffic conditions, so as to ensure effortless movement and safety in the multimode shift. With a 3-degree (longitudinal, lateral, and yaw directions) of the freedom coupled model, a hybrid vehicle platoon controller is proposed using nonsingular terminal sliding-mode control to ensure fast and steady tracking on the hybrid automaton outputs during the multimode shift process. The convergence of the hybrid controller in finite time is also analyzed with the Lyapunov exponential stability. The analysis result proves that the proposed controller not only ensures the stability of the individual vehicle and the vehicle platoon but also ensures the stability of the multimode shift movement system. The proposed cooperative driving strategy for vehicle platoon is evaluated using simulations, where varying traffic conditions and the influence of cutting off are considered in conjunction with demonstration simulations of a vehicle platoon's cruising, following, lane changing, overtaking, and moving in/out of garage functions.