While cooperative perception can overcome the limitations of single-vehicle systems, the practical implementation of vehicle-to-vehicle and vehicle-to-infrastructure systems is often impeded by significant economic barriers. Aerial-ground cooperation (AGC), which pairs ground vehicles with drones, presents a more economically viable and rapidly deployable alternative. However, this emerging field has been held back by a critical lack of high-quality public datasets and benchmarks. To bridge this gap, we present Griffin, a comprehensive AGC 3D perception dataset, featuring over 250 dynamic scenes (37k+ frames). It incorporates varied drone altitudes (20-60m), diverse weather conditions, realistic drone dynamics via CARLA-AirSim co-simulation, and critical occlusion-aware 3D annotations. Accompanying the dataset is a unified benchmarking framework for cooperative detection and tracking, with protocols to evaluate communication efficiency, altitude adaptability, and robustness to communication latency, data loss and localization noise. By experiments through different cooperative paradigms, we demonstrate the effectiveness and limitations of current methods and provide crucial insights for future research.

Cooperative perception is critical for autonomous driving, overcoming the inherent limitations of a single vehicle, such as occlusions and constrained fields-of-view. However, current approaches sharing dense Bird's-Eye-View (BEV) features are constrained by quadratically-scaling communication costs and the lack of flexibility and interpretability for precise alignment across asynchronous or disparate viewpoints. While emerging sparse query-based methods offer an alternative, they often suffer from inadequate geometric representations, suboptimal fusion strategies, and training instability. In this paper, we propose SparseCoop, a fully sparse cooperative perception framework for 3D detection and tracking that completely discards intermediate BEV representations. Our framework features a trio of innovations: a kinematic grounded instance query that uses an explicit state vector with 3D geometry and velocity for precise spatio-temporal alignment; a coarse-to-fine aggregation module that effectively integrates information from both matched and unmatched instances; and a cooperative instance denoising task that provides stable, abundant supervision to accelerate and stabilize training. Experiments on V2X-Seq and Griffin datasets show SparseCoop achieves state-of-the-art performance. Notably, it delivers this performance with superior computational efficiency and a highly competitive transmission cost, while showing remarkable robustness to real-world challenges like communication latency.

This paper addresses a generalization problem of Multi-Agent Pathfinding (MAPF), called Collaborative Task Sequencing - Multi-Agent Pathfinding (CTS-MAPF), where agents must plan collision-free paths and visit a series of intermediate task locations in an optimized order before reaching their final destinations. To address this problem, we propose a new approach, Collaborative Task Sequencing - Conflict-Based Search (CTS-CBS), which conducts a two-level search. In the high level, it generates a search forest, where each tree corresponds to a joint task sequence derived from the jTSP solution. In the low level, CTS-CBS performs constrained single-agent path planning to generate paths for each agent while adhering to high-level constraints. Furthermore, we integrate Explicit Estimation Search into CTS-CBS to enhance the search efficiency. We also provide theoretical guarantees of its solution completeness and optimality (or sub-optimality with a bounded parameter). To evaluate the performance of CTS-CBS, we utilize three datasets with different map sizes, map difficulty and agent/task numbers, and conduct comprehensive experiments. The results show that our algorithms demonstrate significant improvements in success rate and runtime, while maintaining competitive solution quality. Finally, practical robot simulation and tests demonstrate the algorithm’s applicability in real-world scenarios.

Safety is one of the most crucial challenges of autonomous driving vehicles, and one solution to guarantee safety is to employ an additional control revision module after the planning backbone. Control Barrier Function (CBF) has been widely used because of its strong mathematical foundation on safety. However, the incompatibility with heterogeneous perception data and incomplete consideration of traffic scene elements make existing systems hard to be applied in dynamic and complex real-world scenarios. In this study, we introduce a generalized control revision method for autonomous driving safety, which adopts both vectorized perception and occupancy grid map as inputs and comprehensively models multiple types of traffic scene constraints based on a new proposed barrier function. Traffic elements are integrated into one unified framework, decoupled from specific scenario settings or rules. Experiments on CARLA, SUMO, and OnSite simulator prove that the proposed algorithm could realize safe control revision under complicated scenes, adapting to various planning backbones, road topologies, and risk types. Physical platform validation also verifies the real-world application feasibility.

Multi-vehicle trajectory planning (MVTP) is one of the key challenges in multi-robot systems (MRSs) and has broad applications across various fields. This paper presents ESCoT, an enhanced step-based coordinate trajectory planning method for multiple car-like robots. ESCoT incorporates two key strategies: collaborative planning for local robot groups and replanning for duplicate configurations. These strategies effectively enhance the performance of step-based MVTP methods. Through extensive experiments, we show that ESCoT 1) in sparse scenarios, significantly improves solution quality compared to baseline step-based method, achieving up to 70% improvement in typical conflict scenarios and 34% in randomly generated scenarios, while maintaining high solving efficiency; and 2) in dense scenarios, outperforms all baseline methods, maintains a success rate of over 50% even in the most challenging configurations. The results demonstrate that ESCoT effectively solves MVTP, further extending the capabilities of step-based methods. Finally, practical robot tests validate the algorithm’s applicability in real-world scenarios.

Accurate and effective road boundary extraction plays a significant role in the navigation and decision-making processes of self-driving cars. Nevertheless, the reliable detection of road boundaries via 3D LiDAR is particularly difficult due to uneven point cloud density and chaotic vegetation areas. Conventional methods often require time-consuming fitting or clustering algorithms to enhance performance. To this end, this paper presents a road boundary extraction approach particularly focuses on the curbs and vegetation areas, utilizing LiDAR data without fitting or clustering, enabling the generation of detailed road maps. We exploit both single and multiple frames of data in the design, which enables spatio-temporal feature extraction. This strategy is realized by integrating the road boundary detection algorithm and the SLAM technology. The algorithm contains three stages: 1) Coarse Ground Segmentation (CGS), 2) Adaptive Spatial Feature Extraction (A-SFE), 3) Iterative Multi-scale Refinement (I-MSR). Experiments on the KITTI dataset are conducted for verification. The proposed method not only outperforms traditional methods with an average of 85% in key metrics but also demonstrates comparable performance to state-of-the-art deep learning models.

Coordinating connected and automated vehicles (CAVs) poses challenges in achieving safe and optimal pathfinding simultaneously. In this paper, we propose safer conflict-based search (Safer-CBS), an approach based on conflict-based search that generates risk-constrained optimal paths for multiple CAVs. Safer-CBS addresses these challenges by incorporating risk assessment and optimization techniques. By defining the conflict as the risk between two vehicles exceeding a predetermined threshold, the resultant conflict-free paths can provide safety guarantees. An optimization over risk constraints is devised to enhance the efficiency of the tree search. Additional improvements are also incorporated to expedite the search process. Theoretical analysis demonstrates the optimality of Safer-CBS in finding risk-constrained paths. Furthermore, Safer-CBS can seamlessly integrate trajectory planning to generate risk-constrained trajectories for CAVs. Numerical calculations on grid maps validate the effectiveness of the proposed optimization and additional improvements, resulting in increased success rates (by over 43% for 10 vehicles) and reduced runtime (by over 59%) when compared to the method without the optimization and improvements for Safer-CBS. Simulations in both unstructured and structured environments confirm that Safer-CBS, combined with trajectory planning, ensures that trajectory risks strictly remain below predetermined thresholds, while achieving a superior balance between safety and efficiency compared to the benchmarks.Note to Practitioners—Applying multi-CAV technologies in areas such as warehousing, transportation, and operations typically requires the planning of cooperative paths or trajectories for the CAVs. In this planning process, safety and efficiency issues are centrally considered and balanced. Cooperative performance can be ensured as long as each individual vehicle adheres to the central instructions. This study develops a method for risk-constrained cooperative planning, aiming to generate optimal paths and suboptimal trajectories for multiple CAVs. The output trajectories consist of path points with fixed time intervals, which the vehicle controllers can track accordingly. By maintaining the risk level among the vehicles below predetermined thresholds, overall safety can be achieved. Meanwhile, the planning process is optimized, thus improving overall efficiency.

Odometry in adverse weather conditions, such as fog, rain, and snow, presents significant challenges, as traditional vision- and LiDAR-based methods often suffer from degraded performance. Radar-Inertial Odometry (RIO) has emerged as a promising solution due to its resilience in such environments. In this paper, we present RINO, a non-iterative RIO framework implemented in an adaptively loosely coupled manner. Building upon ORORA as the baseline for radar odometry, RINO introduces several key advancements, including improvements in keypoint extraction, motion distortion compensation, and pose estimation via an adaptive voting mechanism. This voting strategy facilitates efficient polynomial-time optimization while simultaneously quantifying the uncertainty in the radar module’s pose estimation. The estimated uncertainty is subsequently integrated into the maximum a posteriori (MAP) estimation within a Kalman filter framework. Unlike prior loosely coupled odometry systems, RINO not only retains the global and robust registration capabilities of the radar component but also dynamically accounts for the real-time operational state of each sensor during fusion. Experimental results conducted on publicly available datasets demonstrate that RINO reduces translation and rotation errors by 1.06% and 0.09°/100m, respectively, when compared to the baseline method, thus significantly enhancing its accuracy. Furthermore, RINO achieves performance comparable to state-of-the-art methods. Our code is available at https://github.com/yangsc4063/rino.

Cooperative planning of Connected and Autonomous vehicles (CAVs) is a promising way to reshape the intelligent transportation system, and planning under scenarios mixed with human-driven vehicles is one of the critical challenges. Many existing studies have proposed vehicle group planning methods towards mixed traffic scenes. By regarding the reward of all CAVs as a unified entity, the overall average driving performance was improved. However, one limitation is that during the collective planning process, individual demands are sacrificed for lack of considering agent personalized preferences. To balance the group common benefit and individual diversity, this paper proposes a novel cooperative multi-vehicle planning method combining collective decision-making and individual preference evaluation. First, a bi-level collective planning framework is designed including region-driven behavior selection and conflict-free trajectory generation. To further consider personalized features, an individual preference evaluation system is established based on Social Value Orientation. Then the evaluation results are merged into the group planning process so that vehicles can generate various decisions according to personal demands. Experimental results show that the driving personalization levels of safety, time efficiency, and comfort are increased by 1.28%, 3.78%, and 8.80%, correspondingly.

Decision-making is one of the critical modules of autonomous driving, and how to evaluate its performance precisely remains a challenge. Although some current evaluation methods have considered multiple aspects of driving experience and been able to give a comprehensive evaluation result, the emphasis on individual personalization diversity is still limited. Existing methods mainly fit the evaluation model to an average human model, neglecting the individual demand features. In this paper, we propose a novel integrated decision-making evaluation method considering individual personalization diversity. First, we build an integrated evaluation model to represent the average human evaluation. Based on the four fundamental single-factor model including safety, time efficiency, comfort, and energy consumption, a segmental linear model is applied to combine various elements into one unified framework. Then a personalized weight fluctuation mechanism is proposed which adjusts the relative term weights in the integrated model dynamically according to the users’ preferences. Finally, a corresponding online individual demand distribution estimation method is designed to assess human personal diversity. The experiments on the D2E dataset prove that the proposed evaluation system can better adapt to individual preference diversity, reducing the evaluation mean absolute error by 7.40%. The higher the degree of personalization for the users, the greater the improvement this method can generate.

Deploying a certain trajectory tracking control algorithm to a different autonomous vehicle is time-consuming as most control parameters need to be fine-tuned, which necessitates a series of experiments and computations to ascertain the appropriate parameters considering factors such as tire stiffness, inertia, wheelbase, and speed levels. This paper introduces a data-driven method for automatic optimization of the parameters for fast migration. We leverage preview control as the fundamental algorithm whose effectiveness has been proven on an automated Lincoln MKZ in our previous studies and aim to deploy it to other autonomous cars more efficiently. To learn the control parameters, naturalistic driving data is collected, but one challenge is that the data lacks tracking error information. The proposed strategy decouples the algorithm into feedforward control and feedback control, then learns the two control parts separately with well-designed experiments. Multiple schemes of neural networks are devised to identify feedforward and feedback gains, then the optimal scheme is selected to form the preview control for the given target vehicle. The deployment and validation take place on both simulation and a 4×4 off-road platform. Tests show that the proposed pipeline can learn the control gains for a certain speed range with only around 0.5 hours of driving data, which accelerates the control deployment compared to empirical human-based fine tuning. Compared to the pure pursuit control, the control identified by the proposed method demonstrates outstanding performance in terms of accuracy and stability at the specific speed range.

Global path planning is an essential component of intelligent vehicle study. This paper designs a two-layer global path planning method based on an improved A* algorithm and quadratic programming for UGVs in a large-scale off-road environment. In the first layer, we generate a global path from the current node to the target node via an improved A* algorithm, with a grid map containing the information of non-accessible areas and uncertainty of off-road terrains as input. The second layer smooths the entire path based on quadratic programming. We adopt efficiency improvement methods in both layers, which ensure the real-time performance of the algorithm. The planner has been verified by simulation and experiments, and the results validate the practicability and real-time performance of the designed method.

Autonomous vehicle (AV) is expected to reshape the future transportation system, and its decision-making is one of the most critical modules. Many current decision-making modules are designed scenario-by-scenario and thus are not capable of meeting high-level AV requirements for lack of scalability to cope with the diversity of drivers’ demands and the infinity of traffic elements. This paper surveys the decision-making design inspired by driver intelligence and environment reasoning with better scalability for future high-level AVs. It involves three aspects: human factors in driving, environment reasoning, and detailed decision methods. The current state of the art of these three sections is surveyed in this paper. The characteristics of distinguished drivers, their decision mechanism, and the factors influencing decision-making are reviewed first to learn from excellent drivers. Environment reasoning is introduced following as a three-layer structure consisting of restriction, interaction, and attention. The optimization-based decision-making algorithms are then reviewed from the aspect of optimization targets, frameworks, applied scenarios, and limitations. Inspired by the existing research on driver intelligence and environment reasoning, a promising decision-making framework is also introduced for high-level AV design.

Reinforcement learning (RL) can be used to design smart driving policies in complex situations where traditional methods cannot. However, they are frequently black-box in nature, and the resulting policy may perform poorly, including in scenarios where few training cases are available. In this paper, we propose a method to use RL under two conditions: (i) RL works together with a baseline rule-based driving policy; and (ii) the RL intervenes only when the rule-based method seems to have difficulty handling and when the confidence of the RL policy is high. Our motivation is to use a not-well trained RL policy to reliably improve AV performance. The confidence of the policy is evaluated by Lindeberg-Levy Theorem using the recorded data distribution in the training process. The overall framework is named “confidence-aware reinforcement learning” (CARL). The condition to switch between the RL policy and the baseline policy is analyzed and presented. Driving in a two-lane roundabout scenario is used as the application case study. Simulation results show the proposed method outperforms the pure RL policy and the baseline rule-based policy.

This article presents a model-based motion planning and control system for autonomous vehicles and its experimental validation. The system consists of four modules: 1) global routing; 2) behavior planner; 3) local trajectory generation; and 4) trajectory tracking. The algorithm and software of each module are detailed, including a behavior planner with unified models to handle typical scenarios in both highway and urban driving, a deterministic sampling algorithm for robust responsive trajectory generation, and a dynamics-and-delay-aware preview algorithm to achieve accurate trajectory tracking. The developed system is implemented and tested at the Mcity test facility with a full-size automated car and a dozen of challenging traffic scenarios.

Lane detection on road segments with complex topologies such as lane merge/split and highway ramps is not yet a solved problem. This paper presents a novel graph-embedded solution. It consists of two key parts, a learning-based low-level lane feature extraction algorithm, and a graph-embedded lane inference algorithm. The former reduces the over-reliance on customized annotated/labeled lane data. We leveraged several open-source semantic segmentation datasets (e.g., Cityscape, Vistas, and Apollo) and designed a dedicated network that can be trained across these heterogeneous datasets to extract lane attributes. The latter algorithm constructs a graph to represent the lane geometry and topology. It does not rely on strong geometric assumptions such as lane lines are a set of parallel polynomials. Instead, it constructs a graph based on detected lane nodes. The lane parameters in the world coordinate are inferred by efficient graph-based searching and calculation. The performance of the proposed method is verified on both open source and our own collected data. On-vehicle experiments were also conducted and the comparison with Mobileye EyeQ2 shows favorable results.

Exiting from highways in crowded dynamic traffic is an important path planning task for autonomous vehicles (AVs). This task can be challenging because of the uncertain motion of surrounding vehicles and limited sensing/observing window. Conventional path planning methods usually compute a mandatory lane change (MLC) command, but the lane change behavior (e.g., vehicle speed and gap acceptance) should also adapt to traffic conditions and the urgency for exiting. In this paper, we propose a reinforcement learning-enhanced highway-exit planner. The learning-based strategy learns from past failures and adjusts the vehicle motion when the AV fails to exit. The reinforcement learning is based on the Monte Carlo tree search (MCTS) approach. The proposed learning-enhanced highway-exit planner is tested 6000 times in stochastic simulations. The results indicate that the proposed planner achieves a higher probability of successful highway exiting than a benchmark MLC planner.

Delay and lag deteriorate path tracking accuracy and system stability. If not properly compensated, they can cause instability or limit the driving speed of autonomous vehicles. This paper presents a preview steering control design considering both communication delay and steering lag to achieve accurate, smooth, and computationally efficient path tracking for highly automated vehicles. Two strategies are adopted for the delay: forward state predictor and delay augmentation. The steering lag is approximated by a first-order lag system. The path tracking problem with delay and lag is solved by the preview control theory. The resulted controller is in an analytical form and is computationally efficient for online implementation. We also analyze the system stability and closed-loop responses in both the time and frequency domain. The control is implemented on an automated vehicle platform and tested inside Mcity and on open roads. Experimental results showed lower tracking errors and significantly improved stability margin compared to the controls ignoring delay and lag.

It is necessary to evaluate the safety of highly automated vehicles (HAVs) rigorously before their deployment on public roads. This paper describes a procedure to conduct behavioral competence tests for HAVs using the unprotected left-turn as the illustrating scenario. We first describe a model-based method for test case generation. Subsequently, we propose two methods for synchronizing the motions of the primary other vehicle (POV) and the vehicle under test (VUT), one using fixed speed profile, the other using model predictive control (MPC). Finally, we implement the POV algorithm for the left-turn scenario on an experimental vehicle, and conduct field tests using both virtual and real VUT in the Mcity test facility.

This paper presents a preview steering control algorithm and its closed-loop system analysis and experimental validation for accurate, smooth, and computationally inexpensive path tracking of automated vehicles. The path tracking issue is formulated as an optimal control problem with dynamic disturbance, i.e., the future road curvature. A discrete-time preview controller is then designed on the top of a linear augmented error system, in which the disturbances within a finite preview window are augmented as part of the state vector. The obtained optimal steering control law is in an analytic form and consists of two parts: 1) a feedback control responding to tracking errors and 2) a feedforward control dealing with the future road curvatures. The designed control's nature, capacity, computation load, and underlying mechanism are revealed by the analysis of system responses in the time domain and the frequency domain, theoretical steady-state error, and comparison with the model predictive control (MPC). The algorithm was implemented on an automated vehicle platform, a hybrid Lincoln MKZ. The experimental and simulation results are then presented to demonstrate the improved performance in tracking accuracy, steering smoothness, and computational efficiency compared to the MPC and the full-state feedback control.

Testing and evaluation are critical steps in the development of connected and automated vehicle (CAV) technology. One limitation of closed CAV testing facilities is that they merely provide empty roadways, in which testing CAVs can only interact with a limited number of other CAVs and infrastructure. This paper presents an augmented reality environment for CAV testing and evaluation. A real-world testing facility and a simulation platform are combined together. Movements of testing CAVs in the real world are synchronized with simulation and information of background traffic is fed back to testing CAVs. Testing CAVs can interact with virtual background traffic as if in a realistic traffic environment. The proposed system mainly consists of three components: a simulation platform, testing CAVs, and a communication network. Testing scenarios that have safety concerns and/or require interactions with other vehicles can be performed. Two exemplary test scenarios are designed and implemented to demonstrate the capabilities of the system.

This paper presents a preview servo-loop speed control algorithm to achieve smooth, accurate, and computationally inexpensive speed tracking for connected automated vehicles (CAVs). Differing from methods neglecting the future road slope and target speed information, the proposed controller focuses on taking advantages of this accessible future information to achieve better speed tracking performance. It integrates the future slope and target speed into an augmented optimal control problem, by solving which we obtain the optimal control law in an analytical form. The brake/throttle control laws consist of five parts,i.e., three feedback controls of system states and two feedforward items-preview of road slope and preview of target speed. This controller and its degenerate form,i.e., a classic PID, are implemented and applied to our automated vehicle platform, a Hybrid Lincoln MKZ. Experimental results show three major benefits of the proposed control-lower speed tracking errors, more gentle operations, and smoother brake/throttle behaviors.

This paper presents two fuel-prioritized feedback controllers, which are called the estimated minimum principle (EMP) and kinetic energy conversion (KEC), to realize eco-cruising on varying slopes for vehicles with conventional powertrains. The former is derived from the minimum principle with an estimated Hamiltonian, and the latter is designed based on the equivalent conversion between the kinetic-energy change of vehicle body and the fuel consumption of the engine. They are implemented with analytical control laws and rely on current road slope information only without look-ahead prediction. This feature results in a very light computing load, with the average computing time of each step less than one millisecond. Their fuel-saving performances are quantitatively studied and compared with a model predictive control and a constant speed control. As an expansion, the control rule for avoiding rear-end collision is also designed by using a safety-guaranteed car-following model to constrain the high-risk behaviors.

This paper studies the principles and mechanism of a fuel-optimal strategy in cruising scenarios, i.e., the pulse and glide (PnG) operation, for road vehicles equipped with a step-gear transmission. In the PnG strategy, the control of the engine and the transmission determines the fuel-saving performance, and it is obtained by solving an optimal control problem (OCP). Due to a discrete gear ratio, strong nonlinear engine fuel characteristics, and different dynamics in the pulse/glide mode, the OCP is a switching nonlinear mixed-integer problem. This challenging problem is converted by a knotting technique and the Legendre pseudospectral method to a nonlinear programming problem, which then solves the optimal engine torque and transmission gear position. The optimization results show the significant fuel saving of the PnG operation as compared with the constant-speed cruising strategy. The underlying fuel-saving mechanism of the PnG strategy is explained graphically. For a real-time implementation, a near-optimal practical rule that enables a driver and/or an automatic control system to fast select gear positions and engine torque profile is proposed with only slightly deteriorated fuel saving.

Eco-driving technologies are able to largely reduce the fuel consumption of ground vehicles. This paper presents how to determine the fuel-optimized operating strategies of passenger cars under cruising process. The design naturally casts into an optimal control problem with the S-shaped engine fueling rate as the integrand of cost function. The solutions are numerically solved by the Legendre pseudospectral method, of which many are found to demonstrate periodic behaviors. In the periodic operation, the engine switches between the minimum brake specific fuel consumption (BSFC) point and the idling point, while the vehicle speed oscillates between its upper and lower bounds. The formation of periodic operation are analyzed and explained by the π-test theory and steady state analysis method.

This paper presents a computational framework to solve optimal control problems (OCPs) using Legendre Pseudospectral (PS) method and its application to obtain eco-driving strategies for ground vehicles. Both control and state variables of OCPs are approximated by Lagrange interpolating polynomials at the Legendre-Gauss-Lobatto (LGL) collocation points. The OCP is converted into a nonlinear programming (NLP) problem, and numerically solved by matured optimization algorithms. To implement the PS method, we developed a computational package, called Pseudospectral Optimal control Problem Solver (POPS) in Matlab environment. Further, the POPS is applied to obtain fuel-optimized driving strategies for automated vehicles in hilly road conditions.