While cooperative perception can overcome the limitations of single-vehicle systems, the practical implementation of vehicle-to-vehicle and vehicle-to-infrastructure systems is often impeded by significant economic barriers. Aerial-ground cooperation (AGC), which pairs ground vehicles with drones, presents a more economically viable and rapidly deployable alternative. However, this emerging field has been held back by a critical lack of high-quality public datasets and benchmarks. To bridge this gap, we present Griffin, a comprehensive AGC 3D perception dataset, featuring over 250 dynamic scenes (37k+ frames). It incorporates varied drone altitudes (20-60m), diverse weather conditions, realistic drone dynamics via CARLA-AirSim co-simulation, and critical occlusion-aware 3D annotations. Accompanying the dataset is a unified benchmarking framework for cooperative detection and tracking, with protocols to evaluate communication efficiency, altitude adaptability, and robustness to communication latency, data loss and localization noise. By experiments through different cooperative paradigms, we demonstrate the effectiveness and limitations of current methods and provide crucial insights for future research.

Cooperative perception is critical for autonomous driving, overcoming the inherent limitations of a single vehicle, such as occlusions and constrained fields-of-view. However, current approaches sharing dense Bird's-Eye-View (BEV) features are constrained by quadratically-scaling communication costs and the lack of flexibility and interpretability for precise alignment across asynchronous or disparate viewpoints. While emerging sparse query-based methods offer an alternative, they often suffer from inadequate geometric representations, suboptimal fusion strategies, and training instability. In this paper, we propose SparseCoop, a fully sparse cooperative perception framework for 3D detection and tracking that completely discards intermediate BEV representations. Our framework features a trio of innovations: a kinematic grounded instance query that uses an explicit state vector with 3D geometry and velocity for precise spatio-temporal alignment; a coarse-to-fine aggregation module that effectively integrates information from both matched and unmatched instances; and a cooperative instance denoising task that provides stable, abundant supervision to accelerate and stabilize training. Experiments on V2X-Seq and Griffin datasets show SparseCoop achieves state-of-the-art performance. Notably, it delivers this performance with superior computational efficiency and a highly competitive transmission cost, while showing remarkable robustness to real-world challenges like communication latency.

The depth completion task is a critical problem in autonomous driving, involving the generation of dense depth maps from sparse depth maps and RGB images. Most existing methods employ a spatial propagation network to iteratively refine the depth map after obtaining an initial dense depth. In this paper, we propose DenseFormer, a novel method that integrates the diffusion model into the depth completion task. By incorporating the denoising mechanism of the diffusion model, DenseFormer generates the dense depth map by progressively refining an initial random depth distribution through multiple iterations. We propose a feature extraction module that leverages a feature pyramid structure, along with multi-layer deformable attention, to effectively extract and integrate features from sparse depth maps and RGB images, which serve as the guiding condition for the diffusion process. Additionally, this paper presents a depth refinement module that applies multi-step iterative refinement across various ranges to the dense depth results generated by the diffusion process. The module utilizes image features enriched with multi-scale information and sparse depth input to further enhance the accuracy of the predicted depth map. Extensive experiments on the KITTI outdoor scene dataset demonstrate that DenseFormer outperforms classical depth completion methods.

This paper addresses a generalization problem of Multi-Agent Pathfinding (MAPF), called Collaborative Task Sequencing - Multi-Agent Pathfinding (CTS-MAPF), where agents must plan collision-free paths and visit a series of intermediate task locations in an optimized order before reaching their final destinations. To address this problem, we propose a new approach, Collaborative Task Sequencing - Conflict-Based Search (CTS-CBS), which conducts a two-level search. In the high level, it generates a search forest, where each tree corresponds to a joint task sequence derived from the jTSP solution. In the low level, CTS-CBS performs constrained single-agent path planning to generate paths for each agent while adhering to high-level constraints. Furthermore, we integrate Explicit Estimation Search into CTS-CBS to enhance the search efficiency. We also provide theoretical guarantees of its solution completeness and optimality (or sub-optimality with a bounded parameter). To evaluate the performance of CTS-CBS, we utilize three datasets with different map sizes, map difficulty and agent/task numbers, and conduct comprehensive experiments. The results show that our algorithms demonstrate significant improvements in success rate and runtime, while maintaining competitive solution quality. Finally, practical robot simulation and tests demonstrate the algorithm’s applicability in real-world scenarios.

Four-wheel independent steering (4WIS) vehicles have attracted increasing attention for their superior maneuverability. Human drivers typically choose to cross or drive over low-profile obstacles (e.g., plastic bags) to efficiently navigate through narrow spaces, while existing planners neglect obstacle attributes, leading to suboptimal efficiency or planning failures. To address this issue, we propose a novel multimodal trajectory planning framework that employs a neural network for scene perception, integrates 4WIS hybrid A* search to generate a warm start, and formulates an optimal control problem (OCP) for trajectory optimization. Specifically, a multimodal perception network fusing visual information and vehicle states is employed to capture semantic and contextual scene information, enabling the planner to adapt the strategy according to scene complexity (hard or easy planning task). For hard tasks, guided points are introduced to decompose complex tasks into local subtasks, improving search efficiency. The multiple steering modes of 4WIS vehicles—Ackermann, diagonal, and zero-turn—are also incorporated as kinematically feasible motion primitives. Moreover, a hierarchical obstacle handling strategy, which categorizes obstacles as “non-traversable”, “crossable”, and “drive-over”, is incorporated into the node expansion process, explicitly linking obstacle attributes to planning actions to enable efficient decisionmaking. Furthermore, to address dynamic obstacles with motion uncertainty, we introduce a probabilistic risk field model, constructing risk-aware driving corridors that serve as linear collision constraints in the OCP. Experimental results demonstrate the proposed framework’s effectiveness in generating safe, efficient, and smooth trajectories for 4WIS vehicles, especially in constrained environments.

Controlling mixed platoons, which consist of both connected and automated vehicles (CAVs) and human-driven vehicles (HDVs), poses significant challenges due to the uncertain and unknown human driving behaviors. Data-driven control methods offer promising solutions by leveraging available trajectory data, but their performance can be compromised by noise and attacks. To address this issue, this paper proposes a Robust Data-EnablEd Predictive Leading Cruise Control (RDeeP-LCC) framework based on data-driven reachability analysis. The framework over-approximates system dynamics under noise and attack using a matrix zonotope set derived from data, and develops a stabilizing feedback control law. By decoupling the mixed platoon system into nominal and error components, we employ data-driven reachability sets to recursively compute error reachable sets that account for noise and attacks, and obtain tightened safety constraints of the nominal system. This leads to a robust data-driven predictive control framework, solved in a tube-based control manner. Human-in-the-loop experiments demonstrate that theRDeeP-LCCmethod significantly improves robustness against noise and attacks, while enhancing tracking accuracy, control efficiency, energy economy, driving comfort, and driving safety.

End-to-end autonomous driving offers a stream-lined alternative to the traditional modular pipeline, integrating perception, prediction, and planning within a single framework. While Deep Reinforcement Learning (DRL) has recently gained traction in this domain, existing approaches often overlook the critical connection between feature extraction of DRL and perception. In this paper, we bridge this gap by mapping the DRL feature extraction network directly to the perception phase, en-abling clearer interpretation through semantic segmentation. By leveraging Bird's-Eye- View (BEV) representations, we propose a novel DRL-based end-to-end driving framework that utilizes multi-sensor inputs to construct a unified three-dimensional understanding of the environment. This BEV-based system extracts and translates critical environmental features into high-level abstract states for DRL, facilitating more informed control. Extensive experimental evaluations demonstrate that our approach not only enhances interpretability but also significantly outperforms state-of-the-art methods in autonomous driving control tasks, reducing the collision rate by 20 %.

Safety is one of the most crucial challenges of autonomous driving vehicles, and one solution to guarantee safety is to employ an additional control revision module after the planning backbone. Control Barrier Function (CBF) has been widely used because of its strong mathematical foundation on safety. However, the incompatibility with heterogeneous perception data and incomplete consideration of traffic scene elements make existing systems hard to be applied in dynamic and complex real-world scenarios. In this study, we introduce a generalized control revision method for autonomous driving safety, which adopts both vectorized perception and occupancy grid map as inputs and comprehensively models multiple types of traffic scene constraints based on a new proposed barrier function. Traffic elements are integrated into one unified framework, decoupled from specific scenario settings or rules. Experiments on CARLA, SUMO, and OnSite simulator prove that the proposed algorithm could realize safe control revision under complicated scenes, adapting to various planning backbones, road topologies, and risk types. Physical platform validation also verifies the real-world application feasibility.

Visual bird’s eye view (BEV) perception, dute to its excellent perceptual capabilities, is progressively replacing costly LiDAR-based perception systems, especially in the realm of urban intelligent driving. However, this type of perception still relies on LiDAR data to construct ground truth databases, a process that is both cumbersome and time-consuming. Additionally, most mass-produced autonomous driving systems are equipped solely with surround camera sensors and lack the LiDAR data necessary for precise annotation. To tackle this challenge, we propose a fine-tuning method for BEV perception network based on visual 2D semantic perception, aimed at enhancing the model’s generalization capabilities in new scene data. Leveraging the maturity of 2D perception technologies, our method utilizes only 2D semantic segmentation labels and monocular depth estimations, thereby significantly reducing the dependence on expensive BEV ground truths and offering strong potential for industrial deployment. Extensive experiments and comparative analyses on the nuScenes and Waymo datasets demonstrate the effectiveness of our method. Specifically, it improves mAP and NDS by 2.51% and 1.93% on nuScenes, and by 1.21% and 0.78% on Waymo, respectively, validating its practical utility and robustness across diverse domains.

Multi-vehicle trajectory planning (MVTP) is one of the key challenges in multi-robot systems (MRSs) and has broad applications across various fields. This paper presents ESCoT, an enhanced step-based coordinate trajectory planning method for multiple car-like robots. ESCoT incorporates two key strategies: collaborative planning for local robot groups and replanning for duplicate configurations. These strategies effectively enhance the performance of step-based MVTP methods. Through extensive experiments, we show that ESCoT 1) in sparse scenarios, significantly improves solution quality compared to baseline step-based method, achieving up to 70% improvement in typical conflict scenarios and 34% in randomly generated scenarios, while maintaining high solving efficiency; and 2) in dense scenarios, outperforms all baseline methods, maintains a success rate of over 50% even in the most challenging configurations. The results demonstrate that ESCoT effectively solves MVTP, further extending the capabilities of step-based methods. Finally, practical robot tests validate the algorithm’s applicability in real-world scenarios.

Evaluating autonomous driving systems in complex and diverse traffic scenarios through controllable simulation is essential to ensure their safety and reliability. However, existing traffic simulation methods face challenges in their controllability. To address this, we propose a novel diffusion-based and LLM-enhanced traffic simulation framework. Our approach incorporates a high-level understanding module and a low-level refinement module, which systematically examines the hierarchical structure of traffic elements, guides LLMs to thoroughly analyze traffic scenario descriptions step by step, and refines the generation by self-reflection, enhancing their understanding of complex situations. Furthermore, we propose a Frenet-frame-based cost function framework that provides LLMs with geometrically meaningful quantities, improving their grasp of spatial relationships in a scenario and enabling more accurate cost function generation. Experiments on the Waymo Open Motion Dataset (WOMD) demonstrate that our method can handle more intricate descriptions and generate a broader range of scenarios in a controllable manner.

In V2X collaborative perception, the domain gaps between heterogeneous nodes pose a significant challenge for effective information fusion. Pose errors arising from latency and GPS localization noise further exacerbate the issue by leading to feature misalignment. To overcome these challenges, we propose V2X-DGPE, a high-accuracy and robust V2X feature-level collaborative perception framework. V2X-DGPE employs a Knowledge Distillation Framework and a Feature Compensation Module to learn domain-invariant representations, effectively reducing the feature distribution gap between vehicles and roadside infrastructure. Historical information is utilized to provide the model with a more comprehensive understanding of the current scene. Furthermore, a Collaborative Fusion Module leverages a heterogeneous self-attention mechanism to extract and integrate heterogeneous representations from vehicles and infrastructure. To address pose errors, V2X-DGPE introduces a deformable attention mechanism, enabling the model to capture feature misalignments by dynamically offsetting sampling points. Extensive experiments on the real-world DAIR-V2X dataset demonstrate that the proposed method outperforms existing approaches, achieving state-of-the-art detection performance. The code is available at https://github.com/wangsch10/V2X-DGPE.

Accurate and effective road boundary extraction plays a significant role in the navigation and decision-making processes of self-driving cars. Nevertheless, the reliable detection of road boundaries via 3D LiDAR is particularly difficult due to uneven point cloud density and chaotic vegetation areas. Conventional methods often require time-consuming fitting or clustering algorithms to enhance performance. To this end, this paper presents a road boundary extraction approach particularly focuses on the curbs and vegetation areas, utilizing LiDAR data without fitting or clustering, enabling the generation of detailed road maps. We exploit both single and multiple frames of data in the design, which enables spatio-temporal feature extraction. This strategy is realized by integrating the road boundary detection algorithm and the SLAM technology. The algorithm contains three stages: 1) Coarse Ground Segmentation (CGS), 2) Adaptive Spatial Feature Extraction (A-SFE), 3) Iterative Multi-scale Refinement (I-MSR). Experiments on the KITTI dataset are conducted for verification. The proposed method not only outperforms traditional methods with an average of 85% in key metrics but also demonstrates comparable performance to state-of-the-art deep learning models.

With increasing driving automation, driving tasks are shifting from drivers to automation systems and their roles are gradually changing from drivers to passengers. While efforts have been made to explore influencing factors of highly automated vehicles (HAVs) usage intention, few studies have linked role adaptation (RA) to variables from existing acceptance models. To fulfill this research gap, a HAV acceptance model was established based on RA, and other factors (i.e., situational trust, anxiety, and perceived usefulness [PU]). The proposed model validity was verified by subjective ratings collected from a driving simulator experiment involving 105 participants each of whom rode vehicles with three different automated driving styles. Results revealed that RA was an important factor in forming HAVs acceptance during initial human-automation interaction stages and could be increased by improving situational trust, PU, or reducing anxiety. Practically, these results can provide valuable guidance for enhancing consumers’ HAV acceptances.

Optimizing mixed vehicle platoons, which consist of connected and automated vehicles (CAVs) with human-driven vehicles (HDVs), is a critical challenge for intelligent transportation systems. While existing predictive control methods have improved modeling accuracy and control robustness, they are often constrained by their reliance on online optimization, limiting their applicability in real-time scenarios. To address this gap, this paper proposes a Robust Explicit Data-Driven Predictive Control (REDDPC) framework designed to provide robust and real-time control for mixed vehicle platoons. The framework begins by utilizing a deep Koopman operator network to learn the nonlinear dynamics of the system. Using this learned representation, the neural network-based control policy is then optimized through backpropagation, eliminating the need for online optimization. To enhance robustness, a reachability-based safety filter is integrated with the learned control policy to dynamically adjust control inputs, ensuring platoon safety under complex conditions. Simulation and experiment results demonstrate that the proposed method achieves superior tracking performance under noise, disturbance, and attack conditions, while significantly reducing online computational time, making it highly suitable for real-world deployment.

Coordinating connected and automated vehicles (CAVs) poses challenges in achieving safe and optimal pathfinding simultaneously. In this paper, we propose safer conflict-based search (Safer-CBS), an approach based on conflict-based search that generates risk-constrained optimal paths for multiple CAVs. Safer-CBS addresses these challenges by incorporating risk assessment and optimization techniques. By defining the conflict as the risk between two vehicles exceeding a predetermined threshold, the resultant conflict-free paths can provide safety guarantees. An optimization over risk constraints is devised to enhance the efficiency of the tree search. Additional improvements are also incorporated to expedite the search process. Theoretical analysis demonstrates the optimality of Safer-CBS in finding risk-constrained paths. Furthermore, Safer-CBS can seamlessly integrate trajectory planning to generate risk-constrained trajectories for CAVs. Numerical calculations on grid maps validate the effectiveness of the proposed optimization and additional improvements, resulting in increased success rates (by over 43% for 10 vehicles) and reduced runtime (by over 59%) when compared to the method without the optimization and improvements for Safer-CBS. Simulations in both unstructured and structured environments confirm that Safer-CBS, combined with trajectory planning, ensures that trajectory risks strictly remain below predetermined thresholds, while achieving a superior balance between safety and efficiency compared to the benchmarks.Note to Practitioners—Applying multi-CAV technologies in areas such as warehousing, transportation, and operations typically requires the planning of cooperative paths or trajectories for the CAVs. In this planning process, safety and efficiency issues are centrally considered and balanced. Cooperative performance can be ensured as long as each individual vehicle adheres to the central instructions. This study develops a method for risk-constrained cooperative planning, aiming to generate optimal paths and suboptimal trajectories for multiple CAVs. The output trajectories consist of path points with fixed time intervals, which the vehicle controllers can track accordingly. By maintaining the risk level among the vehicles below predetermined thresholds, overall safety can be achieved. Meanwhile, the planning process is optimized, thus improving overall efficiency.

Odometry in adverse weather conditions, such as fog, rain, and snow, presents significant challenges, as traditional vision- and LiDAR-based methods often suffer from degraded performance. Radar-Inertial Odometry (RIO) has emerged as a promising solution due to its resilience in such environments. In this paper, we present RINO, a non-iterative RIO framework implemented in an adaptively loosely coupled manner. Building upon ORORA as the baseline for radar odometry, RINO introduces several key advancements, including improvements in keypoint extraction, motion distortion compensation, and pose estimation via an adaptive voting mechanism. This voting strategy facilitates efficient polynomial-time optimization while simultaneously quantifying the uncertainty in the radar module’s pose estimation. The estimated uncertainty is subsequently integrated into the maximum a posteriori (MAP) estimation within a Kalman filter framework. Unlike prior loosely coupled odometry systems, RINO not only retains the global and robust registration capabilities of the radar component but also dynamically accounts for the real-time operational state of each sensor during fusion. Experimental results conducted on publicly available datasets demonstrate that RINO reduces translation and rotation errors by 1.06% and 0.09°/100m, respectively, when compared to the baseline method, thus significantly enhancing its accuracy. Furthermore, RINO achieves performance comparable to state-of-the-art methods. Our code is available at https://github.com/yangsc4063/rino.

This paper proposes a Learning-Enhanced Adaptive Decision-Making (LEAD) framework for autonomous vehicles (AVs) focusing on dynamic merging scenarios. To capture the competitive and strategic nature of vehicle interactions, we develop an interaction behavior model based on non-cooperative game theory. The behavior is modeled as a dynamic game, where each vehicle optimizes its actions using a multifactorial reward function. To optimize the behavior model parameters, maximum entropy inverse reinforcement learning (IRL) is employed to acquire optimal matching parameters. Additionally, a behavioral decision-making framework LEAD adapted to dynamic environments is proposed. By establishing a mapping between environmental variables and behavior model parameters, it enables parameters online learning and recognition, and achieves interactive behavior probabilities of AVs. Quantitative analysis employing naturalistic driving datasets (highD and exiD) and real-vehicle test data validates LEAD’s high consistency with human decision-making. In 188 tested interaction scenarios, the average human-like similarity rate is 81.73%, with a notable 83.12% in the highD dataset. Furthermore, in 145 dynamic interactions, LEAD matches human decisions at 77.12%, with 6913 consistence instances. Moreover, in real-vehicle tests, a 72.73% similarity with 0% safety violations is obtained. Results demonstrate the effectiveness of our LEAD framework in enabling AVs to make informed, adaptive behavior decisions in interactive environments.

Bird’s-eye-view (BEV) semantic segmentation is becoming crucial in autonomous driving systems. It realizes ego-vehicle surrounding environment perception by projecting 2D multi-view images into 3D world space. Recently, BEV segmentation has made notable progress, attributed to better view transformation modules, larger image encoders, or more temporal information. However, there are still two issues: 1) a lack of effective understanding and enhancement of BEV space features, particularly in accurately capturing long-distance environmental features and 2) recognizing fine details of target objects. To address these issues, we propose OE-BevSeg, an end-to-end multimodal framework that enhances BEV segmentation performance through global environment-aware perception and local target object enhancement. OE-BevSeg employs an environment-aware BEV compressor. Based on prior knowledge about the main composition of the BEV surrounding environment varying with the increase of distance intervals, long-sequence global modeling is utilized to improve the model’s understanding and perception of the environment. From the perspective of enriching target object information in segmentation results, we introduce the center-informed object enhancement module, using centerness information to supervise and guide the segmentation head, thereby enhancing segmentation performance from a local enhancement perspective. Additionally, we designed a multimodal fusion branch that integrates multi-view RGB image features with radar/LiDAR features, achieving significant performance improvements. Extensive experiments show that, whether in camera-only or multimodal fusion BEV segmentation tasks, our approach achieves state-of-the-art results by a large margin on the nuScenes dataset for vehicle segmentation, demonstrating superior applicability in the field of autonomous driving. Our code will be released at https://github.com/SunJ1025/OE-BevSeghttps://github.com/SunJ1025/OE-BevSeg.

Safety certificates based on energy functions can provide demonstrable safety for complex robotic systems. However, all recent studies on learning-based energy function synthesis only consider the feasibility of the control policy, which might cause over-conservativeness and even fail to achieve the control goal. To solve the problem of over-conservative controllers, we proposed the magnitude regularization technique to improve the controller performance of safe controllers by reducing the conservativeness inside the energy function, while keeping the promising provable safety guarantees. Specifically, we quantify the conservativeness by the magnitude of the energy function, and we reduce the conservativeness by adding a magnitude regularization term to the synthesis loss. We propose an algorithm using reinforcement learning (RL) for synthesis to unify the learning process of safe controllers and energy functions. We conducted simulation experiments on Safety Gym and real-robot experiments using small quadrotors. Simulation results show that the proposed algorithm does reduce the conservativeness of the energy function and outperforms baselines in terms of controller performance while maintaining safety. Real-robot experiments have shown that the proposed algorithm indeed reduce conservativeness on the small quadrotors.

Cooperative planning of Connected and Autonomous vehicles (CAVs) is a promising way to reshape the intelligent transportation system, and planning under scenarios mixed with human-driven vehicles is one of the critical challenges. Many existing studies have proposed vehicle group planning methods towards mixed traffic scenes. By regarding the reward of all CAVs as a unified entity, the overall average driving performance was improved. However, one limitation is that during the collective planning process, individual demands are sacrificed for lack of considering agent personalized preferences. To balance the group common benefit and individual diversity, this paper proposes a novel cooperative multi-vehicle planning method combining collective decision-making and individual preference evaluation. First, a bi-level collective planning framework is designed including region-driven behavior selection and conflict-free trajectory generation. To further consider personalized features, an individual preference evaluation system is established based on Social Value Orientation. Then the evaluation results are merged into the group planning process so that vehicles can generate various decisions according to personal demands. Experimental results show that the driving personalization levels of safety, time efficiency, and comfort are increased by 1.28%, 3.78%, and 8.80%, correspondingly.

Decision-making is one of the critical modules of autonomous driving, and how to evaluate its performance precisely remains a challenge. Although some current evaluation methods have considered multiple aspects of driving experience and been able to give a comprehensive evaluation result, the emphasis on individual personalization diversity is still limited. Existing methods mainly fit the evaluation model to an average human model, neglecting the individual demand features. In this paper, we propose a novel integrated decision-making evaluation method considering individual personalization diversity. First, we build an integrated evaluation model to represent the average human evaluation. Based on the four fundamental single-factor model including safety, time efficiency, comfort, and energy consumption, a segmental linear model is applied to combine various elements into one unified framework. Then a personalized weight fluctuation mechanism is proposed which adjusts the relative term weights in the integrated model dynamically according to the users’ preferences. Finally, a corresponding online individual demand distribution estimation method is designed to assess human personal diversity. The experiments on the D2E dataset prove that the proposed evaluation system can better adapt to individual preference diversity, reducing the evaluation mean absolute error by 7.40%. The higher the degree of personalization for the users, the greater the improvement this method can generate.

Multi-lane formation control can significantly enhance the efficiency of traffic systems in multi-lane scenarios. However, existing multi-lane formation control methods are mostly developed for fully Connected and Automated Vehicle (CAV) environments and lack a mechanism for multi-lane formation control in mixed traffic. This paper proposes a CAV formation grouping method and an interaction mechanism of CAVs and Human-driven Vehicles (HDVs), which is suitable for different traffic volumes and penetration rates. Depending on the positional relationship between the CAV formation and HDVs, it flexibly chooses between conventional formation control methods or improved mixed traffic formation control methods. By conducting simulations at various input flow volumes and penetration rates, the applicability of this method to different conditions is verified. The paper also explores the extent to which multi-lane formation control methods can improve traffic efficiency and their relationship with traffic volume and penetration rate.

Deploying a certain trajectory tracking control algorithm to a different autonomous vehicle is time-consuming as most control parameters need to be fine-tuned, which necessitates a series of experiments and computations to ascertain the appropriate parameters considering factors such as tire stiffness, inertia, wheelbase, and speed levels. This paper introduces a data-driven method for automatic optimization of the parameters for fast migration. We leverage preview control as the fundamental algorithm whose effectiveness has been proven on an automated Lincoln MKZ in our previous studies and aim to deploy it to other autonomous cars more efficiently. To learn the control parameters, naturalistic driving data is collected, but one challenge is that the data lacks tracking error information. The proposed strategy decouples the algorithm into feedforward control and feedback control, then learns the two control parts separately with well-designed experiments. Multiple schemes of neural networks are devised to identify feedforward and feedback gains, then the optimal scheme is selected to form the preview control for the given target vehicle. The deployment and validation take place on both simulation and a 4×4 off-road platform. Tests show that the proposed pipeline can learn the control gains for a certain speed range with only around 0.5 hours of driving data, which accelerates the control deployment compared to empirical human-based fine tuning. Compared to the pure pursuit control, the control identified by the proposed method demonstrates outstanding performance in terms of accuracy and stability at the specific speed range.

Trajectory prediction for all road participants constitutes a crucial module in decision-making processes. The primary challenge lies in comprehending the interactions between these agents. Conventional models typically rely on historical trajectories of agents to discern their interactions, often falling short of capturing nuanced expertise. This paper introduces VGA, a novel trajectory prediction model based on Virtual Interaction Force (VIF) and a Graph Attention Network. VGA considers the VIF between agents, employing force a feature vector expressing the impact intensity among agents to encapsulate their interactions. Notably, the model utilizes a graph module to transform raw input into an adjacency matrix. The map encoder and VIF encoder leverage an attention network to extract the agent-lane and agent-VIF relationships. Subsequently, the decoder derives features encompassing the agent-map and agent-VIF relationships, facilitating the prediction of future agent trajectories. Experiment results validate VGA’s ability to yield precise predictions in multi-modal trajectory tasks across the Argoverse 2 Motion Forecast dataset and the Suzhou intersection trajectory dataset. Furthermore, the results of ablation experiment also verify that VIF encoder enhances prediction accuracy.

Deep convolutional neural networks are shown to be overkill with high parametric and computational redundancy in many application scenarios, and an increasing number of works have explored model pruning to obtain lightweight and efficient networks. However, most existing pruning approaches are driven by empirical heuristics and rarely consider the joint impact of channels, leading to unguaranteed and suboptimal performance. In this article, we propose a novel channel pruning method via c lass-aware t race r atio o ptimization (CATRO) to reduce the computational burden and accelerate the model inference. Utilizing class information from a few samples, CATRO measures the joint impact of multiple channels by feature space discriminations and consolidates the layerwise impact of preserved channels. By formulating channel pruning as a submodular set function maximization problem, CATRO solves it efficiently via a two-stage greedy iterative optimization procedure. More importantly, we present theoretical justifications on convergence of CATRO and performance of pruned networks. Experimental results demonstrate that CATRO achieves higher accuracy with similar computation cost or lower computation cost with similar accuracy than other state-of-the-art channel pruning algorithms. In addition, because of its class-aware property, CATRO is suitable to prune efficient networks adaptively for various classification subtasks, enhancing handy deployment and usage of deep networks in real-world applications.

Global path planning is an essential component of intelligent vehicle study. This paper designs a two-layer global path planning method based on an improved A* algorithm and quadratic programming for UGVs in a large-scale off-road environment. In the first layer, we generate a global path from the current node to the target node via an improved A* algorithm, with a grid map containing the information of non-accessible areas and uncertainty of off-road terrains as input. The second layer smooths the entire path based on quadratic programming. We adopt efficiency improvement methods in both layers, which ensure the real-time performance of the algorithm. The planner has been verified by simulation and experiments, and the results validate the practicability and real-time performance of the designed method.

Agent trajectory prediction of traffic scenarios is a significant module of environment reasoning and autonomous vehicle decision, and the core challenge is the ability to interaction reasoning under complex scenes. Previous prediction models are either not precise enough or require massive computational costs. In this paper, we propose VIF-GNN, a novel traffic agent trajectory prediction framework based on the Virtual Interaction Force (VIF) concept and Graph Neural Network, which consists of semantic feature engineering, a subgraph encoder, a global graph, and the trajectory decoder. In particular, this method extracts vectorized features including VIF adjacent matrix from raw inputs and transfers them into graph nodes through the subgraph encoder. The global graph module obtains spatiotemporal reasoning information from four various interaction layers combined with the VIF prior knowledge. And the decoder translates the graph into trajectories of the target agent. Experiments prove that VIF-GNN could achieve precise forecasting on both single and multi-modal prediction task compared with the baselines while maintaining a relatively light parameter size scale, ensuring the real-time performance of vehicle platform applications.

Autonomous vehicle (AV) is expected to reshape the future transportation system, and its decision-making is one of the most critical modules. Many current decision-making modules are designed scenario-by-scenario and thus are not capable of meeting high-level AV requirements for lack of scalability to cope with the diversity of drivers’ demands and the infinity of traffic elements. This paper surveys the decision-making design inspired by driver intelligence and environment reasoning with better scalability for future high-level AVs. It involves three aspects: human factors in driving, environment reasoning, and detailed decision methods. The current state of the art of these three sections is surveyed in this paper. The characteristics of distinguished drivers, their decision mechanism, and the factors influencing decision-making are reviewed first to learn from excellent drivers. Environment reasoning is introduced following as a three-layer structure consisting of restriction, interaction, and attention. The optimization-based decision-making algorithms are then reviewed from the aspect of optimization targets, frameworks, applied scenarios, and limitations. Inspired by the existing research on driver intelligence and environment reasoning, a promising decision-making framework is also introduced for high-level AV design.

Most of fatal traffic accidents occur in unexpected emergency conditions, such as, post-impact, tire blowout etc, in which vehicle attitudes are immediately changed due to external disturbances and internal perturbations. It is an extremely challenging task for human driver to effectively and timely stop or control such a vehicle, especially for the inexperienced driver. Towards this end, this paper proposes a driver-automation collaborative control scheme for vehicles subjected to unexpected emergency conditions by assisting human driver’s steering manipulation. To begin with, a model predictive lateral controller is constructed to enhance dynamical stability and collision avoidance capability considering model uncertainty and external disturbances. After that, a collaborative steering control authority allocator is designed for adaptively allocating control weighting of respective steering angles, in which the parameterized human driver activation is formulated considering driving action and state. As well, an optimal preview acceleration driver model combined with neuromuscular dynamics is developed for imitating human driver’s steering manipulation while harmonizing with the controller. Lastly, simulation examples with different experienced human drivers validated the effectiveness and superiority of proposed control scheme and approaches in lateral stability enhancement and collision avoidance capability of intelligent vehicles subjected to unexpected emergency conditions.

Connected and automated vehicles have shown great potential in improving traffic mobility and reducing emissions, especially at unsignalized intersections. Previous research has shown that vehicle passing order is the key influencing factor in improving intersection traffic mobility. In this paper, we propose a graph-based cooperation method to formalize the conflict-free scheduling problem at an unsignalized intersection. Based on graphical analysis, a vehicle’s trajectory conflict relationship is modeled as a conflict directed graph and a coexisting undirected graph. Then, two graph-based methods are proposed to find the vehicle passing order. The first is an improved depth-first spanning tree algorithm, which aims to find the local optimal passing order vehicle by vehicle. The other novel method is a minimum clique cover algorithm, which identifies the global optimal solution. Finally, a distributed control framework and communication topology are presented to realize the conflict-free cooperation of vehicles. Extensive numerical simulations are conducted for various numbers of vehicles and traffic volumes, and the simulation results prove the effectiveness of the proposed algorithms.

Comprehensive and accurate perception of the real 3D world is the basis of autonomous driving. However, many perceptual methods focus on a single task or object type, and the accuracy of existing multi-task or multi-object methods is difficult to balance against their real-time performance. This paper presents a unified framework for concurrent dynamic multi-object joint perception, which introduces a real-time monocular joint perception network termed MJPNet. In MJPNet relative weightings are automatically learned by a series of developed network branches. By training an end-to-end deep convolutional neural network on a shared feature encoder and many proposed decoding sub-branches, the information of the 2D category and 3D position/pose/size of an object are reconstructed both simultaneously and accurately. Moreover, the effective information among subtasks is transferred by multi-stream learning, guaranteeing the accuracy of each task. Compared to various state-of-the-arts, comprehensive evaluations on the benchmark of challenging image sequences demonstrate the superior performance of our 2D detection and 3D reconstruction of depth, lateral distance, orientation, and heading angle. Moreover, on the KITTI test set, the real-time runtime (up to 15 fps) of MJPNet significantly outran the public state-of-the-art visual detection methods. Accompanying video:https://youtu.be/Z-goToOlI94.

This paper presents a road detection method for autonomous driving based on an end-to-end neural network model. Our method takes advantage of both the characteristics of road boundary and multi-task learning of a deep convolutional network. By reassigning the label and rebalancing the loss of road pixels, we focus on the learning of hard examples on the boundary to refine its performance. Then, a road geometric transformation-based data augmentation method is proposed to enable the network model to be robust under traffic scenes. Based on these two novel methods, a unified architecture consisting of a shared deep residual encoder network and multi-branch decoder sub-networks is integrated. It adopts road scene classification as a supervised learning task to realize road segmentation and scene classification simultaneously. Experiments demonstrate that the proposed method has achieved the highest MaxF value in most road scenes. Both qualitative and quantitative evaluations on the KITTI-Road benchmark demonstrate our superior performance.

Humanlike automated driving (AD) strategies which are inspired by drivers’ cognition ways may show advantages in dealing with complicated scenarios. However, many humanlike AD strategies just mimic drivers’ behaviors or some specific characteristic. Learning algorithms are powerful technics to realize these strategies, but the architectures in learning-based strategies are too simple or with no detailed foundations. Therefore, we mean to summarize drivers’ cognition characteristics and design a comprehensive and well-founded architecture for humanlike AD solutions. We review the massive studies about drivers with human driving or AD and summarize the characteristics from three perspectives, cognition foundation, cognition process, and cognition strategies. As for cognition foundation, we propose a simple analogy to show the working mechanisms of biological neural networks; as for cognition foundation, the important role of previous experience is highlighted; as for cognition strategies, we discuss drivers’ cognition compensation strategies under the influences of environment, vehicle automation, and personal states systematically. After the above review of drivers’ characteristics, we classify the methods to model drivers. We find that models based on cognition processes can maintain more cognition details, and thus we design a driving-dedicated cognitive architecture. This architecture works by the cooperation of several modules including long-term memory, management module, and so on. It has solid theoretical and factual foundations and can reflect drivers’ cognition characteristics comprehensively. Finally, we discuss what needs to be done in the near future for us to improve humanlike AD solutions gradually.

Multi-horizon time series forecasting plays an important role in many industrial and business decision processes. To grasp complex and various patterns across different time series is the crucial step in achieving promising performance. However, most deep learning-based forecasting approaches simply take series-specific static (i.e. time-invariant) covariates as input features, which can fail to capture the complex pattern variation for each possible time series. In this paper, we propose a novel multiplicative attention-based architecture to tackle such forecasting problem. Our modification to multi-head attention layers leverages the series-specific covariates to build flexible attention functions for each possible time series. This improvement contributes to greater representation capacity to grasp different patterns across related time series. Experiment results demonstrate that our approach achieves state-of-the-art performance on a variety of real-world datasets.

Intelligent vehicles should be capable to understand the intention of other traffic participants when driving on urban roads. Yet, current approaches mostly emphasize the importance of the crossing/not-crossing (C/NC) problem and neglect the intention estimation task. To this end, we propose a pedestrian intention estimation method based on the extended theory of planned behavior (TPB). In contrast to previous qualitative modeling based on surveys and questionnaires, neural networks and hand-crafted rules are designed to quantitatively model the components of the extended TPB in the proposed architecture. Besides, the interaction between the components is simulated by a mixed classification strategy. Our pedestrian intention estimation model achieves 82% accuracy and outperforms the baseline method by 3% on the pedestrian intention estimation (PIE) dataset.

The next-item recommendation has been in the central of interest in real-world applications such as e-commerce. However, it is challenging to infer what a user may purchase next due the complex interactions in the historical sessions and the changing semantics of an item over time. Most existing methods employ separate models to generate the general preference and the sequential patterns for the next-item recommendation without considering the interactions between the two factors or use a simple linear combination of the two factors. In this paper, we propose a deep adaptable co-embedding neural network (ACENet) to address these limitations. ACENet not only adaptably balances the combination of general preference and sequential patterns but also introduces dynamic attention for each factor in hybrid representations. Extensive experiments on two real-world datasets show the superiority of ACENet compared with other state-of-the-art methods.

Deep convolutional neural networks have achieved great success in many computer vision tasks. However, they can be attacked by adversarial examples which are input-data with small intentional feature perturbations to fool machine learning models. This vulnerability to adversarial examples poses a potential threat to their widespread application, especially in security-sensitive scenarios. In this paper, we revisit adversarial examples in the frequency domain referring to the computational theory of edge detection, and propose a noveldensely high-frequency convolution neural network(DiFNet) to effectively defend against adversarial attacks. DiFNet introduces classical edge detection operations into the network structure to enhance the detection ability for high-frequency components in the image. It works well to defend against the imperceptible perturbations attacks even without adversarial examples. Experiments demonstrate that DiFNet outperforms handcraft-designed CNNs in terms of prediction accuracy with improved robustness against state-of-the-art adversarial attacks (FGSM, PGD, etc.).

Connected and automated vehicles (CAVs) have a great potential to improve traffic efficiency in mixed traffic systems, which has been demonstrated by multiple numerical simulations and field experiments. However, some fundamental properties of mixed traffic flow, including controllability and stabilizability, have not been well understood. This paper analyzes the controllability of mixed traffic systems and designs a system-level optimal control strategy. Using the Popov-Belevitch-Hautus (PBH) criterion, we prove for the first time that a ring-road mixed traffic system with one CAV and multiple heterogeneous human-driven vehicles is not completely controllable, but is stabilizable under a very mild condition. Then, we formulate the design of a system-level control strategy for the CAV as a structured optimal control problem, where the CAV’s communication ability is explicitly considered. Finally, we derive an upper bound for reachable traffic velocity via controlling the CAV. Extensive numerical experiments verify the effectiveness of our analytical results and the proposed control strategy. Our results validate the possibility of utilizing CAVs as mobile actuators to smooth traffic flow actively.

Neural architecture search (NAS) is usually divided into two phases: model search, where candidate architectures go through an early training for a small number of epochs (e.g., 20) and a search strategy is used to find one or multiple top candidates, and model tuning, where the top candidates are trained fully (e.g., for 600 epochs) and one final best architecture is chosen. The top M-best strategy (M-Best) is typically used to help find better candidates during model search. However, the top M best solutions may concentrate in narrow similar areas and do not have enough diversity. Furthermore, empirical evidence suggests that performance distribution of the models which only go through the early training does not have a strong correlation with that of the models trained fully. Therefore, many of the M best solutions may turn out to be sub-optimal simultaneously because of their similarity, which limits the ability to find true top architectures. To alleviate the problems, we define diverse M-best architectures that are both of high quality and sufficiently different from each other based on a novel graph-based architecture distance. The concept is very general and is applicable to existing architecture search methods using top M-Best. To the best of our knowledge, this is the first time that diversity is introduced into architecture search. We applied the method in the progressive neural architecture search (PNAS) algorithm (Liu et al. 2018a). Experimental results show that our diverse M-Best is indeed beneficial for finding better architectures.

The interactions between driver, vehicle and environment generate vehicle's behaviors, and the interactions of vehicle groups shape the traffic modes. Therefore, the conditionally or fully automated driving technologies should be developed and tested in the driver-vehicle-environment (DVE) system rather than being developed and tested individually. To build DVE system dynamics, firstly, we propose an architecture to cope with the complicated interactions in automated vehicles (AVs) and in mixed traffic situations. Then we summarize the driver behavior models and compare the differences of intelligence between human driver and automated vehicle. Finally, we summarize the feasible modeling approaches of DVE system into five categories. The primary distinctions are the modeling methods of human drivers' roles, which are realized by human driver per se, human driver's cognitive architecture, psychological motivation model, mechanism imitation, and specific mechanism transfer respectively. Taking the applications of human machine interface and AD strategy developments as examples, we analyze the benefits and drawbacks of these approaches.

Lane keeping is a key task in driving, and it plays an important role in staying safe while driving. However, conventional lane-keeping assistance systems (LKASs) have a limited level of automation, while fully autonomous lane-keeping systems have shortcomings regarding reliability. To address these issues, an advanced LKAS with two switchable assistance modes, namely, the lane departure prevention mode and the lane-keeping co-pilot mode, is proposed in this paper. First, the system structure is constructed. Then, the functions and control strategies for the two assistance modes are defined. Next, the controller algorithms are designed using the learning-based model predictive control (LBMPC) method to compensate for modeling errors. Within the framework of LBMPC, an oracle is built to learn the unmodeled dynamics using an extended Kalman filter. Moreover, optimization problems are formulated to achieve the control objectives regarding driving safety and driver acceptance. Finally, driver-in-the-loop experiments carried out on a driving simulator prove that both assistance modes are effective. Furthermore, the lane-keeping co-pilot mode can help reduce the driving burden in the sense of avoiding frequent steering correction.

Generating videos with semantic meaning, such as gestures in sign language, is a challenging problem. The model should not only learn to generate videos with realistic appearance, but also take notice of crucial details in frames to convey precise information. In this paper, we focus on the problem of generating long-term gesture videos containing precise and complete semantic meanings. We develop a novel architecture to learn the temporal and spatial transforms in regions of interest, i.e., gesticulating hands or face in our case. We adopt a hierarchical approach for generating gesture videos, by first making predictions on future pose configurations, and then using the encoder-decoder architecture to synthesize future frames based on the predicted pose structures. We develop the scheme of action progress in our architecture to represent how far the action has been performed during its expected execution, and to instruct our model to synthesize actions with various paces. Our approach is evaluated on two challenging datasets for the task of gesture video generation. Experimental results show that our method can produce gesture videos with more realistic appearance and precise meaning than the state-of-the-art video generation approaches.

This paper presents a general design of an autonomous driving platform. Firstly, we design a physic architecture and three-bus power system to guarantee safety and power redundancy. The logical architecture of the platform is designed in five sub-functions. In most of the cases, planning and control function is the critical and universal function, where we use curvature and environmental perception information to calculate vehicle speed. We also apply feedforward-feedback controller with preview point method to calculate steering angle. In the end, we conduct field test and road test to verify the functionality of the platform. Different navigation methods are also compared in the tests.

Pedestrian orientation is essential for trajectory prediction, intention and behavior analysis. Existing methods using low-level features result in imperfect performance. Aiming at continuous pedestrian orientation estimation from a single frame, we propose a novel approach based on high-level semantic features extracted from human keypoints locations. The proposed method first predicts human keypoints locations by pose estimation algorithm, based on human pose, body motion limitations and body parts occlusion. High-level semantic features are then extracted from keypoints locations. Finally, this method applies a soft classifier and interpolation to produce continuous results. Experiments conducted on public datasets show that the proposed method outperforms the state-of-the-art by a large margin, demonstrating the critical role of high-level semantic feature representation for the task of continuous pedestrian orientation estimation.

This paper is concerned with coordinated formation assignment and control of multiple connected vehicles in highway scenario. A dynamical interlaced layered formation generation method is introduced to provide safe distance among vehicles and efficiency for coordinated lane changing and formation switching simultaneously in real time. To assign vehicles to the generated formation, the optimal problem is modeled by introducing the function of numbers of changed lanes for all the vehicles with respect to the kinematic constraints, and on-board local controllers for the vehicles are developed. Simulation result indicates that, comparing to existing methods for lane assignment in multiple traffic scenarios, the method provided by this paper could increase the traffic efficiency by utilizing maximum road capacity while decreasing travel time for all the vehicles.

To enable safe and efficient navigations through the urban environment, autonomous vehicles need to anticipate the future motions of the walking pedestrians who might collide with them. The dynamic and stochastic behavior characteristics of the pedestrians make the trajectory prediction challengeable for most kinematics-based approaches. This paper presents a comparative study of six state-of-the-art learning-based methods for pedestrian trajectory prediction, including Gaussian Process (GP), LSTM, GP-LSTM, Character-based LSTM, Sequence-to-Sequence (Seq2Seq), and attention-based Seq2Seq. The trajectory prediction is formulated as the regression task or sequence generation problem that predicts future trajectories based on observed trajectories. We evaluate the performance of the learning-based methods on a public real-world pedestrian dataset. To address the concern of data scarcity, we employ three forms of data augmentation (i.e., translation, rotation, and stretch) to enlarge the dataset, which produce the transformed trajectories from the original trajectories. By comparison, those learning-based approaches are ranked based on prediction accuracy from high to low as Seq2Seq, attention-based Seq2Seq, C-LSTM, LSTM, GP, and GP-LSTM. Particularly, Seq2Seq model outperforms all baseline approaches, with the mean and final point errors less than 15cm in normal scenarios when predicting 1s ahead.

Connected and automated vehicles (CAVs) have a great potential to actively influence traffic systems. This has been demonstrated by large-scale numerical simulations and small-scale real experiments, whereas a comprehensive theoretical analysis is still lacking. In this paper, we focus on mixed traffic systems with one single CAV and heterogeneous human-driven vehicles, and present rigorous controllability analysis and optimal controller synthesis. Using the PBH controllability criterion, we reveal controllability properties of a linearized mixed traffic system in a ring road. It is proved that the mixed traffic flow can be stabilized by one single CAV under a very mild condition. We formulate the problem of designing CAV control strategies under a pre-specified communication topology as structured optimal controller synthesis. This formulation considers a system-level performance index that allows the CAV to actively dampen undesired perturbations in traffic flow. Numerical experiments verify the effectiveness of our results.

We present an experimental study comparing various Recurrent Neural Network architectures for the task of Vulnerable Road User (VRU) motion trajectory prediction in the intelligent vehicle domain. Making use of temporal motion cues and visual appearance features, we design multi-cue RNN-based architectures with dedicated optimization process to predict future moving trajectories from historical consecutive frames. Experiments are performed on image sequences recorded from on-board a moving vehicle and public tracking datasets. In particular, the Tsinghua-Daimler Cyclist Benchmark (TDCB) has been augmented with additional annotations (vari-ous VRU types) to support the evaluation of object tracking approaches and trajectory prediction methods. This newly introduced dataset is termed TDCB-Track. We demonstrate the effectiveness of the proposed RNN architectures on the public MOT16 dataset and the TDCB-Track dataset. We show that the proposed approaches outperform simpler baseline methods and stay ahead with the state-of-the-art.

Only a few existing works exploit multiple modalities of data for road detection task in the context of autonomous driving. In this work, a deep learning based approach is developed to fuse LIDAR point cloud and camera image features over a bird's eye view representation. A two-stream fully-convolutional network is designed as encoder to extract general features of two types of data. Instead of limiting the fusion processing at a single stage or to a predefined extent, we propose a multi-stage residual fusion strategy to merge the feature maps in a residual learning fashion, and integrate the information at different network depth. Experiments conduct on KITTI road benchmark show that our proposed method has a significant improvement over single modality methods and other fusion approaches. And it is also among the top-performing algorithms.

Signalized intersections play an important role in transportation efficiency and vehicle fuel economy in urban areas. This paper proposes a cooperative method of traffic signal control and vehicle speed optimization for connected automated vehicles, which optimizes the traffic signal timing and vehicles' speed trajectories at the same time. The method consists of two levels, i.e., roadside traffic signal optimization and onboard vehicle speed control. The former calculates the optimal traffic signal timing and vehicles' arrival time to minimize the total travel time of all vehicles; the latter optimizes the engine power and brake force to minimize the fuel consumption of individual vehicles. The enumeration method and the pseudospectral method are applied in roadside and onboard optimization, respectively. Simulation studies are conducted to compare the proposed method with benchmark methods. The results show significant improvement of transportation efficiency and fuel economy by the cooperation method.

This paper presents an object classification method for vision and light detection and ranging (LIDAR) fusion of autonomous vehicles in the environment. This method is based on convolutional neural network (CNN) and image upsampling theory. By creating a point cloud of LIDAR data upsampling and converting into pixel-level depth information, depth information is connected with Red Green Blue data and fed into a deep CNN. The proposed method can obtain informative feature representation for object classification in autonomous vehicle environment using the integrated vision and LIDAR data. This method is also adopted to guarantee both object classification accuracy and minimal loss. Experimental results are presented and show the effectiveness and efficiency of object classification strategies.

This paper presents a shared control driver assistance system based on the driving intention identification and situation assessment to avoid obstacles. A constrained linear-time-varying model predictive controller is designed to follow the obstacle-avoidance path, which is obtained by the artificial potential method in real time. A human driver's driving intention and the desired maneuver are recognized by the inductive multilabel classification with an unlabeled data approach that is trained based on the lateral offset and lateral velocity to the road center line. In addition, the situation assessment of the collision risk is represented by the time to collision and the performance evaluation is designed according to lateral deviation. All of them are employed for the design of the shared control fuzzy controller. The cooperative coefficient, denoting the control authority between the controller and a human driver, is determined by three fuzzy controllers in different conditions, which are the consistent, the advanced inconsistent, and the lagged inconsistent fuzzy controller, respectively. More importantly, there are two scenarios studies provided to verify the proposed system. The results prove that the shared control driver assistance system can successfully help drivers to avoid obstacles and obtains great vehicle stability performance in different scenarios.

Existing traffic signal optimization and vehicle speed optimization at signalized intersections cannot work together for the lack of proper cooperation methods. We propose the V2I (vehicle to infrastructure) based cooperation between traffic signal and approaching vehicles which optimizes the traffic signal and vehicles' speed trajectories simultaneously. The cooperation consists of roadside traffic signal optimization and onboard speed control, of which the former calculates the optimal traffic signal timing and vehicles' arriving time to minimize trip time and the latter optimizes the vehicle engine power and brake force to minimize the fuel consumption in the whole trip. A simulation study is conducted to compare the proposed cooperation method and the actuated signal control method. The simulation results show significant improvement of transportation efficiency and vehicle fuel economy by using the cooperation method.

Extensive research interest has been focused on protecting vulnerable road users in recent years, particularly pedestrians and cyclists, due to their attributes of vulnerability. However, comparatively little effort has been spent on detecting pedestrian and cyclist together, particularly when it concerns quantitative performance analysis on large datasets. In this paper, we present a unified framework for concurrent pedestrian and cyclist detection, which includes a novel detection proposal method (termed UB-MPR) to output a set of object candidates, a discriminative deep model based on Fast R-CNN for classification and localization, and a specific postprocessing step to further improve detection performance. Experiments are performed on a new pedestrian and cyclist dataset containing 30 490 annotated pedestrian and 26 771 cyclist instances in over 50 000 images, recorded from a moving vehicle in the urban traffic of Beijing. Experimental results indicate that the proposed method outperforms other state-of-the-art methods significantly.

The ability to perform long-range pedestrian detection is essential for autonomous vehicles. However, for 3-D LIDAR, an object's point cloud becomes sparse when it is away, directly affecting its detection as a result. In this paper, a novel density enhancement method is proposed to improve the quality of a sparse point cloud. The input of the method is an object's raw point cloud; first, a high-quality local coordinate system of the point cloud is built using a new evaluation metric, and then radial basis function (RBF)-based interpolation is performed based on the local coordinate system. Finally, a resampling algorithm is used to generate a new point cloud that not only meets a density requirement but also fits the object's geometric shape. Novel features of our method are its evaluation metric of a local coordinate system and method to choose a good shape parameter and kernel in RBF-based interpolation step. The effectiveness of this method is demonstrated using naturalistic data and three experiments.

In this paper, we consider the problem of developing a microscopic model for a signalized traffic intersection using Petri nets (PNs). We propose a two-module timed PN representation, where the first module is to model the traffic intersection and the second module is to model its traffic signal system. We use both deterministic and stochastic transitions in our model, and we describe in detail how they operate by considering both the event and time constraints associated with the physical traffic intersection. We also compare our model with some existing models in literature, and we explicitly state the potential advantages of our model.

Significant effort has been made on designing user-acceptable driver assistance systems. To adapt to driver characteristics, this paper proposes a forward collision warning (FCW) algorithm that can adjust its warning thresholds in a real-time manner according to driver behavior changes, including both behavioral fluctuation and individual difference. This adaptive FCW algorithm overcomes the limit of traditional FCW with fixed risk evaluation models and fixed triggering thresholds by continuously monitoring driver braking behaviors in multiple lanes. A real-time identification algorithm for the warning thresholds is designed by using the recursive least squares method. Based on naturalistic experimental data, offline simulations show that this algorithm can match driver behavioral fluctuation and individual difference in long-time driving condition, and as time goes on, the adaptability to driver behavior is gradually improved, thus decreasing the false-alarm rate of FCW.

In addition to decentralized controllers, the information flow among vehicles can significantly affect the dynamics of a platoon. This paper studies the influence of information flow topology on the internal stability and scalability of homogeneous vehicular platoons moving in a rigid formation. A linearized vehicle longitudinal dynamic model is derived using the exact feedback linearization technique, which accommodates the inertial delay of powertrain dynamics. Directed graphs are adopted to describe different types of allowable information flow interconnecting vehicles, including both radar-based sensors and vehicle-to-vehicle (V2V) communications. Under linear feedback controllers, a unified internal stability theorem is proved by using the algebraic graph theory and Routh-Hurwitz stability criterion. The theorem explicitly establishes the stabilizing thresholds of linear controller gains for platoons, under a large class of different information flow topologies. Using matrix eigenvalue analysis, the scalability is investigated for platoons under two typical information flow topologies, i.e., 1) the stability margin of platoon decays to zero as 0(1/N2) for bidirectional topology; and 2) the stability margin is always bounded and independent of the platoon size for bidirectional-leader topology. Numerical simulations are used to illustrate the results.

The platooning of autonomous ground vehicles has potential to largely benefit the road traffic, including enhancing highway safety, improving traffic utility and reducing fuel consumption. The main goal of platoon control is to ensure all the vehicles in the same group to move at consensual speed while maintaining desired spaces between adjacent vehicles. This paper presents an overview of vehicular platoon control techniques from networked control perspective, which naturally decomposes a platoon into four interrelated components, i.e., 1) node dynamics (ND), 2) information flow topology (IFT), 3) distributed controller (DC) and, 4) geometry formation (GF). Under the four-component framework, existing literature are categorized and analyzed according to their technical features. Three main performance metrics, i.e. string stability, stability margin and coherence behavior, are also discussed.

To address the problem caused by a conventional adaptive cruise control (ACC) system, which hinders drivers from changing lanes, in this study we propose a novel coordinated ACC system with a lane-change assistance function, which enables dual-target tracking, safe lane change, and longitudinal ride comfort. We first analyze lane-change risk by calculating minimum safety spacing between the host vehicle and surrounding vehicles and then develop a coordinated control algorithm using model predictive control theory. Tracking performance is designed on the basis of tracking errors of the host car and two leading vehicles, safety performance is realized by considering the safe distance between the host car and surrounding vehicles, and ride comfort performance is realized by limiting the vehicle's longitudinal acceleration. Driver-in-the-loop tests performed on a driving simulator confirm that the proposed ACC system can overcome the disadvantages of conventional ACC and achieves multiobjective coordination in the lane-change process.

Vehicle driving safety is influenced by many factors, including drivers, vehicles, and road environments. The interactions among them are quite complex. Consequently, existing methods that evaluate driving safety perform inadequately because they only consider limited factors and their interactions. As such, it is difficult for kinematics-based and dynamics-based vehicle driving safety assistant systems to adapt to increasingly complex traffic environments. In this paper, we propose a new concept, i.e.,the driving safety field. The concept makes use of field theory to represent risk factors owing to drivers, vehicles, road conditions, and other traffic factors. A unified model of the driving safety field is constructed, which includes the following three parts: 1) a potential field, which is determined by nonmoving objects on the roads, such as a stopped vehicle; 2) a kinetic field, which is determined by the moving objects on roads, such as vehicles and pedestrians; and 3) a behavior field, which is determined by the individual characteristics of drivers. Moreover, the applications of the model are proposed, and its application to a typical car-following scenario is illustrated, which evaluates the risks caused by multiple traffic factors. The driving safety field can reveal driver–vehicle–road interactions and their influences on driving safety, as well as predict driving safety trends owing to dynamic changes. In addition, the model can provide a new foundation for establishing driving safety measures and active vehicle control under complex traffic environments.

A driver intention recognition method designed for lateral driving assistance systems is proposed based on comprehensive lane-change environment assessment. A new symbol, Comprehensive Decision Index is designed using fuzzy method to assess the influence of surrounding traffic environment on drivers' lane-change decisions. Meanwhile, Hidden Markov Model is applied to recognize driver intention. In the model structure, multiple observation variables are used and the elements' density functions in observation matrix are given the form of Gaussian 3-component mixture model. Finally, data of lane changes performed on driving simulator are used to testify the performance of the proposed method. The results show that the Comprehensive Decision Index is able to make an effective assessment on the environment's influence on drivers' lane-change decisions and the algorithm using it as one of the observation signals can both guarantee the accuracy of recognition results and improve the real-time performance.

A prototype of a longitudinal driving-assistance system, which is adaptive to driver behavior, is developed. Its functions include adaptive cruise control and forward collision warning/avoidance. The research data came from driver car-following tests in real traffic environments. Based on the data analysis, a driver model imitating the driver's operation is established to generate the desired throttle depression and braking pressure. Algorithms for collision warning and automatic braking activation are designed based on the driver's pedal deflection timing during approach (gap closing). A self-learning algorithm for driver characteristics is proposed based on the recursive least-square method with a forgetting factor. Using this algorithm, the parameters of the driver model can be identified from the data in the manual operation phase, and the identification result is applied during the automatic control phase in real time. A test bed with an electronic throttle and an electrohydraulic brake actuator is developed for system validation. The experimental results show that the self-learning algorithm is effective and that the system can, to some extent, adapt to individual characteristics.

The concept of an intelligent environment-friendly vehicle (i-EFV) is proposed in this paper. It integrates three components, i.e., clean-energy powertrain, electrified chassis, and intelligent information interaction devices. By employing such technologies as structure sharing, data fusion, and control coordination, more comprehensive performances are achievable, in terms of traffic safety, fuel efficiency, and environmental protection. Based on its definition and configuration, some key technologies, including design for resource effectiveness, driving environment identification, and coordinated control, are studied. As a basic application, a platform of an intelligent hybrid electric vehicle (i-HEV), which incorporates a hybrid powertrain with adaptive cruise control, has been designed and implemented. Both simulation and experimental results demonstrated that the i-EFV performed better than a conventional vehicle.

A target selection method based on multi features fusion is proposed to improve the accuracy of target vehicle selection. The parameters consisting of the longitudinal distance, lateral distance, relative speed between objects and the host vehicle, the in-lane probability of objects are regarded as the features of individual vehicles. Firstly, some pre-processes of features data are carried out including Distance Compensation Factor (DCF) correcting and Kalman filtering, which are used to correct the in-lane probability data provided by lidar, track and predict the relative distance and speed of objects to lower the missing rate of vehicle detection respectively. Furthermore a two-layer BP neural network is designed to train the sample data and obtain the importance weight of feature variables; the training output is finally utilized as the index for target recognition. The selection method utilizes the valid information collected by sensors through the fusion of multi vehicle features. Experiments show that the vehicle detection results can be improved and the target selection and tracking accurately can be fulfilled through the proposed method. Even under cut-in conditions, the target can also be switched to the cut-in vehicle in time.