Deep learning (DL) has shown state-of-the-art performance in trajectory prediction, which is critical to safe navigation in autonomous driving (AD). However, most DL-based methods suffer from catastrophic forgetting, where adapting to a new distribution may cause significant performance degradation in previously learned ones. Such inability to retain learned knowledge limits their applicability in the real world, where AD systems need to operate across varying scenarios with dynamic distributions. As revealed by neuroscience, the hippocampal circuit plays a crucial role in memory replay, effectively reconstructing learned knowledge based on limited resources. Inspired by this, we propose a hippocampal circuit-inspired continual learning method (H2C) for trajectory prediction across varying scenarios. H2C retains prior knowledge by selectively recalling a small subset of learned samples. First, two complementary strategies are developed to select the subset to represent learned knowledge. Specifically, one strategy maximizes inter-sample diversity to represent the distinctive knowledge, and the other estimates the overall knowledge by equiprobable sampling. Then, H2C updates via a memory replay loss function calculated by these selected samples to retain knowledge while learning new data. Experiments based on various scenarios from the INTERACTION dataset are designed to evaluate H2C. Experimental results show that H2C reduces catastrophic forgetting of DL baselines by 22.71% on average in a task-free manner, without relying on manually informed distributional shifts. The implementation is available at https://github.com/BIT-Jack/H2C-lifelong.

Traffic knowledge is essential for safe, efficient, and regulation-compliant autonomous driving. While existing path planning methods often encode only partial traffic rules or focus on specific scenarios, they lack adaptability to diverse and dynamic environments. The Internet of Things (IoT) enables vehicles to access rich, real-time traffic knowledge, yet most IoT-based approaches emphasize long-distance route optimization and overlook local path planning. This paper proposes a traffic knowledge-augmented path planning model (KARS) that integrates a knowledge graph representation of traffic knowledge with a reachable set-based planning framework in IoT environments. The knowledge graph systematically encodes spatial, temporal, and speed constraints, while the reachable set method enables the planner to dynamically adapt to real-time traffic knowledge. KARS is evaluated in multi-constraint simulation environments with static, semi-dynamic, and dynamic traffic knowledge settings. Compared to representative baselines, it improves driving safety by increasing traffic success rates in complex environments, raises maximum driving speed to enhance operational efficiency, and produces smoother acceleration profiles for improved ride comfort. These results validate the effectiveness of combining traffic knowledge with reachable set planning for safe, efficient, and adaptable autonomous driving in IoT-enabled urban scenarios.

Risk assessment plays an essential role in the improvement of driving safety for intelligent vehicles. Current methods ignoring the predictive and personalized impact of driving policies weaken the effectiveness of risk assessment and lead to human-machine conflicts. By combining subjective cognition of drivers and objective risk metrics, a policy-oriented cognitive risk map (POCRM) is proposed in this paper to encode different driving policies in risk assessment for lane-changing scenarios. To obtain the objective safety metrics, insecurity quantification is built based on the fuzzy theory and fault tree analysis. The subjective cognition of drivers for different driving policies is modeled by deep successor representation and encoded in POCRM using deep reinforcement learning. Driving data collected from the public dataset for realistic traffic environment are used to evaluate the proposed POCRM. The experimental results show that the risk map can take into account future risks and provide driving advice that balances human-machine conflicts with safety in scenarios where drivers can or cannot correctly perceive risk.

Accurate assessment of the cyclist risk is a crucial task for the safety system of autonomous vehicles (AVs). This paper proposes a framework for defining and evaluating cyclist risk levels, considering behavioral cues. The framework comprises three modules: the cyclist graph construction (CGC) module, the risk label generation (RLG) module, and the risk assessment (RA) module. The CGC module constructs a spatiotemporal graph model of the cyclist with both the behavioral and risk information. The RLG module leverages the graph representation method (GRM) to extract features and assigns risk labels using unsupervised learning. The RA module employs spatiotemporal graph convolutional networks (ST-GCN) to extract features from the cyclist graph. Additionally, it facilitates feature fusion through interactions between the human body and the two-wheeler and between hierarchical levels. The fused features, along with the risk labels, are used to train a classifier for the risk assessment of cyclists. The proposed framework is validated using real-world data, and the comparative results with state-of-the-art methods demonstrate the effectiveness and accuracy of the proposed approach in cyclist risk assessment in mixed traffic.

Heterogeneous autonomous vehicle (HAV) coordinated motion planning must guide each vehicle out of the conflict zone based on the differences in vehicle platform characteristics. Decomposing complex driving tasks into primitives is an effective way to improve algorithm efficiency. Hence, the purpose of this paper is to complete the coordinated motion planning tasks through offline driving behavior primitive (DBP) library generation, online extension and selection of DBPs. The proposed algorithm applies dynamic movement primitives and singular value decomposition to learn driving behavior patterns from driving data, integrates them into a model-based optimization generation method as constraints, and builds a DBP library by fusing driving data and vehicle model. Based on the generated DBP library and primitive association probabilities learned from labeled driving segments via stochastic context-free grammar, the planning method completes the independent DBP extension of each vehicle in the conflict zone, generates an interaction DBP tree, and uses the mixed-integer linear programming algorithm to optimally select the primitives to be executed. Fig 1. shows the flowchart of the proposed coordinated motion planning method. We also present how to utilize the DBP libraries to obtain coordinated motion planning results with spatiotemporal information in the form of DBP extension and selection. The results obtained by real vehicle platforms and simulation show that the proposed method can accomplish coordinated motion planning tasks without relying on specific scene elements and highlight the unique motion characteristics of HAVs.

Autonomous vehicles require an accurate understanding of the surrounding road scene for navigation. One crucial task in this understanding is the bird’s-eye view (BEV) road network estimation. However, accurately extracting the BEV road network around the vehicle in complex scenes, considering variations in lane curvature and shape, remains a challenge. This paper aims to accurately represent and learn the BEV road network around the vehicle for structured road scene understanding. Specifically, we propose a road network representation, i.e., representing the lane centerline as an ordered point set and the road network as a directed graph, which accurately describes lane centerline instances and lane topological relationships in complex scenes. Then, we introduce an online road network estimation framework that takes onboard surround-view video as input and utilizes hierarchical query embedding to extract the BEV road network around the vehicle. Furthermore, we present a temporal aggregation module to alleviate occlusion issues in road scenes and enhance the accuracy of road network estimation by incorporating historical frame information flexibly. Finally, we conducted extensive experiments on the nuScenes dataset to validate the effectiveness of the proposed method in structured BEV road scene understanding.

Navigating off-road terrains is crucial for military, agricultural, and rescue operations. Existing algorithms for off-road path planning offer limited adaptability to complex terrains and often lack the computational efficiency required for real-time applications. This is largely due to the nonconvex and nonsmooth characteristics of terrain geometry. Our research introduces an innovative terrain representation technique that streamlines the complexity of the terrain into a manageable path optimization problem, focusing on optimizing vehicle attitude concerning the path. By employing discrete curves to represent lateral terrain elevation changes, our method facilitates the direct integration of vehicle attitude into the optimization framework, thereby diminishing the need for computationally intensive traversability maps typical of traditional approaches. We tackle the resulting nonlinear optimization problem with a constrained iterative linear quadratic regulator (iLQR), achieving real-time path planning capabilities. The proposed method demonstrates improved computational efficiency and enhanced path quality, demonstrating significant time savings in planning while ensuring high-quality outcomes.

Trajectory prediction is a fundamental problem and challenge for autonomous vehicles. Early works mainly focused on designing complicated architectures for deep-learning-based prediction models in normal-illumination environments, which fail in dealing with low-light conditions. The paper proposes a novel approach for trajectory prediction in low-illumination scenarios by leveraging multi-stream information fusion, which integrates image, optical flow, and object trajectory information. This is achieved by applying Convolutional Neural Network-based (CNN) Long Short-term Memory (LSTM) networks to extract temporal information from the image channel, Spatial-Temporal Graph Convolutional Network (ST-GCN) to model relative motion between adjacent camera frames through the optical flow channel, and recognizing high-level interactions between vehicles in the trajectory channel. Further, to investigate the reliability of the model in low-illumination scenarios, epistemic uncertainty estimation is conducted by applying Monte Carlo Dropout. The proposed approach is validated on HEV-I and newly generated Dark-HEV-I datasets focusing on graph-based interaction understanding and low illumination conditions. The experimental results show improved performance compared to baselines in both standard and low-illumination scenarios. Importantly, our approach is generic and applicable to scenarios with different types of perception data. The source code is available at https://github.com/TommyGong08/MSIF.

The deep understanding of the behaviors of traffic participants is essential to guarantee the safety of automated vehicles (AV) in mixed traffic with vulnerable road users (VRUs). Precise trajectory prediction of traffic participants can provide reasonable solution space for motion planning of AV. Early works mainly focused on handcrafting the feature representation and designing complicated architectures in deep learning-based prediction models. However, these approaches overlooked the fact that different road users perceive the safety of the same interaction differently and also exhibit heterogeneous risk-taking styles. In this paper, we will develop a model for trajectory prediction based on risk-taking styles. The model accounts for the expected positions and occupancy of traffic participants in the surrounding environment. It consists of two sequential steps: risk-taking styles of multi-modal road users under interactive scenes are first clustered, and then reformulated in the heterogeneous graph model for trajectory prediction. The model is validated by the driving data collected on the urban road using a public dataset. Comparative experiments demonstrate that the proposed method can predict the trajectory of traffic participants much more accurately than the state-of-the-art methods.

Developing autonomous vehicles (AVs) helps improve the road safety and traffic efficiency of intelligent transportation systems (ITS). Accurately predicting the trajectories of traffic participants is essential to the decision-making and motion planning of AVs in interactive scenarios. Recently, learning-based trajectory predictors have shown state-of-the-art performance in highway or urban areas. However, most existing learning-based models trained with fixed datasets may perform poorly in continuously changing scenarios. Specifically, they may not perform well in learned scenarios after learning the new one. This phenomenon is called “catastrophic forgetting”. Few studies investigate trajectory predictions in continuous scenarios, where catastrophic forgetting may happen. To handle this problem, first, a novel continual learning (CL) approach for vehicle trajectory prediction is proposed in this paper. Then, inspired by brain science, a dynamic memory mechanism is developed by utilizing the measurement of traffic divergence between scenarios, which balances the performance and training efficiency of the proposed CL approach. Finally, datasets collected from different locations are used to design continual training and testing methods in experiments. Experimental results show that the proposed approach achieves consistently high prediction accuracy in continuous scenarios without re-training, which mitigates catastrophic forgetting compared to non-CL approaches. The implementation of the proposed approach is publicly available athttps://github.com/BIT-Jack/D-GSM.

Heterogeneous autonomous vehicle (HAV) coordinated motion planning must guide each vehicle out of the conflict zone based on the differences in vehicle platform characteristics. Decomposing complex driving tasks into primitives is an effective way to improve algorithm efficiency. Hence, the purpose of this paper is to complete the coordinated motion planning tasks through offline driving behavior primitive (DBP) library generation, online extension and selection of DBPs. The proposed algorithm applies dynamic movement primitives and singular value decomposition to learn driving behavior patterns from driving data, integrates them into a model-based optimization generation method as constraints, and builds a DBP library by fusing driving data and vehicle model. Based on the generated DBP library and primitive association probabilities learned from labeled driving segments via stochastic context-free grammar, the planning method completes the independent DBP extension of each vehicle in the conflict zone, generates an interaction DBP tree, and uses the mixed-integer linear programming algorithm to optimally select the primitives to be executed. This study demonstrates that the generated DBP library not only expands the types of primitives, but also distinguishes the characteristics of HAVs. We also present how to utilize the DBP libraries to obtain coordinated motion planning results with spatiotemporal information in the form of DBP extension and selection. The results obtained by real vehicle platforms and simulation show that the proposed method can accomplish coordinated motion planning tasks without relying on specific scene elements and highlight the unique motion characteristics of HAVs.

Accurate recognition of driver behaviours is the basis for a reliable driver assistance system. This paper proposes a novel fusion framework for driver behaviour recognition that utilises the traffic scene and driver gaze information. The proposed framework is based on the graph neural network (GNN) and contains three modules, namely, the gaze analysing (GA) module, scene understanding (SU) module and the information fusion (IF) module. The GA module is used to obtain gaze images of drivers, and extract the gaze features from the images. The SU module provides trajectory predictions for surrounding vehicles, motorcycles, bicycles and other traffic participants. The GA and SU modules are parallel and the outputs of both modules are sent to the IF module that fuses the gaze and scene information using the attention mechanism and recognises the driving behaviours through a combined classifier. The proposed framework is verified on a naturalistic driving dataset. The comparative experiments with the state-of-the-art methods demonstrate that the proposed framework has superior performance for driving behaviour recognition in various situations.

Precisely modeling interactions and accurately predicting trajectories of surrounding vehicles are essential to the decision-making and path-planning of intelligent vehicles. This paper proposes a novel framework based on ensemble learning to improve the performance of trajectory predictions in interactive scenarios. The framework is termed Interactive Ensemble Trajectory Predictor (IETP). IETP assembles interaction-aware trajectory predictors as base learners to build an ensemble learner. Firstly, each base learner in IETP observes historical trajectories of vehicles in the scene. Then each base learner handles interactions between vehicles to predict trajectories. Finally, an ensemble learner is built to predict trajectories by applying two ensemble strategies on the predictions from all base learners. Predictions generated by the ensemble learner are final outputs of IETP. In this study, three experiments using different data are conducted based on the NGSIM dataset. Experimental results show that IETP improves the predicting accuracy and decreases the variance of errors compared to base learners. In addition, IETP exceeds baseline models with 50% of the training data, indicating that IETP is data-efficient. Moreover, the implementation of IETP is publicly available at https://github.com/BIT-Jack/IETP.

The path planning of unmanned differential steering vehicles (UDSVs) in the off-road environment not only needs to consider the non-complete constraints of vehicles but also faces the challenges of complex off-road terrains and obstacles. In this paper, an integrated path planning system is proposed to handle the influence of kinematic vehicle model, off-road terrains and obstacles systematically for the path planning of UDSVs. To improve the planning efficiency, a Pre-planning is designed and carried out using the Voronoi diagram established in the 3D environment with obstacles. By combining the potential field functions (PFF) related to passable obstacles and 3D terrains, an integrated PFF is defined to represent the movement cost of UDSV in the nonlinear optimal control (NOC) problem. Based on the NOC, a channel path planning (CPP) problem is formulated to avoid the untraceable path caused by the traditional line path planning (LPP). Simulation results show that the proposed system can plan a feasible path fast with the constraints from vehicle kinematics, obstacle avoidance and off-road terrains.

In complex and dynamic urban traffic scenarios, the accurate prediction of trajectories of surrounding traffic participants (vehicles, pedestrians, etc) with interactive behaviours plays an important role in the navigation and the motion planning of the ego vehicle. In this paper, based on the graph neural network (GNN), we propose a hierarchical GNN framework to model interactions of heterogeneous traffic participants (vehicles, pedestrians and riders) combined with LSTM to predict their trajectories. The proposed framework consists of two modules with two GNNs for interactive events recognition (IER) and trajectory prediction (TP). The IER module is used to recognise interactive events between traffic participants and the ego vehicle. With the recognised results as the input, the TP module is built for interactive trajectory prediction. In addition, to realise the multi-step prediction, a long short-term memory network (LSTM) is combined with GNN in the TP module. The proposed hierarchical framework is verified by the naturalistic driving data collected from the urban traffic environment. Comparative results with state-of-the-art methods indicate that the hierarchical GNN framework obtains an outstanding performance in the recognition of interactive events and the prediction of interactive behaviours.

Driver model adaptation (DMA) plays an essential role for driving behaviour modelling when there is a lack of sufficient data for training the new model. A new data-driven DMA method is proposed in this paper to realise the instance-level knowledge transfer between individual drivers. Using the importance-weighted transfer learning (IWTL), the data collected from one driver (source driver) can be directly used to train the model of another driver (target driver). Under the framework of IWTL, the relationship between two different drivers can be modelled by the importance weight (IW). Two estimation methods Kullback-Leibler (KL) Divergence and least-squares (LS), are used to estimate IW for each data instance by modelling the importance-weight function as a radial basis function (RBF). Experiments based on the driving simulator and real vehicle are carried out to test the performance of TL for steering behaviour adaptation during the overtaking manoeuvre. The experimental results show that the TL method can transfer the knowledge observed from one driver to another when training the new driver model without sufficient data by keeping the modelling error at a low level.

Omni-directional mobile robot (OMR) systems have been very popular in academia and industry for their superb maneuverability and flexibility. Yet their potential has not been fully exploited, where the extra degree of freedom in OMR can potentially enable the robot to carry out extra tasks. For instance, gimbals or sensors on robots may suffer from a limited field of view or be constrained by the inherent mechanical design, which will require the chassis to be orientation-aware and respond in time. To solve this problem and further develop the OMR systems, in this paper, we categorize the tasks related to OMR chassis into orientation transition tasks and position transition tasks, where the two tasks can be carried out at the same time. By integrating the parallel task goals in a single planning problem, we proposed an orientation-aware planning architecture for OMR systems to execute the orientation transition and position transition in a unified and efficient way. A modified trajectory optimization method called orientation-aware timed-elastic-band (OATEB) is introduced to generate the trajectory that satisfies the requirements of both tasks. Experiments in both 2D simulated environments and real scenes are carried out. A four-wheeled OMR is deployed to conduct the real scene experiment and the results demonstrate that the proposed method is capable of simultaneously executing parallel tasks and is applicable to real-life scenarios.

This paper presents a novel hierarchical reinforcement learning (HRL) framework for automated overtaking. The proposed framework is developed based on the semi-Markov decision process (SMDP) and motion primitives (MPs) which can be applied to different overtaking phases. Unlike the high-level decision and low-level control which are usually independent with each other, the high-level decision making and low-level control are combined by defining MPs with different time intervals. As for the high-level decision making, a SMDP Q-learning algorithm is adopted to realize decision-making of MPs. Besides, a development method of MPs used in the low-level control of automated overtaking is proposed. The performance of the HRL framework is tested in the simulation environment built in a driving simulator called CARLA. The results show that the HRL framework can determine the optimal trajectory under different driving styles of the overtaken vehicle.

Driver model adaptation (DMA) provides a way to model the target driver when sufficient data are not available. Traditional DMA methods running at the model level are restricted by the specific model structures and cannot make full use of the historical data. In this paper, a novel DMA framework based on transfer learning (TL) is proposed to deal with the adaptation of driver models in lane-changing scenarios at the data level. Under the proposed DMA framework, a new TL approach named DTW-LPA that combines dynamic time warping (DTW) and local Procrustes analysis (LPA) is developed. Using the DTW, the relationship between the datasets for different drivers can be found automatically. Based on this relationship, the LPA can transfer the data in the historical dataset to the dataset of a newly-involved driver (target driver). In this way, sufficient data can be obtained for the target driver. After the data transferring process, a proper modeling method, such as the Gaussian mixture regression (GMR), can be applied to train the model for the target driver. Data collected from a driving simulator and realistic driving scenes are used to validate the proposed method in various experiments. Compared with the GMR-only and GMR-MAP methods, the DTW-LPA shows better performance on the model accuracy with much lower predicting errors in most cases.

Developing an autonomous driving system which can generate human-like actions requires the ability to utilize the basic driving skills learned from the driving data. The efficiency of the algorithm can be significantly improved if we can decompose the complex driving tasks into motion primitives (MPs) which represent the elementary composition of driving skills. Therefore, the purpose of this paper is to represent MPs, extract MPs from unlabeled driving data, and then connect the learned MPs in the established library. By applying a probabilistic inference based on an Expectation-Maximization (EM) algorithm and initial segmentation, the extraction method segments the observed trajectories while learning a set of MPs represented by the modified dynamic movement primitives (DMPs). Moreover, the proposed connection algorithm transforms the connection problem into the re-representation problem of the MP sequence. This paper demonstrates that the modified DMP method can not only represent the driver's trajectory with acceptable accuracy but also have strong generalization ability. We also present how to utilize the mutual dependency between the representation and extraction to achieve MP segmentation and MP library establishment. Besides, this paper shows how the proposed connection algorithm correlates the independent MPs in the sequence to ensure a smooth transition and evaluates the tracking accuracy. The results show that the proposed method realizes the extraction of MPs and the re-generation of trajectory by making use of the interdependence relationship that is often neglected between the representation of a single MP, extraction of different types of MP and combination of multiple MPs.

To safely and efficiently change lanes among human drivers, autonomous vehicles (AVs) should make human-like decisions and seamlessly cooperate with surrounding vehicles. Both overaggressive and over-conservative cut-in maneuvers will have adverse effects on traffic efficiency and safety. However, it is still not entirely clear how much influence of the AV's cut-in behavior would lay on the surrounding drivers in urban traffic. To investigate this question, we design a series of driving scenarios and analyze the impact of different cut-in maneuvers performed by the human-like AV on the surrounding drivers' comfort. Ten volunteer drivers participate in our experiment and take a series of trials in a driving simulator. The experimental results demonstrate that the relative distance between the AV and the target car on the adjacent lane has a more significant effect on the surrounding drivers' comfort than the relative speed does. In addition, different parameters should be considered with different cut-in scenarios. This conclusion could provide practical support to make a friendly cut-in decision for the AVs.

Because of the high accuracy and low cost, learning-based methods have been widely used to model driver behaviors in various scenarios. However, the performance of learning-based methods depend heavily on the quantity and coverage of the driving data. When the new driver with insufficient data is considered, the accuracy of these methods cannot be guaranteed any more. To solve this problem, the balanced distribution adaptation (BDA) is used to build the new driver's decision making model in the lane change (LC) scenario. Meanwhile, a transfer learning (TL) based regression model, modified BDA (MBDA) is proposed to predict the driver's steering behavior during the LC maneuver. Cross validation (CV) based model selection (MS) method is developed to obtain the optimal parameters in model training process. A series of experiments are carried out based on the simulated and naturalistic driving data to verify the TL based classification and regression models. The experimental results indicate that the BDA and MBDA have an outstanding ability in knowledge transfer. Compared with support vector machine (SVM) and Gaussian mixture regression (GMR), the proposed methods show a better performance in the decision making of lane keep/change and the prediction of the driver's steering operation.

How to integrate human factors into the motion planning system is of great significance for improving the acceptance of intelligent vehicles. Decomposing motion into primitives and then accurately and smoothly joining the motion primitives (MPs) is an essential issue in the motion planning system. Therefore, the purpose of this paper is to regenerate and join the learned MPs in the library. By applying a representation algorithm based on the modified dynamic movement primitives (DMPs) and singular value decomposition (SVD), our method separates the basic shape parameters and fine-tuning shape parameters from the same type of demonstration trajectories in the MP library. Moreover, we convert the MP joining problem into a re-presentation problem and use the characteristics of the proposed representation algorithm to achieve an accurate and smooth transition. This paper demonstrates that the proposed method can effectively reduce the number of shape adjustment parameters when the MPs are regenerated without affecting the accuracy of the representation. Besides, we also present the ability of the proposed method to smooth the velocity jump when the MPs are connected and evaluate its effect on the accuracy of tracking the set target points. The results show that the proposed method can not only improve the adjustment ability of a single MP in response to different motion planning requirements but also meet the basic requirements of MP joining in the generation of MP sequences.

Autonomous vehicle path planning involves finding a collision-free and kinematic-feasible path from the start to the goal. Evidently, collision-free and kinematic-feasible are the two most important parts of the path planning process. In this paper, we propose a collision checking method that considers dwarf or negative obstacles, which can improve the traversability and stability under the promise of ensuring collision-free, and we use a kind of motion primitives which considering non-holonomic constraints for path planning to generate a kinematic-feasible path. Specifically, some existing approaches of collision checking deal with lofty and dwarf obstacles, or wide and narrow, negative obstacles in the same manner that require the entire vehicle body to steer away from them. However, some dwarf obstacles are shorter than the chassis of the vehicle while some negative obstacles are narrower than the vehicle track. In those cases, overstriding maneuvers will prove much more efficient and reasonable. Moreover, dealing with these obstacles in the same way as lofty obstacles can dramatically reduce the flexibility and disturb the stability of path planning under clustered environments, especially when the vehicle moves at higher speed. This paper proposes a double layer collision checking (DLCC) method that deals with lofty and dwarf obstacles, or wide and narrow, negative obstacles separately, effectively increasing the flexibility under the complex scenarios where dwarf and narrow negative obstacles exist.

In this paper, we introduce a practical two-layer planning framework for autonomous vehicles operating in unknown off-road environment. The first layer refers to a global path planning layer, searching a shortest global path from road network according to given task points. We build up road network through pre-processing on Google earth and refactoring network in terms of vehicle real historical trajectory on real-time. The second layer refers to a local planning layer, solving a real-time planning problem to generate a collision-free and kinematic-feasible local path by a hybrid trajectory planning method. Our method has been verified in real off-road environment. Experimental results show that the proposed planning method performs well in off-road environment.

Fast recognition of a driver's decision-making style when changing lanes plays a pivotal role in a safety-oriented and personalized vehicle control system design. This article presents a time-efficient recognition method by integrating k-means clustering (k-MC) with the K-nearest neighbor (KNN) algorithm, called kMC-KNN. Mathematical morphology is implemented to automatically label the decision-making data into three styles (moderate, vague, and aggressive), while the integration of k-MC and the KNN algorithm helps to improve the recognition speed and accuracy. Our developed mathematical-morphology-based clustering algorithm is then validated by a comparison with agglomerative hierarchical clustering. Experimental results demonstrate that the developed kMC-KNN method, in comparison with the traditional KNN algorithm, can shorten the recognition time by more than 72.67% with a recognition accuracy of 90-98%. In addition, our developed kMCKNN method also outperforms a support vector machine in terms of recognition accuracy and stability. The developed time-efficient recognition approach would have great application potential for in-vehicle embedded solutions with restricted design specifications.

A new driver model adaptation (DMA) method is proposed in this paper to help the model adaptation between different individual drivers. This method is based on transfer learning which can improve the DMA process at data level. The Gaussian mixture model (GMM)-based method is used to model the steering behaviour of drivers during the overtaking manoeuvre. Based on the GMM model, an alignment-based transfer learning technique named local Procrustes analysis (LPA) is modified to formulate the transfer learning problem for driver steering behaviour. A series of experiments based on the data collected from a driving simulator are carried out to evaluate the proposed modified LPA (MLPA). The experimental results verify the ability of MLPA for knowledge transfer. Compared with the GMM-only method and LPA, MLPA shows better performance on the prediction accuracy with much lower predicting errors in most cases.

The decision-making model of merging behavior is one of the key technologies of unmanned vehicles. In order to solve the problem of unmanned vehicles' merging decision making, this paper presents a merging strategy based on Least squares Policy Iteration (LSPI) algorithm, and selects the basis function which includes reciprocal of TTC, relative distance and relative speed to represent state space and discretizes action space. This study synthetically takes consideration o safety, the success of the task, the merging efficiency and comfort in setting reward function, compares the Q-learning with LSPI algorithm, and verifies its adaptability by using NGSIM data. The algorithm can ultimately achieve a success rate of 86%. This research can provide theoretic support and technical basis for the merging decision-making of unmanned vehicles.

This paper presents an Enhanced Adaptive Cruise Control (EACC) framework that can work in different modes according to the forward targets. The EACC system, which was proposed in this paper, is based on a unified model and can achieve speed tracking, stop & go and autonomous emergency braking (AEB). Notably, speed tracking does not require a real preceding vehicle, a virtual vehicle can be set in front of the EACC vehicle. The mathematical method of setting the virtual preceding vehicle and the switching logic between the different working modes of the EACC system were given. Employing a constraints softening method to avoid computing infeasibility, an optimal control law is numerically calculated using the CVXGEN solver. Finally, real vehicle tests show that the EACC framework provides significant benefits in terms of speed-tracking capability, safety and comfort requirements while satisfying driver desired car following characteristics for different driving situations.

Simultaneous localization and mapping (SLAM), as an important tool for vehicle positioning and mapping, plays an important role in the unmanned vehicle technology. This paper mainly presents a new solution to the LIDAR-based SLAM for unmanned vehicles in the off-road environment. Many methods have been proposed to solve the SLAM problems well. However, in complex environment, especially off-road environment, it is difficult to obtain stable positioning results due to the rough road and scene diversity. We propose a SLAM algorithm based on grid which combining probability and feature by Expectation-maximization (EM). The algorithm is mainly divided into three steps: data preprocessing, pose estimation, updating feature grid map. Our algorithm has strong robustness and real-time performance. We have tested our algorithm with our datasets of the multiple off-road scenes which obtained by LIDAR. Our algorithm performs pose estimation and feature map updating in parallel, which guarantees the real-time performance of the algorithm. The average processing time of each frame is about 55ms, and the average relative translation error is around 0.94%. Compared with several state-of-the-art algorithms, our algorithm has better performance in robustness and location accuracy.

Establishing a personalized driver model to predict the driving behavior plays a significant role in the promotion of driver assistance system and automated driving system. In this paper, we propose two different learning-based pathtracking personalized driver models to predict the lookahead distance based on pure pursuit algorithm using the naturalistic driving data collected from BIT intelligent vehicle platform. Based on Gaussian Mixture Model (GMM), one stochastic driver model is the velocity-based Gaussian Mixture Regression (GMR) approach established by combining Gaussian classification process and Gaussian mixture regression, while another driver model is the general GMR approach. The predicting results obtained from the stochastic models are analyzed based on the numbers of GMM components. Statistical analyses show that both personalized driver models perform well, and the velocity-based GMR approach demonstrate higher accuracy than general GMR approach in predicting lookahead distance with the preferred number of the GMM components 10-12 and better performance in tracking the given path.

Simultaneous Localization And Mapping (SLAM) plays a more and more important role in the environment perception system of Unmanned Ground Vehicle (UGV), most SLAM technologies used to be applied indoor or in urban scenarios, we present a real-time 6D SLAM approach suitable for large scale natural terrain with the help of an Inertial Measurement Unit(IMU) and two 3D Lidars. Besides dividing the entire map into many submaps which consists of large numbers of tree structure based voxels, we use probabilistic methods to represent the possibility of one voxel being occupied/null. A Sparse Pose Adjustment (SPA) method has been used to solve 6D global pose optimization with some relative poses as pose constraints and relative motions computed from IMU data as kinetics constraints. A place recognition method integrated a method named Rotation Histogram Matching (RHM) and a Branch and Bound Search (BBS) based Iterative Closest Points (ICP) algorithm is applied to realize a real-time loop closure detection. We complete global pose optimization with the help of Ceres. Experimental results obtained from a real large scale natural environment shows an effective reduction for Lidar odometry pose accumulative error and a good performance for 3D mapping.

Considering the driving habits which are learned from the naturalistic driving data in the path-tracking system can significantly improve the acceptance of intelligent vehicles. Therefore, the goal of this paper is to generate the prediction results of lateral commands with confidence regions according to the reference based on the learned motion primitives. We present a two-level structure for learning and generalizing motion primitives through demonstrations. The lower-level motion primitives are generated under the path segmentation and clustering layer in the upper-level. The Gaussian Mixture Model (GMM) is utilized to represent the primitives and Gaussian Mixture Regression (GMR) is selected to generalize the motion primitives. We show how the upper-level can help to improve the prediction accuracy and evaluate the influence of different time scales and the number of Gaussian components. The model is trained and validated by using the driving data collected from the Beijing Institute of Technology (BIT) intelligent vehicle platform. Experiment results show that the proposed method can extract the motion primitives from the driving data and predict the future lateral control commands with high accuracy.

This paper proposes the decision-making framework of lane change behavior based on Hierarchical State Machine (HSM) and we build distributed control system architecture based on RCS (Real-Time Control System) to test the model. Environment perception module, decision planning module and execution control module are put into the distributed system architecture based on RCS to improve real-time and ensure that several modules run simultaneously. Besides, the decision-making framework of lane change behavior consists of two parts: miniature scene information model and decision-making model of lane change behavior based on multi-attribute decision-making. The decision-making model of lane change behavior is based on HSM and it sets two top-level state machines: free lane change model and mandatory lane change model. Free lane change model changes the state by using lane reward model to judge and assess driving condition of each lane, while mandatory lane change model uses strategy of multi-source information fusion tojudge whether to lane change. In the end, the unmanned platform BYD Tang using vehicle embedded platform is used to verify the reliability and effectiveness of the lane change decision-making algorithm proposed in this paper in the real road environment.

To safely, efficiently change lanes among human drivers, autonomous vehicles (AV) should act as close as possible to human to decide when to friendly cut in and seamlessly cooperate with surrounding vehicles. However, it is still not fully clear about how AV cut-in behavior would influence on human drivers. This paper comprehensively analyzes the influence of AV cut-behavior on human drivers in terms of comfort levels over different cut-in scenarios in a driving simulator. The experiment results demonstrate that the relative distance has great influence on the comfort level of human drivers compared to relative speed; and the integrated influence of relative distance and speed between AV and the behind target vehicle have an important role in the influence. These findings could provide an empirical basis for the decision-making design of autonomous vehicles.

Single image defogging algorithms based on prior assumptions or constraints have captured much attention because of their simplicity and practicality. However, they still have some challenges to deal with foggy images captured under weather conditions where these assumptions or constraints may not be effective or efficient enough. In this paper, we aim to develop a novel image defogging algorithm by directly predicting the fog density of recovered images rather than adopting prior assumptions or constraints. In order to achieve this goal, two specific steps are introduced. First, we adopt three fog-relevant statistical features derived from foggy images, and further develop a simple fog density evaluator (SFDE) by creating a linear combination of these fog-relevant features. This proposed evaluator can efficiently perceive the fog density of a single image without reference to a corresponding fog-free image and has a low computational load compared with an existing method. Second, a physics-based mathematical relationship between the transmission and the fog density score of the recovered image is developed via SFDE, thus image defogging can be posed as a minimization problem on the fog density score of the recovered image. As a result, two optimal transmission models, called an optimal transmission model via SFDE (OTSFDE) and a simpler optimal transmission models via SFDE (SOTSFDE), are present to determine the key transmission map for efficient fog removal. Compared to OTSFDE, SOTSFDE has low computational complexity with slight performance degradation. Experimental results demonstrate that the proposed algorithms can effectively remove fog and are not confined by any assumptions or constraints, both quantitatively and qualitatively, compared with some existing algorithms.

This paper develops a learning model for personalized adaptive cruise control that can learn from human demonstration online and mimic a human driver's driving strategies in the dynamic traffic environment. Under the framework of the proposed model, reinforcement learning is used to capture the human-desired driving strategy, and the proportion-integration-differentiation controller is adopted to convert the learning strategy to low-level control commands. The performance of the learning model is tested in the simulation environment built in a driving simulator using PreScan. Experimental results show that the learning model can duplicate human driving strategies with acceptable errors. Moreover, compared with the traditional adaptive cruise control, the proposed model can provide better driving comfort and smoothness in the dynamic situation.

The longitudinal control of an autonomous vehicle usually suffers from lateral interruptions, such as the cutting in/out of the lead vehicle, deteriorating its performance and even endangering driving safety. To address this problem, we present a model predictive-based approach for longitudinal control in autonomous driving by taking the lateral interruptions into account. First, a virtual lead vehicle scheme is introduced to predict the future behavior of the actual lead vehicle. By following the virtual lead vehicle rather than the actual lead vehicle, the control of the host vehicle is simplified to keep a proper following gap problem. Then, a strategic car-following gap (CFG) model, generated from highway naturalistic driving data, is employed to describe the safety hazard and the probability of cut-ins by other vehicles. A model predictive controller, incorporating the strategic CFG model as well as the acceleration and jerk limitations in the objective function, is designed for the longitudinal control of the host vehicle. Solving the optimal control problem can not only smooth the oscillation and overshoots caused by the lateral interruptions but also reduce the probability of cut-ins from the adjacent lanes. The proposed approach is simulated and validated through some predefined test scenarios in CarSim software.

Complex robotic systems often have to operate in large environments. At the same time, their dynamic is complex enough that path planning algorithms need to reason about the kinodynamic constraints of these systems. On the other hand, such robotic systems are typically expected to operate with speed that is commensurate with that of humans. This poses stringent limitation on available planning time. These will result in a contradiction between planning efficiency and the dimensions of the state space determined by the kinodynamic constraints. In this paper we present an anytime path planning algorithm to solving this problem. First, a graduated state space which consists of state lattices and grids is constructed for planning. Then, ARA* algorithm is utilized to search the graduated state space to find a path that satisfies the kinodynamic constraints and available runtime of planning.

This paper presents a novel method to realize on-board visual odometry system. Vehicular kinematic constrain is used in the motion estimation algorithms. The work is a extension from planar steering model to 3-dof in which vehicle's motion is modeled more reasonable and accurate. By virtue of appropriate simplification, the close-form solution of motion parameters can be obtained only need to find real roots of a cubic equation. Then optimization based refine method can bring the winner solution to accurate solution utilizing inliers founded. The algorithm has been tested on both simulation platform and real car test and achieved promising results.

The 2011 Intelligent Vehicle Future Challenge (11'IVFC) tested self-driving systems in real urban scenarios. The entry of Beijing Institute of Technology: BIT-III finished the 10-kilometer long track in 28 minutes without human operation and obeyed traffic regulations in most circumstances. This paper presented the design and implementation of BIT-III. As a universal self-driving system, BIT-III valued extensibility and featured modularized system architecture. For a better compatibility with diverse sensing devices, BIT-III classified perception to be either OGM (Occupancy Grid Map)-oriented or object-oriented based on the output mode. To work in environments with uncertainties, BIT-III gave first priority to safety and stability in driving, and realized them in the core-level components as the instinct of the system. Even in the unknown environment in the ll'IVFC, BIT-III was able to drive smoothly without crashes.

This paper describes a nonholonomic robotic wheeled vehicle ripple tentacle motion planning method, aiming to improve the vehicle's trajectory smoothness and avoid frequent weight parameters adjustment in different environments. In the regular tentacle motion planning algorithm, the planning result is selected among the drivable tentacles using a weighted sum cost function. Though the method is simple and easy to understand, it is difficult to adjust the weighted coefficients in different environments. To solve this problem, a geometrical ripple tentacles technique is used to choose a tentacle as a sub-optimal path. Compared with the regular tentacles algorithm, the proposed ripple tentacle algorithm can get a better performance in vehicle's trajectory smoothness with an acceptable runtime expense. And another two traits can also distinguish this method: (a) it can avoid weight parameter adjustment in different environments and varied vehicle's states, and (b) it can be used in both unknown environment and partly known environment with goal point and global reference path. In the totally unknown environment, it acts as a pure obstacle avoidance algorithm, and when there is a global path, it can follow the reference path and avoid hazards simultaneously.

We present an iterative linear quadratic regulator (ILQR) method for trajectory tracking control of a wheeled mobile robot system. The proposed scheme involves a kinematic model linearization technique, a global trajectory generation algorithm, and trajectory tracking controller design. A lattice planner, which searches over a 3D ( x, y, θ ) configuration space, is adopted to generate the global trajectory. The ILQR method is used to design a local trajectory tracking controller. The effectiveness of the proposed method is demonstrated in simulation and experiment with a significantly asymmetric differential drive robot. The performance of the local controller is analyzed and compared with that of the existing linear quadratic regulator (LQR) method. According to the experiments, the new controller improves the control sequences ( ν, ω ) iteratively and produces slightly better results. Specifically, two trajectories, ‘S’ and ‘8’ courses, are followed with sufficient accuracy using the proposed controller.

This paper presents an approach to detect and recognize traffic signs present in the urban scenes in China. The algorithm is composed of three steps that are color segmentation, shape detection and pictogram recognition. In the first step Ridge Regression is used to obtain a precise segmentation in RGB color space and achieves the same good performance as many machine learning based methods while using less computation time. Recognition process include a novel feature extraction involves OTSU method, and the feature extracted is robust against illumination variations and distortions. The algorithm has been run on several thousands of images with promising results.

Rank and census transforms provide high resistance to radiometric distortion, vignette, and noise because they are based on the relative ordering of local pixel intensity values rather than the pixel values themselves. These transforms are widely used in many computer vision applications. An important step of computing these transforms is to compare or rank two grayscale values, which is very much like measuring color difference in color image. Color difference between two color points at any part of a uniform color space corresponds to the perceptual difference between the two colors by the human vision system. Based on this idea, we propose to use perceptual color contrast to implement color rank and census transforms and achieve this without significantly increasing the amount of data to process and without complicated computations. Furthermore, we demonstrate the feasibility of using these new transforms to find correspondences for stereo vision.

The recognition and tracking of traffic lights for intelligent vehicles based on a vehicle-mounted camera are studied in this paper. The candidate region of the traffic light is extracted using the threshold segmentation method and the morphological operation. Then, the recognition algorithm of the traffic light based on machine learning is employed. To avoid false negatives and tracking loss, the target tracking algorithm CAMSHIFT (Continuously Adaptive Mean Shift), which uses the color histogram as the target model, is adopted. In addition to traffic signal pre-processing and the recognition method of learning, the initialization problem of the search window of CAMSHIFT algorithm is resolved. Moreover, the window setting method is used to shorten the processing time of the global HSV color space conversion. The real vehicle experiments validate the performance of the presented approach.

The first Chinese unmanned ground vehicles competition - The 2009 Future Challenge: Intelligent Vehicles and Beyond (FC'09) pushed China's unmanned vehicles out of laboratories and into application environments. In order to further promote the development of unmanned vehicle technologies, the test and evaluation system for unmanned vehicles needs to be studied. The design method of test environment is proposed in accordance with the definition and classification of test environment elements. Based on the multi-platform and multi-sensor, an omnidirectional video monitoring test system of unmanned vehicles is built. The fuzzy comprehensive evaluation method combined with AHP (analytic hierarchy process) is applied to the comprehensive evaluation of unmanned vehicles. The evaluation examples of unmanned vehicles show that the proposed evaluation system can quantitatively evaluate the overall technical performance and individual technical performance of unmanned vehicles.

The 2009 Future Challenge - Intelligent Vehicle and Beyond (FC'09) was held in Xi'an, China. Our intelligent vehicle named BIT participated in all competitions at this event. This paper describes BIT's system structure and its capabilities. BIT combines a global path planning method and local path planning to drive the vehicle to address the challenges posted by the unknown competition environment. A novel curve tracking strategy based on preview and curve bisector is developed for complex paths such as U-turn. For recognizing traffic lights, Haar feature and AdaBoost algorithm are used to train and obtain traffic light classifiers. Normalization of every candidate region in RGB and HSV spaces is performed and compared with a threshold to fulfill the verification. The experiment describes BIT's performance and the conclusion sets forth the main work in the next step.

In this paper, a 3D laser point cloud-based navigation method for autonomous ground vehicles in complex environment is proposed. With a coordinate transformation of laser data from sphere to cylinder, environment perception cylinder is configured. In the cylinder, terrain traversability is predicted through analysis on radial and tangential slope of 3D point cloud. In addition, the candidate point cloud of traversable region has been extracted. Furthermore, navigation circle contained with direction information is built up based on the candidate point cloud. Extended experimental results demonstrate that the method allows autonomous ground vehicle to move safely and correctly in complex environment.

Road detection is an important problem with application to driver assistance systems and autonomous, self-guided vehicles. The focus of this paper is on the problem of feature extraction and classification for front-view road detection. Specifically, we propose using Support Vector Machines (SVM) for road detection and effective approach for self-supervised online learning. The proposed road detection algorithm is capable of automatically updating the training data for online training which reduces the possibility of misclassifying road and non-road classes and improves the adaptability of the road detection algorithm. The algorithm presented here can also be seen as a novel framework for self-supervised online learning in the application of classification-based road detection algorithm on intelligent vehicle.

Many people die each year in the world in single vehicle roadway departure crashes caused by driver inattention, especially on the freeway. Lane Departure Warning System (LDWS) is a useful system to avoid those accident, in which, the lane detection is a key issue. In this paper, after a brief overview of existing methods, we present a robust lane detection algorithm based on geometrical model and Gabor filter. This algorithm is based on two assumptions: the road in front of vehicle is approximately planar and marked which are often correct on the highway and freeway where most lane departure accidents happen. The lane geometrical model we build in this paper contains four parameters which are starting position, lane original orientation, lane width and lane curvature. The algorithm is composed of three stages: the first stage is called off-line calibration which just runs once after the camera is mounted and fixed in the vehicle. The parameters of camera used for lane detection is accurately estimated by the 2D calibration method; The second stage is called lane model parameters estimation and lane model candidates construction, the first three parameters, starting position, lane original orientation and lane width will be estimated using dominant orientation estimation and local Hough transform. Then the construction of lane model candidates is implemented for the final lane model matching; the third stage is model matching. The proposed lane module matching algorithm is implemented to match the best fitted lane model. The combination of these modules can overcome the universal lane detection problems due to inaccuracies in edge detection such as shadow of tree and passengers on the road. Experimental results on real road will be presented to prove the effectiveness of the proposed lane detection algorithm.