The learning efficiency remains a critical impediment to the practical application of connected and automated vehicles (CAVs). This paper proposes a transferable model-based reinforcement learning (TMBRL) strategy to enhance the sample efficiency and learning rate of CAVs. Specifically, a surrogate policy model is established by capturing state transitions between the actual environment and the vehicle within traffic scenarios. Then, a model-based reinforcement learning (MBRL) approach is established utilizing a surrogate model and a soft actor-critic algorithm. To improve the learning efficiency of platoon control algorithm, a transfer learning method is implemented to MBRL framework. Specifically, the trained surrogate model of vehicles in the source domain is transferred to vehicles in the target domain, and the latter just should update the surrogate model in terms of the individual dynamic characteristics and tasks. Finally, a platoon experiment platform with Prescan software is conducted. The experimental evaluation demonstrates that the TMBRL strategy significantly outperforms conventional reinforcement learning approaches, achieving higher average cumulative reward of 47 and demonstrating a 16% improvement in training success rate. Comparative analysis further reveals that the proposed TMBRL strategy exhibits superior robustness in platoon tracking tasks, maintaining enhanced trajectory tracking precision and stability under dynamic environmental conditions.

Connected and automated vehicles (CAVs) have emerged as a potential solution to the future challenges of developing safe, efficient, and eco-friendly transportation systems. However, CAV control presents significant challenges significant challenges due to the complexity of interconnectivity and co-ordination required among vehicles. Multi-agent reinforcement learning (MARL), which has shown notable advancements in addressing complex problems in autonomous driving, robotics, and human-vehicle interaction, emerges as a promising tool to enhance CAV capabilities. Despite its potential, there is a notable absence of current reviews on mainstream MARL algorithms for CAVs. To fill this gap, this paper offers a comprehensive review of MARL’s application in CAV control. The paper begins with an introduction to MARL, explaining its unique advantages in handling complex and multi-agent scenarios. It then presents a detailed survey of MARL applications across various control dimensions for CAVs, including critical scenarios such as platooning control, lane-changing, and unsignalized intersections. Additionally, the paper reviews prominent simulation platforms essential for developing and testing MARL algorithms for CAVs. Lastly, it examines the current challenges in deploying MARL for CAV control, including safety, communication, mixed traffic, and sim-to-real challenges. Potential solutions discussed include hierarchical MARL, decentralized MARL, adaptive interactions, and offline MARL. The work has been summarized in MARL_in_CAV_Control_Repository.

Automated Vehicles (AVs) will coexist with Human-Driven Vehicles (HDVs) for a long time. AVs must navigate safely among HDVs while maintaining smooth traffic flow. To facilitate this, the decision making system of AVs must accurately assess HDV intentions while accounting for inherent uncertainties. Current HDV intention prediction models often misclassify these intentions, leading to unsafe navigation decisions. This study introduces a three-stage Bayesian sequential game-based decision making architecture designed for AV operation. In the first stage, the AV utilizes a temporal neural network to classify vehicle intentions. In the second stage, a sequential game is solved to determine optimal actions by predicting future HDV states. The final stage, serving as a validation stage, identifies and corrects misclassifications from the first stage by predicting HDV future positions, incorporating models that account for potential deviations from the ground truth. Simulation results indicate a 93.5±0.5% accuracy in initial intention predictions, facilitating swift and effective decision making. The validation stage further enhances safety by promptly correcting errors, ensuring reliable navigation for AVs in HDV environments.

Dedicated hybrid engine technology using auxiliary electronic components has been proven as an energy-saving solution to public concerns about energy consumption and carbon emissions. This paper proposes a generic approach of K-fold filter-collaborated feature selection (KFFC-FS) to robustly identify the key parameters of three engine performance indicators, i.e., volumetric efficiency, thermal efficiency, and fuel consumption. By using this approach, five filters are collaborated to provide a robust rank of feature importance and avoid the feature overestimation caused by the single filter. Meanwhile, the K-fold cross validation method is introduced to avoid random precision issues and overfitting, further enhancing the robustness of key parameter identification for the independent engine performance indicators. In this research, the modelling data is collected from an experimental test bench with a BYD 1.5L gasoline engine. Under the basics of the studied three engine performance indicators by using a multiple-layer perceptron network, the proposed approach further reduces by at least 10.3% root-mean-square error (RMSE) and at least 30% reduction of the model inputs.

The accurate prediction of soft sensors is essential for the development of modern combustion engines to achieve better performance, lower emissions, and reduced fuel consumption. To precisely predict engine performance, i.e., indicated thermal efficiency, volumetric efficiency, and fuel consumption rate of a hybrid engine, in this article, we propose a novel data-driven approach of statistics-guided accelerated swarm feature selection to find the most effective features for engine soft sensors. Differing from the existing filter or wrapper feature selection approaches, this approach uses external measure information to direct velocity updates in the accelerated swarm feature selection. Several filter and wrapper methods are developed and comprehensively compared. The experimental dataset is collected from a BYD 1.5 L gasoline engine. Validated by bench test, the results demonstrate that the proposed approach finds the most effective features and optimal network structure for data-driven performance prediction of the hybrid engine that was studied.

This article proposes a new form of algorithm environment for the multiobjective optimization of an energy management system in plug-in hybrid vehicles (PHEVs). The surrogate-assisted strength Pareto evolutionary algorithm (SSPEA) is developed to optimize the power-split control parameters guided by the data from the physical PHEV and its digital twins (DTs). By introducing a “confidence factor,” the SSPEA uses the fused data of physically measured and virtually simulated vehicle performances (energy consumption and remaining battery state of charge) to converge the optimization process. Gaussian noisy models are adopted to emulate the real vehicle system on the hardware-in-the-loop platform for experimental evaluation. The testing results suggest that the proposed SSPEA requires less R&D costs than the model-free method that only uses the physical information, and more than 44.6% energy can be saved during the R&D process. Driven by the SSPEA, the optimized energy management system surpasses other non-DT-assisted systems by saving more than 4.8% energy.

Essential decision-making tasks such as power management in future vehicles will benefit from the development of artificial intelligence technology for safe and energy-efficient operations. To develop the technique of using neural network and deep learning in energy management of the plug-in hybrid vehicle and evaluate its advantage, this article proposes a new adaptive learning network that incorporates a deep deterministic policy gradient (DDPG) network with an adaptive neuro-fuzzy inference system (ANFIS) network. First, the ANFIS network is built using a new global K-fold fuzzy learning (GKFL) method for real-time implementation of the offline dynamic programming result. Then, the DDPG network is developed to regulate the input of the ANFIS network with the real-world reinforcement signal. The ANFIS and DDPG networks are integrated to maximize the control utility (CU), which is a function of the vehicle's energy efficiency and the battery state-of-charge. Experimental studies are conducted to testify the performance and robustness of the DDPG-ANFIS network. It has shown that the studied vehicle with the DDPG-ANFIS network achieves 8% higher CU than using the MATLAB ANFIS toolbox on the studied vehicle. In five simulated real-world driving conditions, the DDPG-ANFIS network increased the maximum mean CU value by 138% over the ANFIS-only network and 5% over the DDPG-only network.

This article introduces the concept of the driver-identified supervisory control system, which forms a novel architecture of adaptive energy management for hybrid electric vehicles (HEVs). As a man-machine system, the proposed system can accurately identify the human driver from natural operating signals and provides driver-identified globally optimal control policies as opposed to mere control actions. To help improve the identifiability and efficiency of this control system, the method of spectrum-guided fuzzy feature extraction (SFFE) is developed. First, the configuration of the HEV model and its control system are analyzed. Second, design procedures of the SFFE algorithm are set out to extract 15 groups of features from primitive operating signals. Third, long-term and short-term memory networks are developed as a driver recognizer and tested by the features. The driver identity maps to corresponding control policies optimized by dynamic programming. Finally, the comparative study includes involved extraction methods and their identification system performance as well as their application to HEV systems. The results demonstrate that with help of the SFFE, the driver recognizer improves identifiability by at least 10% compared to that obtained using other involved extraction methods. The improved HEV system is a significant advance over the 5.53% reduction on fuel consumption obtained by the fuzzy-logic-based system.

This paper researches on a cyber-physical energy-saving control framework for a plug-in hybrid aircraft-towing tractor, in which, an online optimization methodology named the online swarm intelligent programming (OSIP) is proposed. The new methodology obtains real-time optimal control signals from the vehicle to everything (V2X) network, and the widely used charge depleting/charge sustaining strategy is upgraded to a more adaptive and intelligent level. The energy flow of the hybrid aircraft-towing tractor with connectivity is first analyzed and modeled for OSIP. The optimal control problem is then formulated as an online integer optimization and the OSIP algorithm based on chaos-enhanced accelerated swarm optimization is developed to minimize the powertrain power loss in real-time. Finally, the advantages of the new energy management system are demonstrated and evaluated by hardware-in-the-loop testing. The results show that up to 17% fuel and 13% total energy loss can be saved via the proposed cyber-physical control.