Reinforcement learning (RL) is a promising approach for end-to-end autonomous driving, but its practical deployment remains challenging due to low sample efficiency and sensitivity to reward design. To address these challenges, this study presents a novel Q-advantage integrated human-guided reinforcement learning (QIHG-RL) framework that effectively combines the strengths of machine learning and human expertise. The QIHG-RL framework features: 1) an ensemble Q-advantage function that aggregates multiple value networks to enhance value estimation, and 2) an integration mechanism that embeds the Q-advantage into both the actor-critic network and the prioritized experience replay. This design allows the agent to leverage sparse and sub-optimal human demonstrations, accelerating policy learning in the early training phase while gradually enhancing exploration as training progresses. The framework is evaluated across three safety-critical driving tasks. Experimental results show a 167% improvement in sample efficiency compared to standard RL methods and a 14% performance gain over a state-of-the-art human-guided RL baseline. Furthermore, a Sim2Real pipeline combining domain randomization and semantic denoised remapping facilitates successful deployment on a real-world autonomous vehicle.

A novel autonomous driving motion planning framework for unsignalized intersections is presented. To achieve an effective balance between safety and efficiency in the decision-making process, a motion planning decision strategy tailored for discrete action spaces is developed based on the discrete soft actor-critic algorithm. In response to the challenges posed by the complexity of feature information in dense intersection environments, a multi-level attention mechanism–integrating both feature-level and vehicle-entity-level information–is introduced to significantly enhance feature extraction and processing capabilities. Furthermore, to mitigate the issues of temporal sample distribution imbalance and low utilization of high-value samples in a single experience pool, a dual experience buffer prioritized replay mechanism is proposed, thereby improving training stability. Experimental results indicate that, compared with alternative methods, the proposed framework not only achieves a superior balance between efficiency and safety but also exhibits enhanced interpretability and generalization performance.

The intelligent transportation system furnishes electrified vehicles with multi-source traffic information, thereby enhancing the potential for greater energy efficiency. Eco-driving and internal energy management represent dual pathways to achieving these efficiencies. Departing from existing studies that typically investigate these pathways independently, this paper introduces a collaborative hierarchical reinforcement learning (RL) method that synchronizes the optimization of both eco-driving and energy management in an integrated solution. To advance RL performance, we propose a novel information bridge scheme at the methodological level. This scheme optimizes the actor-critic RL algorithm’s learning mechanism, facilitating improved information exchange between the dual agents: eco-driving (upper layer) and energy management (lower layer). The critic value of the lower-level agent as a conduit for transmitting condensed state information into the upper, improving the holistic performance. Furthermore, convolutional networks are employed to enhance traffic information extraction. The effectiveness of our method is demonstrated through SUMO simulations, showing a 30.28% improvement in energy efficiency with only a 7.53% compromise in timeliness compared to prevailing Krauss eco-driving. Results also indicate improved training stability and adaptability of our method. This research not only contributes to the optimization of eco-driving for electrified vehicles but also has values in other multi-agent collaborative optimization fields.

Autonomous driving (AD) is critically dependent on intelligent decision-making technology, which is the crucial ingredient in driving safety and overall vehicle performance. And comprehensive consideration of driving heterogeneity, decision synergy, and game interaction is also the cornerstones. Accordingly, this paper constructs a cooperative decision-making framework for autonomous vehicles (AVs) that integrates driving styles within a hierarchical architecture based on deep reinforcement learning (DRL). The upper layer adopts the action shielding mechanism-based dueling-double deep Q-network (D3QN) algorithm incorporating the lane advantages into shared state space to complete the prompt lane-changing (LC) decision, the lower layer applies the soft actor-3-critic (SA3C) algorithm based on the clipped triple Q-learning to provide the continuous speed adaptive control. Three personalized collaborative decision strategies are formulated for particular driving styles in multi-objective optimization preference combined with style-incentive prioritized experience replay (SIPER). The experimental results confirm that the proposed framework can satisfy the personalized driving demands in complex traffic scenarios, effectively explore the prospective LC opportunities, and enhance the driving efficiency by 35.40% with aggressive strategy and the comfort by 56.46% with defensive strategy compared with normal strategy, while maintaining the safety.

The development of a high-performance energy management strategy is of significant importance for reducing the operational costs of fuel cell hybrid electric vehicles. Current energy management strategies lack the quantification of the cost of energy source degradation and suffer from insufficient global optimality. Therefore, this study first quantifies the driving costs, including energy source degradation, and introduces a hierarchical energy management strategy. Specifically, the upper-level driving condition predictor provides accurate driving condition prediction information, while the lower-level power distribution controller uses the established driving cost to create a reward function. By optimizing the overall driving costs of the vehicle within a broad range of driving conditions, the developed energy management strategy combines good global optimality with high computational efficiency, demonstrating potential for practical applications.

In addressing the trade-off between prediction model accuracy and computational cost in the context of path tracking control, this paper proposes a learning-based model predictive control (LB-MPC) strategy for autonomous buses. A three-degree-of-freedom (DOF) single-track vehicle dynamic model is established, and an in-depth analysis is conducted on its step response error with respect to variations in vehicle speed, pedal position, and front wheel steering angle compared to the IPG TruckMaker model. Methods for constructing error datasets and receding horizon updates are designed, and a Gaussian process regression (GPR) is employed to establish an error fitting model for real-time error compensation and correction of the nominal single-track model. The error correction model is utilized as the prediction model, and a path tracking cost function is designed to formulate a quadratic programming (QP) optimization problem, proposing an LB-MPC path tracking control architecture. Through joint simulations using the IPG TruckMaker & Simulink platform and real bus experiment, the real-time performance and effectiveness of the proposed GPR error correction model and LB-MPC path tracking control strategy are verified. Results demonstrate that compared to traditional MPC path tracking control strategy, the proposed LB-MPC strategy reduces the average path tracking error by 79.00%.

Deep reinforcement learning (DRL) has been widely used in the field of automotive energy management. However, DRL is computationally inefficient and less robust, making it difficult to be applied to practical systems. In this article, a customized energy management strategy based on the deep reinforcement learning-model predictive control (DRL-MPC) self-regulation framework is proposed for fuel cell electric vehicles. The soft actor critic (SAC) algorithm is used to train the energy management strategy offline, which minimizes system comprehensive consumption and lifetime degradation. The trained SAC policy outputs the sequence of fuel cell actions at different states in the prediction horizon as the initial value of the nonlinear MPC solution. Under the MPC framework, iterative computation is carried out for nonlinear optimization problems to optimize action sequences based on SAC policy. In addition, the vehicle's usual operation dataset is collected to customize the update package for further improvement of the energy management effect. The DRL-MPC can optimize the SAC policy action at the state boundary to reduce system lifetime degradation. The proposed strategy also shows better optimization robustness than SAC strategy under different vehicle loads. Moreover, after the update package application, the total cost is reduced by 5.93% compared with SAC strategy, which has better optimization under comprehensive condition with different vehicle loads.

Accurate traffic prediction is crucial for effective traffic control and risk assessment. Traffic data exhibits a distinct nature, characterized by the interplay of swift, sudden short-term variations and enduring, extended long-term trends within specific regions. This intricate intermingling and interaction give rise to diverse spatial propagation patterns. Successful traffic prediction models necessitate mastering multi-scale temporal and dynamic spatial correlations, as well as their intricate interrelationships. In this study, we present a novel spatial-temporal traffic prediction framework namedInteractiveSpatial-EnhancedGraphConvolutionNetwork (ISGCN). Our key innovation lies in the introduction of a novel dynamic graph convolution module, which not only captures overarching spatial correlations but also unveils the concealed evolution of dynamic spatial correlations over time. By seamlessly integrating the graph convolutional module with temporal sample convolution and interaction blocks, we adeptly bridge multi-scale temporal correlations with the acquired dynamic spatial correlations. Additionally, we harness diverse temporal granularities data to comprehensively capture global temporal correlations. Experiments conducted on four real-world traffic datasets illustrate that ISGCN outperforms diverse types of state-of-the-art baseline models.

The predictive energy management strategy (PEMS) offers potential advantages in enhancing the driving economy of electrified vehicles using vehicle speed prediction. However, realizing accurate predictions in practical contexts remains a challenge. Departing from conventional PEMS that rely on historical speed or static traffic data, we introduce a real-time traffic-aware PEMS for improved performance. To better understand the interplay between the host vehicle and its surrounding traffic, we use a Transformer network as the predictor that employs the speeds and relative distances of the surrounding six vehicles to forecast future speed sequences for the host vehicle. To augment this data-driven approach, we develop a dual-predictor strategy based on the deep ensemble technique. This strategy measures the Transformer’s output uncertainty to gauge prediction reliability and introduce an automated threshold mechanism. Based on this threshold and real-time uncertainties, the strategy chooses between the Transformer and an exponential predictor to achieve improved prediction outcomes. A reinforcement learning method is integrated as the PEMS optimizer. For validation, we generate training data with traffic information based on the next generation simulation (NGSIM) dataset and create a test scenario in the SUMO simulator. The results confirm that speed predictions based on real-time traffic data surpass traditional PEMS, either directly inputting traffic data or excluding it. The Transformer predictor significantly outperforms the state-of-the-art predictor. Importantly, our dual-predictor design amplifies prediction accuracy by 27.2% against the standard single-network predictor under non-training conditions. Overall, our PEMS enhances driving economy by 11.1% relative to traffic-unaware models and 8.0% over non-Transformer schemes.

Accurate detection of capacity degradation is critical to the safe and efficient utilization of battery systems. Many data-driven capacity estimators were proposed based on emerging intelligent algorithms, but their accuracy depends on the data of complete charged/discharged process and complex algorithm structures. This article developed a computer vision (CV)-based method, constructing battery multidimensional aging features as the key image to estimate capacity using specific charging data segment. Specifically, the designed image-aging recognition method is used to extract multidimensional aging features from the partial charging current sequence and then establish map inputs for a computer vision model that recognizes the constructed feature maps. Consequently, the mapping relationship between the charging information and capacity degradation can be obtained as the 2-D grayscale images that contain massive extracted features in their small size hence greatly simplify the network structure in CV model so as to improve estimation accuracy and efficiency significantly. More importantly, since the model input is a specific charging current segment rather than the data of complete charging process, the model applicability to the random and incomplete charging process of electric vehicles can be greatly improved. Battery cycling data from different types of Li-ion cells were utilized for performance verification. Compared with the conventional estimation methods proposed previously, the proposed method demonstrates the great superiority in terms of the model applicability, estimation accuracy, and computational efficiency for online capacity estimation in actual battery usage.

A precisely parameterized battery model is the prerequisite of the model-based management of lithium-ion battery. However, the unexpected sensing of noises may discount the identification of model parameters in practical applications. This article focuses on the noise effect compensation and online parameter identification for the widely used equivalent circuit model. A novel degree of freedom (DOF) eliminator is proposed and combined with the Frisch scheme in a recursive fashion, for the first time, to coestimate the noise statistics and unbiased model parameters. A computationally tractable numerical solver is further proposed for the DOF eliminator to improve the real-time performance. Simulations and experiments are performed to validate the proposed method from theoretical to practical perspective. Results show that the proposed method can effectively mitigate the noise-induced identification biases and outperform the existing methods in terms of the accuracy and the robustness to noise corruption.

Internal short circuit (ISC) has been proven to be responsible for the thermal runaway failure of lithium-ion battery (LIB). The accurate detection of the ISC failure at the early stage is critical to improve the safety of electric vehicles. In this paper, a ISC detection method with self-diagnostic feature is proposed according to the onboard measured load current and terminal voltage. The state of charge (SOC) is first estimated based on the extended Kalman filter (EKF). The ISC current of the cell is self-calibrated leveraging the EKF-estimated SOC and the measured load current. The estimated ISC current is further median filtered to reduce the stochastic error caused by the uncertainty of SOC estimation. Finally, the filtered ISC current are used to identify the ISC resistance online with the recursive least squares with variable forgetting factor (RLSVF) algorithm. Results suggest that the proposed method can identify the internal short circuit resistance online accurately with a high robustness to the noise disturbance.

Lithium-ion batteries (LIBs) are widely used in electric vehicles (EVs) attributed to their advantages of high energy density and long cycle life. In this vision, fast charging of the LIB system has been a crucial technology to promote the large-scale penetration of EVs in the existing automotive market. Motivated by this, a thermal-constrained fast charging method is proposed based on the model predictive control (MPC) concept in this paper. A coupled electro-thermal model is established, based on which two model-based observers are devised to estimate the state of charge (SOC) and internal temperature of LIB. On this premise, an MPC-based controller is exploited to trade-off smartly the charging fastness and the physical constraints. Comparative results show that the proposed method can optimize the charging towards high speed while keep the terminal voltage and battery internal temperature both within the safety region, which forms an obvious superiority over the traditionally-used constant-current-constant-voltage (CC-CV) protocol.

A stochastic model predictive controller (SMPC) of air conditioning (AC) system is proposed to improve the energy efficiency of electric vehicles (EVs). A Markov-chain based velocity predictor is adopted to provide a sense of the future disturbances over the SMPC control horizon. The sensitivity of electrified AC plant to solar radiation, ambient temperature, and relative air flow speed is quantificationally analyzed from an energy efficiency perspective. Three control approaches are compared in terms of the electricity consumption, cabin temperature, and comfort fluctuation, which include the proposed SMPC method, a generally used bang-bang controller, and dynamic programming as the benchmark. Real solar radiation and ambient temperature data are measured to validate the effectiveness of the SMPC. Comparison results illustrate that SMPC is able to improve the AC energy economy by 12% compared to the rule-based controller. The cabin temperature variation is reduced by more than 50.4%, resulting with a much better cabin comfort.