Deep Reinforcement Learning (DRL) is becoming a prominent method for autonomous driving due to its strong capability to generate complex driving policy. However, DRL motion planning still has limitations in safety performance including learning quality, convergence speed and the safety guarantee. To this end, this work proposes a safety-enhanced deep reinforcement learning method with dynamic safety guidance (DSG-DRL) for lane-change motion planning. It bears the following key features: 1) Able to learn a safer DRL driving policy by additionally including potentially unsafe behaviors; 2) Able to accelerate learning a safe policy by making dangerous driving experiences impressive; 3) Able to further enhance the driving safety by avoiding unexpected reckless action. The proposed DSG-DRL motion planner dares to make mistakes to learn the safe driving policy better and faster. By evaluating anticipated risk, it learns not only from the maneuvers right at the moments of collisions, but also from the dangerous maneuvers leading towards collisions. Besides, risk driving experiences are enhanced with additional memory batches and sampling prioritization. Moreover, reckless actions can be prevented by dynamic constraints both in training and testing, which further improves the safety performance. Simulation validation shows that the proposed method can learn a safer driving policy with faster convergence speed, achieving the high safety performance while keeping the driving efficiency.

Integrating the understanding and reasoning capabilities of Large Language Models (LLM) with the self-learning capabilities of Reinforcement Learning (RL) enables more reliable driving performance under complex driving conditions. There has been a lot of work exploring LLM-Dominated RL methods in the field of autonomous driving motion planning. These methods, which utilize LLM to directly generate policies or provide decisive instructions during policy learning of RL agent, are centrally characterized by an over-reliance on LLM outputs. However, LLM outputs are susceptible to hallucinations. Evaluations show that state-of-the-art LLM indicates a non-hallucination rate of only approximately 57.95\% when assessed on essential driving-related tasks. Thus, in these methods, hallucinations from the LLM can directly jeopardize the performance of driving policies. This paper argues that maintaining relative independence between the LLM and the RL is vital for solving the hallucinations problem. Consequently, this paper is devoted to propose a novel LLM-Hinted RL paradigm. The LLM is used to generate semantic hints for state augmentation and policy optimization to assist RL agent in motion planning, while the RL agent counteracts potential erroneous semantic indications through policy learning to achieve excellent driving performance. Based on this paradigm, we propose the HCRMP (LLM-Hinted Contextual Reinforcement Learning Motion Planner) architecture, which is designed that includes ①Augmented Semantic Representation Module to extend state space. ②Contextual Stability Anchor Module enhances the reliability of multi-critic weight hints by utilizing information from the knowledge base. ③Semantic Cache Module is employed to seamlessly integrate LLM low-frequency guidance with RL high-frequency control. Extensive experiments in CARLA validate HCRMP's strong overall driving performance. HCRMP achieves a task success rate of up to 80.3\% under diverse driving conditions with different traffic densities. Under safety-critical driving conditions, HCRMP significantly reduces the collision rate by 11.4\%, which effectively improves the driving performance in complex scenarios.

Lane-change is a common maneuver. However, completely egocentric lane-changing behavior may increase potential driving risks and cause oscillations in the movement of surrounding vehicles, referred to as disharmony. To improve the overall safety and efficiency of both ego vehicle and their surroundings, this paper proposes a convergent harmonious reinforcement learning (CHRL) approach to generate harmonious lane-changing strategies. It introduces a game-based model to measure the overall harmony cost. On this basis, a prosocial critic network is established to guide the policy toward harmony by decreasing the harmony cost. Meanwhile, CHRL identifies and penalizes discordant behaviors that may lead to high risk, accelerating the RL agent’s learning of harmonious driving strategies through expert demonstrations of the game model. Simulation and real-data tests validate that CHRL, compared to other lane-change methods and human drivers, improves the overall harmony of lane changes for autonomous vehicles.

Autonomous driving requires efficient and safe decision making and motion planning in dynamic and uncertain environments. Future movement of surrounding vehicles is often difficult to represent. Besides, most existing studies consider decision making and planning/control separately. Both them may lead to the oscillation and unsafe for autonomous driving. This paper proposes an integrated framework of decision making and motion planning with oscillation-free capability. The proposed approach overcomes the shortcomings of autonomous driving for lane change/keeping maneuvers and is able to: i) make oscillation-free behavior decisions given biased prediction; ii) cut through in the traffic efficiently and safely when being in squeezed; iii) accelerate computation efficiency by building a state transfer model based on prediction uncertainty; iv) reduce the dissonance between decision-making and motion planning. A belief decision planner is designed with the uncertainty of the prediction trajectories. Lateral and longitudinal drivable corridors including the reference state and the related boundary constraints are built, which provide better suited information for planning to solve the optimal motion sequence more quickly and stably, and improve its consistency with decision module. Finally, the problem is formulated as an optimal control problem considering the vehicle dynamics and some soft constraints and the motion trajectory is solved by OSQP. Simulation and experimental tests are implemented to evaluate the feasibility and effectiveness of the proposed approach. Test results show that the integrated approach can make proper, safe and continuous decision and planning for autonomous vehicles and the calculation time is very low.