Robust and accurate path tracking for autonomous vehicle navigation is a complex task, especially when it comes to managing system uncertainties such as inertia, slippage, and action delays. Although model-based controllers are efficient, their performance can be limited by such uncertainties and by the complexity of the gain tuning process. To address this, our study evaluates the effectiveness of four strategies using reinforcement learning (RL) with a controller, to provide either - steering correction, full gain tuning, gain correction, or end-to-end learning without any controller - to improve trajectory tracking. These methods are trained on geometric controllers (Pure Pursuit, Stanley) and model predictive controllers (Romea, EBSF). Our results show that all RL methods improve tracking at high speeds, with steering correction proving the most consistently effective in all cases.

During the off-road path following of a wheeled mobile robot in presence of poor grip conditions, the longitudinal velocity should be limited in order to maintain safe navigation with limited tracking errors, while at the same time being high enough to minimize travel time. Thus, this paper presents a new approach of online speed fluctuation, capable of limiting the lateral error below a given threshold, while maximizing the longitudinal velocity. This is accomplished using a neural network trained with a reinforcement learning method. This speed modulation is done side-by-side with an existing model-based predictive steering control, using a state estimator and dynamic observers. Simulated and experimental results show a decrease in tracking error, while maintaining a consistent travel time when compared to a classical constant speed method and to a kinematic speed fluctuation method.

Stable-Baselines3 provides open-source implementations of deep reinforcement learning (RL) algorithms in Python. The implementations have been benchmarked against reference codebases, and automated unit tests cover 95% of the code. The algorithms follow a consistent interface and are accompanied by extensive documentation, making it simple to train and compare different RL algorithms. Our documentation, examples, and source-code are available at https://github.com/DLR-RM/stable-baselines3.

International audience

This paper proposes a new approach for online control law gains adaptation, through the use of neural networks and the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) algorithm, in order to optimize the behavior of the robot with respect to an objective function. The neural network considered takes as input the current observed state as well as its uncertainty, and provides as output the control law gains. It is trained, using the CMA-ES algorithm, on a simulator reproducing the vehicle dynamics. Then, it is tested in real conditions on an agricultural mobile robot at different speeds. The transferability of this method from simulation to a real system is demonstrated, as well as its robustness to environmental changes, such as GPS signal degradation or ground variation. As a result, path following errors are reduced, while ensuring tracking stability.

International audience