This paper presents experimental results of mobile robot navigation using two predictive controllers – a conventional model-predictive control and a Q-learning predictive controller. The latter essentially substitutes the running objective roll-outs with predicted action-value (Q-function) estimates. The idea behind such an approach is to integrate capabilities of reinforcement learning agents into the setting of model-predictive control while retaining the safety guarantees of the latter. Noteworthy the action sequence calculation step in both algorithms is of the same computational complexity. Yet, as we observed in our experiments, the learning predictive controller was able to outperform the model-predictive baseline. The code for the environment simulation may be found under https://github.com/thd-research/RL-autonomous-navigation.

Traditional control approaches for robotic systems, such as linear quadratic regulator (LQR) or model predictive control (MPC), often rely on a known model of the environment. However, in the real world, uncertainty is a common feature of control problems hence models have imperfections. In this work, we address reward engineering for underactuated robotic systems with parameter uncertainty. We introduce a novel reinforcement learning (RL) method to plan the reward function, specifically designed for underactuated robotic systems with parameter uncertainty. We present and validate a new algorithm called Greedy-Divide and Conquer. We implement this algorithm with a single RL agent to address the challenge of swinging up and balancing a Pendubot system with uncertain parameters and give another example with a 2D-Drone with body mass uncertainty. Our ultimate objective is to enhance the system’s ability to adapt and perform reliably in the face of varying uncertainties.