In this article, the underwater target tracking control problem of a biomimetic underwater vehicle (BUV) is addressed. Since it is difficult to build an effective mathematic model of a BUV due to the uncertainty of hydrodynamics, target tracking control is converted into the Markov decision process and is further achieved via deep reinforcement learning. The system state and reward function of underwater target tracking control are described. Based on the actor-critic reinforcement learning framework, the deep deterministic policy gradient actor-critic algorithm with supervision controller is proposed. The training tricks, including prioritized experience replay, actor network indirect supervision training, target network updating with different periods, and expansion of exploration space by applying random noise, are presented. Indirect supervision training is designed to address the issues of low stability and slow convergence of reinforcement learning in the continuous state and action space. Comparative simulations are performed to show the effectiveness of the training tricks. Finally, the proposed actor-critic reinforcement learning algorithm with supervision controller is applied to the physical BUV. Swimming pool experiments of underwater object tracking of the BUV are conducted in multiple scenarios to verify the effectiveness and robustness of the proposed method.

This paper addresses the problem of path following for a biomimetic underwater vehicle (BUV) propelled by undulatory fins with uncertain model and unknown disturbance. The mechanical structure of the BUV is briefly described. Moreover, the general kinematics and dynamics models of the vehicle are presented and the path following problem is formulated. The controller combining line-of-sight (LOS) guidance system with active disturbance rejection control (ADRC) technique is designed to maneuver the BUV to follow a predefined parameterized curve. Specifically, a guidance system based on LOS principle is implemented to decouple the multi-variable system to steer the surge speed and the course respectively. Furthermore, in order to deal with model uncertainty, ADRC is used in development of the surge speed controller and the course controller. Finally, simulations and experimental results validated the performance of the proposed path following control scheme.