Online approximation of the optimal station-keeping strategy for a marine craft subject to an irrotational current is considered. An approximate policy that minimizes a user-defined cost function over an infinite time horizon is obtained using an actor-critic-identifier-based adaptive dynamic programming technique. The hydrodynamic drift dynamics are assumed to be unknown; therefore, a concurrent learning-based system identifier is developed to identify the unknown model parameters. The identified model is used to implement an adaptive model-based reinforcement learning technique to estimate the unknown value function. The developed policy guarantees uniformly ultimately bounded convergence of the vehicle to the desired station and uniformly ultimately bounded convergence of the approximated policies to the optimal polices without the requirement of persistence of excitation. The developed strategy is validated using an autonomous underwater vehicle, where the three degrees-of-freedom in the horizontal plane are regulated. The experiments are conducted in a second-magnitude spring located in central Florida.

This brief paper provides an approximate online adaptive solution to the infinite-horizon optimal tracking problem for control-affine continuous-time nonlinear systems with unknown drift dynamics. To relax the persistence of excitation condition, model-based reinforcement learning is implemented using a concurrent-learning-based system identifier to simulate experience by evaluating the Bellman error over unexplored areas of the state space. Tracking of the desired trajectory and convergence of the developed policy to a neighborhood of the optimal policy are established via Lyapunov-based stability analysis. Simulation results demonstrate the effectiveness of the developed technique.

This study focuses on the development of a nonlinear control design for a fully-actuated autonomous underwater vehicle (AUV) using a continuous robust integral of the sign of the error control structure to compensate for system uncertainties and sufficiently smooth bounded exogenous disturbances. A Lyapunov stability analysis is included to prove semiglobal asymptotic tracking. The resulting controller is experimentally validated on an AUV developed at the University of Florida in both controlled and open-water environments.