This article focuses on solving a finite-horizon nonlinear optimal control problem by using the Pontryagin's maximum principle. In practical applications, linearization is a common approach for solving nonlinear dynamical systems. However, it is not universally applicable due to various reasons, such as instability and low accuracy. In contrast to linearization, the inherent challenge in directly solving the above nonlinear optimal control problem lies in addressing the highly coupled nonlinear forward and backward differential equations. In order to address this problem, an equivalent relationship is established between these equations and a new optimization problem. By exploiting the inherent relationship between supervised learning and an optimization problem from the view of a dynamical system, a deep neural network framework is constructed for describing the new optimization problem. Furthermore, a numerical algorithm for optimal control, which is very powerful for a large variety of nonlinear dynamical systems, is implemented by training a deep residual network. Finally, the effectiveness of the algorithm is demonstrated by solving a trajectory tracking control problem for automatic guided vehicle. The obtained results reveal that the proposed control scheme can achieve high-precision tracking.

The linear quadratic regulation (LQR) problem for discrete-time networked control systems (NCSs) is investigated in this article. The difference from most previous works is that input delay and packet losses occur simultaneously in the communication channel connecting the controller to the actuator. Moreover, the data packet dropout is modeled as a time-homogeneous Markov process which poses challenges due to the temporal correlation. The contributions of this article are twofold. First, by applying the maximum principle involving Markov jumps and delay, the linear quadratic optimal control problem in finite horizon is solved and the solution is given in terms of a forward and a backward stochastic difference equations (FBSDEs-M). Second, under a basic assumption, the infinite horizon optimal control problem is solved and the necessary and sufficient condition for mean square stabilization is given in terms of the solutions to coupled algebraic Riccati-type equations with Markov jumps. The presented results are new to the best of our knowledge since there is no existing work that tackles delay and Markovian packet dropouts simultaneously. It is a generalization of the previous work in which the packet dropouts is modeled as an independent identically distributed Bernoulli process.