This study addresses the optimal control problem for continuous-time nonlinear systems with unknown dynamics. A policy iterative-based optimization algorithm is proposed to solve this problem by leveraging a novel neural network representation termed multivariable neural network linear differential inclusion (MVNNLDI). MVNNLDI approximates the initial nonlinear model with a linear differential equation formulation that includes bounded disturbances. Based on this linearized representation, the relevant adaptive optimal control and disturbance compensation approach are derived to tackle the nonlinear optimization problem. Capitalizing on model-free control principles, the optimal solutions can be obtained using only measured state and input data, thus simplifying algorithmic complexity and accelerating convergence speed substantially. Finally, we use two simulation experiments to demonstrate the feasibility and effectiveness of the proposed method.

In this article, we investigate the optimal control problem for semicar active suspension systems (SCASSs). First, we model the SCASSs by Newtonian dynamics as well as considering the uncertainties and nonlinear dynamics of the actuator. Second, in order to solve the complexity brought by uncertainties, we apply the Takagi–Sugeno (T-S) fuzzy approach to transform the SCASSs as multilinear systems, as well as solving the optimal control problem as a zero-sum problem to find the solution of Nash-equilibrium. Third, we construct a novel self-learning method based on the reinforcement learning framework, and propose two algorithms to solve the fuzzy game algebraic Riccati equation. Especially, in the second algorithm, without using any model information of the SCASSs, we only use the state and input information in control design by a self-learning manner removing the traditional dependence problem, which is more preferable for practical applications. Finally, we give a simulation result of the SCASSs to demonstrate the effectiveness and practicability for the designed self-learning algorithms.

A novel completely mode-free integral reinforcement learning (CMFIRL)-based iteration algorithm is proposed in this article to compute the two-player zero-sum games and the Nash equilibrium problems, that is, the optimal control policy pairs, for tidal turbine system based on continuous-time Markov jump linear model with exact transition probability and completely unknown dynamics. First, the tidal turbine system is modeled into Markov jump linear systems, followed by a designed subsystem transformation technique to decouple the jumping modes. Then, a completely mode-free reinforcement learning algorithm is employed to address the game-coupled algebraic Riccati equations without using the information of the system dynamics, in order to reach the Nash equilibrium. The learning algorithm includes one iteration loop by updating the control policy and the disturbance policy simultaneously. Also, the exploration signal is added for motivating the system, and the convergence of the CMFIRL iteration algorithm is rigorously proved. Finally, a simulation example is given to illustrate the effectiveness and applicability of the control design approach.

This article explores a novel adaptive optimal control strategy for a class of sophisticated discrete-time nonlinear Markov jump systems (DTNMJSs) via Takagi–Sugeno fuzzy models and reinforcement learning (RL) techniques. First, the original nonlinear system model is represented by fuzzy approximation, while the relevant optimal control problem is equivalent to designing fuzzy controllers for linear fuzzy systems with Markov jumping parameters. Subsequently, we derive the fuzzy coupled algebraic Riccati equations for the fuzzy-based discrete-time linear Markov jump systems by using Hamiltonian–Bellman methods. Following this, an online fuzzy optimization algorithm for DTNMJSs as well as the associated equivalence proof is given. Then, a fully model-free off-policy fuzzy RL algorithm is derived with proved convergence for the DTNMJSs without using the information of system dynamics and transition probability. Finally, two simulation examples, respectively, related to the single-link robotic arm and the half-car active suspension are given to verify the effectiveness and good performance of the proposed approach.

This paper studies the online adaptive optimal controller design for a class of nonlinear systems through a novel policy iteration (PI) algorithm. By using the technique of neural network linear differential inclusion (LDI) to linearize the nonlinear terms in each iteration, the optimal law for controller design can be solved through the relevant algebraic Riccati equation (ARE) without using the system internal parameters. Based on PI approach, the adaptive optimal control algorithm is developed with the online linearization and the two-step iteration, i.e., policy evaluation and policy improvement. The convergence of the proposed PI algorithm is also proved. Finally, two numerical examples are given to illustrate the effectiveness and applicability of the proposed method.