This brief presents the adaptive optimal prescribed performance tracking solutions for the multiplayer nonlinear systems based on the adaptive critic learning scheme, where the tracking errors are constrained to a predefined bounded set. First, the general optimal tracking solutions of multiplayer nonlinear systems are presented. Every optimal tracking solution of multiple players consists of the steady-state part and the adaptive feedback part. The steady-state part can be obtained directly according to the tracking signal and system dynamics. Then, the adaptive feedback part can be studied with the prescribed performance constraints and adaptive critic learning such that multiple value functions achieve a Nash equilibrium with error constraints. Moreover, the convergence of the critic network weight is analyzed by the Lyapunov algorithm. Finally, simulation results and experiments are presented to demonstrate the satisfactory performance of the proposed method.

This paper proposes an adaptive finite-time sliding optimal tracking control (AFSOTC) for uncertain linear systems by combining finite-time optimal tracking with integral sliding mode technology. The AFSOTC includes two components: an optimal finite-time feedback control based on the finite-time control technique for a nominal model, and an integral finite-time sliding control to attenuate external disturbances. First, a sliding model is constructed to transform the tracking problem into a regular control one. The finite-time optimal controller based on the fixed-time stability technique and adaptive dynamic programming is proposed for the uncertain sliding model system, where an improved finite-time algebraic-Riccati-equation (ARE) with fixed-time convergence is online solved by a novel adaptive law. Then, an integral finite-time sliding control is constructed to resist the uncertain dynamics and interferences. Moreover, Lyapunov theory is applied to prove the parameter convergence and the stability of the closed-loop system with the AFSOTC. Finally, a linear system with comparisons and an autonomous underwater vehicle (AUV) with dynamic uncertainties and disturbance are applied to verify the correctness of the proposed method, and a linearization method for AUV underwater is presented.

In this paper, a novel adaptive integral reinforcement learning (AIRL) is utilized to online handle the finite-horizon optimum control policies of the partially unknown multi-input nonlinear system. Firstly, the concept of Nash equilibrium is introduced to make the multiple cost functions reach the saddle point. Then, dual neural networks (NNs) are applied to approach the performance index functions based on the integral reinforcement signal. Simultaneously, two novel learning algorithms are proposed to update the NN weights, in which the convergence of weights is proved. Then, the optimal strategies can be obtained by using the obtained weights. The designed controllers based on the data-driven AIRL scheme can avoid the internal state of the system and the derivatives of NN activations in the weight learning process. Finally, the stability of the controlled system is analyzed. An F-16 aircraft model and another nonlinear system are utilized to prove the validity and rationality of the algorithm.Note to Practitioners—There exist many multi-input systems in practical engineering, which includes multi-engine driven F-16 aircraft, large radar servo system and large artillery systems, etc. The finite-horizon optimal control of these systems is crucial for the better performance of system state. However, an accurate engineering model is difficult to obtain, and the finite-horizon cannot generally be achieved. To address these issues, this paper proposes the finite-horizon optimal control for these systems based on a novel adaptive integral reinforcement learning (AIRL). The AIRL can realize an optimal performance for these multi-input systems in finite-time without internal system dynamics, which is a good development for the multi-input system in practical engineering.

Model-free control does not require precise system dynamic information, but rather meets performance requirements by directly designing a control law. This method is particularly suitable for adaptive cruise control (ACC) systems in cyber-physical system (CPS) environments, as it can effectively handle the dynamic uncertainty and external disturbances of the system. Thus, this paper develops a novel online adaptive$H_{\infty }$control scheme for ACC systems. The main contribution of our study lies in achieving model-free learning by using the homotopy strategy, removing the necessity of prior model knowledge of initial stabilizing control policies, which has been a significant challenge in existing policy learning and ACC studies. This resolves a long-standing issue and enhances the applicability of our findings. For this purpose, a continuous time ACC system is first constructed with unknown system dynamics. Then, based a designed offline policy learning algorithm, a novel online policy algorithm based on system input-output data is introduced to solve the Riccati equation without any model information. Finally, experimental results demonstrate that the proposed control method significantly improves system performance, especially in terms of computational speed, it has improved by about 45% compared to classical reinforcement learning (RL) algorithm.

Conventional ACC method has great fluctuation and deviation when solving speed and distance control problems. Thus, this paper develops a reinforcement learning (RL) based robust optimal prescribed performance controller for ACC systems. To this end, we first construct a continuous time ACC system with unknown system dynamics (e.g., target vehicle acceleration, sensor and actuator attacks, etc). To estimate the unknown system dynamics, an unknown system dynamic estimator (USDE) is designed, where the unknown system dynamic can be accurately estimated by using the input-output information, this is helpful for controller design. Then, a RL based optimal control method is developed, where the prescribed performance function (PPF) is applied, the system states can be effectively defined within a certain range. To realize the online solution for optimal control, we design a new adaptive law based on the adaptive dynamic programming (ADP) framework to online learn the critic neural network (NN) weights, because of the strong convergence, the proposed learning algorithm can be effectively applied in practical industrial systems. Finally, the efficacy of the proposed control technique is tested through simulations and experiments.

The path tracking control problem for autonomous vehicle with uncertain dynamics requires simultaneous consideration of control optimality and safety-based performance constraints. In this paper, an adaptive optimal control method with prescribed performance is proposed to solve this problem, which contains two contributions: 1) by introducing a prescribed performance function (PPF) into adaptive dynamic programming (ADP), the controller can constrain the tracking error of the system within a specified performance boundary while optimizing the control cost; 2) the critic-only ADP is used for the controller design, which simplifies the commonly used actor-critic ADP scheme, and the convergence of the estimation error is guaranteed under FE conditions. On this basis, the neural network identification technique is introduced to deal with the unknown dynamic parameters of the vehicle system. The control scheme is able to strictly guarantee user-defined vehicle performance specifications with approximately optimal control performance. The stability of the closed-loop system is rigorously demonstrated by the Lyapunov method. In addition, the controller also embeds a radial basis function neural network (RBFNN) compensator to approximate the nonlinear external disturbances of the autonomous vehicle. Finally, the efficiency of the controller to achieve autonomous vehicle path tracking is verified by CarSim-Simulink simulation.

This article presents and practically validates an identifier-critic-based approximate dynamic programming (ADP) method to online address the optimal tracking control problem for nonlinear continuous-time unknown systems. The imposed assumption on precisely known system dynamics is obviated via a neural network (NN) identifier. A static control is first adopted to retain the steady-state tracking response, while an optimal control derived via the ADP method is proposed to regulate the tracking error by minimizing a cost function. A critic NN is then trained online to obtain the solution of the associated Hamilton–Jacobi–Bellman (HJB) equation. The learning of the identifier NN and critic NN is performed online simultaneously by tailoring a novel adaptation method, which can guarantee the convergence of the estimated NN weights. Consequently, the critic NN can be used to construct the optimal control policy directly, such that the actor NN used in the previous ADP schemes is avoided. Simulations are performed to verify the suggested control, and experiments on a helicopter plant are carried out to show its feasibility and improved control response.

Although robust control has been studied for decades, the output-feedback robust control design is still challenging in the control field. This article proposes a new approach to address the output-feedback robust control for continuous-time uncertain systems. First, we transform the robust control problem into an optimal control problem of the nominal linear system with a constructive cost function, which allows simplifying the control design. Then, a modified algebraic Riccati equation (MARE) is constructed by further investigating the corresponding relationship with the state-feedback optimal control. To solve the derived MARE online, the vectorization operation and Kronecker's product are applied to reformulate the output Lyapunov function, and then, a new online data-driven learning method is suggested to learn its solution. Consequently, only the measurable system input and output are used to derive the solution of the MARE. In this case, the output-feedback robust control gain can be obtained without using the unknown system states. The control system stability and convergence of the derived solution are rigorously proved. Two simulation examples are provided to demonstrate the efficacy of the suggested methods.