Aiming at the path-tracking problem of autonomous ground vehicle systems (AGVSs) under randomly activated network attacks, this article proposes a real-time dynamic integrated scheduling (RT-DIS) mechanism. First, the uncertain vehicle–road dynamics model is described by the Takagi–Sugeno fuzzy model through the time-varying speed of the vehicle. Second, a class of switching gain-scheduling controller is designed based on the difference of the normalized fuzzy membership functions in the vertical dimension of time and two different horizontal dimensions, which is fully adapted to all potential system dynamics behaviors of AGVSs. At the same time, by designing the homogeneous polynomially parameter-dependent Lyapunov function, the mean-square exponential stability satisfying the premise of${H_\infty }$performance is derived in a unified analysis framework. Finally, the simulation results demonstrate that the proposed RT-DIS mechanism can effectively enhance the vehicle lane-keeping performance.

This article proposes a hierarchical neural learning (HNL) algorithm for optimal tracking control (OTC) of nonlinear strict-feedback systems (SFSs) with unmatched disturbances (uMDs) and unknown dynamics. Leveraging the recursive structure of SFSs, we introduce the virtual target (VT) construction scheme in which each VT is a nonlinear mapping of the current state and desired output, thereby eliminating the noncausal that typically plagues discrete-time SFS control. The VTs serve as auxiliary inputs for low-order subsystems, while a time-varying affine Hamilton-Jacobi-Isaacs (HJI) formulation establishes an explicit relationship between the auxiliary control and the disturbance. The controller is synthesized directly from input-output data, removing the need for an accurate plant model. Within an adaptive dynamic programming (ADP) framework, we further enhance the neural architecture by replacing the conventional action network with a tracking network (T-network) whose energy function merges gradient information with future tracking errors, ensuring that each policy update simultaneously reduces control effort and improves tracking accuracy. Simulations confirm that the proposed HNL scheme achieves outstanding performance in both (optimal) tracking modes, exhibiting strong robustness to uMDs and significant model uncertainties.

This article investigates adaptive dynamic programming (ADP)-based optimal control issue of nonlinear stochastic systems with asymmetric input constraints. The solution starts with developing generalized fuzzy hyperbolic model (GFHM) in the stochastic system, which aims to approximate unknown nonlinear terms. By establishing a nonquadratic cost function, the constrained$H_{\infty }$control problem is converted into zero-sum game and Hamilton–Jacobi–Isaacs equation (HJIE) is derived. To solve the HJIE, the ADP algorithm is developed by constructing a single-network adaptive critic framework. Assisted by GFHM, the updating process obviates the necessity for the dynamics of unknown nonlinear terms. Under the designed controller, the stability of the stochastic system is guaranteed by the Lyapunov method. Two illustrative examples validate the presented method.

This article explores the challenge of triggered optimal control for random differential equations (RDEs) with Markovian switching. We initially address the inherent contradiction between whether to comply with or bypass the event-triggered mechanism. By navigating this challenge, we ensure noise-to-state stability (NSS) for RDEs through event-triggered control (ETC). Furthermore, we establish that random nonlinear systems utilizing self-triggered control (STC) can achieve NSS, by setting a minimum triggering time to prevent Zeno behavior. Lastly, by adopting the adaptive dynamic programming (ADP) strategy, we develop self-triggered optimal control for random systems with Markovian switching, ensuring the uniform ultimate boundedness (UUB) of the signals in all closed-loop systems. This article addresses three key gaps in the field of RDE optimal control, contributing substantially to both theoretical and practical advancements. To demonstrate the method’s feasibility, we include a representative example with simulation results.

This paper studies the mixed$H_{2}/H_{\infty} $control problem for nonlinear closed-loop Stackelberg games via adaptive dynamic programming (ADP) technology. Firstly, by constructing a cost function with the Lagrange multiplier, the hierarchical Stackelberg game problem is transformed into a coupled Hamilton-Jacobi-Isaacs (HJI) equations problem. In the second place, the critic-actor neural networks (NNs) framework is established to approximate the cost functions and control strategies of Stackelberg game. The corresponding algorithm flow is given. This is a novel idea that NNs are used to approximate the control strategies in Stackelberg game that cannot be given a specific form. Finally, the algorithm is applied to the load frequency control (LFC) problem of single-area power system. The optimal neural network weights are obtained by the designed ADP algorithm. And the comparison results of the three control schemes show that the mixed$H_{2}/H_{\infty} $based on Stackelberg game can not only achieve better frequency response, but also have better performance indicators for leader. Note to Practitioners—$H_{\infty} $control studies the anti-interference ability of the system, and the designed controller should attenuate the suppression coefficient of adjusting error and external disturbance to a given minimum level.$H_{2}$control is optimal control, and the controller is designed to minimize the cost function of system state and system input. In general, in$H_{\infty} $or$H_{2}$control, only a single control performance can be achieved. In the control system design, if the control system satisfies both$H_{\infty} $and$H_{2}$performance, the overall performance of the system will be greatly improved. Therefore, in engineering practice, in order to achieve satisfactory system performance, the mixed$H_{2}/H_{\infty} $control is more attractive, because it integrates the advantages of$H_{\infty} $control and$H_{2}$control. On the other hand, in a real game, Stackelberg game is the best choice if two players are not on the same level. This paper takes Stackelberg game as the main research object. Stackelberg game is a kind of game problem in which the participants have different decision-making status, and there is one or more decision makers higher than other decision makers. As a traditional control strategy, PID control and SMC control are not uncommon in the research of LFC. With the development of control theory and application, state space and optimal control strategy are also applied to the design of load frequency control system. Intelligent control is the current development trend of LFC research.

This article investigate the cooperative differential game (CDG) for continuous-time linear Itô stochastic systems with markovian jumps (SSMJ) to obtain the Pareto solutions. Different from most existing works studying nonzero-sum games, this article studies the CDG on the quadratic infinite horizon for the Itô-type SSMJ with unknown system matrix and transition probability. A novel$Q$-learning online algorithm is developed, which consists of that (i) the optimal control problem is equivalent to solving a stochastic algebraic Riccatic equation (ARE); (ii) the joint cost function is approximated by a critic neural network (NN) and Pareto efficient is approximated by two actor NNs. The rigorous stability analysis shows that the system state for SSMJ and the NN weight errors are uniformly ultimately bounded (UUB). Finally, the theory analysis is validated by a numerical example with detailed discussions.Note to Practitioners—In practical applications, many systems are often affected by the change of external environment or the failure of internal components, which leads to the random jump of system parameters. Markovian jump system can effectively describe the above problems. And when the system model is disturbed by the internal parameters, the system control input and external environment, the random errors of state measurement, and other random factors, the deterministic model can no longer accurately describe the controlled system. Therefore, the SSMJ can describe practical problems more accurately. By cooperation, in general, the cost one specific player incurs is not uniquely determined anymore. If all players decide, for example, to use their control variables to reduce the cost of player 1 as much as possible, a different minimum is attained for player 1 compared with that in the case where all players agree collectively to help a different player in minimizing his cost. So, depending on how the players choose to ‘divide’ their control efforts, a player incurs different ‘minima’. Therefore, we will design an online learning algorithm to obtain pareto solutions with different weights. On the other hand, in practice, it is difficult to obtain an accurate system model. In order to solve this problem, a novel scheme is designed by using$Q$-learning technology, which does not need system matrix.

In this paper, to tackle the optimal control problem, we propose a$\mathcal{Q}$-Learning approach for continuous-time nonlinear systems without any dynamic information. Primarily, the Hamiltonian and optimum cost functions are utilized to articulate the$\mathcal{Q}$-function of continuous-time affine systems. To reduce the dependence of algorithms on system information, a novel$\mathcal{Q}$-Learning approach is derived to obtain optimal solutions of nonlinear continuous-time systems without requiring knowledge of either the drift information$p(x)$or input gain$q(x)$. To implement this approach, critic and actor neural networks can be iterated alternately using an integral reinforcement learning method to estimate the$\mathcal{Q}$-function. Furthermore, all signals in closed-loop system are demonstrated to be ultimate uniform bounded (UUB). It is worth noting that there exist rare literatures focused on the optimal control problem of continuous-time nonlinear uncertain systems via the$\mathcal{Q}$-Learning for actor/critic networks iteration. Finally, two simulations are used to confirm the effectiveness of the proposed algorithm.Note to Practitioners—Nonlinear continuous-time systems, being ubiquitous in engineering practice, are widely employed due to their versatility and effectiveness. Aiming at such systems, a$\mathcal{Q}$-learning approach with optimal feature is proposed to strengthen control efficiency while reduce costs. However, it is well known that accurately capturing all the dynamic information of the system is a formidable task in practical operation. This defect inevitably weakens the feasibility of model-based control algorithms. Since the$\mathcal{Q}$-learning algorithm presented in this paper does not require any dynamic knowledge of systems, it is promising enabler in enhancing the effectiveness and flexibility of engineering activities.

Distributed control of DC microgrid is becoming more and more important in modern power system. An important control goal is to ensure voltage stability and current sharing of DC bus. In the presence of constant power loads and uncertainties, a novel distributed quadratic optimum control technique based on reinforcement learning (RL) is developed in order to ensure correct current sharing and adequate performance. Firstly, the system model with power coupling is established and transformed into a linear heterogeneous multi-agent system (MAS) with unknown disturbances. Subsequently, a neural network (NN)-based adaptive model-free observer is developed. Since not all followers have direct access to the leader’s information, a distributed cooperation performance index with discount component is created by fusing the dynamics of the observer and the follower. The$Q$-learning technique is applied to obtain the optimal control strategy to achieve voltage stabilization and current sharing without using system dynamics. Finally, simulation and experimental results show the effectiveness of this strategy.

This article focuses on the Pareto optimal issues of nonlinear game systems with asymmetric input saturation under dynamic event-triggered mechanism (DETM). First, the safe control is guaranteed by transforming the system with safety constraints into the one without state constraints utilizing barrier function. The united cost function integrating nonquadratic utility function is constructed to provide the foundation to achieve the Pareto optimal solutions. Then, the adaptive dynamic programming method with concurrent learning is proposed to approximate the Pareto optimal strategies wherein both current and historical data are utilized. To further lessen the consumptions of computation/communication resources, the DETM is integrated into the adaptive algorithm framework which can avoid Zeno phenomena. All the signals of the closed-loop system are proved to be uniformly ultimately bounded. Finally, the simulation results are given to validate the effectiveness of the proposed method from several aspects.

This article addresses the solution of continuous-time linear Itô stochastic systems with Markovian jumps using an online policy iteration (PI) approach grounded in -learning. Initially, a model-dependent offline algorithm, structured according to traditional optimal control strategies, is designed to solve the algebraic Riccati equation (ARE). Employing Lyapunov theory, we rigorously derive the convergence of the offline PI algorithm and the admissibility of the iterative control law through mathematical analysis. This article represents the first attempt to tackle these technical challenges. Subsequently, to address the limitations inherent in the offline algorithm, we introduce a novel online -learning algorithm tailored for Itô stochastic systems with Markovian jumps. The proposed -learning algorithm obviates the need for transition probabilities and system matrices. We provide a thorough stability analysis of the closed-loop system. Finally, the effectiveness and applicability of the proposed algorithms are demonstrated through a simulation example, underpinned by the theorems established herein.

In order to solve the two-player Stackelberg game for the continuous-time nonlinear stochastic system, using the Takagi–Sugeno (T-S) fuzzy stochastic model, this paper defines the novel$Q$-functions and suggests an adaptive dynamic programming (ADP)-based approach that is completely model-free. First, based on the T-S fuzzy model, the overall fuzzy control policies with corresponding cost functions are designed where coupled penalty functions are considered. Subsequently, we create a novel two-level algorithm based on integral reinforcement learning and provide the proof of convergence to overcome challenge of computing the optimal cost functions analytically. On this basis, in order to achieve entirely model-free learning, which is the first attempt in solving fuzzy stochastic nonlinear continuous-time systems with the Stackelberg game problem, the innovative action-dependent$Q$-functions are developed. Fuzzy linearization technique and$Q$-learning algorithm are ingeniously combined in this article to solve their respective difficulties. In addition, the Lyapunov approach under the ADP-based control scheme ensures the stability of the closed-loop nonlinear stochastic system based on fuzzy approximation and is characterized by asymptotic stability. Finally, a numerical simulation is offered to show the efficacy of the existing ADP-based control technique.

In this article, an integral reinforcement learning (IRL)-based event-triggered guarantee cost control (GCC) approach is proposed for stochastic systems which are modulated by randomly time-varying parameters. First, with the aid of the RL algorithm, the optimal GCC (OGCC) problem is converted into an optimal zero-sum game by solving a modified Hamilton-Jacobin-Isaac (HJI) equation of the auxiliary system. Moreover, in order to address the stochastic zero-sum game, we propose an on-policy IRL-based control approach involved by the multivariate probabilistic collocation method (MPCM), which can accurately predict the mean value of uncertain functions with randomly time-varying parameters. Furthermore, a novel GCC method, which combines the explorized IRL algorithm and MPCM, is designed to relax the restriction of knowing the system dynamics for the class of stochastic systems. On this foundation, for the purpose of reducing computation cost and avoiding the waste of resources, we propose an event-triggered GCC approach involved with explorized IRL and MPCM by utilizing critic-actor-disturbance neural networks (NNs). Meanwhile, the weight vectors of three NNs are updated simultaneously and aperiodically according to the designed triggering condition. The ultimate boundedness (UB) properties of the controlled systems have been proved by means of the Lyapunov theorem. Finally, the effectiveness of the developed GCC algorithms is illustrated via two simulation examples.

In order to solve the two-player Stackelberg differential game (SDG) for the continuous-time nonlinear Markov jump system (MJS), this article defines a unique$Q$-function and suggests a novel adaptive dynamic programming (ADP) method which is completely independent of system information. First, the optimal policies for the leader and follower are determined from down to the top, and it is further demonstrated that these policies are what make up the Stackelberg–Nash equilibrium point. Then, a novel action-dependent$Q$-function is established in order to attain completely model-free learning, which is the first attempt for SDG-based nonlinear MJS. Furthermore, the Lyapunov direct approach is employed to guarantee the stability of the closed-loop uncertain nonlinear MJS under the control scheme based on ADP, ensuring uniform ultimate boundedness (UUB). Ultimately, a numerical simulation is presented to validate the efficacy of the aforementioned ADP-based control approach.

This technical note discusses the design process of dynamic event-triggered control (DETC) with the mixed$H_{2}/H_\infty $for partially unknown nonlinear stochastic systems. The purpose of this problem is to design a controller to make the closed-loop system achieve the expected$H_{2}$performance under the condition that the$H_\infty $continuous attenuation level is protected. Firstly, a two-player non-zero-sum game for stochastic system is given. Then we prove the optimal control strategy and the worst interference, which constitute the Nash equilibrium solution and can be derived from the corresponding Hamiltonian functions. Furthermore, two neural networks (NNs) are used to realize Nash equilibrium. Under the condition of dynamic event-triggered mechanism (DETM), the control strategy only is updated at the trigger moment. In addition, the stability and weights convergence of the system are proved mathematically. Finally, two numerical example are given to prove it.Note to Practitioners—In the practice of control engineering, mixed$H_{2}/H_\infty $control can not only evaluate the$H_{2}$performance index of the transient behavior of the system, but also measure the$H_\infty $performance index of the system’s robustness to external disturbances and parameter uncertainty. And many useful signals and interference vary randomly. Therefore, the optimal control of stochastic systems and dynamics play an important role in the modern industry. Saving control resources is very important for actual production. Therefore DETC is considered in this paper. On the other hand, in practice, accurate system models are difficult to obtain. In order to tackle this difficulty, by designing a novel scheme via integral reinforcement learning technique, the system relaxes the requirement of drift dynamic and Zeno behavior is avoided.

This article presents an online off-policy policy iteration (PI) algorithm using reinforcement learning (RL) to optimize the distributed synchronization problem for nonlinear multiagent systems (MASs). First, considering that not every follower can directly obtain the leader's information, a novel adaptive model-free observer based on neural networks (NNs) is designed. Moreover the feasibility of the observer is strictly proved. Subsequently, combined with the observer and follower dynamics, an augmented system and a distributed cooperative performance index with discount factors are established. On this basis, the optimal distributed cooperative synchronization problem changes into solving the numerical solution of the Hamilton-Jacobian-Bellman (HJB) equation. Finally, an online off-policy algorithm is proposed, which can be used to optimize the distributed synchronization problem of the MASs in real time based on measured data. In order to prove the stability and convergence of the online off-policy algorithm more conveniently, an offline on-policy algorithm whose stability and convergence are proved is given before the online off-policy algorithm is proposed. We give a novel mathematical analysis method for establishing the stability of the algorithm. The effectiveness of the theory is verified by simulation results.

In this article, a dynamic event-triggered stochastic adaptive dynamic programming (ADP)-based problem is investigated for nonlinear systems with a communication network. First, a novel condition of obtaining stochastic input-to-state stability (SISS) of discrete version is skillfully established. Then, the event-triggered control strategy is devised, and a near-optimal control policy is designed using an identifier-actor-critic neural networks (NNs) with an event-sampled state vector. Above all, an adaptive static event sampling condition is designed by using the Lyapunov technique to ensure ultimate boundedness (UB) for the closed-loop system. However, since the static event-triggered rule only depends on the current state, regardless of previous values, this article presents an explicit dynamic event-triggered rule. Furthermore, we prove that the lower bound of sampling interval for the proposed dynamic event-triggered control strategy is greater than one, which avoids the so-called triviality phenomenon. Finally, the effectiveness of the proposed near-optimal control pattern is verified by a simulation example.

In this article, the neural network (NN)-based adaptive dynamic programming (ADP) event-triggered control method is presented to obtain the near-optimal control policy for the model-free finite-horizon H∞ optimal tracking control problem with constrained control input. First, using available input-output data, a data-driven model is established by a recurrent NN (RNN) to reconstruct the unknown system. Then, an augmented system with event-triggered mechanism is obtained by a tracking error system and a command generator. We present a novel event-triggering condition without Zeno behavior. On this basis, the relationship between event-triggered Hamilton-Jacobi-Isaacs (HJI) equation and time-triggered HJI equation is given in Theorem 3. Since the solution of the HJI equation is time-dependent for the augmented system, the time-dependent activation functions of NNs are considered. Moreover, an extra error is incorporated to satisfy the terminal constraints of cost function. This adaptive control pattern finds, in real time, approximations of the optimal value while also ensuring the uniform ultimate boundedness of the closed-loop system. Finally, the effectiveness of the proposed near-optimal control pattern is verified by two simulation examples.

The time-varying output bipartite formation containment (TVOBFC) problem for linear multiagent systems (MASs) under directed graphs is an important problem. However, the methods in existing works rely on the global information of the MASs or do not use event-triggered communication. This article investigates two kinds of TVOBFC problems for heterogeneous linear MASs under signed digraphs by event-triggered communication. For the first case where leaders have the same dynamics, the innovative fully distributed event-triggered protocol for the follower is proposed. In this case, the followers form the preset formation shape. For the second case where leaders have different dynamics, the leaders are divided into two groups. One group can directly obtain the output information of the virtual leader, while the other group cannot. In order to make leaders achieve the formation shape and track the virtual leader, two kinds of innovative observers are designed for two kinds of leaders to estimate the state of the virtual leader, and the control protocol is designed for each leader based on the designed observers. Then, the control law for each follower is designed to solve the formation containment problem. Finally, two examples are introduced to illustrate the main results.

For nonlinear systems, a novel adaptive dynamic programming (ADP) algorithm of self-triggered control (STC) strategy is proposed. This is a novel attempt to introduce self-triggering into the ADP algorithm. First, an identifier based on a generalized fuzzy hyperbolic model (GFHM) is established, which only uses input–output data to reconstruct the unknown system, thus reducing the requirements for system dynamics. Then, the critic neural network (NN) adjusts continuously, while actor NN updates the control strategy only at triggering instants. The event-triggered control (ETC) reduces the use of control resources and improves the anti-interference capability. However, it requires dedicated hardware to monitor whether triggering rules are violated, which is not feasible on most general-purpose devices. Hence, we propose a novel technique, which uses the current state of the device to determine the state measurement at the next moment, calculate the control law, and then abandon persistently monitoring of the plant. This technique is called STC. Finally, the closed-loop system is guaranteed to be ultimate uniform boundedness (UUBs). Furthermore, a simulation example is given.

In this article, the mixed$H_{2}/H_{\infty }$control problem with closed-loop Stackelberg games for nonlinear stochastic systems is studied. With the help of the value functions of leader and follower, the Stackelberg game problem is transformed into the numerical solution of Hamilton–Jacobi (HJ) equations with generalized Hamiltonian functions. On the premise of ensuring that the system state is uniformly ultimate bounded (UUB), an adaptive neural networks (NNs) algorithm is proposed to approach the closed-loop Stackelberg equilibrium. This is the first time that adaptive dynamic programming (ADP) is used to solve the closed-loop Stackelberg games control problem of nonlinear stochastic systems. Further, in order to reduce computational load and save communication resources, the event-triggering control strategy is obtained. Moreover, a novel stochastic comparison lemma is given, which is used to exclude Zeno behavior during the learning process. Finally, a simulation example is used to demonstrate the usefulness of the suggested near-optimal control scheme.

This article concentrates on the neural network (NN)-based adaptive sliding-mode control (SMC) for fuzzy fractional-order system (FOS),$\alpha \in (0,1)$. First of all, a novel method of optimal SMC approach is developed for fuzzy FOSs by using the adaptive dynamic program (ADP), integral sliding mode, and NN with unmatched disturbances and time-varying delays. Next, to weaken the influence of the nonlinearities, the SMC strategy is proposed for the specific system, which is established on the corresponding SMD to ensure that the FOS reach the SMS in a finite time. Moreover, it shows that the matrix of SMS can be described by the linear matrix inequality (LMI). Furthermore, the Hamilton–Jacobi–Bell man (HJB) equation can be approximated by a single NN method, and the Lyapunov stability principle proves that the weight errors are convergent, further guaranteeing the asymptotically stability of the fuzzy FOS. Finally, to display that the above-presented policy is effective, simulation results are presented.