This article proposes a dynamic self-triggered (DST) control strategy and a switching signal for a cascaded switched continuous stirred tank reactor (CSTR) system subject to asymmetric input constraints. First, the switched subsystems are established, and an improved nonquadratic performance function (PF) is designed, enabling the decentralized constrained switched control problem (CP) to be decomposed into a set of unconstrained optimal switched CPs for the individual subsystems. To reduce communication and computational burdens while eliminating the need for dedicated trigger-monitoring hardware, a dynamic self-triggering mechanism is incorporated into the controller. Subsequently, the DST Hamilton–Jacobi–Bellman equation associated with the transformed optimal switched CP is derived. To circumvent repeated trial-and-error and ease the selection of activation functions, a dual heuristic programming method utilizing echo state networks (ESNs) is adopted to approximate the PF. According to the Lyapunov theorem, the output weights of the ESN and the states of the closed-loop switched subsystems are proven to be uniformly ultimately bounded. Finally, the effectiveness of the proposed DST control strategy and switching signals is validated on a cascaded switched CSTR system comprising three reactors.

In this paper, the problem of adaptive fault-tolerant optimal control is investigated for nonstrict-feedback nonlinear systems with deferred output and performance constraints (DOPCs). By skillfully constructing the shifting transformation and finite-time constraining function, a novel error-dependent barrier function is innovated to achieve the preassigned time tracking performance without the conservative initial limitations, helping the property of DOPCs be preserved. Neural networks (NNs)-based reinforcement learning (RL) is exploited, which takes advantage of approximation means to solve the Hamiltonian equation under the backstepping framework, optimally reconciling the tracking performance and control behavior. By utilizing NNs to online estimate the unaware faults and uncertainty, an adaptive fault-tolerant optimal controller is designed. Finally, the efficiency of the proposed method is confirmed by simulations.

In this article, a finite-time learning-based optimal tracking problem for nonlinear systems with preassigned performance constraint is investigated. By designing a state predictor, a fuzzy approximator driven by prediction errors rather than tracking errors is formulated to precisely compensate the effect of the unknown uncertainties. The design realizes a decoupling of control and estimation loops, effectively ensuring transient approximation performance and avoiding chattering induced by nonzero initial tracking errors. Then, based on the estimated components, a robust steady-state control scheme embedded with a prescribed performance mechanism is tailored to guarantee that the output state can converge to a predefined range within a preassigned time. This endows the designed controller with a specified time tracking capability independence on control parameters. To make a tradeoff between tracking precision and energy cost, a finite-time learning-based optimal control policy is exploited by utilizing adaptive dynamic programming technique to serve as an adaptive supplementary controller, where single critic neural network is trained for acquiring the solution of the Hamilton–Jacobi–Bellman equation. Compared with the traditional gradient descent method, the established learning law is updated by introducing an auxiliary variable, which enhances learning performance and guarantees finite-time convergence of adaptive weights. Simulation examples examine the effectiveness and superiority of the suggested scheme.

This article explores the challenge of triggered optimal control for random differential equations (RDEs) with Markovian switching. We initially address the inherent contradiction between whether to comply with or bypass the event-triggered mechanism. By navigating this challenge, we ensure noise-to-state stability (NSS) for RDEs through event-triggered control (ETC). Furthermore, we establish that random nonlinear systems utilizing self-triggered control (STC) can achieve NSS, by setting a minimum triggering time to prevent Zeno behavior. Lastly, by adopting the adaptive dynamic programming (ADP) strategy, we develop self-triggered optimal control for random systems with Markovian switching, ensuring the uniform ultimate boundedness (UUB) of the signals in all closed-loop systems. This article addresses three key gaps in the field of RDE optimal control, contributing substantially to both theoretical and practical advancements. To demonstrate the method’s feasibility, we include a representative example with simulation results.

In this article, an adaptive dynamic programming (ADP)-based optimal control strategy for a series of fractional-order nonlinear systems (FONS) with unknown control directions is investigated. To eliminate the challenges posed by unknown control directions, fractional-order Nussbaum-type functions are introduced for FONS, expanding the range of possible applications. Additionally, since system performance is compromised by disturbances, a fractional-order disturbance observer is designed to counteract the effects of external disturbances and enhance system robustness. Furthermore, differential geometric methods are employed to investigate FONS, constructing appropriate diffeomorphism that provide equivalent systems for decoupled linearization. Then, an optimal control method is studied for a class of strictly feedback FONS, in which Nussbaum-type functions are combined with ADP theory during the backstepping design process. Finally, based on fractional Lyapunov stability theory and backstepping method, it is guaranteed that all signals of the closed-loop FONS are uniformly ultimately bounded (UUB). Numerical simulation and a PMSM model are utilized to verify the effectiveness of the presented method.

To address the finite-horizon coupled two-player mixedH2/H∞control challenge within a continuous-time affine nonlinear system, this research introduces a distinctiveQ-function and presents an innovative adaptive dynamic programming (ADP) method that operates autonomously of system-specific information. Initially, we formulate the time-varying Hamilton–Jacobi–Isaacs (HJI) equations, which pose a significant challenge for resolution due to their time-dependent and nonlinear nature. Subsequently, a novel offline policy iteration (PI) algorithm is introduced, highlighting its convergence and reinforcing the substantive proof of the existence of Nash equilibrium points. Moreover, a novel action-dependentQ-function is established to facilitate entirely model-free learning, representing the initial foray into the mixedH2/H∞control problem involving coupled players. The Lyapunov direct approach is employed to ensure the stability of the closed-loop uncertain affine nonlinear system under the ADP-based control scheme, guaranteeing uniform ultimate boundedness (UUB). Finally, a numerical simulation is conducted to validate the effectiveness of the aforementioned ADP-based control approach.

In this paper, to tackle the optimal control problem, we propose a$\mathcal{Q}$-Learning approach for continuous-time nonlinear systems without any dynamic information. Primarily, the Hamiltonian and optimum cost functions are utilized to articulate the$\mathcal{Q}$-function of continuous-time affine systems. To reduce the dependence of algorithms on system information, a novel$\mathcal{Q}$-Learning approach is derived to obtain optimal solutions of nonlinear continuous-time systems without requiring knowledge of either the drift information$p(x)$or input gain$q(x)$. To implement this approach, critic and actor neural networks can be iterated alternately using an integral reinforcement learning method to estimate the$\mathcal{Q}$-function. Furthermore, all signals in closed-loop system are demonstrated to be ultimate uniform bounded (UUB). It is worth noting that there exist rare literatures focused on the optimal control problem of continuous-time nonlinear uncertain systems via the$\mathcal{Q}$-Learning for actor/critic networks iteration. Finally, two simulations are used to confirm the effectiveness of the proposed algorithm.Note to Practitioners—Nonlinear continuous-time systems, being ubiquitous in engineering practice, are widely employed due to their versatility and effectiveness. Aiming at such systems, a$\mathcal{Q}$-learning approach with optimal feature is proposed to strengthen control efficiency while reduce costs. However, it is well known that accurately capturing all the dynamic information of the system is a formidable task in practical operation. This defect inevitably weakens the feasibility of model-based control algorithms. Since the$\mathcal{Q}$-learning algorithm presented in this paper does not require any dynamic knowledge of systems, it is promising enabler in enhancing the effectiveness and flexibility of engineering activities.

In this article, fractional-order nonautonomous system (FONAS) with the input delay and nonlinear terms are considered and investigated using adaptive fuzzy control method based on Q-learning. With the novel estimation model, the defined predictions for the error system determines the weights of the fuzzy logic system (FLS). On this basis, an error derivative-based cost function is introduced, which not only deals with the classic problem that quadratic term cost function is unbounded in infinite time, but also resolves the challenge that the exponential discount factor cost function fails to stabilize asymptotically. For the unmeasurable part of the state, the designed fuzzy observer eliminates the restriction on the gain parameters. Furthermore, based on the measured information and the actor–critic architecture of the online training FLSs, the improved adaptive fault-tolerant control (FTC) input approximate the optimal control. Utilizing a fractional-order Lyapunov method, the stability of FONAS with actuator faults is discussed, and a sufficient criterion for stability is obtained, which is easier to perform with convex optimization tools. Finally, numerical simulations are shown to display the effectiveness of the optimal adaptive fuzzy FTC strategy.

In order to solve the two-player Stackelberg differential game (SDG) for the continuous-time nonlinear Markov jump system (MJS), this article defines a unique$Q$-function and suggests a novel adaptive dynamic programming (ADP) method which is completely independent of system information. First, the optimal policies for the leader and follower are determined from down to the top, and it is further demonstrated that these policies are what make up the Stackelberg–Nash equilibrium point. Then, a novel action-dependent$Q$-function is established in order to attain completely model-free learning, which is the first attempt for SDG-based nonlinear MJS. Furthermore, the Lyapunov direct approach is employed to guarantee the stability of the closed-loop uncertain nonlinear MJS under the control scheme based on ADP, ensuring uniform ultimate boundedness (UUB). Ultimately, a numerical simulation is presented to validate the efficacy of the aforementioned ADP-based control approach.

This article investigates optimal regulation scheme between tumor and immune cells based on the adaptive dynamic programming (ADP) approach. The therapeutic goal is to inhibit the growth of tumor cells to allowable injury degree and maximize the number of immune cells in the meantime. The reliable controller is derived through the ADP approach to make the number of cells achieve the specific ideal states. First, the main objective is to weaken the negative effect caused by chemotherapy and immunotherapy, which means that the minimal dose of chemotherapeutic and immunotherapeutic drugs can be operational in the treatment process. Second, according to the nonlinear dynamical mathematical model of tumor cells, chemotherapy and immunotherapeutic drugs can act as powerful regulatory measures, which is a closed-loop control behavior. Finally, states of the system and critic weight errors are proved to be ultimately uniformly bounded with the appropriate optimization control strategy and the simulation results are shown to demonstrate the effectiveness of the cybernetics methodology.