This article presents a decentralized dynamic event-sampled control (ESC) strategy for multi-unit power systems (MUPSs) subject to mismatched interconnections. Initially, with the introduction of a dynamic event-sampling mechanism, the decentralized ESC problem of MUPSs is converted into a set of event-sampled optimal control problems of auxiliary subsystems. It is demonstrated that all the solutions of the event-sampled Hamilton-Jacobi-Bellman equations (ES-HJBEs) arising in these optimal control problems together constitute the decentralized dynamic ESC law. Then, in order to solve the ES-HJBEs, the critic neural networks (CNNs) within the adaptive dynamic programming framework are constructed. The CNNs’ weights are updated via simultaneously using the gradient descent method and concurrent learning. The benefits of such a tuning rule are that it not only makes full use of state data (including historical and instantaneous state data) but also no longer requires the persistence of excitation condition. Moreover, uniform ultimate boundedness of the closed-loop auxiliary subsystem states and the CNNs’ weight estimation errors are proved via Lyapunov approach. Finally, simulations of a three-unit power system are given to validate the present decentralized dynamic ESC scheme.

We develop a dynamic decentralized event-triggered tracking control strategy for cascade interconnected continuous stirred tank reactor (CSTR) systems subject to asymmetric input limits. Initially, we construct auxiliary augmented subsystems related to the cascade interconnected CSTR systems. Then, by introducing modified nonquadratic cost functions for the auxiliary augmented subsystems, we convert the decentralized constrained tracking control problem into an array of unconstrained optimal regulation problems. After that, with the construction of dynamic event-triggering mechanisms, we propose the event-triggered Hamilton-Jacobi-Bellman equations (ET-HJBEs) associated with the transformed optimal regulation problems. To approximately solve the ET-HJBEs, we design critic neural networks (CNNs) in the adaptive dynamic programming framework with the CNNs’ weights being updated through the gradient descent method. Furthermore, we use Lyapunov method to prove that the CNNs’ weight estimation errors and the tracking error are stable in the sense of uniform ultimate boundedness. Finally, simulation results of the three-reactor cascade interconnected CSTR systems are provided to validate the present dynamic decentralized event-triggered tracking control scheme.

Robust decoding performance is essential for the practical deployment of brain-computer interface (BCI) systems. Existing EEG decoding models often rely on large amounts of annotated data collected through specific experimental setups, which fail to address the heterogeneity of data distributions across different domains. This limitation hinders BCI systems from effectively managing the complexity and variability of real-world data. To overcome these challenges, we propose Synchronized Self-Training Domain Adaptation (SSTDA) for cross-domain motor imagery classification. Specifically, SSTDA leverages labeled signals from a source domain and applies self-training to unlabeled signals from a target domain, enabling the simultaneous training of a more robust classifier. The raw EEG signals are mapped into a latent space by a feature extractor for discriminative representation learning. A domain-shared latent space is then learned by optimizing the feature extractor with both source and target samples, using an easy-tohard self-training process. We validate the method with extensive experiments on two public motor imagery datasets: Dataset IIa of BCI Competition IV and the High Gamma dataset. In the inter-subject task, our method achieves classification accuracies of 64.43% and 80.40%, respectively. It also outperforms existing methods in the inter-session task. Moreover, we develope a new six-class motor imagery dataset and achieve test accuracies of 77.09% and 80.18% across different datasets. All experimental results demonstrate that our SSTDA outperforms existing algorithms in inter-session, inter-subject, and inter-dataset validation protocols, highlighting its capability to learn discriminative, domain-invariant representations that enhance EEG decoding performance.

We consider a robust dynamic event-driven control (EDC) problem of nonlinear systems having both unmatched perturbations and unknown styles of constraints. Specifically, the constraints imposed on the nonlinear systems' input could be symmetric or asymmetric. Initially, to tackle such constraints, we construct a novel nonquadratic cost function for the constrained auxiliary system. Then, we propose a dynamic event-triggering mechanism relied on the time-based variable and the system states simultaneously for cutting down the computational load. Meanwhile, we show that the robust dynamic EDC of original nonlinear-constrained systems could be acquired by solving the event-driven optimal control problem of the constrained auxiliary system. After that, we develop the corresponding event-driven Hamilton-Jacobi-Bellman equation, and then solve it through a unique critic neural network (CNN) in the reinforcement learning framework. To relax the persistence of excitation condition in tuning CNN's weights, we incorporate experience replay into the gradient descent method. With the aid of Lyapunov's approach, we prove that the closed-loop auxiliary system and the weight estimation error are uniformly ultimately bounded stable. Finally, two examples, including a nonlinear plant and the pendulum system, are utilized to validate the theoretical claims.

This article investigates distributed adaptive leader-following consensus tracking optimal control problem for nonlinear multiagent systems (MASs) subject to unknown nonlinearities and uncertain external disturbances. In contrast to traditional centralized control, the primary challenge is that only partial subsystems can access the desired reference trajectory. To address this, positive time-varying smooth function compensating terms are introduced to counteract the effects of uncertain external disturbances and unknown desired trajectories. Then, by fusing consensus errors into the backstepping technique, feedforward controllers are given. On this basis, the controlled nonlinear systems are transformed into an equivalent affine form, and feedback optimal controllers are designed using actor and critic neural networks (NNs) to execute control behavior and evaluate control performance. The whole control laws comprise both feedforward and feedback controllers. The proposed distributed adaptive consensus control protocol can simultaneously achieve desired optimal control performance and minimize the cost function, as demonstrated through theoretical analysis and simulation results.

This article considers the robust dynamic event-driven tracking control problem of nonlinear systems having mismatched disturbances and asymmetric input constraints. Initially, to tackle the asymmetric constraints, a novel nonquadratic value function is constructed for the original system. This makes the asymmetrically constrained tracking control problem transformed into an unconstrained optimal regulation problem. Then, a dynamic event-driven mechanism is proposed. Meanwhile, the event-driven Hamilton-Jacobi-Bellman equation (ED-HJBE) is developed for the optimal regulation problem in order to acquire the optimal control with distinctly decreased computational burden. To solve the ED-HJBE, a single critic neural network (CNN) is designed in the adaptive dynamic programming framework. Meanwhile, the gradient descent method is employed to update the CNN's weights. After that, both the weight estimation error and the tracking error are proved to be uniformly ultimately bounded via Lyapunov's direct method. Finally, simulations of the spring-mass-damper system and the pendulum plant are separately utilized to validate the established theoretical claims.

A robust decoding model that can efficiently deal with the subject and period variation is urgently needed to apply the brain-computer interface (BCI) system. The performance of most electroencephalogram (EEG) decoding models depends on the characteristics of specific subjects and periods, which require calibration and training with annotated data prior to application. However, this situation will become unacceptable as it would be difficult for subjects to collect data for an extended period, especially in the rehabilitation process of disability based on motor imagery (MI). To address this issue, we propose an unsupervised domain adaptation framework called iterative self-training multisubject domain adaptation (ISMDA) that focuses on the offline MI task. First, the feature extractor is purposefully designed to map the EEG to a latent space of discriminative representations. Second, the attention module based on dynamic transfer matches the source domain and target domain samples with a higher coincidence degree in latent space. Then, an independent classifier oriented to the target domain is employed in the first stage of the iterative training process to cluster the samples of the target domain through similarity. Finally, a pseudolabel algorithm based on certainty and confidence is employed in the second stage of the iterative training process to adequately calibrate the error between prediction and empirical probabilities. To evaluate the effectiveness of the model, extensive testing has been performed on three publicly available MI datasets, the BCI IV IIa, the High gamma dataset, and Kwon et al. datasets. The proposed method achieved 69.51%, 82.38%, and 90.98% cross-subject classification accuracy on the three datasets, which outperforms the current state-of-the-art offline algorithms. Meanwhile, all results demonstrated that the proposed method could address the main challenges of the offline MI paradigm.

We develop a decentralized dynamic event-based control strategy for nonlinear systems subject to matched interconnections. To begin with, we introduce a dynamic event-based sampling mechanism, which relies on the system's states and the variables generated by time-based differential equations. Then, we prove that the decentralized event-based controller for the whole system is composed of all the optimal event-based control policies of nominal subsystems. To derive these optimal event-based control policies, we design a critic-only architecture to solve the related event-based Hamilton-Jacobi-Bellman equations in the reinforcement learning framework. The implementation of such an architecture uses only critic neural networks (NNs) with their weight vectors being updated through the gradient descent method together with concurrent learning. After that, we demonstrate that the asymptotic stability of closed-loop nominal subsystems and the uniformly ultimate boundedness stability of critic NNs' weight estimation errors are guaranteed by using Lyapunov's approach. Finally, we provide simulations of a matched nonlinear-interconnected plant to validate the present theoretical claims.

This article develops an intermittent feedback optimal control scheme for nonlinear systems with asymmetric input saturation using a dynamic event-triggering mechanism. First, an infinite horizon nonquadratic value function with a novel integrand is formulated for the studied system to evaluate the performance, tackle the asymmetric input saturation, and remove certain rigorous assumptions in prior related studies. Second, a critic neural network (CNN) in the adaptive dynamic programming framework is constructed to obtain the optimal event-triggered control (ETC). An improved concurrent learning technique is then developed to update the CNN’s weights without requiring the persistence of excitation condition. Compared with the static ETC scheme, the present dynamic ETC strategy consumes fewer computational resources. Third, the uniform ultimate boundedness of the state, the weight estimation error, and the internal dynamic variable are assured, and the Zeno behavior is excluded. Finally, a rotational-translational actuator system is given to validate the developed intermittent feedback optimal control scheme.

This article considers the$H_{\infty }$control problem of nonlinear systems having unavailable dynamics and asymmetric saturating actuators. Initially, such an$H_{\infty }$control problem is converted into the zero-sum game with a nonquadratic cost function being introduced. Then, in order to solve the Hamilton–Jacobi–Isaacs equation arising in the zero-sum game, a simultaneous policy iteration (SPI) algorithm is developed under the adaptive dynamic programming framework. Meanwhile, it is proved that the convergence of the SPI algorithm in essence amounts to the convergence of the sequential PI algorithm. To implement the SPI algorithm, the critic, the actor, and the perturbation neural networks (NNs) are, respectively, constructed to estimate the cost function, the control policy, and the perturbation. The three NNs’ weights are simultaneously determined by using the least-squares method together with the Monte Carlo integration technique. A remarkable characteristic of such an SPI algorithm is that arbitrary control policies and perturbations are applicable in the learning process. This makes system’s information be able to be replaced by the data collected along system’s trajectories in advance. More importantly, the persistence of the excitation condition is not required. Finally, simulations of two nonlinear examples are given to validate the present SPI algorithm.

We study the dynamic event-driven H∞ constrained control problem through approximate dynamic programming (ADP). Differing from the existing literature considering systems with either symmetric constraints or asymmetric constraints, we consider the two different constraints simultaneously. Initially, by constructing a generalized nonquadratic value function, we transform the H∞ constrained control problem into an unconstrained two-player zero-sum game. Then, we present an event-driven Hamilton–Jacobi–Isaacs equation (ED-HJIE) corresponding to the zero-sum game for lowering down the computational load. To solve the ED-HJIE, we propose a dynamic triggering mechanism together with a sole critic neural network (CNN) being built under the ADP framework. The CNN’s weights are tuned via the gradient descent approach. After that, we prove uniform ultimate boundedness of the closed-loop system and the CNN’s weight estimation error via Lyapunov’s method. Finally, we separately use an F16 aircraft plant and an inverted pendulum system to validate the present theoretical claims.

We consider the decentralized control problem of a class of continuous-time nonlinear systems with mismatched interconnections. Initially, with the discounted cost functions being introduced to auxiliary subsystems, we have the decentralized control problem converted into a set of optimal control problems. To derive solutions to these optimal control problems, we first present the related Hamilton-Jacobi-Bellman equations (HJBEs). Then, we develop a novel critic learning method to solve these HJBEs. To implement the newly developed critic learning approach, we only use critic neural networks (NNs) and tune their weight vectors via the combination of a modified gradient descent method and concurrent learning. By using the present critic learning method, we not only remove the restriction of initial admissible control but also relax the persistence-of-excitation condition. After that, we employ Lyapunov's direct method to demonstrate that the critic NNs' weight estimation error and the states of closed-loop auxiliary systems are stable in the sense of uniform ultimate boundedness. Finally, we separately provide a nonlinear-interconnected plant and an unstable interconnected power system to validate the present critic learning approach.

We study the decentralized event-driven control problem of nonlinear dynamical systems with mismatched interconnections and asymmetric input constraints. To begin with, by introducing a discounted cost function for each auxiliary subsystem, we transform the decentralized event-driven constrained control problem into a group of nonlinear$H_{2}$-constrained optimal control problems. Then, we develop the event-driven Hamilton–Jacobi–Bellman equations (ED-HJBEs), which arise in the nonlinear$H_{2}$-constrained optimal control problems. Meanwhile, we demonstrate that all the solutions of the ED-HJBEs together keep the overall system stable in the sense of uniform ultimate boundedness (UUB). To solve the ED-HJBEs, we build a critic-only architecture under the framework of adaptive critic designs. The architecture only employs critic neural networks and updates their weight vectors via the gradient descent method. After that, based on the Lyapunov approach, we prove that the UUB stability of all signals in the closed-loop auxiliary subsystems is assured. Finally, simulations of an illustrated nonlinear interconnected plant are provided to validate the present designs.

This article considers the decentralized stabilization problem of continuous-time nonlinear systems subject to unmatched interconnections and asymmetric input constraints. Initially, with the nonquadratic value functions being introduced to constrained auxiliary subsystems, the decentralized stabilization problem is converted into an array of nonlinear optimal control problems. It is proved that all solutions of these nonlinear optimal control problems together assure asymptotic stability of the entire system. Then, in the framework of adaptive critics, the critic-only architecture is built to solve the Hamilton–Jacobi–Bellman equations associated with these solutions. The critic-only architecture is implemented via critic neural networks (NNs) with their weight vectors being tuned through an improved gradient descent method. A remarkable feature of the present gradient descent approach is that it simultaneously utilizes previously stored and instantaneous state data, which makes the persistence of excitation conditions relaxed. After that, with Lyapunov’s techniques being employed, asymptotic stability of the closed-loop auxiliary subsystems and uniform ultimate boundedness of the critic NNs’ weight estimation errors are demonstrated. Finally, simulations of an unmatched interconnected nonlinear plant are provided to validate the present decentralized control method.

In this article, a novel distributed policy iteration algorithm is established for infinite horizon optimal control problems of continuous-time nonlinear systems. In each iteration of the developed distributed policy iteration algorithm, only one controller's control law is updated and the other controllers' control laws remain unchanged. The main contribution of the present algorithm is to improve the iterative control law one by one, instead of updating all the control laws in each iteration of the traditional policy iteration algorithms, which effectively releases the computational burden in each iteration. The properties of distributed policy iteration algorithm for continuous-time nonlinear systems are analyzed. The admissibility of the present methods has also been analyzed. Monotonicity, convergence, and optimality have been discussed, which show that the iterative value function is nonincreasingly convergent to the solution of the Hamilton-Jacobi-Bellman equation. Finally, numerical simulations are conducted to illustrate the effectiveness of the proposed method.

In this article, we propose a novel decentralized event-triggered control (ETC) scheme for a class of continuous-time nonlinear systems with matched interconnections. The present interconnected systems differ from most of the existing interconnected plants in that their equilibrium points are no longer assumed to be zero. Initially, we establish a theorem to indicate that the decentralized ETC law for the overall system can be represented by an array of optimal ETC laws for nominal subsystems. Then, to obtain these optimal ETC laws, we develop a reinforcement learning (RL)-based method to solve the Hamilton-Jacobi-Bellman equations arising in the discounted-cost optimal ETC problems of the nominal subsystems. Meanwhile, we only use critic networks to implement the RL-based approach and tune the critic network weight vectors by using the gradient descent method and the concurrent learning technique together. With the proposed weight vectors tuning rule, we are able to not only relax the persistence of the excitation condition but also ensure the critic network weight vectors to be uniformly ultimately bounded. Moreover, by utilizing the Lyapunov method, we prove that the obtained decentralized ETC law can force the entire system to be stable in the sense of uniform ultimate boundedness. Finally, we validate the proposed decentralized ETC strategy through simulations of the nonlinear-interconnected systems derived from two inverted pendulums connected via a spring.

This article considers an event-driven$H_{\infty }$control problem of continuous-time nonlinear systems with asymmetric input constraints. Initially, the$H_{\infty }$-constrained control problem is converted into a two-person zero-sum game with the discounted nonquadratic cost function. Then, we present the event-driven Hamilton–Jacobi–Isaacs equation (HJIE) associated with the two-person zero-sum game. Meanwhile, we develop a novel event-triggering condition making Zeno behavior excluded. The present event-triggering condition differs from the existing literature in that it can make the triggering threshold non-negative without the requirement of properly selecting the prescribed level of disturbance attenuation. After that, under the framework of adaptive critic learning, we use a single critic network to solve the event-driven HJIE and tune its weight parameters by using historical and instantaneous state data simultaneously. Based on the Lyapunov approach, we demonstrate that the uniform ultimate boundedness of all the signals in the closed-loop system is guaranteed. Finally, simulations of a nonlinear plant are presented to validate the developed event-driven$H_{\infty }$control strategy.

In this paper, we study the constrained optimization problem of a class of uncertain nonlinear interconnected systems. First, we prove that the solution of the constrained optimization problem can be obtained through solving an array of optimal control problems of constrained auxiliary subsystems. Then, under the framework of approximate dynamic programming, we present a simultaneous policy iteration (SPI) algorithm to solve the Hamilton-Jacobi-Bellman equations corresponding to the constrained auxiliary subsystems. By building an equivalence relationship, we demonstrate the convergence of the SPI algorithm. Meanwhile, we implement the SPI algorithm via an actor-critic structure, where actor networks are used to approximate optimal control policies and critic networks are applied to estimate optimal value functions. By using the least squares method and the Monte Carlo integration technique together, we are able to determine the weight vectors of actor and critic networks. Finally, we validate the developed control method through the simulation of a nonlinear interconnected plant.

Gas-liquid two-phase flow is of great importance in various industrial processes. How to accurately measure the flow parameters in the gas-liquid two-phase flow remains a challenging problem. In this article, we develop a novel deep learning based soft measure technique to predict the gas void fraction, which is one key parameter in a gas-liquid two-phase flow. We conduct the vertical upward gas-liquid two-phase flow experiments to measure the flow signals by using the four-sector distributed conductance sensor. Then, we design a novel multitask-based temporal-channelwise convolutional neural network (MTCCNN) to predict the gas void fraction. In MTCCNN, we first utilize the decomposed convolutional block to extract temporal dependence and channel connection from fluid data. After further fusion by the dense layer, we apply multitask learning to make full use of the extracted features through both classification branch and gas void fraction prediction branch. We compare our MTCCNN with its variations to demonstrate the proposed improvements. We also present other competitive methods for comparisons, which shows that our MTCCNN presents a better performance in gas void fraction prediction.

This article presents an optimal event-driven control scheme for a continuous stirred tank reactor (CSTR) system. The CSTR system differs from most of studied plants in that its equilibrium point is nonzero. In order to obtain the optimal event-driven control without coordinate transformations, we first introduce a discounted cost for such a system. Then, under the framework of adaptive critic designs, we use a single critic network to solve the Hamilton-Jacobi-Bellman equation related to the discounted-cost optimal event-driven control problem. To update the critic network weights, we employ the gradient descent method together with the experience replay technique. An advantage of the experience replay technique is that it brings about an easy-checked persistency of excitation-like condition. The stability analysis of all the signals in the closed-loop system is conducted via the Lyapunov approach. Finally, we validate the present optimal event-driven control strategy through simulations of the CSTR system.

This article studies an optimal event-triggered control (ETC) problem of nonlinear continuous-time systems subject to asymmetric control constraints. The present nonlinear plant differs from many studied systems in that its equilibrium point is nonzero. First, we introduce a discounted cost for such a system in order to obtain the optimal ETC without making coordinate transformations. Then, we present an event-triggered Hamilton-Jacobi-Bellman equation (ET-HJBE) arising in the discounted-cost constrained optimal ETC problem. After that, we propose an event-triggering condition guaranteeing a positive lower bound for the minimal intersample time. To solve the ET-HJBE, we construct a critic network under the framework of adaptive critic learning. The critic network weight vector is tuned through a modified gradient descent method, which simultaneously uses historical and instantaneous state data. By employing the Lyapunov method, we prove that the uniform ultimate boundedness of all signals in the closed-loop system is guaranteed. Finally, we provide simulations of a pendulum system and an oscillator system to validate the obtained optimal ETC strategy.

Electroencephalogram (EEG), obtained by wearable devices, can realize effective human health monitoring. Traditional methods based on artificially designed features have achieved valid results in EEG-based recognition, and numerous studies start to apply deep learning techniques in this area. In this article, we propose a coincidence-filtering-based method to build a connection between artificial-features-based methods and convolutional neural networks (CNNs), and design CNNs through simulating the information extraction pattern of artificial-features-based methods. Based on this method, we propose a novel, simple, and effective CNNs structure for EEG-based classification. We implement two experiments to obtain EEG data, and perform experiments based on the two health monitoring tasks. The results illustrate that the proposed network can achieve a prominent average accuracy on the emotion recognition and fatigue driving detection task. Due to its generality, the proposed framework design of CNNs is expected to be useful for broader applications in health monitoring areas.

This paper presents a novel decentralized control strategy for a class of uncertain nonlinear large-scale systems with mismatched interconnections. First, it is shown that the decentralized controller for the overall system can be represented by an array of optimal control policies of auxiliary subsystems. Then, within the framework of adaptive dynamic programming, a simultaneous policy iteration (SPI) algorithm is developed to solve the Hamilton-Jacobi-Bellman equations associated with auxiliary subsystem optimal control policies. The convergence of the SPI algorithm is guaranteed by an equivalence relationship. To implement the present SPI algorithm, actor and critic neural networks are applied to approximate the optimal control policies and the optimal value functions, respectively. Meanwhile, both the least squares method and the Monte Carlo integration technique are employed to derive the unknown weight parameters. Furthermore, by using Lyapunov's direct method, the overall system with the obtained decentralized controller is proved to be asymptotically stable. Finally, the effectiveness of the proposed decentralized control scheme is illustrated via simulations for nonlinear plants and unstable power systems.

In this paper, we study the event-triggered robust stabilization problem of nonlinear systems subject to mismatched perturbations and input constraints. First, with the introduction of an infinite-horizon cost function for the auxiliary system, we transform the robust stabilization problem into a constrained optimal control problem. Then, we prove that the solution of the event-triggered Hamilton-Jacobi-Bellman (ETHJB) equation, which arises in the constrained optimal control problem, guarantees original system states to be uniformly ultimately bounded (UUB). To solve the ETHJB equation, we present a single network adaptive critic design (SN-ACD). The critic network used in the SN-ACD is tuned through the gradient descent method. By using Lyapunov method, we demonstrate that all the signals in the closed-loop auxiliary system are UUB. Finally, we provide two examples, including the pendulum system, to validate the proposed event-triggered control strategy.

In this paper, we develop a decentralized event-triggered control (ETC) strategy for a class of nonlinear systems with uncertain interconnections. To begin with, we show that the decentralized ETC policy for the whole system can be represented by a group of optimal ETC laws of auxiliary subsystems. Then, under the framework of adaptive critic learning, we construct the critic networks to solve the event-triggered Hamilton-Jacobi-Bellman equations related to these optimal ETC laws. The weight vectors used in the critic networks are updated by using the gradient descent approach and the experience replay (ER) technique together. With the aid of the ER technique, we can conquer the difficulty arising in the persistence of excitation condition. Meanwhile, by using classic Lyapunov approaches, we prove that the estimated weight vectors used in the critic networks are uniformly ultimately bounded. Moreover, we demonstrate that the obtained decentralized ETC can force the overall system to be asymptotically stable. Finally, we present an interconnected nonlinear plant to validate the proposed decentralized ETC scheme.

This paper develops a novel event-triggered robust control strategy for continuous-time nonlinear systems with unknown dynamics. To begin with, the event-triggered robust nonlinear control problem is transformed into an event-triggered nonlinear optimal control problem by introducing an infinite-horizon integral cost for the nominal system. Then, a recurrent neural network (RNN) and adaptive critic designs (ACDs) are employed to solve the derived event-triggered nonlinear optimal control problem. The RNN is applied to reconstruct the system dynamics based on collected system data. After acquiring the knowledge of system dynamics, a unique critic network is proposed to obtain the approximate solution of the event-triggered Hamilton-Jacobi-Bellman equation within the framework of ACDs. The critic network is updated by using simultaneously historical and instantaneous state data. An advantage of the present critic network update law is that it can relax the persistence of excitation condition. Meanwhile, under a newly developed event-triggering condition, the proposed critic network tuning rule not only guarantees the critic network weights to converge to optimums but also ensures nominal system states to be uniformly ultimately bounded. Moreover, by using Lyapunov method, it is proved that the derived optimal event-triggered control (ETC) guarantees uniform ultimate boundedness of all the signals in the original system. Finally, a nonlinear oscillator and an unstable power system are provided to validate the developed robust ETC scheme.

This paper develops an optimal control scheme for continuous-time unknown nonlinear systems using the event-triggering mechanism. Different from designing controllers using the time-triggering mechanism, the event-triggered controller is updated only when the system state deviates more than a certain threshold from a prescribed value. To obtain the event-triggered optimal controller, we develop an identifier-critic architecture under the framework of reinforcement learning. The identifier network, composed of a feedforward neural network (FNN), aims to derive the knowledge of unknown system dynamics, and the critic network, constituted of an FNN, intends to derive the event-triggered optimal controller. The identifier network is tuned via the combination of a standard back-propagation algorithm and an e-modification method, and the critic network is updated using a modification of the gradient descent method. By introducing an additional stability term to update the critic network, the initial admissible control is no longer required. Meanwhile, by using historical and instantaneous state data together, the persistence of excitation condition is relaxed. A stability analysis of the closed-loop system is provided based on the Lyapunov method. The effectiveness of the proposed designs is illustrated through simulations of a nonlinear example and a single link robot arm system.

Sum of squares (SOS) polynomials have provided a computationally tractable way to deal with inequality constraints appearing in many control problems. It can also act as an approximator in the framework of adaptive dynamic programming. In this paper, an approximate solution to the optimal control of polynomial nonlinear systems is proposed. Under a given attenuation coefficient, the Hamilton-Jacobi-Isaacs equation is relaxed to an optimization problem with a set of inequalities. After applying the policy iteration technique and constraining inequalities to SOS, the optimization problem is divided into a sequence of feasible semidefinite programming problems. With the converged solution, the attenuation coefficient is further minimized to a lower value. After iterations, approximate solutions to the smallest -gain and the associated optimal controller are obtained. Four examples are employed to verify the effectiveness of the proposed algorithm.

This paper focuses on establishing an event-based constrained robust control strategy for a class of continuous-time affine nonlinear systems by incorporating the adaptive critic mechanism (ACM). The main objective is to integrate the event-based framework, the constrained optimal control method, and the neural network learning ability, thereby achieving the nonlinear robust state feedback of input-constrained nonlinear systems under event-based environment. Through theoretical analysis, it is shown that the nonlinear robust control law subject to input limitations can be obtained by designing an event-based constrained optimal controller with respect to the nominal system. Then, the ACM is adopted to facilitate the constrained optimal control implementation, where a critic neural network is constructed to serve as the learning approximator. The system stability issue is proved by employing the Lyapunov theory and the constrained robust control performance is illustrated through simulation experiments of several dynamical plants.

The H ∞ control problem is considered for linear parabolic partial differential equation (PDE) systems with completely unknown system dynamics. We propose a model-free policy iteration (PI) method for learning the H ∞ control policy by using measured system data without system model information. First, a finite-dimensional system of ordinary differential equation (ODE) is derived, which accurately describes the dominant dynamics of the parabolic PDE system. Based on the finite-dimensional ODE model, the H ∞ control problem is reformulated, which is theoretically equivalent to solving an algebraic Riccati equation (ARE). To solve the ARE without system model information, we propose a least-square based model-free PI approach by using real system data. Finally, the simulation results demonstrate the effectiveness of the developed model-free PI method.

The design of stabilizing controller for uncertain nonlinear systems with control constraints is a challenging problem. The constrained-input coupled with the inability to identify accurately the uncertainties motivates the design of stabilizing controller based on reinforcement-learning (RL) methods. In this paper, a novel RL-based robust adaptive control algorithm is developed for a class of continuous-time uncertain nonlinear systems subject to input constraints. The robust control problem is converted to the constrained optimal control problem with appropriately selecting value functions for the nominal system. Distinct from typical action-critic dual networks employed in RL, only one critic neural network (NN) is constructed to derive the approximate optimal control. Meanwhile, unlike initial stabilizing control often indispensable in RL, there is no special requirement imposed on the initial control. By utilizing Lyapunov's direct method, the closed-loop optimal control system and the estimated weights of the critic NN are proved to be uniformly ultimately bounded. In addition, the derived approximate optimal control is verified to guarantee the uncertain nonlinear system to be stable in the sense of uniform ultimate boundedness. Two simulation examples are provided to illustrate the effectiveness and applicability of the present approach.

The data-driven H∞ control problem of nonlinear distributed parameter systems is considered in this paper. An off-policy learning method is developed to learn the H∞ control policy from real system data rather than the mathematical model. First, Karhunen-Loève decomposition is used to compute the empirical eigenfunctions, which are then employed to derive a reduced-order model (ROM) of slow subsystem based on the singular perturbation theory. The H∞ control problem is reformulated based on the ROM, which can be transformed to solve the Hamilton-Jacobi-Isaacs (HJI) equation, theoretically. To learn the solution of the HJI equation from real system data, a data-driven off-policy learning approach is proposed based on the simultaneous policy update algorithm and its convergence is proved. For implementation purpose, a neural network (NN)- based action-critic structure is developed, where a critic NN and two action NNs are employed to approximate the value function, control, and disturbance policies, respectively. Subsequently, a least-square NN weight-tuning rule is derived with the method of weighted residuals. Finally, the developed data-driven off-policy learning approach is applied to a nonlinear diffusion-reaction process, and the obtained results demonstrate its effectiveness.

In this paper, a novel iterative adaptive dynamic programming (ADP)-based infinite horizon self-learning optimal control algorithm, called generalized policy iteration algorithm, is developed for nonaffine discrete-time (DT) nonlinear systems. Generalized policy iteration algorithm is a general idea of interacting policy and value iteration algorithms of ADP. The developed generalized policy iteration algorithm permits an arbitrary positive semidefinite function to initialize the algorithm, where two iteration indices are used for policy improvement and policy evaluation, respectively. It is the first time that the convergence, admissibility, and optimality properties of the generalized policy iteration algorithm for DT nonlinear systems are analyzed. Neural networks are used to implement the developed algorithm. Finally, numerical examples are presented to illustrate the performance of the developed algorithm.

This paper presents a novel observer-critic architecture for solving the near-optimal control problem of uncertain nonlinear continuous-time systems. Two neural networks (NNs) are employed in the architecture: an observer NN is constructed to get the knowledge of uncertain system dynamics and a critic NN is utilized to derive the optimal control. The observer NN and the critic NN are tuned simultaneously. By using the recorded and instantaneous data together, the optimal control can be derived without the persistence of excitation condition. Meanwhile, the closed-loop system is guaranteed to be stable in the sense of uniform ultimate boundedness. No initial stabilizing control is required in the developed algorithm. An illustrated example is provided to demonstrate the effectiveness of the present approach.

In this paper, the infinite horizon optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems is investigated using neural-network-based online solution of Hamilton-Jacobi-Bellman (HJB) equation. By establishing an appropriate bounded function and defining a modified cost function, the optimal robust guaranteed cost control problem is transformed into an optimal control problem. It can be observed that the optimal cost function of the nominal system is nothing but the optimal guaranteed cost of the original uncertain system. A critic neural network is constructed to facilitate the solution of the modified HJB equation corresponding to the nominal system. More importantly, an additional stabilizing term is introduced for helping to verify the stability, which reinforces the updating process of the weight vector and reduces the requirement of an initial stabilizing control. The uniform ultimate boundedness of the closed-loop system is analyzed by using the Lyapunov approach as well. Two simulation examples are provided to verify the effectiveness of the present control approach.

In this paper, a new iterative adaptive dynamic programming (ADP) algorithm is developed to solve optimal control problems for infinite horizon discrete-time nonlinear systems with finite approximation errors. First, a new generalized value iteration algorithm of ADP is developed to make the iterative performance index function converge to the solution of the Hamilton-Jacobi-Bellman equation. The generalized value iteration algorithm permits an arbitrary positive semi-definite function to initialize it, which overcomes the disadvantage of traditional value iteration algorithms. When the iterative control law and iterative performance index function in each iteration cannot accurately be obtained, for the first time a new "design method of the convergence criteria" for the finite-approximation-error-based generalized value iteration algorithm is established. A suitable approximation error can be designed adaptively to make the iterative performance index function converge to a finite neighborhood of the optimal performance index function. Neural networks are used to implement the iterative ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the developed method.