This paper investigates the event-triggered prescribed performance intermittent (ETPPI) control for constrained nonlinear systems via adaptive dynamic programming (ADP). By integrating the ADP algorithm and prescribed performance control, an optimal control policy is designed to ensure both system states and control inputs are constrained in prescribed boundaries while guaranteeing that system states converge within a prescribed time. Subsequently, an event-triggered intermittent control mechanism is established to reduce the consumption of computing and communication resources. Additionally, a critic neural network is built to approximate the solution of the Hamilton-Jacobi-Bellman equation, which enables the derivation of an approximate optimal control policy. Theoretical analysis demonstrates that under the developed ETPPI control approach, the closed-loop system achieves asymptotically stable and the Zeno behavior is precluded. Finally, the effectiveness of the developed approach is validated through two simulation cases.

This article addresses the predefined-time optimal motion control problem of an autonomous surface vehicle (ASV) with disturbances under dynamic self-triggered frameworks via reinforcement learning (RL). Initially, to eliminate the influence of disturbance on the ASV, a predefined-time second-order integral sliding mode control (SOISM) strategy is formulated by establishing a novel integral sliding mode (ISM) function and a terminal sliding mode function. Subsequently, a predefined-time approximate optimal motion (AOM) control strategy is further developed to ensure the ASV maintains a stable state. Furthermore, a single critic network is used to obtain an approximate solution of the Hamilton-Jacobi-Bellman (HJB) equation. The above two strategies are established under the dynamic self-triggered framework, which relies on the current information to predict the next updating time, effectively reducing the computational and communication burden while avoiding the continuous monitoring of the ASV state. In the theoretical analysis, the main challenges lie in the design of Lyapunov functions and triggered conditions to ensure the stability of the sliding mode dynamics and the disturbed ASV. By applying the Lyapunov stability principle and designing two novel Lyapunov functions and triggered conditions that both contain dynamic variables, we demonstrate that the developed control strategies can ensure the stability within the specified time frame. Ultimately, simulation results verify the efficacy of the proposed motion control approach.

This paper investigates the path tracking control of unmanned agricultural vehicles with disturbances under a dynamic self-triggered mechanism via reinforcement learning (RL). To begin with, a path tracking offset system is constructed based on the kinematic model of the unmanned agricultural vehicle, which transforms the path tracking control problem into an optimal control problem. Subsequently, a novel dynamic self-triggered second-order integral sliding mode control policy is developed to mitigate the impact of disturbances and to derive a nominal path tracking offset model. Afterward, to further alleviate the computing and communication burdens, a novel dynamic self-triggered mechanism is proposed for the optimal control policy. It can predict the next update time based on current information, thus avoiding the need for continuous monitoring of the triggering condition. Furthermore, a single critic network architecture is constructed to obtain an approximate path tracking control policy, and it is proven by Lyapunov stability theory that this policy ensures unmanned agricultural vehicles can maintain the predefined working path in the presence of disturbances. Finally, the effectiveness of the proposed path tracking control method is demonstrated by simulation experiments.

In this paper, the model-free dynamic event-driven safe (MFDEDS) control of unknown nonaffine systems with state and input constraints is investigated via adaptive dynamic programming. To begin with, by introducing a dynamic compensator and performing system transformation, the safe control problem with state and input constraints is transformed into an optimal regulation problem of an unconstrained system. Afterwards, an integral reinforcement learning algorithm is applied to the unconstrained system to derive an optimal safe control policy independent of the original system model, which achieves model-free approximate optimal control for the original system. To conserve computing and communication resources, a novel game-based dynamic event-driven mechanism is established, which models the control policy and the event-driven error as players in a zero-sum game, with the aim of obtaining the worst event-driven error to maximize the triggering interval. Furthermore, an approximate solution to the Hamilton-Jacobi-Bellman equation is derived by constructing a single-critic learning structure, which results in an approximate optimal safe control policy. Theoretical analysis demonstrates that the proposed MFDEDS control scheme ensures the closed-loop system is asymptotically stable. Ultimately, the efficacy of the developed approach is corroborated through two simulation examples.

In this paper, the event-triggered robust hierarchical synchronization (ETRHS) control of unmanned surface vehicles (USVs) is investigated via reinforcement learning. In the ETRHS control problem, there exists one dominant USV and many following USVs. The dominant USV chooses a motion control policy based on the responses of all following USVs, and then each following USV takes corresponding optimal responses to the dominant USV’s policy. This paper converts the ETRHS control problem to an event-triggered optimal synchronization control problem by designing novel value functions for the dominant and following USVs. Subsequently, critic-only structures are established and the ETRHS control laws of all USVs are obtained to form the Stackelberg equilibrium. In order to reduce the computing and communication burden, a novel event-triggering condition is designed for each USV, and the corresponding control law is updated when the condition is triggered. Theoretical analysis demonstrates that the developed reinforcement learning-based ETRHS controllers guarantee all following USVs synchronize with the dominant USV even when dynamic uncertainties exist. Finally, simulation results verify the effectiveness of the developed reinforcement learning-based ETRHS control scheme.

In this article, the distributed optimal containment (DOC) control of wheeled mobile robots (WMRs) is investigated via adaptive dynamic programming. To begin with, a novel performance index function which contains containment errors and their derivatives is designed for each following WMR without requiring the discount factor, which simplifies the controller design process and enhances the practicality of the control method. Subsequently, the DOC control of WMRs is formulated as a differential graphical game whose Nash equilibrium can be formed by using the optimal responses of all following WMRs. Moreover, a critic-only structure is built to obtain an approximate DOC control law, which provides a solution for the coupled Hamilton–Jacobi–Bellman equation of each following WMR. Stability analysis demonstrates that the containment error of each following WMR is uniformly ultimately bounded. Finally, a group of WMRs are utilized to verify the effectiveness of the present DOC control scheme.

This article develops a novel online adaptive dynamic programming algorithm with adjustable convergence rate to address the optimal control problem of nonlinear systems. Relaxation factors are introduced to tune the convergence rate of value function sequence online. A novel update law based on recursive least squares is developed to adjust the weight of critic neural network at the sampling instant. The uniform ultimate boundedness of the neural network estimation error and the closed-loop system state are analyzed by utilizing the Lyapunov technique. Finally, the effectiveness of the present algorithm is demonstrated by executing three simulation examples.

In this article, the reinforcement learning-based distributed robust bipartite consensus control of multispacecraft systems with dynamic uncertainties is investigated. The developed control structure includes two parts, i.e., integral sliding mode control and distributed optimal bipartite consensus control. In the first step, an integral sliding mode controller is designed for each following spacecraft to address matched uncertainties such that the dynamics of nominal spacecraft is obtained. In the second step, a novel performance index function, which contains consensus errors and their derivatives, is designed for each nominal spacecraft. As a result, the system assumption of zero equilibrium and the discount factor in performance index function are not required, which simplifies the controller design process and improves the practicability of the developed control method. Moreover, in order to solve the coupled Hamilton–Jacobi–Bellman equation of each following spacecraft, a novel policy iteration algorithm is designed and its properties are analyzed. Finally, a group of spacecraft is employed to verify the effectiveness of the present control scheme.

This article develops a distributed fault-tolerant consensus control (DFTCC) approach for multiagent systems by using adaptive dynamic programming. By establishing a local fault observer, the potential actuator faults of each agent are estimated. Subsequently, the DFTCC problem is transformed into an optimal consensus control problem by designing a novel local value function for each agent which contains the estimated fault, the consensus errors, and the control laws of the local agent and its neighbors. In order to solve the coupled Hamilton-Jacobi-Bellman equation of each agent, a critic-only structure is established to obtain the approximate local optimal consensus control law of each agent. Moreover, by using Lyapunov's direct method, it is proven that the approximate local optimal consensus control law guarantees the uniform ultimate boundedness of the consensus error of all agents, which means that all following agents with potential actuator faults synchronize to the leader. Finally, two simulation examples are provided to validate the effectiveness of the present DFTCC scheme.

In this article, an event-triggered decentralized integral sliding mode control (ETDISMC) method is investigated for a class of input-constrained nonlinear large-scale systems with actuator failures based on adaptive dynamic programming (ADP). An integral sliding mode control method is developed to maintain the subsystem trajectories on the sliding mode surface, eliminate the effect of actuator failures, and obtain the sliding mode dynamics (SMDs). Then, the control problem is transformed into an optimal control (OC) problem for the nominal form of the SMDs by constructing a modified local value function. To obtain the event-triggered OC law, a critic-only structure is applied to approximate the local optimal value function of the nominal subsystem for solving the event-triggered Hamilton–Jacobi–Bellman equation. An event-triggered ADP control method is developed to decrease the updating frequency of the OC law and to reduce the computational burden. In addition, an experience replay-based weight updating policy is presented to relax the persistence of excitation condition. Furthermore, we prove that the developed method can guarantee the closed-loop system to be asymptotically stable by using Lyapunov’s direct method. Finally, a numerical example and a practical system are employed for simulation to demonstrate the effectiveness of the proposed ETDISMC scheme.

In this article, the event-triggered robust control of unknown multiplayer nonlinear systems with constrained inputs and uncertainties is investigated by using adaptive dynamic programming. To relax the requirement of system dynamics, a neural network-based identifier is constructed by using the system input-output data. Subsequently, by designing a nonquadratic value function, which contains the bounded functions, the system states, and the control inputs of all players, the event-triggered robust stabilization problem is converted into an event-triggered constrained optimal control problem. To obtain the approximate solution of the event-triggered Hamilton-Jacobi (HJ) equation, a critic network for each player is established with a novel weight updating law to relax the persistence of excitation condition based on the experience replay technique. Furthermore, according to the Lyapunov stability theorem, the present event-triggered robust optimal control ensures the multiplayer system to be uniformly ultimately bounded. Finally, two simulation examples are employed to show the effectiveness of the present method.

This article develops a cooperative motion/force control (CMFC) scheme based on adaptive dynamic programming (ADP) for modular reconfigurable manipulators (MRMs) with the joint task assignment approach. By separating terms depending on local variables only, the dynamic model of the entire MRM system can be regarded as a set of joint modules interconnected by coupling torque. In addition, the Jacobian matrix, which reflects the interaction force of the MRM end-effector, can be mapped into each joint. Using this approach, both the motion and force tasks on the end-effector of the entire MRM system can be assigned to each joint module cooperatively. Then, by substituting the actual states of coupled joint modules with their desired ones, the norm-boundedness assumption on the interconnection of joint module can be relaxed. By using the measured input-output data of each joint module, a neural network (NN)-based robust decentralized observer, which guarantees the observation error to be asymptotically stable is established. An improved local value function is constructed for each joint module to reflect the interconnection. Then, the local Hamilton-Jacobi-Bellman equation is solved by constructing a local critic NN with a nested learning structure. Hereafter, the ADP-based CMFC is obtained by the assistance of force feedback compensation. Based on the Lyapunov stability analysis, the closed-loop MRM system is guaranteed to be uniformly ultimately bounded under the present ADP-based CMFC scheme. The simulation on a two-degree of freedom MRM system demonstrates the effectiveness of the present control approach.

In order to address zero-sum game problems for discrete-time (DT) nonlinear systems, this article develops a novel event-triggered control (ETC) approach based on the deterministic policy gradient (PG) adaptive dynamic programming (ADP) algorithm. By adopting the input and output data, the proposed ETC method updates the control law and the disturbance law with a gradient descent algorithm. Compared with the conventional PG ADP-based control scheme, the present controller is updated aperiodically to reduce the computational and communication burden. Then, the actor-critic-disturbance framework is adopted to obtain the optimal control law and the worst disturbance law, which guarantee the input-to-state stability of the closed-loop system. Moreover, a novel neural network weight updating law which guarantees the uniform ultimate boundedness of weight estimation errors is provided based on the experience replay technique. Finally, the validity of the present method is verified by simulation of two DT nonlinear systems.

To solve the problems of poor performance of traditional spectrum sensing method under low signal‐to‐noise ratio, a new spectrum sensing method based on Empirical Mode Decomposition algorithm and K‐means clustering algorithm is proposed. Firstly, the Empirical Mode Decomposition algorithm and the wavelet threshold algorithm are used to remove the noise components in the spectrum sensing signal, and K‐means clustering algorithm is used to determine whether the primary user exists. The method can remove the redundant components such as noise in the nonstationary or nonlinear sampling signal in the real environment and does not need to know the prior information such as signal, channel, and noise, so it can well handle the complicated sensing signal in real environment. This method can reduce the impact of noise on the spectrum sensing system and thus can improve the sensing performance of the system. In the experimental part, the difference between maximum and minimum eigenvalues and the difference between the maximum eigenvalue and the average energy in the random matrix are selected as signal features. Experiments also show that the proposed method is better than the traditional spectrum sensing methods.