This paper presents a prescribed-time optimal resilient control strategy for multiquadrotor systems under composite attacks, including sensor attacks and denial of service (DoS) attacks. To counter the sensor attacks, a distributed resilient extended state observer, integrated with a sensor attack compensator, is proposed for each follower to estimate its own states and the unknown lumped disturbances. Furthermore, a prescribed-time fully distributed resilient observer is constructed to provide each follower with the estimation of the reference signals even under DoS attacks. Based on these estimations, a prescribed-time optimal controller is designed by combining a novel coordinate transformation and the experience replay technique. The stability analysis is presented to verify that the attitude tracking error can converge to a small neighborhood of zero within a prescribed time. Finally, a simulation example is given to validate the effectiveness of the proposed framework.

This paper proposes inverse reinforcement learning (IRL) algorithms to solve the optimal synchronization problems for multi-agent systems (MASs) subject to disturbances. A framework of expert-learner MASs is developed to reconstruct the unknown cost functions of expert MAS. Within this framework, the learner MAS imitates the behavior trajectories of expert MAS under observation. We first present a model-based IRL algorithm including two iteration loops: an inner loop on optimal control and an outer loop on inverse optimal control. Moreover, a modelfree IRL algorithm is further proposed using the observed input and state data without requiring the dynamics knowledge of two MASs. Then generalized fuzzy hyperbolic models are utilized to implement the model-free algorithm, and we provide the convergence and stabilization proofs of our proposed algorithms. Finally, the effectiveness of the proposed algorithms is verified through a simulation example.

To address the security and privacy challenges in Internet of Things (IoT) enabled systems, this paper introduces a differential privacy-based distributed model-free adaptive control framework for secure consensus control of nonlinear IoT-based multi-agent systems (MASs). First, by embedding a differential privacy mechanism into inter-agent communications, the privacy protection accuracy is theoretically guaranteed while ensuring data confidentiality. Second, a distributed model-free adaptive control algorithm is developed, which only uses local input-output (I/O) data, thereby removing the need for explicit system models and global network topology information, offering high scalability and robustness for large-scale IoT networks with unknown dynamics. Furthermore, an integrated control framework is proposed to achieve consensus control for heterogeneous nonlinear MASs under simultaneous FDI attacks and privacy perturbations. It is theoretically established that the method maintains mean-square boundedness for estimation errors, consensus errors, and output trajectories despite the joint impact of adversarial attacks and privacy noise. Simulation results demonstrate that the approach effectively resists stealthy data tampering attacks while maintaining communication privacy and system stability.

This paper presents a distributed data-driven inverse reinforcement learning (IRL) framework for multiagent systems using state trajectories only. By exploiting the state observations, the approach achieves optimal consensus among agents while simultaneously inferring unknown cost functions of all agents. Concurrent learning-based adaptive laws and parameter estimators are designed to estimate feedback control gains and unknown system’s dynamics in real time, which mitigates the requirement for the persistency of excitation condition. Building on these learned models, we develop a distributed data-driven IRL approach and analyze the ultimate boundedness of both the weight estimate errors and local neighbor consensus errors. Finally, the proposed approach is validated through simulations on a DC islanded microgrid.

This article proposes an online data-based inverse reinforcement learning (IRL) scheme to solve optimal control problem for nonlinear systems with control constraints, in which the unknown reward functions are recovered based on the systems’ demonstrated state and input data. To deal with control constraint, we introduce a saturation function to formulate the original constrained optimal control problem into a new unconstrained optimal control problem. Then a data-based identifier using neural network (NN) approximation technique is designed to estimate the system’s dynamics. Subsequently, we develop a data-based IRL approach to learn the unknown reward function and establish the weight tuning law of the value function and reward function using demonstrated state and input data. The proof of the uniform boundedness of the weight estimation error is presented. A simulation example is provided to verify the effectiveness of the proposed approach.

This article addresses the optimal containment problem of heterogeneous multiagent systems (MASs) with dynamic leaders via reinforcement learning (RL), where the dynamics of all agents are all completely unknown. A distributed model-free observer is constructed for each follower to estimate the leaders’ dynamics and the output trajectories inside the convex hull formed by the leaders. Based on the designed observers, the optimal containment problem is formulated as an optimal tracking control issue. Then the discounted performance functions are introduced to obtain algebraic Riccati equations (AREs). And a model-free RL algorithm is developed to learn the AREs online. To implement this algorithm, we design a single critic neural network structure for each follower to approximate Q-function, and estimate optimal control policy and worst-case adversarial input policy. Finally, a numerical simulation is provided to demonstrate the effectiveness of the proposed algorithm.

In this paper, we investigate pursuit-evasion game problem for multiple-pursuer single-evader using reinforcement learning. A performance function including the states and limited inputs of all players is developed to evaluate the whole system's cost. Then we construct the Hamiltonian equation and design the optimal control polices for the pursuers and the evaders using the Bellman optimization principle. And we present a policy iteration (PI) algorithm to learn the value function and control policies of all players. In order to implement PI algorithm, we construct a neural network to approximate the optimal value function and optimal control strategy. Finally, a simulation example is provided to demonstrate the effectiveness of the algorithm.

This article considers integral sliding-mode control (ISMC) for multiagent systems with matched disturbances through adaptive dynamic programming (ADP) method. A distributed disturbance observer (DDO) is designed for each agent to generate the disturbance estimation. Using the disturbance estimation, an ISMC scheme is proposed to reject the input disturbance and obtain the equivalent local neighbor consensus sliding-mode dynamics. Then, a discount performance function is designed for each agent, and ADP technique is utilized to address the optimal consensus control for the equivalent sliding-mode dynamics online. Based on the gradient descent algorithm, we propose the adaptive weight tuning laws for the critic-actor neural networks (NNs) to carry out the ADP method. Furthermore, the local neighbor consensus errors and the weight estimation errors for the critic-actor NNs are proved to be uniformly ultimately bounded (UUB). Finally, the practical example is applied to validate the effectiveness of our results.

This paper investigates finite-horizon optimal consensus control problem for unknown multiagent systems with state delays. It is well known that optimal consensus control is the solutions to the coupled Hamilton-Jacobi-Bellman (HJB) equations. An off-policy reinforcement learning (RL) algorithm is developed to learn the two-stage optimal consensus solutions to the coupled time-varying HJB equations using the measurable state data instead of the knowledge of the state-delayed system dynamics. Subsequently, for each agent, a single critic neural network (NN) is utilized to approximate the time-varying cost function and help to calculate optimal consensus control policy. Based on the method of weighted residuals, adaptive weight update laws for the critic NNs are proposed. Finally, the simulation results are provided to illustrate the effectiveness of the proposed off-policy RL method.

This paper is concerned with data-driven distributed optimal consensus control for unknown multiagent systems (MASs) with input delays. The input-delayed MAS model is first converted into a delay-free form using a model reduction method. By establishing an equivalent relationship on the predesigned performance indices of the two MASs, optimal consensus control of input-delayed MAS can be fully transformed to that of delay-free MAS. Based on the coupled Hamilton-Jacobi equations and Bellman's optimality principle, optimal consensus control policies are derived for the transformed delay-free MAS. Then a policy iteration algorithm based on distributed asynchronous update mechanism is proposed to learn the coupled Hamilton-Jacobi-Bellman equations online. To perform the proposed data-driven adaptive dynamic programming algorithm, we adopt the measured data-based critic-actor neural networks to approximate the value functions and the control policies, respectively. Finally, a simulation example is given to illustrate the effectiveness of the proposed method.

This paper addresses the problem of distributed optimal consensus control for a continuous-time heterogeneous linear multiagent system subject to time varying input delays. First, by discretization and model transformation, the continuous-time input-delayed system is converted into a discrete-time delay-free system. Two delicate performance index functions are defined for these two systems. It is shown that the performance index functions are equivalent and the optimal consensus control problem of the input-delayed system can be cast into that of the delay-free system. Second, by virtue of the Hamilton-Jacobi-Bellman (HJB) equations, an optimal control policy for each agent is designed based on the delay-free system and a novel value iteration algorithm is proposed to learn the solutions to the HJB equations online. The proposed adaptive dynamic programming algorithm is implemented on the basis of a critic-action neural network (NN) structure. Third, it is proved that local consensus errors of the two systems and weight estimation errors of the critic-action NNs are uniformly ultimately bounded while the approximated control policies converge to their target values. Finally, two simulation examples are presented to illustrate the effectiveness of the developed method.