The presence of nonlinearities and actuator faults poses significant challenges to achieving optimal control of quadrotors. This article investigates an adaptive dynamic programming algorithm to solve the optimal attitude regulation problem of quadrotors with nonlinear hysteresis. The optimal regulation problem is modeled as a graphical game, and optimality is defined by the Nash equilibrium. A nonlinear hysteresis inverse model is first designed to compensate for the nonlinearity of the hysteresis. To eliminate matched disturbances caused by compensation errors, an integral sliding mode mechanism is proposed, improving fault tolerance. Through problem transformation, the Nash equilibrium is achieved by solving interconnected Hamilton–Jacobi–Bellman equations. A data-based policy iteration algorithm is then developed to numerically solve the approximate optimal solution using a least-squares approach. Finally, a quadrotor team is employed to verify the effectiveness of the proposed algorithm.

Causal discovery aims to learn causal relationships between variables from targeted data, making it a fundamental task in machine learning. However, causal discovery algorithms often rely on unverifiable causal assumptions, which are usually difficult to satisfy in real-world data, thereby limiting the broad application of causal discovery in practical scenarios. Inspired by these considerations, this work extensively benchmarks the empirical performance of various mainstream causal discovery algorithms, which assume i.i.d. data, under eight model assumption violations. Our experimental results show that differentiable causal discovery methods exhibit robustness under the metrics of Structural Hamming Distance and Structural Intervention Distance of the inferred graphs in commonly used challenging scenarios, except for scale variation. We also provide the theoretical explanations for the performance of differentiable causal discovery methods. Finally, our work aims to comprehensively benchmark the performance of recent differentiable causal discovery methods under model assumption violations, and provide the standard for reasonable evaluation of causal discovery, as well as to further promote its application in real-world scenarios.

This article explores the application of the multiagent reinforcement learning (MARL) algorithm in addressing the large-scale traffic signal control (TSC) problem. To address the TSC problem in complex urban traffic networks, most existing algorithms focus on optimizing local traffic flow at each intersection through decentralized training based on either the local observations or messages from its neighboring intersections, which, however, lacks the concept of cooperative learning. To conquer such limitations, a novel distributed critic with decentralized actor (DCDA) framework is proposed, which allows the communication messages and temporal difference (TD) losses to be exchanged among neighboring intersections. Specially, by considering the traffic network as a communication network of agents (more precisely, intersections and lanes are considered as agents and edges, respectively), a distributed global average TD loss estimation algorithm is designed in the distributed critic step to estimate the global average TD loss estimation and enhance collaboration among agents. Moreover, in the decentralized actor step, the policy gradient decomposition method is adopted for each agents to learn its local policy solely based on its local action-value function. By adhering to the DCDA framework, a novel distributed policy gradient decomposition (DPGD) algorithm is further proposed to address the TSC problem. Empirical experiments demonstrate that the efficiency, robustness, and stability of the DPGD algorithm outperform the state-of-the-art MARL algorithms in both the environments of cooperative adaptive cruise control and adaptive traffic signal control.

In this article, the fully distributed neural learning algorithms by neural network approximation for networked multiagent reinforcement learning (NMARL) are studied. To tackle the convergence analysis of methods in NMARL with tremendous state-action space, most of the existing distributed algorithms are designed by linear function approximation, which however would fall into a situation of poor expression. To conquer such limitation, the distributed neural learning algorithms are developed by using a novel neural network approximation that bridges the theory and practice of deep NMARL (DMARL). Specifically, inspired by the overparametrization method for minimizing mean-squared projected bellman error (MSPBE), the distributed neural learning algorithms with population semigradients and stochastic semigradients are respectively, proposed to solve the NMARL problem. Furthermore, the convergence of the proposed algorithms are strictly given by employing the overparametrization method to establish the approximate stationary point of MSPBE to characterize the algorithms toward the global optimum. Finally, some numerical simulations demonstrate the effectiveness of the distributed neural learning algorithms.

Actor-critic (AC) cooperative multiagent reinforcement learning (MARL) over directed graphs is studied in this article. The goal of the agents in MARL is to maximize the globally averaged return in a distributed way, i.e., each agent can only exchange information with its neighboring agents. AC methods proposed in the literature require the communication graphs to be undirected and the weight matrices to be doubly stochastic (more precisely, the weight matrices are row stochastic and their expectation are column stochastic). Differently from these methods, we propose a distributed AC algorithm for MARL over directed graph with fixed topology that only requires the weight matrix to be row stochastic. Then, we also study the MARL over directed graphs (possibly not connected) with changing topologies, proposing a different distributed AC algorithm based on the push-sum protocol that only requires the weight matrices to be column stochastic. Convergence of the proposed algorithms is proven for linear function approximation of the action value function. Simulations are presented to demonstrate the effectiveness of the proposed algorithms.

In this article, the asymptotic tracking consensus problem of higher-order multiagent systems (MASs) with general directed communication graphs is addressed via designing event-triggered control strategies. One common assumption utilized in most existing results on such tracking consensus problem that the inherent dynamics of the leader are the same as those of the followers is removed in this article. In particular, two cases that the dynamics of the leader are subjected, respectively, to bounded input and unknown nonlinearity are considered. To do this, distributed event-triggered observers are first constructed to estimate the state information of the leader. Then, local event-triggered tracking control protocols are designed for each follower to complete the goal of tracking consensus. One distinguishing feature of the present distributed observers lies in the fact that they could avoid the continuous monitoring for the states of the neighbors' observer states. It is also worth pointing out that the present tracking consensus control strategies are fully distributed as no global information related to the directed communication graph is involved in designing the strategies. Two simulation examples are finally presented to verify the efficiency of the theoretical results.

In this paper, the finite-time output consensus problem is considered for a class of second-order multiagent systems (MASs), where the mismatched disturbance exists in the dynamics of each agent, and the communication topology is directed. First of all, a basic backstepping control protocol is proposed to solve the finite-time consensus problem without mismatched disturbance. Then, a finite-time disturbance observer is designed to estimate the mismatched disturbance, based on which, two adaptive finite-time consensus protocols are proposed to solve the finite-time output consensus and tracking consensus problems without using any global information with respect to the communication topology. Finally, two simulation examples are illustrated to verify the theoretical results.

In this paper, finite-time consensus problem is considered for a class of second-order multi-agent systems with a directed communication topology. A fully-distributed control protocol is proposed for solving the finite-time consensus problem, without using any global information. A simulation example is presented to verify the theoretical result and to show the effectiveness of the protocol.

In this paper, the new decoupled distributed sliding-mode control (DSMC) is first proposed for second-order consensus in multiagent systems, which finally solves the fundamental unknown problem for sliding-mode control (SMC) design of coupled networked systems. A distributed full-order sliding-mode surface is designed based on the homogeneity with dilation for reaching second-order consensus in multiagent systems, under which the sliding-mode states are decoupled. Then, the SMC is applied to the decoupled sliding-mode states to reach their origin in finite time, which is the sliding-mode surface. The states of agents can first reach the designed sliding-mode surface in finite time and then move to the second-order consensus state along the surface in finite time as well. The DSMC designed in this paper can eliminate the influence of singularity problems and weaken the influence of chattering, which is still very difficult in the SMC systems. In addition, DSMC proposes a general decoupling framework for designing SMC in networked multiagent systems. Simulations are presented to verify the theoretical results in this paper.