Direct current microgrids (DCmGs) are attractive due to their high efficiency and ease of deployment, yet decentralized controllers remain vulnerable to false-data-injection (FDI) attacks and abrupt load changes. This article develops a resilient decentralized control architecture that achieves performance guarantees under bounded FDI attacks using only local measurements. The key idea is a decentralized high-gain observer that reconstructs each power buffer input impedance and stored energy while explicitly accounting for network coupling. On this basis, a smooth nonlinear feedback term provides attack compensation without chattering. In contrast to existing schemes, our design comes with explicit linear matrix inequality conditions that, first, certify asymptotic stability in the attack-free case and, second, ensure uniform ultimate boundedness of the closed loop under FDI, thereby yielding transparent tuning ranges for the controller and observer gains. Hardware-in-the-loop experiments, in which the DCmG is subjected to staged FDI attack windows and load steps, are conducted to demonstrate the proposed design with robust and practically tunable resilience for DCmGs.

This article addresses the stabilization challenges of nonholonomic systems under the threat of false data injection (FDI) attacks, which compromise the integrity of state information. A novel adaptive control strategy using Nussbaum-type gains is proposed to ensure the asymptotic stability of the closed-loop system while maintaining signal boundedness. The approach extends conventional Nussbaum designs to handle multiple unknown control directions. It integrates online learning mechanisms to mitigate the impact of FDI attacks. Additionally, adaptive backstepping and fuzzy-logic systems are utilized to approximate and compensate for unknown nonlinear dynamics. The methodology transforms nonholonomic systems into equivalent cascade structures to address inherent constraints and enable secure control input design. Simulation studies validate the effectiveness and resilience of the proposed control strategy, demonstrating significant improvements in stability and robustness in the presence of FDI attacks.

This article proposes an event-based asymptotic tracking control method for nonlinear strict-feedback systems with multiple unknown control directions. The system is characterized by multiple unknown control directions, which pose challenges to its performance. In contrast to traditional Nussbaum-type methods, we propose a novel Nussbaum-type function to handle multiple Nussbaum-type gains, ensuring robust asymptotic tracking. Additionally, two event-triggered mechanisms are developed to alleviate the computational complexity of adaptive Nussbaum design. The static event-triggered mechanism significantly improves the system’s responsiveness to dynamic changes by employing dynamically decreasing thresholds. Building on this, a dynamic event-triggered mechanism is introduced, incorporating an internal variable that continuously adjusts the triggering conditions over time. Furthermore, the proposed design not only achieves asymptotic tracking control but also ensures that both event-triggered mechanisms avoid the Zeno phenomenon. To validate the proposed design schemes, a simulation example of a marine surface vehicle is presented.

In this article, we propose a way to enhance the learning framework for zero-sum games with dynamics evolving in continuous time. In contrast to the conventional centralized actor-critic learning, a novel cooperative finitely excited learning approach is developed to combine the online recorded data with instantaneous data for efficiency. By using an experience replay technique for each agent and distributed interaction amongst agents, we are able to replace the classical persistent excitation condition with an easy-to-check cooperative excitation condition. This approach also guarantees the consensus of the distributed actor-critic learning on the solution to the Hamilton-Jacobi-Isaacs (HJI) equation. It is shown that both the closed-loop stability of the equilibrium point and convergence to the Nash equilibrium can be guaranteed. Simulation results demonstrate the efficacy of this approach compared to previous methods.

In this article, we propose an adaptive fuzzy control design for the distributed competitive control problem of multiagent systems (MASs) with multiple unknown control directions. The bipartite synchronization control is investigated by using the fuzzy backstepping control framework and fuzzy logic systems. To broaden the application field for the distributed protocol design, we consider practical bipartite synchronization for a group of MASs consisting of followers subject to heterogeneous unknown control directions. To address these multiple unknown control directions, a novel Nussbaum-type function is developed. Moreover, to reduce the communication bandwidth, this article proposes two threshold strategies for event-triggered control to avoid any unnecessary sampling while taking flexibility into consideration, further improving the efficiency and feasibility of the developed bipartite protocol design. The experimental results indicate that the proposed control method can effectively realize bipartite synchronization of MASs with multiple unknown control directions.

This article presents a novel efficient experience-replay-based adaptive dynamic programming (ADP) for the optimal control problem of a class of nonlinear dynamical systems within the Hamiltonian-driven framework. The quasi-Hamiltonian is presented for the policy evaluation problem with an admissible policy. With the quasi-Hamiltonian, a novel composite critic learning mechanism is developed to combine the instantaneous data with the historical data. In addition, the pseudo-Hamiltonian is defined to deal with the performance optimization problem. Based on the pseudo-Hamiltonian, the conventional Hamilton-Jacobi-Bellman (HJB) equation can be represented in a filtered form, which can be implemented online. Theoretical analysis is investigated in terms of the convergence of the adaptive critic design and the stability of the closed-loop systems, where parameter convergence can be achieved under a weakened excitation condition. Simulation studies are investigated to verify the efficacy of the presented design scheme.

This article presents a model-free λ -policy iteration ( λ -PI) for the discrete-time linear quadratic regulation (LQR) problem. To solve the algebraic Riccati equation arising from solving the LQR in an iterative manner, we define two novel matrix operators, named the weighted Bellman operator and the composite Bellman operator. Then, the λ -PI algorithm is first designed as a recursion with the weighted Bellman operator, and its equivalent formulation as a fixed-point iteration with the composite Bellman operator is shown. The contraction and monotonic properties of the composite Bellman operator guarantee the convergence of the λ -PI algorithm. In contrast to the PI algorithm, the λ -PI does not require an admissible initial policy, and the convergence rate outperforms the value iteration (VI) algorithm. Model-free extension of the λ -PI algorithm is developed using the off-policy reinforcement learning technique. It is also shown that the off-policy variants of the λ -PI algorithm are robust against the probing noise. Finally, simulation examples are conducted to validate the efficacy of the λ -PI algorithm.

In this article, we consider an iterative adaptive dynamic programming (ADP) algorithm within the Hamiltonian-driven framework to solve the Hamilton-Jacobi-Bellman (HJB) equation for the infinite-horizon optimal control problem in continuous time for nonlinear systems. First, a novel function, "min-Hamiltonian," is defined to capture the fundamental properties of the classical Hamiltonian. It is shown that both the HJB equation and the policy iteration (PI) algorithm can be formulated in terms of the min-Hamiltonian within the Hamiltonian-driven framework. Moreover, we develop an iterative ADP algorithm that takes into consideration the approximation errors during the policy evaluation step. We then derive a sufficient condition on the iterative value gradient to guarantee closed-loop stability of the equilibrium point as well as convergence to the optimal value. A model-free extension based on an off-policy reinforcement learning (RL) technique is also provided. Finally, numerical results illustrate the efficacy of the proposed framework.

This article considers the robust optimal control problem for a class of nonlinear systems in the presence of unmodeled dynamics. An adaptive optimal controller is designed using the online actor–critic learning and is robustified against unmodeled dynamics. To deal with unmodeled dynamics, an auxiliary signal with the system state as its input signal is designed to capture the input-to-state stability. In addition to the critic network for value function approximation, a novel robustifying term is developed and introduced into the actor network to ensure robustness during the learning process. It is shown that both the actor and the critic weights learning converge to their optimal values while guaranteeing the boundedness of all the signals in the closed loop. Simulation examples are conducted to verify the efficacy of the presented scheme.

This article considers the distributed adaptive neuro-fuzzy output feedback control protocol design to solve the output synchronization problem for heterogeneous multiagent systems with nonlinear strict-feedback agent dynamics. The output constraints and actuator saturation are considered simultaneously. First, a distributed high-gain observer is employed to estimate the unmeasured agent state and relax the requirement of the Lipschitz continuity of nonlinear follower dynamics. Second, an asymmetric barrier Lyapunov function with time-varying constraint is presented to deal with both the transient and the steady-state constraints on the output synchronization error. To avoid the “explosion of complexity,” the dynamic surface control technique is employed to filter the virtual control signal for each follower. To deal with the actuator saturation, a distributed auxiliary dynamical system is designed for each follower. The fuzzy logic system is employed to compensate for the uncertain follower dynamics with guaranteed semiglobal uniformly ultimately boundedness of all closed-loop signals. Finally, a simulation example is conducted to verify the efficacy of the presented adaptive neuro-fuzzy controller design.

This article presents a model-based hybrid adaptive dynamic programming (ADP) framework consisting of continuous feedback-based policy evaluation and policy improvement steps as well as an intermittent policy implementation procedure. This results in an intermittent ADP with a quantifiable performance and guaranteed closed-loop stability of the equilibrium point. To investigate the effect of aperiodic sampling on the communication bandwidth and the control performance of the intermittent ADP algorithms, we use a Hamiltonian-driven unified framework. With such a framework, it is shown that there is a tradeoff between the communication burden and the control performance. We finally show that the developed policies exhibit Zeno-free behaviors. Simulation examples show the efficiency of the proposed framework along with quantifiable comparisons of the policies with different intermittent information.

This article develops a novel distributed intermittent control framework with the ultimate goal of reducing the communication burden in containment control of multiagent systems communicating via a directed graph. Agents are assumed to be under disturbance and communicate on a directed graph. Both static and dynamic intermittent protocols are proposed. Intermittent H∞containment control design is considered to attenuate the effect of the disturbance and the game algebraic Riccati equation (GARE) is employed to design the coupling and feedback gains for both static and dynamic intermittent feedback. A novel scheme is then used to unify continuous, static, and dynamic intermittent containment protocols. Finally, simulation results verify the efficacy of the proposed approach.

In this article, we present an intermittent framework for safe reinforcement learning (RL) algorithms. First, we develop a barrier function-based system transformation to impose state constraints while converting the original problem to an unconstrained optimization problem. Second, based on optimal derived policies, two types of intermittent feedback RL algorithms are presented, namely, a static and a dynamic one. We finally leverage an actor/critic structure to solve the problem online while guaranteeing optimality, stability, and safety. Simulation results show the efficacy of the proposed approach.

This paper presents a model-free solution to the robust stabilization problem of discrete-time linear dynamical systems with bounded and mismatched uncertainty. An optimal controller design method is derived to solve the robust control problem, which results in solving an algebraic Riccati equation (ARE). It is shown that the optimal controller obtained by solving the ARE can robustly stabilize the uncertain system. To develop a model-free solution to the translated ARE, off-policy reinforcement learning (RL) is employed to solve the problem in hand without the requirement of system dynamics. In addition, the comparisons between on- and off-policy RL methods are presented regarding the robustness to probing noise and the dependence on system dynamics. Finally, a simulation example is carried out to validate the efficacy of the presented off-policy RL approach.

This paper presents a data-driven method based on off-policy integral reinforcement learning to solve the semi-global output regulation of continuous-time linear systems with input saturation. A family of state feedback laws for the input constrained output regulation problem is designed based on solving an algebraic Riccati equation. In contrast to the existing methods, complete knowledge of the system dynamics is no longer required in this paper. Instead, the data collected from online implementation is efficiently utilized to design the controller. Therefore, the controller design in this paper is data-driven. It is shown that the presented method can find feedback control inputs with constraint of amplitude saturation and stabilize a given linear system with all its poles inside or on the imaginary axis. Finally, a simulation example is conducted to show the validity of the presented approach to solve the semi-global output regulation of continuous-time linear systems with input saturation.

This paper develops optimal control protocols for the distributed output synchronization problem of leader-follower multiagent systems with an active leader. Agents are assumed to be heterogeneous with different dynamics and dimensions. The desired trajectory is assumed to be preplanned and is generated by the leader. Other follower agents autonomously synchronize to the leader by interacting with each other using a communication network. The leader is assumed to be active in the sense that it has a nonzero control input so that it can act independently and update its control to keep the followers away from possible danger. A distributed observer is first designed to estimate the leader's state and generate the reference signal for each follower. Then, the output synchronization of leader-follower systems with an active leader is formulated as a distributed optimal tracking problem, and inhomogeneous algebraic Riccati equations (AREs) are derived to solve it. The resulting distributed optimal control protocols not only minimize the steady-state error but also optimize the transient response of the agents. An off-policy reinforcement learning algorithm is developed to solve the inhomogeneous AREs online in real time and without requiring any knowledge of the agents' dynamics. Finally, two simulation examples are conducted to illustrate the effectiveness of the proposed algorithm.

In this paper, based on the Hamiltonian, an alternative interpretation about the iterative adaptive dynamic programming (ADP) approach from the perspective of optimization is developed for discrete time nonlinear dynamic systems. The role of the Hamiltonian in iterative ADP is explained. The resulting Hamiltonian driven ADP is able to evaluate the performance with respect to arbitrary admissible policies, compare two different admissible policies and further improve the given admissible policy. The convergence of the Hamiltonian ADP to the optimal policy is proven. Implementation of the Hamiltonian-driven ADP by neural networks is discussed based on the assumption that each iterative policy and value function can be updated exactly. Finally, a simulation is conducted to verify the effectiveness of the presented Hamiltonian-driven ADP.

This paper presents a Hamiltonian-driven framework of adaptive dynamic programming (ADP) for continuous time nonlinear systems, which consists of evaluation of an admissible control, comparison between two different admissible policies with respect to the corresponding the performance function, and the performance improvement of an admissible control. It is showed that the Hamiltonian can serve as the temporal difference for continuous-time systems. In the Hamiltonian-driven ADP, the critic network is trained to output the value gradient. Then, the inner product between the critic and the system dynamics produces the value derivative. Under some conditions, the minimization of the Hamiltonian functional is equivalent to the value function approximation. An iterative algorithm starting from an arbitrary admissible control is presented for the optimal control approximation with its convergence proof. The implementation is accomplished by a neural network approximation. Two simulation studies demonstrate the effectiveness of Hamiltonian-driven ADP.