This paper focuses on the Nash equilibrium problem for non-cooperative games over general directed graphs with application to energy allocation. Unlike previous Nash equilibrium-seeking works that typically required strongly connected or undirected graphs, we now relax the network to contain a spanning tree. To overcome the inaccessibility caused by the general graph, we introduce an information collector who gathers player profiles and acts as the leader. Under this framework, each player preserves the state observation from leader through the spanning-tree network, and uses gradient descent to adjust variables for converging to an equilibrium point. We also present an enhanced version that invests the solo coupling gain with adaptation through distributed consensus error. The proposed schemes ensure the globally asymptotic stability of the Nash equilibrium-seeking process through Lyapunov analysis. Finally, numerical studies, as well as hardware-in-the-loop experiments for energy allocation on the RT-Lab platform, are provided to demonstrate the effectiveness of the proposed protocols.

Trajectory planning with a learning-based approach has emerged as a crucial element in autonomous unmanned systems and has attracted substantial interest from both academia and industry. However, unresolved issues persist concerning data efficiency, safety, convergence, and generalization within the control pipeline. To address this gap, this work presents a trajectory planning method that combines the differential flatness of wheeled vehicle with global convergence property. Our proposed framework transforms the trajectory planning problem, integrating kinematic constraints into a motion planning paradigm. This transformation significantly reduces the state space associated with trajectory planning. Initially, Gaussian mixture regression (GMR) is employed to learn the nonlinear mapping from flat input, leveraging a limited number of demonstrations solved by the optimal control method. Subsequently, we design an asymmetric quadratic Lyapunov function that incorporates both random barrier information and the potential convergence property of the demonstration trajectories. Based on the optimized parameterized Lyapunov function incorporating the convergence and safety criteria, the analytical supplementary control is subsequently obtained by solving a quadratic programming problem to compensate for the prediction errors of GMR, which make the framework complete. Both numerical and real-world experiments are performed to validate the effectiveness of our framework.Note to Practitioners—This work is motivated by the concept of robot motion primitives. While direct segmented planning based on dynamical system movement is feasible for robots with multiple degrees of freedom, it cannot be directly applied to wheeled vehicles with non-holonomic constraints. In this paper, we propose a method that utilizes the differential flatness properties of wheeled vehicles to bridge the gap between these two application objects. By converting the trajectory planning problem of the mobile vehicles into an optimal control problem, we can obtain a satisfactory optimal trajectory. Solving the optimal control problem involves an iterative process due to its nature as an initial value problem. While some learning-based method offer end-to-end learning capabilities and can perform real-time optimal control, they requires a substantial number of training dataset (different optimal trajectories) and lack convergence validation. The foundation of this work is the imitation learning paradigm. Although learning-based approaches are also capable of directly learning from demonstrations, our approach realizes a combination of Lyapunov theory, which allows us to extract the planning task’s potential properties from a limited number of demonstrations, and the vehicle’s differential flat properties to construct a framework with analytical solution. The framework can effectively re-plan obstacle-free trajectories, even in scenarios with obstacles not given in learning phase. This effectively compensates for the shortcomings of learning-based methods in terms of data reliance, convergence, and interpretability.

This article addresses the optimal control problem for nonlinear multiagent systems (MASs) with an uncertain nonlinear leader subject to intermittent Denial-of-Service (DoS) attacks. The main challenge is estimating the leader's dynamics when the uncertain nonlinear dynamics of the leader are unknown to all followers and communication between subsystems is intermittently disrupted by attacks. Furthermore, the uncertainty in the followers' dynamics adds complexity, making it difficult to eliminate reliance on the identifier network. To overcome these challenges, we develop a learning-based adaptive distributed observer to estimate the leader's dynamics under attacks. Based on this observer, a single-critic optimal consensus tracking control scheme is proposed to solve the leader-follower consensus problem in uncertain MASs without requiring an identifier network. It is proven that all system signals are uniformly ultimately bounded (UUB), and consensus tracking is achieved. The effectiveness of the proposed method is validated through a simulation example.

Accurate prediction of lithium-ion battery capacity is a critical task in BMSs. However, existing multisource domain adaptation methods often ignore the different contributions of each source domain, focusing solely on aligning the global distributions of source and target domains. This limitation can result in negative transfer. To address this issue, this article proposes a multisource feature separation and weighted (MFSW) network for lithium-ion battery capacity estimation. First, private and common features of both source and target domains are disentangled through feature separation. An adversarial mechanism is employed to guide the common feature extractor to learn domain-invariant features. Then, the features are further aligned using a multiorder metric. Finally, a multisource dynamic weighting method is introduced to adaptively adjust the weight of each source domain. Compared with other multisource domain adaptation methods, the proposed method reduces the average MSE and MAE by 56.3% and 28.8% on the MIT dataset, and by 44.0% and 38.6% on the XJTU dataset, respectively. Extensive experimental results demonstrate that the proposed method effectively mitigates negative transfer and exhibits superior performance and robustness.

This paper proposes two online policy-iteration (PI) algorithms for solving linear continuous-time$H_\infty$regulation problems with unknown dynamics. Our results are completely learning-orientated in the sense that prior model knowledge of initial stabilizing control policies arising from solving the Game Algebraic Riccati Equation (GARE) associated with the$H_\infty$regulation problem is now removed, which thereby resolves a long-standing challenge in the existing PI works to achieve model-free learning. To this end, two offline PI algorithms, consisting of the single-looped and the double-looped, are first proposed by nesting a homotopy-based initialization to solve a series of Lyapunov equations associated with the GARE. Then, two online PI algorithms are further proposed by utilizing the system data to avoid the model requirement for online solving the GARE. The single-looped PI algorithm has the feature of simultaneously learning control and disturbance policies, while a double-looped PI updates control policies before carrying out a series of learning disturbance policies. These two online PI algorithms work in a model-free manner and do not require prior knowledge of the system matrices over the whole learning period including the control policy initialization and can lead to the desired control policy with satisfactory system performance. We demonstrate the effectiveness of the proposed learning algorithms with an example of a power systemNote to Practitioners—Solving the$H_\infty$regulation problem for linear continuous-time systems can be achieved by finding the Nash equilibrium of the two-player zero-sum game. However, it is a challenge for control practitioners to design the$H_\infty$regulation controller with completely unknown dynamics due to the fact that it is nontrivial to obtain precise prior knowledge of models/dynamics for many engineering systems. The current methods usually utilized system data to solve the Nash equilibrium solution by offline or online iterative computation, but most of them still needed prior knowledge of the system dynamics for policy seeking such as stabilizing/admissible control or disturbance policies in the initialization. To address such a challenge, this paper develops two homotopy-based online PI algorithms that solve the$H_\infty$regulation problem in a fully model-free manner. It is shown that the developed algorithms can find the Nash equilibrium solution by online measuring the system data, and overcomes the difficulty of finding an initial stabilizing control policy with unknown system dynamics. The validity of the algorithms is illustrated through a simulation study.

Policy iteration (PI), an iterative method in reinforcement learning, has the merit of interactions with a little-known environment to learn a decision law through policy evaluation and improvement. However, the existing PI-based results for output-feedback (OPFB) continuous-time systems relied heavily on an initial stabilizing full state-feedback (FSFB) policy. It thus raises the question of violating the OPFB principle. This article addresses such a question and establishes the PI under an initial stabilizing OPFB policy. We prove that an off-policy Bellman equation can transform any OPFB policy into an FSFB policy. Based on this transformation property, we revise the traditional PI by appending an additional iteration, which turns out to be efficient in approximating the optimal control under the initial OPFB policy. We show the effectiveness of the proposed learning methods through theoretical analysis and a case study.

This article focuses on the problem of adaptive optimal tracking control of interconnected switched systems on dwell time. The objective is to establish interconnection between switched systems, stabilize each switched system, and achieve optimal performance over the dwell time. An actor-critic learning technique is utilized to design the optimal controller by evaluating the performance via the critic and improving the policy via the actor. During the design of the cost function and optimization process, we employed non-zero-sum game theory to incorporate the interconnection factors among systems. The proposed control scheme designs a novel relaxed stability condition for switched subsystems and validates the effectiveness by using a Lyapunov function with a value function. Finally, the designed control scheme can be verified by simulation.

In this article, an optimal output synchronization solution to the$H_{\infty}$optimization of linear discrete-time (DT) multiagent systems is investigated. Compared with current approaches, the issue of designated convergence rate is handled with system optimality, while less computation cost is required. Specifically, the internal model principle is employed to derive a cooperative regulation problem of DT systems, wherein no explicit solution to output regulation equations is needed for learning. Then, we introduce a convergence rate parameter to construct a group of auxiliary cooperative systems, based on which the zero-sum game in$H_{\infty}$optimization is formulated. The data-efficient off-policy reinforcement learning and output-feedback technique are applied to solve the enhanced Bellman equations with a designated convergence rate. This results in an online optimal synchronization solution learning from only the input–output data along the system trajectories. It is shown that the proposed optimal synchronization protocol achieves asymptotic synchronization for the original systems with the consensus error converging to zero at a designated rate. The effectiveness of the proposed approach is verified by the simulation results.

In this article, we study the distributed resilient cooperative control problem for directed networked Lagrangian systems under denial-of-service (DoS) attacks. The DoS attacks will block the communication channels between the agents. Compared with the existing methods for the linear networked systems, the considered nonlinear networked Lagrangian systems with asymmetric channels under DoS attacks are more challenging and still not well explored. In order to solve this problem, a novel resilient cooperative control scheme is proposed by using the sampling control approach. Sufficient conditions are first derived in the absence of DoS attacks according to a multidimensional small-gain scheme. Then, in the presence of DoS attacks, the proposed resilient scheme works in a switching manner. Inspired by multidimensional small-gain techniques, the Lyapunov approach is used to analyze the closed-loop system, which enables us to establish sufficient stability conditions for the control gains in terms of the duration and frequency of the DoS attacks.

In this article, we consider the resilience problem in the presence of communication faults encountered in distributed secondary voltage and frequency control of an islanded alternating current microgrid. Such faults include the partial failure of communication links and some classes of data manipulation attacks. This practical and important yet challenging issue has been taken into limited consideration by existing approaches, which commonly assume that the measurement or communication between the distributed generations (DGs) is ideal or satisfies some restrictive assumptions. To achieve communication resilience, a novel adaptive observer is first proposed for each individual DG to estimate the desired reference voltage and frequency under unknown communication faults. Then, to guarantee the stability of the closed-loop system, voltage and frequency restoration, and accurate power sharing regardless of unknown communication faults, sufficient conditions are derived. Some simulation results are presented to verify the effectiveness of the proposed secondary control approach.

In this article, we address the bearing-only formation control problem of 3-D networked robotic systems with parametric uncertainties. The contributions of this article are two-fold: 1) the bearing-rigid theory is extended to solve the nonlinear robotic systems with the Euler-Lagrange-like model and 2) a novel almost global stable distributed bearing-only formation control law is proposed for the nonlinear robotic systems. Specifically, the robotic systems subject to nonholonomic constraints and dynamics are first transformed into a Euler-Lagrange-like model. By exploring the bearing-rigid graph theory, a backstepping approach is used to design the distributed formation controller. Simulations for 3-D robotics are given to demonstrate the effectiveness of the proposed control law. Compared to the distance-rigid formation control approach, the bearing-rigid approach guarantees almost global stability while naturally excluding flip ambiguities.

This paper investigates a problem of designing an adaptive asymptotic cooperative control scheme for nonlinear time-varying multiagent systems, which can simultaneously tolerate unknown actuator failures and unknown control directions. To address such the problem, we propose a conditional inequality, which allows multiple piecewise Nussbaum functions to acquire the control robustness. Benefiting from this robustness, a part of failure uncertainties and system errors are compensated for, while the remaining parts are handled by adaptive control technique. Moreover, structural properties of the proposed adaptive laws are utilized so that Barbalat's lemma is applicable to make all the followers asymptotically converge to the leader based on the neighborhood information.

In this paper, we propose an adaptive neural-network-based asymptotic control algorithm for a class of nonlinear systems subject to unknown actuator quantization. To this end, we exploit the sector property of the quantization nonlinearity and transform actuator quantization control problem into analyzing its upper bounds, which are then handled by a dynamic loop gain function-based approach. In our adaptive control scheme, there is only one parameter required to be estimated online for updating weights of neural networks. Within the framework of Lyapunov theory, it is shown that the proposed algorithm ensures that all the signals in the closed-loop system are ultimately bounded. Moreover, an asymptotic tracking error is obtained by means of introducing Barbalat's lemma to the proposed adaptive law.

Most existing control results for pure-feedback stochastic systems are limited to a condition that tracking errors are bounded in probability. Departing from such bounded results, this paper proposes an asymptotic fuzzy neural network control for pure-feedback stochastic systems. The control goal is realized by proposing a novel semi-Nussbaum function-based technique and employing it in adaptive backstepping controller design. The proposed Nussbaum function is integrated with adaptive control technique to guarantee that the tracking error is asymptotically stable in probability.

This paper proposes an adaptive fuzzy asymptotic control method for multiple input multiple output (MIMO) nonlinear systems with unknown input coefficients, with a focus on handling unknown input nonlinearities and control directions. For all the existing Nussbaum gain-based approaches, it is difficult to investigate unknown input coefficients problem since multiple time-varying coefficients and disturbances coexist and should be simultaneously tackled in the stability analysis. To overcome the above difficulty, we propose a robust Nussbaum gain-based approach for the adaptive fuzzy asymptotic control of MIMO nonlinear systems. Benefiting from the proposed Nussbaum gain-based approach, bounded disturbances including unmodeled system dynamics and universal approximation errors are handled. Furthermore, the proposed approach helps extend the bounded fuzzy control result to the asymptotic convergence. Hence, both the control robustness and control accuracy are prompted within the frame of the developed Nussbaum gain approach. Finally, a simulation example is carried out to illustrate the effectiveness of the proposed control method.

This paper presents an asymptotic tracking control design method for stochastic strict-feedback systems via fuzzy logic systems. In the existing results, stochastic controls are usually limited to be bounded in probability. However, how to realize the asymptotic tracking control for stochastic strict-feedback systems remains a control dilemma. This paper achieves the asymptotic tracking control by proposing a novel gain suppressing inequality approach. Specifically, the three-part construction is performed to achieve such control objective for stochastic systems. First, the novel gain suppressing inequality technique is developed to lay the foundation for carrying out the Lyapunov stability analysis. Second, the developed inequality technique is integrated with each step of the backstepping-based adaptive control design procedure. Third, analyses are provided to realize the asymptotic tracking control of stochastic strict-feedback systems.

To achieve satisfying motion/force objectives, it is required for the multiarm robot manipulation to tackle unknown nonlinearities and uncertainties. Existing control schemes have a requirement that the deadzone nonlinearity in the sensor channel could be ignored. However, the sensor deadzone is widely spread in real world applications; and its existence significantly limits the robotic performances. Moreover, the kinematics and dynamics of the manipulated object cannot be accurately known as the prior knowledge. Hence, such robotic systems may not be perfectly controlled by conventional approaches. In this paper, a coordinated motion/force control method is presented to handle unknown sensor deadzone and object's uncertainty. The proposed method ensures the motion and internal force errors be bounded in a small neighborhood around the origin. Finally, comparative studies are presented to show the effectiveness and robustness of the proposed scheme.

This paper presents a saturated Nussbaum function based approach for robotic systems with unknown actuator dynamics. To eliminate the effect of the control shock from the traditional Nussbaum function, a new type of the saturated Nussbaum function is developed with the idea of time-elongation. Moreover, by exploiting properties of the proposed Nussbaum function, a promising theorem is established to deal with unknown multiple actuator nonlinearities. In what follows, the proposed theorem is integrated with the adaptive control technique such that the stability analysis of the robotic system is completed. It thus guarantees that the state of the robotic system asymptotically converges to the desired trajectory. Finally, comparative studies are carried out to validate the effectiveness and the superiority of the proposed approach.

To achieve an excellent dual-arm coordination of the humanoid robot, it is essential to deal with the nonlinearities existing in the system dynamics. The literatures so far on the humanoid robot control have a common assumption that the problem of output hysteresis could be ignored. However, in the practical applications, the output hysteresis is widely spread; and its existing limits the motion/force performances of the robotic system. In this paper, an adaptive neural control scheme, which takes the unknown output hysteresis and computational efficiency into account, is presented and investigated. In the controller design, the prior knowledge of system dynamics is assumed to be unknown. The motion error is guaranteed to converge to a small neighborhood of the origin by Lyapunov's stability theory. Simultaneously, the internal force is kept bounded and its error can be made arbitrarily small.

Due to the fact that backlash nonlinearity is widespread in actuators, it is impossible to ignore its existence and achieve excellent and desirable mechanical system performance. In this paper, the problem of the humanoid robot grasping a common object with unknown actuator backlash is investigated. To tackle the nonsmooth backlash nonlinearity, a smooth adaptive backlash inverse is incorporated to compensate the line-segment effect. Moreover, a decentralized robust fuzzy adaptive control is constructed and developed to guarantee the object's motion and internal forces converge to the predefined values. The stabilities of the signals in the closed-loop system are proven by utilizing the Lyapunov method. In the end, experiments and simulations involving humanoid robot manipulation are conducted to validate the effectiveness of the proposed algorithms.