In goal-oriented dialogue tasks, the main challenge is to steer the interaction towards a given goal within a limited number of turns. Existing approaches either rely on elaborate prompt engineering, whose effectiveness is heavily dependent on human experience, or integrate policy networks and pre-trained policy models, which are usually difficult to adapt to new dialogue scenarios and costly to train. Therefore, in this paper, we present Nested Rollout Policy Adaptation for Goal-oriented Dialogue (NRPA-GD), a novel dialogue policy planning method that completely avoids specific model training by utilizing a Large Language Model (LLM) to simulate behaviors of user and system at the same time. Specifically, NRPA-GD constructs a complete evaluation mechanism for dialogue trajectories and employs an optimization framework of nested Monte Carlo simulation and policy self-adaptation to dynamically adjust policies during the dialogue process. The experimental results on four typical goal-oriented dialogue datasets show that NRPA-GD outperforms both existing prompt engineering and specifically pre-trained model-based methods. Impressively, NRPA-GD surpasses ChatGPT and pre-trained policy models with only a 0.6-billion-parameter LLM. The proposed approach further demonstrates the advantages and novelty of employing planning methods on LLMs to solve practical planning tasks.

Point cloud processing methods leverage local and global point features to cater to downstream tasks, yet they often overlook the task-level context inherent in point clouds during the encoding stage. We argue that integrating task-level information into the encoding stage significantly enhances performance. To that end, we propose SMTransformer which incorporates task-level information into a vector-based transformer by utilizing a soft mask generated from task-level queries and keys to learn the attention weights. Additionally, to facilitate effective communication between features from the encoding and decoding layers in high-level tasks such as segmentation, we introduce a skip-attention-based up-sampling block. This block dynamically fuses features from various resolution points across the encoding and decoding layers. To mitigate the increase in network parameters and training time resulting from the complexity of the aforementioned blocks, we propose a novel shared point position encoding strategy. This strategy allows various transformer blocks to share the same position information over the same resolution points, thereby reducing network parameters and training time without compromising accuracy. Experimental comparisons with existing methods on multiple datasets demonstrate the efficacy of SMTransformer and skip-attention-based up-sampling for semantic segmentation task. In particular, we achieve state-of-the-art semantic segmentation results of 73.9% mIoU on S3DIS Area 5 and 62.4% mIoU on SWAN dataset. Note to Practitioners—Point cloud processing underpins automation tasks such as robotic perception, navigation, and inspection, where accurate 3D understanding is essential. Existing methods often prioritize vision benchmarks while overlooking automation needs like efficiency on limited hardware and robustness in real-world environments. The proposed SMTransformer embeds task-level guidance into feature learning and employs skip-attention up-sampling to improve segmentation accuracy with practical efficiency. It is well-suited for robotic manipulation, autonomous driving, and inspection applications. Current limitations include reliance on GPUs and sensitivity to extreme density variations. Future work will target edge-device deployment and multi-task extensions.

This paper addresses the intricate control challenges posed by unknown Euler–Lagrange systems operating under actuator faults and subject to asymmetric error constraints. A novel prescribed performance control (PPC) strategy is proposed, leveraging a dual-error transformation technique. The first transformation is designed to decouple the initial tracking errors from the predefined performance functions, thereby significantly relaxing the stringent initial condition dependencies typical of conventional PPC schemes and ensuring errors remain confined within adjustable asymmetric boundaries. The second transformation introduces exponential decay constraint functions to dynamically regulate the convergence rate and steady-state accuracy of the tracking errors. Theoretical analysis rigorously demonstrates that the proposed strategy, without requiring prior knowledge of the system’s complex nonlinearities, guarantees global boundedness of all closed-loop signals. Furthermore, it ensures that tracking errors converge to prespecified asymmetric residual zones at a preset rate, even in the presence of actuator faults. The efficacy and superiority of the proposed strategy are validated through comparative simulations conducted on a two-link robotic manipulator. The experimental source code is available at https:// github.com/hclll22/Dual-Error-Transformation-Approach-PPC.

Significant theoretical and practical challenges arise in the cooperative control of distributed nonlinear multi-agent systems (MAS), particularly when they involve nonstrict-feedback interconnections and unknown state-dependent control gains. Conventional neural adaptive controllers, while versatile, often operate as “black-box” models, leading to solutions that may lack physical plausibility and exhibit compromised robustness. This paper addresses this critical gap by introducing a novel Physics-Regularized Adaptive Control (PRAC) framework, implemented via neural backstepping. Central to PRAC is the design of novel Physics-Regularized Neural Networks (PRNNs), which are realized using Radial Basis Function Neural Networks (RBFNNs) as their architectural foundation in this paper. Instead of treating the neural network as a simple approximator, the PRAC methodology embeds physical priors, such as equilibrium conditions, system smoothness, and energy dissipation principles, into the PRNNs’ online adaptive laws as differentiable regularization terms. The gradients of these terms actively constrain the PRNN weight adaptation, transforming the learning process into a Lyapunov-guided constrained optimization. This enhances the physical consistency and interpretability of the learned dynamics while simultaneously improving control performance. By synergistically combining this physics-regularized architecture with Dynamic Surface Control (DSC) to manage computational complexity, the proposed scheme guarantees cooperative uniformly ultimately bounded (CUUB) tracking of a leader’s trajectory. Rigorous Lyapunov analysis substantiates the theoretical guarantees, which are further validated by comprehensive numerical simulations and a practical networked inverted-pendulum example demonstrating superior tracking accuracy and robustness over conventional neural adaptive controllers.

The control moment gyroscope (CMG) is widely used for high-precision attitude control in aerospace applications due to its excellent torque output and energy efficiency. However, the micro-vibrations generated during CMG operation can propagate through flexible vibration isolation platforms, which may affect the performance of sensitive spacecraft payloads. To address this critical issue, this article proposes a systematic algorithm that combines coupled dynamic modeling with intelligent control. First, a six-degree-of-freedom (six-DoF) dynamic model accounting for the flexibility of the vibration isolator is developed, which combines the flywheel rotor, gimbal system, and the vibration isolator with a flexible platform to characterize their coupled dynamics. Based on this model, an active-passive isolation control algorithm is designed, which combines feedforward disturbance compensation with a policy iteration algorithm based on adaptive dynamic programming (ADP). The feedforward component enhanced by neural networks effectively approximates and cancels out the interference caused by CMG in real time, while ADP optimization ensures adaptive and optimal vibration suppression. The simulation results demonstrate the performance of the proposed algorithm, confirming a significant reduction in vibration transmission and a marked improvement in isolation efficiency.

The task planning and motion planning problems in multi task and multi robot collaboration are usually considered as independent parts to be solved. However, in long-horizon, multi-step collaborative scenarios, this independent solution approach is difficult to effectively handle the coupling problem between task planning and motion planning, such as the inability to simultaneously consider robot motion collisions during task allocation. To address these limitations, we develop a unified hierarchical reinforcement learning framework that enables agents to learn effective policies in multi task and multi robot collaborative motion planning, supplemented by two techniques: 1) using a shared graph attention network and distributed police networks to assign target tasks to each robot at a high level, and 2) training decentralized motion planning policies for each robot at a low level to control the workspace state and target end actuator posture. The high-level and low-level policies are trained in parallel and stabilized through expert dataset guidance. To verify the effectiveness of the method, we conduct experiments on a general object placement platform and an engine hydraulic column assembly platform. The results indicate that this method can more efficiently complete multi task and multi robot collaborative work.

The large-scale injection of new energy systems into active distribution networks (ADNs) has caused voltage violations, challenging the stable operation of power grids. Recently, deep reinforcement learning (DRL) has emerged with great advantages in replacing traditional optimization methods for voltage regulation in ADNs. However, existing DRL studies face sampling inefficiency and overlook the robustness issue resulting from the uncertainties brought by renewable energy systems in ADNs, which seriously affects the application of DRL in the real world. In this paper, a novel uncertainty-aware model-based multi-agent deep reinforcement learning (MADRL) framework is proposed for robust active voltage control (AVC). First, a probabilistic ensemble of neural networks with different initializations is designed for uncertainties in environment model learning, and a hybrid data augmentation method is proposed to improve the learning efficiency and final performance of MADRL. Then, a multi-agent distributional soft actor-critic (MADSAC) framework is developed for robust voltage regulation by tackling various uncertainties in the ADN environment. Simulations are performed on the IEEE 33-bus distribution network and IEEE 141-bus distribution network to validate that the proposed model-based MADSAC algorithm can significantly improve sampling efficiency, robustness and performance in AVC.

With growing renewable integration, optimizing bidding strategies is critical for efficient capacity allocation and generator revenue. However, using real operational data poses significant privacy risks. Thus, this article proposes a novel electricity market bidding strategy optimization framework based on reward learning and enhanced by conditional tabular generative adversarial networks (CTGAN). Initially, a CTGAN is employed to synthesize market data, effectively implementing data augmentation and mitigating privacy leakage risks. Subsequently, a reward learning algorithm grounded in the maximum entropy principle is developed to infer the implicit reward functions. Finally, leveraging the identified reward functions, a deep$Q$-network algorithm generates enhanced bidding strategies. Experimental results indicate that the CTGAN method more accurately replicates real-data distributions than conventional methods. In bidding simulations, the strategy derived from the identified reward function demonstrates enhanced flexibility and strategic behavior compared to predefined reward approaches, ultimately increasing generator profits and improving market efficiency.

In this paper, trajectory planning is realized for an autonomous vehicle in urban road environments based on a digital twin (DT) system. A feedforward-feedback control framework is also proposed to complete virtual-physical synchronous in the DT system. An A${}^{\ast }$algorithm is improved to obtain a global path based on a high-precision map in OpenDRIVE format. The global path is utilized by a virtual vehicle with a model predictive controller to produce a reference trajectory and feedforward inputs for a physical vehicle. A backstepping controller with the feedforward inputs is designed for the physical vehicle to track the reference trajectory. Experimental results demonstrate effectiveness of the trajectory planning and feedforward-feedback control method based on the DT system.

This article presents an accelerated distributed optimization algorithm for online optimization problems over large-scale networks. The proposed algorithm's iteration only relies on local computation and communication. To effectively adapt to dynamic changes and achieve a fast convergence rate while maintaining good convergence performance, we design a new algorithm called NGTAdam. This algorithm combines the Nesterov acceleration technique with an adaptive moment estimation method. The convergence of NGTAdam is evaluated by evaluating its dynamic regret through the use of linear system inequality. For online convex optimization problems, we provide an upper bound on the dynamic regret of NGTAdam, which depends on the initial conditions and the time-varying nature of the optimization problem. Moreover, we show that if the time-varying part of this upper bound is sublinear with time, the dynamic regret is also sublinear. Through a variety of numerical experiments, we demonstrate that NGTAdam outperforms state-of-the-art distributed online optimization algorithms.

This article investigates safe cooperative control problem for nonlinear multiagent systems (MASs) with composite constraints and multiactuator faults. Specifically, the composite constraints primarily consist of state constraints and terminal time constraints. For the former, state-mapping functions and universal barrier functions are introduced to address asymmetric dynamic delayed constraints, which are further enhanced through state transformation to create new state variables. This approach eliminates the limitations imposed by initial system conditions and the influence of unknown terms, while accommodating both unconstrained and constrained scenarios. The latter introduces terminal time constraints on the basis of the former framework to rapidly satisfy system performance requirements. Furthermore, neural networks (NNs) are employed to approximate the reconstructed multiple fault information and system uncertainties. Then, an actor-critic-identifier architecture based on fuzzy reinforcement learning (RL) is constructed via an optimal backstepping (OB) method, enabling the solution of the Hamilton–Jacobi–Bellman equation within each subsystem without the need for persistent excitation. Finally, by integrating Lyapunov stability theory with graph theory, it is proven that all signals are bounded, and the followers ultimately achieve dynamic consensus with the leader. Simulation results are provided to demonstrate the effectiveness of this control strategy.

This article investigates finite/fixed-time synchronization (FTS/FXTS) for delayed inertial memristive neural networks (IMNNs) with time-varying actuator faults, a more practical scenario compared to the widely studied constant-fault case. Novel fault-tolerant controllers, including state-feedback and event-triggered schemes, are proposed to achieve synchronization under mixed delays. By establishing algebraic criteria via a nonreduced-order approach, explicit settling time estimates are derived while excluding Zeno behavior. The theoretical results are verified through simulations, and the proposed method is further applied to secure communication using synchronized IMNNs for image encryption.

In this letter, we propose a new method for visual locomotion controller in quadruped robots, aimed at enhancing their capability to traverse challenging terrain. Our approach integrates computer vision techniques with robust locomotion control to improve terrain traversal. To facilitate terrain perception, we use onboard cameras and body sensors to collect real-world visual and proprioceptor data, and utilize forward kinematics to convert joint angles into precise foot positions. This enables accurate estimation of terrain height, which serves as supervised training data for our visual motion controller. This integrated approach improves the robot's ability to anticipate and adapt to diverse terrain conditions, potentially advancing quadruped locomotion in unstructured environments. Our model is deployed on A1 robot from Unitree. Experimental results show that our proposed method can enable the robot to stably climb stairs and pass through sand, grass, snow, and uneven roads.

This article presents a safety-critical control framework for navigation in complex environments with numerous obstacles. An online robust path planning scheme is developed by integrating reinforcement learning (RL) with control barrier functions (CBFs). First, a disturbance observer is designed to estimate the unknown disturbance along with a derived upper bound of the estimation error. Then, a nominal controller is designed using RL, where a critic neural network (NN) structure is established by using state-following (StaF) kernel function. Additionally, by employing a state extrapolation technique, the learning process leverages both real-time and simulated experience data. To ensure safety, obstacle avoidance is formulated as a forward invariance problem of safe sets defined by CBFs. Subsequently, the CBF-based safety-critical constraints are integrated into a quadratic programming (QP) framework to modify the nominal controller. Furthermore, these CBFs are incorporated into a composite CBF using smooth approximation, enabling efficient constraint consolidation. Then, an explicit safe control policy is proposed that guarantees collision-free path planning. Finally, the effectiveness of the proposed scheme is demonstrated through numerical simulations, and comparative results show the advantages over the existing methods in motion trajectory.

In this paper, the fixed-time (FxT) and practical FxT consensus of discontinuous multi-agent systems (MASs) via adaptive intermittent control strategy are studied. New intermittent FxT stability lemmas incorporating the indefinite function and the unified exponent condition are established, which can include the existing periodically intermittent stability lemmas and aperiodic intermittent lemmas with negative definite derivatives. Considering that the MASs cannot realize the exact consensus under the inevitable external interferences, intermittent-type practical FxT stability lemmas are further obtained. The intermittent controllers and adaptive intermittent controllers are designed, which demonstrates the improvements over the existing control strategies since the chattering phenomenon and singularity do not appear. Notably, the control gains are time-varying, which are also different from the previous time-invariant ones. Based on the established stability lemmas, the FxT consensus and practical FxT consensus are achieved. Clearly, the practical FxT consensus of discontinuous MASs under the discontinuous-time controllers is achieved for the first time. The simulation confirms the applicability of the proposed methods and shows strong support for our theoretical findings.

Due to the limited bandwidth and energy of communication channels among agents in practical applications, the communication-efficient distributed optimization method has emerged as a pressing research topic in recent years. The distributed economic dispatch problem with restricted data communication/finite communication bandwidth is investigated in this study, where the communication among agents can be described as a strongly connected directed network. For this purpose, a robust push-pull distributed optimization algorithm with a dynamic scaling quantization mechanism is developed based on the gradient tracking technique. A novel surplus variable is designed to prevent the accumulation of quantization errors, and then, a heavy-ball momentum is introduced to speed up convergence performance. In addition, a linear convergence rate of the developed approach is deduced for the strongly convex and Lipschitz smooth cost function. Finally, we offer two instances for illustration. Note to Practitioners—This paper proposes a robust quantization-based algorithm for the economic dispatch problem, in which the broadcasting information is quantized before sending to its neighboring generators. Therefore, this method reduces duplicate transmission of agents and improves the use of communication resources. Furthermore, the developed method can be extended to similar constrained optimization problems, such as the resource allocation problem in wireless networks, and the network utility maximization problem in the Internet.

This paper addresses the problem of distributed formation-containment tracking control for uncertain multi-agent systems (MASs) with completely unknown system nonlinearities, denial-of-service (DOS) attacks, and switching communication topologies. To enhance the system robustness, neural networks (NNs) are utilized to identify the unknown nonlinear terms. Additionally, a novel distributed observer is designed to reconstruct the external unmeasured attack dynamics. To handle the switching topologies of MASs, a mechanism using piecewise continuous functions is proposed to counteract the unexpected controller actions during switching time instants. Furthermore, the clever design of barrier Lyapunov functions aids in achieving the required predefined performance. A fractional power nonlinear filter is introduced to tackle the problem of computational complexity. By applying the local neighborhood states information and the lyapunov stability theory, the presented control method evaluates the stability of the MASs and shows that the designed controller not only enables the system output to track a formation trajectory in the presence of external attacks but also converges the consensus errors into a predefined set. Finally, simulation results are provided to validate the effectiveness of the proposed control strategy.

Decentralized secure control faces significant challenges in handling unknown mismatched interconnections and reducing fault-tolerant delays. To address these issues, this paper proposes a synergetic learning-based decentralized secure tracking control scheme for nonlinear interconnected systems with multiple actuator faults. Replacing actual states with desired ones in the coupled system relaxes the assumption of requiring a known upper bound for interconnections, and a neural network observer is designed to estimate the replaced interconnections. To reduce fault-tolerant delays, the secure tracking control problem is reformulated as an adversarial evolution problem between fault signals and control inputs, eliminating the need for fault compensation. To achieve optimal tracking control, an augmented subsystem is constructed by integrating the dynamics of tracking error and the reference trajectory. A modified cost function is designed for the augmented subsystem, and a critic network with two cooperative updating laws is developed to solve the Hamilton–Jacobi–Isaacs equation, providing a synergetic approximate solution for the control input and fault assistance signal. It is proven that the tracking error converges to a small neighborhood of the equilibrium. Simulation results demonstrate the effectiveness of the proposed approach.

In this article we focus on solving the optimal control problem for stochastic game systems while considering multiple constraints. Initially, applying the asymmetric time-varying mapping functions, the constrained nonzero-sum stochastic differential games are transformed into the ones in unconstrained forms. Moreover, the actor-critic architecture is designed to achieve the Nash equilibrium, the critic fuzzy logic systems (FLSs) and actor FLSs are utilized to approximate the optimal value functions and the control policies, respectively. Subsequently, a dynamic event-triggered mechanism is devised and an additional dynamic variable is defined to characterize the past triggering information, which provides a larger inter-event time. Mathematical analysis reveals the stability of closed-loop system, the weight convergence and the characteristics of dynamic event-triggered mechanism. Finally, simulation results prove the effectiveness of the proposed method.

This article introduces hybrid-dependent event-triggered schemes for global asymptotical synchronization (GAS) of Takagi–Sugeno fuzzy memristive neural networks (FMNNs) with nondifferentiable time-varying delays. First, some schemes incorporating an exponential decay rate function with tunable parameters are established. Under these schemes, sufficient conditions for GAS of FMNNs are derived by formulating a delayed differential inequality. This approach eliminates the necessity for constructing complex Lyapunov functionals and effectively accommodates nondifferentiable delay terms. Notably, this is accomplished by a nonchattering event-triggered controller. In addition, Zeno behavior is precluded by proving the existence of a positive lower bound on the interexecution time. Finally, numerical simulation is provided to validate the effectiveness of these theoretical results.

The growth of global wind capacity necessitates efficient maintenance and inspection, particularly in offshore wind farms. However, existing manual inspection method is time consuming and lacks scalability. This article introduces an autonomous unmanned aerial vehicle (UAV) inspection method for offshore wind farms by combing heuristic algorithm with deep reinforcement learning (DRL). Our approach integrates UAVs deployment and routing optimization, factoring in environmental conditions, such as wind and weather. The inspection task is divided into UAVs deployment and routing optimization, with models constructed for UAVs, offshore wind farms, and wind conditions. A heuristic algorithm is used to optimize UAVs deployment, ensuring efficient coverage despite the UAVs’ limited range. For routing optimization, we develop a DRL model for effective path planning of UAVs simultaneously. Experimental results demonstrate the superior efficiency of our approach in terms of coverage, energy consumption, and adaptability to environmental changes. By introducing the hybrid DRL algorithm, a scalable and robust solution for offshore wind farm inspections is developed, enhancing both operational efficiency and reliability.

Multiple change point detection seeks to identify potential shifts in the data properties. While existing detection methods primarily focus on univariate and multivariate data, they often fall short in detecting variations in profile data, which represent the functional relationships between explanatory and response variables. This paper introduces a novel change point detection method tailored for profile data, employing a smooth profile decomposition (SPD) strategy that accommodates arbitrary designs and heteroscedasticity. This approach facilitates a comprehensive representation of both overall trends and fluctuations within the profiles. Furthermore, we propose an order-splitting screening estimator (OSE) to construct a detection statistic, allowing for precise estimation of change points while ensuring a significant theoretical guarantee regarding the false discovery rate (FDR). We validate the performance and robustness of the proposed method through numerical experiments and a real case study involving wind turbines.

With the gradual application of reinforcement learning (RL), safety has emerged as a paramount concern. This article presents a novel data-model hybrid-driven safe RL (SRL) scheme to address the challenge of avoidance control in the operation domain containing multiple moving unsafe zones. First, the avoidance problem is transformed into the optimal control problem of an augmented system by encoding a barrier function (BF) term into the cost function. Then, using the idea of integral RL (IRL), an adaptive learning algorithm is proposed for generating safe control policies, in which the actor-critic neural network (NN) structure is established with the aid of state-following (StaF) kernel function. The policy iteration process is executed by this structure; specifically, the critic network undergoes gradient-descent adaptation, while the actor network employs gradient projection updating. Particularly, via a state extrapolation technique, both real-time experience and simulated experience are utilized in the learning process. Next, closed-loop stability and weight convergence are theoretically substantiated. Finally, the effectiveness of the proposed scheme is demonstrated on a single integrator system, a nonlinear numerical system, and a unicycle kinematic system; besides, its advantages over the existing control methods are illustrated by comparisons.

In this article, the synchronization problem of memristive neural networks (MNNs) subjected to multiple failures is investigated. First, a general form of fault model is introduced into the MNNs, which can represent and summarize various process faults, actuator faults, and their coupling. Subsequently, with the help of designing intermediate variables, two types of fault function observers based on state feedback and output feedback are constructed, and their effectiveness is verified through a generalization of Halanay-type inequalities. Then, based on the designed observers and the event-triggered strategy, two classes of fault-tolerant synchronization schemes are designed for the considered MNNs. By adjusting the controller parameter conditions, finite-time and fixed-time synchronization or quasi-synchronization of the considered MNNs system can be achieved, respectively. Finally, the effectiveness of the provided fault observers and synchronization strategies is verified through simulation and comparison experiments.

In this article, we proposed a novel fault-tolerant control scheme for quadrotor unmanned aerial vehicles (UAVs) based on spiking neural networks (SNNs), which leverages the inherent features of neural network computing to significantly enhance the reliability and robustness of UAV flight control. Traditional control methods are known to be inadequate in dealing with complex and real-time sensor data, which results in poor performance and reduced robustness in fault-tolerant control. In contrast, the temporal processing, parallelism, and nonlinear capacity of SNNs enable the fault-tolerant control scheme to process vast amounts of sensory data with the ability to accurately identify and respond to faults. Furthermore, SNNs can learn and adjust to new environments and fault conditions, providing effective and adaptive flight control. The proposed SNN-based fault-tolerant control scheme demonstrates significant improvements in control accuracy and robustness compared with conventional methods, indicating its potential applicability and suitability for a range of UAV flight control scenarios.

This article investigates the finite-time stabilization problem of inertial memristive neural networks (IMNNs) with bounded and unbounded time-varying delays, respectively. To simplify the theoretical derivation, the nonreduced order method is utilized for constructing appropriate comparison functions and designing a discontinuous state feedback controller. Then, based on the controller, the state of IMNNs can directly converge to 0 in finite time. Several criteria for finite-time stabilization of IMNNs are obtained and the setting time is estimated. Compared with previous studies, the requirement of differentiability of time delay is eliminated. Finally, numerical examples illustrate the usefulness of the analysis results in this article.

This study proposes a neural-network (NN)-based adaptive fixed-time control method for a two-degree-of-freedom (2-DOF) nonlinear helicopter system with input quantization and output constraints. First, a hysteresis quantizer is employed to mitigate chattering during signal quantization, and adaptive variables are utilized to eliminate errors in the quantization process. Subsequently, the system uncertainties are approximated using a radial basis function NN. Simultaneously, a logarithmic barrier Lyapunov function (BLF) is constructed to prevent the system outputs from violating the constraint boundaries. Based on a rigorous Lyapunov stability analysis and the fixed-time stability criterion, the signals of the closed-loop system are proven to be bounded within a fixed time. Finally, numerical simulations and experiments verified the feasibility of the proposed method.

In this article, a novel model-free dynamic event-triggered adaptive learning control scheme is developed for continuous-time linear multiagent systems. This control scheme is different from model-based control scheme in the sense that prior knowledge of the system's model is not required. To further reduce transmission data, an event-triggered control policy based on static event-triggered mechanism (SETM) and dynamic event-triggered mechanism (DETM) is proposed. Compared to SETM, DETM may significantly produce larger average event intervals and maintain control performance. In addition, based on off-policy integral reinforcement learning, an adaptive iteration method is proposed with convergence proof. Numerical tests on both linear and nonlinear multiagent systems are conducted to demonstrate that the proposed scheme can guarantee learning performance and larger triggering intervals. Finally, the learning control scheme is tested on the multiarea power system, which can illustrate the reliability and practicality of this method. Specifically, the load frequency control problem of the multiarea power system is studied using three control schemes, revealing that DETM can achieve a better frequency response at the lowest information transmission rate and ensure the overall quality and reliability of the power system.

In this study, the fixed-time synchronization (FTSn) problem is investigated for coupled reaction-diffusion systems (RDSs) with time-varying delay based on an aperiodically intermittent control (AIC) strategy. For the fixed-time control, the convergence time can be estimated in advance, irrespective of initial states. Additionally, unlike the previous studies with the semi-intermittent control strategy, the FTSn is achieved for the coupled RDSs via completely AIC by adopting the average control rate, then the mechanism is more general. Meanwhile, the utilization of average control rate indicates that the results obtained are less conservative. Furthermore, a new auxiliary function is designed to demonstrate that the fixed-time convergence of the coupled RDSs can be guaranteed with or without the presence of time-varying delay. Finally, numerical examples are provided to verify the effectiveness of the theoretical results.

This article investigates distributed adaptive leader-following consensus tracking optimal control problem for nonlinear multiagent systems (MASs) subject to unknown nonlinearities and uncertain external disturbances. In contrast to traditional centralized control, the primary challenge is that only partial subsystems can access the desired reference trajectory. To address this, positive time-varying smooth function compensating terms are introduced to counteract the effects of uncertain external disturbances and unknown desired trajectories. Then, by fusing consensus errors into the backstepping technique, feedforward controllers are given. On this basis, the controlled nonlinear systems are transformed into an equivalent affine form, and feedback optimal controllers are designed using actor and critic neural networks (NNs) to execute control behavior and evaluate control performance. The whole control laws comprise both feedforward and feedback controllers. The proposed distributed adaptive consensus control protocol can simultaneously achieve desired optimal control performance and minimize the cost function, as demonstrated through theoretical analysis and simulation results.

In dynamic systems with communication limitations and delay, the input-to-state stability (ISS) is crucial for ensuring system performance. In this article, the finite-time ISS (FTISS) of time-delay neural networks (NNs) with disturbances is investigated. First, by further considering the idea of finite-time contractive stability (FTCS), the prescribed performance is proposed for the considered NNs system, thereby achieving better learning ability and robustness. Next, in order to achieve the above research objectives, some stability conditions for the considered NNs with disturbances are given by constructing sequentially two classes of Lyapunov functions and a finite-time contractive function. Subsequently, a stabilization strategy is proposed to further reduce the parameter requirements of the NNs system and improve its application value. Finally, the numerical simulation and comparative experiments have verified the effectiveness of the stability strategy provided in this article.

In this paper, an aperiodically intermittent event-based control strategy is developed to investigate the practical fixed-time consensus (FTC) tracking problem of nonlinear multi-agent systems (MASs). Different from the traditional event-based scheme, we incorporate the event-based scheme into the intermittent control mechanism, and the aperiodically intermittent event-based mechanism is developed, which can significantly save resources, particularly in terms of reducing the energy consumption of communication. Additionally, our proposed mechanism enables practical intermittent event-based FTC tracking for a directed graph, while eliminating the dependence on initial states for convergence time estimation. Moreover, the measurement error and intermittent event-based controller are constructed based on the hyperbolic tangent function, then the non-differentiable problem and Zeno behavior can be avoided. Furthermore, an improved triggering mechanism of the event-based scheme is designed to avoid continuous monitoring in control intervals. Hence, resource consumption can be further reduced. Finally, the multiple ground vehicles and Chua’s circuit are considered in simulation examples to verify the effectiveness of theoretical results.Note to Practitioners—This paper addresses the FTC tracking problem of MASs via intermittent event-based control for a directed graph, which can be applied to multiple ground vehicles and Chua’s circuit system. Unlike the asymptotic and finite-time stability results, the upper bound of the convergence time can be estimated, which is unrelated to the initial states and can better satisfy the application requirements. Considering the limitation of communication bandwidth and saving resources, we take the event-based scheme into the intermittent control mechanism, and a new aperiodic intermittent event-based controller is designed under the fixed-time convergence. Contrary to the traditional fixed-time control strategies via intermittent control or event-based control, the proposed algorithms in this study can effectively reduce the update frequency of the controller and significantly save energy under intermittent monitoring, which is more friendly for control engineers. The feasibility of the obtained results is demonstrated by examples of multiple ground vehicles and Chua’s circuit. Potential applications of the proposed control algorithms include smart grid, cooperative search and exploration.

This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method.Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

This study focuses on the novel reinforcement learning control strategy of a nonlinear two-degrees-of-freedom (2-DOF) helicopter system for tracking the desired trajectory while minimizing the tracking error. First, gradient descent algorithm is incorporated in the context of the reinforcement learning control scheme to obtain the adaptive laws. Subsequently, considering the uncertainties in the nonlinear system, radial basis function (RBF) neural networks (NNs) are exploited to approximate the unknown internal dynamics. In contrast to the previous studies, aiming at accelerating the convergence in reinforcement learning control, a barrier Lyapunov function is constructed to constrain the states to ensure that the tracking error rapidly converges to a neighborhood of zero. Under the proposed control strategy, the states of the closed-loop system are proven to be semi-globally uniformly ultimately bounded through rigorous Lyapunov analyses, and the state constraints are satisfied. Furthermore, the simulations and experiments conducted on a Quanser laboratory platform reveal that the proposed control functions are suitable and effective. Note to Practitioners—This paper is motivated by designing a reinforcement learning control strategy to enhance online learning capability and control performance of the controller for a nonlinear 2-DOF helicopter system. The control framework is divided into the design of the critic and actor NNs, responsible primarily for evaluating the control performance and approximating uncertainties in the system separately. Unlike the adaptive NN control, the actor NN weights are updated by combining information of states and inputs from the critic NN. In addition, aiming at accelerating the convergence, a barrier Lyapunov function is constructed to constrain the states to ensure that the tracking error rapidly converges to a neighborhood of zero. Finally, the proposed control strategy is validated in simulation and experiment on the Quanser laboratory platform.

This article focuses on the fixed-time (FxT) synchronization of discontinuous fuzzy competitive neural networks (DFCNNs) with leakage-delays and state-dependent switching. Discontinuous activations, fuzzy terms, leakage delays, and state-dependent switching are considered into the competitive neural networks to increase the generality in practical applications. Before studying the synchronization, as an important theoretical basis, a new FxT stability lemma is established based on the method of nonscaling integral denominator, which is low conservative. Five different settling-time estimates are explicitly given by discussing the range of the exponents. Previous results are included and greatly improved. In order to achieve the FxT synchronization of the addressed DFCNNs, a continuously bounded control scheme is proposed by introducing a saturation function rather than a sign function, which cannot only solve the chattering phenomenon, but also ensure the boundedness of the control input. Some new leakage-delay-dependent synchronization criteria are derived for the DFCNNs. Finally, the validity of the main results is verified by numerical simulations.

A dynamic event-based optimal learning scheme is provided in the paper for nonlinear systems subject to constrained inputs and stochastic disturbances. An actor-critic structure is constructed to learn the stochastic optimal solution, which includes critic fuzzy logic system (FLS) and actor FLS. The experience replay technique and gradient-descent adaption method are used to periodically tune the critic FLS, which can approximate the optimal cost function. Based on the static event-triggered control mechanism (ETCM), a dynamic ETCM is designed, which can incorporate past triggering information and generate a longer inter-event time. The actor FLS is updated at aperiodic jumping points, which can approximate the optimal control policy. The combination of dynamic ETCM and learning structure ensures the stochastic stability of closed-loop system. The efficiency of the controller is illustrated on a numerical example and a manipulator system.

This article investigates a generalized type of multistability about almost periodic solutions for memristive Cohen–Grossberg neural networks (MCGNNs). As the inevitable disturbances in biological neurons, almost periodic solutions are more common in nature than equilibrium points (EPs). They are also generalizations of EPs in mathematics. According to the concepts of almost periodic solutions and$\Psi$-type stability, this article presents a generalized-type multistability definition of almost periodic solutions. The results show that$(K+1)^n$generalized stable almost periodic solutions can coexist in a MCGNN with$n$neurons, where$K$is a parameter of the activation functions. The enlarged attraction basins are also estimated based on the original state space partition method. Some comparisons and convincing simulations are given to verify the theoretical results at the end of this article.

This article is devoted to analyzing the multistability and robustness of competitive neural networks (NNs) with time-varying delays. Based on the geometrical structure of activation functions, some sufficient conditions are proposed to ascertain the coexistence of equilibrium points, of them are locally exponentially stable, where represents a dimension of system and is the parameter related to activation functions. The derived stability results not only involve exponential stability but also include power stability and logarithmical stability. In addition, the robustness of stable equilibrium points is discussed in the presence of perturbations. Compared with previous papers, the conclusions proposed in this article are easy to verify and enrich the existing stability theories of competitive NNs. Finally, numerical examples are provided to support theoretical results.

In this study, a cooperative control methodology is developed for an air-ground system under a complicated environment. The air-ground system consists of a car-like mobile robot (CLMR) with road bumps and a quadrotor with gust winds. For the quadrotor, a cooperative integral sliding-mode controller is suggested to achieve bidirectional signal connection. A fixed-time extended state observer (ESO) is used to assess the external disruptions brought on by gust winds. For the CLMR, a double closed-loop tracking control approach is utilized to track the trajectory. An outer loop kinematics controller generates desired velocities. An inner loop nonlinear ESO estimates velocities and compensates for road bumps. Based on the Lyapunov method, stability analyses are given for the air-ground systems. Experimental results verify that the proposed approach for the air-ground system is effective.

This article focuses on the performance analysis of distributed controllers for general linear multiagent systems in the sense of convergence time and energy consumption. First, the specified-time optimal controller is obtained using the linear operator theory-based method, and then the optimal topology is deduced. Second, to analyze the impact of communication topology on energy consumption, two distributed, suboptimal specified-time controllers are developed for undirected and directed graphs, respectively. By utilizing the inverse optimality method and Lyapunov function scaling, the performance in terms of the bounds of the gaps between the energy consumption of the suboptimal and optimal control laws is derived, which evaluates the effectiveness of the suboptimal controllers. Finally, as the simulation results show, the performance can specify appropriate settling times for applications with different energy budgets and facilitate optimizing the communication topology to reduce the energy gap.

This article develops an intermittent feedback optimal control scheme for nonlinear systems with asymmetric input saturation using a dynamic event-triggering mechanism. First, an infinite horizon nonquadratic value function with a novel integrand is formulated for the studied system to evaluate the performance, tackle the asymmetric input saturation, and remove certain rigorous assumptions in prior related studies. Second, a critic neural network (CNN) in the adaptive dynamic programming framework is constructed to obtain the optimal event-triggered control (ETC). An improved concurrent learning technique is then developed to update the CNN’s weights without requiring the persistence of excitation condition. Compared with the static ETC scheme, the present dynamic ETC strategy consumes fewer computational resources. Third, the uniform ultimate boundedness of the state, the weight estimation error, and the internal dynamic variable are assured, and the Zeno behavior is excluded. Finally, a rotational-translational actuator system is given to validate the developed intermittent feedback optimal control scheme.

This article mainly studies the multistability of complex-valued neural networks (CVNNs) with general periodic-type activation functions. In order to improve the storage capacity of associative memory, a general periodic-type activation function is introduced which obtains three different numbers of equilibrium points (EPs), including unique, finite, and countable infinite. The existence and stability of equilibria are investigated based on Brouwer’s fixed point theorem andM-matrix method. By means of a sign function on complex numbers, stability is confirmed using a new norm on the absolute values of the real and imaginary parts. The attraction basins of exponentially stable equilibria are estimated, which are bigger than the subspaces of the original division. Also, the design of associative memory is given. Finally, two numerical simulation examples verify the obtained results.

This brief discusses the reachable set estimation (RSE) problem for memristive neural networks (MNNs) involving time-varying delays and bounded disturbances. The reachable sets of the considered MNNs under zero and nonzero initial conditions are estimated by two novel algebraic criteria, respectively. Compared with the existing results, the conclusions are easy to verify, and the obtained reachable sets are more accurate. Finally, the validity of the theoretical results is illustrated by two examples.

With the increasing scale of cloud data centers (CDCs), the energy consumption of CDCs is sharply increasing. In this article, an efficient energy-saving strategy is proposed for CDCs. The greedy virtual machine (VM) deployment strategy is obtained by using the least number of servers, the heuristic VM migration strategy is obtained by using the improved double threshold algorithm, and the comprehensive VM scheduling strategy of severs is obtained by combining deployment and migration strategies. Furthermore, for the privacy security of VM scheduling, a safety-oriented energy-saving scheme based on information difference is proposed to ensure the dataset availability under privacy protection, comparing with$\varepsilon$-differential privacy algorithm and$(\varepsilon, \delta)$-differential privacy algorithm. Simulation results show that the safe energy-saving strategy can significantly reduce the energy consumption in CDCs with guaranteeing the security and availability of the important datasets.

In this article, a data-based feedback relearning (FR) algorithm is developed for the uncertain nonlinear systems with control channel disturbances and actuator faults. Uncertain problems will influence the accuracy of collected data episodes, and in turn affect the convergence and optimality of the data-based reinforcement learning (RL) algorithm. The proposed FR algorithm can update the strategy online by relearning from the empirical data. The strategy can continuously approach the optimal solution, which improves the convergence and optimality of the algorithm. Moreover, based on the experience replay technology, a data processing method is designed to further improve the data utilization efficiency and the algorithm convergence. A neural network (NN)-based fault observer is used to achieve the model-free fault compensation. The polynomial activation function is redesigned by using the sigmoid function/hyperbolic tangent activation function, to reduce the difficulty of NNs design for an unknown nonlinear system and improve the generalization. In the face of disturbances and actuator faults, the control performance, algorithm convergence, and optimality of the proposed strategy can be well guaranteed through comparative simulation.

Classification of electroencephalogram-based motor imagery (MI-EEG) tasks is crucial in brain–computer interface (BCI). EEG signals require a large number of channels in the acquisition process, which hinders its application in practice. How to select the optimal channel subset without a serious impact on the classification performance is an urgent problem to be solved in the field of BCIs. This article proposes an end-to-end deep learning framework, called EEG channel active inference neural network (EEG-ARNN), which is based on graph convolutional neural networks (GCN) to fully exploit the correlation of signals in the temporal and spatial domains. Two channel selection methods, i.e., edge-selection (ES) and aggregation-selection (AS), are proposed to select a specified number of optimal channels automatically. Two publicly available BCI Competition IV 2a (BCICIV 2a) dataset and PhysioNet dataset and a self-collected dataset (TJU dataset) are used to evaluate the performance of the proposed method. Experimental results reveal that the proposed method outperforms state-of-the-art methods in terms of both classification accuracy and robustness. Using only a small number of channels, we obtain a classification performance similar to that of using all channels. Finally, the association between selected channels and activated brain areas is analyzed, which is important to reveal the working state of brain during MI.

The optimal control of multi-input system can be described by a multiplayer nonzero-sum differential game. This article theoretically presents an event-based adaptive learning scheme to approximate the Nash equilibrium, and practically addresses the cruise control problem for Caltech vehicle systems. This design is deployed in two aspects. On one hand, the reinforcement learning is implemented through critic neural network architecture and recalling stored experience data. On the other hand, in view of that each player’s preference is different, the decentralized triggering manner is considered to reduce communication. Based on the continuous state, the local sampled state is defined for each player, and a static triggering mechanism is formulated first. The decentralized dynamic triggering is then promoted by designing an auxiliary variable whose dynamics are constructed using static triggering information. Next, the proposed learning scheme is examined on a four-player numerical system. Finally, the learning-based controller is tested on a single-vehicle system under different tracking commands, and then, it is extended to multivehicle systems to realize cooperative optimization by introducing a novel game-in-game structure.

This article presents a nearly optimal solution to the cooperative formation control problem for large-scale multiagent system (MAS). First, multigroup technique is widely used for the decomposition of the large-scale problem, but there is no consensus between different subgroups. Inspired by the hierarchical structure applied in the MAS, a hierarchical leader-following formation control structure with multigroup technique is constructed, where two layers and three types of agents are designed. Second, adaptive dynamic programming technique is conformed to the optimal formation control problem by the establishment of performance index function. Based on the traditional generalized policy iteration (PI) algorithm, the multistep generalized policy iteration (MsGPI) is developed with the modification of policy evaluation. The novel algorithm not only inherits the advantages of high convergence speed and low computational complexity in the generalized PI algorithm but also further accelerates the convergence speed and reduces run time. Besides, the stability analysis, convergence analysis, and optimality analysis are given for the proposed multistep PI algorithm. Afterward, a neural network-based actor-critic structure is built for approximating the iterative control policies and value functions. Finally, a large-scale formation control problem is provided to demonstrate the performance of our developed hierarchical leader-following formation control structure and MsGPI algorithm.

An adaptive neural network (NN) control is proposed for an unknown two-degree of freedom (2-DOF) helicopter system with unknown backlash-like hysteresis and output constraint in this study. A radial basis function NN is adopted to estimate the unknown dynamics model of the helicopter, adaptive variables are employed to eliminate the effect of unknown backlash-like hysteresis present in the system, and a barrier Lyapunov function is designed to deal with the output constraint. Through the Lyapunov stability analysis, the closed-loop system is proven to be semiglobally and uniformly bounded, and the asymptotic attitude adjustment and tracking of the desired set point and trajectory are achieved. Finally, numerical simulation and experiments on a Quanser's experimental platform verify that the control method is appropriate and effective.

In this article, the global asymptotic stability of a class of recurrent neural networks (RNNs) with generalized piecewise constant argument (GPCA) is investigated. By the Banach fixed point theorem (BFPT) and comparison principle, a set of improved criteria are presented to guarantee the existence and uniqueness (EU) of the solutions and global asymptotic stability of equilibrium point for the considered RNNs. Compared with the existing results, this article not only reduces the requirements for system parameters, but also provides more criteria in different forms, which greatly improve the feasible range of the obtained criteria. The effectiveness of the obtained results are tested by some numerical examples.

This article presents an event-sampled integral reinforcement learning algorithm for partially unknown nonlinear systems using a novel dynamic event-triggering strategy. This is a novel attempt to introduce the dynamic triggering into the adaptive learning process. The core of this algorithm is the policy iteration technique, which is implemented by two neural networks. A critic network is periodically tuned using the integral reinforcement signal, and an actor network adopts the event-based communication to update the control policy only at triggering instants. For overcoming the deficiency of static triggering, a dynamic triggering rule is proposed to determine the occurrence of events, in which an internal dynamic variable characterized by a first-order filter is defined. Theoretical results indicate that the impulsive system driven by events is asymptotically stable, the network weight is convergent, and the Zeno behavior is successfully avoided. Finally, three examples are provided to demonstrate that the proposed dynamic triggering algorithm can reduce samples and transmissions even more, with guaranteed learning performance.

We propose an adaptive neural-network-based fault-tolerant control scheme for a flexible string considering the input constraint, actuator gain fault, and external disturbances. First, we utilize a radial basis function neural network to compensate for the actuator gain fault. In addition, an observer is used to handle composite disturbances, including unknown approximation errors and boundary disturbances. Then, an auxiliary system eliminates the effect of the input constraint. By integrating the composite disturbance observer and auxiliary system, adaptive fault-tolerant boundary control is achieved for an uncertain flexible string. Under rigorous Lyapunov stability analysis, the vibration scope of the flexible string is guaranteed to remain within a small compact set. Numerical simulations verify the high control performance of the proposed control scheme.

Blockchain has been utilized to manage distributed multicloud storage in the industrial Internet of Things. Existing approaches commonly use trusted third-party servers or middlewares to search data allocation strategies and use blockchain to enhance security. However, finding a fair data allocation strategy is hard when the third-party brokers are manipulated. Moreover, the complex computing in generating blocks reduces efficiency and heavy communication cost in consensus leads to critical challenges to scalability. To address that, this article proposes a scalable two-layer blockchain system for distributed multi-cloud storage (STSM). We design a novel consensus mechanism called proof of storage allocation, which integrates data placement problems into leader selection to achieve fair strategy and high QoS of data storage. We also incorporate asynchronous consensus groups into the consensus process to enhance scalability. Extensive experiments verify that STSM gains high scalability and increases efficiency while achieving high QoS in distributed multicloud data allocation.

Static event-triggering-based control problems have been investigated when implementing adaptive dynamic programming algorithms. The related triggering rules are only current state-dependent without considering previous values. This motivates our improvements. This article aims to provide an explicit formulation for dynamic event-triggering that guarantees asymptotic stability of the event-sampled nonzero-sum differential game system and desirable approximation of critic neural networks. This article first deduces the static triggering rule by processing the coupling terms of Hamilton-Jacobi equations, and then, Zeno-free behavior is realized by devising an exponential term. Subsequently, a novel dynamic-triggering rule is devised into the adaptive learning stage by defining a dynamic variable, which is mathematically characterized by a first-order filter. Moreover, mathematical proofs illustrate the system stability and the weight convergence. Theoretical analysis reveals the characteristics of dynamic rule and its relations with the static rules. Finally, a numerical example is presented to substantiate the established claims. The comparative simulation results confirm that both static and dynamic strategies can reduce the communication that arises in the control loops, while the latter undertakes less communication burden due to fewer triggered events.

Energy consumption is an important issue for resource-constrained wireless neural recording applications with limited data bandwidth. Compressed sensing (CS) is a promising framework for addressing this challenge because it can compress data in an energy-efficient way. Recent work has shown that deep neural networks (DNNs) can serve as valuable models for CS of neural action potentials (APs). However, these models typically require impractically large datasets and computational resources for training, and they do not easily generalize to novel circumstances. Here, we propose a new CS framework, termed APGen, for the reconstruction of APs in a training-free manner. It consists of a deep generative network and an analysis sparse regularizer. We validate our method on two in vivo datasets. Even without any training, APGen outperformed model-based and data-driven methods in terms of reconstruction accuracy, computational efficiency, and robustness to AP overlap and misalignment. The computational efficiency of APGen and its ability to perform without training make it an ideal candidate for long-term, resource-constrained, and large-scale wireless neural recording. It may also promote the development of real-time, naturalistic brain-computer interfaces.

This article presents a learning-based controller to solve the cooperative formation control problem for multiagent system (MAS) with collision avoidance. First, the consensus problem of first-order MAS is mostly solved by linear matrix inequality (LMI) without consideration of energy loss. To overcome these difficulties, an adaptive dynamic programming (ADP) technique is fit to solve the consensus problem and similar formation control problem for second-order MAS by the establishment of a performance index function. Besides, we introduce the generalized policy iteration (GPI) algorithm as a kind of ADP technique without the problem of low convergence speed and high computational complexity. Combined with previous works, it can be found that our proposed structure can be extended to high-order cases based on the structure of local neighborhood formation error and algorithm. Afterward, the convergence analysis, optimality analysis, and stability analysis are given. Neural networks (NNs) are also implemented to approximate the iterative control policies and value functions, respectively. Moreover, we realize that many collisions may occur in the formation control problem. Inspired by the idea of the artificial potential field (APF) technique, the concept of the repulsive force field is introduced based on our proposed learning-based structure to avoid collisions simply and efficiently. Finally, a simulation is provided to demonstrate the effectiveness of our proposed method.

For the practical control system, the controller is normally implemented on a digital platform with a time-triggered scheme. This scheme maybe produces redundant control and resources wasting, and hence, an event-triggered scheme is gradually favored. In this article, the robust learning control scheme is proposed aiming at a class of disturbed control systems, in which the system information is processed by a novel dynamic event communication. First, the robust optimal control problem with external disturbances is redescribed as a zero-sum differential game, and with integral reinforcement learning, a model-independent weight tuning law is devised for a critic neural network. Then, in order to further reduce the computational burden, an additional dynamic variable is put forward to incorporate the past triggering information. The application of a single-link joint arm system demonstrates that the proposed scheme can guarantee learning performance and robust control effect, along with larger triggering intervals. Finally, the load frequency control problem of single-area power system is studied. On one hand, the comparative results of five control schemes reveal that the dynamic event scheme can achieve the better frequency response at the lowest information transmission rate. On the other hand, the advantages of the proposed method are illustrated by comparing with other three event-triggered schemes.

This article investigates the optimal control problem for nonlinear nonzero-sum differential game in the environment of no initial admissible policies while considering the control constraint. An adaptive learning algorithm is thus developed based on policy iteration technique to approximately obtain the Nash equilibrium using real-time data. A two-player continuous-time system is used to present this approximate mechanism, which is implemented as a critic-actor architecture for every player. The constraint is incorporated into this optimization by introducing the nonquadratic value function, and the associated constrained Hamilton-Jacobi equation is derived. The critic neural network (NN) and actor NN are utilized to learn the value function and the optimal control policy, respectively, in the light of novel weight tuning laws. In order to tackle the stability during the learning phase, two stable operators are designed for two actors. The proposed algorithm is proved to be convergent as a Newton's iteration, and the stability of this closed-loop system is also ensured by Lyapunov analysis. Finally, two simulation examples demonstrate the effectiveness of the proposed learning scheme by considering different constraint scenes.

The extensive applications in the Internet of Things (IoT) have inspired a growing network scale. However, due to the resource-limited IoT devices and the numerous cyber attacks against applications, maintaining the robustness and communication capabilities for the applications is increasingly challenging. In this article, we consider IoT network topologies that provide robust communication for heterogeneous networks and study the networking stability of IoT devices and the intelligent evolution computing in network architectures. We explicate the network robustness problem both for the network architecture and the resistance to cyber attacks. For the network architecture, we optimize the robustness of IoT network topology with a scale-free network model which has good performance in random attacks. In the case with the resistance to cyber attacks, a deep deterministic learning policy (DDLP) algorithm is proposed to improve the stability for large-scale IoT applications. Simulations show that the proposed algorithms greatly advance the robustness of IoT network topology compared to other algorithms, with a less computational cost.

Differential games have been extensively applied to optimal control problems. Nash equilibrium captures the tradeoff among players' policies when every player independently tries to minimize a predefined index. When considering potential cooperation, Pareto equilibrium plays an important role in cooperative differential games. This article studies the cooperative control of multiplayer systems on the quadratic infinite horizon. First, by defining a joint cost function using a parameter set, a cooperative differential game is reformulated as a general optimal control problem, where all players form a grand coalition. Then, the joint cost function is approximated by a critic neural network, and for the first time, a novel adaptive dynamic programming algorithm with two learning stages is proposed to determine the parameter selection and then obtain Pareto optimal solutions. A numerical example demonstrates that this algorithm can achieve optimal policies and Pareto frontier. As for its application, the cooperative control of a two-area interconnected power system is investigated, where the primary frequency control and secondary frequency control are regarded as two players. Simulation results indicate that the proposed scheme can obtain binding cooperation agreements, such that cooperative control scheme can get better overall performance compared to Nash control method and another three control methods.

In this paper, a learning-based robust tracking control scheme is proposed for a quadrotor unmanned aerial vehicle system. The quadrotor dynamics are modeled including time-varying and coupling uncertainties. By designing position and attitude tracking error subsystems, the robust tracking control strategy is conducted by involving the approximately optimal control of associated nominal error subsystems. Furthermore, an improved weight updating rule is adopted, and neural networks are applied in the learning-based control scheme to get the approximately optimal control laws of the nominal error subsystems. The stability of tracking error subsystems with time-varying and coupling uncertainties is provided as the theoretical guarantee of learning-based robust tracking control scheme. Finally, considering the variable disturbances in the actual environment, three simulation cases are presented based on linear and nonlinear models of quadrotor with competitive results to demonstrate the effectiveness of the proposed control scheme.

Due to the rapidly expanding complexity of the cyber-physical power systems, the probability of a system malfunctioning and failing is increasing. Most of the existing works combining smart grid (SG) security and game theory fail to replicate the adversarial events in the simulated environment close to the real-life events. In this article, a repeated game is formulated to mimic the real-life interactions between the adversaries of the modern electric power system. The optimal action strategies for different environment settings are analyzed. The advantage of the repeated game is that the players can generate actions independent of the previous actions' history. The solution of the game is designed based on the reinforcement learning algorithm, which ensures the desired outcome in favor of the players. The outcome in favor of a player means achieving higher mixed strategy payoff compared to the other player. Different from the existing game-theoretic approaches, both the attacker and the defender participate actively in the game and learn the sequence of actions applying to the power transmission lines. In this game, we consider several factors (e.g., attack and defense costs, allocated budgets, and the players' strengths) that could affect the outcome of the game. These considerations make the game close to real-life events. To evaluate the game outcome, both players' utilities are compared, and they reflect how much power is lost due to the attacks and how much power is saved due to the defenses. The players' favorable outcome is achieved for different attack and defense strengths (probabilities). The IEEE 39 bus system is used here as the test benchmark. Learned attack and defense strategies are applied in a simulated power system environment (PowerWorld) to illustrate the postattack effects on the system.

In this paper, an approximately optimal control strategy is developed for the tracking control of a class of continuous-time nonlinear systems with unmatched uncertainties. By transforming the unmatched uncertain term, the auxiliary system associated with the uncertain nonlinear system is established. The auxiliary system is divided into steady and transient parts, and the related controllers are separately solved, meanwhile the transient tracking error system is also obtained by introducing the reference system. A neural network-based adaptive dynamic programming method is used to get the approximately optimal tracking control law of uncertain nonlinear systems with a predefined cost function. Furthermore, the ultimately uniform boundedness of neural network weights and the stability of tracking error systems are both proved through Lyapunov theory. Two cases of nonlinear systems with unmatched uncertainties are investigated to illustrate the effectiveness of the proposed robust tracking control strategy.

Driver fatigue evaluation is of great importance for traffic safety and many intricate factors would exacerbate the difficulty. In this paper, based on the spatial-temporal structure of multichannel electroencephalogram (EEG) signals, we develop a novel EEG-based spatial-temporal convolutional neural network (ESTCNN) to detect driver fatigue. First, we introduce the core block to extract temporal dependencies from EEG signals. Then, we employ dense layers to fuse spatial features and realize classification. The developed network could automatically learn valid features from EEG signals, which outperforms the classical two-step machine learning algorithms. Importantly, we carry out fatigue driving experiments to collect EEG signals from eight subjects being alert and fatigue states. Using 2800 samples under within-subject splitting, we compare the effectiveness of ESTCNN with eight competitive methods. The results indicate that ESTCNN fulfills a better classification accuracy of 97.37% than these compared methods. Furthermore, the spatial-temporal structure of this framework advantages in computational efficiency and reference time, which allows further implementations in the brain-computer interface online systems.

This paper presents a data-based finite-horizon optimal control approach for discrete-time nonlinear affine systems. The iterative adaptive dynamic programming (ADP) is used to approximately solve Hamilton-Jacobi-Bellman equation by minimizing the cost function in finite time. The idea is implemented with the heuristic dynamic programming (HDP) involved the model network, which makes the iterative control at the first step can be obtained without the system function, meanwhile the action network is used to obtain the approximate optimal control law and the critic network is utilized for approximating the optimal cost function. The convergence of the iterative ADP algorithm and the stability of the weight estimation errors based on the HDP structure are intensively analyzed. Finally, two simulation examples are provided to demonstrate the theoretical results and show the performance of the proposed method.

Increasingly advanced technology allows the monitoring of complex systems from a wide variety of perspectives. But the exploration of such systems from a multichannel sensor information viewpoint remains a complicated challenge of ongoing interest. In this paper, first, based on a well-designed double-layer distributed-sector conductance (DLDSC) sensor, systematic oil-water and gas-liquid two-phase flow experiments are carried out to capture abundant spatiotemporal flow information. Second, well flow parameter measurement performance of the DLDSC sensor is effectively validated from the perspective of normalized conductance. Third, a novel multiplex network-based model is presented to implement data mining and characterize the evolution of flow dynamics. The results demonstrate that the model is powerful for the exploration of the spatial flow behaviors from heterogeneity to randomness in the studied two-phase flows.

With the technology development of intelligent power generation and consumption, more and more electric vehicles (EVs), smart loads, and renewable energy are linked up to smart grids. In this paper, due to the great response ability of EVs, a large number of aggregate EVs are considered to improve the frequency response. From this point of view, for an EV-integrated smart grid with loads and renewable energy, a novel hierarchically adaptive control strategy is developed including both the primary controller and the EV controller when power mismatches happen. The sliding-mode primary controller is used to stabilize the frequency by operating power plants, while the EV controller is designed by using adaptive dynamic programming to deal with most of the frequency deviation. Simulation is validated on a benchmark smart grid by comparative studies to demonstrate the superior performance of hierarchically adaptive frequency control as well as the benefit of using EVs in the frequency regulation.

Due to the existence of dynamical uncertainties, it is important to pay attention to the robustness of nonlinear control systems, especially when designing adaptive critic control strategies. In this paper, based on the neural network learning component, the robust stabilization scheme of nonlinear systems with general uncertainties is developed. Through system transformation and employing adaptive critic technique, the approximate optimal controller of the nominal plant can be applied to accomplish robust stabilization for the original uncertain dynamics. The neural network weight vector is very convenient to initialize by virtue of the improved critic learning formulation. Under the action of the approximate optimal control law, the stability issues for the closed-loop form of nominal and uncertain plants are analyzed, respectively. Simulation illustrations via a typical nonlinear system and a practical power system are included to verify the control performance.

In this paper, based on the adaptive critic learning technique, the control for a class of unknown nonlinear dynamic systems is investigated by adopting a mixed data and event driven design approach. The nonlinear control problem is formulated as a two-player zero-sum differential game and the adaptive critic method is employed to cope with the data-based optimization. The novelty lies in that the data driven learning identifier is combined with the event driven design formulation, in order to develop the adaptive critic controller, thereby accomplishing the nonlinear control. The event driven optimal control law and the time driven worst case disturbance law are approximated by constructing and tuning a critic neural network. Applying the event driven feedback control, the closed-loop system is built with stability analysis. Simulation studies are conducted to verify the theoretical results and illustrate the control performance. It is significant to observe that the present research provides a new avenue of integrating data-based control and event-triggering mechanism into establishing advanced adaptive critic systems.

A second-order integral flux observer (SOIFO-FLL) with dc component and harmonics attenuation capability is proposed for PMSM sensorless control in this paper. Due to the nonzero integral initial value, parameter mismatch, converter nonlinearities and detection errors, the estimated rotor flux by the conventional methods, pure integrator and low-pass filter (LPF) suffer the dc component and harmonics. And a novel rotor flux estimation method, SOIFO-FLL is proposed on basis of the second-order generalized integrator (SOGI) structure. Combined with the frequency-locked loop (FLL), the dc and high-order harmonics of the estimated rotor flux can be removed by the SOIFO. Lastly, the rotor position are exacted accurately by a quadrature normalized phase locked loop (Q-PLL). Simulations and experimental results verify the excellent performance of the proposed SOIFO-FLL based rotor flux observation and sensorless control strategy.

Due to the uncertainty of power load demand and the stochastic power generation from renewable energy, frequency fluctuation becomes a major concern of power system, especially for a microgrid. In this paper, an improved proportional-integral (PI) controller based on the neural adaptive control method is proposed to deal with the load frequency control (LFC) problem in a microgrid with wind power. The designed neural adaptive control auxiliary controller is used to provide the adaptive supplement control signal to PI controller in a real-time manner. Simulation studies on a benchmark microgrid system are carried out between the proposed compound controller and traditional PI controller. The simulation results demonstrate the proposed method has a superior performance for stabilizing the frequency over the traditional PI control under disturbances from the load change and wind power.

In this paper, active disturbance rejection control is designed for position/force control of a holonomic constrained mobile manipulator in presence of uncertainties and disturbances. A dynamic model of the mobile manipulator is derived, based on the Lagrange formulation. The basic idea of the proposed control design is to estimate the unknown internal dynamics of the mobile manipulator and external disturbances by the linear extended state observer and to actively compensate them by the control input. The stability of the proposed control system is analyzed. Simulation results validate the effectiveness and strong robustness of the proposed control system.

In this paper, coordinated sliding mode control (SMC) and adaptive dynamic programming (ADP) strategy is proposed for near-space aerospace vehicle (NSASV) adaptive attitude tracking control. In this design, the NSASV attitude angle control is implemented as classical cascade control scheme with two control loops in the model. The outer one is a slow control loop for the attitude angle tracking, and the inner one is a fast control loop for the attitude angular rate tracking. Both of these two control loops are designed by using SMC, which can provide exact control performance near the operating point. To improve the control performance and robustness under parameter variations and external disturbances, ADP based supplementary control is introduced and incorporated into the inner fast control loop to provide adaptive compensation for the reference signal. Simulation study is carried out in Matlab/Simulink environment, and the results demonstrate that the proposed cooperative control could provide quite satisfied tracking performance in terms of overshoot and oscillation.

A data-driven adaptive tracking control approach is proposed for a class of continuous-time nonlinear systems using a recent developed goal representation heuristic dynamic programming (GrHDP) architecture. The major focus of this paper is on designing a multivariable tracking scheme, including the filter-based action network (FAN) architecture, and the stability analysis in continuous-time fashion. In this design, the FAN is used to observe the system function, and then generates the corresponding control action together with the reference signals. The goal network will provide an internal reward signal adaptively based on the current system states and the control action. This internal reward signal is assigned as the input for the critic network, which approximates the cost function over time. We demonstrate its improved tracking performance in comparison with the existing heuristic dynamic programming (HDP) approach under the same parameter and environment settings. The simulation results of the multivariable tracking control on two examples have been presented to show that the proposed scheme can achieve better control in terms of learning speed and overall performance.

In this paper, we propose a data-driven supplementary control approach with adaptive learning capability for air-breathing hypersonic vehicle tracking control based on action-dependent heuristic dynamic programming (ADHDP). The control action is generated by the combination of sliding mode control (SMC) and the ADHDP controller to track the desired velocity and the desired altitude. In particular, the ADHDP controller observes the differences between the actual velocity/altitude and the desired velocity/altitude, and then provides a supplementary control action accordingly. The ADHDP controller does not rely on the accurate mathematical model function and is data driven. Meanwhile, it is capable to adjust its parameters online over time under various working conditions, which is very suitable for hypersonic vehicle system with parameter uncertainties and disturbances. We verify the adaptive supplementary control approach versus the traditional SMC in the cruising flight, and provide three simulation studies to illustrate the improved performance with the proposed approach.

In this paper, we construct an event-driven adaptive robust control approach for continuous-time uncertain nonlinear systems through a neural dynamic programming (NDP) strategy. Through system transformation and theoretical analysis, the robustness of the original uncertain system can be achieved by designing an event-driven optimal controller with respect to the nominal system under a suitable triggering condition. In addition, it is also observed that the event-driven controller has a certain degree of gain margin. Then, the NDP technique is employed to perform the main controller design task, followed by the uniform ultimate boundedness stability proof with the feedback action of the event-driven adaptive control law. The comparative effect of the present control strategy is also illustrated via two simulation examples. The established method provides a new avenue of combining adaptive dynamic programming-based self-learning control, event-triggered adaptive control, and robust control, to investigate the nonlinear adaptive robust feedback design under uncertain environment.

This paper focuses on establishing an event-based constrained robust control strategy for a class of continuous-time affine nonlinear systems by incorporating the adaptive critic mechanism (ACM). The main objective is to integrate the event-based framework, the constrained optimal control method, and the neural network learning ability, thereby achieving the nonlinear robust state feedback of input-constrained nonlinear systems under event-based environment. Through theoretical analysis, it is shown that the nonlinear robust control law subject to input limitations can be obtained by designing an event-based constrained optimal controller with respect to the nominal system. Then, the ACM is adopted to facilitate the constrained optimal control implementation, where a critic neural network is constructed to serve as the learning approximator. The system stability issue is proved by employing the Lyapunov theory and the constrained robust control performance is illustrated through simulation experiments of several dynamical plants.

The drive performance of permanent magnet synchronous motor (PMSM) can be deteriorated due to various disturbances. In this paper, the problem of speed control for a PMSM system with parameter uncertainties is investigated. A new control algorithm based on nonsingular terminal sliding mode control (NTSMC) is proposed, where the controller is developed for speed regulation. Compared with conventional strategies, this new controller provides improved performance for speed regulation of PMSM when subject to parameter uncertainties, in that it achieves fast dynamic response and strong robustness. Simulation studies are conducted to verify the effectiveness of this proposed method.

In this paper, we propose a cascade dictionary learning algorithm for action recognition. In the first stage, a dictionary for basic sparse coding is learned based on local descriptors. And then spatial pyramid features are extracted to represent all the images in the same dimensions. Instead of performing dimension reduction, all the features are regrouped and then fed into second dictionary learning. In the second stage, a supervised dictionary for block and group sparse coding is learned to get discriminative representations based on the regrouped features. Without lowering classification performance, the size of the second dictionary is much smaller than other dictionary based on spatial pyramid features. We evaluate our algorithm on two publicly available databases about action recognition: Willows and People Playing Music Instrument. The numerical results show the effectiveness of the proposed algorithm.

This paper mainly studies the speed control for a permanent magnet synchronous motor system. The relationship between the reference quadrature axis current and the speed output is approximately considered as a second-order model. Based on this second-order model, a composite control strategy is adopted, where a continuous sliding mode controller is designed for the speed regulation without chattering and a disturbance observer is introduced as a compensator to resist disturbances and to reduce control gains. Simulation results have been presented to illustrate that the proposed method has good responses to reference speed signals with torque load disturbances.

The algorithm of least square support vector machine (LSSVM) based on fuzzy c-means (FCM) clustering is presented in this paper, which can select the number of clusters automatically depending on different parameters and samples. We adopt the method to identify the inverse system with crucial spanless process variables and the inenarrable nonlinear character. In the course of identification, we construct the allied inverse system by the left inverse soft-sensing function and the right inverse system, then utilize the proposed method to approach the nonlinear allied inverse system via offline training. Simulation experiments are performed and indicate that the proposed method is effective and provides satisfactory performance with excellent accuracy and low computational cost comparing with the conventional method using LSSVM.