In this article, a fixed-time deep reinforcement learning (DRL) formation hunting control problem is investigated for a multi-marine surface vehicle (MSV) system. First, considering the lack of dynamic adaptability caused by the conventional deep neural network (DNN) framework, an online adaptive DNN method is proposed for the high-dimensional multi-MSV system. Second, a novel DRL framework is developed for designing fixed-time formation hunting controllers, which integrates the online adaptive DNNs method with the actor–critic-based reinforcement learning (RL) algorithm. Finally, a nonsmooth fixed-time stability analysis is established for the nonsmooth closed-loop system induced by the DRL-based structure, which rigorously demonstrates that all signals converge within a fixed-time interval independent of initial states. The simulation example demonstrates the practical viability of the presented scheme.

This paper investigates the event-triggered train formation control problem for multiple autonomous surface vehicles (ASVs) formation system in polar communication interference environment. Firstly, a distributed resilient guidance algorithm is introduced to generate the reference route based on waypoints. In the guidance algorithm, the distributed resilient leader predictor (RLP) is applied to obtain the states of ice-breaking ship when communication fails, and the resilient train formation scheme is designed to compute the reference signals for ASVs. Subsequently, an adaptive neural event-triggered train formation control algorithm is developed. In the control algorithm, the neural networks (NNs) are conducted to approximate model uncertainties, and event-triggered control (ETC) is employed to minimize controller updates. Furthermore, the threshold of the event-triggered mechanism (ETM) can be dynamically adjusted by states of system. It is proved that the formulated algorithm can ensure the prediction errors converge and multiple ASVs system is stable in polar communication interference environment. Finally, two simulation experiments are adopted to illustrate the effectiveness of the proposed algorithm.

In this article, an adaptive reinforcement learning (RL) control problem is explored for a class of nonstrict-feedback discrete-time systems. First, different from the existing results, considering the noncausal problem which may exist in the backstepping design procedure, a universal system transformation method is first proposed for a class of nonstrict-feedback discrete-time systems. Second, by defining a compensation term to compensate the controller and utilizing the property of radial-basis-function neural network (RBFNN), an RL-based direct adaptive control strategy is developed via a backstepping method to achieve optimal control, and the multigradient recursive (MGR) algorithm is employed to estimate the weight vector. Finally, the stability of the control system is guaranteed and all signals in the closed-loop system are semiglobal uniformly ultimately bounded (SGUUB) on the basis of the Lyapunov theory. In addition, a universal system transformation is first proposed which breaks through the limitations on the controller design for the discrete-time nonstrict-feedback nonlinear system by using the traditional method. The validity of this strategy is verified by two simulation examples that include a course keeping system of the marine vessel.

In this article, the tracking control problem of event-triggered multigradient recursive reinforcement learning is investigated for nonlinear multiagent systems (MASs). Attention is focused on the distributed reinforcement learning approach for MASs. The critic neural network (NN) is applied to estimate the long-term strategic utility function, and the actor NN is designed to approximate the uncertain dynamics in MASs. The multigradient recursive (MGR) strategy is tailored to learn the weight vector in NN, which eliminates the local optimal problem inherent in gradient descent method and decreases the dependence of initial value. Furthermore, reinforcement learning and event-triggered mechanism can improve the energy conservation of MASs by decreasing the amplitude of the controller signal and the controller update frequency, respectively. It is proved that all signals in MASs are semiglobal uniformly ultimately bounded (SGUUB) according to the Lyapunov theory. Simulation results are given to demonstrate the effectiveness of the proposed strategy.

This article investigates the model-free fault-tolerant containment control problem for multiagent systems (MASs) with time-varying actuator faults. Depending on the relative state information of neighbors, a distributed containment control method based on reinforcement learning (RL) is adopted to achieve containment control objective without prior knowledge on the system dynamics. First, based on the information of agent itself and its neighbors, a containment error system is established. Then, the optimal containment control problem is transformed into an optimal regulation problem for the containment error system. Furthermore, the RL-based policy iteration method is employed to deal with the corresponding optimal regulation problem, and the nominal controller is proposed for the original fault-free system. Based on the nominal controller, a fault-tolerant controller is further developed to compensate for the influence of actuator faults on MAS. Meanwhile, the uniform boundedness of the containment errors can be guaranteed by using the presented control scheme. Finally, numerical simulations are given to show the effectiveness and advantages of the proposed method.

In this article, the asynchronous fault detection (FD) strategy is investigated in frequency domain for nonlinear Markov jump systems under fading channels. In order to estimate the system dynamics and meet the fact that not all the running modes can be observed exactly, a set of asynchronous FD filters is proposed. By using statistical methods and the Lynapunov stability theory, the augmented system is shown to be stochastic stable with a prescribed$l_{2}$gain even under fading transmissions. Then, a novel lemma is developed to capture the finite frequency performance. Some solvable conditions with less conservatism are subsequently deduced by exploiting novel decoupling techniques and additional slack variables. Besides, the FD filter gains could be calculated with the aid of the derived conditions. Finally, the effectiveness of the proposed method is shown by an illustrative example.

This paper focuses on the design of a reduced adaptive fuzzy tracking controller for a class of high-order stochastic nonstrict feedback nonlinear systems with full-state constraints. In the proposed approach, reduced fuzzy systems are used to approximate uncertain functions which involve all state variables and a high-order tan-type barrier Lyapunov function (BLF) is considered to deal with full-state constraints of the controlled system. With this BLF and a combination of the reduced fuzzy control and adding a power integrator, a novel control scheme is constructed to ensure that tracking error is within a very small range of the origin almost surely, meanwhile, the constraints on the system states are not breached almost surely during the operation. Two examples are proposed to show the effectiveness of the design scheme.

This article investigates an adaptive reinforcement learning (RL) optimal control design problem for a class of nonstrict-feedback discrete-time systems. Based on the neural network (NN) approximating ability and RL control design technique, an adaptive backstepping RL optimal controller and a minimal learning parameter (MLP) adaptive RL optimal controller are developed by establishing a novel strategic utility function and introducing external function terms. It is proved that the proposed adaptive RL optimal controllers can guarantee that all signals in the closed-loop systems are semiglobal uniformly ultimately bounded (SGUUB). The main feature is that the proposed schemes can solve the optimal control problem that the previous literature cannot deal with. Furthermore, the proposed MPL adaptive optimal control scheme can reduce the number of adaptive laws, and thus the computational complexity is decreased. Finally, the simulation results illustrate the validity of the proposed optimal control schemes.

In this paper, an adaptive neural network (NN) control problem is investigated for discrete-time nonlinear systems with input saturation. Radial-basis-function (RBF) NNs, including critic NNs and action NNs, are employed to approximate the utility functions and system uncertainties, respectively. In the previous works, a gradient descent scheme is applied to update weight vectors, which may lead to local optimal problem. To circumvent this problem, a multigradient recursive (MGR) reinforcement learning scheme is proposed, which utilizes both the current gradient and the past gradients. As a consequence, the MGR scheme not only eliminates the local optimal problem but also guarantees faster convergence rate than the gradient descent scheme. Moreover, the constraint of actuator input saturation is considered. The closed-loop system stability is developed by using the Lyapunov stability theory, and it is proved that all the signals in the closed-loop system are semiglobal uniformly ultimately bounded (SGUUB). Finally, the effectiveness of the proposed approach is further validated via some simulation results.

In this paper, an adaptive backstepping controller based on nonlinear disturbance observer (DOB) is proposed for the nonlinear fin stabilizer system. DOB is responsible for disturbance rejection and uncertainty compensation, while the adaptive backstepping scheme is proposed for dealing with uncertain parameters of the fin stabilizer model. The designed controller guarantees uniform ultimate boundedness of all the signals in the closed-loop system and the tracking errors converge to a small neighborhood of the origin. The advantages of the proposed control scheme comprise that the DOB shorten the response time and observes the whole disturbances of the fin stabilizer system. Simulation example is presented to show the preciseness and robustness of the stabilization control realized by the proposed method.