This paper presents a novel Nesterov accelerated gradient-based fixed-time convergent actor-critic (NAG-FxT-AC) scheme for the optimal control of nonlinear systems. The proposed approach integrates the Nesterov accelerated gradient method with FxT concurrent learning to achieve rapid convergence while ensuring optimal performance. Actor-critic neural networks (NN) are employed to approximate the optimal value function and control policy, where the accelerated gradient mechanism introduces auxiliary variables to enhance learning efficiency. A FxT concurrent learning algorithm is developed to update the NN weights, guaranteeing convergence to bounded regions within fixed time independent of initial conditions. Lyapunov stability analysis proves that both the closed-loop system states and NN estimation errors are ultimately uniformly bounded with FxT NN weights convergence properties. Simulation results on a nonlinear system validate the effectiveness of the proposed control scheme. Compared to the baseline methods like FxT-ADP, the proposed NAG-FxT-AC reduces the state convergence time by up to 12% and the weight convergence time by 84%, demonstrating superior learning efficiency.

This work investigates the formation tracking issue for multirobot manipulator end-effectors under input constraints. A distributed formation control law is designed to guarantee the finite-time boundedness of tracking errors within the framework. To estimate the significant bias of dynamics discovered during practical multirobot collaborative manipulation tasks, a bias radial basis function neural network (RBFNN) is integrated, along with a designed adaptive updating law for expeditious approximation. In addition, an anti-windup compensator within a finite-time framework is specifically introduced to mitigate the input saturation issue arising from torque limitations in joint actuators. Finally, the system’s semi-global practical finite-time boundedness (SGPFTB) is rigorously established through Lyapunov theory. Five planar manipulators are employed in comparative computational experiments to validate the feasibility of the presented control strategy.

ABSTRACT Recently, routing problems have made significant progress and exhibited remarkable performance across various domains. However, they still suffer from severe issues, including high computational complexity and path intersection phenomenon, which curtail their broader applicability. This study focuses on the challenging and practical Single Depot‐Multiple Traveling Salesman Problem (SD‐MTSP) with a min‐max objective, which aims to minimize the maximum tour length among all salesmen. To tackle these challenges, we propose a Hierarchical Self Organizing Map (HiSOM) based on a divide‐and‐conquer framework to decompose the complex scheduling of SD‐MTSP into task allocation and route planning subproblems, with a strong emphasis on reducing computational complexity. Specifically, in the task allocation stage, we introduce the concept of soft labels to precisely characterize the strength of association between salesmen and their traversal cities, and devise an SGD‐SOM framework to optimize the sum square error with smooth gradient descents. In the route planning stage, we design a TOSOM framework to generate a topologically ordered tour with minimal length, ensuring strict adherence to the convex hull property and effectively mitigating path intersection. Comprehensive experiments on both synthetic and real‐world datasets demonstrate that our proposed HiSOM outperforms numerous baseline methods by up to 6.91%.

This paper proposes a hierarchical safe reinforcement learning with prescribed performance control (HSRL-PPC) scheme to address the challenges of interconnected leader-follower systems operating in complex environments. The framework consists of two levels: at the higher level, the leader agent detects and avoids moving obstacles while planning optimal paths; at the lower level, the follower agent tracks the leader within strict prescribed performance bounds. We formulate the optimal prescribed performance safe control problem and solve it using the Hamilton-Jacobi-Bellman (HJB) equation. Due to system nonlinearity and obstacle complexity, we approximate the leader’s optimal value function using a state-following neural network that efficiently extrapolates training data to neighboring states, while employing a regular critic neural network for the follower’s value function approximation. Lyapunov stability analysis demonstrates the closed-loop system’s theoretical guarantees. Experimental results from two simulation examples and hardware tests with a quadcopter-vehicle system validate the effectiveness of the proposed approach in achieving safe navigation and precise tracking performance in dynamic environments. Note to Practitioners—Challenges exist in unpredictable obstacles and agent limitations for the interconnected leader-follower system. To provide a safe, efficient, and reliable control scheme, hierarchical safe reinforcement learning with prescribed performance control is proposed in this paper. The hierarchical structure is utilized to coordinate the leader and follower agents in the interconnected system, where the leader agent plans the optimal path and avoids obstacles, and the follower agent tracks the leader within prescribed performance bounds. Based on the proposed hierarchical structure, engineers can design efficient and safe control schemes for interconnected leader-follower systems with moving obstacles. In future work, we will address the problem of external disturbances and uncertainties in the interconnected leader-follower system.

Real-world applications of nonlinear systems tracking control are always challenging due to the existence of uncertainties and disturbances. To design a robust optimal tracking controller for uncertain nonlinear systems with disturbances and actuator saturation, this paper investigates the prescribed performance robust optimal tracking control problem. A prescribed performance mechanism is constructed to convert the dynamics of tracking error into transformed error dynamics, which keeps the system’s operating states within specific bounds, ensuring tracking with predefined error constraints. For the optimal tracking controller design, an optimal index is established to optimize the performance of tracking control, and a robust optimal index is established to optimize the disturbance effect on the tracking error. To achieve robust optimal tracking control that minimizes both optimal and robust optimal indexes, a Stackelberg game is constructed, which provides a hierarchical game structure for the optimal controller and the worst disturbance. The robust optimal controller is approximated online using reinforcement learning techniques. An actor-critic-identifier algorithm is designed to approximate the optimal value function, optimal controller, and drifted system parameters. Lyapunov theory is utilized to analyze the closed-loop system’s stability. To demonstrate the effectiveness of the proposed robust optimal control method, two numerical simulations and a hardware experiment on a quadcopter system are conducted. The experiment results demonstrate that our method successfully achieves prescribed performance tracking control when actuators are saturated and disturbances are present. Note to Practitioners—In this paper, the probelm of mixed$H_{2}/H_{\infty }$prescribed-performance optimal tracking control for nonlinear systems with input saturation is investigated. To constrain the operating states of the system within certain bounds, the prescribed performance transformation is designed to achieve tracking with predefined error constraints. For the optimal controller design, the$H_{2}$index is established to minimize the optimal tracking performance, and the$H_{\infty }$index is designed to minimize the disturbance effect on the tracking error. A Stackelberg-based non-zero sum game between the optimal controller and the worst disturbance is established to design the mixed$H_{2}/H_{\infty }$optimal tracking controller. The designed optimal controller is approximated online using reinforcement learning. Effectiveness of the proposed method is demonstrated by two numerical simulations and a hardware experiment on a quadcopter system. Based on the proposed high-performance controller, engineers can design a high-performance robust optimal tracking controller for uncertain nonlinear systems with extreme conditions of disturbances and actuator saturation.

Multiagent formation tracking tasks in constrained space commonly require the formation to transform its pattern adaptively. Thus, the multiagent systems can effectively avoid spatial constraints and safely pass through the constrained region. Aiming to achieve such control objectives, a formation tracking control strategy based on the leader-following framework is designed in this article. An adaptive formation scaling mechanism called orientational scaling is first designed. A time-varying matrix introduces real-time information about the formation shape and obstacles into the formation tracking controller. With the action of the controller, the formation can perform an orientational scaling action on its pattern according to the constraints of the external space during the tracking of a given trajectory. Besides, a spatial receding control mechanism is also designed to handle possible collisions in scaling transformations in multiagent systems, reducing the collision risk and improving movement efficiency. The stability conditions of the system are given by adopting the Lyapunov stability theory. Finally, the effectiveness of the designed control strategy is illustrated by simulation results.