This brief presents the adaptive optimal prescribed performance tracking solutions for the multiplayer nonlinear systems based on the adaptive critic learning scheme, where the tracking errors are constrained to a predefined bounded set. First, the general optimal tracking solutions of multiplayer nonlinear systems are presented. Every optimal tracking solution of multiple players consists of the steady-state part and the adaptive feedback part. The steady-state part can be obtained directly according to the tracking signal and system dynamics. Then, the adaptive feedback part can be studied with the prescribed performance constraints and adaptive critic learning such that multiple value functions achieve a Nash equilibrium with error constraints. Moreover, the convergence of the critic network weight is analyzed by the Lyapunov algorithm. Finally, simulation results and experiments are presented to demonstrate the satisfactory performance of the proposed method.

Multi-modal object Re-IDentification (ReID) aims to retrieve specific objects by utilizing complementary information from various modalities. However, existing methods focus on fusing heterogeneous visual features, neglecting the potential benefits of text-based semantic information. To address this issue, we first construct three text-enhanced multi-modal object ReID benchmarks. To be specific, we propose a standardized multi-modal caption generation pipeline for structured and concise text annotations with Multi-modal Large Language Models (MLLMs). Besides, current methods often directly aggregate multi-modal information without selecting representative local features, leading to redundancy and high complexity. To address the above issues, we introduce IDEA, a novel feature learning framework comprising the Inverted Multi-modal Feature Extractor (IMFE) and Cooperative Deformable Aggregation (CDA). The IMFE utilizes Modal Prefixes and an InverseNet to integrate multi-modal information with semantic guidance from inverted text. The CDA adaptively generates sampling positions, enabling the model to focus on the interplay between global features and discriminative local features. With the constructed benchmarks and the proposed modules, our framework can generate more robust multi-modal features under complex scenarios. Extensive experiments on three multi-modal object ReID benchmarks demonstrate the effectiveness of our proposed method.

This paper proposes an adaptive finite-time sliding optimal tracking control (AFSOTC) for uncertain linear systems by combining finite-time optimal tracking with integral sliding mode technology. The AFSOTC includes two components: an optimal finite-time feedback control based on the finite-time control technique for a nominal model, and an integral finite-time sliding control to attenuate external disturbances. First, a sliding model is constructed to transform the tracking problem into a regular control one. The finite-time optimal controller based on the fixed-time stability technique and adaptive dynamic programming is proposed for the uncertain sliding model system, where an improved finite-time algebraic-Riccati-equation (ARE) with fixed-time convergence is online solved by a novel adaptive law. Then, an integral finite-time sliding control is constructed to resist the uncertain dynamics and interferences. Moreover, Lyapunov theory is applied to prove the parameter convergence and the stability of the closed-loop system with the AFSOTC. Finally, a linear system with comparisons and an autonomous underwater vehicle (AUV) with dynamic uncertainties and disturbance are applied to verify the correctness of the proposed method, and a linearization method for AUV underwater is presented.

In this paper, a novel adaptive integral reinforcement learning (AIRL) is utilized to online handle the finite-horizon optimum control policies of the partially unknown multi-input nonlinear system. Firstly, the concept of Nash equilibrium is introduced to make the multiple cost functions reach the saddle point. Then, dual neural networks (NNs) are applied to approach the performance index functions based on the integral reinforcement signal. Simultaneously, two novel learning algorithms are proposed to update the NN weights, in which the convergence of weights is proved. Then, the optimal strategies can be obtained by using the obtained weights. The designed controllers based on the data-driven AIRL scheme can avoid the internal state of the system and the derivatives of NN activations in the weight learning process. Finally, the stability of the controlled system is analyzed. An F-16 aircraft model and another nonlinear system are utilized to prove the validity and rationality of the algorithm.Note to Practitioners—There exist many multi-input systems in practical engineering, which includes multi-engine driven F-16 aircraft, large radar servo system and large artillery systems, etc. The finite-horizon optimal control of these systems is crucial for the better performance of system state. However, an accurate engineering model is difficult to obtain, and the finite-horizon cannot generally be achieved. To address these issues, this paper proposes the finite-horizon optimal control for these systems based on a novel adaptive integral reinforcement learning (AIRL). The AIRL can realize an optimal performance for these multi-input systems in finite-time without internal system dynamics, which is a good development for the multi-input system in practical engineering.

Conventional ACC method has great fluctuation and deviation when solving speed and distance control problems. Thus, this paper develops a reinforcement learning (RL) based robust optimal prescribed performance controller for ACC systems. To this end, we first construct a continuous time ACC system with unknown system dynamics (e.g., target vehicle acceleration, sensor and actuator attacks, etc). To estimate the unknown system dynamics, an unknown system dynamic estimator (USDE) is designed, where the unknown system dynamic can be accurately estimated by using the input-output information, this is helpful for controller design. Then, a RL based optimal control method is developed, where the prescribed performance function (PPF) is applied, the system states can be effectively defined within a certain range. To realize the online solution for optimal control, we design a new adaptive law based on the adaptive dynamic programming (ADP) framework to online learn the critic neural network (NN) weights, because of the strong convergence, the proposed learning algorithm can be effectively applied in practical industrial systems. Finally, the efficacy of the proposed control technique is tested through simulations and experiments.

This paper provides an inverse-model-based iterative learning control (ILC) for the unknown multi-input multi-output (MIMO) nonlinear system with neural network (NN), where a novel gradient adaptive law is used to update the NN weights both hidden and output layers such a faster convergence can be achieved. First, a three-layer NN structure is introduced to observe the MIMO nonlinear system with input–output data, and a new gradient algorithm is proposed to update the unknown parameters of both hidden and output layers. Then, the input dynamic can be obtained with the NN observer, and the inversion-model-based control is designed. Moreover, the ideal inversion control can be obtained based on the reference signal, and the inverse ILC is designed. The stability of the NN observer and the convergence of the inverse-model-based control are analyzed. Finally, a SCARA manipulator MIMO model is simulated to illustrate the correctness of the proposed methods.

This article proposes an integral${Q}$-learning scheme to study the optimum management strategy of the battery for energy trading in the microgrid. The obtained optimal strategies are managed to minimize the cost of the microgrid, and simultaneously guarantee the better performance of the battery such that service life of battery is extended. First, the microgrid model is constructed, where renewable energy and profiled loads are considered. To satisfy the microgrid demand and the environmental energy consumption, an integral${Q}$-learning scheme will be developed for the input policy of the microgrid energy system such that the model dynamics are avoided. Moreover, the minimized cost and the property of the battery are considered in the performance function. Rather than the traditional${Q}$-learning iterative scheme, this article proposes a self-learning architecture for the integral-${Q}$learning scheme with two neural networks. The first network can learn the optimal${Q}$-function. Another network is used to learn the optimal action such that the${Q}$-function is minimized and load demand can be satisfied. The network weights are updated with the gradient method and the stability analysis is presented. Finally, the experiment data is used to verify the proposed integral${Q}$-learning scheme.

This article studies the multi- [Formula: see text] controls for the input-interference nonlinear systems via adaptive dynamic programming (ADP) method, which allows for multiple inputs to have the individual selfish component of the strategy to resist weighted interference. In this line, the ADP scheme is used to learn the Nash-optimization solutions of the input-interference nonlinear system such that multiple [Formula: see text] performance indices can reach the defined Nash equilibrium. First, the input-interference nonlinear system is given and the Nash equilibrium is defined. An adaptive neural network (NN) observer is introduced to identify the input-interference nonlinear dynamics. Then, the critic NNs are used to learn the multiple [Formula: see text] performance indices. A novel adaptive law is designed to update the critic NN weights by minimizing the Hamiltonian-Jacobi-Isaacs (HJI) equation, which can be used to directly calculate the multi- [Formula: see text] controls effectively by using input-output data such that the actor structure is avoided. Moreover, the control system stability and updated parameter convergence are proved. Finally, two numerical examples are simulated to verify the proposed ADP scheme for the input-interference nonlinear system.

This article presents and practically validates an identifier-critic-based approximate dynamic programming (ADP) method to online address the optimal tracking control problem for nonlinear continuous-time unknown systems. The imposed assumption on precisely known system dynamics is obviated via a neural network (NN) identifier. A static control is first adopted to retain the steady-state tracking response, while an optimal control derived via the ADP method is proposed to regulate the tracking error by minimizing a cost function. A critic NN is then trained online to obtain the solution of the associated Hamilton–Jacobi–Bellman (HJB) equation. The learning of the identifier NN and critic NN is performed online simultaneously by tailoring a novel adaptation method, which can guarantee the convergence of the estimated NN weights. Consequently, the critic NN can be used to construct the optimal control policy directly, such that the actor NN used in the previous ADP schemes is avoided. Simulations are performed to verify the suggested control, and experiments on a helicopter plant are carried out to show its feasibility and improved control response.

Inspired by Nash game theory, a multiplayer mixed-zero-sum (MZS) nonlinear game considering both two situations [zero-sum and nonzero-sum (NZS) Nash games] is proposed in this paper. A synchronous reinforcement learning (RL) scheme based on the identifier-critic structure is developed to learn the Nash equilibrium solution of the proposed MZS game. First, the MZS game formulation is presented, where the performance indexes for players 1 to N - 1 and N NZS Nash game are presented, and another performance index for players N and N + 1 zero-sum game is presented, such that player N cooperates with players 1 to N - 1, while competes with player N + 1, which leads to a Nash equilibrium of all players. A single-layer neural network (NN) is then used to approximate the unknown dynamics of the nonlinear game system. Finally, an RL scheme based on NNs is developed to learn the optimal performance indexes, which can be used to produce the optimal control policy of every player such that Nash equilibrium can be obtained. Thus, the widely used actor NN in RL literature is not needed. To this end, a recently proposed adaptive law is used to estimate the unknown identifier coefficient vectors, and an improved adaptive law with the error performance index is further developed to update the critic coefficient vectors. Both linear and nonlinear simulations are presented to demonstrate the existence of Nash equilibrium for MZS game and performance of the proposed algorithm.

In this paper, an alternative solution for adaptive optimal tracking control of nonlinear completely unknown systems is proposed. Firstly, an adaptive identifier is used to estimate the unknown system dynamics. Then, a recently developed system augmentation approach is adopted to design the optimal control, where the reference signal is incorporated into the augmented system. Thus, both the feedforward control and feedback control can be obtained simultaneously. Then, a critic neural network (NN) is used to estimate the augmented performance index, and calculate the optimal control action. Thus, the widely used actor NN is not needed. Finally, a new adaptive law recently proposed by the authors is used to online update the NN weight. The closed-loop stability and the convergence of the optimal control are all proved. The feasibility of the suggested approach is demonstrated by a simulation example.