In this paper, a bias-Lyapunov iteration method is proposed to solve the optimal control problem of unknown Markovian jump linear systems. By incorporating a bias parameter into the conventional Lyapunov iteration method, the proposed method eliminates initial admissible control requirements. A model-based theoretical framework is subsequently established, accompanied by a rigorous convergence proof for the modified iteration process. Subsequently, a data-driven version of bias-Lyapunov iteration is developed to learn an optimal control for Markovian jump linear systems with completely unknown dynamics. Simulation examples validate the efficacy and advantage of the proposed bias-Lyapunov iteration method.

This paper presents the bias-policy iteration, a modified adaptive dynamic programming method, to achieve optimal control design of discrete-time nonlinear systems. Firstly, the formulation of the bias-policy iteration method and the thorough convergence analysis are provided. By leveraging the bias parameter, the constraint of admissible control is relaxed while the fast convergence of traditional policy iteration is inherited. The actor-critic framework is utilized to realize the implementation of the proposed method accordingly. Finally, the proposed method is applied to optimal control problem of the inverted pendulum system. The simulation is conducted to verify the effectiveness of the bias-policy iteration approach.

This paper delves into the adaptive prescribed-time leader-following consensus control within a class of nonlinear networked multi-agent systems. Firstly, the nonlinear multi-agent network subjected to matched disturbances employs parameterization of the non-identical unknown nonlinear dynamics. Distributed bounded protocols leveraging parametric Lyapunov equation and adaptive laws are introduced, incorporating local consensus errors and relative state feedback. The proposed solution attains prescribed-time consensus, ensuring the boundedness of estimated parameters. Subsequently, building upon these findings, fully distributed adaptive bounded protocols for the nonlinear multi-agent networks in the lower triangular structure are presented. These fully distributed protocols rely solely on the relative states between the neighboring agents and do not necessitate information about the underlying communication topology to attain a prescribed-time consensus. Finally, the established results are substantiated through numerical examples, illustrating their effectiveness.

In this article, the λ -policy iteration ( λ -PI) method for the optimal control problem of discrete-time linear systems is reconsidered and restated from a novel aspect. First, the traditional λ -PI method is recalled, and some new properties of the traditional λ -PI are proposed. Based on these new properties, a modified λ -PI algorithm is introduced with its convergence proven. Compared with the existing results, the initial condition is further relaxed. The data-driven implementation is then constructed with a new matrix rank condition for verifying the feasibility of the proposed data-driven implementation. A simulation example verifies the effectiveness of the proposed method.