Designing a safe policy for uncertain environments is crucial in real-world control systems. However, this challenge remains inadequately addressed within the Markov decision process (MDP) framework. This paper presents the first algorithm guaranteed to identify a near-optimal policy in a robust constrained MDP (RCMDP), where an optimal policy minimizes cumulative cost while satisfying constraints in the worst-case scenario across a set of environments. We first prove that the conventional policy gradient approach to the Lagrangian max-min formulation can become trapped in suboptimal solutions. This occurs when its inner minimization encounters a sum of conflicting gradients from the objective and constraint functions. To address this, we leverage the epigraph form of the RCMDP problem, which resolves the conflict by selecting a single gradient from either the objective or the constraints. Building on the epigraph form, we propose a bisection search algorithm with a policy gradient subroutine and prove that it identifies an $\varepsilon$-optimal policy in an RCMDP with $\widetilde{\mathcal{O}}(\varepsilon^{-4})$ robust policy evaluations.

We study the reinforcement learning (RL) problem in a constrained Markov decision process (CMDP), where an agent explores the environment to maximize the expected cumulative reward while satisfying a single constraint on the expected total utility value in every episode. While this problem is well understood in the tabular setting, theoretical results for function approximation remain scarce. This paper closes the gap by proposing an RL algorithm for linear CMDPs that achieves $\widetilde{\mathcal{O}}(\sqrt{K})$ regret with an episode-wise zero-violation guarantee. Furthermore, our method is computationally efficient, scaling polynomially with problem-dependent parameters while remaining independent of the state space size. Our results significantly improve upon recent linear CMDP algorithms, which either violate the constraint or incur exponential computational costs.

Optimal and safety-critical control are fundamental problems for stochastic systems, and are widely considered in real-world scenarios such as robotic manipulation and autonomous driving. In this paper, we consider the problem of efficiently finding optimal and safe control for high-dimensional systems. Specifically, we propose to use dimensionality reduction techniques from a comparison theorem for stochastic differential equations together with a generalizable physics-informed neural network to estimate the optimal value function and the safety probability of the system. The proposed framework results in substantial sample efficiency improvement compared to existing methods. We further develop an autoencoder-like neural network to automatically identify the low-dimensional features in the system to enhance the ease of design for system integration. We also provide experiments and quantitative analysis to validate the efficacy of the proposed method. Source code is available at https://github.com/jacobwang925/path-integral-PINN.

This paper discusses an application of finite-time stabilization methods to the control of quadcopters. The finite-time stabilization methods guarantee the finite-time convergence to the equilibrium state and exhibit robustness property with respect to disturbances and modeling errors. The application of the finite-time stabilization to the control of UAV is expected to be effective. This paper shows an application of a finite-time stabilization to quadcopters and investigates the effectiveness of the stabilization via the physics simulation based on ROS and Gazebo. The performance of the finite-time stabilization method is compared with those of an exponentially stabilizing feedback controller.

An observer is an important mechanism to estimate the state variables that cannot be directly measured. The paper proposes new observer design methods for nonlinear systems via Takagi-Sugeno model that can describe a general class of nonlinear systems. Extensive research on the observer problem for nonlinear systems has been carried out and the obtained design methods have shown effective results. However, there is still a way to improve the existing observer design methods. This paper investigates new observer design methods based on new observer design conditions. In order to obtain such new observer design conditions, a new multiple Lyapunov matrix method, which contains multiple integrals of the membership function, is introduced. Moreover, a generalized relaxation lemma is applied for further reduction of the conservatism on the observer design conditions. Finally, illustrative examples are shown to illustrate the comparison of our observer methods and others.

This paper is concerned with output feedback control design for a discrete-time Takagi-Sugeno fuzzy system where the premise variable is not necessarily measurable. The Takagi-Sugeno fuzzy system generally describes nonlinear systems by employing local linear system representations, and a fuzzy system to be considered in this paper describes even a wider class of nonlinear systems. For a fuzzy system, a design method of a stabilizing output feedback controller is proposed by employing a fuzzy multiple Lyapnunov functional and descriptor system approach. These techniques reduce the conservatism in stabilizing controller design conditions. Such conditions are given in terms of a set of linear matrix inequalities(LMIs), which are easily numerically solvable. In addition, a relaxation method is used to further reduce the conservatism in design conditions. A proposed controller can be of reduced-order, which is practically useful for large-scale systems. Finally, numerical examples are given to illustrate our nonlinear control design and to show the effectiveness over other existing results.

The paper is concerned with non-fragile control design via output feedback controller for uncertain fuzzy systems. In control design for physical systems, there are chances that malfunction in actuator happens and an exact value of the control input may not be applied. Hence, controller gain variations as well as uncertainty in the system parameters should be considered in the control design. For an uncertain fuzzy system, a design method of a non-fragile output feedback controller is proposed by introducing a new class of non-parallel distributed compensators(non-PDCs) where the integrals of the membership functions are involved. Non-PDC is a generalized controller of PDC, which is a traditional controller for fuzzy systems. A non-PDC non-fragile output feedback controller for uncertain fuzzy systems is obtained from new fuzzy multiple Lyapunov functions and its control design conditions are given in terms of a set of linear matrix inequalities(LMIs), which are easily numerically solvable. The descriptor system approach, which leads to relaxation in controller design conditions, is also employed. Finally, a numerical example is given to illustrate our nonlinear control design and to show the effectiveness over other existing results.

The paper discusses non-fragile output feedback control design Takagi-Sugeno fuzzy systems with uncertain parameters. In control design for physical systems, there are chances that malfunction in actuator and round-off error of the control gain calculation may occur. Hence, controller gain variations as well as uncertainty in the system parameters should be considered in the control design. For an uncertain fuzzy system, a design method of a non-fragile output feedback controller is proposed by introducing a new type of controllers where the integrals of the membership functions are involved. A non-parallel distributed compensator(Non-PDC) is a generalization of a parallel distributed compensator(PDC), which is a traditional controller for fuzzy systems, and is used for the control design. A non-fragile non-PDC output feedback controller for uncertain fuzzy systems is obtained from new fuzzy multiple Lyapunov functions and its control design conditions are given in terms of a set of linear matrix inequalities(LMIs), which are easily numerically solvable. The descriptor system formulation, which leads to relaxation in controller design conditions, is also employed. These approaches reduce the conservatism of the control design conditions. Finally, numerical examples are given to illustrate our nonlinear control design and to show the effectiveness over other existing results.

The paper is concerned with non-fragile control design via output feedback controller for uncertain systems. In control design for physical systems, there are chances that malfunction in actuator happens and an exact value of the control input may not be applied. Hence controller gain variations should be considered in the control design. Additionally, many results on the state feedback control design have been given in the literature. However, in the actual situation, the state of the system is not always measured and the output feedback control design is desired. In this paper, non-fragile guaranteed cost control via output feedback control for uncertain systems is considered and a control design of stabilizing controllers with guaranteed cost and robustness against uncertainties of system parameters and control gains is proposed. The robust stability analysis of the closed-loop system and controller design are given in terms of LMI conditions, which are less conservative than the existing results.

The paper is concerned with control design via static output feedback controller for fuzzy systems. Many results on the state feedback control design have been given in the literature. In the actual situation, however, the state of the system is rarely measured and the output feedback control design is desired. In this paper, guaranteed cost control via output feedback control for fuzzy systems is considered and a control design of stabilizing controllers with guaranteed cost is proposed. The stability analysis of the closed-loop system and controller design are given in terms of LMI conditions, which are less conservative than the existing results.

This paper introduces a novel image recognition method based on the neural oscillator network. In the present method, the dynamics of neurons is determined from neural coupling strengths relevant to the similarities in pixel levels, which have been used in the conventional image segmentation, as well as geometrical local features obtained from the Gabor filtering. Then the image regions under resultant neural synchronization are identified. It is shown that the accuracy of the present method, applied to the image recognition for pairs of identical and different alphabetical letters, is 76.9% and 62.2%, respectively. Moreover, the simple polygons are also correctly recognized by the present method.