This paper presents a model-free reinforcement learning (RL) algorithm to synthesize a control policy that maximizes the satisfaction probability of complex tasks, which are expressed by linear temporal logic (LTL) specifications. Due to the consideration of environment and motion uncertainties, we model the robot motion as a probabilistic labeled Markov decision process (PL-MDP) with unknown transition probabilities and probabilistic labeling functions. The LTL task specification is converted to a limit deterministic generalized Büchi automaton (LDGBA) with several accepting sets to maintain dense rewards during learning. The novelty of applying LDGBA is to construct an embedded LDGBA (E-LDGBA) by designing a synchronous tracking-frontier function, which enables the record of non-visited accepting sets of LDGBA at each round of the repeated visiting pattern, to overcome the difficulties of directly applying conventional LDGBA. With appropriate dependent reward and discount functions, rigorous analysis shows that any method, which optimizes the expected discount return of the RL-based approach, is guaranteed to find the optimal policy to maximize the satisfaction probability of the LTL specifications. A model-free RL-based motion planning strategy is developed to generate the optimal policy in this paper. The effectiveness of the RL-based control synthesis is demonstrated via simulation and experimental results.

In the field of high voltage and large capacity renewable energy grid connection system, electric machine variable frequency drive system, five-level converter has been received more and more attention. The five-level active neutral point clamped topology is evolved from the active neutral point clamped three-level structure and flying capacitor structure. It has simple structure and flexible control, which is the future development direction. For the three-phase five-level neutral point clamped grid-connected inverter, this paper proposes a spatial domain subdivision optimization based model predictive control strategy. With the control objectives of output current control, neutral point potential control, voltage control of suspended capacitor and switch state switching limitation, the small sector which the reference voltage vector is located, is determined in advance in the 60 degree angle coordinate system of space, and the voltage vector of the small sector is used for rolling optimization and iterative optimization of model predictive control, which greatly reduces the calculation time while achieving high-performance control. Simulation results verify the effectiveness of the proposed control algorithm.