This paper presents a model-free reinforcement learning (RL) algorithm to synthesize a control policy that maximizes the satisfaction probability of complex tasks, which are expressed by linear temporal logic (LTL) specifications. Due to the consideration of environment and motion uncertainties, we model the robot motion as a probabilistic labeled Markov decision process (PL-MDP) with unknown transition probabilities and probabilistic labeling functions. The LTL task specification is converted to a limit deterministic generalized Büchi automaton (LDGBA) with several accepting sets to maintain dense rewards during learning. The novelty of applying LDGBA is to construct an embedded LDGBA (E-LDGBA) by designing a synchronous tracking-frontier function, which enables the record of non-visited accepting sets of LDGBA at each round of the repeated visiting pattern, to overcome the difficulties of directly applying conventional LDGBA. With appropriate dependent reward and discount functions, rigorous analysis shows that any method, which optimizes the expected discount return of the RL-based approach, is guaranteed to find the optimal policy to maximize the satisfaction probability of the LTL specifications. A model-free RL-based motion planning strategy is developed to generate the optimal policy in this paper. The effectiveness of the RL-based control synthesis is demonstrated via simulation and experimental results.

Most multiple‐flow‐direction algorithms (MFDs) use a flow‐partition coefficient (exponent) to determine the fractions draining to all downslope neighbours. The commonly used MFD often employs a fixed exponent over an entire watershed. The fixed coefficient strategy cannot effectively model the impact of local terrain conditions on the dispersion of local flow. This paper addresses this problem based on the idea that dispersion of local flow varies over space due to the spatial variation of local terrain conditions. Thus, the flow‐partition exponent of an MFD should also vary over space. We present an adaptive approach for determining the flow‐partition exponent based on local topographic attribute which controls local flow partitioning. In our approach, the influence of local terrain on flow partition is modelled by a flow‐partition function which is based on local maximum downslope gradient (we refer to this approach as MFD based on maximum downslope gradient, MFD‐md for short). With this new approach, a steep terrain which induces a convergent flow condition can be modelled using a large value for the flow‐partition exponent. Similarly, a gentle terrain can be modelled using a small value for the flow‐partition exponent. MFD‐md is quantitatively evaluated using four types of mathematical surfaces and their theoretical ‘true’ value of Specific Catchment Area (SCA). The Root Mean Square Error (RMSE) shows that the error of SCA computed by MFD‐md is lower than that of SCA computed by the widely used SFD and MFD algorithms. Application of the new approach using a real DEM of a watershed in Northeast China shows that the flow accumulation computed by MFD‐md is better adapted to terrain conditions based on visual judgement.