Reinforcement learning (RL) based flying ad-hoc network (FANET) routing enables unmanned aerial vehicles (UAVs) to choose the next-hop to increase the packet delivery ratio, but the routing latency and energy consumption have to be further reduced over inaccurate feedback for large-scale networks. In this paper, we propose an RL based energy-efficient fast routing for each UAV to choose the forwarding decision and the power. Based on the state consisting of the battery level, channel conditions and forwarding decisions of the one-hop neighbors, the routing policy is chosen to enhance the utility as the weighted sum of the delivery success indicator, the latency and the energy consumption. The number of the latency violations and the learning parameters shared among the one-hop neighbors are exploited in the update of the routing policy distribution following the latency constraint with the reduced energy consumption. The deep neural networks address the state quantization error of the latency and the channel gain for UAVs with high mobility under large-scale networks. The performance bound regarding the end-to-end latency and the energy consumption is derived in terms of network topology and channel gain based on the packet forwarding game. The performance gain over the benchmark is provided via both simulation and experimental results.

Reinforcement learning (RL) enables flying ad-hoc networks (FANETs) to choose the next hop unmanned aerial vehicles (UAV s) with shorter routing path, but may raise the retransmission rate and fails to guarantee the quality of service (QoS) under the high mobility and fast fading channels. In this paper, we propose an RL based routing scheme that optimizes both the routing and the power allocation to protect the latency QoS and save routing energy consumption of the FANET. Based on the routing history, the channel conditions, the battery level and the shared knowledge from the neighbors, this scheme formulates the routing policy distribution with safe exploration to select the stable path and thus reduce the retransmission rate. Specifically, the risk value with respect to end-to-end latency constraint is designed to evaluate the routing policy and reduce the exploration probability of the high-latency routing. Based on the distributed value function approach, the learning parameter such as the state value functions shared among neighbors is exploited to accelerate the routing process and enhance the routing stability under the dynamic network topology. Simulation results verify the routing performance gain of our proposed scheme over the benchmark.