This paper presents a deep reinforcement learning (DRL) algorithm for orientation estimation using inertial sensors combined with a magnetometer. Lyapunov’s method in control theory is employed to prove the convergence of orientation estimation errors. The estimator gains and a Lyapunov function are parametrised by deep neural networks and learned from samples based on the theoretical results. The DRL estimator is compared with three well-known orientation estimation methods on both numerical simulations and real dataset collected from commercially available sensors. The results show that the proposed algorithm is superior for arbitrary estimation initialisation and can adapt to a drastic angular velocity profile for which other algorithms can be hardly applicable. To the best of our knowledge, this is the first DRL-based orientation estimation method with an estimation error boundedness guarantee.

Inertial measurement units are widely used in different fields to estimate the attitude. Many algorithms have been proposed to improve estimation performance. However, most of them still suffer from 1) inaccurate initial estimation, 2) inaccurate initial filter gain, and 3) non-Gaussian process and/or measurement noise. This paper will leverage reinforcement learning to compensate for the classical extended Kalman filter estimation, i.e., to learn the filter gain from the sensor measurements. We also analyse the convergence of the estimate error. The effectiveness of the proposed algorithm is validated on both simulated data and real data.

Auto-exploration is a task for self-driving robots to explore unknown environments, which becomes much complicated when they move on irregular outdoor terrains. To improve the situation, a new frontier-based exploration algorithm is presented in this paper. It starts from original 3D cloud points of the environment to analyze the traversability of the scanned area, and further provides a reachability map to mark all map grid cells as reachable, dangerous or unknown. Frontier candidates are obtained from the reachable map, then clustered and reduced using an improved K-means. Finally, the target of next exploration step is selected from the frontiers left by evaluating their travel cost. The algorithm is validated on an irregular outdoor terrain and shows the capability for a field robot to explore on an irregular terrain.