Decision making under uncertainty is a challenging task, especially when dealing with complex robotics scenarios. The Partially Observable Markov Decision Process (POMDP) framework, designed to solve this problem, was subject to much work lately. Most POMDP solvers, however, focus on planning in discrete state, action and/or observations spaces, which does not truly reflect the complexity of most real world problems. This paper addresses the issue by devising a method for solving POMDPs with continuous state, action and observations spaces. The proposed planner, Continuous Belief Tree Search (CBTS), uses Bayesian Optimisation (BO) to dynamically sample promising actions while constructing a belief tree. This dynamic sampling allows for richer action selection than offline action discretisation. CBTS is complemented by a novel trajectory generation technique, relying on the theory of Reproducing Kernel Hilbert Spaces (RKHS), yielding trajectories amenable for robotics applications. The resulting trajectory planner kCBTS outperforms other continuous planners on space modelling and robot parking problems.

Bayesian Optimization has gained much popularity lately, as a global optimization technique for functions that are expensive to evaluate or unknown a priori. While classical BO focuses on where to gather an observation next, it does not take into account practical constraints for a robotic system such as where it is physically possible to gather samples from, nor the sequential nature of the problem while executing a trajectory. In field robotics and other real-life situations, physical and trajectory constraints are inherent problems. This paper addresses these issues by formulating Bayesian Optimization for continuous trajectories within a Partially observable Markov Decision Process (POMDP) framework. The resulting POMDP is solved using Monte-Carlo Tree Search (MCTS), which we adapt to using a reward function balancing exploration and exploitation. Experiments on monitoring a spatial phenomenon with a UAV illustrate how our BO-POMDP algorithm outperforms competing techniques.

Environmental monitoring with mobile robots requires solving the informative path planning problem. A key challenge is how to compute a continuous path over space and time that will allow a robot to best sample the environment for an initially unknown phenomenon. To address this problem we devise a layered Bayesian Optimisation approach that uses two Gaussian Processes, one to model the phenomenon and the other to model the quality of selected paths. By using different acquisition functions over both models we tackle the exploration-exploitation trade off in a principled manner. Our method optimises sampling over continuous paths and allows us to find trajectories that maximise the reward over the path. We test our method on a large scale experiment for modelling ozone concentration in the US, and on a mobile robot modelling the changes in luminosity. Comparisons are presented against information based criteria and point-based strategies demonstrating the benefits of our method.

A key challenge for long-term autonomy is to enable a robot to automatically model properties of the environment while actively searching for better decisions to accomplish its task. This amounts to the problem of exploration-exploitation in the context of active perception. This paper addresses active perception and presents a technique to incrementally model the roughness of the terrain a robot navigates on while actively searching for waypoints that reduce the overall vibration experienced during travel. The approach employs Gaussian processes in conjunction with Bayesian optimisation for decision making. The algorithms are executed in real-time on the robot while it explores the environment. We present experiments with an outdoor vehicle navigating over several types of terrains demonstrating the properties and effectiveness of the approach.

Environmental Monitoring (EM) is typically performed using sensor networks that collect measurements in predefined static locations. The possibility of having one or more autonomous robots to perform this task increases versatility and reduces the number of necessary sensor nodes to cover the same area. However, several problems arise when making use of autonomous moving robots for EM. The main challenges are how to build an accurate spatial-temporal model while choosing locations for measuring the phenomenon. This paper addresses the problem by using Bayesian Optimisation for choosing sensing locations, and presents a new utility function that takes into account the distance travelled by a moving robot. The proposed methodology is tested in simulation and in a real environment. Compared to existing strategies, our approach exhibits slightly better accuracy in terms of RMSE error and considerably reduces the total distance travelled by the robot.