Many prediction, decision-making, and control architectures rely on online learned Gaussian process (GP) models. However, most existing GP regression algorithms assume a single generative model, leading to poor predictive performance when the data are nonstationary, i.e., generated from multiple switching processes. Furthermore, existing methods for GP regression over nonstationary data require significant computation, do not come with provable guarantees on correctness and speed, and many only work in batch settings, making them ill-suited for real-time prediction. We present an efficient online GP framework, GP-non-Bayesian clustering (GP-NBC), which addresses these computational and theoretical issues, allowing for real-time changepoint detection and regression using GPs. Our empirical results on two real-world data sets and two synthetic data set show that GP-NBC outperforms state-of-the-art methods for nonstationary regression in terms of both regression error and computation. For example, it outperforms Dirichlet process GP clustering with Gibbs sampling by 98% in computation time reduction while the mean absolute error is comparable.

This paper derives sample complexity results for using Gaussian Processes (GPs) in both model-based and model-free reinforcement learning (RL). We show that GPs are KWIK learnable, proving for the first time that a model-based RL approach using GPs, GP-Rmax, is sample efficient (PAC-MDP). However, we then show that previous approaches to model-free RL using GPs take an exponential number of steps to find an optimal policy, and are therefore not sample efficient. The third and main contribution is the introduction of a model-free RL algorithm using GPs, DGPQ, which is sample efficient and, in contrast to model-based algorithms, capable of acting in real time, as demonstrated on a five-dimensional aircraft simulator.

A growing concern with deploying Unmanned Aerial Vehicles (UAVs) in urban environments is the potential violation of human privacy, and the backlash this could entail. Therefore, there is a need for UAV path planning algorithms that minimize the likelihood of invading human privacy. We formulate the problem of human-aware path planning as a nonstationary Markov Decision Process, and provide a novel model-based reinforcement learning solution that leverages Gaussian process clustering. Our algorithm is flexible enough to accommodate changes in human population densities by employing Bayesian nonparametrics, and is real-time computable. The approach is validated experimentally on a large-scale long duration experiment with both simulated and real UAVs.