A growing concern with deploying Unmanned Aerial Vehicles (UAVs) in urban environments is the potential violation of human privacy, and the backlash this could entail. Therefore, there is a need for UAV path planning algorithms that minimize the likelihood of invading human privacy. We formulate the problem of human-aware path planning as a nonstationary Markov Decision Process, and provide a novel model-based reinforcement learning solution that leverages Gaussian process clustering. Our algorithm is flexible enough to accommodate changes in human population densities by employing Bayesian nonparametrics, and is real-time computable. The approach is validated experimentally on a large-scale long duration experiment with both simulated and real UAVs.