We want robots to complete assigned tasks even when unexpected task pressures arise, either from the robot or the environment. This paper presents a method of both learning sources of task failure in situ and rapidly planning new motions on-the-fly to accommodate them. This “risk-adaptive” approach to robot control uses a few encounters with a novel failure mode to generate a probabilistic failure model which we use to optimize a risk-mitigating motion plan. We demonstrate two toy problems, where risk-adaptive double-integrator agents are introduced to separate environments, each with their own tasks and modes of failure. The agents are not aware a priori of any risks the environments might present, but after one failure, the agents quickly adapt their motion plans and ensure task completion. We further conduct numerical experiments to characterize the algorithm's speed of adaptation with respect to environmental uncertainty. We see this framework as a natural extension for the myriad of robotic applications using model-based motion planners.