In a multi-agent setting, the optimal policy of a single agent is largely dependent on the behavior of other agents. We investigate the problem of multi-agent reinforcement learning, focusing on decentralized learning in non-stationary domains for mobile robot navigation. We identify a cause for the difficulty in training non-stationary policies: mutual adaptation to sub-optimal behaviors, and we use this to motivate a curriculum-based strategy for learning interactive policies. The curriculum has two stages. First, the agent leverages policy gradient algorithms to learn a policy that is capable of achieving multiple goals. Second, the agent learns a modifier policy to learn how to interact with other agents in a multi-agent setting. We evaluated our approach on both an autonomous driving lane-change domain and a robot navigation domain.

While mobile robots reliably perform each service task by accurately localizing and safely navigating avoiding obstacles, they do not respond in any other way to their surroundings. We can make the robots more responsive to their environment by equipping them with models of multiple tasks and a way to interrupt a specific task and switch to another task based on observations. However the challenges of a multiple task model approach include selecting a task model to execute based on observations and having a potentially large set of observations associated with the set of all individual task models. We present a novel two-step solution. First, our approach leverages the tasks' policies and an abstract representation of their states, and learns which task should be executed at each given world state. Secondly, the algorithm uses the learned tasks and identifies the observation stimuli that trigger the interruption of one task and the switch to another task. We show that our solution using the switching stimuli compares favorably to the naive approach of learning a combined model for all the tasks. Moreover, leveraging the stimuli significantly decreases the amount of sensory input processing during the execution of tasks.

The assumption that we can preprogram robots with all the information necessary for their function becomes impractical as the range of robotics applications grows. One widely proposed solution is the specification of reusable task plans, or recipes, consisting of a sequence of primitive actions. However, the primitive actions, such as unscrewing a nut in a car maintenance task, cannot always be predefined and thus must be learned for a particular robot platform and workspace. In this work, we present a novel algorithm that enables a robot to identify a reusable motion trajectory associated with each primitive action of a plan. Our approach segments the motion data captured during the demonstration of a human performing the given task, while also leveraging the human's verbal cues. We evaluated our algorithms in a pilot study with 6 users executing 90 primitive actions.

We present a novel algorithm to identify reusable motion trajectories corresponding to the primitive actions in a human demonstration of a symbolic plan with accompanying narration. Our approach involves a multi-step process starting with time-series pattern mining applied to raw motion-capture data. We evaluated our algorithm on recordings of human motions and showed that it identifies reusable trajectories with 86% of the accuracy of human experts.

We have developed learning and interaction algorithms to support a human teaching hierarchical task models to a robot using a single demonstration in the context of a mixed-initiative interaction with bi-directional communication. In particular, we have identified and implemented two important heuristics for suggesting task groupings based on the physical structure of the manipulated artifact and on the data flow between tasks. We have evaluated our algorithms with users in a simulated environment and shown both that the overall approach is usable and that the grouping suggestions significantly improve the learning and interaction.

Current approaches to learning partial ordering constraints by demonstration require demonstrating all (or almost all) possible completion orders. We have developed an algorithm that, for plans involving relative placement of objects, learns the partial ordering constraints from a single demonstration by letting the user specify naturally conceived reference frame information. This work is an example of a broader research agenda that involves applying principles of human collaboration to robot learning from demonstration.