A long term goal of Interactive Reinforcement Learning is to incorporate non-expert human feedback to solve complex tasks. State-of-the-art methods have approached this problem by mapping human information to reward and value signals to indicate preferences and then iterating over them to compute the necessary control policy. In this paper we argue for an alternate, more effective characterization of human feedback: Policy Shaping. We introduce Advise, a Bayesian approach that attempts to maximize the information gained from human feedback by utilizing it as direct labels on the policy. We compare Advise to state-of-the-art approaches and highlight scenarios where it outperforms them and importantly is robust to infrequent and inconsistent human feedback.

The ability to form object categories is an important milestone in human infant development. We propose a framework that allows a robot to form a unified object categorization from several interactions with objects. This framework is consistent with the principle that robot learning should be ultimately grounded in the robot's perceptual and behavioral repertoire. This paper builds upon our previous work by adding more exploratory behaviors (now 6 instead of 1) and by employing consensus clustering for finding a single, unified object categorization. The framework was tested on a container/non-container categorization task with 20 objects.

This paper describes an approach to interactive object categorization that couples exploratory behaviors and their resulting acoustic signatures to form object categories. The framework was tested with an upper-torso humanoid robot on a container/non-container categorization task. The robot used six exploratory behaviors (drop block, grasp, move, shake, flip, and drop object) and applied them to twenty objects. The results from this large-scale experimental study show that the robot was able to learn meaningful object categories using only acoustic information. The results also show that the quality of the categorization depends on the exploratory behavior used to derive it as some behaviors elicit more salient acoustic signatures than others.