This paper introduces a method, Generative Adversarial Networks for Detecting Erroneous Results (GANDER), leveraging Generative Adversarial Networks to provide online error detection in manipulation tasks for autonomous robot systems. GANDER relies on mapping input images of a trained task to a learned manifold that contains only positive task executions and outcomes. When reconstructed through this manifold, the input images from successful task executions will remain largely unchanged, while the images from a failed task will change significantly. Using this insight, GANDER enables inspection and task outcome verification capabilities using a large number of positive examples but only a small set of negative examples, thus increasing the applicability of autonomous robot systems. We detail the design of GANDER and provide results of a proof-of-concept system, establishing its efficacy in an autonomous inspection, maintenance, and repair task. GANDER produces favorable results compared to baseline approaches and is capable of correctly identifying off-nominal behavior with 91.65% accuracy in our test task. Ablation studies were also performed to quantify the amount of data ultimately needed for this approach to succeed.

Sample-based motion planning approaches, such as RRT*, have been widely adopted in robotics due to their support for high-dimensional state spaces and guarantees of completeness and optimality. This paper introduces an RRT* approach (ORRT*) that leverages opportunism to (1) find solutions quickly, (2) reduce wasted compute, and (3) improve data efficiency. The key insight of the approach is to make the most of compute when expanding the search tree by adding the last viable configurations found when connecting new nodes rather than rejecting the sampled nodes outright, allowing for more productive exploration of the space. We evaluate the proposed approach in a set of mobility and manipulator postural control domains, contrasting the performance of the opportunistic approach with state-of-the-art RRT* variants. Our analysis shows that such an approach has desirable characteristics and warrants further exploration.

This paper presents recent advances to the Affordance Template (AT) task description language. Affordance Templates provide standardized, easy-to-use tools for defining robot manipulation tasks that provide a high level of augmented reality capabilities to facilitate human-in-the-loop operation, but can also be used to support robot autonomy when coupled with various planning tools. While initially defined in terms of end effector waypoint sequences for bimanual robots, such as the NASA Valkyrie and Robonaut 2, this paper extends the original specification to support integrated mobile manipulation, object-centric template definitions, autonomous grasp determination, and integration with custom and off-the-shelf collision free motion planners. ATs have proved highly adaptable to new robots and new domains by both the authors and third-party groups, and have been used in numerous contexts for NASA, DOD, and industry. As such, we believe that the AT framework provides a strong foundation for robot development in multiple real-world contexts that can be increasingly built upon and expanded to meet the challenges of many new applications.

Deep learning has been successful in a variety of applications that have high-dimensional state spaces such as object recognition, video games, and machine translation. Deep neural networks can automatically learn important features from high-dimensional state given large training datasets. However, the success of deep learning in robot systems in the realworld is limited due to the cost of obtaining these large datasets. To overcome this problem, we propose an information-theoretic, intrinsically motivated, self-labeling mechanism using closed-loop control states. Taking this approach biases exploration to informative interactions-as such, a robot requires much less training to achieve reliable performance. In this paper, we explore the impact such an approach has on learning how to grasp objects. We evaluate different intrinsic motivators present in the literature applied appropriately in our framework and discuss the benefits and drawbacks of each.

This paper presents an active, model-based recognition system. It applies information theoretic measures in a belief-driven planning framework to recognize objects using the history of visual and manual interactions and to select the most informative actions. A generalization of the aspect graph is used to construct forward models of objects that account for visual transitions. We use populations of these models to define the belief state of the recognition problem. This paper focuses on the impact of the belief-space and object model representations on recognition efficiency and performance. A benchmarking system is introduced to execute controlled experiments in a challenging mobile manipulation domain. It offers a large population of objects that remain ambiguous from single sensor geometry or from visual or manual actions alone. Results are presented for recognition performance on this dataset using locomotive, pushing, and lifting controllers as the basis for active information gathering on single objects. An information theoretic approach that is greedy over the expected information gain is used to select informative actions, and its performance is compared to a sequence of random actions.

The design and implementation of a robotic chess agent is described. Shallow Blue, a competitor in the AAAI 2011 Small Scale Manipulation Challenge, is constructed with low-cost components including Lego NXT bricks and is programmed using Java and Lejos.