The need for combined task and motion planning in robotics is well understood. Solutions to this problem have typically relied on special purpose, integrated implementations of task planning and motion planning algorithms. We propose a new approach that uses off-the-shelf task planners and motion planners and makes no assumptions about their implementation. Doing so enables our approach to directly build on, and benefit from, the vast literature and latest advances in task planning and motion planning. It uses a novel representational abstraction and requires only that failures in computing a motion plan for a high-level action be identifiable and expressible in the form of logical predicates at the task level. We evaluate the approach and illustrate its robustness through a number of experiments using a state-of-the-art robotics simulator and a PR2 robot. These experiments show the system accomplishing a diverse set of challenging tasks such as taking advantage of a tray when laying out a table for dinner and picking objects from cluttered environments where other objects need to be re-arranged before the target object can be reached.

Delivering on the promise of real-world robotics will require robots that can communicate with humans through natural language by learning new words and concepts through their daily experiences. Our research strives to create a robot that can learn the meaning of haptic adjectives by directly touching objects. By equipping the PR2 humanoid robot with state-of-the-art biomimetic tactile sensors that measure temperature, pressure, and fingertip deformations, we created a platform uniquely capable of feeling the physical properties of everyday objects. The robot used five exploratory procedures to touch 51 objects that were annotated by human participants with 34 binary adjective labels. We present both static and dynamic learning methods to discover the meaning of these adjectives from the labeled objects, achieving average F1 scores of 0.57 and 0.79 on a set of eight previously unfelt items.

We propose a system for human-robot interaction that learns both models for spatial prepositions and for object recognition. Our system grounds the meaning of an input sentence in terms of visual percepts coming from the robot's sensors in order to send an appropriate command to the PR2 or respond to spatial queries. To perform this grounding, the system recognizes the objects in the scene, determines which spatial relations hold between those objects, and semantically parses the input sentence. The proposed system uses the visual and spatial information in conjunction with the semantic parse to interpret statements that refer to objects (nouns), their spatial relationships (prepositions), and to execute commands (actions). The semantic parse is inherently compositional, allowing the robot to understand complex commands that refer to multiple objects and relations such as: “Move the cup close to the robot to the area in front of the plate and behind the tea box”. Our system correctly parses 94% of the 210 online test sentences, correctly interprets 91% of the correctly parsed sentences, and correctly executes 89% of the correctly interpreted sentences.

We propose an evolutionary algorithm to autonomously improve the performances of a robotics skill. The algorithm extends a previously proposed graphical evolutionary skills building approach to allow a robot to autonomously collect use cases where a skill fails and use them to improve the skill. Here we define a computational graph as a generic model to hierarchically represent skills and to modify them. The computational graph makes use of embedded neural networks to create generic skills. We tested our proposed algorithm on a real robot implementing a “move to reach” action. Four experiments show the evolution of the computational graph as it is adapted to solve increasingly complex problems.

This paper presents an approach for the simultaneous learning and recognition of places applied to autonomous robotics. While noteworthy results have been achieved with respect to off-line training process for appearance-based navigation, novel issues arise when recognition and learning are simultaneous and unsupervised processes. The approach adopted here uses a Gaussian mixture model estimated by a novel incremental MML-EM to model the probability distribution of features extracted by image-preprocessing. A place detector decides which features belong to which place integrating odometric information and a hidden Markov model. Tests demonstrate that the proposed system performs as well as the ones relying on batch off-line environmental learning.

In this paper we propose a three-layered neural network for binary pattern recognition and memorization. Unlike the classic approach to pattern recognition, our net works organizing itself in an unsupervised way, to distinguish beetween different patterns or to recognize similar ones. If we present a binary input to the first layer, after some time steps we could read the output of the net in the third layer, as one and only one neuron activating with high firing rate; the middle layer will act as a generalization layer, i.e. similar pattern will have similar (or the same) representation in the middle layer. We used learning algorithms inspired from other works or from biological data to achieve network stability and a correct pattern memorization. The network can be used for pattern recognition or generalization by selecting output signals from the selection layer or the generalization layer.