This paper presents a data-efficient approach to learning transferable forward models for robotic push manipulation. Our approach extends our previous work on contact-based predictors by leveraging information on the pushed object's local surface features. We test the hypothesis that, by conditioning predictions on local surface features, we can achieve generalisation across objects of different shapes. In doing so, we do not require a CAD model of the object but rather rely on a point cloud object model (PCOM). Our approach involves learning motion models that are specific to contact models. Contact models encode the contacts seen during training time and allow generating similar contacts at prediction time. Predicting on familiar ground reduces the motion models' sample complexity while using local contact information for prediction increases their transferability. In extensive experiments in simulation, our approach is capable of transfer learning for various test objects, outperforming a baseline predictor. We support those results with a proof of concept on a real robot.

Dexterous grasping of objects with uncertain pose is a hard unsolved problem in robotics. This paper solves this problem using information gain re-planning. First we show how tactile information, acquired during a failed attempt to grasp an object can be used to refine the estimate of that object's pose. Second, we show how this information can be used to replan new reach to grasp trajectories for successive grasp attempts. Finally we show how reach-to-grasp trajectories can be modified, so that they maximise the expected tactile information gain, while simultaneously delivering the hand to the grasp configuration that is most likely to succeed. Our main novel outcome is thus to enable tactile information gain planning for Dexterous, high degree of freedom (DoFs) manipulators. We achieve this using a combination of information gain planning, hierarchical probabilistic roadmap planning, and belief updating from tactile sensors for objects with non-Gaussian pose uncertainty in 6 dimensions. The method is demonstrated in trials with simulated robots. Sequential replanning is shown to achieve a greater success rate than single grasp attempts, and trajectories that maximise information gain require fewer re-planning iterations than conventional planning methods before a grasp is achieved.

This paper presents an algorithm for planning sequences of pushes, by which a robotic arm equipped with a single rigid finger can move a manipulated object (or manipulandum) towards a desired goal pose. Pushing is perhaps the most basic kind of manipulation, however it presents difficult challenges for planning, because of the complex relationship between manipulative pushing actions and resulting manipulandum motions. The motion planning literature has well developed paradigms for solving e.g. the piano-mover's problem, where the search occurs directly in the configuration space of the manipulandum object being moved. In contrast, in pushing manipulation, a plan must be built in the action space of the robot, which is only indirectly linked to the motion space of the manipulandum through a complex interaction for which inverse models may not be known. In this paper, we present a two stage approach to planning pushing operations. A global RRT path planner is used to explore the space of possible manipulandum configurations, while a local push planner makes use of predictive models of pushing interactions, to plan sequences of pushes to move the manipulandum from one RRT node to the next. The effectiveness of the algorithm is demonstrated in simulation experiments in which a robot must move a rigid body through complex 3D transformations by applying only a sequence of simple single finger pushes.