Robots acting in open environments need to be able to handle novel objects. Based on the observation that objects within a category are often similar in their shapes and usage, we propose an approach for transferring grasping skills from known instances to novel instances of an object category. Correspondences between the instances are established by means of a non-rigid registration method that combines the Coherent Point Drift approach with subspace methods. The known object instances are modeled using a canonical shape and a transformation which deforms it to match the instance shape. The principle axes of variation of these deformations define a low-dimensional latent space. New instances can be generated through interpolation and extrapolation in this shape space. For inferring the shape parameters of an unknown instance, an energy function expressed in terms of the latent variables is minimized. Due to the class-level knowledge of the object, our method is able to complete novel shapes from partial views. Control poses for generating grasping motions are transferred efficiently to novel instances by the estimated non-rigid transformation.

Robotic picking from cluttered bins is a demanding task, for which Amazon Robotics holds challenges. The 2017 Amazon Robotics Challenge (ARC) required stowing items into a storage system, picking specific items, and packing them into boxes. In this paper, we describe the entry of team NimbRo Picking. Our deep object perception pipeline can be quickly and efficiently adapted to new items using a custom turntable capture system and transfer learning. It produces high-quality item segments, on which grasp poses are found. A planning component coordinates manipulation actions between two robot arms, minimizing execution time. The system has been demonstrated successfully at ARC, where our team reached second places in both the picking task and the final stow-and-pick task. We also evaluate individual components.

Coordination between visual sensors and robot manipulators is necessary for successful manipulation. This paper proposes a novel visual-motor coordination method that performs online parameter estimation of an RGB-D camera mounted in a robot head without any external markers. Through self-observation of a dual-arm robot manipulator, the method updates parameters to reduce the discrepancy between observed point cloud data and 3D mesh models of the current robot con guration. With the estimated parameters at each time step, visual data is adjusted to the focused workspace of the 14DOF dual-arm robot manipulator. The online and realtime algorithm was developed by using a GPU-based particle filtering method. Experimental results show that our method outperforms state-of-the-art offline registration methods in terms of accuracy and computation time. We also analyzed the dependence of the results on prior parameters to demonstrate the online capability of our method.