We present a data-driven approach for effective bin picking from clutter. Recent bin picking solutions usually lead to a direct pinch grasp on a target object without addressing any other potential contact interaction in clutter. However, appropriate physical interaction can be essential to successful singulation and subsequent secure picking, the goal of bin picking. In this work, we contribute a framework that learns physically interactive actions for object picking end-to-end from a visual input in a self-supervised manner. The learned actions enable the robot to purposefully interact with a target object by performing a digging operation through the clutter. By leveraging a fully convolutional network (FCN), we predict picking success probabilities for a set of interactive action primitives that will in turn specify an optimal action to perform. The FCN is trained in a simulated environment through trial and error. Moreover, new datasets are collected using the latest network through iterative self-supervision. Extensive real-world bin picking experiments show the effectiveness and generalizability of the approach.

This paper introduces the technique of tilt-and-pivot manipulation, a new method for picking thin, rigid objects lying on a flat surface through robotic dexterous in-hand manipulation. During the manipulation process, the gripper is controlled to reorient about the contact with the object such that its finger can get in the space between the object and the supporting surface, which is formed by tilting up the object, with no relative sliding motion at the contact. As a result, a pinch grasp can be obtained on the faces of the thin object with ease. We discuss issues regarding the kinematics and planning of tilt-and-pivot, effector shape design, and the overall practicality of the manipulation technique, which is general enough to be applicable to any rigid convex polygonal objects. We also present a set of experiments in a range of bin picking scenarios.