We present a method that estimates graspability measures on a single depth map for grasping objects randomly placed in a bin. Our method represents a gripper model by using two mask images, one describing a contact region that should be filled by a target object for stable grasping, and the other describing a collision region that should not be filled by other objects to avoid collisions during grasping. The graspability measure is computed by convolving the mask images with binarized depth maps, which are thresholded differently in each region according to the minimum height of the 3D points in the region and the length of the gripper. Our method does not assume any 3-D model of objects, thus applicable to general objects. Our representation of the gripper model using the two mask images is also applicable to general grippers, such as multi-finger and vacuum grippers. We apply our method to bin picking of piled objects using a robot arm and demonstrate fast pick-and-place operations for various industrial objects.

Progress in machine vision algorithms has led to widespread adoption of these techniques to automate several industrial assembly tasks. Nevertheless, shiny or specular objects which are common in industrial environments still present a great challenge for vision systems. In this paper, we take a step towards this problem under the context of vision-aided robotic assembly. We show that when the illumination source moves, the specular highlights remain in a region whose radius is inversely proportional to the surface curvature. This allows us to extract regions of the object that have high surface curvature. These points of high curvature can be used as features for specular objects. Further, an inexpensive multi-flash camera (MFC) design can be used to reliably extract these features. We show that one can use multiple views of the object using the MFC in order to triangulate and obtain the 3D location and pose of the shiny objects. Finally, we show a system consisting of a robot arm with an MFC that can perform automated detection and pose estimation of shiny screws within a cluttered bin, achieving position and orientation errors less than 0.5 mm and 0.8° respectively.

In this paper we present a robust alignment algorithm for correcting the effects of out-of-plane rotation to be used for automatic alignment of the Computed Tomography (CT) volumes and the generally low quality fluoroscopic images for radiotherapy applications. Analyzing not only in-plane but also out-of-plane rotation effects on the Dignitary Reconstructed Radiograph (DRR) images, we develop simple alignment algorithm that extracts a set of implicit features from DRR. Using these SIFT-based features, we align DRRs with the fluoroscopic images of the patient and evaluate the alignment accuracy. We compare our approach with traditional techniques based on gradient-based operators and show that our algorithm performs faster while in most cases delivering higher accuracy.

We propose a novel solution to object detection, localization and pose estimation with applications in robot vision. The proposed method is especially applicable when the objects of interest may not be richly textured and are immersed in heavy clutter. We show that a multi-flash camera (MFC) provides accurate separation of depth edges and texture edges in such scenes. Then, we reformulate the problem, as one of finding matches between the depth edges obtained in one or more MFC images to the rendered depth edges that are computed offline using 3D CAD model of the objects. In order to facilitate accurate matching of these binary depth edge maps, we introduce a novel cost function that respects both the position and the local orientation of each edge pixel. This cost function is significantly superior to traditional Chamfer cost and leads to accurate matching even in heavily cluttered scenes where traditional methods are unreliable. We present a sub-linear time algorithm to compute the cost function using techniques from 3D distance transforms and integral images. Finally, we also propose a multi-view based pose-refinement algorithm to improve the estimated pose. We implemented the algorithm on an industrial robot arm and obtained location and angular estimation accuracy of the order of 1 mm and 2° respectively for a variety of parts with minimal texture.

This paper presents a probing-based method for probabilistic localization in automated robotic assembly. We consider peg-in-hole problems in which a needle-like peg has a single point of contact with the object that contains the hole, and in which the initial uncertainty in the relative pose (3D position and 3D angle) between the peg and the object is much greater than the required accuracy (assembly clearance). We solve this 6 degree-of-freedom (6-DOF) localization problem using a Rao-Blackwellized particle filter, in which the probability distribution over the peg's pose is factorized into two components: The distribution over position (3-DOF) is represented by particles, while the distribution over angle (3-DOF) is approximated as a Gaussian distribution for each particle, updated using an extended Kalman filter. This factorization reduces the number of particles required for localization by orders of magnitude, enabling real-time online 6-DOF pose estimation. Each measurement is simply the contact position obtained by randomly repositioning the peg and moving towards the object until there is contact. To compute the likelihood of each measurement, we use as a map a mesh model of the object that is based on the CAD model but also explicitly models the uncertainty in the map. The mesh uncertainty model makes our system robust to cases in which the actual measurement is different from the expected one. We demonstrate the advantages of our approach over previous methods using simulations as well as physical experiments with a robotic arm and a metal peg and object.

In a robotic cell, an assembly robot has to grasp various parts robustly even under some uncertainties in their initial poses. For this purpose, it is necessary to design robust grasping strategies for robotic hands. In this paper, we propose a method to derive an optimal robust grasping strategy from a given initial pose error region of a target object. Based on the pushing operation analysis, it is possible to simulate multi-fingered hand grasping and derive a permissible initial pose error region of a target object from which planned grasping is successful. Adopting an active search algorithm proposed by the authors, we can find the optimal grasping strategy efficiently. As an example, we derive the optimal grasping strategy for grasping a circular object by a three-fingered hand.

This paper presents our real-time vision system for robots. The system employs a new internal image representation, in which the scene is encoded into three intensity level images. This representation is generated by Laplacian-Gaussian filtering followed by dual-thresholding. We refer to this image as three-level broad-edge representation. It reduces the computational cost of the normalized cross correlation. Thus the system performs the searching visual task as well as the object tracking task, which is not reliably achieved by the mean absolute error based correlators. Our prototype system achieves 32/spl times/32 pixel normalized cross correlation based matching from the 128/spl times/128 pixel search area in 2 ms under 9 MHz pixel clock image processor. This speed is fast enough for searching and tracking a single object at video frame rate.