Previous work has introduced the planning coordinator, a logical extension to the coordination level of the intelligent machine model that provides a platform for error recovery and online planning by autonomous and semi-autonomous robotic systems. This paper refines the design architecture of the planning coordinator focusing on the primitive structure database and node/link weighting mechanism. This architecture facilitates the role of the planning coordinator as the mechanism by which an existing robotic system can be made more autonomous.

The potential for the use of fuzzy logic in the assignment of fuzzy weights to the existing relational links of the Primitive Structure Database is explored. The assignment of these weights is crucial to establishing robust error recovery and a competent online planning scheme as the weights provide a mechanism whereby the extent of the strength of a relation can be established. Through the appropriate defuzzification of the combined weights in the path of a single plan or multiple plans, a more realistic expectation of the potential success of a particular plan can be extracted than can be expected if the assigned weights were initially crisp numbers. The defuzzified weighting can then be used to hierarchically structure several different plans with differing potentials for success. This hierarchy permits alternative plans to be made available should a plan with the highest potential for success rating field. In addition, a limiting agent (threshold) established through the defuzzification of the relational weights can be used to limit the number of potential plans that will be examined.

An object representation scheme which allows fast Euclidean distance calculation is presented. The object model extends the polytope model by representing objects as the convex hull of a finite set of spheres. Objects with rounded surfaces can be represented with far fewer vertices than would be required with the polytope model. An algorithm for calculating distances between objects is developed which is linear in the total number of spheres specifying the two objects. To determine the efficiency of the distance computation algorithms, a program was written and tested using randomly generated objects with varying numbers of vertices.>

The Planning Coordinator is a logical extension to the Coordination Level of the Intelligent MachineMOdel, func- tioning to provide a heretofore unavailable platform for robust error recovery and dynamic on-line planning by autonomous and semi-autonomous robotic systems. This paper introduces the design architecture of the Planning Coordinator and focuses upon its macro-structure, interfaces and functional description, with respect to its role as the mechanism whereby an existing robotic system requiring significant human intervention can be made more autonomous, thus becoming more robust.

An examination is made of the fundamental trade-off between exploration and exploitation in a genetic algorithm (GA). An immigration operator is introduced that infuses random members into successive GA populations. It is theorized that immigration maintains much of the exploitation of the GA while increasing exploration. To test this theory, a set of functions that often require the GA to perform an excessive number of evaluations to find the global optimum of the function is designed. For These functions, it is shown experimentally that a GA enhanced with immigration (1) reduces the number of trials that require an excessive number of evaluations and (2) decreases the average number of evaluations needed to find the optimum function.>

A new scheme for the representation of objects, the successive spherical approximation (SSA), facilitates the rapid planning of collision-free paths in a dynamic, three-dimensional environment. The hierarchical nature of the SSA allows collisions to be determined efficiently while still providing an exact representation of objects. The rapidity with which collisions can be detected, less than one second per environment object per path, makes it possible to use a generate-and-test path-planning strategy driven by human conceptual knowledge to determine collision-free paths in a matter of seconds on a Sun 3/180 computer. A hierarchy of rules, based on the concept of a freespace cell, is used to find heuristically satisfying collision-free paths in a structured environment.>

The authors present the design and implementation of a robot system equipped with multiple sensors. The robotic assembly system couples the task sequence of the robot with information from different sensors, enabling the system to handle uncertainties encountered during task execution. The target task involves the insertion of a printed circuit board into a card cage. The sensors used are an overhead three-dimensional vision system, a fingertip cross-fire sensor, and a six-degree-of-freedom force/torque sensor. The vision system is used to locate the printed circuit boards and the insertion slots. Information provided by the cross-fire sensor is used to complement the information from the vision system in order to obtain a proper grasping location on the printed circuit board. A fuzzy-logic-based system is used to interpret the forces and torques generated by the force/torque sensor during the insertion process. Using a fuzzy controller, an experiment is presented which successfully inserts printed circuit boards into target insertion slots. The repeated success of this assembly task over many trials verifies the modeling and controller assumptions made by fuzzy set theory.>

The authors propose a robotic assembly system that is based on the least-commitment principle, in which the successive stages proceeding from planning to execution become progressively more specific. The system is partitioned into an offline automatic programming system and an online execution manager. The framework and objectives are presented, followed by an overview of the global system. The submodules of a system that has actually executed a simple pick/place task are described. Some conclusions and perspectives are outlined.>

To increase task flexibility in a robotic assembly environment, a hierarchical planning and execution system is being developed which will map user-specified, three-dimensional part assembly tasks into various target robotic workcells, and execute these tasks efficiently using manipulators and sensors available in the workcell. One level of this hierarchy, the supervisor, is responsible for assigning subtasks of a system-generated task plan to a set of task specific specialists, and online coordination of the activity of these specialists to accomplish the user-specified assembly. The design of the supervisor is broken down into five major functional blocks: resource management; concurrency detection; task scheduling; error recovery; and interprocess communication. Using concepts from operating systems, the design of each of these supervisor blocks is examined.>

Vertical integration of transformation of abstract assembly task descriptions into task execution commands is needed to overcome the fundamental factors which limit the widespread application of current robotic systems. These factors are the requirements for precise control of the robot task environment, custom-designed fixtures and end-effectors, and on-site robot control programming. These requirements arise because current robotic systems do not have an understanding of the robot cell environment. Further, these systems are not able to react rationally to non-deterministic events which occur during task execution. A research plan which is aimed at the systematic incorporation of limited amounts of machine intelligence into robot task planning and execution is outlined.

Gripper mounted infrared proximity sensors can provide essential information concerning the neighborhood of the end effector whether it be for collision avoidance, workpiece location, or workpiece recognition. These sensors, along with a hierarchical control through an intelligent sensor interface computer, could offer simple, inexpensive, and reliable sensory feedback control of a robot in 3D space. This paper explains how infrared proximity sensors, strategically mounted in the finger tips of a general purpose parallel-jaw gripper, aid in the control of a robot arm. A sensor operating system was written in assembly language for the gripper control microcomputer to provide high level communications to a host computer. The system's input/output capabilities allow handshaking and sensory control between it and the robot.

Least square error methods are employed to improve the precision of a robot world coordinate system model. A solution which reduces the large scale errors of the robot is presented. This method is then applied to the task of vision directed workpiece reorientation.

Currently, a major difficulty for the widespread use of robots in assembly and material handling comes from the necessity of feeding accurately positioned workpieces to robots. ``Bin picking'' techniques help reduce this constraint. This paper presents the application of matched filters for enabling robots with vision to acquire workpieces randomly stored in bins. This approach complements heuristic methods already reported. The concept of matched filter is an old one. Here, however, it is redefined to take into account robot end-effector features, in terms of geometry and mechanics. In particular, the proposed filters match local workpiece structures where the robot end-effector is likely to grasp successfully and hold workpieces. The local nature of the holdsites is very important as computation costs are shown to vary with the fifth power of structure size. In addition, the proposed filters tend to have a narrow angular bandwidth. An example, which features a parallel-jaw hand is developed in detail, using both statistical and Fourier models. Both approaches concur in requiring a very small number of filters (typically four), even if a good orientation accuracy is expected (two degrees). Success rates of about 90 percent in three or fewer attempts have been experimentally obtained on a system which includes a small minicomputer, a 128 × 128 pixel solidstate camera, a prototype Cartesian robot, and a ``universal'' parallel-jaw hand.

Surface normals can be computed from three images of a workpiece taken under three distinct lighting conditions without requiring surface continuity. Radiometric methods are susceptible to systematic errors such as: errors in the measurement of light source orientations; mismatched light source irradiance; detector nonlinearity; the presence of specular reflection or shadows; the spatial and spectral distribution of incident light; surface size, material, and microstructure; and the length and properties of the light source to target path. Typically, a 1° error in surface orientation of a Lambertian workpiece is caused by a 1 percent change in image intensity due to variations in incident light intensity or a 1° change in orientation of a collimated light source. Tests on a white nylon sphere indicate that by using modest error prevention and calibration schemes, surface angles off the camera axis can be computed within 5°, except at edge pixels. Equations for the sensitivity of surface normals to major error sources have been derived. Results of surface normal estimation and edge extraction experiments on various non-Lambertian and textured workpieces are also presented.

The feasibility of robots employing vision to acquire randomly oriented cylinders has been demonstrated for the first time. An experimental robot system using vision and a parallel jaw gripper was able to acquire randomly oriented cylindrical workpieces piled in bins. Binary image analysis was adequate to guide the gripper into the multilayered, piles. Complementary information was provided by simple sensors on the gripper. Experiments were performed using titanium cylinders 6 cm × 15 cm diameter and 7.6 cm × 3 cm diameter. Cycle times to acquire a cylinder and deliver it to a receiving chute ranged from 8 to 10 s when a single supply of one-size cylinders was used. By using a dual supply bin configuration with one bin of each size and overlapping arm motion and image analysis tasks, the cycle times for one cylinder from alternate bins ranged from 5.5 to 7.5 s per piece. In the future robots with such capabilities can be applied to enhance automation applications, especially in small batch production.

Bin-picking robots acquire a workpiece from a bin, reorient it, and place it into a machine. However, to achieve adaptability the system must incorporate general purpose hands capable of working with a wide variety of workpieces. To this end, eight guidelines for the design of general purpose hands for bin-picking robots have been identified on the basis of functional requirements. Using the eight guidelines as a framework, three hands were designed. The first design utilized a single vacuum cup which can sense contact, adapt to unknown surface angles, sense grip, and maintain a fixed hand-to-workpiece relationship. The second is a contour adapting vacuum gripper comprised of an array of 20 vacuum-gripping units which are capable of sensing discrete surface contour points while grasping. The third is a parallel jaw arrangement similar to those commonly found in industry, except that it has the capability of sensing overload and the presence of a workpiece within the hand. Each of the hands is described in terms of design features, control systems, and mode of operation. Advantages and disadvantages are indicated.

A robot system has been tested which can acquire a class of workpieces that are unoriented in bins and transport them, one at a time, to a destination with the proper orientation. This kind of robot has numerous applications in industries which manufacture discrete-part products in batches. A variety of approaches to the design of an orienting robot have been identified. For acquisition, gripper design and vision algorithms should be considered together. A vacuum gripper which adapts to surface angle and an algorithm which finds the center of smooth regions in the bin image have been shown to form an effective combination. Knowledge of the position and orientation of the workpieces near the top of the bin is not necessary for acquisition. Collisions with workpieces in the bin whose poses are not known can be avoided by having the gripper follow a line of sight approach path. To simplify the computation of workpiece orientation, it was found to be useful to employ a gripper which locked a workpiece in one of a finite number of states and which did not obscure the workpiece. For an orienting robot which is designed to service a variety of different workpieces, using an arm with six degrees of freedom greatly facilitates workpiece transportation.

Features extracted by operators which examine diameter-limited gradient direction histograms are important because they describe images of industrial workpieces efficiently and have the potential for rapid computation via special purpose hardware. When such operators are passed over an image in raster fashion, features such as the following are detected: a strong peak in the direction histogram indicating the presence of a relatively straight edge, a second strong direction indicating a corner, a wide direction group indicating a curved edge, and a uniform distribution of directions over the histogram indicating small holes. Diameter-limited optimization can be used to substantially reduce the number of pixels which have been given feature labels by such operators without losing descriptive power. Feature labels for direction histograms having a second strong direction, a wide direction group, or a uniform distribution might be retained only if all other pixels within a circular aperture have a lower bin value. Pixels with a strong direction label might be retained only if all other pixels within a circular aperture and along the gradient direction have a smaller bin value. Tracking can then be applied to the remaining strong direction pixels in the direction perpendicular to the gradient to achieve representation of edges by endpoints. Experimental results indicate that descriptions which are compatible with human interpretation can be achieved.