This paper presents methods to compute the locus of all viewpoints from which features on known polyhedral objects can be viewed in their entirety without being occluded by anything in the environment. Convex and concave polyhedra with or without holes and the viewing model of perspective projection are employed in this work. Initially, properties of the occlusion-free and occluded loci of viewpoints are determined. Based on these properties, two methods to construct these loci together with their complexity analysis are presented. In the first method, a boundary representation of the occlusion-free locus is obtained. In the second method, the locus of occluded viewpoints is expressed in terms of a constructive solid geometry representation that consists of a union of component solids. Implementation results and comparison of the two methods are given.

A survey of research in the area of vision sensor planning is presented. The problem can be summarized as follows: given information about the environment as well as information about the task that the vision system is to accomplish, develop strategies to automatically determine sensor parameter values that achieve this task with a certain degree of satisfaction. With such strategies, sensor parameters values can be selected and can be purposefully changed in order to effectively perform the task at hand. The focus here is on vision sensor planning for the task of robustly detecting object features. For this task, camera and illumination parameters such as position, orientation, and optical settings are determined so that object features are, for example, visible, in focus, within the sensor field of view, magnified as required, and imaged with sufficient contrast. References to, and a brief description of, representative sensing strategies for the tasks of object recognition and scene reconstruction are also presented. For these tasks, sensor configurations are sought that will prove most useful when trying to identify an object or reconstruct a scene.>

The MVP (machine vision planner) model-based sensor planning system for robotic vision is presented. MVP automatically synthesizes desirable camera views of a scene based on geometric models of the environment, optical models of the vision sensors, and models of the task to be achieved. The generic task of feature detectability has been chosen since it is applicable to many robot-controlled vision systems. For such a task, features of interest in the environment are required to simultaneously be visible, inside the field of view, in focus, and magnified as required. In this paper, we present a technique that poses the vision sensor planning problem in an optimization setting and determines viewpoints that satisfy all previous requirements simultaneously and with a margin. In addition, we present experimental results of this technique when applied to a robotic vision system that consists of a camera mounted on a robot manipulator in a hand-eye configuration.>

Methods for computing the locus of all viewpoints from which features on known polyhedral objects can be viewed in their entirety without being occluded by anything in the environment are presented. Convex and concave polyhedra with or without holes and the viewing model of perspective projection are used in this work. Based on properties of the occlusion-free and occluded loci of viewpoints, two methods for constructing these loci together with their complexity analysis are presented. In one method, a boundary representation of the occlusion-free locus is obtained. In the other, the locus of occluded viewpoints is expressed in terms of a constructive solid geometry representation that consists of a union of component solids. Implementation results are shown.>

The authors present calibration techniques to determine the relationships that map the parameters associated with a camera viewpoint to the parameters that can actually be controlled in a reconfigurable vision system. These relationships can be used to achieve the desired values of the camera viewpoint parameters by setting the controllable parameters to the appropriate values. The sensor setup consisted of a camera lens with zoom, focus, and aperture control. The calibration techniques were applied to the H 6*12.5 R Fujinon zoom lens and experimental results are shown.>

The authors present techniques with which camera viewpoints can be automatically determined in order to satisfy the generic feature detectability requirements of resolution, focus, and field-of-view. These optical constraints are characterized analytically for a general thick lens model, and complete viewpoint loci that satisfy these constraints are obtained. The results discussed are useful for automating the vision system design process, as well as for programming the vision system itself. In addition, such planning techniques can also automate robot imaging systems that reconfigure themselves in an intelligent manner in order to optimize imaging quality.>

A new approach is described for geometric calibration of Cartesian robots. This is part of a set of procedures for real-time 3-D robotics eye, eye-to-hand, and hand calibration which uses a common setup and calibration object, common coordinate systems, matrices, vectors, symbols, and operations and is especially suited to machine vision systems. The robot makes a series of automatically planned movement with a camera rigidly mounted at the gripper. At the end of each move, it takes a total of 90 ms to grab an image, extract image feature coordinates, and perform camera-extrinsic calibration. After the robot finishes all the movements, it takes only a few milliseconds to do the calibration. The key of this technique is that only one rotary joint is moving for each movement. This allows the calibration parameters to be fully decoupled, and converts a multidimensional problem into a series of one-dimensional problems. Another key is that eye-to-hand transformation is not needed at all during the computation.>

The authors describe a novel technique for computing position and orientation of a camera relative to the last joint of a robot manipulator in an eye-on-hand configuration. It takes only about 100+64N arithmetic operations to compute the hand/eye relationship after the robot finishes the movement, and incurs only additional 64 arithmetic operations for each additional station. The robot makes a series of automatically planned movements with a camera rigidly mounted at the gripper. At the end of each move, it takes a total of 90 ms to grab an image, extract image feature coordinates, and perform camera extrinsic calibration. After the robot finishes all the movements, it takes only a few milliseconds to do the calibration. A series of generic geometric properties or lemmas are presented, leading to the derivation of the final algorithms, which are aimed at simplicity, efficiency, and accuracy while giving ample geometric and algebraic insights. Critical factors influencing the accuracy are analyzed, and procedures for improving accuracy are introduced. Test results of both simulation and real experiments on an IBM Cartesian robot are reported and analyzed.>

A novel approach is described to geometric calibration of Cartesian robots. This is one-third of an overall approach to real-time 3-D robotics eye, eye-to-hand, and hand calibration, which uses a common setup and calibration object, and is especially suited to machine vision community. The robot makes a series of automatically planned movements with a camera rigidly mounted at the gripper. At the end of each move, it takes a total of 90 ms to grab an image, extract image feature coordinates, and perform camera extrinsic calibration. After the robot finishes all the movements, it takes only a few milliseconds to do the calibration. Only one single rotary joint is moving for each movement while the robot motion can still be planned such that the calibration object remains within the field of view. This allows the calibration parameters to be fully decoupled, and converts a multidimensional problem into a series of one-dimensional problems. Eye-to-hand transformation is not needed at all during the computation.>

A technique is described for computing 3-D position and orientation of a camera relative to the last joint of a robot manipulator in an eye-on-hand configuration. The calibration can be done within a fraction of a millisecond after the robot finishes the movement. The setup is simple (a planar set of calibration points arbitrarily placed on the work table, in addition to robot and camera) and is the same as that for a common camera calibration. This method is claimed to be faster, simpler, and more accurate than any existing technique for hand/eye calibration. Generic geometric properties of lemmas are presented, leading to the derivation of the final algorithms, which are aimed at simplicity, efficiency, and accuracy while giving ample geometric and algebraic insights. Besides describing the technique, critical factors influencing the accuracy are analysed, and procedures for improving accuracy are introduced. Tests results of both simulation and real experiments on an IBM Cartesian robot are reported.>

Techniques are described for calibrating certain intrinsic camera parameters for machine vision. The parameters to be calibrated are the horizontal scale factor, and the image center. The scale factor calibration uses a one-dimensional fast Fourier transform and is accurate and efficient. It also permits the use of only one coplanar set of calibration points for general camera calibration. Three groups of techniques for center calibration are presented: Group I requires using a laser and a four-degree-of-freedom adjustment of its orientation, but is simplest in concept and is accurate and reproducible; Group II is simple to perform, but is less accurate than the other two; and the most general, Group II, is accurate and efficient, but requires a good calibration plate and accurate image feature extraction of calibration points. Group II is recommended most highly for machine vision applications. Results of experiments are presented and compared with theoretical predictions. Accuracy and reproducibility of the calibrated parameters are reported, as well as the improvement in actual 3-D measurement due to center calibration.>

This paper describes techniques for calibrating certain intrinsic camera parameters for machine vision. The parameters to be calibrated are the horizontal scale factor, i.e. the factor that relates the sensor element spacing of a discrete array camera to the picture element spacing after sampling by the image acquisition circuitry, and the image center, i.e. the intersection of the optical axis with the camera sensor. The scale factor calibration uses a 1D-FFT and is accurate and efficient. It also permits the use of only one coplanar set of calibration points for general camera calibration. Three groups of techniques for center calibration are presented: Group I requires using a laser and a four-degree of freedom adjustment of its orientation, but is simplest in concept, and is accurate and reproducible. Group II is simple to perform, but is less accurate than the other two. The most general Group III is accurate and efficient, but requires accurate image feature extraction of calibration points with known 3D coordinates. A feasible setup is described. Results of real experiments are presented and compared with theoretical predictions. Accuracy and reproducibility of the calibrated parameters are reported, as well as the improvement in actual 3D measurement due to center calibration.

A new technique for three-dimensional (3D) camera calibration for machine vision metrology using off-the-shelf TV cameras and lenses is described. The two-stage technique is aimed at efficient computation of camera external position and orientation relative to object reference coordinate system as well as the effective focal length, radial lens distortion, and image scanning parameters. The two-stage technique has advantage in terms of accuracy, speed, and versatility over existing state of the art. A critical review of the state of the art is given in the beginning. A theoretical framework is established, supported by comprehensive proof in five appendixes, and may pave the way for future research on 3D robotics vision. Test results using real data are described. Both accuracy and speed are reported. The experimental results are analyzed and compared with theoretical prediction. Recent effort indicates that with slight modification, the two-stage calibration can be done in real time.

Two main results are established in this paper. First, we show that seven point correspondences are sufficient to uniquely determine from two perspective views the three-dimensional motion parameters (within a scale factor for the translations) of a rigid object with curved surfaces. The seven points should not be traversed by two planes with one plane containing the origin, nor by a cone containing the origin. Second, a set of ``essential parameters'' are introduced which uniquely determine the motion parameters up to a scale factor for the translations, and can be estimated by solving a set of linear equations which are derived from the correspondences of eight image points. The actual motion parameters can subsequently be determined by computing the singular value decomposition (SVD) of a 3×3 matrix containing the essential parameters.

This paper discusses a technique for estimation of 3-D motion parameters and object surface orientation of a rigid moving object from the correspondence of a conic arc segment in several perspective views; the correspondences of the intermediate points on the conic arcs are not known a priori. The paper focusses on a theorem and a corollary regarding the validity of simultaneously representing a pair of corresponding conic arc segments, one before and the other after the motion, using rational quadratic parameterization scheme with each corresponding pair of points on the two conic arcs characterized by the same running parameter. This theorem is essential to the proof of uniqueness and the derivation of solutions for the 3-D motion parameters and object surface orientation; these topices are covered in a later paper[15].

This paper presents two new methods, the Joint Moment Method (JMM) and the Window Variance Method (WVM), for image matching and 3-D object surface reconstruction using multiple perspective views. Like the conventional two-frame correlation method, the JMM and WVM require finding the extrema of 1-D curves, which are proved to theoretically approach a delta function exponentially as the number of frames increases for the JMM and are much sharper than the two-frame correlation function for both the JMM and the WVM. The theoretical findings have been supported by simulations.

This paper presents two new methods, the Joint Moment Method (JMM) and the Window Variance Method (WVM), for image matching and 3-D object surface reconstruction using multiple perspective views. The viewing positions and orientations for these perspective views are known a priori, as is usually the case for such applications as robotics and industrial vision as well as close range photogrammetry. Like the conventional two-frame correlation method, the JMM and WVM require finding the extrema of 1-D curves, which are proved to theoretically approach a delta function exponentially as the number of frames increases for the JMM and are much sharper than the two-frame correlation function for both the JMM and the WVM, even when the image point to be matched cannot be easily distinguished from some of the other points. The theoretical findings have been supported by simulations. It is also proved that JMM and WVM are not sensitive to certain radiometric effects. If the same window size is used, the computational complexity for the proposed methods is about n - 1 times that for the two-frame method where n is the number of frames. Simulation results show that the JMM and WVM require smaller windows than the two-frame correlation method with better accuracy, and therefore may even be more computationally feasible than the latter since the computational complexity increases quadratically as a function of the window size.

Past work in motion-compensated interframe coding has been restricted to two-dimensional translation. It is the purpose of this paper to report some experimental results of a study on the potential advantage of considering more general types of motion. More specifically, we shall consider general two-dimensional motion including translation, rotation, and scale change (zooming). The entropy of the histogram of motion-compensated frame differences will be used as a performance measure. For comparison, this quantity will be measured for the case of using only two-dimensional translation as well as the case of using general two-dimensional motion.

This paper considers the three-view problem in motion estimation from image sequences. It is proved using the properties of Singular Value Decomposition and analytical geometry that, given four image point correspondences in three perspective views taken by a single camera of a planar patch undergoing general three-dimensional rigid-body motion, the number of solution for the motion parameters is one aside from a scale factor for the translations, as opposed to two when only two perspective views are given.

We present a two-step method of estimating the parameters of general three-dimensional motion of rigid bodies based on two time-sequential perspective views. The first step is to estimate the image-space shifts of a number of points. Then, the second step is to determine the motion parameters from these shifts. The second step involves the solution of equations where the unknowns can be partitioned into two subsets appearing bilinearly. An iterative method for solving such equations is suggested. In the special cases of three-dimensional translation and of two-dimensional motion with zooming, the equations become linear.

A new method for moving image restoration and registration is established. The observations are sequences of low-resolution, possibly undersampled, discrete frames. The result is a restored high-resolution image. The restoration part is attractive for the purpose of real time implementation since the computation consists of only a few complex operations per pel of the resultant high resolution image. Tests on a class of waveforms with different bandwidths show that the performance is superior to that of the cubic spline technique in terms of the signal to noise ratio. The registration part has two new features: 1) The relative shifts of several frames can be estimated in one process. 2) The estimate is accruate even if the observation frames are severely undersampled, so long as the number of frames available is big enough.