Experience has shown that changes in illumination conditions may influence the robustness of robot control based on machine vision even if such illumination changes appear practically negligible to a human. CMOS cameras have become available with a larger dynamic range than conventional (high quality) CCD cameras. The use of such higher dynamic range cameras for robot control by machine vision is compared with the use of conventional cameras under various illumination conditions. The task to be performed by the robot consists in the removal of a battery from the engine compartment of a used car. In all cases, the servo task could be accomplished successfully. The stop of CCD cameras had to be adjusted manually, however, whereas an automatic adaptation to the current lighting conditions could be implemented for the other camera type, thus allowing us to deal automatically with significant changes of illumination.

Automatic disassembly of used products can not assume that CAD databases will provide precise knowledge about the pose, size, and shape of components to be manipulated because some of the components may have been repaired, repositioned, or replaced, thereby possibly invalidating the original construction data. In principle, the missing information can be provided by machine vision which often becomes more robust if it can rely on some knowledge about the workpiece to be manipulated, for example, in the form of generic descriptions. Use of the latter necessitates, however, that not only an unknown pose, but in addition unknown size and shape parameters of a component have to be determined. A solution to the automatic determination of location, orientation, size, and shape of truncated cones is demonstrated by the example of shock absorber lids in the engine compartment of a used car.

Over the past thirty years, technical means to record, digitize, store, process, and present video image sequences have multiplied in capacity by several orders of magnitude. This large quantitative improvement is about to turn into a qualitative change regarding the research to be pursued within this area in the future. Whereas up to now an exploratory type of approach has prevailed, the increasingly available technical means for image sequence evaluation facilitate the investigation of large amounts of data. This requires to build encompassing systems and to study their behavior which in turn is expected to stimulate theoretical developments: even small effects of an assumption can be studied in detail within a system environment, thereby enabling stepwise improvements of theoretical approaches underlying the construction of system solutions. The following discussion will emphasize fundamental experimental and theoretical advances rather than details of particular approaches.

The interpretation of road traffic scenes recorded by a stationary video camera requires a sufficiently large field of view in order to capture non-trivial maneuvers. Vehicle images will thus be small. Their reliable tracking needs to combine edge-based approaches for precision with area-oriented approaches, for example based on a match between estimated and predicted Optic Flow fields, for stability. The approach reported here estimates the steering angle of vehicles to be tracked and compares this as well as a velocity estimate with actual measurements performed in a suitably equipped test vehicle recorded while crossing a road intersection. In addition, tracking results for numerous other vehicles recorded in the same scene have been assessed interactively and are reported. Computation of the Jacobi matrix for the Kalman-Filter is facilitated by a "dual number" representation for partial derivatives.

Model-based vehicle tracking in traffic image sequences can be made more robust by matching expected displacement rates of vehicle surface points to optical flow (OF) vectors computed from an image sequence. The capability to track vehicles uninterruptedly in this manner over extended image sequences results in the ability to investigate even small errors in OF estimation. It turns out that the OF magnitudes are systematically underestimated. The-albeit small-bias can be corrected by analyzing the influence of explicitly modeled grey value noise on the precision of OF values estimated by means of the neighborhood sampling method.

It is advantageous to use an independently mobile camera in visually servoed disassembly operations since its degrees of freedom can be used to bring the camera into an optimal view-pose. Apart from the vision-based closed-loop control that guides the disassembly robot, a second vision-based closed-loop control can be set up which upgrades the camera from a passive to an active sensor and optimizes the view-pose according to changes in the scene domain. The objective of this contribution is to present an approach to model-based pose initialization and pose refinement steps of an active camera which is able to simultaneously observe and track several objects moving to some extent independently from each other. In both steps, constraints regarding object occlusions, the limited field-of-view, and the uncertainty of pose estimation are exploited to determine an optimal camera pose. The feasibility of this approach to model-based pose initialization is experimentally validated and documented for the case of the manipulation of a shock absorber lid.

The disassembly of engines of used cars based on visually servoed manipulators requires a state-of-the-art vision system which is able to evaluate images of dirty, partly occluded or possibly rearranged quasi- or non-polyhedral parts in real-time. As the pose of a stationary rigid part has, in general, six degrees of freedom, at least six degrees of freedom of a manipulator have to be visually controlled. Two pose estimators relying on different types of image features are discussed within a common framework. A position-based solution to the visually servoed manipulation of non-polyhedrals is presented that extends to real time and meets all given requirements. Experiments with the manipulation of a battery, which is partly occluded by wires, show the feasibility of the proposed approach.

Presents results of a model-based approach to visual tracking and pose estimation for a moving polyhedral tool in position-based visual servoing. This enables the control of a robot in look-and-move mode to achieve six degree of freedom goal configurations. Robust solutions of the correspondence problem-known as "matching" in the static case and "tracking" in the dynamic one-are crucial to the feasibility of such an approach in real-world environments. The object's motion along an arbitrary trajectory in space is tracked using visual pose estimates through consecutive images. Subsequent positions are predicted from robot joint angle measurements. To deal with inaccurate models and to relax calibration requirements, adaptive online calibration of the kinematic chain is proposed. The kinematic predictions enable unambiguous feature matching by a pessimistic algorithm. The performance of the suggested algorithms and the robustness of the proposed system are evaluated on real image sequences of a moving gripper. The results fulfill the requirements of visual servoing, and the computational demands are sufficiently low to allow for real-time implementation.

This contribution presents a position-based approach to visual servoing that allows the manipulation of quasi- or nonpolyhedral objects in a complex scene, namely the engine compartment of a used car. During a manipulation, the moving disassembly robot with its end-effector-mounted tool as well as the-possibly moving-object are observed by an independently mobile, binocular camera configuration. A simple proportional control law is used to guide the robot to its manipulation position. Vision-based tracking as implemented here tolerates mutual occlusions of tool and object. Moreover, it does not require additional feature configurations such as white circular patterns on homogeneous black background to reliably pursue tool and object. We demonstrate simultaneous movements of tool, object and independently mobile camera configuration during 6-DOF closed-loop control.

Our image sequence interpretation process, which generates conceptual descriptions of the behaviour of vehicles in real-world traffic scenes, essentially treated only a single vehicle (the agent) so far. Simultaneous behaviours of other vehicles in the scene have been formulated only relative to the agent. The approach discussed in this contribution allows us to quantify occurrences and thus to generate more global conceptual descriptions of behaviour by using natural language quantifiers. The semantics of such quantified occurrences are represented by special logic structures. Natural language descriptions are derived from these internal knowledge representations.

In order to automatically run a dismantling task, first a workpiece has to be recognized and its pose has to be estimated. This step is solved by a model-based recognition process using the information from three stationary cameras. Our workpiece model could be extracted from CAD-data and consists of planar patches and circular features, where the circular features are suitable to model parts of connection elements like screws. Using the initial estimation of the workpiece pose we control the robot by evaluating the image sequence given by a camera mounted on the robot hand. We use a multi-level 3D tracking approach to continuously estimate the workpiece pose using an iterative extended Kalman-filter process and integrate velocity estimation based on optical flow. To provide enough computing power for real-time image processing, we combine our dedicated image processing hardware with a heterogeneous parallel computer. Our approach has been tested in numerous experiments on loosening screws with a powered screwdriver and dismantling a workpiece consisting of two blocks of different sizes connected by two screws.

Addresses the problem of pose estimation and tracking of vehicles in image sequences from traffic scenes recorded by a stationary camera. In a new algorithm, the vehicle pose is estimated by directly fitting image gradients to polyhedral vehicle models without an edge segment extraction process. The new approach is significantly more robust than approaches that rely on feature extraction because the new approach exploits more information from the image data. We can track vehicles that are partially occluded by textured objects, e.g. foliage, where classical approaches based on edge segment extraction fail. Results from various experiments with real-world traffic scenes are presented.>

The error sensitivity in the estimation of the 3-D motion and the normal of a planar surface from an instantaneous motion field is studied. The statistical theory or the Cramer-Rao lower bound is used for error covariance in the estimated motion and structure parameters. This enables the derivation of results valid for any unbiased estimator under the assumption of Gaussian noise in the motion field. The obtained lower-bound-matrix is studied analytically with respect to measurement noise, size of the field of view, and the motion-geometry configuration. The coupling between translation and rotation is exacerbated if the field of view and the slant of the plane become smaller, and the deviation of the translation from the viewing direction becomes larger. The relationships of the uncertainty bounds for every unknown motion parameter to the angle between translation and the plane-normal, the size of the field of view, and the distance from the perceived plane and the translation magnitude are discussed.>

Images of vehicles which move in traffic scenes recorded by a stationary camera have been detected and tracked without operator intervention. The resulting vehicle trajectories were projected from the image plane onto the street plane. A suitable system internal representation of about 90 German motion verbs was then exploited in order to automatically characterize trajectory segments in terms of natural language concepts. A multiresolution approach for feature matching has been developed which is robust enough to track vehicle images across hundreds of frames, despite considerable variations in size and projected velocity. Results from experiments with image sequences from real-world traffic scenes are presented. >

The commonly used constraint equation Delta g/sup T/u+g/sub t/=0 for the estimation of optical flow can only be justified by assumptions that are, in general, far too restrictive for image sequences of real world scenes. B.G. Schunck (1985, 86) recently argued that a constraint equation for the estimation of what he called image flow has to include a term containing the divergence of this image flow without presenting, however, a stringent derivation based on perspective projection from 3-D scene space. The present author derives a constraint equation based on a combination of perspective projection and notions from differential geometry. In addition, he demonstrates the quantitive effects of taking into account radiometric considerations based on the use of Lambertian reflection properties and isotropic illumination in scene space.>

Many investigations of image sequences can be understood on the basis of a few concepts for which computational approaches become increasingly available. The estimation of optical flow fields is discussed, exhibiting a common foundation for feature-based and differential approaches. The interpretation of optical flow fields is mostly concerned so far with approaches which infer the 3-D structure of a rigid point configuration in 3-D space and its relative motion with respect to the image sensor from an image sequence. The combination of stereo and motion provides additional incentives to evaluate image sequences, especially for the control of robots and autonomous vehicles. Advances in all these areas lead to the desire to describe the spatio-temporal development recorded by an image sequence not only at the level of geometry, but also at higher conceptual levels, for example by natural language descriptions.

A mapping between one frame from an image sequence and the preceding or following frame can be represented as a displacement vector field. In most situations, the mere gray value variations do not provide sufficient information in order to estimate such a displacement vector field. Supplementary constraints are necessary, for example the postulate that a displacement vector field varies smoothly as a function of the image position. Taken as a general requirement, this creates difficulties at gray value transitions which correspond to occluding contours. Nagel therefore introduced the ``oriented smoothness'' requirement which restricts variations of the displacement vector field only in directions with small or no variation of gray values. This contribution reports results of an investigation about how such an ``oriented smoothness'' constraint may be formulated and evaluated.

The count of events where sample areas from the second and subsequent frames of a TV-image sequence are incompatible with the corresponding sample area of the first frame are accumulated in a first-order difference picture (FODP). Analysis of this FODP provides a separate estimate for images of moving objects and of stationary scene components. We start from the hypothesis that the first frame represents the stationary scene component. Once it has been recognized that a subarea of this initial estimate corresponds to the image of a moving object, the grey values in this subarea are replaced by later estimates of the stationary background at this position. No knowledge specific to a particular scene is utilized in the algorithm. The results for two scene sequences are presented.

Abstract There can be a considerable difference between a portable compiler and a usable one which is a high level language implementation with a fair chance of acceptance by the general public in competition with other—vendor supported—systems. The experiences gained in pushing a portable PASCAL compiler across such an acceptance barrier are discussed in detail. Comments on some shortcomings of the PASCAL definition and suggested remedies are included. As a result, it seems feasible to reduce drastically the effort required for a bootstrap by supplementing a bare bone portable compiler with features hitherto considered as luxuries in this context. Only language Implementations which are portable in this broader sense might spread sufficiently easily to survive the competition, especially if the impact of increasingly available access to computer networks is considered.