Head tracking is an important primitive for smart environments and perceptual user interfaces where the poses and movements of body parts need to be determined. Most previous solutions to this problem are based on intensity images and, as a result, suffer from a host of problems including sensitivity to background clutter and lighting variations. Our approach avoids these pitfalls by using stereo depth data together with a simple human torso model to create a head tracking system that is both fast and robust. We use stereo data to derive a depth model of the background which is then employed to provide accurate foreground segmentation. We then use directed local edge detectors on the foreground to find occluding edges which are used as features to fit to a torso model. Once we have the model parameters, the location and orientation of the head can be easily estimated. A useful side effect from using stereo data is the ability to track head movement through a room in three dimensions. Experimental results on real image sequences are given.

The lure of using motion vision as a fundamental element in the perception of space drives this effort to use flow features as the sole cues for robot mobility. Real-time estimates of image flow and flow divergence provide the robot's sense of space. The robot steers down a conceptual corridor, comparing left and right peripheral flows. Large central flow divergence warns the robot of impending collisions at "dead ends". When this occurs, the robot turns around and resumes wandering. Behavior is generated by directly using flow-based information in the 2D image sequence. Active mechanical gaze stabilization simplifies the visual interpretation problems by reducing camera rotation. By combining corridor following and dead-end deflection, the robot has wandered around the lab at 30 cm/s for as long as 20 min without collision. The ability to support this behavior in real-time with current equipment promises expanded capabilities as computational power increases in the future.

A real-time robot vision system is described which uses only the divergence of the optical flow field for both steering control and collision detection. The robot has wandered about the lab at 20 cm/s for as long as 26 minutes without collision. The entire system is implemented on a single ordinary UNIX workstation without the benefit of real-time operating system support. Dense optical flow data are calculated in real-time across the entire wide-angle image. The divergence of this optical flow field is calculated everywhere and used to control steering and collision behavior. Divergence alone has proven sufficient for steering past objects and detecting imminent collision. The major contribution is the demonstration of a simple, robust, minimal system that uses flow-derived measures to control steering and speed to avoid collision in real time for extended periods. Such a system can be embedded in a general, multi-level perception/control system.

This paper is concerned with the task of visual motion-based navigation. A critical requirement of the task is the ability to estimate 3D depth and motion from visual information. Recent studies have demonstrated that the relevant cues are contained in motion parallax or optical flow from which flow field divergence and hence time-to-contact can be extracted. We present a new concept called image gradient evolution, which utilizes the change of image spatial gradients over time as a threat cue; an approaching object induces 2D expanding motion and causes the image spatial structure to stretch so the image gradients decrease. Based on this idea, other method offers a one-step solution directly from image gradients, instead of optical flow and its derived properties. We use a technique that is local and linear so the implementation can be very fast. The threat map is expectedly noisy but sufficiently informative. As seen in demonstrations on several real images these two aspects, fast implementation and useful qualitative information, provide a viable solution to navigational tasks.

The lure of using motion vision as a fundamental element in the perception of space drives this effort to use flow features as the sole cues for robot mobility. Real-time estimates of image flow and flow divergence provide the robot's sense of space. The robot steers down a conceptual corridor comparing left and right peripheral flows. Large central flow divergence warns the robot of impending collisions at "dead ends." When this occurs, the robot turns around and resumes wandering. Behavior is generated by directly using flow-based information in the 2D image sequence; no 3D reconstruction is attempted. Active mechanical gate stabilization simplifies the visual interpretation problems by reducing camera rotation. By combining corridor following and dead-end deflection, the robot has wandered around the lab at 30 cm/s for as long as 20 minutes without collision. The ability to support this behavior in real-time with current equipment promises expanded capabilities as computational power increases in the future.>

An image is ideally a projection of the 3-D scene. However the imaging process is always imperfect and constrained by the physical environment, for example, viewing through a window with reflections. This paper is concerned with image sequences acquired in such situations, the so-called transparency. When it occurs, the image sequence contains undesirable transparent motion, for example, of the window reflections. This complicates the already difficult motion estimation problem. We present an algorithm to segment transparent motion based on a spatio-temporal filtering technique-3-D Hermite polynomial differentiation filters. With motion segmentation accomplished, we can then focus on the scene analysis. The implementation of our algorithm is fast and accurate.

Classic optical flow algorithms assume local image translational motion and apply some primitive image smoothing. Recent studies have taken two separate approaches toward improving accuracy: the application of spatio-temporal filtering schemes and the use of generalized motion models such as the affine model. Each has achieved improvement in its specialized situations. We analyze the interdependency between them and propose a unified theory. The generalized motion we adopt models arbitrary 3D steady motion. Under perspective projection, we derive an image motion equation that describes the spatio-temporal relation in an image sequence, thus making 3D spatio-temporal filtering possible. Hence we establish a theory of Hermite polynomial differentiation filters, whose orthogonality and Gaussian derivative properties ensure numerical stability. The use of higher order motion constraint equations to accommodate more complex motion is justified by the algorithm's reliable performance, as demonstrated by evaluating our algorithm in the scheme established by Barron, et al. (1994).

Both camera fixation and vision-based road following are problems that involve tracking or fixating on 3-D points and features. This paper presents a unified theoretical approach to analyzing camera fixation and vision-based road following. The approach is based on the concept of equal flow circles (EFCs) and zero flow circles (ZFCs). Using EFCs it is possible to locate points in space relative to the fixation point, and predict the behavior. The camera's instantaneous direction of translation and the fixation point determine the plane on which the EFCs can be found. We show that points on an EFC inside the ZFC produce optical flow that is opposite in sign to that produced by points outside the ZFC. When a point in space crosses a ZFC it produces zero flow. For explanation purposes we analyzed a special case of motion. However, a similar approach can be taken for a more general motion of the camera. The analysis for the current motion can also be extended to find equal flow curves.>

A technique for the calibration of an active camera system is presented. The calibration of manipulator, camera-to-manipulator, camera, and base-to-world is treated in a unified and elegant way. In this approach, the camera frames and manipulator link frames are all related to the world frame, therefore the camera-to-manipulator and base-to-world calibration is very straightforward. The approach is simple, since it uses the form of one equation solving one parameter. Two experiments that verify the accuracy of the technique are reported.>

To operate autonomous vehicles safely, obstacles must be detected before any path planning and obstacle avoidance activity is undertaken. In the paper, a novel approach to obstacle detection is developed. New visual linear invariants based on optical flow have been developed. Employing the linear invariance property, obstacles an be directly detected by using a reference flow line obtained from measured optical flow. This method can be used for ground vehicles to navigate through man-made roadways or natural outdoor terrain or for air vehicles to land on known or unknown terrain.>

A fixation point is a point in 3-D space that projects to zero optical flow in an image over some period of time while the camera is moving. Quantitative aspects of fixation for a static scene are treated. For the case where the rotation axis of the camera is perpendicular to the instantaneous translation vector, it is shown that there is an infinite number of points that produce zero instantaneous optical flow. These points lie on a circle (called the zero flow circle, or ZFC) and a line. The ZFC changes its location and radius as a function of time, and the intersection of all the ZFCs is a fixation point. Points inside the ZFC produce optical flow that is opposite in sign to those that are outside the ZFC. This fact explains in a more quantitative way phenomena due to fixation. In particular, points in the neighborhood of the fixation point may change the sign of their optical flow as the camera moves. In a set of experiments, it is shown how the concept of the ZFC can be used to explain the optical flow produced by 3-D points near the fixation point.>

The US National Bureau of Standard's multiple autonomous underwater vehicles (MAUV) project involves the development of a real-time intelligent control system that performs sensing, world modeling, planning, and execution for underwater robot vehicles. The goal of the project is to have multiple vehicles exhibiting intelligent, autonomous, cooperative behavior. Initial tests have involved two identical vehicles engaged in various scenarios in Lake Winnipesaukee in New Hampshire. All software for controlling the vehicles reside on computer boards mounted onboard the vehicles.>

Issues dealing with fast, 3-D, collision-free motion planning are discussed, and a fast path planning system under development at NBS is described. The components of a general motion planner are outlined, and some of their computational aspects are discussed. It is argued that an octree representation of the obstacles in the world leads to fast path planning algorithms. The system we are developing uses such an octree representation. The robot and its swept-volume paths are approximated by primitive shapes so as to result in fast collision detection algorithms. The search for a path is performed in the octree space, and combines hypothesize and test, hill climbing, and A.

Techniques for extracting detailed, complete descriptions of polyhedral objects from light-stripe rangefinder data are described. The descriptions are in the form of 3D faces, edges, vertices, and their topology and geometry. A range image is first segmented into edge points. A line drawing is then obtained by fitting linear segments to the points in the image, and refining the segments to eliminate gaps. Faces are then generated from the line drawing. Interestingly, although the final description is in 3D. most of the processing is done in the 2D image space. This paper describes one part of a larger project whose goal is to obtain a full symbolic description of a scene from range data obtained from multiple viewpoints.

We describe the current state of the 3-D Mosaic project, whose goal is to incrementally acquire a 3-D model of a complex urban scene from images. The notion of incremental acquisition arises from the observations that 1) single images contain only parfial information about a scene, 2) complex images are difficult to fully interpret, and 3) different features of a given scene tend to be easier to extract in different images because of differences in viewpoint and lighting conditions. In our approach, multiple images of the scene are sequentially analyzed so as to incrementaly construct the model. Each new image provides information which refines the model. We describe some experiments toward this end. Our method of extracting 3-D shape information from the images is stereo analysis. Because we are dealing with urban scenes, a junction-based matching technique proves very useful. This technique produces rather sparse wire-frame descriptions of the scene. A reasoning system that relies on task-specific knowledge generates an approximate model of the scene from the stereo output. Gray scale information is also acquired for the faces in the model. Finally, we describe an experiment in combining two views of the scene to obtain a rermed model.