This paper describes the theory and real-time implementation using an active camera platform of a method of planar trajectory recovery, and of the use of those trajectories to facilitate prediction over delays in the visual feedback loop. Image-based position and velocity demands for tracking are generated by detecting and segmenting optical flow within a central region of the image, and a projective construct is used to map the camera platform's joint angles into a Euclidean coordinate system within a plane, typically the ground plane, in the scene. A set of extended Kalman filters with different dynamics is implemented to analyze the trajectories, and these compete to provide the best description of the motion within an interacting multiple model. Prediction from the optimum motion model is used within the visual feedback loop to overcome visual latency. It is demonstrated that prediction from the 3D planar description gives better tracking performance than prediction based on a filtered description of observer-based 2D motion trajectories.

In this paper we describe the use of a steerable camera and active vision system in the real-time recovery of trajectories of objects moving on a planar surface in the scene. The system first self-calibrates the transformation from camera platform joint angles to scene plane coordinates by active observation of static scene geometry which has been stored as a model. The transformation is encoded as a homography. When distracted by motion, the camera saccades to the object and subsequently tracks it, and the 2D Cartesian components of the motion trajectory are recovered within the scene plane.>

We describe the implementation of, and results from, a realtime active surveillance vision system which detects moving objects in an everyday environment, directs the gaze of a headkye platform towards the objects and subsequently pursues them smoothly. Target detection and pursuit arc performed purely on the basis of image motion, and can continue over extended periods. Two independent parallel processes derive (i) coarse resolution motion over the entire image to direct saccadic shifts in attention over a wide field of view, and (ii) fine resolution motion in a small central region of the image used to perform smooth-pursuit. A gaze controller which selects results from the two visual processes and controls the movement of the head platform is implemented as a finite state machine.

The authors present an active vision system which performs a surveillance task in everyday dynamic scenes. The system is based around simple, rapid motion processors and a control strategy which uses both position and velocity information. The surveillance task is defined in terms of two separate behavioral subsystems, saccade and smooth pursuit, which are demonstrated individually on the system. It is shown how these and other elementary responses to 2D motion can be built up into behavior sequences, and how judicious close cooperation between vision and control results in smooth transitions between the behaviors. These ideas are demonstrated by an implementation of a saccade to smooth pursuit surveillance system on a high-performance robotic hand/eye platform.