This paper addresses the problem of reconstructing surface models of indoor scenes from sparse 3D scene structure captured from N camera views. Sparse 3D measurements of real scenes are readily estimated from image sequences using structure-from-motion techniques. Currently there is no general method for reconstruction of 3D models of arbitrary scenes from sparse data. We previously introduced an algorithm for recursive integration of sparse 3D structure to obtain a consistent model. In this paper we focus on incorporating uncertainty information into model to achieve reliable reconstruction of real-scenes in the presence of noise. A statistical geometric framework is described that provides a unified approach to probabilistic scene reconstruction from sparse or even dense 3D scene structure.

This work investigates the propagation of errors from the camera motion to the epipolar constraint. A relation between the perturbation of the motion parameters and the error in the epipolar constraint is derived. Based on this relation, the sensitivity of the motion parameters to the epipolar constraint is characterised, and a constraint on the allowed perturbation in a motion parameter in response to a threshold for the produced error in the epipolar constraint is determined. The presented error propagation model is useful to vision systems such as a mobile robot where the camera motion is provided by an odometry sensor. Experimental results on real images are presented.

In this paper we present a geometric theory for reconstruction of surface models from sparse 3D data captured from N camera views which are consistent with the data visibility. Sparse 3D measurements of real scenes are readily estimated from image sequences using structure-from-motion techniques. Currently there is no general method for reconstruction of 3D models of arbitrary scenes from sparse data. We introduce an algorithm for recursive integration of sparse 3D structure to obtain a consistent model. This algorithm is shown to converge to the real scene structure as the number of views increases and to have a computational cost which is linear in the number of views. Results are presented for real and synthetic image sequences which demonstrate correct reconstruction for scenes containing significant occlusions.

We present a new formulation of sequential least-squares applied to scene and motion reconstruction from image features. We argue that recursive techniques will become more important both for real-time control applications and also interactive vision applications. The aim is to approximate as well as possible the result of the batch bundle adjustment method. In previously published work we described an algorithm which works well if the same features are visible throughout the sequence. Here we show how to deal with new features in a way that avoids deterioration of the results. The main theoretical advance here is showing how to adjust the system information matrix when scene/camera parameters are removed from the reconstruction. We show how this procedure affects the sparseness of the information matrix, and thus how to achieve an efficient recursive solution to the reconstruction problem.

For the problem of tracking vehicles on freeways using machine vision, existing systems work well in free-flowing traffic. Traffic engineers, however, are more interested in monitoring freeways when there is congestion, and current systems break down for congested traffic due to the problem of partial occlusion. We are developing a feature-based tracking approach for the task of tracking vehicles under congestion. Instead of tracking entire vehicles, vehicle sub-features are tracked to make the system robust to partial occlusion. In order to group together sub-features that come from the same vehicle, the constraint of common motion is used. In this paper we describe the system, a real-time implementation using a network of DSP chips, and experiments of the system on approximately 44 lane hours of video data.

This paper presents a new technique for calibrating a camera mounted on a controllable head/eye platform. It uses the trajectories of an arbitrary number of tracked corner features to improve the calibration parameter estimates over time, utilizing a novel variable state dimension form of recursive filter. No special visual stimuli are required and no assumptions are made about the structure of the scene, other than that it is stationary relative to the head. The algorithm runs at 4 frames per second on a single Inmos T805 transputer, and is fully integrated into a real-time active vision system. Updated calibration parameters are regularly passed to the vision modules that require them. Although the algorithm requires an initial estimate of camera focal length, results are presented from real experiments demonstrating that convergence is achieved for initial errors up to 50%.

The paper proposes a statistical framework that enables 3D structure and motion to be computed optimally from an image sequence, on the assumption that feature measurement errors are independent and Gaussian distributed. The analysis and results demonstrate that computing both camera/scene motion and 3D structure is essential to computing either with any accuracy. Having computed optimal estimates of structure and motion over a small number of initial images, a recursive version of the algorithm (previously reported) recomputes sub optimal estimates given new image data. The algorithm is designed explicitly for real time implementation, and the complexity is proportional to the number of tracked features. 3D projective, affine and Euclidean models of structure and motion recovery have been implemented, incorporating both point and line features into the computation. The framework can handle any feature type and camera model that may be encapsulated as a projection equation from scene to image.>

We describe the implementation of, and results from, a realtime active surveillance vision system which detects moving objects in an everyday environment, directs the gaze of a headkye platform towards the objects and subsequently pursues them smoothly. Target detection and pursuit arc performed purely on the basis of image motion, and can continue over extended periods. Two independent parallel processes derive (i) coarse resolution motion over the entire image to direct saccadic shifts in attention over a wide field of view, and (ii) fine resolution motion in a small central region of the image used to perform smooth-pursuit. A gaze controller which selects results from the two visual processes and controls the movement of the head platform is implemented as a finite state machine.

The authors demonstrate four real-time reactive responses to movement in everyday scenes using an active head/eye platform. They first describe the design and realization of a high-bandwidth four-degree-of-freedom head/eye platform and visual feedback loop for the exploration of motion processing within active vision. The vision system divides processing into two scales and two broad functions. At a coarse, quasi-peripheral scale, detection and segmentation of new motion occurs across the whole image, and at fine scale, tracking of already detected motion takes place within a foveal region. Several simple coarse scale motion sensors which run concurrently at 25 Hz with latencies around 100 ms are detailed. The use of these sensors are discussed to drive the following real-time responses: (1) head/eye saccades to moving regions of interest; (2) a panic response to looming motion; (3) an opto-kinetic response to continuous motion across the image and (4) smooth pursuit of a moving target using motion alone.>

The authors present an active vision system which performs a surveillance task in everyday dynamic scenes. The system is based around simple, rapid motion processors and a control strategy which uses both position and velocity information. The surveillance task is defined in terms of two separate behavioral subsystems, saccade and smooth pursuit, which are demonstrated individually on the system. It is shown how these and other elementary responses to 2D motion can be built up into behavior sequences, and how judicious close cooperation between vision and control results in smooth transitions between the behaviors. These ideas are demonstrated by an implementation of a saccade to smooth pursuit surveillance system on a high-performance robotic hand/eye platform.