This paper presents a method to fit a 3D Morphable Model to a sequence of extracted facial features. The approach is a direct extension of a single view method. The novelty is presented as a new mathematical derivation of the same method but for multiple views where identity and pose are known. The new fitting method exploits point-to-model correspondences and can deal with occlusion since features are not required to correspond across views. The mathematics is explained in detail and the methods single tunable parameter is empirically determined using a database of scanned heads.

This paper presents a fast fitting method for 3D Morphable Models (3DMMs). In most cases fitting a Morphable Model to an image is done using slow non-linear optimization processes. We avoid this by introducing a relationship to Active Appearance Models (AAMs) that can be used to linearize the non-linear optimization problem of 3DMM fitting. Using the combination of a constrained AAM and closed form 2D-3D fitting method for 3DMMs we show that a perceptually close 3D shape can be extracted from the AAM fittings. We show preliminary results of the method and a simplified exterior orientation solution. Finally we conclude with a tracking algorithm that is based on the combination of the AAM and 3DMM, presenting the tracking and reconstruction errors.

Robust, real-time tracking of objects from visual data requires probabilistic fusion of multiple visual cues. Previous approaches have either been ad hoc or relied on a Bayesian network with discrete spatial variables which suffers from discretisation and computational complexity problems. We present a new Bayesian modality fusion network that uses continuous domain variables. The network architecture distinguishes between cues that are necessary or unnecessary for the object's presence. Computationally expensive and inexpensive modalities are also handled differently to minimise cost. The method provides a formal, tractable and robust probabilistic method for simultaneously tracking multiple objects. While instantaneous inference is exact, approximation is required for propagation over time.

Tracking interacting human body parts from a single two-dimensional view is difficult due to occlusion, ambiguity and spatio-temporal discontinuities. We present a Bayesian network method for this task. The method is not reliant upon spatio-temporal continuity, but exploits it when present. Our inference-based tracking model is compared with a CONDENSATION model aug-mented with a probabilistic exclusion mechanism. We show that the Bayesian network has the advantages of fully modelling the state space, explicitly rep-resenting domain knowledge, and handling complex interactions between variables in a globally consistent and computationally effective manner. 1

Tracking multiple people and their behaviours is a fundamental task for visually mediated interaction. We present VIGOUR, a platform for simultaneously tracking of multiple people and recognition of their behaviours for high-level interpretation. Through perceptual integration, different types of visual information are fused to combine the benefits of different techniques. Robust low-level visual cues such as skin colour and motion are used to focus attention and facilitate real-time tracking. VIGOUR tracks behaviours using gestures and head pose to produce a high-level behaviour representation for subsequent interpretation. The system is able to track three people and recognise their gestures simultaneously in real time.

While full computer understanding of dynamic visual scenes containing several people may be currently unattainable, we propose a computationally efficient approach to determine areas of interest in such scenes. We present methods for modelling and interpretation of multi-person human behaviour in real time to control video cameras for visually mediated interaction.

An approach to multi-view face detection based on head pose estimation is presented in this paper. Support vector regression is employed to solve the problem of pose estimation. Three methods, the eigenface method the support vector machine (SVM) based method, and a combination of the two methods, are investigated. The eigenface method, which seeks to estimate the overall probability distribution of patterns to be recognised, is fast but less accurate because of the overlap of confidence distributions between face and non-face classes. On the other hand, the SVM method, which tries to model the boundary of two classes to be classified is more accurate but slower as the number of support vectors is normally large. The combined method can achieve an improved performance by speeding up the computation and keeping the accuracy to a preset level. It can be used to automatically detect and track faces in face verification and identification systems.

The paradigm of perceptual integration provides robust solutions to computer vision problems. By combining the outputs of multiple vision modules, the assumptions and constraints of each module are factored out to result in a more robust system overall. The integration of dierent modules can be regarded as a form of data fusion. To this end, we propose a framework for fusing dierent information sources through estimation of covariance from observations. The framework is demonstrated in a pose tracking method that fuses similarity-to-prototypes measures and skin colour to track head pose and face position. The use of data fusion through covariance introduces constraints that allow the tracker to robustly estimate head pose and track face position. Contact Author Jamie R. Sherrah Contact Address Department of Computer Science Queen Mary and Westeld College Mile End, E1 4NS London UK Contact email [email protected] Contact Tel. (+44) (0)171 975 5230 Contact Fax (+44) (0)181 980 Bri...

Identity-independent estimation of head pose from prototype im-ages is a perplexing task, requiring pose-invariant face detection. The problem is exacerbated by changes in illumination, identity and facial position. Facial images must be transformed in such a way as to em-phasise dierences in pose, while suppressing dierences in identity. We investigate appropriate transformations for use with a similarity-to-prototypes philosophy. The results show that orientation-selective Gabor lters enhance dierences in pose, and that dierent lter ori-entations are optimal at dierent poses. In contrast, PCA was found to provide an identity-invariant representation in which similarities can be calculated more robustly. We also investigate the angular resolution at which pose changes can be resolved using our methods. An angular resolution of 10 was found to be suciently discriminable at some poses but not at others, while 20 is quite acceptable at most poses. 1

Symmetry detection is important in the area of computer vision. A 3D symmetry detection algorithm is presented in this paper. The symmetry detection problem is converted to the correlation of the Gaussian image. Once the Gaussian image of the object has been obtained, the algorithm is independent of the input format. The algorithm can handle different kinds of images or objects. Simulated and real images have been tested in a variety of formats, and the results show that the symmetry can be determined using the Gaussian image.