Facial keypoint localization is an important step for face recognition. The “Average of Synthetic Exact Filter (ASEF)” approach [2] finds a correlation filter for each training image and averages them together. The resulting classifier is efficient as the filtering can be implemented in the Fourier domain and performance is good for frontal images. However, it cannot cope with a range of poses. In this paper, we generalize this approach to find keypoints using a technique that (i) combines together information from training images in a more principled way than averaging, (ii) can be extended to form non-linear combinations of filters and (iii) can adapt based on context (e.g. pose). These innovations are presented within a greedy boosting-style probabilistic framework. We demonstrate state of the art performance of these algorithms using a challenging data set.

Automatic estimation of head pose faciliates human facial analysis. It has widespread applications such as, gaze direction detection, video teleconferencing and human computer interaction (HCI). It can also be integrated in a multi-view face detection and recognition system. Most current methods estimate pose in a limited range or treat pose as a classification problem by assigning the face to one of many discrete poses [1,2]. Mainly tested on images taken in controlled environments e.g. the FacePix dataset [3] (Fig. 1a).

Advances in object detection have made it possible to collect large databases of certain objects. In this paper we exploit these datasets for within-object classification. For example, we classify gender in face images, pose in pedestrian images and phenotype in cell images. Previous work has mainly targeted the above tasks individually using object specific representations. Here, we propose a general Bayesian framework for within-object classification. Images are represented as a regular grid of non-overlapping patches. In training, these patches are approximated by a predefined library. In inference, the choice of approximating patch determines the classification decision. We propose a Bayesian framework in which we marginalize over the patch frequency parameters to provide a posterior probability for the class. We test our algorithm on several challenging “real world” databases.

Many previous studies have investigated gender classification in well-lit frontal images. In this paper we consider images where the pose, expression and lighting are relatively unconstrained. We localize faces using a standard sliding-window detector. We preprocess the facial region by convolving with Gabor filters at at four scales and four orientations. We sample these responses and concatenate them to form a feature vector. We develop a classifier based on an additive sum of non-linear functions of one-dimensional projections of the data. In particular we investigate arc tangent and weighted sums of Gaussians. We describe a training method based on increasing the binomial log likelihood. We demonstrate that our system on two databases and show that it performs well relative to the state of the art.

Most face recognition algorithms use a "distance- based" approach: gallery and probe images are projected into a low dimensional feature space and decisions about matching are based on distance in this space. In this paper we use a very different representation, where each face is approximated by a regular grid of patches (a mosaicface). Each of these patches is chosen from a library. Faces are now represented as a list of indices to this library. Since there is no obvious way to measure distance between two such lists, we use a probabilistic approach in which the observed face data is explained by a generative model. There are two phases: (i) Learning - we estimate library contents and associated variability (noise), (ii) Recognition - we evaluate the probability that probe and gallery images were generated from the same library patches. Our method performs significantly better than contemporary approaches in the presence of large illumination changes. Variation in viewing conditions is handled by extending this model to learn equivalences between multiple patch appearances. We demonstrate that our method provides a major improvement on the lighting subset of the XM2VTS database compared to "distance-based" methods.