In many computer vision systems, the same object can be observed at varying viewpoints or even by different sensors, which brings in the challenging demand for recognizing objects from distinct even heterogeneous views. In this work we propose a Multi-view Discriminant Analysis (MvDA) approach, which seeks for a single discriminant common space for multiple views in a non-pairwise manner by jointly learning multiple view-specific linear transforms. Specifically, our MvDA is formulated to jointly solve the multiple linear transforms by optimizing a generalized Rayleigh quotient, i.e., maximizing the between-class variations and minimizing the within-class variations from both intra-view and inter-view in the common space. By reformulating this problem as a ratio trace problem, the multiple linear transforms are achieved analytically and simultaneously through generalized eigenvalue decomposition. Furthermore, inspired by the observation that different views share similar data structures, a constraint is introduced to enforce the view-consistency of the multiple linear transforms. The proposed method is evaluated on three tasks: face recognition across pose, photo versus. sketch face recognition, and visual light image versus near infrared image face recognition on Multi-PIE, CUFSF and HFB databases respectively. Extensive experiments show that our MvDA achieves significant improvements compared with the best known results.

Face recognition with still face images has been widely studied, while the research on video-based face recognition is inadequate relatively, especially in terms of benchmark datasets and comparisons. Real-world video-based face recognition applications require techniques for three distinct scenarios: 1) Videoto-Still (V2S); 2) Still-to-Video (S2V); and 3) Video-to-Video (V2V), respectively, taking video or still image as query or target. To the best of our knowledge, few datasets and evaluation protocols have benchmarked for all the three scenarios. In order to facilitate the study of this specific topic, this paper contributes a benchmarking and comparative study based on a newly collected still/video face database, named COX(1) Face DB. Specifically, we make three contributions. First, we collect and release a largescale still/video face database to simulate video surveillance with three different video-based face recognition scenarios (i.e., V2S, S2V, and V2V). Second, for benchmarking the three scenarios designed on our database, we review and experimentally compare a number of existing set-based methods. Third, we further propose a novel Point-to-Set Correlation Learning (PSCL) method, and experimentally show that it can be used as a promising baseline method for V2S/S2V face recognition on COX Face DB. Extensive experimental results clearly demonstrate that video-based face recognition needs more efforts, and our COX Face DB is a good benchmark database for evaluation.

Due to the misalignment of image features, the performance of many conventional face recognition methods degrades considerably in across pose scenario. To address this problem, many image matching-based methods are proposed to estimate semantic correspondence between faces in different poses. In this paper, we aim to solve two critical problems in previous image matching-based correspondence learning methods: 1) fail to fully exploit face specific structure information in correspondence estimation and 2) fail to learn personalized correspondence for each probe image. To this end, we first build a model, termed as morphable displacement field (MDF), to encode face specific structure information of semantic correspondence from a set of real samples of correspondences calculated from 3D face models. Then, we propose a maximal likelihood correspondence estimation (MLCE) method to learn personalized correspondence based on maximal likelihood frontal face assumption. After obtaining the semantic correspondence encoded in the learned displacement, we can synthesize virtual frontal images of the profile faces for subsequent recognition. Using linear discriminant analysis method with pixel-intensity features, state-of-the-art performance is achieved on three multipose benchmarks, i.e., CMU-PIE, FERET, and MultiPIE databases. Owe to the rational MDF regularization and the usage of novel maximal likelihood objective, the proposed MLCE method can reliably learn correspondence between faces in different poses even in complex wild environment, i.e., labeled face in the wild database.

This paper proposes a novel hierarchical summarization system for surveillance video in which synopsis video and original video are at the lower layer and dynamic collage at the higher layer. A user can efficiently browse the video content through dynamic zooming into collage images and further skimming synopsis video or original video for details, enabling an interactive interface for video navigation. We call this system Dynamic VideoBook because its structure is akin to a book, that is, dynamic collage images resemble the cover and the contents while synopsis video gives an abridged edition of the video. Unlike previous collage techniques for films or entertainment videos, collage for surveillance video is challenging since such video imposes restrictions on spatiotemporal relationship and focus on multiple foreground objects. So we put forward a new collage method which better conveys the spatiotemporal motion and structurally extracts the visual contents from the video simultaneously. And we also enable dynamic zooming in to reveal more information in real-time. In addition, by mapping the upper layer of sketchy visualization to the lower layer of visual data, this system can efficiently explore and browse the video content, highlight one motion or interactively search one specific fragment.

In this paper we propose a simple and effective solution to the important and challenging problem of across-camera person re-identification. We focus on the common case in video surveillance where multiple images or video frames are available for each person. Instead of exploring new features, the proposed approach aims at making a better use of such images/frames. It builds a collaborative representation over all the gallery images (of known person individuals) to best approximate the query images (containing an unknown person) via affine combinations. The approximation is measured by the nearest point distance between the two affine hulls constructed by the query images and gallery images, respectively. By enforcing the sparsity of the samples used for approximating the two nearest points, the relative importance of the gallery images belonging to different persons has the ability to reveal the identity of the querying person. Extensive experiments on public benchmark datasets demonstrate that the proposed approach greatly outperforms the state-of-the-art methods.

Video-based Face Recognition (VFR) can be converted to the matching of two image sets containing face images captured from each video. For this purpose, we propose to bridge the two sets with a reference image set that is well-defined and pre-structured to a number of local models offline. In other words, given two image sets, as long as each of them is aligned to the reference set, they are mutually aligned and well structured. Therefore, the similarity between them can be computed by comparing only the corresponded local models rather than considering all the pairs. To align an image set with the reference set, we further formulate the problem as a quadratic programming. It integrates three constrains to guarantee robust alignment, including appearance matching cost term exploiting principal angles, geometric structure consistency using affine invariant reconstruction weights, smoothness constraint preserving local neighborhood relationship. Extensive experimental evaluations are performed on three databases: Honda, MoBo and YouTube. Compared with competing methods, our approach can consistently achieve better results.

Linear Discriminant Analysis (LDA) is an efficient image feature extraction technique by supervised dimensionality reduction. In this paper, we extend LDA to Structured Sparse LDA (SSLDA), where the projecting vectors are not only constrained to sparsity but also structured with a pre-specified set of shapes. While the sparse priors deal with small sample size problem, the proposed structure regularization can also encode higher-order information with better interpretability. We also propose a simple and efficient optimization algorithm to solve the proposed optimization problem. Experiments on face images show the benefits of the proposed structured sparse LDA on both classification accuracy and interpretability.

Vehicle detection in traffic scenes is a fundamental task for intelligent transportation system and has many practical applications as diverse as traffic monitoring, intelligent scheduling and autonomous navigation. In recent years, the number of detection approaches in monocular images has grown rapidly. However, most of them focus on detecting other objects (such as face, pedestrian, cat, dog, etc.) and also there lacks of vehicle datasets with various conditions for vehicle detection and comprehensive comparisons. To address these problems, we perform an extensive evaluation of many state-of-the-art detection approaches on vehicles. Our main contributions are: (1) we collect a large dataset of real-world vehicles in frontal/rear view with 30° ~ -30° yaw changes and 5° ~ 45° pitch changes under different weather conditions (snowy, rainy, sunny and cloudy) and illumination variations, and then (2) we evaluate six types of state-of-the-art features in Real AdaBoost framework on the adequate dataset collected by ourselves and a public dataset using the same evaluation protocol. Our study presents a fair comparison and deep analysis of these features in vehicle detection. From these experiments, we explore the characteristics of good features for vehicle detection. (3) Finally, we exploit these characteristics and propose a relatively effective and efficient detector, balancing performance, speed and memory cost which can be put into practical use.

Person re-identification is one of the most challenging issues in network-based surveillance. The difficulties mainly come from the great appearance variations induced by illumination, camera view and body pose changes. Maybe influenced by the research on face recognition and general object recognition, this problem is habitually treated as a verification or classification problem, and much effort has been put on optimizing standard recognition criteria. However, we found that in practical applications the users usually have different expectations. For example, in a real surveillance system, we may expect that a visual user interface can show us the relevant images in the first few (e.g. 20) candidates, but not necessarily before all the irrelevant ones. In other words, there is no problem to leave the final judgement to the users. Based on such an observation, this paper treats the re-identification problem as a ranking problem and directly optimizes a listwise ranking function named Mean Reciprocal Rank (MRR), which is considered by us to be able to generate results closest to human expectations. Using a maximum-margin based structured learning model, we are able to show improved re-identification results on widely-used benchmark datasets.

The performance of an offline-trained classifier can be improved on-site by adapting the classifier towards newly acquired data. However, the adaptation rate is a tuning parameter affecting the performance gain substantially. Poor selection of the adaptation rate may worsen the performance of the original classifier. To solve this problem, we propose a conservative model adaptation method by considering the worst case during the adaptation process. We first construct a random cover of the set of the adaptation data from its partition. For each element in the cover (i.e. a portion of the whole adaptation data set), we define the cross-entropy error function in the form of logistic regression. The element in the cover with the maximum cross-entropy error corresponds to the worst case in the adaptation. Therefore we can convert the conservative model adaptation into the classic min-max optimization problem: finding the adaptation parameters that minimize the maximum of the cross-entropy errors of the cover. Taking the object detection as a testbed, we implement an adapted object detector based on binary classification. Under different adaptation scenarios and different datasets including PASCAL, ImageNet, INRIA, and TUD-Pedestrian, the proposed adaption method achieves significant performance gain and is compared favorably with the state-of-the-art adaptation method with the fine tuned adaptation rate. Without the need of tuning the adaptation rates, the proposed conservative model adaptation method can be extended to other adaptive classification tasks.

It is still a challenging work for face recognition to deal with changes due to long term aging. Other than the complicated method using aging models to compensate the aging effect on face recognition, we use the age estimation result as a clue for the database retrieval. The age difference of input image and registered database image is estimated, and then it is used to limit the range of database retrieval and weight the recognition score for sorting. We show that the age estimation results can be used to improve the face recognition accuracy when long term aging effects are necessary to be considered.

In this paper, we propose a method for dot text detection based on FAST points. This problem is different from general scene text detection because of discontinuous text stroke. Unlike many other methods which assume that text is horizontally oriented, our method is able to deal with slant dot text. We extract interesting patches from FAST points and define four features based on the stroke and gray value similarity of dot text to describe a patch. Then, we generate some candidate regions from these patches and utilize SVM to filter out non-dot text ones with the first and second order moments of FAST points in them. Experimental results show that the proposed method is effective and fast to detect dot text.

Crowd counting which aims at obtaining the number of people within a scene is an important computer vision task. While most previous methods try to count people within one frame, this paper addresses this problem using the detection flow which is defined as a set of object detection responses along the temporal video sequence. We argue that counting based on detection flow provides a better way to estimate the crowd size with following merits: 1) it can greatly alleviate the common weakness of an object detector including miss detection and false alarms; 2) it is robust to temporal object occlusions and noises; 3) it is more competent to give specific descriptions of the crowd, e.g. crowd moving directions and target locations. Experiment results on PETS 2009 dataset demonstrate the potential of this method.

Multiple object tracking (MOT) is a very challenging task yet of fundamental importance for many practical applications. In this paper, we focus on the problem of tracking multiple players in sports video which is even more difficult due to the abrupt movements of players and their complex interactions. To handle the difficulties in this problem, we present a new MOT algorithm which contributes both in the observation modeling level and in the tracking strategy level. For the observation modeling, we develop a progressive observation modeling process that is able to provide strong tracking observations and greatly facilitate the tracking task. For the tracking strategy, we propose a dual-mode two-way Bayesian inference approach which dynamically switches between an offline general model and an online dedicated model to deal with single isolated object tracking and multiple occluded object tracking integrally by forward filtering and backward smoothing. Extensive experiments on different kinds of sports videos, including football, basketball, as well as hockey, demonstrate the effectiveness and efficiency of the proposed method.

There is a large number of possible facial attributes such as hairstyle, with/without glasses, with/without mustache, etc. Considering large number of facial attributes and their combinations, it is difficult to build attributes classifiers for all possible combinations needed in various applications, especially at the designing stage. To tackle this important and challenging problem, we propose a novel efficient facial attributes recognition algorithm using a learned spatial codebook. The Maximum Entropy and Maximum Orthogonality (MEMO) criterion is followed to learn the spatial codebook. With a spatial codebook constructed at the designing stage, attribute classifiers can be trained on demand with a small number of exemplars with high accuracy on the testing data. Meanwhile, up to 600 times speedup is achieved in the on-demand training process, compared to current state-of-the-art method. The effectiveness of the proposed method is supported by convincing experimental results.

In this paper, a novel human articulated pose estimation method based on AdaBoost algorithm is presented. The human articulated pose is estimated by locating major human joint positions. We learn the classifiers on a normalized image for classifying each pixel position into a certain category. Two different kinds of classifiers, bottom-up joint position classifier and top-down skeleton classifier, are combined to achieve final results. HOG (Histogram of Oriented Gradient) feature is used for training both classifiers. Our human pose estimation system consists of three models, human detection, view classification, and pose estimation. The implemented system can automatically estimate human pose of different views. Experiment results are reported to show our proposed method can work on relatively small-size human images without using human silhouettes as a prerequisite, which is very efficient, robust and accurate enough for potential applications in visual surveillance.

This paper focuses on the problem of tracking multiple humans in dense environments which is very challenging due to recurring occlusions between different humans. To cope with the difficulties it presents, an offline boosted multi-view upper-body detector is used to automatically initialize a new human trajectory and is capable of dealing with partial human occlusions. What is more, an online learning process is proposed to learn discriminative human observations, including discriminative interest points and color patches, to effectively track each human when even more occlusions occur. The offline and online observation models are neatly integrated into the particle filter framework to robustly track multiple highly interactive humans. Experiments results on CAVIAR dataset as well as many other challenging real-world cases demonstrate the effectiveness of the proposed method.

In this paper, a novel feature named adaptive contour feature (ACF) is proposed for human detection and segmentation. This feature consists of a chain of a number of granules in oriented granular space (OGS) that is learnt via the AdaBoost algorithm. Three operations are defined on the OGS to mine object contour feature and feature co-occurrences automatically. A heuristic learning algorithm is proposed to generate an ACF that at the same time define a weak classifier for human detection or segmentation. Experiments on two open datasets show that the ACF outperform several well-known existing features due to its stronger discriminative power rooted in the nature of its flexibility and adaptability to describe an object contour element.

This paper presents an online detection-based two-stage multi-object tracking method in dense visual surveillances scenarios with a single camera. In the local stage, a particle filter with observer selection that could deal with partial object occlusion is used to generate a set of reliable tracklets. In the global stage, the detection responses are collected from a temporal sliding window to deal with ambiguity caused by full object occlusion to generate a set of potential tracklets. The reliable tracklets generated in the local stage and the potential tracklets generated within the temporal sliding window are associated by Hungarian algorithm on a modified pairwise tracklets association cost matrix to get the global optimal association. This method is applied to the pedestrian class and evaluated on two challenging datasets. The experimental results prove the effectiveness of our method.

Most motion-based tracking algorithms assume that objects undergo rigid motion, which is most likely disobeyed in real world. In this paper, we present a novel motion-based tracking framework which makes no such assumptions. Object is represented by a set of local invariant features, whose motions are observed by a feature correspondence process. A generative model is proposed to depict the relationship between local feature motions and object global motion, whose parameters are learned efficiently by an on-line EM algorithm. And the object global motion is estimated in term of maximum likelihood of observations. Then an updating mechanism is employed to adapt object representation. Experiments show that our framework is flexible and robust in dealing with appearance changes, background clutter, illumination changes and occlusion.

Linear discrimination analysis (LDA) is one of the most popular feature extraction and classifier design techniques. It maximizes the Fisher-ratio between between-class scatter matrix and within-class scatter matrix under a linear transformation, and the transformation is composed of the generalized eigenvectors of them. However, Fisher criterion itself can not decide the optimum norm of transformation vectors for classification. In this paper, we show that actually the norm of the transformation vectors has strong influence on classification performance, and we propose a novel method to estimate the optimum norm of LDA under the ranking loss, re-weighting LDA. On artificial data and real databases, the experiments demonstrate the proposed method can effectively improve the performance of LDA classifiers. And the algorithm can also be applied to other LDA variants such as non parametric discriminant analysis (NDA) to improve theirs performance further.

We developed a fast and robust head pose and gaze estimation system. This system can detect facial feature points and estimate 3D pose angles and gaze direction in various conditions including changes in facial expression, partial occlusion, etc. The system needs only one face image as input and doesn't need any special devices such as blinking LED or stereo camera. Moreover, no calibration process is needed. It shows 95% of head pose estimation accuracy and 81% of gaze estimation accuracy (when the error margin is 15 degrees). The processing time is approximately 15ms/frame (Pentium4 3.2 GHz). Acceptable range of facial pose is within +/- 60 degrees in yaw (left-right) and within +/- 30 degrees in pitch (up-down).

This paper studies face recognition and person-specific face image retrieval in unconstrained environments. The proposed method consists of two parts: offline and online learning. In offline stage, we take advantage of both global and local features in a Bayesian framework for generic face recognition. In online stage, the offline learned classifier is adapted according to the query images of a given person, from which a person-specific face image retriever can be obtained. Our method is applied to the ldquolabeled faces in the wildrdquo dataset, which is more realistic than usual face recognition datasets. We show that the combination of offline and online learning can yield very promising results.

In this paper a novel ranking-based face recognition (FR) scheme is proposed. Compared with classical two-class (intra/extra person) and multi-class (each person a single class) schemes, the ranking-based method only takes into account the most relevant information in training data to find a solution, and therefore is more consistent with the objective of FR. In our approach, given a feature set and its similarity measure, all interested image pairs will be ordered by similarity. The solution to FR then becomes to explore a ranking function that can rank each intra-personal similarity prior to its relevant extra-personal similarities, which can be readily solved by rank boost algorithm. Furthermore in this paper, a logit-rank boost algorithm is proposed which can achieve better recognition performance, and a pruning technique is adopted to deal with the large amount of data that results in further improvement in recognition accuracy. Extensive experimental results on a consumer image collection and the FERET dataset are reported to show the effectiveness of our approach.

Real-time face alignment in video is very critical in many applications such as facial expression analysis, driver fatigue monitoring, etc. This paper presents a real time algorithm for face alignment in video that combines Active Shape Model (ASM) based face alignment and spatial-temporal continuity based tracking strategy. To guarantee the correctness of the tracked shape in each frame, a verification procedure is introduced so that when inter-frame shape tracking failed the intra-frame ASM algorithm can be restored to initialize a new shape for tracking. Experiments show that the implemented system can run totally automatic with a quite good accuracy that may have many practical applications.

For an image, texture unit (TU) is a small complete unit which characterizes the local texture of a given pixel and its neighborhood. Recently, TU-based approaches have been widely used in face recognition. This paper proposes a novel face representation and recognition approach based on TU. We make three major contributions: (1) we introduce a novel TU feature, Local Gabor Quarternary Pattern (LGQP), which incorporates both Gabor magnitude and phase information in a single TU code; (2) similarity measure of two TU images is treated as a tracking problem between two images, and we present a novel point-to-point matching (PPM) approach for TU similarity measure; (3) based on an integral histogram technique, the PPM similarity can be computed very rapidly. Experimental results on CMU-PIE and FERET data sets show that our method is able to reach very promising results.

Face alignment is a critical problem in many face related applications such as facial expression analysis, face recognition, etc. This paper presents a novel real time face alignment algorithm based on active shape model (ASM). In our algorithm, local textures of each label point is used to predict the displacement of each label point by applying boost non-linear regressions on the local search stage of the ASM framework. Experiments on different datasets show that our algorithm is much faster in speed and more robust to the initialization than previous ASM method.

Online Boosting is an effective incremental learning method which can update weak classifiers efficiently according to the object being trackedt. It is a promising technique for online object tracking to adapt tothe appearance variations of objects during tracking process. However, proposed online-boosting based tracking methods update and select weak classifiers from fixed the offline learned weak classifiers, which might not be an optimal selection for object appearance variations. In this paper, we propose a new feature adjusting strategy for online boosting called Soft Decision Feature. We combine it with online real AdaBoost to achieve better tracking performance in scenes with human pose and posture variations. Experiment result demonstrates that it can successfully deal with the human posture variation scenes that conventional online boosting tracking methods fails to deal with.

Tracking objects in low frame rate (LFR) video or with abrupt motion poses two main difficulties which most conventional tracking methods can hardly handle: 1) poor motion continuity and increased search space; 2) fast appearance variation of target and more background clutter due to increased search space. In this paper, we address the problem from a view which integrates conventional tracking and detection, and present a temporal probabilistic combination of discriminative observers of different lifespans. Each observer is learned from different ranges of samples, with different subsets of features, to achieve varying levels of discriminative power at varying cost. An efficient fusion and temporal inference is then done by a cascade particle filter which consists of multiple stages of importance sampling. Experiments show significantly improved accuracy of the proposed approach in comparison with existing tracking methods, under the condition of LFR data and abrupt motion of both target and camera.

Tracking object in low frame rate video or with abrupt motion poses two main difficulties which conventional tracking methods can barely handle: 1) poor motion continuity and increased search space; 2) fast appearance variation of target and more background clutter due to increased search space. In this paper, we address the problem from a view which integrates conventional tracking and detection, and present a temporal probabilistic combination of discriminative observers of different lifespans. Each observer is learned from different ranges of samples, with different subsets of features, to achieve varying level of discriminative power at varying cost. An efficient fusion and temporal inference is then done by a cascade particle filter which consists of multiple stages of importance sampling. Experiments show significantly improved accuracy of the proposed approach in comparison with existing tracking methods, under the condition of low frame rate data and abrupt motion of both target and camera.

Scale invariant feature transform (SIFT) proposed by Lowe has been widely and successfully applied to object detection and recognition. However, the representation ability of SIFT features in face recognition has rarely been investigated systematically. In this paper, we proposed to use the person-specific SIFT features and a simple non-statistical matching strategy combined with local and global similarity on key-points clusters to solve face recognition problems. Large scale experiments on FERET and CAS-PEAL face databases using only one training sample per person have been carried out to compare it with other non person-specific features such as Gabor wavelet feature and local binary pattern feature. The experimental results demonstrate the robustness of SIFT features to expression, accessory and pose variations.

Robust visual tracking is always a challenging but yet intriguing problem owing to the appearance variability of target objects. In this paper we propose a novel method to handle large changes in appearance based on online real-value boosting, which is utilized to incrementally learn a strong classifier to distinguish between objects and their background. By incorporating online real boosting into a particle filter framework, our tracking algorithm shows a strong adaptability for different target objects which undergo severe appearance changes during the tracking process.

In recent years, boosting has been successfully applied to many practical problems in pattern recognition and computer vision fields such as object detection and tracking. As boosting is an offline training process with beforehand collected data, once learned, it cannot make use of any newly arriving ones. However, an offline boosted detector is to be exploited online and inevitably there must be some special cases that are not covered by those beforehand collected training data. As a result, the inadaptable detector often performs badly in diverse and changeful environments which are ordinary for many real-life applications. To alleviate this problem, this paper proposes an incremental learning algorithm to effectively adjust a boosted strong classifier with domain-partitioning weak hypotheses to online samples, which adopts a novel approach to efficient estimation of training losses received from offline samples. By this means, the offline learned general-purpose detectors can be adapted to special online situations at a low extra cost, and still retains good generalization ability for common environments. The experiments show convincing results of our incremental learning approach on challenging face detection problems with partial occlusions and extreme illuminations.

We present a real-time liveness detection approach against photograph spoofing in face recognition, by recognizing spontaneous eyeblinks, which is a non-intrusive manner. The approach requires no extra hardware except for a generic webcamera. Eyeblink sequences often have a complex underlying structure. We formulate blink detection as inference in an undirected conditional graphical framework, and are able to learn a compact and efficient observation and transition potentials from data. For purpose of quick and accurate recognition of the blink behavior, eye closity, an easily-computed discriminative measure derived from the adaptive boosting algorithm, is developed, and then smoothly embedded into the conditional model. An extensive set of experiments are presented to show effectiveness of our approach and how it outperforms the cascaded Adaboost and HMM in task of eyeblink detection.

On-line discovery of some auxiliary objects to verify the tracking results is a novel approach to achieving robust tracking by balancing the need for strong verification and computational efficiency. However, the applicability and effectiveness of this approach highly depend on how to reliably validate the motion correlation between the target and the auxiliary objects so as to estimate the motion model. In this paper, we extend the algorithm of mining auxiliary objects for tracking by incorporating multibody grouping to detect the motion correlation and estimate the motion model, which imposes more general motion correlation constraints. The proposed method discovers the auxiliary objects that exhibit strong affine motion correlation and estimates the closed-form affine models. The proposed tracking algorithm shows good performance in real-world test sequences.

In this paper we propose a domain partitioning RankBoost approach for face recognition. This method uses Local Gabor Binary Pattern Histogram (LGBPH) features for face representation, and adopts RankBoost to select the most discriminative features. Unlike the original RankBoost algorithm in Freund et al. (2003), weak hypotheses in our method make their predictions based on a partitioning of the similarity domain. Since the domain partitioning approach handles the loss function of a ranking problem directly, it can achieve a higher convergence speed than the original approach. Furthermore, in order to improve the algorithm's generalization ability, we introduce some constraints to the weak classifiers being searched. Experiment results on FERET database show the effectiveness of our approach.

Correlation is one of the most widely used similarity measures in machine learning like Euclidean and Mahalanobis distances. However, compared with proposed numerous discriminant learning algorithms in distance metric space, only a very little work has been conducted on this topic using correlation similarity measure. In this paper, we propose a novel discriminant learning algorithm in correlation measure space, Correlation Discriminant Analysis (CDA). In this framework, based on the definitions of within-class correlation and between-class correlation, the optimum transformation can be sought for to maximize the difference between them, which is in accordance with good classification performance empirically. Under different cases of the transformation, different implementations of the algorithm are given. Extensive empirical evaluations of CDA demonstrate its advantage over alternative methods. 1.

Rotation invariant multiview face detection (MVFD) aims to detect faces with arbitrary rotation-in-plane (RIP) and rotation-off-plane (ROP) angles in still images or video sequences. MVFD is crucial as the first step in automatic face processing for general applications since face images are seldom upright and frontal unless they are taken cooperatively. In this paper, we propose a series of innovative methods to construct a high-performance rotation invariant multiview face detector, including the Width-First-Search (WFS) tree detector structure, the Vector Boosting algorithm for learning vector-output strong classifiers, the domain-partition-based weak learning method, the sparse feature in granular space, and the heuristic search for sparse feature selection. As a result of that, our multiview face detector achieves low computational complexity, broad detection scope, and high detection accuracy on both standard testing sets and real-life images.

Many tracking methods face a fundamental dilemma in practice: tracking has to be computationally efficient but verifying if or not the tracker is following the true target tends to be demanding, especially when the background is cluttered and/or when occlusion occurs. Due to the lack of a good solution to this problem, many existing methods tend to be either computationally intensive with the use of sophisticated image observation models, or vulnerable to the false alarms. This greatly threatens long-duration robust tracking. This paper presents a novel solution to this dilemma by integrating into the tracking process a set of auxiliary objects that are automatically discovered in the video on the fly by data mining. Auxiliary objects have three properties at least in a short time interval: (1) persistent co-occurrence with the target; (2) consistent motion correlation with the target; and (3) easy to track. The collaborative tracking of these auxiliary objects leads to an efficient computation as well as a strong verification. Our extensive experiments have exhibited exciting performance in very challenging real-world testing cases.

We developed a fast and robust head pose and gaze estimation system. This system can detect facial points and estimate 3D pose angles and gaze direction under various conditions including facial expression changes and partial occlusion. We need only one face image as input and do not need special devices such as blinking LEDs or stereo cameras. Moreover, no calibration is needed. The system shows a 95% head pose estimation accuracy and 81% gaze estimation accuracy (when the error margin is 15 degrees). The processing time is about 15 ms/frame (Pentium4 3.2 GHz). Acceptable range of facial pose is within a yaw (left-right) of ±60 degrees and within a pitch (up-down) of ±30 degrees.

For applications based on facial image processing, pose variation is a difficult problem. In this paper, we propose a gender and age estimation system that is robust against pose variations. The acceptable facial pose range is a yaw (left-right) from -30 degrees to +30 degrees and a pitch (up-down) from -20 degrees to +20 degrees. According to our experiments on several large databases collected under real environments, the gender estimation accuracy is 84.8% and the age estimation accuracy is 80.9% (subjects are divided into 5 classes). The average processing time is about 70 ms/frame for gender estimation and 95 ms/frame for age estimation (Pentium4 3.2 GHz). The system can be used to automatically analyze shopping customers and pedestrians using surveillance cameras.

Nowadays mobile devices maintain a lot of private information such as payment information, personal photographs and so on. In Japan, people are starting to use cellular phones as means of payment like pre-paid cards. If owners lost these devices and someone else picked it up, what would happen? To solve this problem, password is now widely used. In the case of password, it has to be complicated and long enough from the viewpoint of security, however it is troublesome for users to type such a long password. If someone set very easy password to avoid the troublesome process, it would be of no use. To solve both security and usability problem simultaneously, various types of biometrics have been proposed. Among many biometric solutions, since nowadays almost all of cellular phones have cameras, we propose face recognition that utilize these cameras. Through the verification process, users have only to take their facial photo using a camera equipped on mobile devices and wait for about one second. The system is effective enough to be used in the mobile devices in terms of performance, usability and hardware requirement.

In this paper, we propose a novel tree-structured multiview face detector (MVFD), which adopts the coarse-to-fine strategy to divide the entire face space into smaller and smaller subspaces. For this purpose, a newly extended boosting algorithm named vector boosting is developed to train the predictors for the branching nodes of the tree that have multicomponents outputs as vectors. Our MVFD covers a large range of the face space, say, +/-45/spl deg/ rotation in plane (RIP) and +/-90/spl deg/ rotation off plane (ROP), and achieves high accuracy and amazing speed (about 40 ms per frame on a 320 /spl times/ 240 video sequence) compared with previous published works. As a result, by simply rotating the detector 90/spl deg/, 180/spl deg/ and 270/spl deg/, a rotation invariant (360/spl deg/ RIP) MVFD is implemented that achieves real time performance (11 fps on a 320 /spl times/ 240 video sequence) with high accuracy.

In this paper, a multi-view face pose classification method is introduced. Each face region is normalized by two eye-centers and mouth center, and then multi-class classifiers are trained for face pose classification. The face pose classification is organized into a tree structure that deals with off-image plane pose variation. As a specific application in video, it can estimate the face pose in each frame that results in a face pose variation trajectory, which may have important applications such as in security monitoring of car drivers. Experiment results on a very large set compared with a PCA reconstruction method as a benchmark are reported to show its effectiveness.

We propose a robust face alignment algorithm with a novel discriminative local texture model. Different from the conventional descriptive PCA local texture model in ASM, classifiers using LUT-type Haar-like features are trained from a large data set as local texture model. The strong discriminative power of the classifier greatly improves the accuracy and robustness of local searching on faces with expression variation and ambiguous contours. A Bayesian framework is configured for shape parameter optimization and the algorithm is implemented in a hierarchical structure for efficiency. Extensive experiments are reported to show its accuracy and robustness.

We propose an omni-directional face detection method based on the confidence-rated AdaBoost algorithm, called real AdaBoost, proposed by R.E. Schapire and Y. Singer (see Machine Learning, vol.37, p.297-336, 1999). To use real AdaBoost, we configure the confidence-rated look-up-table (LUT) weak classifiers based on Haar-type features. A nesting-structured framework is developed to combine a series of boosted classifiers into an efficient object detector. For omni-directional face detection, our method has achieved a rather high performance and the processing speed can reach 217 ms per 320/spl times/240 image. Experiment results on the CMU+MIT frontal and the CMU profile face test sets are reported to show its effectiveness.

We propose a framework for face recognition performed in 3D space. A 3D facial model consisting of a sparse depth map is constructed from stereo images using isoluminance lines for the stereo matching. By searching for arcs whose radiuses are of certain ranges, we can locate the candidate irises very efficiently. After the pose of the face is detected, the 3D model is transformed into a canonical pose. Recognition is performed by calculating the mean differences in depth between the corresponding data points in the 3D test model and all the models in the database. The corresponding data points refer to a pair of closest data points in two 3D models. We show that even without using any 2D features, we can do face recognition depending on the depth information only. The 3D pose detecting algorithm makes it possible for our face recognition algorithm to be pose invariant.