Face recognition is one of the most intensively studied topics in computer vision and pattern recognition, but few are focused on how to robustly recognize faces with expressions under the restriction of one single training sample per class. A constrained optical flow algorithm, which combines the advantages of the unambiguous correspondence of feature point labeling and the flexible representation of optical flow computation, has been developed for face recognition from expressional face images. In this paper, we propose an integrated face recognition system that is robust against facial expressions by combining information from the computed intraperson optical flow and the synthesized face image in a probabilistic framework. Our experimental results show that the proposed system improves the accuracy of face recognition from expressional face images.

Face recognition is one of the most intensively studied topics in computer vision and pattern recognition. A constrained optical flow algorithm, which combines the advantages of the unambiguous correspondence of feature point labeling and the flexible representation of optical flow computation, has been developed for face recognition from expressional face images. In this paper, we propose an integrated face recognition system that is robust against facial expressions by combining information from the computed intra-person optical flow and the synthesized face image in a probabilistic framework. Our experimental results show that the proposed system improves the accuracy of face recognition from expressional face images.

Face recognition is one of the most intensively studied topics in computer vision and pattern recognition, but few are focused on how to robustly recognize expressional faces with one single training sample per class. In this paper, we modify the regularization-based optical flow algorithm by imposing constraints on some given point correspondences to compute precise pixel displacements and intensity variations. By using the optical flow computed for the input expression variant face with respect to a reference neutral face image, we remove the expression from the face image by elastic image warping to recognize the subject with facial expression. Experimental validation is given to show that the proposed expression normalization algorithm significantly improves the accuracy of face recognition on expression variant faces.

Face recognition is one of the most intensively studied topics in computer vision and pattern recognition. A constrained optical flow algorithm, which combines the advantages of the unambiguous correspondence of feature point labeling and the flexible representation of optical flow computation has been proposed in our pervious work. Facial expression normalization, from expressive to neutral facial images, based on optical flow analysis is discussed in this paper. In addition, we propose a new algorithm for nonnegative coefficient projection algorithm for projecting optical flow onto a facial expression subspace. Experimental validation is given to show that the proposed systems improve the accuracy of face and expression recognition on expressional face images.

The performance of a face recognition system degrades incredibly due to the variation of facial appearance with different pose, which is well known as one of the bottlenecks in face recognition. One of the possible solutions is generating virtual frontal view from any given non-frontal view to obtain a virtual face. The ideal solution is to reconstruct a 3D model from the input images and synthesize the virtual image with corresponding pose, which might be too complex to be implemented in a real-time application. By formulating this kind of solutions as a nonlinear pose normalization problem, we will propose an algorithm integrating the nonlinearity of kernel function and the efficiency of linear regression method, which modifies the linear assumption in local linear regression (LLR) method and makes the solution more resembling to the ideal one. Some discussions and experiments on CMU PIE database are carried out, and show that our proposed method performs well.

Virtual avatars in many applications are constructed manually or by a single speech-driven model which needs a lot of training data and long training time. It’s an essential problem to build up a user-dependent model more efficiently. In this paper, a new adaptation method, called the partial linear regression (PLR), is proposed and adopted in an audio-driven talking head application. This method allows users to adapt the partial parameters from the available adaptive data while keeping the others unchanged. In our experiments, the PLR algorithm can retrench the hours of time spent on retraining a new user dependent model, and adjust the user-independent model to a more personalized one. The animated results with adapted models were 36% closer to the user-dependent model than using the pre-trained user-independent model.

Most of the facial animation applications, such as automatic face detection, recognition, and tracking are sensitive to the different lighting conditions and complicated background environment, especially for accurate expression analysis in virtual conferencing system. The main work in this research is to detect the faces under these complex situations and reduce the influence of different lighting. In addition, the pose estimation is the bottleneck of the whole framework, the speed of which is to be improved. An adaptive skin color model, scanning over the downsampled skin map, a modified mean shift algorithm and a new skin mask are used to refine and speed up the head tracking procedure. The lighting distribution is estimated by a second order function, and the lighting effect is compensated. Finally, we modify the good feature selection algorithm to the texture map to pick out the features, and then perform the analysis-by-synthesis pose estimation with the texture map of good features to speed up without losing accuracy.

Fast, reliable, and marker-free facial expression analysis still remains to be a difficult task in computer vision research. In this paper, the concept of speech-assisted facial expression analysis and synthesis is proposed, which shows that the speech-driven facial animation technique not only can be used for expression analysis. From the input speech, the mouth shape can is estimated from the audio-visual model. Thus, the large search space of mouth appearance is reduced for mouth tracking. Similarly, the modeling technique is extended from modeling speech and mouth shape to facial movements and detail facial texture changes. In this way, a virtual conferencing system with video realistic avatars is realized to meet real-time requirement.