When a picture contains non-translational motions in it, a picture-level parametric motion model can be more efficient than the block-based translational motion model because the former has small number of parameters that can replace many motion vectors of individual blocks. In addition, the former can represent the deformation of the image better than the latter. Based on this idea, we detected multiple homography transformations between a reference and a current picture. Then, we generated warped reference pictures1corresponding to the homographies and inserted the one of the warped reference pictures into reference picture lists. We measured the performance of the proposed algorithm using the test condition that is proposed by MPEG and ITU recently. Experimental result showed 3.1% overall bitrate saving under low delay & high efficiency condition and 3.5% overall bitrate saving under random access & high efficiency condition compared with TMuC 0.7.3.

This paper proposes a pose-robust face tracking and facial expression recognition method using a view-based 2D$+$3D active appearance model (AAM) that extends the 2D$+$3D AAM to the view-based approach, where one independent face model is used for a specific view and an appropriate face model is selected for the input face image. Our extension has been conducted in many aspects. First, we use principal component analysis with missing data to construct the 2D$+$3D AAM due to the missing data in the posed face images. Second, we develop an effective model selection method that directly uses the estimated pose angle from the 2D$+$3D AAM, which makes face tracking pose-robust and feature extraction for facial expression recognition accurate. Third, we propose a double-layered generalized discriminant analysis (GDA) for facial expression recognition. Experimental results show the following: 1) The face tracking by the view-based 2D$+$3D AAM, which uses multiple face models with one face model per each view, is more robust to pose change than that by an integrated 2D$+$3D AAM, which uses an integrated face model for all three views; 2) the double-layered GDA extracts good features for facial expression recognition; and 3) the view-based 2D$+$3D AAM outperforms other existing models at pose-varying facial expression recognition.

Faces show both global and local motions, where the former represents rigid head movements due to 3D translation and rotation and the local motion represents non-rigid deformation due to speech, or facial expressions. Although non- rigid face models can represent both types of the facial motions, they are not enough to track the facial motions correctly. The non-rigid face models have large number of model parameters to explain various deformation of the face and the high dimensionality of their model parameter space make them sensitive to initial model parameters, apt to be stuck to local minimum, and difficult to be recovered (re-initialized) from failure when iterative gradient descent optimization techniques are used. To alleviate these problems, we propose to use two types of face trackers that are suitable for estimating the global and local motions, respectively. In the proposed algorithm, the global motion estimator is applied at first and the estimated global motion is used to compute proper initial model parameters of the local motion estimator to make it converge correctly. In this paper, we used active appearance model (MM) and cylinder head model (CHM) as the representative examples of the non- rigid and rigid face models. Experimental results showed that face tracking combining AAMs and CHMs improved the face tracking performance than that of AAMs in terms of 170% higher tracking rate and the 115% wider pose coverage.

This paper proposes a real-time 3D face tracking system. This system uses a stereo active appearance model fitting (STAAM) algorithm that uses multiple calibrated perspective cameras to compute the 3D shape and rigid motion parameters. The use of calibration information reduces the number of model parameters, restricts the degree of freedom in the model parameters, and increases the accuracy and speed of fitting. The real-time face tracking system works alternating two different modes: the detection mode detects the face and eyes in an image and the tracking mode tracks the detected face using the STAAM. In addition, it utilizes a histogram matching to make the system robust to lighting conditions, and uses the motion information to compensate the temporal fluctuation of the captured images. The experimental results show that the proposed system operates robustly under varying lighting conditions and fast in the real-time manner at above 8 frames/sec.

This paper proposes a stereo active appearance model fitting algorithm (STAAM), that fits a 2D+3D active appearance model to stereo images acquired from calibrated perspective cameras. The STAAM uses geometrical relationship between cameras to compute the 3D shape and rigid motion parameters. The use of calibration information reduces the number of model parameters, restricts the degree of freedom in the model parameters, and increases the accuracy and speed of fitting. Also, the STAAM uses a modified simultaneous update fitting method that reduces the fitting computation greatly. Experimental results show that (2) the STAAM shows a better fitting stability than the existing multi-view AAM, (2) the modified simultaneous update algorithm accelerates the AAM fitting speed.

Because the Active Appearance Model (AAM) is sensitive to the initial parameters, it is difficult to track an object that shows a large motion. This paper proposes an active contour combined Active Appearance Model that can track an object whose motion is large. The proposed AAM fitting algorithm consists of two alternating procedures: active contour fitting to find the contour sample that best fits the face image and then the active appearance model fitting that begins from the estimated motion parameters. Experimental results show that the proposed active contour combined AAM provides better accuracy and convergence characteristics in terms of RMS error and convergence rate than the existing AAM. The combination of the existing robust AAM and the proposed active contour based AAM (AC-R-AAM) had the best accuracy and convergence performances.

This paper represents a 3D model-based object tracking algorithm to extract rigid and non-rigid motion of an object simultaneously. Previous AAM is extended by replacing 2D shape model with 3D shape model, and appropriate model fitting algorithm is derived. Proposed algorithm is applied to face tracking in video sequence to extract nonrigidly deforming facial shape.

Since pose-varying face images form nonlinear convex manifold in high dimensional image space, it is difficult to model their pose distribution in terms of a simple probabilistic density function. To solve this difficulty, we divide the pose space into many constituent pose classes and treat the continuous pose estimation problem as a discrete pose-class identification problem. We propose to use a hierarchically structured ML (Maximum Likelihood) pose classifiers in the reduced feature space to decrease the computation time for pose identification, where pose space is divided into several pose groups and each group consists of a number of similar neighboring poses. We use the CONDENSATION algorithm to find a newly appearing face and track the face with a variety of poses in real-time. Simulation results show that our proposed pose identification using the hierarchically structured ML pose classifiers can perform a faster pose identification than conventional pose identification using the flat structured ML pose classifiers. A real-time facial pose tracking system is built with high speed hierarchically structured ML pose classifiers.