This paper proposes a novel method that preserves the geometrical structure created by variation of multiple factors in analysis of multiple factor models, i.e., multifactor analysis. We use factor-dependent submanifolds as constituent elements of the factor-dependent geometry in a multiple factor framework. In this paper, a submanifold is defined as some subset of a manifold in the data space, and factor-dependent submanifolds are defined as the submani-folds created for each factor by varying only this factor. In this paper, we show that MPCA is formulated using factor-dependent submanifolds, as is our proposed method. We show, however, that MPCA loses the original shapes of these submanifolds because MPCA's parameterization is based on averaging the shapes of factor-dependent subman-ifolds for each factor. On the other hand, our proposed multifactor analysis preserves the shapes of individual factor-dependent submanifolds in low-dimensional spaces. Because the parameters obtained by our method do not lose their structures, our method, unlike MPCA, sufficiently covers original factor-dependent submanifolds. As a result of sufficient coverage, our method is appropriate for accurate classification of each sample.

We propose the use of a multifactor model that extends Grassmann manifold to multiple factor frameworks. Both manifold learning algorithms and multifactor analysis are state-of-the-art dimension reduction techniques that are suitable to model variations of face images. In this paper, we demonstrate that Grassmann manifold can be extended to Mul-tifactor Grassmann manifold when used in conjunction with Multilinear PCA (MPCA). Indeed, the multifactor manifold learning algorithm proposed in this paper can be interpreted as MPCA's kernel-based extension using a kernel function that is defined in terms of geodesic distance. As a result, we first propose the use of Multifactor Grassmann manifold, which can learn both a multifactor structure and an underlying manifold in a given set of face images. We then demonstrate that our proposed method, Multifactor Grassmann manifold, produces more reliable results in the context of face recognition than the traditional dimension reduction techniques.

Lately, Multilinear Principal Component Analysis (MPCA) has been successfully applied to face recognition since MPCA provides analysis of multiple factors of face images such as people's identities, viewpoints, and lighting conditions. MPCA employees multiple linear subspaces constructed by varying factors. In this paper, we propose nonlinear submanifold analysis, which can represent the variation of each factor more accurately than the conventional multilinear subspace analysis. Based on submanifold learning, we propose an extension of the multiple factor analysis. This paper proposes the kernel-based extension of MPCA whose definition of a kernel function and neighbors of each sample is robust for submanifold learning. The experimental results in this paper demonstrate that the proposed methods produce a synergetic advantage for face recognition. This is because our method offers the combined virtues of both multifactor analysis and manifold learning.

A novel method for robust super-resolution of face images is proposed in this paper. Face super-resolution is a particular interest in video surveillance where face images have typically very low-resolution quality and there is a need to apply face enhancement or super-resolution algorithms. In this paper, we apply a manifold learning method which has hardly been used for super-resolution. A manifold is a natural generalization of a Euclidean space to a locally Euclidean space. Manifold learning algorithms are more powerful than other pattern recognition methods which analyze a Euclidean space because they can reveal the underlying nonlinear distribution of the face space; however, there are some practical problems which prevent these algorithms from being applied to super-resolution. Almost all of the manifold learning methods cannot generate mapping functions for new test images which are absent from a training set. Another factor is that super-resolution seeks to recover a high-dimensional image from a lower-dimensional one while manifold learning methods perform the exact opposite as they are applied to dimensionality reduction. In this paper, we break the limitation of applying manifold learning methods for face super-resolution by proposing a novel method using locality preserving projections (LPP).

Facial images change appearance due to multiple factors such as different poses, lighting variations, and facial expressions. Tensors are higher order extensions of vectors and matrices, which make it possible to analyze different appearance factors of facial variation. Using higher order tensors, we can construct a multilinear structure and model the multiple factors of face variation. In particular, among the appearance factors, the factor of a person's identity modeled by a tensor structure can be used for face recognition. However, this tensor-based face recognition creates difficulty in factorizing the unknown parameters of a new test image and solving for the person-identity parameter. In this paper, to break this limitation of applying the tensor-based methods to face recognition, we propose a novel tensor approach based on an individual-modeling method and nonlinear mappings. The proposed method does not require the problematic tensor factorization and is more efficient than the traditional TensorFaces method with respect to computation and memory. We set up the problem of solving for the unknown factors as a least squares problem with a quadratic equality constraint and solve it using numerical optimization techniques. We show that an individual-multilinear approach reduces the order of the tensor so that it makes face-recognition tasks computationally efficient as well as analytically simpler. We also show that nonlinear kernel mappings can be applied to this optimization problem and provide more accuracy to face-recognition systems than linear mappings. In this paper, we show that the proposed method, Individual Kernel TensorFaces, produces the better discrimination power for classification. The novelty in our approach as compared to previous work is that the Individual Kernel TensorFaces method does not require estimating any factor of a new test image for face recognition. In addition, we do not need to have any a priori knowledge of or assumption about the factors of a test image when using the proposed method. We can apply Individual Kernel TensorFaces even if the factors of a test image are absent from the training set. Based on various experiments on the Carnegie Mellon University Pose, Illumination, and Expression database, we demonstrate that the proposed method produces reliable results for face recognition.