Tangent Distance (TD) is one classical method for invariant pattern classification. However, conventional TD need pre-obtain tangent vectors, which is difficult except for image objects. This paper extends TD to more general pattern classification tasks. The basic assumption is that tangent vectors can be approximately represented by the pattern variations. We propose three probabilistic subspace models to encode the variations: the linear subspace, nonlinear subspace, and manifold subspace models. These three models are addressed in a unified view, namely Probabilistic Tangent Subspace (PTS). Experiments show that PTS can achieve promising classification performance in non-image data sets.

Kernel PCA is an efficient method for nonlinear feature extraction. We address two issues in kernel PCA based feature extraction and classification. First, it extracts features without utilizing sample label information. Second, it does not provide a practical means to choose the dimensionality for principal subspace. In this paper, one kind of side-information is incorporated into kernel PCA to solve the first problem. And a complete probabilistic density function is estimated in kernel space so that the choice of dimensionality for principal subspace becomes less important. The proposed model is named probabilistic kernel feature subspace (PKFS). Experiments show that it achieves promising performance and outperforms many other algorithms in classification.

Gaussian mixture model (GMM) is an efficient method for parametric clustering. However, traditional GMM can't perform clustering well on data set with complex structure such as images. In this paper, kernel trick, successfully used by SVM and kernel PCA, is introduced into EM algorithm for solving parameter estimation of GMM, which is so called kernel GMM (kGMM). The basic idea of kernel GMM is to apply kernel based GMM in feature space instead of in input data space. In order to avoid the curse of dimension, the proposed kGMM also embeds a step to automatically select discriminative features in feature space. kGMM is employed for the task of image binarization. Result shows that the proposed approach is feasible.