Kernel methods have been successfully applied to the areas of pattern recognition and data mining. In this paper, we mainly discuss the issue of propagating labels in kernel space. A Kernel-Induced Label Propagation (Kernel-LP) framework by mapping is proposed for high-dimensional data classification using the most informative patterns of data in kernel space. The essence of Kernel-LP is to perform joint label propagation and adaptive weight learning in a transformed kernel space. That is, our Kernel-LP changes the task of label propagation from the commonly-used Euclidean space in most existing work to kernel space. The motivation of our Kernel-LP to propagate labels and learn the adaptive weights jointly by the assumption of an inner product space of inputs, i.e., the original linearly inseparable inputs may be mapped to be separable in kernel space. Kernel-LP is based on existing positive and negative LP model, i.e., the effects of negative label information are integrated to improve the label prediction power. Also, Kernel-LP performs adaptive weight construction over the same kernel space, so it can avoid the tricky process of choosing the optimal neighborhood size suffered in traditional criteria. Two novel and efficient out-of-sample approaches for our Kernel-LP to involve new test data are also presented, i.e., (1) direct kernel mapping and (2) kernel mapping-induced label reconstruction, both of which purely depend on the kernel matrix between training set and testing set. Owing to the kernel trick, our algorithms will be applicable to handle the high-dimensional real data. Extensive results on real datasets demonstrate the effectiveness of our approach.

We propose a robust inductive semi-supervised label prediction model over the embedded representation, termed adaptive embedded label propagation with weight learning (AELP-WL), for classification. AELP-WL offers several properties. First, our method seamlessly integrates the robust adaptive embedded label propagation with adaptive weight learning into a unified framework. By minimizing the reconstruction errors over embedded features and embedded soft labels jointly, our AELP-WL can explicitly ensure the learned weights to be joint optimal for representation and classification, which differs from most existing LP models that perform weight learning separately by an independent step before label prediction. Second, existing models usually precalculate the weights over the original samples that may contain unfavorable features and noise decreasing performance. To this end, our model adds a constraint that decomposes original data into a sparse component encoding embedded noise-removed sparse representations of samples and a sparse error part fitting noise, and then performs the adaptive weight learning over the embedded sparse representations. Third, our AELP-WL computes the projected soft labels by trading-off the manifold smoothness and label fitness errors over the adaptive weights and the embedded representations for enhancing the label estimation power. By including a regressive label approximation error for simultaneous minimization to correlate sample features with the embedded soft labels, the out-of-sample issue is naturally solved. By minimizing the reconstruction errors over features and embedded soft labels, classification error and label approximation error jointly, state-of-the-art results are delivered.

We propose a new transductive label propagation method, termed Adaptive Neighborhood Propagation (Adaptive-NP) by joint L2,1-norm regularized sparse coding, for semi-supervised classification. To make the predicted soft labels more accurate for predicting the labels of samples and to avoid the tricky process of choosing the optimal neighborhood size or kernel width for graph construction, Adaptive-NP seamlessly integrates sparse coding and neighborhood propagation into a unified framework. That is, the sparse reconstruction error and classification error are combined for joint minimization, which clearly differs from traditional methods that explicitly separate graph construction and label propagation into independent steps, which may result in inaccurate predictions. Note that our Adaptive-NP alternately optimize the sparse codes and soft labels matrices, where the sparse codes are used as adaptive weights for neighborhood propagation at each iteration, so the tricky process of determining neighborhood size or kernel width is avoided. Besides, for enhancing sparse coding, we use the L2,1-norm constraint on the sparse coding coefficients and the reconstruction error at the same time for delivering more accurate and robust representations. Extensive simulations show that our model can deliver state-of-the-art performances on several public datasets for classification.

We propose a Joint nuclear norm based nonlinear Manifold Learning through linear embedding with Classification, called JMLC. By including a feature approximation error into the existing nonlinear manifold learning framework to correlate manifold features with embedded features by a linear projection, the learnt projection can handle the outside points efficiently by embedding. Besides, to encode the neighborhood reconstruction error in manifold learning part, we apply a more reliable nuclear norm based distance metric, since nuclear norm is proved to be more reliable than both L1-norm and Frobenius norm. To make learnt nonlinear features be optimal for classification so that the accuracy can be enhanced, we also minimize a robust L2,1-norm based regressive classification error over the embedded manifold features, which ensures the classification process to be robust to noise and outliers in data. Based on performing joint manifold learning and classification alternately, our JMLC can obtain a low-dimensional embedding, a linear projection and a multi-class classifier simultaneously. Extensive results demonstrate the validity of our proposed algorithm for feature extraction and robust classification, compared with other related models.

We propose a new discriminative dictionary learning framework termed Latent Label Consistent K-SVD (LLC-KSVD) for representing and classifying machine faults. Our LLC-KSVD handles the task by minimizing the reconstruction, discriminative sparse-code and classification errors at the same time. To enhance the representation and classification powers, LLC-KSVD aim to decompose given data into a sparse reconstruction part, a salient feature part and an error part. The salient features are learnt by embedding data onto a projection and a classifier is then trained over extracted salient features so that features are ensured to be optimal for classification. Thus, the classification approach of our LLC-KSVD is very efficient, since there is no need to involve a time-consuming sparse reconstruction process with well-trained dictionary for each test signal as existing models. Besides, to make the classifier be robust to noise and outliers, we regularize the l2,¿-norm on classifier so that the predictions are more accurate. Simulations on several machine fault datasets demonstrate the state-of-the-art performance by our LLC-KSVD.