In the face of high-dimensional and complex data, effective subspace can preserve specific statistical properties and provide an appropriate representation of data, which generally facilitates the underlying tasks such as clustering or classification. Meanwhile, multiple kernel learning is a technique to combine multiple kernels from different feature spaces effectively. Thus, by incorporating multiple kernels into the process of subspace learning, different feature spaces can be projected into a unified subspace. This paper proposes the Multiple Kernel Subspace Learning (MKSL) for embedding the original space into a unified subspace. Multiple kernels of different feature spaces are combined by MKSL in the process of learning, which can extend the suitability for various applications. Moreover, to generate the optimal combination kernel of subspace learning, we propose a two-step iteration strategy to learn the appropriate kernel weights and transformation matrix of projecting simultaneously. Furthermore, our proposed formulation of MKSL can introduce different prior knowledge such as class information and neighborhood relationships. Thus it is competent to the unsupervised learning, semi-supervised learning, and supervised learning. Extensive experiments are conducted on diverse datasets, and the performances are comprehensively evaluated on different tasks. The experimental results indicate that the proposed algorithm is outstanding in unsupervised clustering task and effective in supervised and semi-supervised classification tasks.

Multiple empirical kernel learning (MEKL) is a scalable and efficient supervised algorithm based on labeled samples. However, there is still a huge amount of unlabeled samples in the real-world application, which are not applicable for the supervised algorithm. To fully utilize the spatial distribution information of the unlabeled samples, this paper proposes a novel semi-supervised multiple empirical kernel learning (SSMEKL). SSMEKL enables multiple empirical kernel learning to achieve better classification performance with a small number of labeled samples and a large number of unlabeled samples. First, SSMEKL uses the collaborative information of multiple kernels to provide a pseudo labels to some unlabeled samples in the optimization process of the model, and SSMEKL designs pseudo-empirical loss to transform learning process of the unlabeled samples into supervised learning. Second, SSMEKL designs the similarity regularization for unlabeled samples to make full use of the spatial information of unlabeled samples. It is required that the output of unlabeled samples should be similar to the neighboring labeled samples to improve the classification performance of the model. The proposed SSMEKL can improve the performance of the classifier by using a small number of labeled samples and numerous unlabeled samples to improve the classification performance of MEKL. In the experiment, the results on four real-world data sets and two multiview data sets validate the effectiveness and superiority of the proposed SSMEKL.

Support Vector Data Description (SVDD) is an extremely hot topic issue in One-Class Classification (OCC), which has displayed outstanding performance in dealing with many novelty detection problems. However, SVDD just takes the data description by the kernel-based distance among each instance into consideration rather than considering the distribution of the data. Therefore, Fuzzy Support Vector Data Description (Fuzzy-SVDD) has been developed to distribute a fuzzy membership to each input sample so that different samples cause different contributions to classification boundary. The majority of the methods in Fuzzy-SVDD are based on the sample density, but there are remaining two problems. These density-based Fuzzy-SVDD methods would decrease the contribution of support vectors (SVs) in low densities. What is more, these methods cannot get a precise density when there are few target samples. These two problems would lead to a poor classification boundary. To overcome these drawbacks, a novel method called Boundary-based Fuzzy-SVDD (BF-SVDD) is proposed in this paper. BF-SVDD uses a new definition called local–global center distance to search for the samples near the boundary. Then, it enhances fuzzy memberships of these samples because they carry more significant information for the decision boundary than other data. The contribution of this paper can be summarized into three main points. First a novel concept called local–global center distances is proposed to find the SVs better. Second, fuzzy memberships with local–global center distance make SVs more informative to create the decision boundary. Furthermore, the experiments based on University of California, Irvine and Knowledge Extraction based on Evolutionary Learning also show that the proposed method has excellent performances. Even for the minority class in imbalance data sets, the proposed method can also have a good classification.

Sampling method is one of the most commonly used techniques in dealing with imbalanced data. Most of the existing undersampling methods randomly select samples from negative class with replacement. However, it may lose some important information of the training data. Moreover, increasing the positive data by oversampling in high imbalanced situations may cause the overlapping problem. To overcome these problems, this paper proposes a hybrid sampling method. The method takes the distributions of the training data into consideration by the information entropy, thus distinguishing the important samples in the undersampling procedure. Meanwhile, since the positive data only extend to the size of each subset of the negative class in the oversampling, the overlapping problem is relieved. Further, the method retains all the data in the training procedure and generates various data views from the original training data. Then each view is handled with an individual basic classifier. Finally, all the basic classifiers are combined by the ensemble method. The newly proposed method is named as Entropy-based Hybrid Sampling Ensemble Learning (EHSEL). In addition, the EHSEL is applied to three different kinds of basic classifiers to validate its robustness. Experiments results show the great effectiveness of the EHSEL on real-world imbalanced data sets.