Multi-kernel k-means clustering (MKC) aims to learn a composite kernel from multiple precomputed basic kernels to better reflect the data distribution. In the existing MKC models, the optimal composite kernel is linearly combined by basic kernels with varying weights, subject to different constraints. While some state-of-the-art models have achieved satisfactory clustering performance, they often do so at the expense of model interpretability. To address this issue, this paper proposes a new Multi-Kernel K-means Clustering model with maximized Entropy regularization (MKKC-E). In the new model, convex combination of basic kernels is used to learn the optimal composite kernel to enhance interpretability. Meanwhile, an entropy regularization term is introduced to prevent the kernel weights from becoming overly sparse, thereby improving the model’s robustness. Experimental results demonstrate that the proposed model’s interpretability and robustness are validated on synthetic datasets, while its superior clustering performance is confirmed on benchmark datasets. In conclusion, the MKKC-E model not only achieves excellent clustering performance but also offers significant interpretability.

K-Means algorithm is a popular clustering method. However, it has two limitations: 1) it gets stuck easily in spurious local minima, and 2) the number of clusters$k$has to be given a priori. To solve these two issues, a multi-prototypes convex merging based K-Means clustering algorithm (MCKM) is presented. First, based on the structure of the spurious local minima of the K-Means problem, a multi-prototypes sampling (MPS) is designed to select the appropriate number of multi-prototypes for data with arbitrary shapes. Then, a merging technique, called convex merging (CM), merges the multi-prototypes to get a better local minima without$k$being given a priori. Specifically, CM can obtain the optimal merging and estimate the correct$k$. By integrating these two techniques with K-Means algorithm, the proposed MCKM is an efficient and explainable clustering algorithm for escaping the undesirable local minima of K-Means problem without given$k$first. Two theoretical proofs are given to guarantee that the cost of MCKM (MPS+CM) can achieve a constant factor approximation to the optimal cost of the K-Means problem. Experimental results performed on synthetic and real-world data sets have verified the effectiveness of the proposed algorithm.

Based on Hoeffding’s inequality, many popular regression and classification models in supervised learning relax the expected risk minimization problem to the empirical risk minimization problem . Nevertheless, the recent theoretical results disclose that the bound of Bernstein’s inequality - which includes variance information - is often significantly tighter than the bound of Hoeffding’s inequality. In this paper, based on the empirical Bernstein bound, we proposed a risk-averse learning machine, which can achieve better generalization performance by trading off good loss performance (approximation error) and small variance (estimation error) as well as suitable complexity of the model. We prove that the resulting learning machine is tractable for many popular loss functions. Moreover, to solve our model which is mostly non-convex because of the square root term , we introduce an extra variable to get rid of the square root and obtain an optimization problem that is convex in most cases for two variables respectively. Then Newton’s method can be used to solve it alternately. The experimental results on artificial and benchmark datasets demonstrate that the proposed models can achieve better performance compared to other existing empirical risk minimization models.

Fuzzy C-Means (FCM) is a widely used clustering method. However, FCM and its many accelerated variants have low efficiency in the mid-to-late stage of the clustering process. In this stage, all samples are involved in updating their non-affinity centers, and the membership grades of most samples, whose assignments remain unchanged, are still updated by calculating the sample-center distances. All these factors lead to the algorithms converging slowly. In this paper, a new affinity filtering technique is developed to recognize a complete set of non-affinity centers for each sample with low computations. Then, a new membership scaling technique is suggested to set the membership grades between each sample and its non-affinity centers to 0 and maintain the fuzzy membership grades for others. By integrating these two techniques, FCM based on new affinity filtering and membership scaling (AMFCM) is proposed to accelerate the whole convergence process of FCM. Numerous experimental results performed on synthetic and real-world data sets have shown the feasibility and efficiency of the proposed algorithm. Compared with state-of-the-art algorithms, AMFCM is significantly faster and more effective. For example, AMFCM reduces the number of FCM iterations by 80$\%$on average.

Fuzzy c-means (FCM) is one of the most frequently used methods for clustering. However, with increasing amount of data, FCM suffers from slow convergence and a large amount of calculation because all samples are involved in updating the solutions per iteration without considering the current clustering results. In this article, a new membership scaling FCM (MSFCM) is proposed, based on the observation that the samples, whose nearest cluster center isv, aid the convergence ofv, whereas the remaining samples prevent the convergence ofv. In the new algorithm, many samples whose nearest cluster centers do not change in the next iteration are chosen by using the triangle inequality. A new scheme for scaling the membership degrees of the chosen samples is suggested to boost the effect of the in-cluster samples and to weaken the effect of the out-of-cluster samples in the clustering process. The new scheme not only accelerates the convergence of the algorithm but also maintains the high clustering quality. Many experimental results on synthetic and real-world data sets have verified the effectiveness of the proposed algorithm in improving the speed of the convergence of the fuzzy clustering. In particular, compared with FCM, MSFCM saves at least two thirds of the total rounds of iterations without significantly increasing the cost per iteration.

Two-dimensional principal component analysis (2DPCA), as a state-of-the-art method for dimensionality reduction, has been widely used in face recognition. However, it is very sensitive to outliers since it minimizes the sum of squared F-norm, which is least-squares loss in nature. Recently, angle 2DPCA was presented to alleviate this problem by minimizing the sum of Fnorm, which is corresponding to L1 loss. But a vital unresolved problem of angle 2DPCA is that it needs many more coefficients for image representation because it works only in the row direction. In this letter, we first give a new angle 2DPCA called Sin-2DPCA by minimizing the relative error, which has a better explanation than the original one. Furthermore, in order to obtain better performance with fewer reduced coefficients, we project the input image to a lower dimension from right and left simultaneously, and then, the bilateral angle 2DPCA (BA2DPCA) is proposed. The experimental results on two benchmark face recognition datasets with outlier noises illustrate that the Sin-2DPCA has the similar performance with original angle 2DPCA, and BA2DPCA can obtain the highest performance in all compared algorithms with the minimal number of representation coefficients.

For support vector machine (SVM) learning, least squares SVM (LSSVM), derived by duality LSSVM (D-LSSVM), is a widely used model, because it has an explicit solution. One obvious limitation of the model is that the solution lacks sparseness, which limits it from training large-scale problems efficiently. In this paper, we derive an equivalent LSSVM model in primal space LSSVM (P-LSSVM) by the representer theorem and prove that P-LSSVM can be solved exactly at some sparse solutions for problems with low-rank kernel matrices. Two algorithms are proposed for finding the sparse (approximate) solution of P-LSSVM by Cholesky factorization. One is based on the decomposition of the kernel matrix K as P P(T) with the best low-rank matrix P approximately by pivoting Cholesky factorization. The other is based on solving P-LSSVM by approximating the Cholesky factorization of the Hessian matrix with rank-one update scheme. For linear learning problems, theoretical analysis and experimental results support that P-LSSVM can give the sparsest solutions in all SVM learners. Experimental results on some large-scale nonlinear training problems show that our algorithms, based on P-LSSVM, can converge to acceptable test accuracies at very sparse solutions with a sparsity level <1%, and even as little as 0.01%. Hence, our algorithms are a better choice for large-scale training problems.