Clustering on high-dimensional and low sample size (HDLSS) data remains a critical, persistent challenge where extreme sparsity and noise confound cluster analysis. This creates a dilemma: spectral methods fail as distance metrics degrade, while deep clustering tends to over-fit scarce data. To break this dilemma, a Self-Enhanced Density Clustering (SEDC) framework that integrates the cluster structure discovery and embedding representation learning into an iterative enhancement process is proposed in this paper. Specifically, SEDC uses adaptive density-derived centroids to parameterize probabilistic soft labels, which in turn supervise a lightweight multilayer perceptron (MLP) to learn the low-dimensional embedding from data. The resulting embedding provides a refined metric space for further generating superior labels in the subsequent interaction process. This feedback forms a mutual reinforcement that progressively enhances the discrimination of embedding while rigorously mitigating over-fitting. Extensive experiments on 43 challenging HDLSS datasets demonstrate state-of-the-art performance, substantially outperforming popular clustering methods. This work delivers a principled and promising solution for robust data clustering in HDLSS situations.

As data sources become ever more numerous, classification for multi-view data represented by heterogeneous features has been involved in many data mining applications. Most existing methods either directly concatenate all views or separately tackle each view, neglecting the correlation and diversity among views. Moreover, they often encounter an extra hyper-parameter that needs to be manually tuned, degenerating the applicability of models. In this paper, we present a robust supervised learning framework for multi-view classification, seeking a better representation and fusion of multiple views. Specifically, our framework discriminates different views with adaptively optimized view-wise weight factors and coalesces them to learn a joint projection subspace compatible across multiple views in an adaptive-weighting manner, thereby avoiding the intractable hyper-parameter. Meanwhile, the consensus and complementary information of original views can be naturally integrated into the learned subspace, in turn enhancing the discrimination of the subspace for subsequent classification. An efficient convergent algorithm is developed to iteratively optimize the formulated framework. Experiments on real datasets demonstrate the effectiveness and superiority of the proposed method.

Due to the interpretability and robustness, Markov boundary (MB) has received much attention and been widely applied to causal feature selection. However, enormous empirical studies show that, existing algorithms achieve outstanding performance only on the standard Bayesian network data. While on the real-world data, they could not identify some of the relevant features since the large conditioning set and the ignored multivariate dependence lead to performance degradation. In this paper, we propose a tolerant MB discovery algorithm (TLMB), which maps the feature space and target space to a reproducing kernel Hilbert space through the conditional covariance operator, to measure the causal information carried by a feature. Specifically, TLMB uses a score function to filter the redundant features first and then minimize the trace of the conditional covariance operator, where both of the score function and the optimization problem work in the reproducing kernel Hilbert space so that TLMB can select features with not only pairwise dependence but also multivariate dependence. Moreover, as a MB-based method, TLMB can automatically determine the number of selected features due to the property of MB.

Sparse Bayesian learning is a state-of-the-art supervised learning algorithm that can choose a subset of relevant samples from the input data and make reliable probabilistic predictions. However, in the presence of high-dimensional data with irrelevant features, traditional sparse Bayesian classifiers suffer from performance degradation and low efficiency due to the incapability of eliminating irrelevant features. To tackle this problem, we propose a novel sparse Bayesian embedded feature selection algorithm that adopts truncated Gaussian distributions as both sample and feature priors. The proposed algorithm, called probabilistic feature selection and classification vector machine (PFCVM LP ) is able to simultaneously select relevant features and samples for classification tasks. In order to derive the analytical solutions, Laplace approximation is applied to compute approximate posteriors and marginal likelihoods. Finally, parameters and hyperparameters are optimized by the type-II maximum likelihood method. Experiments on three datasets validate the performance of PFCVM LP along two dimensions: classification performance and effectiveness for feature selection. Finally, we analyze the generalization performance and derive a generalization error bound for PFCVM LP . By tightening the bound, the importance of feature selection is demonstrated.

Semi-supervised learning (SSL) concerns the problem of how to improve classifiers’ performance through making use of prior knowledge from unlabeled data. Many SSL methods have been developed to integrate unlabeled data into the classifiers based on either the manifold or cluster assumption in recent years. In particular, the graph-based approaches, following the manifold assumption, have achieved a promising performance in many real-world applications. However, most of them work well on small-scale data sets only and lack probabilistic outputs. In this paper, a scalable graph-based SSL framework through sparse Bayesian model is proposed by defining a graph-based sparse prior. Based on the traditional Bayesian inference technique, a sparse Bayesian SSL algorithm (SBS$^2$L) is obtained, which can remove the irrelevant unlabeled samples and make probabilistic prediction for out-of-sample data. Moreover, in order to scale SBS$^2$L to large-scale data sets, an incremental SBS$^2$L (ISBS$^2$L) is derived. The key idea of ISBS$^2$L is employing an incremental strategy and sequentially selecting parts of unlabeled samples that contribute to the learning instead of using all available unlabeled samples directly. ISBS$^2$L has lower time and space complexities than previous SSL algorithms with the use of all unlabeled samples. Extensive experiments on various data sets verify that our algorithms can achieve comparable classification effectiveness and efficiency with much better scalability. Finally, the generalization error bound is derived based on robustness analysis.