We propose the DPSM method, a density-based node clustering approach that automatically determines the number of clusters and can be applied in both data space and graph space. Unlike traditional density-based clustering methods, which necessitate calculating the distance between any two nodes, our proposed technique determines density through a propagation process, thereby making it suitable for a graph space. In DPSM, nodes are partitioned into small clusters based on propagated density. The partitioning technique has been proved to be sound and complete. We then extend the concept of spectral clustering from individual nodes to these small clusters, while introducing the CluCut measure to guide cluster merging. This measure is modified in various ways to account for cluster properties, thus provides guidance on when to terminate the merging process. Various experiments have validated the effectiveness of DPSM and the accuracy of these conclusions.

K-means algorithm divides samples into c classes based on their structural characteristics. However, due to the non convex nature of the clustering problem, algorithms are prone to converge to poor local minima. To address the aforementioned issues, we propose the Local Structure Preserving Clustering with Learnable Anchors (LSPC-LA) method. We assume that with a well-designed anchor selection strategy, samples near the same anchor tend to belong to the same cluster, which reveal high confidence Must-Link local structural information for clustering. Based on this observation, we first construct an anchor-based bipartite graph, transforming the sample clustering problem into anchor clustering problem by local structural information, thus reducing the solution space and minimizing the risk of poor local minima. Then we create an anchor guiding matrix to allow anchors to learn the sample structure, improving clustering performance. Subsequently, an alternating iterative algorithm is proposed to optimize the LSPC-LA model. Finally, extensive experiments demonstrate the accuracy of the local structural information and the effectiveness of LSPC-LA.

Principal Component Analysis (PCA) is one of the most famous unsupervised dimensionality reduction algorithms and has been widely used in many fields. However, it is very sensitive to outliers, which reduces the robustness of the algorithm. In recent years, many studies have tried to employ \(\ell_{1}\) -norm to improve the robustness of PCA, but they all lack rotation invariance or the solution is expensive. In this article, we propose a novel robust PCA, namely, Fuzzy Weighted Principal Component Analysis (FWPCA), which still uses squared \(\ell_{2}\) -norm to minimize reconstruction error and maintains rotation invariance of PCA. The biggest bright spot is that the contribution of data is restricted by fuzzy weights, so that the contribution of normal samples is much greater than noise or abnormal data, and realizes anomaly detection. Besides, a more reasonable data center can be obtained by solving the optimal mean to make projection matrix more accurate. Subsequently, an effective iterative optimization algorithm is developed to solve this problem, and its convergence is strictly proved. Extensive experimental results on face datasets and RGB anomaly detection datasets show the superiority of our proposed method.

Fuzzy c-means based on entropy regularization (FCER) is a commonly used machine learning algorithm that uses maximum entropy as the regularization term to realize fuzzy clustering. However, this model has many constraints and is challenging to optimize directly. During the solution process, the membership matrix and cluster centers are alternately optimized, easily converging to poor local solutions, limiting the clustering performance. In this paper, we start with the optimization model and propose an unconstrained fuzzy clustering model (UFCER) equivalent to FCER, which reduces the size of optimization variables from$(n+d)\times c$to$d\times c$. More importantly, there is no need to calculate the membership matrix during the optimization process iteratively. The time complexity is only linear, and the convergence speed is fast. We conduct extensive experiments on real datasets. The comparison of objective function value and clustering performance fully demonstrates that under the same initialization, our proposed algorithm can converge to smaller local minimums and get better clustering performance.

Unsupervised domain adaptation aims to classify unlabeled data points in the target domain using labeled data points from the source domain, while the distributions of data points in two domains are different. To address this issue, we propose a novel method called the anchor guided unsupervised domain adaptation method (AGDA). We minimize distribution divergence in a latent feature subspace using the Maximum Mean Discrepancy (MMD) criterion. Unlike existing unsupervised domain adaptation methods, we introduce anchor points in the original space and impose domains data to the same anchor points rather than center points to further reduce the domain difference. We optimize the anchor-based graph in the subspace to obtain discriminative transformation matrices. This enables our model to perform better on non-Gaussian distribution than methods focusing on global structure. Furthermore, the sparse anchor-based graph reduces time complexity compared to the fully connected graph, enabling exploration of local structure. Experimental results demonstrate that our algorithm outperforms several state-of-the-art methods on various benchmark datasets.

Structured proximity matrix learning, one of the mainstream directions in clustering research, refers to learning a proximity matrix with an explicit clustering structure from the original first-order proximity matrix. Due to the complexity of the data structure, the original first-order proximity matrix always lacks some must-links compared to the groundtruth proximity matrix. It is worth noting that high-order proximity matrices can provide missed must-link information. However, the computation of high-order proximity matrices and clustering based on them are expensive. To solve the above problem, inspired by the anchor bipartite graph, we present a novel high-order bipartite graph proximity matrix and a fast method to compute it. This proposed high-order bipartite graph proximity matrix contains high-order proximity information and can significantly reduce the computational complexity of the whole clustering process. Furthermore, we introduce an efficient and simple high-order bipartite graph fusion framework that can adaptively assign weights to each order of the high-order bipartite graph matrices. Finally, under the Laplace rank constraint, a consensus structured bipartite graph proximity matrix is obtained. At the same time, an efficient solution algorithm is proposed for this model. The model's efficacy is underscored through rigorous experiments, highlighting its superior clustering performance and time efficiency. Code available:https://anonymous.4open.science/r/HBGC-F6C4.

Ensemble clustering can utilize the complementary information among multiple base clusterings, and obtain a clustering model with better performance and more robustness. Despite its great success, there are still two problems in the current ensemble clustering methods. First, most ensemble clustering methods often treat all base clusterings equally. Second, the final ensemble clustering result often relies on$k$-means or other discretization procedures to uncover the clustering indicators, thus obtaining unsatisfactory results. To address these issues, we proposed a novel ensemble clustering method based on structured graph learning, which can directly extract clustering indicators from the obtained similarity matrix. Moreover, our methods take sufficient consideration of correlation among the base clusterings and can effectively reduce the redundancy among them. Extensive experiments on artificial and real-world datasets demonstrate the efficiency and effectiveness of our methods.

Graph Neural Networks (GNNs) have emerged as the predominant approach for analyzing graph data on the web and beyond. Contrastive learning (CL), a self-supervised paradigm, not only mitigates reliance on annotations but also has potential in performance. The hard negative sampling strategy that benefits CL in other domains proves ineffective in the context of Graph Contrastive Learning (GCL) due to the message passing mechanism. Embracing the subspace hypothesis in clustering, we propose a method towards expansive and adaptive hard negative mining, referred to as G raph contR astive leA rning via subsP ace prE serving (GRAPE ). Beyond homophily, we argue that false negatives are prevalent over an expansive range and exploring them confers benefits upon GCL. Diverging from existing neighbor-based methods, our method seeks to mine long-range hard negatives throughout subspace, where message passing is conceived as interactions between subspaces. %Empirical investigations back up this strategy. Additionally, our method adaptively scales the hard negatives set through subspace preservation during training. In practice, we develop two schemes to enhance GCL that are pluggable into existing GCL frameworks. The underlying mechanisms are analyzed and the connections to related methods are investigated. Comprehensive experiments demonstrate that our method outperforms across diverse graph datasets and remains competitive across varied application scenarios\footnoteOur code is available at https://github.com/zz-haooo/WWW24-GRAPE. .

Ensemble clustering has emerged as a powerful framework for analyzing heterogeneous and complex data. Despite the abundance of existing schemes, co-association matrix-based methods remain the mainstream approach. However, focusing solely on pairwise correlations falls short of fully capturing the intricate cluster relationships. Moreover, despite its potential, ensemble clustering has yet to effectively leverage the powerful representation capabilities of neural networks. To address these limitations, we propose a deep ensemble clustering method called Ensemble Clustering with Attentional Representation (ECAR). Our method considers the results of base partition as groups with related information to explore higher-order fusion information. ECAR captures the importance of each sample's association with its related group by employing an attentional network, and encodes this information into a low-dimensional representation. The attentional network is trained by jointly optimizing the clustering loss from soft assignments learned from the embeddings and the reconstruction loss from the weighted graph generated from ensemble clustering. During training, the weights of base partitions are adaptively refined to promote diversity and consistency while reducing the impact of low-quality and redundant base partitions. Extensive experimental results on real-world datasets demonstrate the substantial improvement of our method over existing baseline ensemble clustering methods and deep clustering methods.

The key challenge of multi-view graph-based clustering is to mine consistent clustering structures from multiple graphs. Existing works seek clustering decisions from either multiple spectral embeddings or multiple affinity matrices, ignoring the interactions among them. To address this problem, we propose a Bidirectional Attentive Multi-view Clustering (BAMC) model to explore a consensus space w.r.t.spectral embedding and affinity matrix simultaneously, where they can promote each other to mine richer structural information from multiple graphs. BAMC is composed of a Spectral Embedding Learning (SEL) module, an Affinity Matrix Learning (AML) module, and a Bidirectional Attentive Clustering (BAC) module. SEL seeks consensus spectral embeddings by aligning the distributions of elements sampled from subspaces spanned by multiple spectral embeddings. AML learns a consensus affinity matrix from input affinity matrices. BAC guarantees consistency between the learned consensus spectral embeddings and the affinity matrix. To balance their effects, it also assigns adaptive weights to SEL and AML's objective functions. To solve the optimization problem involved in BAMC, we propose an efficient algorithm based on the Majority-Minimization framework with an ingenious surrogate problem. Extensive experiments on several synthetic and real-world datasets demonstrate the superb performance of BAMC.

Feature selection is one of the important topics of machine learning, and it has a wide range of applications in data preprocessing. At present, feature selection based on$\ell _{2,1}$-norm regularization is a relatively mature method, but it is not enough to maximize the sparsity and parameter-tuning leads to increased costs. Later scholars found that the$\ell _{2,0}$-norm constraint is more conductive to feature selection, but it is difficult to solve and lacks convergence guarantees. To address these problems, we creatively propose a novel Outliers Robust Unsupervised Feature Selection for structured sparse subspace (ORUFS), which utilizes$\ell _{2,0}$-norm constraint to learn a structured sparse subspace and avoid tuning the regularization parameter. Moreover, by adding binary weights, outliers are directly eliminated and the robustness of model is improved. More importantly, a Re-Weighted (RW) algorithm is exploited to solve our$\ell _{p}$-norm problem. For the NP-hard problem of$\ell _{2,0}$-norm constraint, we develop an effective iterative optimization algorithm with strict convergence guarantees and closed-form solution. Subsequently, we provide theoretical analysis about convergence and computational complexity. Experimental results on real-world datasets illustrate that our method is superior to the state-of-the-art methods in clustering and anomaly detection tasks.

Feature selection has achieved unprecedented success in obtaining sparse discriminative features. However, the existing methods almost use the$\ell _{2,p}$-norm constraint on transformation matrix to obtain sparse features, which introduces extra parameters and cannot obtain the features directly. In addition, existing algorithms only focused on the global structure and ignored the local structure, leading to poor performance when solving data with non-Gaussian distributions which a single center point cannot describe precisely. Based on above considerations, we propose a supervised feature selection via multi-center and local structure learning. We further introduce trace ratio criterion into our model in favor of improving the discriminant of features selected. In order to address the overlap problem, we use multiple center points to match the distribution of data and construct a$k$-Nearest Neighbor graph to explore the local structure of the data. In addition, we also propose an efficient method to optimize the transformation matrix with the$\ell _{2,0}$-norm constraint and can directly obtain the sparse features. We evaluate our method on Toy datasets and several real-world datasets, show improvement over state-of-the-art feature selection methods, and demonstrate the effectiveness of our model in dealing with non-Gaussian distributed data problems.

In recent years, graph-based multiview clustering methods have become a research hotspot in the clustering field. However, most existing methods lack consideration of cluster balance in their results. In fact, cluster balance is crucial in many real-world scenarios. Additionally, graph-based multiview clustering methods often suffer from high time consumption and cannot handle large-scale datasets. To address these issues, this paper proposes a novel graph-based multiview clustering method. The method is built upon the bipartite graph. Specifically, it employs a label propagation mechanism to update the smaller anchor label matrix rather than the sample label matrix, significantly reducing the computational cost. The introduced balance constraint in the proposed model contributes to achieving balanced clustering results. The entire clustering model combines information from multiple views through graph fusion. The joint graph and view weight parameters in the model are obtained through task-driven self-supervised learning. Moreover, the model can directly obtain clustering results without the need for the two-stage processing typically used in general spectral clustering. Finally, extensive experiments on toy datasets and real-world datasets are conducted to validate the superiority of the proposed method in terms of clustering performance, clustering balance, and time expenditure.

Since obtaining data labels is a time-consuming and laborious task, unsupervised feature selection has become a popular feature selection technique. However, the current unsupervised feature selection methods are facing three challenges: (1) they rely on a fixed similarity matrix derived from the original data, which will affect their performance; (2) due to the limitation of sparsity, they can only obtain sub-optimal solutions; (3) they have high computational complexity and cannot handle large-scale data. To solve this dilemma, we propose a fast unsupervised feature selection algorithm with bipartite graph and 2;0-norm constraint (BGCFS). We use the original data and the selected anchors to construct an adaptive bipartite graph in the subspace, and apply the l2,0-norm constraint to the projection matrix for feature selection. In this way, we can update the adaptive bipartite graph and the projection matrix simultaneously, and we can get the feature subset directly, without sorting the features. In addition, we propose an iterative algorithm that can solve the proposed problem globally to obtain a closed-form solution, and we provide a strict proof of convergence for it. Experiments on eight real data sets with different scales show that our method can select more valuable feature subsets more quickly

Fuzzy clustering algorithms have been widely used to reveal the possible hidden structure of data. However, with the increasing of data amount, large scale data has brought genuine challenges for fuzzy clustering. Most fuzzy clustering algorithms suffer from the long time-consumption problem since a large amount of distance calculations are involved to update the solution per iteration. To address this problem, we introduce the popular anchor graph technique into fuzzy clustering and propose a scalable fuzzy clustering algorithm referred to as Scalable Fuzzy Clustering with Anchor Graph (SFCAG). The main characteristic of SFCAG is that it addresses the scalability issue plaguing fuzzy clustering from two perspectives: anchor graph construction and membership matrix learning. Specifically, we select a small number of anchors and construct a sparse anchor graph, which is beneficial to reduce the computational complexity. We then formulate a trace ratio model, which is parameter-free, to learn the membership matrix of anchors to speed up the clustering procedure. In addition, the proposed method enjoys linear time complexity with the data size. Extensive experiments performed on both synthetic and real world datasets demonstrate the superiority (both effectiveness and scalability) of the proposed method over some representative large scale clustering methods.

K-means is one of the most simple and popular clustering algorithms, which implemented as a standard clustering method in most of machine learning researches. The goal of K-means clustering is finding a set of cluster centers and minimizing the sum of squared distances between each sample and its nearest clustering center. In this paper, we proposed a novel K-means clustering algorithm, which reformulate the classical K-Means objective function as a trace maximization problem and then replace it with a new formulation. The proposed algorithm does not need to calculate the cluster centers in each iteration and requires fewer additional intermediate variables during the optimization process. In addition, we proposed an efficient iterative re-weighted algorithm to solve the involved optimization problem and provided the corresponding convergence analysis. The proposed algorithm keeps a consistent computational complexity as Lloyd's algorithm,$\mathcal {O}(ndk)$, but shows a faster convergence rate in experiments. Extensive experimental results on real world benchmark datasets show the effectiveness and efficiency of the proposed algorithm.

Multi-view clustering attracts considerable attention due to its effectiveness in unsupervised learning. However, previous multi-view spectral clustering methods include two separated steps: 1) Obtaining a spectral embedding; 2) Performing classical clustering methods. Although these methods have achieved promising performance, there is still some limitations. First, in computing spectral embedding, multi-view spectral clustering approaches exist high computational complexity since they usually need eigenvalue decomposition on laplacian matrix L. Its computational complexity is O(n3) where n is the number of samples; Second, in constructing similarity matrices, previous methods need to compute similarity between any two samples; Third, the two-stage approach only can obtain sub-optimal solution; Fourth, treating equally all views is unreasonable. To address these issues, we propose a Fast Multi-view Clustering via Prototype Graph (FMVPG) method. Specifically, the prototype graph is firstly constructed, and then simultaneously perform spectral embedding to obtain the real matrix and spectral rotation to get the indicator matrix. In addition, the Alternating Direction Method of Multipliers (ADMM) is used to solve the joint optimization problem. Further, we conduct extensive experiments to evaluate the proposed FMVPG approach. These experimental results show the comparable or even better clustering performance than the state-of-the-art approaches.

Spectral clustering is a vital clustering method and has been widely applied for data analysis and pattern reorganization. A routine of solving spectral clustering problem consists of two successive stages: (1) solving a relaxed continuous optimization problem to obtain a real-valued indicator solution (2) transform the real-valued indicator into a 0-1 discrete one as the final clustering result. However, we may lose the optimal solution with such a two-stage process. Besides, the spectral clustering has a high time complexity which limits the analysis of large-scale data. To alleviate these problems, this paper proposes an efficient spectral clustering framework that computes spectral embedding and improved spectral rotation simultaneously (SE-ISR). In addition, we also provide a parameter-free method (SE-ISR-PF) to automatically choose the trade-off parameter. Furthermore, with an anchor-based similarity matrix construction, it is scalable to large-scale data. An effective algorithm with a strict convergence proof is provided to solve the corresponding optimization problem. Experimental results on several benchmark datasets demonstrate that the proposed algorithm outperforms the state-of-art methods.

In recent decades, unsupervised feature selection methods have become increasingly popular. Nevertheless, most of the existing unsupervised feature selection methods suffer from two major problems that lead to suboptimal solutions. Many methods impose a hard linear projection constraint on original data, which is overly strict in nature and not suitable for dealing with data sampled from nonlinear manifolds. Second, most existing methods usel2,p-norm (02S and SF2SOG, which can simultaneously learn optimal flexible projections and obtain an orthogonal sparse projection to directly select discriminative features by applyingl2,0-norm constraint. Moreover, we propose to explore the local structure of flexible embedding through preserving the manifold structure of original data and adaptively constructing an optimal graph in subspace. Thirdly, the novel iterative optimization algorithms are presented to solve objective functions guaranteeing convergence theoretically. Various evaluation experiments on synthetic and real-world datasets demonstrate the effectiveness and superiority of our proposed methods.

Graph-based semi-supervised learning, which further utilizes graph structure behind samples for boosting semi-supervised learning, gains convincing results in several machine learning tasks. Nevertheless, existing graph-based methods have shortcomings from two aspects. On the one hand, many of them concentrate on improving label propagation over the constructed graph through time-saving methods, e.g. path searching, without giving insights on constructing a proper graph accommodated to samples. On the other hand, some models are only devoted to constructing the appropriate graph resulting in a two-stage procedure, which may incur a suboptimal scenario. In this paper, we develop a joint learning method that considers both bipartite graph construction and label propagation simultaneously. With this configuration, the constructed graph is constantly adjusted by the smoothness term in the objective as the algorithm proceeds. The time complexity of our method gets significant improvement compared with traditional graph-based methods, and the experimental results on one synthetic dataset and several real-world benchmarks demonstrate the effectiveness and scalability of our proposed method.

Given an affinity graph of data samples, graph-based clustering aims to partition these samples into disjoint groups based on the affinities, and most previous works are based on spectral clustering. However, two problems among spectral-based methods heavily affect the clustering performance. Firstly, the randomness of post-processing procedures, such as$K$-means, affects the stability of clustering. Secondly, the separated stages of spectral-based methods, including graph construction, spectral embedding learning, and clustering decision, lead to mismatched problems. In this paper, we explore a structured graph learning (SGL) framework that aims to fuse these stages to improve clustering stability. Specifically, SGL adaptively learns a structured affinity graph that contains exact$k$connected components. Each connected component corresponds to a cluster so clustering assignments can be directly obtained according to the connectivity of the learned graph. In this way, SGL avoids the randomness brought by reliance on traditional post-processing procedures. Meanwhile, the graph construction and structured graph learning procedures happen simultaneously, which alleviates the mismatched problem effectively. Moreover, we propose an efficient algorithm to solve the involved optimization problems and discuss the connections between this work and previous works. Numerical experiments on several synthetic and real datasets demonstrate the effectiveness of our methods.

Considering many graph clustering methods are with quadratic or cubic time complexity and need post-processing to obtain the discrete solution. Combining with the anchor graph, we study a novel graph clustering model called Rank-$r$Discrete Matrix Factorization (DMF-RR), which is linear time complexity, and motivated by nonnegative matrix factorization (NMF). Instead of constraining the factor matrices of NMF to be nonnegative as many existed methods, we constrain them to indicator matrices. Thus, DMF-RR can obtain the discrete solution by directly solving the original problem without post-processing. Furthermore, considering the greater similarity between samples of the same category, an anchor graph is constructed as an input to capture essential clustering structure by utilizing the duality information between samples and anchors. Subsequently, an efficient and simple algorithm is proposed due to the nature of indicator matrices. Extensive experiments performed on synthetic and real-world datasets demonstrate the superiority of DMF-RR.

Fuzzy K-Means (FKM) clustering regards each cluster as a fuzzy set and assigns each sample to multiple clusters with a certain degree of membership. However, conventional FKM methods perform clustering on original data directly where the intrinsic structure of data may be corrupted by the noise. According, the performance of these methods would be challenged. In this paper, we present a novel fuzzy K-Means clustering model to conduct clustering tasks on the flexible manifold. Technically, we perform fuzzy clustering based on the shrunk patterns which have desired manifold structure. The shrunk patterns can be viewed as an approximation to the original data; and a penalty term is employed to measure the mismatch between them. Moreover, we integrate the learning of shrunk patterns and the learning of membership degree between shrunk patterns and clusters into a unified framework. Furthermore, we extend the proposed model for projected FKM clustering to find a suitable subspace to fit the non-linear manifold structure of data, reduce the interference of the noise and redundant features and gather homogeneous samples together simultaneously. Two alternating iterative algorithms are derived to solve these two models, respectively. Extensive experimental results demonstrate the feasibility and effectiveness of our proposed clustering algorithms.

With its simplicity and effectiveness, k-means is immensely popular, but it cannot perform well on complex nonlinear datasets. Multiple kernel k-means (MKKM) demonstrates the ability to describe highly complex nonlinear separable data structures. However, its speed requirement cannot scale as well as the data size grows beyond tens of thousands. Nowadays, digital data explosion mandates more scalable clustering methods to assist the machine learning tasks in easy-to-access form. To address the issue, we propose to employ the Nystrom scheme for MKKM clustering, termed scalable multiple kernel k-means clustering. It significantly reduces the computational complexity by replacing the original kernel matrix with a low-rank approximation. Analytically and empirically, we demonstrate that our method performs as well as existing state-of-the-art methods, but at a significantly lower compute cost, allowing us to scale the method more effectively for clustering tasks.

Multiple Kernel K-means (MKKM) uses various kernels from different sources to improve clustering performance. However, most of the existing models are non-convex, which is prone to be stuck into bad local optimum, especially with noise and outliers. To address the issue, we propose a novel Self-Paced and Discrete Multiple Kernel K-Means (SPD-MKKM). It learns the MKKM model in a meaningful order by progressing both samples and kernels from easy to complex, which is beneficial to avoid bad local optimum. In addition, whereas existing methods optimize in two stages: learning the relaxation matrix and then finding the discrete one by extra discretization, our work can directly gain the discrete cluster indicator matrix without extra process. What's more, a well-designed alternative optimization is employed to reduce the overall computational complexity via using the coordinate descent technique. Finally, thorough experiments performed on real-world datasets illustrated the excellence and efficacy of our method.

Semi-supervised learning as one of most attractive problems in machine learning research field has aroused broad attentions in recent years. In this paper, we propose a novel locality preserved dimensionality reduction framework, named Semi-supervised Adaptive Local Embedding learning (SALE), which learns a local discriminative embedding by constructing a$k_1$Nearest Neighbors ($k_1$NN) graph on labeled data, so as to explore the intrinsic structure, i.e., sub-manifolds from non-Gaussian labeled data. Then, mapping all samples into learned embedding and constructing another$k_2$NN graph on all embedded data to explore the global structure of all samples. Therefore, the unlabeled data and their corresponding labeled neighbors can be clustered into same sub-manifold, so as to improve the discriminative power of embedded data. Furthermore, we propose two semi-supervised dimensionality reduction methods with orthogonal and whitening constraints based on proposed SALE framework. An efficient alternatively iterative optimization algorithm is developed to solve the NP-hard problem in our models. Extensive experiments conducted on several synthetic and real-world data sets demonstrate the superiorities of our methods on local structure exploration and classification task.

Fuzzy K-Means (FKM) clustering is of great importance for analyzing unlabeled data. FKM algorithms assign each data point to multiple clusters with some degree of certainty measured by the membership function. In these methods, the fuzzy membership degree matrix is obtained based on the calculation of the distance between data points in the original space. However, this operation may lead to suboptimal results because of the influence of noises and redundant features. Besides, some FKM clustering methods ignore the importance of the weighting exponent. In this paper, we propose a novel FKM method called Fuzzy K-Means Clustering With Discriminative Embedding. Within this method, we simultaneously conduct dimensionality reduction along with fuzzy membership degree learning. To retain most information in the embedding subspace and improve the robustness of this method, principal component analysis is incorporated into our framework. An iterative optimization algorithm is proposed to solve the model. To validate the efficacy of the proposed method, we perform comprehensive analyses, including convergence behavior, parameter determination and computational complexity. Moreover, we also match a appropriate weighting exponent for each data set. Experimental results on benchmark data sets show that the proposed method is more discriminative and effective for clustering tasks.

Feature selection is one of the most important techniques to deal with the high-dimensional data for a variety of machine learning and data mining tasks, such clustering, classification, and retrieval, etc. Fuzziness is a widespread nature of data in nature human society. However, most existing feature selection methods ignore the existence of fuzziness in the data, resulting in sub-optimal feature subsets. To address the problem, we propose a novel unsupervised feature selection method, called Unsupervised Discriminative Projection for Feature Selection (UDPFS) to select discriminative features by conducting fuzziness learning and sparse learning, simultaneously. Specifically, we use projection matrix transform data as its low-dimensional representation, which are partitioned into clusters by using membership matrix with sparse constraint. In addition,$\ell _{2, 1}$-norm regularization is applied to the projection matrix. Then, a discriminative projection matrix with row sparse is obtained by perform fuzziness learning and sparse learning, simultaneously. An effective alternative optimization algorithm is proposed to solve the objective function. Evaluate experimental results on several real-world datasets show the effectiveness and superiority of the proposed unsupervised feature selection method.

Multi-task learning has been observed by many researchers, which supposes that different tasks can share a low-rank common yet latent subspace. It means learning multiple tasks jointly is better than learning them independently. In this paper, we propose two novel multi-task learning formulations based on two regularization terms, which can learn the optimal shared latent subspace by minimizing the exactly k minimal singular values. The proposed regularization terms are the more tight approximations of rank minimization than trace norm. But it's an NP-hard problem to solve the exact rank minimization problem. Therefore, we design a novel re-weighted based iterative strategy to solve our models, which can tactically handle the exact rank minimization problem by setting a large penalizing parameter. Experimental results on benchmark datasets demonstrate that our methods can correctly recover the low-rank structure shared across tasks, and outperform related multi-task learning methods.

Recently, with the explosive increase in Internet data, the traditional Graph-based Semi-Supervised Learning (GSSL) model is not suitable to deal with large scale data as the high computation complexity. Besides, GSSL models perform classification on a fixed input data graph. The quality of initialized graph has a great effect on the classification result. To solve this problem, in this paper, we propose a novel approach, named optimal bipartite graph-based SSL (OBGSSL). Instead of fixing the input data graph, we learn a new bipartite graph to make the result more robust. Based on the learned bipartite graph, the labels of the original data and anchors can be calculated simultaneously, which solves co-classification problem in SSL. Then, we use the label of anchor to handle out-of-sample problem, which preserves well classification performance and saves much time. The computational complexity of OBGSSL is O(ndmt+nm2), which is a significant improvement compared with traditional GSSL methods that need O(n2d+n3), where n, d, m and t are the number of samples, features anchors and iterations, respectively. Experimental results demonstrate the effectiveness and efficiency of our OBGSSL model.

Linear discriminant analysis (LDA) is one of commonly used supervised subspace learning methods. However, LDA will be powerless faced with the no-label situation. In this paper, the unsupervised LDA (Un-LDA) is proposed and first formulated as a seamlessly unified objective optimization which guarantees convergence during the iteratively alternative solving process. The objective optimization is in both the ratio trace and the trace ratio forms, forming a complete framework of a new approach to jointly clustering and unsupervised subspace learning. The extension of LDA into Un-LDA enables to not only complete unsupervised subspace learning via the explicitly presented subspace projection matrix but also simultaneously finish clustering and even clustering out-of-sample data via the explicitly presented transformation matrix. To overcome the difficulty in solving the non-convex objective optimization, we mathematically prove that the Un-LDA optimization in both forms can be transformed into the simple K-means clustering optimization when the subspace is determined. The Un-LDA optimization is eventually completed by alternatively optimizing the clusters using K-means and the subspace using the supervised LDA methods and iterating this whole process until convergence or stopping criterion. The experiments demonstrate that our proposed Un-LDA algorithms are comparable or even much superior to the counterparts.

Dimensionality reduction plays a significant role in high-dimensional data processing, and Linear Discriminant Analysis (LDA) is a widely used supervised dimensionality reduction approach. However, a major drawback of LDA is that it is incapable of extracting the local structure information, which is crucial for handling multimodal data. In this article, we propose a novel supervised dimensionality reduction method named Adaptive Local Linear Discriminant Analysis (ALLDA), which adaptively learns a k -nearest neighbors graph from data themselves to extract the local connectivity of data. Furthermore, the original high-dimensional data usually contains noisy and redundant features, which has a negative impact on the evaluation of neighborships and degrades the subsequent classification performance. To address this issue, our method learns the similarity matrix and updates the subspace simultaneously so that the neighborships can be evaluated in the optimal subspaces where the noises have been removed. Through the optimal graph embedding, the underlying sub-manifolds of data in intra-class can be extracted precisely. Meanwhile, an efficient iterative optimization algorithm is proposed to solve the minimization problem. Promising experimental results on synthetic and real-world datasets are provided to evaluate the effectiveness of proposed method.

Semi-supervised learning models for multiview data are important in image classification tasks, since heterogeneous features are easy to obtain and semi-supervised schemes are economical and effective. To model the view importance, conventional graph-based multiview learning models learn a linear combination of views while assuming a priori weights distribution. In this paper, we present a novel structural regularized semi-supervised model for multiview data, termed Adaptive MUltiview SEmi-supervised model (AMUSE). Our new model learns weights from a priori graph structure, which is more reasonable than weight regularization. Theoretical analysis reveals the significant difference between AMUSE and the prior arts. An efficient optimization algorithm is provided to solve the new model. Experimental results on six real-world data sets demonstrate the effectiveness of the structural regularized weights learning scheme.

In many real-world applications, we are often confronted with high dimensional data which are represented by various heterogeneous views. How to cluster this kind of data is still a challenging problem due to the curse of dimensionality and effectively integration of different views. To address this problem, we propose two parameter-free weighted multi-view projected clustering methods which perform structured graph learning and dimensionality reduction simultaneously. We can use the obtained structured graph directly to extract the clustering indicators, without performing other discretization procedures as previous graph-based clustering methods have to do. Moreover, two parameter-free strategies are adopted to learn an optimal weight for each view automatically, without introducing a regularization parameter as previous methods do. Extensive experiments on several public datasets demonstrate that the proposed methods outperform other state-of-the-art approaches and can be used more practically.