Anchor graph clustering has gained significant attention due to its effectiveness and efficiency. As the most representative points in original data, anchors are applied to connect the sample space to label space. However, when noise is present in original data, the anchor-refined label spreading mechanism may fail. To address this, we propose an Efficient Co-clustering via Anchor-refined Label Spreading (ECALS), which simultaneously clusters original data and anchors. We introduce the size constraint that ensures each cluster contains a minimum number of samples. Our method includes two variants, which are continuous and discrete model, catering to both fuzzy and discrete label matrices. Both models are applicable to out-of-sample problems and demonstrate superior performance on synthetic and real-world datasets.

Anchor-based clustering methods have gained attention for their efficiency in subspace, multi-view, and ensemble clustering tasks. Most existing methods focus on using anchors to reduce computational complexity in the original data space. However, clustering directly on anchors, followed by label propagation to the original data, can significantly improve computational efficiency. In this paper, we propose an Efficient Anchor Graph Clustering (EAGC) method that maximizes within-cluster homogeneity among anchors. Inspired by the relaxation and discretization model in spectral clustering, we propose two corresponding models, namely EAGC-R and EAGC-D. EAGC-R first obtains relaxed spectral embedding of anchors and then the embedding is discretized by K-Means. EAGC-D directly solves the discrete anchor membership matrix by coordinate descent method. Once anchor clustering results are obtained, original data labels can be obtained through anchor label transmission. Extensive experiments conducted on both synthetic and real datasets illustrate the effectiveness of proposed methods.

Fuzzy C-Means algorithm (FCM) is one of the most commonly used fuzzy clustering algorithm, which uses the alternating optimization algorithm to update the membership matrix and the cluster center matrix. FCM achieves effective results in clustering tasks. However, due to many constraints, the objective function is inconvenient to optimize directly and is prone to converges to a suboptimal local minimum, which affects the clustering performance. In this paper, we propose a minimization problem equivalent to FCM. Firstly, we use the optimal solution when fixing the cluster center matrix to replace the membership matrix, transforming the original constrained optimization problem into an unconstrained optimization problem, thus reducing the number of variables. We then use gradient descent instead of alternating optimization to solve the model, so we call this model UC-FCM. Extensive experimental results show that UC-FCM can obtain better local minimum and achieve superior clustering performance compared to FCM under the same initialization. Moreover, UC-FCM is also competitive compared with other advanced clustering algorithms.

As a compromise between supervised and unsupervised learning, semi-supervised learning (SSL) harnesses both labeled and unlabeled data to enhance learning performance. Graph-based semi-supervised learning (GSSL) has emerged as a prominent approach owing to its versatility in representing sample interdependencies via graph structures. However, traditional GSSL methods face high time cost when computing matrix inverses, making them inefficient for large datasets. To address this, some researchers have introduced anchors as a bridge to accelerate the process. Nevertheless, most anchor-based models suffer from one or more of the following issues: (1) The anchor graph-based construction of the adjacency matrix has limitations; (2) The objective functions are typically non-convex, leading to local optima and requiring multiple runs to achieve good performance. To tackle these challenges, we develop a probability-driven approach to build the adjacency matrix, defining sample similarity as the probability of sharing the same anchor. Based on this strategy, we design a model (CFSL) with a strictly convex objective function, guaranteeing a globally optimal solution without iterative optimization. Experiments on multiple datasets indicate that our algorithm yields strong performance.

This paper proposes a deep pseudo-label method for unsupervised feature selection, which learns non-linear representations to generate pseudo-labels and trains a Neural Network (NN) to select informative features via self-Knowledge Distillation (KD). Specifically, the proposed method divides a standard NN into two sub-components: an encoder and a predictor, and introduces a dependency subnet. It works by self-supervised pretraining the encoder to produce informative representations and then alternating between two steps: (1) learning pseudo-labels by combining the clustering results of the encoder's outputs with the NN's prediction outputs, and (2) updating the NN's parameters by globally selecting a subset of features to predict the pseudo-labels while updating the subnet's parameters through self-KD. Self-KD is achieved by encouraging the subnet to locally capture a subset of the NN features to produce class probabilities that match those produced by the NN. This allows the model to self-absorb the learned inter-class knowledge and evaluate feature diversity, removing redundant features without sacrificing performance. Meanwhile, the potential discriminative capability of a NN can also be self-excavated without the assistance of other NNs. The two alternate steps reinforce each other: in step (2), by predicting the learned pseudo-labels and conducting selfKD, the discrimination of the outputs of both the NN and the encoder is gradually enhanced, while the self-labeling method in step (1) leverages these two improvements to further refine the pseudo-labels for step (2), resulting in the superior performance. Extensive experiments show the proposed method significantly outperforms state-of-the-art methods across various datasets.

Anchor-based clustering methods have attracted increasing attention due to their ability to provide efficient and scalable solutions in clustering tasks, such as subspace, multi-view and ensemble clustering. Nevertheless, the majority of anchor-based methods view anchors merely as tools, concentrating on diminishing computational complexity within original data space. However, in fact, clustering can be directly performed on anchors and then the anchor clustering results could be propagated to original data. Due to the much smaller volume of anchors, this could significantly reduce the computational complexity of clustering algorithms. Building upon this idea, in this paper, we propose a fast anchor graph clustering method (FAGC) via maximizing within-cluster similarity. Inspired by the relaxation and discretization model in spectral clustering, we also propose two corresponding models, namely FAGC-R and FAGC-D. FAGC-R first obtains spectral embedding of anchors and then discretizes the embedding to obtain anchor indicator matrix. While FAGC-D directly solves the discrete anchor membership matrix. Once anchor clustering results are obtained, original data labels can be obtained through anchor label transmission. Extensive experiments conducted on synthetic and real datasets illustrate the effectiveness and efficiency of the proposed methods.

In practical applications, multi-view subspace clustering is hindered by data noise that disrupts the ideal block-diagonal structure of self-representation matrices, thereby degrading performance. Moreover, many existing methods rely solely on sample features, overlooking the valuable structural information in affinity matrices (e.g., pairwise relationships). While conventional contrastive learning strategies often introduce false negative pairs due to noise and unreliable sample selection. To address these challenges, we propose a pseudo-label guided bidirectional discriminative deep multi-view subspace clustering method (PBDMSC). Our approach first employs pseudo-label guided contrastive learning, using previous cluster assignments to select reliable positive and negative samples, which mitigates incorrect pairings and enhances low-dimensional representations. Then, a discriminative self-representation learning method is introduced that leverages pseudo-labels to enforce homogeneous expression constraints and incorporates a bidirectional attention mechanism to preserve the structured information from affinity matrices, thereby enhancing robustness. Experimental results on six real-world datasets demonstrate that our proposed method achieves state-of-the-art clustering performance.

Boosting is a well-established ensemble learning approach that aims to enhance overall performance by combining multiple weak learners with a linear combination structure. It operates on the principle of using new learners to compensate for the shortcomings of previous learners and is known for its ability to reduce computational resource requirements while mitigating the risks of overfitting. However, from the perspective of convex optimization, it becomes apparent that classical boosting methods often converge to local optima rather than global optima when minimizing the target loss due to its greedy strategy. In this article, we address the issue and propose a novel optimization framework for the boosting paradigm. Our framework focuses on refining the ensemble model by further minimizing loss function through the reallocation of base learner weights, which results in a more robust and powerful learner. We have conducted experiments on various real-world and synthetic datasets, and our findings confirm that our Reweighted-Boosting model consistently outperforms its counterparts. It also exhibits an increased classification margin for the data, making it a valuable enhancement to original boosting algorithms.

Many recent research works on unsupervised feature selection (UFS) have focused on how to exploit autoencoders (AEs) to seek informative features. However, existing methods typically employ the squared error to estimate the data reconstruction, which amplifies the negative effect of outliers and can lead to performance degradation. Moreover, traditional AEs aim to extract latent features that capture intrinsic information of the data for accurate data recovery. Without incorporating explicit cluster structure-detecting objectives into the training criterion, AEs fail to capture the latent cluster structure of the data which is essential for identifying discriminative features. Thus, the selected features lack strong discriminative power. To address the issues, we propose to jointly perform robust feature selection and -means clustering in a unified framework. Concretely, we exploit an AE with a -norm as a basic model to seek informative features. To improve robustness against outliers, we introduce an adaptive weight vector for the data reconstruction terms of AE, which assigns smaller weights to the data with larger errors to automatically reduce the influence of the outliers, and larger weights to the data with smaller errors to strengthen the influence of clean data. To enhance the discriminative power of the selected features, we incorporate -means clustering into the representation learning of the AE. This allows the AE to continually explore cluster structure information, which can be used to discover more discriminative features. Then, we also present an efficient approach to solve the objective of the corresponding problem. Extensive experiments on various benchmark datasets are provided, which clearly demonstrate that the proposed method outperforms state-of-the-art methods.

The affinity graph is regarded as a mathematical representation of the local manifold structure. The performance of locality-preserving projections (LPPs) and its variants is tied to the quality of the affinity graph. However, there are two drawbacks in current approaches. First, the pre-designed graph is inconsistent with the actual distribution of data. Second, the linear projection way would cause damage to the nonlinear manifold structure. In this article, we propose a nonlinear dimensionality reduction model, named deep locality-preserving projections (DLPPs), to solve these problems simultaneously. The model consists of two loss functions, each employing deep autoencoders (AEs) to extract discriminative features. In the first loss function, the affinity relationships among samples in the intermediate layer are determined adaptively according to the distances between samples. Since the features of samples are obtained by nonlinear mapping, the manifold structure can be kept in the low-dimensional space. Additionally, the learned affinity graph is able to avoid the influence of noisy and redundant features. In the second loss function, the affinity relationships among samples in the last layer (also called the reconstruction layer) are learned. This strategy enables denoised samples to have a good manifold structure. By integrating these two functions, our proposed model minimizes the mismatch of the manifold structure between samples in the denoising space and the low-dimensional space, while reducing sensitivity to the initial weights of the graph. Extensive experiments on toy and benchmark datasets have been conducted to verify the effectiveness of our proposed model.

Clustering aims to partition a set of objects into different groups through the internal nature of these objects. Most existing methods face intractable hyper-parameter problems triggered by various regularization terms, which degenerates the applicability of models. Moreover, traditional graph clustering methods always encounter the expensive time overhead. To this end, we propose a Fast Clustering model with Anchor Guidance (FCAG). The proposed model not only avoids trivial solutions without extra regularization terms, but is also suitable to deal with large-scale problems by utilizing the prior knowledge of the bipartite graph. Moreover, the proposed FCAG can cope with out-of-sample extension problems. Three optimization methods Projected Gradient Descent (PGD) method, Iteratively Re-Weighted (IRW) algorithm and Coordinate Descent (CD) algorithm are proposed to solve FCAG. Extensive experiments verify the superiority of the optimization method CD. Besides, compared with other bipartite graph models, FCAG has the better performance with the less time cost. In addition, we prove through theory and experiment that when the learning rate of PGD tends to infinite, PGD is equivalent to IRW.

Clustering is a fundamental topic in machine learning and various methods are proposed, in which K-Means (KM) and min cut clustering are typical ones. However, they may produce empty or skewed clustering results, which are not as expected. In KM, the constrained clustering methods have been fully studied while in min cut clustering, it still needs to be developed. In this paper, we propose a parameter-insensitive min cut clustering with flexible size constraints. Specifically, we add lower limitations on the number of samples for each cluster, which can perfectly avoid the trivial solution in min cut clustering. As far as we are concerned, this is the first attempt of directly incorporating size constraints into min cut. However, it is a NP-hard problem and difficult to solve. Thus, the upper limits is also added in but it is still difficult to solve. Therefore, an additional variable that is equivalent to label matrix is introduced in and the augmented Lagrangian multiplier (ALM) is used to decouple the constraints. In the experiments, we find that the our algorithm is less sensitive to lower bound and is practical in image segmentation. A large number of experiments demonstrate the effectiveness of our proposed algorithm.

With the development of information technology, a large number of multiview data has emerged, which makes multiview clustering algorithms considerably attractive. Previous graph-based multiview clustering methods usually contain two steps: obtaining the fusion graph or spectral embedding of all views; and performing clustering algorithms. The two-step process cannot obtain optimal results since the two steps cannot negotiate with each other. To address this drawback, a novel algorithm named as multi-view fuzzy clustering based on anchor graph is presented. The proposed method can simultaneously obtain the membership matrix and minimize the disagreement rates of different views. A novel regularization based on trace norm is also presented in this article, which can not only obtain a clear clustering partition to prevent that all samples belonging to each cluster with the same membership value$\frac{1}{c}$, but also balance the size of each cluster. Moreover, we exploit the reweighted method to optimize the proposed model, which can introduce an adaptive weight to each view to deal with the unreliable views. A series of experiments are conducted on different datasets, and the clustering performance verifies the effectiveness and efficiency of the proposed algorithm.

In this paper, we propose a novel algorithm to solve the row-sparse principal component analysis problem without relying on any data structure assumption. Sparse principal component analysis was proposed to improve the interpretability of principal component analysis by restricting the number of non-zero elements in each loading vector, but the varying sparsity patterns among different leading vectors may result in trouble on some occasions, such as feature selection. Then row-sparse principal component analysis was proposed, which demands the same sparse pattern among different loading vectors. However, the optimization of row-sparse principal component analysis problems is NP-hard. Although some algorithms were proposed to solve this problem, but they are only applicable to the specific data structure. In this paper, we transform the original row-sparse principal component analysis problem into a new equivalent problem that can be solved by coordinate descent method without relying on any data structure assumption. Then by carefully eliminating redundant structures to avoid repeating computation, we propose a more efficient coordinate descent method to solve this problem. Furthermore, no parameter needs to be tuned in our algorithm. Finally, extensive experiments are conducted on the real world data sets to demonstrate the superiority of our algorithm.

Fuzzy C-means (FCM) is a kind of classic cluster method, which has been widely used in various fields, such as image segmentation and data mining. Euclidean distance is a frequently used distance metric in FCM, but it is only suitable for data with spherical structure. As an extension of Euclidean distance, Mahalanobis distance has been used in Gustafson–Kessel FCM and its variants to tackle ellipsoidal data. For the convenience of optimizing, most existing Mahalanobis distance-based FCM algorithms only focus on squared Mahalanobis distance. However, squared Mahalanobis distance may not be the best distance metric for FCM because it is easy to enlarge the influence of outliers. In this article, we propose a novel$\ell _{2,p}$-norm and Mahalanobis distance-based FCM model, abbreviated as LM-FCM, which can help FCM improve the ability of tackling ellipsoidal clusters and outliers. Then, in order to reduce computational complexity, we propose a more simplified model, abbreviated as SLM-FCM. Furthermore, we develop an iteratively reweighted optimization algorithm to optimize the proposed models and provide a rigorous monotonous convergence proof. Finally, compared with the existing state-of-the-art FCM algorithms, we conduct extensive experiments on both synthetic and real-world datasets to manifest the superior clustering performance and robustness of the proposed algorithms.

Graph-based subspace learning has been widely used in various applications as the rapid growth of data dimension, while the graph is constructed by affinity matrix of input data. However, it is difficult for these subspace learning methods to preserve the intrinsic local structure of data with the high-dimensional noise. To address this problem, we proposed a novel unsupervised dimensionality reduction approach named unsupervised subspace learning with flexible neighboring (USFN). We learn a similarity graph by adaptive probabilistic neighborhood learning process to preserve the manifold structure of high-dimensional data. In addition, we utilize the flexible neighboring to learn projection and latent representation of manifold structure of high-dimensional data to remove the impact of noise. The adaptive similarity graph and latent representation are jointly learned by integrating adaptive probabilistic neighborhood learning and manifold residue term into a unified objection function. The experimental results on synthetic and real-world datasets demonstrate the performance of the proposed unsupervised subspace learning USFN method.

The rapid growth of the number of data brings great challenges to clustering, especially the introduction of multi-view data, which collected from multiple sources or represented by multiple features, makes these challenges more arduous. How to clustering large-scale data efficiently has become the hottest topic of current large-scale clustering tasks. Although several accelerated multi-view methods have been proposed to improve the efficiency of clustering large-scale data, they still cannot be applied to some scenarios that require high efficiency because of the high computational complexity. To cope with the issue of high computational complexity of existing multi-view methods when dealing with large-scale data, a fast multi-view clustering model via nonnegative and orthogonal factorization (FMCNOF) is proposed in this paper. Instead of constraining the factor matrices to be nonnegative as traditional nonnegative and orthogonal factorization (NOF), we constrain a factor matrix of this model to be cluster indicator matrix which can assign cluster labels to data directly without extra post-processing step to extract cluster structures from the factor matrix. Meanwhile, the F-norm instead of the L2-norm is utilized on the FMCNOF model, which makes the model very easy to optimize. Furthermore, an efficient optimization algorithm is proposed to solve the FMCNOF model. Different from the traditional NOF optimization algorithm requiring dense matrix multiplications, our algorithm can divide the optimization problem into three decoupled small size subproblems that can be solved by much less matrix multiplications. Combined with the FMCNOF model and the corresponding fast optimization method, the efficiency of the clustering process can be significantly improved, and the computational complexity is nearly O(n) . Extensive experiments on various benchmark data sets validate our approach can greatly improve the efficiency when achieve acceptable performance.

Linear discriminant analysis (LDA) is one of commonly used supervised subspace learning methods. However, LDA will be powerless faced with the no-label situation. In this paper, the unsupervised LDA (Un-LDA) is proposed and first formulated as a seamlessly unified objective optimization which guarantees convergence during the iteratively alternative solving process. The objective optimization is in both the ratio trace and the trace ratio forms, forming a complete framework of a new approach to jointly clustering and unsupervised subspace learning. The extension of LDA into Un-LDA enables to not only complete unsupervised subspace learning via the explicitly presented subspace projection matrix but also simultaneously finish clustering and even clustering out-of-sample data via the explicitly presented transformation matrix. To overcome the difficulty in solving the non-convex objective optimization, we mathematically prove that the Un-LDA optimization in both forms can be transformed into the simple K-means clustering optimization when the subspace is determined. The Un-LDA optimization is eventually completed by alternatively optimizing the clusters using K-means and the subspace using the supervised LDA methods and iterating this whole process until convergence or stopping criterion. The experiments demonstrate that our proposed Un-LDA algorithms are comparable or even much superior to the counterparts.

The large-scale hyperspectral image (HSI) clustering problem has attracted significant attention in the field of remote sensing. Most traditional graph-based clustering methods still face challenges in the successful application of the large-scale HSI clustering problem mainly due to their high computational complexity. In this letter, we propose a novel approach, called fast spectral clustering with anchor graph (FSCAG), to efficiently deal with the large-scale HSI clustering problem. Specifically, we consider the spectral and spatial properties of HSI in the anchor graph construction. The proposed FSCAG algorithm first constructs anchor graph and then performs spectral analysis on the graph. With this, the computational complexity can be reduced to O(ndm), which is a significant improvement compared to conventional graph-based clustering methods that need at least O(n2d), where n, d, and m are the number of samples, features, and anchors, respectively. Several experiments are conducted to demonstrate the efficiency and effectiveness of the proposed FSCAG algorithm.

The locality preserving projections (LPP) algorithm is a recently developed linear dimensionality reduction algorithm that has been frequently used in face recognition and other applications. However, the projection matrix in LPP is not orthogonal, thus creating difficulties for both reconstruction and other applications. As the orthogonality property is desirable, orthogonal LPP (OLPP) has been proposed so that an orthogonal projection matrix can be obtained based on a step by step procedure; however, this makes the algorithm computationally more expensive. Therefore, in this paper, we propose a fast and orthogonal version of LPP, called FOLPP, which simultaneously minimizes the locality and maximizes the globality under the orthogonal constraint. As a result, the computation burden of the proposed algorithm can be effectively alleviated compared with the OLPP algorithm. Experimental results on two face recognition data sets and two hyperspectral data sets are presented to demonstrate the effectiveness of the proposed algorithm.