This paper proposes to control ridge estimation with nonlinear transformations. We establish an inclusion relationship between ridges with/without transformations:${\mathcal R}(f\circ p)\subseteq {\mathcal R}(p)$, where${\mathcal R}(p)$is the$d$-dimensional ridge of$p:\mathbb {R}^{D}\to \mathbb {R}$and$f$is a strictly increasing and concave map defined on range$(p)$. This finding reveals the benefit of applying an increasing and concave transformation before ridge estimation. As$f^{q}(y)=y^{q}/q, q\leq 1$are increasing and concave on$\mathbb {R}_+$, we use these power transformations to positive density functions, and then perform ridge estimation. The algorithm, named power-transformed subspace-constrained mean-shift (PSCMS), outperforms its competitors in numerical experiments.

This letter introduces a sparse regularized projection matrix approximation (SPMA) model to recover cluster structures from affinity matrices. The model is formulated as a projection approximation problem with an entry-wise sparsity penalty to encourage sparse solutions. We propose two algorithms to solve this problem: one involves direct optimization on the Stiefel manifold using the Cayley transformation, while the other employs the Alternating Direction Method of Multipliers (ADMM). Numerical experiments on synthetic and real-world datasets demonstrate that our regularized projection matrix approximation approach significantly outperforms state-of-the-art methods in clustering accuracy and performance.

Matrix factorization is a popular framework for modeling low-rank data matrices. Motivated by manifold learning problems, this paper proposes a quadratic matrix factorization (QMF) framework to learn the curved manifold on which the dataset lies. Unlike local linear methods such as the local principal component analysis, QMF can better exploit the curved structure of the underlying manifold. Algorithmically, we propose an alternating minimization algorithm to optimize QMF and establish its theoretical convergence properties. To avoid possible over-fitting, we then propose a regularized QMF algorithm and discuss how to tune its regularization parameter. Finally, we elaborate how to apply the regularized QMF to manifold learning problems. Experiments on a synthetic manifold learning dataset and three real-world datasets, including the MNIST handwritten dataset, a cryogenic electron microscopy dataset, and the Frey Face dataset, demonstrate the superiority of the proposed method over its competitors.

Community detection is an important problem in unsupervised learning. This letter proposes to solve a projection matrix approximation problem with an additional entrywise bounded constraint. Algorithmically, we introduce a new differentiable convex penalty and derive an alternating direction method of multipliers (ADMM) algorithm. Theoretically, we establish the convergence properties of the proposed algorithm. Numerical experiments demonstrate the superiority of our algorithm over its competitors.

Multi-view learning aims to integrate multiple data information from different views to improve the learning performance. The key problem is to handle the unconformities or distortions among view-specific samples or measurements of similarity or dissimilarity. This paper models the view-specific samples as a nonlinear mapping of uniform but latent intact samples for all the views, and the view-specific dissimilarity matrices or similarity matrices are estimated in terms of the uniform latent one. Two methods are then developed for multi-view clustering. One makes use of uniform multidimensional scaling (UMDS) on multi-view dissimilarities or kernels. The other one uses a uniform class assignment (UCA) procedure that optimally extracts the cluster components contained in the view-specific similarity matrices. These two methods result in the same optimization model, subjected to some slightly different constraints. A first-order condition of solutions is given as a nonlinear eigenvalue problem, and a second order condition guarantees local optimality. The nonlinear eigenvalue problem is solved by an iterative algorithm via eigen-space updating, and its convergence is proven. Furthermore, a fast implementation of the algorithm is discussed, which adopts the strategy of restarting subspace extension. Numerical experiments on some real-world data sets provide good support to the proposed methods.