Multimodal fusion is susceptible to modality imbalance, where dominant modalities overshadow weak ones, easily leading to biased learning and suboptimal fusion, especially for incomplete modality conditions. To address this problem, we introduce an Equilibrium Deviation Metric (EDM) to quantify this imbalance and verify, in both theoretical and empirical terms, that the optimization order of modalities plays a critical role in approaching equilibrium. In particular, we demonstrate that an EDM-ranked weak-to-strong schedule achieves the tightest convergence bound among all possible ordering strategies. Leveraging these insights, we design an alternating strategy that dynamically prioritises under-optimised modalities, plus a modality-mapping layer for feature alignment and a memory module for information filtering and inheritance. Our framework is compatible with both conventional and MLLM-based backbones. It achieves new state-of-the-art (SOTA) on four benchmarks (e.g., +3.36% on CREMA-D, +3.51% on Kinetics-400), and remains robust under missing-modality conditions. These findings highlight the value of modality scheduling, offering a principled alternative to conventional joint training.

This paper presents a new Physics-Informed Neural Network (PINN) framework to estimate the State of Health (SOH) of lithium-ion batteries. The proposed architecture, PINN-LSTM-KF, integrates long- and short-term memory (LSTM) networks with extended Kalman filtering under physics-based constraints. Conventional data-driven approaches often fail to generalize across different operating conditions due to non-linear degradation patterns. Our method addresses these challenges by enforcing electrochemical constraints within a multiscale architecture. It simultaneously captures microscopic physical processes, mesoscopic temporal dynamics, and macroscopic uncertainty quantification. Experiments on lithium-ion, lithium iron phosphate, and lithium-sulfur batteries demonstrate that the proposed framework achieves mean absolute percentage errors below 0. 01% under physics-informed configurations. Model compression techniques further reduce memory overhead, enabling real-time deployment in embedded systems. The framework also supports feature-level interpretability by quantifying contributions of physical variables to degradation, offering practical insights for battery design and management. These results highlight the potential of combining physics-based modeling with learning-based estimation to improve reliability and safety in energy storage systems, particularly in critical domains such as electric vehicles and smart grids.

Graph representation provides a more effective method for describing the underlying data relationships. Nonetheless, the vast majority of data consists solely of feature information without a corresponding graph structure, rendering graph representation techniques ineffective. Much of the existing research on graph data has concentrated on how to effectively characterize graph nodes, with little focus on how to adaptively construct internal structures and potential connections between the sample pairs. On the other hand, the existing graph construction techniques generate linear inter-instance affinity distributions based on a probabilistic perspective, which might not give a true picture of the relationships. To overcome the above problems, motivated by the fact that sample and inter-sample affinities can be viewed as the source and strength of the magnetic field, respectively, a novel tangent-based affinity measurement algorithm that utilizes a parameter to dynamically adjust the sparsity of the magnetic field is derived. In addition, Adaptive Magnetic-Graph Clustering (AMGC) is designed for graph representation and clustering. AMGC ensures instance-level and cluster-level consistency using a novel dual decoder, where the reconstructed graph retains local affinity and global topology, and contrastive learning defines new sample pairs based on positive-incentive noise, making the learned embedding more discriminative. Eventually, we perform empirical experiments to demonstrate the superiority of the model.

A conventional principal component analysis (PCA) frequently suffers from the disturbance of outliers, and thus, spectra of extensions and variations of PCA have been developed. However, all the existing extensions of PCA derive from the same motivation, which aims to alleviate the negative effect of the occlusion. In this article, we design a novel collaborative-enhanced learning framework that aims to highlight the pivotal data points in contrast. As for the proposed framework, only a part of well-fitting samples are adaptively highlighted, which indicates more significance during training. Meanwhile, the framework can collaboratively reduce the disturbance of the polluted samples as well. In other words, two contrary mechanisms could work cooperatively under the proposed framework. Based on the proposed framework, we further develop a pivotal-aware PCA (PAPCA), which utilizes the framework to simultaneously augment positive samples and constrain negative ones by retaining the rotational invariance property. Accordingly, extensive experiments demonstrate that our model has superior performance compared with the existing methods that only focus on the negative samples.

The existing matrix completion methods focus on optimizing the relaxation of rank function such as nuclear norm, Schatten- p norm, etc. They usually need many iterations to converge. Moreover, only the low-rank property of matrices is utilized in most existing models and several methods that incorporate other knowledge are quite time-consuming in practice. To address these issues, we propose a novel non-convex surrogate that can be optimized by closed-form solutions, such that it empirically converges within dozens of iterations. Besides, the optimization is parameter-free and the convergence is proved. Compared with the relaxation of rank, the surrogate is motivated by optimizing an upper-bound of rank. We theoretically validate that it is equivalent to the existing matrix completion models. Besides the low-rank assumption, we intend to exploit the column-wise correlation for matrix completion, and thus an adaptive correlation learning, which is scaling-invariant, is developed. More importantly, after incorporating the correlation learning, the model can be still solved by closed-form solutions such that it still converges fast. Experiments show the effectiveness of the non-convex surrogate and adaptive correlation learning.

Deep neural network (DNN) generally takes thousands of iterations to optimize via gradient descent and thus has a slow convergence. In addition, softmax, as a decision layer, may ignore the distribution information of the data during classification. Aiming to tackle the referred problems, we propose a novel manifold neural network based on non-gradient optimization, i.e., the analytical-form solutions. Considering that the activation function is generally invertible, we reconstruct the network via forward ridge regression and low-rank backward approximation, which achieve rapid convergence. Moreover, by unifying the flexible Stiefel manifold and adaptive support vector machine, we devise the novel decision layer which efficiently fits the manifold structure of the data and label information. Consequently, a jointly non-gradient optimization method is designed to generate the network with analytical-form results. Furthermore, an acceleration strategy is utilize to reduce the time complexity for handling high dimensional datasets. Eventually, extensive experiments validate the superior performance of the model.

Since the representative capacity of graph-based clustering methods is usually limited by the graph constructed on the original features, it is attractive to find whether graph neural networks (GNNs), a strong extension of neural networks to graphs, can be applied to augment the capacity of graph-based clustering methods. The core problems mainly come from two aspects. On the one hand, the graph is unavailable in the most general clustering scenes so that how to construct graph on the non-graph data and the quality of graph is usually the most important part. On the other hand, given$n$samples, the graph-based clustering methods usually consume at least$\mathcal {O}(n^{2})$time to build graphs and the graph convolution requires nearly$\mathcal {O}(n^{2})$for a dense graph and$\mathcal {O}(|\mathcal {E}|)$for a sparse one with$|\mathcal {E}|$edges. Accordingly, both graph-based clustering and GNNs suffer from the severe inefficiency problem. To tackle these problems, we propose a novel clustering method,AnchorGAE, with the self-supervised estimation of graph and efficient graph convolution. We first show how to convert a non-graph dataset into a graph dataset, by introducing the generative graph model and anchors. A bipartite graph is built via generating anchors and estimating the connectivity distributions of original points and anchors. We then show that the constructed bipartite graph can reduce the computational complexity of graph convolution from$\mathcal {O}(n^{2})$and$\mathcal {O}(|\mathcal {E}|)$to$\mathcal {O}(n)$. The succeeding steps for clustering can be easily designed as$\mathcal {O}(n)$operations. Interestingly, the anchors naturally lead to siamese architecture with the help of the Markov process. Furthermore, the estimated bipartite graph is updated dynamically according to the features extracted by GNN modules, to promote the quality of the graph by exploiting the high-level information by GNNs. However, we theoretically prove that the self-supervised paradigm frequently results in a collapse that often occurs after 2-3 update iterations in experiments, especially when the model is well-trained. A specific strategy is accordingly designed to prevent the collapse. The experiments support the theoretical analysis and show the superiority of AnchorGAE.

Graph autoencoders (GAEs) are powerful tools in representation learning for graph embedding. However, the performance of GAEs is very dependent on the quality of the graph structure, i.e., of the adjacency matrix. In other words, GAEs would perform poorly when the adjacency matrix is incomplete or be disturbed. In this paper, two novel unsupervised graph embedding methods, unsupervised graph embedding via adaptive graph learning (BAGE) and unsupervised graph embedding via variational adaptive graph learning (VBAGE) are proposed. The proposed methods expand the application range of GAEs on graph embedding, i.e, on the general datasets without graph structure. Meanwhile, the adaptive learning mechanism can initialize the adjacency matrix without being affected by the parameter. Besides that, the latent representations are embedded with the Laplacian graph structure to preserve the topology structure of the graph in the vector space. Moreover, the adjacency matrix can be self-learned for better embedding performance when the original graph structure is incomplete. With adaptive learning, the proposed method is much more robust to the graph structure. Experimental studies on several datasets validate our design and demonstrate that our methods outperform baselines by a wide margin in node clustering, node classification, link prediction, and graph visualization tasks.

Density structure and pairwise structure serve as two different but complementary perspectives for clustering. Either side of road is frequently visited and explored by multiple clustering methods. However, there are seldom approaches, which could mutually exploit both structures for clustering. To address this problem, in this paper, we develop a novel adaptive joint clustering algorithm, which combines unsupervised discrete orthogonal least squares discriminant analysis (DOLSDA) and discrete spectral clustering (DSC) with adaptive neighbors and side information into a unified model. Firstly, we extend supervised OLSDA to a discrete kernel clustering problem. To further achieve a clear pairwise structure, a new similarity with adaptive neighbors is then derived to establish sparse Laplacian matrix. In addition, side information could be incorporated to formulate clearer graph by modifying the proposed similarity. Based on the constructed graph, DSC is embedded with the discrete kernel OLSDA (DKOLSDA) clustering to exploit both cluster and pairwise data structures. Equipped with the proposed framework regarding quadratic weighted optimization, adaptive weight can be obtained automatically to leverage both unsupervised DKOLSDA and DSC. Since the unified problem is still discrete, we develop an increment scheme to achieve the optimal spectral rotation for the approximate solution to the predicted indicator.

As a hot topic in unsupervised learning, clustering methods have been greatly developed. However, the model becomes more and more complex, and the number of parameters becomes more and more with the continuous development of clustering methods. And parameter-tuning in most methods is a laborious work due to its complexity and unpredictability. How to propose a concise and beautiful model in which the parameters can be learned adaptively becomes a very meaningful problem. Aim at tackling this problem, we develop a novel self-weighted unsupervised linear discriminative analysis method, namely SWULDA. The proposed method not only avoids adjusting parameters but also explains the link between k -means and linear discriminant analysis (LDA). To obtain superior structural performance, the idea of minimizing the within-class scatter matrix and maximizing the between-class scatter matrix is embedded in the unsupervised model. Moreover, equipped with the proposed quadratic weighted optimization framework, the parameter can be adaptively learned. The extensive experiments on several datasets are conducted to validate the effectiveness of our method.

Graph clustering, aiming to partition nodes of a graph into various groups via an unsupervised approach, is an attractive topic in recent years. To improve the representative ability, several graph auto-encoder (GAE) models, which are based on semisupervised graph convolution networks (GCN), have been developed and they have achieved impressive results compared with traditional clustering methods. However, all existing methods either fail to utilize the orthogonal property of the representations generated by GAE or separate the clustering and the training of neural networks. We first prove that the relaxed k -means will obtain an optimal partition in the inner-product distance used space. Driven by theoretical analysis about relaxed k -means, we design a specific GAE-based model for graph clustering to be consistent with the theory, namely Embedding GAE (EGAE). The learned representations are well explainable so that the representations can be also used for other tasks. To induce the neural network to produce deep features that are appropriate for the specific clustering model, the relaxed k -means and GAE are learned simultaneously. Meanwhile, the relaxed k -means can be equivalently regarded as a decoder that attempts to learn representations that can be linearly constructed by some centroid vectors. Accordingly, EGAE consists of one encoder and dual decoders. Extensive experiments are conducted to prove the superiority of EGAE and the corresponding theoretical analyses.

With the tremendous growth of mobile social networks (MSNs), people are highly relying on it to connect with friends and further expand their social circles. However, the conventional friendship inference techniques have issues handling such a large yet sparse multi-source data. The related friend recommendation systems are therefore suffering from reduced accuracy and limited scalability. To address this issue, we propose a Two-stage Deep learning framework for Friendship Inference, namely TDFI. This approach enables MSNs to exploit multi-source information simultaneously, rather than hierarchically. Therefore, there is no need to manually set which information is more important and the order in which the various information is applied. In details, we apply an Extended Adjacency Matrix (EAM) to represent the multi-source information. We then adopt an improved Deep Auto-Encoder Network (iDAEN) to extract the fused feature vector for each user. Our framework also provides an improved Deep Siamese Network (iDSN) to measure user similarity. To provide a substantial description and evaluation of the proposed methodology, we evaluate the effectiveness and robustness on three large-scale real-world datasets. Trace-driven evaluation results demonstrate that TDFI can effectively handle the sparse multi-source data while providing better accuracy for friendship inference. Through the comparison with numerous state-of-the-art methods, we find that TDFI can achieve superior performance via real-world multi-source information. Meanwhile, it demonstrates that the proposed pipeline can not only integrate structural information and attribute information, but also be compatible with different attribute information, which further enhances the overall applicability of friend-recommendation systems under information-rich MSNs.

Graph-based clustering plays an important role in the clustering area. Recent studies about graph neural networks (GNN) have achieved impressive success on graph-type data. However, in general clustering tasks, the graph structure of data does not exist such that GNN can not be applied to clustering directly and the strategy to construct a graph is crucial for performance. Therefore, how to extend GNN into general clustering tasks is an attractive problem. In this paper, we propose a graph auto-encoder for general data clustering, AdaGAE, which constructs the graph adaptively according to the generative perspective of graphs. The adaptive process is designed to induce the model to exploit the high-level information behind data and utilize the non-euclidean structure sufficiently. Importantly, we find that the simple update of the graph will result in severe degeneration, which can be concluded as better reconstruction means worse update. We provide rigorous analysis theoretically and empirically. Then we further design a novel mechanism to avoid the collapse. Via extending the generative graph models to general type data, a graph auto-encoder with a novel decoder is devised and the weighted graphs can be also applied to GNN. AdaGAE performs well and stably in different scale and type datasets. Besides, it is insensitive to the initialization of parameters and requires no pretraining.

Principal component analysis (PCA) is warmly welcomed in dimensionality reduction and its applications. Due to the high sensitivity of PCA to outliers, a series of PCA methods are proposed to enhance the robustness of PCA. Besides, the representation ability of the existing PCA methods has limitations as well. To enhance the robustness and representation ability of robust PCA, we elaborate a novel Graph Convolution Robust PCA method (GRPCA) to incorporate the manifold structure into PCA. It constructs a sparse graph based on the local connectivity structure of samples. Graph auto-encoder is utilized to solve the robust PCA problem under the low-rank and sparse constraints. With the dual-decoder, GRPCA learns the low-dimensional embeddings that reconstruct the manifold structure and low-rank approximation simultaneously. Furthermore, since the graph suffers from misconnection triggered by occlusions, the local connectivity structure of low-dimensional embeddings is utilized to modify the graph. Our proposed method excels in both the clustering of low-dimensional embeddings and the low-rank recovery. Lastly, extensive experiments conducted on six real-world datasets demonstrated the efficiency and superiority of the proposed GRPCA.

Ridge regression is frequently utilized by both supervised learning and semisupervised learning. However, the results cannot obtain the closed-form solution and perform manifold structure when ridge regression is directly applied to semisupervised learning. To address this issue, we propose a novel semisupervised feature selection method under generalized uncorrelated constraint, namely SFS. The generalized uncorrelated constraint equips the framework with the elegant closed-form solution and is introduced to the ridge regression with embedding the manifold structure. The manifold structure and closed-form solution can better save data's topology information compared to the deep network with gradient descent. Furthermore, the full rank constraint of the projection matrix also avoids the occurrence of excessive row sparsity. The scale factor of the constraint that can be adaptively obtained also provides the subspace constraint more flexibility. Experimental results on data sets validate the superiority of our method to the state-of-the-art semisupervised feature selection methods.

The performance of graph-based feature selection methods relies heavily on the quality of the construction of the similarity matrix. However, most of the graphs on these methods are initially fixed, where few of them are constrained. Once the graph is determined, it will remain constant in the whole optimization process. In other words, in case that the graph constructed on the raw data is not appropriate, it will drag down the entire algorithm. Aiming to tackle this defect, a novel unsupervised feature selection via adaptive graph learning and constraint (EGCFS) is proposed to select the uncorrelated yet discriminative features by exploiting the embedded graph learning and constraint. The adaptive graph learning method incorporates the structure of the similarity matrix into the optimization process, which not only learns the graph structure adaptively but also obtains the closed-form solution of the graph coefficient. Special graph constraint is embedded with the feature selection process to connect nearer data points with larger probability. The idea of maximizing between-class scatter matrix and the adaptive graph structure is integrated into a uniform framework to obtain excellent structural performance. Moreover, the proposed embedded graph constraint not only performs with manifold structure but also validates the link between graph-based approach and k -means from a unique perspective. Experiments on several benchmark data sets verify the effectiveness and superiority of the proposed method.

Classical generative models in unsupervised learning intend to maximize p(X) . In practice, samples may have multiple representations caused by various transformations, measurements, and so on. Therefore, it is crucial to integrate information from different representations, and lots of models have been developed. However, most of them fail to incorporate the prior information about data distribution p(X) to distinguish representations. In this article, we propose a novel clustering framework that attempts to maximize the joint probability of data and parameters. Under this framework, the prior distribution can be employed to measure the rationality of diverse representations. K -means is a special case of the proposed framework. Meanwhile, a specific clustering model considering both multiple kernels and multiple views is derived to verify the validity of the designed framework and model.

As to unsupervised learning, most discriminative information is encoded in the cluster labels. To obtain the pseudo labels, unsupervised feature selection methods usually utilize spectral clustering to generate them. Nonetheless, two related disadvantages exist accordingly: 1) the performance of feature selection highly depends on the constructed Laplacian matrix and 2) the pseudo labels are obtained with mixed signs, while the real ones should be nonnegative. To address this problem, a novel approach for unsupervised feature selection is proposed by extending orthogonal least square discriminant analysis (OLSDA) to the unsupervised case, such that nonnegative pseudo labels can be achieved. Additionally, an orthogonal constraint is imposed on the class indicator to hold the manifold structure. Furthermore,$\ell _{2,1}$regularization is imposed to ensure that the projection matrix is row sparse for efficient feature selection and proved to be equivalent to$\ell _{2,0}$regularization. Finally, extensive experiments on nine benchmark data sets are conducted to demonstrate the effectiveness of the proposed approach.

Multi-Task Learning attempts to explore and mine the sufficient information within multiple related tasks for the better solutions. However, the performance of the existing multi-task approaches would largely degenerate when dealing with the polluted data, i.e., outliers. In this paper, we propose a novel robust multi-task model by incorporating a flexible manifold constraint (FMC-MTL) and a robust loss. Specifically speaking, multi-task subspace is embedded with a relaxed and generalized Stiefel Manifold for considering point-wise correlation and preserving the data structure simultaneously. In addition, a robust loss function is developed to ensure the robustness to outliers by smoothly interpolating betweenl2,1ℓ2,1-norm and squared Frobenius norm. Equipped with an efficient algorithm, FMC-MTL serves as a robust solution to tackling the severely polluted data. Moreover, extensive experiments are conducted to verify the superiority of our model. Compared to the state-of-the-art multi-task models, the proposed FMC-MTL model demonstrates remarkable robustness to the contaminated data.

Although fuzziness universally diffuses in the real-world data, the fuzzy information is tricky to harness for feature selection such that it is rarely utilized. Therefore, how to efficiently exploit fuzzy information has become the major focus for feature selection recently. In this article, a novel unsupervised feature selection method is proposed via exploiting the sparse fuzzy membership efficiently. In general,$\ell _{2,1}$norm is utilized to induce sparsity, while Frobenius norm is used to prevent overfitting. To obtain sparsity and avoid overfitting simultaneously, adaptive loss regularization is introduced to the least-squares regression, such that a sparse and nontrivial projection matrix can be achieved via continuous interpolation between$\ell _{2,1}$and Frobenius regularization. Additionally, the fuzzy$k$-means problem is further embedded with the adaptive loss regression model to avoid the trivial solution caused by the linearity of fuzzy membership. Therefore, the fuzzy cluster structure of fuzzy$k$-means is exploited for the efficient feature selection. By performing fuzzy clustering and subspace regression simultaneously, the embedded problem is then reformulated into a general quadratic problem with$\ell _1$ball constraint. Equipped with an auxiliary variable and standard augmented Lagrangian method, the quadratic problem, i.e., the corresponding dual problem can be solved with the closed form solutions regarding the fuzzy membership and the projection matrix. Consequently, empirical results are provided to demonstrate the effectiveness of the proposed feature selection approach.

The conventional subspace clustering method obtains explicit data representation that captures the global structure of data and clusters via the associated subspace. However, due to the limitation of intrinsic linearity and fixed structure, the advantages of prior structure are limited. To address this problem, in this brief, we embed the structured graph learning with adaptive neighbors into the deep autoencoder networks such that an adaptive deep clustering approach, namely, autoencoder constrained clustering with adaptive neighbors (ACC_AN), is developed. The proposed method not only can adaptively investigate the nonlinear structure of data via a parameter-free graph built upon deep features but also can iteratively strengthen the correlations among the deep representations in the learning process. In addition, the local structure of raw data is preserved by minimizing the reconstruction error. Compared to the state-of-the-art works, ACC_AN is the first deep clustering method embedded with the adaptive structured graph learning to update the latent representation of data and structured deep graph simultaneously.

Ridge regression is frequently utilized by both supervised and semisupervised learnings. However, the trivial solution might occur, when ridge regression is directly applied for clustering. To address this issue, an uncorrelated constraint is introduced to the ridge regression with embedding the manifold structure. In particular, we choose uncorrelated constraint over orthogonal constraint, since the closed-form solution can be obtained correspondingly. In addition to the proposed uncorrelated ridge regression, a soft pseudo label is utilized with ℓ1ball constraint for clustering. Moreover, a brand new strategy, i.e., a rescaled technique, is proposed such that optimal scaling within the uncorrelated constraint can be achieved automatically to avoid the inconvenience of tuning it manually. Equipped with the rescaled uncorrelated ridge regression with the soft label, a novel clustering method can be developed based on solving the related clustering model. Consequently, extensive experiments are provided to illustrate the effectiveness of the proposed method.

Sponsored Data Plan (SDP) is an emerging pricing model for the wireless data market where the Content Provider (CP) can sponsor the data usage for specific content on behalf of the users. This strategy sheds new light on the data pricing model and receives significant attention from the Internet Service Provider (ISP). However, the existing SDP studies consider traffic price (e.g., sponsorship) as the only factor that affects user decision. The impact of other classic market features, such as the demand for a variety of contents (i.e., love of variety), remains largely unclear. In this paper, we develop a new model to understand the love of variety in the wireless data market under SDPs. Our model has demonstrated that, such variety is important to understand the complex gaming between ISPs, CPs, and users in both short-run and long-run markets. For example, the analysis indicates that the advantage of CPs with higher revenue will be significantly reduced when users have a greater love of variety. Moreover, to help the ISP better adopt the proposed model in the real market, we also develop a practical method to calibrate the related parameters, which can also be applied to quantity the love of variety.

Neural network based clustering methods usually have better performance compared to the conventional approaches due to more efficient feature extraction. Most of existing deep clustering techniques either exploit graph information as prior to extract pivotal deep structure from the raw data and simply utilizes stochastic gradient descent (SGD). However, they often suffer from separating the learning steps regarding dimensionality reduction and clustering. To address these issues, a novel deep model named as deep fuzzy k-means (DFKM) with adaptive loss function and entropy regularization is proposed. DFKM performs deep feature extraction and fuzzy clustering simultaneously to generate a more appropriate nonlinear feature map. Additionally, DFKM incorporates FKM so that fuzzy information is utilized to represent a clear structure of deep clusters. To further promote the robustness of the model, a robust loss function is applied to the objective with adaptive weights. Moreover, an entropy regularization is employed for affinity to provide confidence of each assignment and the corresponding membership and centroid matrices are updated by close-form solutions rather than SGD. Extensive experiments show that DFKM has better performance compared to the state-of-the-art fuzzy clustering techniques under three clustering metrics.

The existing unsupervised feature extraction methods frequently explore low-redundant features by an uncorrelated constraint. However, the constrained models might incur trivial solutions, due to the singularity of scatter matrix triggered by high-dimensional data. In this paper, we propose a regularized regression model with a generalized uncorrelated constraint for feature selection, which leads to three merits: 1) exploring the low-redundant and discriminative features; 2) avoiding the trivial solutions and 3) simplifying the optimization. Besides that, the local cluster structure is achieved via a novel constrained spectral analysis for the unsupervised learning, where MustLinks and Cannot-Links are transformed into a intrinsic graph and a penalty graph respectively, rather than incorporated into a mixed affinity graph. Accordingly, a discriminative and uncorrelated feature selection with constrained spectral analysis (DUCFS) is proposed with adopting σ-norm regularization for interpolating between F-norm and ℓ2,1-norm. Due to the flexible gradient and global differentiability, our model converges fast. Extensive experiments on benchmark datasets among several state-of-the-art approaches verify the effectiveness of the proposed method.

Data reconstruction, which aims at preserving statistical properties of the data during the reconstruction has become a new criterion for feature selection. Although feature selection could benefit from the perspective of data reconstruction, it is unable to exploit other crucial information, namely, graph structure and pairwise constraints. To address previously mentioned deficiency, we propose a novel feature selection approach in this paper, known as unsupervised feature selection via data reconstruction and side information. More specifically, the proposed method takes advantage of the prior knowledge regarding pairwise constraints (side information), the minimization of data reconstruction error, and the graph embedding simultaneously, such that pivotal features are selected with preserving data manifold structure. To obtain the robust solution, a robust loss function is applied to the feature selection problem, which interpolates between ℓ1-norm and ℓ2-norm. Eventually, extensive experiments are conducted to demonstrate the effectiveness of the proposed method.

In this article, the recursive problems of both orthogonal linear discriminant analysis (OLDA) and orthogonal least squares regression (OLSR) are investigated. Different from other works, the associated recursive problems are addressed via a novel recursive regression method, which achieves the dimensionality reduction in the orthogonal complement space heuristically. As for the OLDA, an efficient method is developed to obtain the associated optimal subspace, which is closely related to the orthonormal basis of the optimal solution to the ridge regression. As for the OLSR, the scalable subspace is introduced to build up an original OLSR with optimal scaling (OS). Through further relaxing the proposed problem into a convex parameterized orthogonal quadratic problem, an effective approach is derived, such that not only the optimal subspace can be achieved but also the OS could be obtained automatically. Accordingly, two supervised dimensionality reduction methods are proposed via obtaining the heuristic solutions to the recursive problems of the OLDA and the OLSR.

Existing image reconstruction methods frequently improve their robustness by using various nonsquared loss functions, which are still potentially sensitive to the outliers. More specifically, when certain samples in data sets encounter severe contamination, these methods cannot identify and filter out the ill ones, and thus lead to the functional degeneration of the associated models. To address this issue, we propose a general framework, named robust and sparse weight learning (RSWL), to compute the adaptive weights based on an objective for robustness and sparsity. More importantly, the degree of the sparsity is steerable, such that only k well-reserved samples are activated during the optimization of our model. As a result, the severely polluted or damaged samples are eliminated, and the robustness is ensured. The framework is further leveraged against a 2-D image reconstruction task. Theoretical analysis and extensive experiments are presented to demonstrate the superiority of the proposed method.

Most existing feature selection methods select the top-ranked features according to certain criterion. However, without considering the redundancy among the features, the selected ones are frequently highly correlated with each other, which is detrimental to the performance. To tackle this problem, we propose a framework regarding adaptive redundancy minimization (ARM) for the feature selection. Unlike other feature selection methods, the proposed model has the following merits: (1) The redundancy matrix is adaptively constructed instead of presetting it as the priori information. (2) The proposed model could pick out the discriminative and non-redundant features via minimizing the global redundancy of the features. (3) ARM can reduce the redundancy of the features from both supervised and unsupervised perspectives.

Deep neural network clustering is superior to the conventional clustering methods due to deep feature extraction and nonlinear dimensionality reduction. Nevertheless, deep neural network leads to a rough representation regarding the inherent relationship of the data points. Therefore, it is still difficult for deep neural network to exploit the effective structure for direct clustering. To address this issue, we propose a robust embedded deep K-means clustering (RED-KC) method. The proposed RED-KC approach utilizes the δ-norm metric to constrain the feature mapping process of the auto-encoder network, so that data are mapped to a latent feature space, which is more conducive to the robust clustering. Compared to the existing auto-encoder networks with the fixed prior, the proposed RED-KC is adaptive during the process of feature mapping. More importantly, the proposed RED-KC embeds the clustering process with the auto-encoder network, such that deep feature extraction and clustering can be performed simultaneously. Accordingly, a direct and efficient clustering could be obtained within only one step to avoid the inconvenience of multiple separate stages, namely, losing pivotal information and correlation. Consequently, extensive experiments are provided to validate the effectiveness of the proposed approach.

In this paper, we propose a novel unsupervised feature selection method, which is to minimize the data reconstruction error between each sample and a linear combination of its neighbors. Different from the conventional reconstruction-based feature selection method, we impose a nonnegative orthogonal constraint on the reconstruction weight matrix, so that an ideal neighbor assignment is adaptively captured. To enhance the robustness of the residual term and select the most valuable features, ℓ2,1-norm is applied to both reconstruction error term and feature selection matrix. At last, we derive an iterative algorithm to effectively solve the proposed objective function, and perform extensive experiments on four benchmark datasets to validate the effectiveness of the proposed method.

Due to the explosive growth of social network services, friendship inference has been widely adopted by Online Social Service Providers (OSSPs) for friend recommendation. The conventional techniques, however, have limitations in accuracy or scalability to handle such a large yet sparse multi-source data. For example, the OSSPs will be required to manually give the order in which the various information is applied. This unavoidably reduces the applicability of existing friend recommendation systems. To address this issue, we propose a Two-stage Deep learning framework for Friendship Inference (TDFI). This approach can utilize multi-source information simultaneously with low complexity. In particular, we apply an Extended Adjacency Matrix (EAM) to represent the multi-source information. We then adopt an improved Deep AutoEncoder Network (iDAEN) to extract the fused feature vector for each user. The TDFI framework also provides an improved Deep Siamese Network (iDSN) to measure user similarity from iDAEN. Finally, we evaluate the effectiveness and robustness of TDFI on three large-scale real-world datasets. It shows that TDFI can effectively handle the sparse multi-source data while providing better accuracy for friend recommendation.

Suppose certain data points are overly contaminated, then the existing principal component analysis (PCA) methods are frequently incapable of filtering out and eliminating the excessively polluted ones, which potentially lead to the functional degeneration of the corresponding models. To tackle the issue, we propose a general framework namely robust weight learning with adaptive neighbors (RWL-AN), via which adaptive weight vector is automatically obtained with both robustness and sparse neighbors. More significantly, the degree of the sparsity is steerable such that only exact k well-fitting samples with least reconstruction errors are activated during the optimization, while the residual samples, i.e., the extreme noised ones are eliminated for the global robustness. Additionally, the framework is further applied to PCA problem to demonstrate the superiority and effectiveness of the proposed RWL-AN model.

Most existing feature selection methods rank all the features by a certain criterion, via which the top ranking features are selected for the subsequent classification or clustering tasks. Due to neglecting the feature redundancy, the selected features are frequently correlated with each other such that performance could be compromised. To address this issue, we propose a novel auto-weighted feature selection framework via global redundancy minimization (AGRM) in this paper. Different from other feature selection methods, the proposed method can truly select the representative and non-redundant features, since the redundancy among the features can be largely reduced from the global perspective. In addition, AGRM is extended to a compact (C-AGRM) framework, which is more concise and efficient. Moreover, both of the proposed frameworks are auto-weighted, i.e., parameterfree, so that they are pragmatic in real applications. In general, the proposed frameworks serve as post-processing system, which can be applied to the existing supervised and unsupervised feature selection methods to refine the original feature score for the non-redundant features. Eventually, extensive experiments on nine benchmark datasets are conducted to demonstrate the effectiveness and the superiority of our proposed frameworks.

In this paper, we propose a novel bilinear projections model for unsupervised learning, which can be directly applied to dimension reduction and feature extraction for image analysis. Compared to previous 2-dimensional (2D) methods, our method seeks multiple rank-k bilinear projections [MRBP], which makes a balance between two opposite problems of enhancing the degree of freedom and avoiding the problem of over-fitting by adjusting k. Besides, each renewed feature in our method is extracted independently rather than dimensions are reduced by row or by column like before, and thus our reduced dimension is not restricted by the size of a sample. Additionally, we demonstrate that our model could be optimized via two equivalent optimization problems based on criteria of maximum separability and nearest reconstruction respectively. Motivated by this, the corresponding robust version [RMRBP] achieving a better performance to occluded data is introduced as well. Extensive experiments on several datasets have been done to verify the effectiveness and superiority of our methods.

As one of the most widely used clustering techniques, the fuzzy k-means (FKM) assigns every data point to each cluster with a certain degree of membership. However, conventional FKM approach relies on the square data fitting term, which is sensitive to the outliers with ignoring the prior information. In this paper, we develop a novel and robust fuzzy k-means clustering algorithm, namely, joint learning of fuzzy k-means and nonnegative spectral clustering with side information. The proposed method combines fuzzy k-means and nonnegative spectral clustering into a unified model, which can further exploit the prior knowledge of data pairs such that both the quality of affinity graph and the clustering performance can be improved. In addition, for the purpose of enhancing the robustness, the adaptive loss function is adopted in the objective function, since it smoothly interpolates between 11-norm and 12-norm. Finally, experimental results on benchmark datasets verify the effectiveness and the superiority of our clustering method.

Unsupervised feature selection always occupies a key position as a preprocessing in the tasks of classification or clustering due to the existence of extra essential features within high-dimensional data. Although lots of efforts have been made, the existing methods neglect to consider the redundancy of features, and thus select redundant features. In this brief, by virtue of a generalized uncorrelated constraint, we present an improved sparse regression model [generalized uncorrelated regression model (GURM)] for seeking the uncorrelated yet discriminative features. Benefited from this, the structure of data is kept in the Stiefel manifold, which avoids the potential trivial solution triggered by a conventional ridge regression model. Besides that, the uncorrelated constraint equips the model with the closed-form solution. In addition, we also incorporate a graph regularization term based on the principle of maximum entropy into the GURM model (URAFS), so as to embed the local geometric structure of data into the manifold learning. An efficient algorithm is designed to perform URAFS by virtue of the existing generalized powered iteration method. Extensive experiments on eight benchmark data sets among seven state-of-the-art methods on the task of clustering are conducted to verify the effectiveness and superiority of the proposed method.

As for semisupervised learning, both label information and side information serve as pivotal indicators for the classification. Nonetheless, most of related research works utilize either label information or side information instead of exploiting both of them simultaneously. To address the referred defect, we propose a graph-based semisupervised learning (GSL) problem according to both given label information and side information. To solve the GSL problem efficiently, two novel self-weighted strategies are proposed based on solving associated equivalent counterparts of a GSL problem, which can be widely applied to a spectrum of biobjective optimizations. Different from a conventional technique to amalgamate must-link and cannot-link into a single similarity for convenient optimization, we derive a new parameter-free similarity, upon which intrinsic graph and penalty graph can be separately developed. Consequently, a novel semisupervised classification algorithm can be summarized correspondingly with a theoretical analysis.

Since multiple criteria can be adopted to estimate the similarity among the given data points, problem regarding diverse representations of pairwise relations is brought about. To address this issue, a novel self-adaptive multimeasure (SAMM) fusion problem is proposed, such that different measure functions can be adaptively merged into a unified similarity measure. Different from other approaches, we optimize similarity as a variable instead of presetting it as a priori, such that similarity can be adaptively evaluated based on integrating various measures. To further obtain the associated subspace representation, a graph-based dimensionality reduction problem is incorporated into the proposed SAMM problem, such that the related subspace can be achieved according to the unified similarity. In addition, sparsity-inducing ℓ2,0regularization is introduced, such that a sparse projection is obtained for efficient feature selection (FS). Consequently, the SAMM-FS method can be summarized correspondingly.

As one of the most widely used clustering techniques, the fuzzy K-Means (also called FKM or FCM) assigns every data point to each cluster with a certain degree of membership. However, conventional FKM approach relies on the square data fitting term which is not robust to data outliers and ignores the prior information, which leads to unsatisfactory clustering results. In this paper, we present a novel and robust fuzzy K-Means clustering algorithm, namely Embedding Fuzzy K-Means with Nonnegative Spectral Clustering via Incorporating Side Information. The proposed method combines fuzzy K-Means with nonnegative spectral clustering into a unified model, and further takes the advantage of the prior knowledge of data pairs such that the quality of similarity graph is enhanced and the clustering performance is effectively improved. Besides, the ℓ2,1-norm loss function is adopted in the objective function, which achieves better robustness to outliers. Last, experimental results on benchmark datasets verify the effectiveness and superiority of the proposed clustering method.

The ridge regression has been widely applied in multiple domains and gains the promising performance. However, due to the unavailability of labels, the ridge regression easily incurs the trivial solution towards unsupervised learning. In this paper, we investigate unsupervised feature selection by virtue of an uncorrelated and nonnegative ridge regression model (UN-RFS). To be specific, a generalized uncorrelated constraint on the projection matrix, and a nonnegative orthogonal constraint on the indicator matrix are imposed upon the proposed regression model. With the proposed method, the most uncorrelated features on the embedded Stiefel manifold is exploited for feature selection and trivial solutions of projection matrix are avoided as well. Besides, equipped with a generalized scatter matrix, the proposed uncorrelated constraint is superior to conventional uncorrelated constraint, since the closed form solution can be achieved directly. In addition, owing to the nonnegative of real labels, the nonnegative orthogonal constraint is employed to suppress the indicator matrix such that the learned labels confront to reality further.

The task of feature selection is to find the optimal feature subset such that an appropriate criterion is optimized. It can be seen as a special subspace learning task, where the projection matrix is constrained to be selection matrix. In this paper, a novel unsupervised graph embedded feature selection (GEFS) method is derived from the perspective of incorporating the projected k-means with Stiefel manifold regularization. To achieve more statistical and structural properties, we directly embed unsupervised feature selection algorithm into a clustering algorithm via sparse learning to suppress the projected matrix to be row sparse. Comparative experiments demonstrate the effectiveness of our proposed algorithm in comparison with the traditional methods for feature selection.

In this brief, a novel self-weighted orthogonal linear discriminant analysis (SOLDA) problem is proposed, and a self-weighted supervised discriminative feature selection (SSD-FS) method is derived by introducing sparsity-inducing regularization to the proposed SOLDA problem. By using the row-sparse projection, the proposed SSD-FS method is superior to multiple sparse feature selection approaches, which can overly suppress the nonzero rows such that the associated features are insufficient for selection. More specifically, the orthogonal constraint ensures the minimal number of selectable features for the proposed SSD-FS method. In addition, the proposed feature selection method is able to harness the discriminant power such that the discriminative features are selected. Consequently, the effectiveness of the proposed SSD-FS method is validated theoretically and experimentally.

In this paper, a novel embedded clustering (EC) method is derived from the perspective of extending the supervised orthogonal least square discriminant analysis (OLSDA) method to the unsupervised case, which proves to be closely related to k-means. To achieve more statistical and structural properties, the robust learning of unsupervised OLSDA is investigated to further derive the unsupervised robust OLSDA (ROLSDA) problem. For the convenience of solving the proposed ROLSDA problem, re-weighted counterpart of ROLSDA is utilized with self-adaptive weight, such that the smaller weight would be assigned to the term with larger outliers automatically. Consequently, aforementioned EC method is proposed with not only the robust outliers but also the optimal weighted cluster centroids. Comparative experiments are presented to show the effectiveness of the EC method under the proposed ROLSDA problem.

As for the semi-supervised learning, both label and side information serve as pretty significant indicators for the classification. However, majority of the associated works only focus on one side of the road. In other words, either the label information or the side information is utilized instead of taking both of them into consideration simultaneously. To address the referred defect, we propose a graph-based semi-supervised learning (GSL) problem via building the intrinsic graph and the penalty graph upon both label and side information. To efficiently unravel the proposed GSL problem, a novel quadratic trace ratio (QTR) method is proposed based on solving the associated QTR problem, which is the equivalent counterpart of the GSL problem. Besides, a parameter-free similarity is further derived and utilized. Consequently, a novel semi-supervised classification (SC) algorithm can be summarized by virtue of the proposed GSL problem and QTR method.

Two-dimensional principal component analysis (2DPCA) serves as an efficient approach for both dimensionality reduction and high-quality reconstruction. However, conventional 2DPCA method is sensitive to the outliers such that associated results could be compromised. To strengthen the robustness of conventional 2DPCA method, we try to propose a novel robust two-dimensional principal component analysis with optimal mean (R2DPCA-OM) method to automatically achieve the optimal mean. Besides, the experimental results illustrate that the proposed R2DPCA-OM method could obtain the optimal subspaces and mean, such that dimensionality is reduced with less reconstruction error. Consequently, superiority and effectiveness of the proposed R2DPCA-OM method could be verified analytically and empirically.

The orthogonal least square regression (OLSR) serves as a pretty significant problem for the dimensionality reduction. Due to lack of the scale change in OLSR, the scaling term is at first introduced to OLSR to build up a novel orthogonal least square regression with optimal scaling (OLSR-OS) problem. However, OLSR-OS is still sensitive to the outliers, such that associated results could be fallacious. To strengthen the robustness of OLSR-OS, we propose an original robust OLSR-OS (ROLSR-OS) problem in ℓ2,1-norm. To tackle a more ill-defined situation, ROLSR-OS in ℓ2,1-norm can be further extended to ROLSR-OS in capped ℓ2-norm. Besides, the associated ROLSR-OS methods could be derived by solving the re-weighted counterparts of ROLSR-OS problems in both norms. Moreover, the equivalence between the re-weighted counterparts and the original ROLSR-OS problems is also provided along with the convergence analysis of the proposed ROLSR-OS methods. Accordingly, both the optimal scaling and weight can be achieved automatically via the proposed ROLSR-OS approaches. Specifically, the proposed ROLSR-OS methods are self-adaptive, such that the smaller weight would be automatically assigned to the term with larger outliers to enhance the robustness. Consequently, projected clustering and modified projected clustering under the proposed ROLSR-OS problems are further investigated both theoretically and experimentally.

In this paper, we mainly focus on how to achieve the translated subspace representation for each class, which could simultaneously indicate the distribution of the associated class and the differences from its complementary classes. By virtue of the reconstruction problem, the class-specific subspace classifier (CSSC) problem could be represented as a series of biobjective optimization problems, which minimize and maximize the reconstruction errors of the related class and its complementary classes, respectively. Besides, the regularization term is specifically introduced to ensure the whole system's stability. Accordingly, a regularized class-specific subspace classifier (RCSSC) method can be further proposed based on solving a general quadratic ratio problem. The proposed RCSSC method consistently converges to the global optimal subspace and translation under the variations of the regularization parameter. Furthermore, the proposed RCSSC method could be extended to the unregularized case, which is known as unregularized CSSC (UCSSC) method via orthogonal decomposition technique. As a result, the effectiveness and the superiority of both proposed RCSSC and UCSSC methods can be verified analytically and experimentally.In this paper, we mainly focus on how to achieve the translated subspace representation for each class, which could simultaneously indicate the distribution of the associated class and the differences from its complementary classes. By virtue of the reconstruction problem, the class-specific subspace classifier (CSSC) problem could be represented as a series of biobjective optimization problems, which minimize and maximize the reconstruction errors of the related class and its complementary classes, respectively. Besides, the regularization term is specifically introduced to ensure the whole system's stability. Accordingly, a regularized class-specific subspace classifier (RCSSC) method can be further proposed based on solving a general quadratic ratio problem. The proposed RCSSC method consistently converges to the global optimal subspace and translation under the variations of the regularization parameter. Furthermore, the proposed RCSSC method could be extended to the unregularized case, which is known as unregularized CSSC (UCSSC) method via orthogonal decomposition technique. As a result, the effectiveness and the superiority of both proposed RCSSC and UCSSC methods can be verified analytically and experimentally.