Time series analysis plays a vital role in fields such as finance, healthcare, industry, and meteorology, underpinning key tasks including classification, forecasting, and anomaly detection. Although deep learning models have achieved remarkable progress in these areas in recent years, constructing an efficient, multi-task compatible, and generalizable unified framework for time series analysis remains a significant challenge. Existing approaches are often tailored to single tasks or specific data types, making it difficult to simultaneously handle multi-task modeling and effectively integrate information across diverse time series types. Moreover, real-world data are often affected by noise, complex frequency components, and multi-scale dynamic patterns, which further complicate robust feature extraction and analysis. To ameliorate these challenges, we propose FusAD, a unified analysis framework designed for diverse time series tasks. FusAD features an adaptive time-frequency fusion mechanism, integrating both Fourier and Wavelet transforms to efficiently capture global-local and multi-scale dynamic features. With an adaptive denoising mechanism, FusAD automatically senses and filters various types of noise, highlighting crucial sequence variations and enabling robust feature extraction in complex environments. In addition, the framework integrates a general information fusion and decoding structure, combined with masked pre-training, to promote efficient learning and transfer of multi-granularity representations. Extensive experiments demonstrate that FusAD consistently outperforms state-of-the-art models on mainstream time series benchmarks for classification, forecasting, and anomaly detection tasks, while maintaining high efficiency and scalability. Code is available at https://github.com/zhangda1018/FusAD.

Spectral clustering has received widespread attention for its effectiveness in handling nonconvex geometries. The classic methods of spectral clustering include Ratio Cut (Rcut), Normalized Cut (Ncut), and Min-Max Cut (MMcut). Among them, the objective function of MMcut is more reasonable. Unfortunately, existing methods cannot solve MMcut problem without relaxing the discrete or nonnegative constraints. To this end, based on coordinate descent, we propose a basic optimization algorithm to solve MMcut problem without relaxing any constraint. And then, a fast version of the solver is proposed to improve the computational efficiency. Besides, we prove the convergence of the proposed solver, evaluate its computational complexity, and discuss the connection between MMcut and NCut. Finally, extensive experiments are performed to evaluate the effectiveness of our proposed method.

Multi-view clustering (MVC) with bipartite graph has been extensively studied to rapidly handle multi-source heterogeneous information via sparse anchors. However, most existing methods follow a two-stage learning paradigm that first learns continuous label matrix and then discretizes it, not only bringing extra trade-off parameters but yielding suboptimal solutions. Also, numerous methods still exhibit limited scalability for large-scale problems. Thus, this paper proposes two novel models for discrete, trade-off parameter-free and rapid MVC. First, the Bipartite Graph Discrete Reconstruction (BGDR) model uniquely leverages the discrete label matrices of both samples and anchors to dynamically reconstruct a consensus bipartite graph across views. This concise reconstruction style eliminates redundant computations, and anchor labels enable to enrich cluster partition information during reconstruction, enhancing both accuracy and efficiency. The final clustering outcomes are directly acquired via discrete sample labels. Second, to free the optimization time overheads from the limitation of sample size, we further devise the Compact Graph Discrete Reconstruction (CGDR) model, which reconstructs a smaller compact affinity graph among anchors for significant acceleration. Original sample labels are then gained by label propagation. Systematic experiments illuminate that both models reach superior outcomes in term of efficacy and efficiency.

We propose the DPSM method, a density-based node clustering approach that automatically determines the number of clusters and can be applied in both data space and graph space. Unlike traditional density-based clustering methods, which necessitate calculating the distance between any two nodes, our proposed technique determines density through a propagation process, thereby making it suitable for a graph space. In DPSM, nodes are partitioned into small clusters based on propagated density. The partitioning technique has been proved to be sound and complete. We then extend the concept of spectral clustering from individual nodes to these small clusters, while introducing the CluCut measure to guide cluster merging. This measure is modified in various ways to account for cluster properties, thus provides guidance on when to terminate the merging process. Various experiments have validated the effectiveness of DPSM and the accuracy of these conclusions.

Principal Component Analysis (PCA) is one of the most famous unsupervised dimensionality reduction algorithms and has been widely used in many fields. However, it is very sensitive to outliers, which reduces the robustness of the algorithm. In recent years, many studies have tried to employ \(\ell_{1}\) -norm to improve the robustness of PCA, but they all lack rotation invariance or the solution is expensive. In this article, we propose a novel robust PCA, namely, Fuzzy Weighted Principal Component Analysis (FWPCA), which still uses squared \(\ell_{2}\) -norm to minimize reconstruction error and maintains rotation invariance of PCA. The biggest bright spot is that the contribution of data is restricted by fuzzy weights, so that the contribution of normal samples is much greater than noise or abnormal data, and realizes anomaly detection. Besides, a more reasonable data center can be obtained by solving the optimal mean to make projection matrix more accurate. Subsequently, an effective iterative optimization algorithm is developed to solve this problem, and its convergence is strictly proved. Extensive experimental results on face datasets and RGB anomaly detection datasets show the superiority of our proposed method.

As a compromise between supervised and unsupervised learning, semi-supervised learning (SSL) harnesses both labeled and unlabeled data to enhance learning performance. Graph-based semi-supervised learning (GSSL) has emerged as a prominent approach owing to its versatility in representing sample interdependencies via graph structures. However, traditional GSSL methods face high time cost when computing matrix inverses, making them inefficient for large datasets. To address this, some researchers have introduced anchors as a bridge to accelerate the process. Nevertheless, most anchor-based models suffer from one or more of the following issues: (1) The anchor graph-based construction of the adjacency matrix has limitations; (2) The objective functions are typically non-convex, leading to local optima and requiring multiple runs to achieve good performance. To tackle these challenges, we develop a probability-driven approach to build the adjacency matrix, defining sample similarity as the probability of sharing the same anchor. Based on this strategy, we design a model (CFSL) with a strictly convex objective function, guaranteeing a globally optimal solution without iterative optimization. Experiments on multiple datasets indicate that our algorithm yields strong performance.

Traditional spectral clustering methods struggle with scalability and robustness in large datasets due to their reliance on similarity matrices and EigenValue Decomposition. We introduce two innovative models: Rcut-based Coordinate Descent Clustering (R-CDC) and Ncut-based Doubly Stochastic Clustering (N-DSC). These models integrate graph construction and segmentation into a unified process optimized through the coordinate descent method, significantly enhancing clustering efficacy. A novel graph structure enhances robustness against noise and outliers, simplifying the clustering process and improving outcomes across diverse datasets. Our extensive experiments show that these models surpass existing spectral clustering techniques in managing large-scale data and complex structures.

This paper proposes a deep pseudo-label method for unsupervised feature selection, which learns non-linear representations to generate pseudo-labels and trains a Neural Network (NN) to select informative features via self-Knowledge Distillation (KD). Specifically, the proposed method divides a standard NN into two sub-components: an encoder and a predictor, and introduces a dependency subnet. It works by self-supervised pretraining the encoder to produce informative representations and then alternating between two steps: (1) learning pseudo-labels by combining the clustering results of the encoder's outputs with the NN's prediction outputs, and (2) updating the NN's parameters by globally selecting a subset of features to predict the pseudo-labels while updating the subnet's parameters through self-KD. Self-KD is achieved by encouraging the subnet to locally capture a subset of the NN features to produce class probabilities that match those produced by the NN. This allows the model to self-absorb the learned inter-class knowledge and evaluate feature diversity, removing redundant features without sacrificing performance. Meanwhile, the potential discriminative capability of a NN can also be self-excavated without the assistance of other NNs. The two alternate steps reinforce each other: in step (2), by predicting the learned pseudo-labels and conducting selfKD, the discrimination of the outputs of both the NN and the encoder is gradually enhanced, while the self-labeling method in step (1) leverages these two improvements to further refine the pseudo-labels for step (2), resulting in the superior performance. Extensive experiments show the proposed method significantly outperforms state-of-the-art methods across various datasets.

Graph-based clustering technique has garnered significant attention due to precise information characterization by pairwise graph similarity. Nevertheless, the post-processing step in traditional methods often limits clustering effects because of crucial information loss. Therefore, the Constrained Laplacian Rank (CLR) theory emerges to directly obtain discrete labels from optimally structural graph, achieving desirable outcomes. However, CLR suffers from substantial time overhead, making it infeasible for large-scale data analysis. To overcome this issue, we propose Anchor-based CLR (ACLR), a simple yet effective method for efficient large-scale clustering. The ACLR method comprises four stages: (1) anchors that roughly cover original data are opted to prepare bipartite graph construction; (2) a novel two-step probability transition (TSPT) strategy initializes a small-scale graph with random walk probability among anchors; (3) the main ACLR model alternately optimizes the graph connected structure and directly produces discrete anchor labels, achieving a time complexity independent of the number of samples due to dramatically reduced graph scale; and (4) labels are propagated from anchors to samples using$K$-NN algorithm. Extensive experiments demonstrate that ACLR yields superior accuracy and efficiency, particularly when applied to large-scale data. The codes are available athttps://github.com/MarathonZhenyuMa/2025-TKDE-ACLR.

Anchor-based clustering methods have attracted increasing attention due to their ability to provide efficient and scalable solutions in clustering tasks, such as subspace, multi-view and ensemble clustering. Nevertheless, the majority of anchor-based methods view anchors merely as tools, concentrating on diminishing computational complexity within original data space. However, in fact, clustering can be directly performed on anchors and then the anchor clustering results could be propagated to original data. Due to the much smaller volume of anchors, this could significantly reduce the computational complexity of clustering algorithms. Building upon this idea, in this paper, we propose a fast anchor graph clustering method (FAGC) via maximizing within-cluster similarity. Inspired by the relaxation and discretization model in spectral clustering, we also propose two corresponding models, namely FAGC-R and FAGC-D. FAGC-R first obtains spectral embedding of anchors and then discretizes the embedding to obtain anchor indicator matrix. While FAGC-D directly solves the discrete anchor membership matrix. Once anchor clustering results are obtained, original data labels can be obtained through anchor label transmission. Extensive experiments conducted on synthetic and real datasets illustrate the effectiveness and efficiency of the proposed methods.

In practical applications, multi-view subspace clustering is hindered by data noise that disrupts the ideal block-diagonal structure of self-representation matrices, thereby degrading performance. Moreover, many existing methods rely solely on sample features, overlooking the valuable structural information in affinity matrices (e.g., pairwise relationships). While conventional contrastive learning strategies often introduce false negative pairs due to noise and unreliable sample selection. To address these challenges, we propose a pseudo-label guided bidirectional discriminative deep multi-view subspace clustering method (PBDMSC). Our approach first employs pseudo-label guided contrastive learning, using previous cluster assignments to select reliable positive and negative samples, which mitigates incorrect pairings and enhances low-dimensional representations. Then, a discriminative self-representation learning method is introduced that leverages pseudo-labels to enforce homogeneous expression constraints and incorporates a bidirectional attention mechanism to preserve the structured information from affinity matrices, thereby enhancing robustness. Experimental results on six real-world datasets demonstrate that our proposed method achieves state-of-the-art clustering performance.

Graph representation provides a more effective method for describing the underlying data relationships. Nonetheless, the vast majority of data consists solely of feature information without a corresponding graph structure, rendering graph representation techniques ineffective. Much of the existing research on graph data has concentrated on how to effectively characterize graph nodes, with little focus on how to adaptively construct internal structures and potential connections between the sample pairs. On the other hand, the existing graph construction techniques generate linear inter-instance affinity distributions based on a probabilistic perspective, which might not give a true picture of the relationships. To overcome the above problems, motivated by the fact that sample and inter-sample affinities can be viewed as the source and strength of the magnetic field, respectively, a novel tangent-based affinity measurement algorithm that utilizes a parameter to dynamically adjust the sparsity of the magnetic field is derived. In addition, Adaptive Magnetic-Graph Clustering (AMGC) is designed for graph representation and clustering. AMGC ensures instance-level and cluster-level consistency using a novel dual decoder, where the reconstructed graph retains local affinity and global topology, and contrastive learning defines new sample pairs based on positive-incentive noise, making the learned embedding more discriminative. Eventually, we perform empirical experiments to demonstrate the superiority of the model.

Structured proximity matrix learning, one of the mainstream directions in clustering research, refers to learning a proximity matrix with an explicit clustering structure from the original first-order proximity matrix. Due to the complexity of the data structure, the original first-order proximity matrix always lacks some must-links compared to the groundtruth proximity matrix. It is worth noting that high-order proximity matrices can provide missed must-link information. However, the computation of high-order proximity matrices and clustering based on them are expensive. To solve the above problem, inspired by the anchor bipartite graph, we present a novel high-order bipartite graph proximity matrix and a fast method to compute it. This proposed high-order bipartite graph proximity matrix contains high-order proximity information and can significantly reduce the computational complexity of the whole clustering process. Furthermore, we introduce an efficient and simple high-order bipartite graph fusion framework that can adaptively assign weights to each order of the high-order bipartite graph matrices. Finally, under the Laplace rank constraint, a consensus structured bipartite graph proximity matrix is obtained. At the same time, an efficient solution algorithm is proposed for this model. The model's efficacy is underscored through rigorous experiments, highlighting its superior clustering performance and time efficiency. Code available:https://anonymous.4open.science/r/HBGC-F6C4.

Ensemble clustering can utilize the complementary information among multiple base clusterings, and obtain a clustering model with better performance and more robustness. Despite its great success, there are still two problems in the current ensemble clustering methods. First, most ensemble clustering methods often treat all base clusterings equally. Second, the final ensemble clustering result often relies on$k$-means or other discretization procedures to uncover the clustering indicators, thus obtaining unsatisfactory results. To address these issues, we proposed a novel ensemble clustering method based on structured graph learning, which can directly extract clustering indicators from the obtained similarity matrix. Moreover, our methods take sufficient consideration of correlation among the base clusterings and can effectively reduce the redundancy among them. Extensive experiments on artificial and real-world datasets demonstrate the efficiency and effectiveness of our methods.

Although graph neural networks (GNNs) can extract the latent relationship-level knowledge among the graph nodes and have achieved excellent performance in unsupervised scenarios, it is weak in learning the instance-level knowledge in contrast to the convolution neural networks (CNNs). Besides, lacking of the graph structure limits the extension of GNNs on non-graph datasets. To solve these problems, we propose a novel unsupervised multi-level knowledge fusion network. It successfully unifies the instance-level and relationship-level knowledge on the non-graph data by distillation from a pre-trained CNN teacher to a GNN student. Meanwhile, a sparse weighted strategy is designed to adaptively extract the sparse graph topology and extend the GNN on non-graph datasets. By optimization of distillation loss, the "boosted'' GNN student can learn the multi-level knowledge and extract more discriminative deep embeddings for clustering. Finally, extensive experiments show it has achieved excellent performance compared with the current methods.

Ensemble clustering has emerged as a powerful framework for analyzing heterogeneous and complex data. Despite the abundance of existing schemes, co-association matrix-based methods remain the mainstream approach. However, focusing solely on pairwise correlations falls short of fully capturing the intricate cluster relationships. Moreover, despite its potential, ensemble clustering has yet to effectively leverage the powerful representation capabilities of neural networks. To address these limitations, we propose a deep ensemble clustering method called Ensemble Clustering with Attentional Representation (ECAR). Our method considers the results of base partition as groups with related information to explore higher-order fusion information. ECAR captures the importance of each sample's association with its related group by employing an attentional network, and encodes this information into a low-dimensional representation. The attentional network is trained by jointly optimizing the clustering loss from soft assignments learned from the embeddings and the reconstruction loss from the weighted graph generated from ensemble clustering. During training, the weights of base partitions are adaptively refined to promote diversity and consistency while reducing the impact of low-quality and redundant base partitions. Extensive experimental results on real-world datasets demonstrate the substantial improvement of our method over existing baseline ensemble clustering methods and deep clustering methods.

Learning to hash is of particular interest in information retrieval for large-scale data due to its high efficiency and effectiveness. Most studies in hashing concentrate on constructing new hashing models, but rarely touch the correlation and redundancy between hash bits derived. In this article, we first introduce a general schema of hash bit reduction to derive compact and informative binary codes for hashing techniques. Further, we take locality sensitive hashing, one of the most widely-used hashing methods, as an example and propose a novel and two-stage binary code refinement method under the reduction schema. Specifically, the proposed method includes two stages, i.e., bit evaluation and bit refinement. The former stage aims to initially extract a small portion of informative hash bits in terms of their importance and quality evaluated by bit balance and similarity preservation. Then, the representation capabilities of the reduced hash bits are strengthened further by refining their binary values. The purpose of refinement is to lessen the correlations and redundancies between the reduced bits, making themselves more discriminative. The experimental results on three widely-used data collections confirm the effectiveness of the proposed bit reduction method and its superiority over the state-of-the-art hashing methods, as well as a bit selection method.

The key challenge of multi-view graph-based clustering is to mine consistent clustering structures from multiple graphs. Existing works seek clustering decisions from either multiple spectral embeddings or multiple affinity matrices, ignoring the interactions among them. To address this problem, we propose a Bidirectional Attentive Multi-view Clustering (BAMC) model to explore a consensus space w.r.t.spectral embedding and affinity matrix simultaneously, where they can promote each other to mine richer structural information from multiple graphs. BAMC is composed of a Spectral Embedding Learning (SEL) module, an Affinity Matrix Learning (AML) module, and a Bidirectional Attentive Clustering (BAC) module. SEL seeks consensus spectral embeddings by aligning the distributions of elements sampled from subspaces spanned by multiple spectral embeddings. AML learns a consensus affinity matrix from input affinity matrices. BAC guarantees consistency between the learned consensus spectral embeddings and the affinity matrix. To balance their effects, it also assigns adaptive weights to SEL and AML's objective functions. To solve the optimization problem involved in BAMC, we propose an efficient algorithm based on the Majority-Minimization framework with an ingenious surrogate problem. Extensive experiments on several synthetic and real-world datasets demonstrate the superb performance of BAMC.

In this paper, we propose a novel algorithm to solve the row-sparse principal component analysis problem without relying on any data structure assumption. Sparse principal component analysis was proposed to improve the interpretability of principal component analysis by restricting the number of non-zero elements in each loading vector, but the varying sparsity patterns among different leading vectors may result in trouble on some occasions, such as feature selection. Then row-sparse principal component analysis was proposed, which demands the same sparse pattern among different loading vectors. However, the optimization of row-sparse principal component analysis problems is NP-hard. Although some algorithms were proposed to solve this problem, but they are only applicable to the specific data structure. In this paper, we transform the original row-sparse principal component analysis problem into a new equivalent problem that can be solved by coordinate descent method without relying on any data structure assumption. Then by carefully eliminating redundant structures to avoid repeating computation, we propose a more efficient coordinate descent method to solve this problem. Furthermore, no parameter needs to be tuned in our algorithm. Finally, extensive experiments are conducted on the real world data sets to demonstrate the superiority of our algorithm.

Feature selection is one of the important topics of machine learning, and it has a wide range of applications in data preprocessing. At present, feature selection based on$\ell _{2,1}$-norm regularization is a relatively mature method, but it is not enough to maximize the sparsity and parameter-tuning leads to increased costs. Later scholars found that the$\ell _{2,0}$-norm constraint is more conductive to feature selection, but it is difficult to solve and lacks convergence guarantees. To address these problems, we creatively propose a novel Outliers Robust Unsupervised Feature Selection for structured sparse subspace (ORUFS), which utilizes$\ell _{2,0}$-norm constraint to learn a structured sparse subspace and avoid tuning the regularization parameter. Moreover, by adding binary weights, outliers are directly eliminated and the robustness of model is improved. More importantly, a Re-Weighted (RW) algorithm is exploited to solve our$\ell _{p}$-norm problem. For the NP-hard problem of$\ell _{2,0}$-norm constraint, we develop an effective iterative optimization algorithm with strict convergence guarantees and closed-form solution. Subsequently, we provide theoretical analysis about convergence and computational complexity. Experimental results on real-world datasets illustrate that our method is superior to the state-of-the-art methods in clustering and anomaly detection tasks.

Unsupervised feature selection (UFS)is an important technology for dimensionality reduction and has gained great interest in a wide range of fields. Recently, most popular methods are spectral-based which frequently use adaptive graph constraints to promote performance. However, no literature has considered the grouping characteristic of the data features, which is the most basic and important characteristic for arbitrary data. In this paper, based on the spectral analysis method, we first simulate the data feature grouping characteristic. Then, the similarity between data is adaptively reconstructed through the similarity between groups, which can explore the more fine-grained relationship between data than the previous adaptive graph methods. In order to achieve the aforementioned goal, the local similarity matrix and the global similarity matrix are defined, and the weighted KL entropy is used to constrain the relationship between the global similarity matrix and the local similarity matrices. Furthermore, the symmetrical self-representation structure is used to improve the performance of the reconstruction error term in the conventional spectral-based methods. After the model is constructed, a simple but efficient algorithm is proposed to solve the full model. Extensive experiments on 8 benchmark dataset with different types to show the effectiveness of the proposed method.

Feature selection has achieved unprecedented success in obtaining sparse discriminative features. However, the existing methods almost use the$\ell _{2,p}$-norm constraint on transformation matrix to obtain sparse features, which introduces extra parameters and cannot obtain the features directly. In addition, existing algorithms only focused on the global structure and ignored the local structure, leading to poor performance when solving data with non-Gaussian distributions which a single center point cannot describe precisely. Based on above considerations, we propose a supervised feature selection via multi-center and local structure learning. We further introduce trace ratio criterion into our model in favor of improving the discriminant of features selected. In order to address the overlap problem, we use multiple center points to match the distribution of data and construct a$k$-Nearest Neighbor graph to explore the local structure of the data. In addition, we also propose an efficient method to optimize the transformation matrix with the$\ell _{2,0}$-norm constraint and can directly obtain the sparse features. We evaluate our method on Toy datasets and several real-world datasets, show improvement over state-of-the-art feature selection methods, and demonstrate the effectiveness of our model in dealing with non-Gaussian distributed data problems.

In recent years, graph-based multiview clustering methods have become a research hotspot in the clustering field. However, most existing methods lack consideration of cluster balance in their results. In fact, cluster balance is crucial in many real-world scenarios. Additionally, graph-based multiview clustering methods often suffer from high time consumption and cannot handle large-scale datasets. To address these issues, this paper proposes a novel graph-based multiview clustering method. The method is built upon the bipartite graph. Specifically, it employs a label propagation mechanism to update the smaller anchor label matrix rather than the sample label matrix, significantly reducing the computational cost. The introduced balance constraint in the proposed model contributes to achieving balanced clustering results. The entire clustering model combines information from multiple views through graph fusion. The joint graph and view weight parameters in the model are obtained through task-driven self-supervised learning. Moreover, the model can directly obtain clustering results without the need for the two-stage processing typically used in general spectral clustering. Finally, extensive experiments on toy datasets and real-world datasets are conducted to validate the superiority of the proposed method in terms of clustering performance, clustering balance, and time expenditure.

This paper proposes a novel method, Enhanced Discrete Multi-modal Hashing (EDMH), which learns binary codes and hash functions simultaneously from the pairwise similarity matrix of data for large-scale cross-view retrieval. EDMH distinguishes itself from existing methods by considering not just the binarization constraint but also the balance and decorrelation constraints. Although those additional discrete constraints make the optimization problem of EDMH look a lot more complicated, we are actually able to develop a fast iterative learning algorithm in the alternating optimization framework for it, as after introducing a couple of auxiliary variables each subproblem of optimization turns out to have closed-form solutions. It has been confirmed by extensive experiments that EDMH can consistently deliver better retrieval performances than state-of-the-art MH methods at lower computational costs.

An explicit network structure plays an important role when analyzing and understanding diffusion processes. In many scenarios, however, the interactions between nodes in an underlying network are unavailable. Although many methods for inferring a network structure from observed cascades have been proposed, they did not perceive the relationship between pairwise interactions in a cascade. Therefore, this paper proposes a Pairwise-interactions-based Bayesian Inference method (named PBI) to infer the underlying diffusion network structure. More specifically, to get more accurate inference results, we measure the weights of each candidate pairwise interaction in different cascades and add them to the likelihood of a contagion process. In addition, a pre-pruning work is introduced for candidate edges to further improve the inference efficiency. Experiments on synthetic and real-world networks show that PBI achieves significantly better results.

The rapid diffusion of hazardous information in large-flow-based social media causes great economic losses and potential threats to society. It is crucial to infer the inner information source as early as possible to prevent further losses. However, existing localization methods wait until all deployed sensors obtain propagation information before starting source inference within a network, and hence the best opportunity to control propagation is missed. In this paper, we propose a new localization strategy based on finite deployed sensors, named Greedy-coverage-based Rapid Source Localization (GRSL), to rapidly, flexibly and accurately infer the source in the early propagation stage of large-scale networks. There are two phases in GRSL. In the first phase, the Greedy-based Strategy (GS) greedily deploys sensors to rapidly achieve wide area coverage at a low cost. In the second phase, when a propagation event within a network is observed by a part of the sensors, the Inference Strategy (IS) with an earlier response mechanism begins executing the source inference task in an earlier small infected area. Comprehensive experiments with the SOTA methods demonstrate the superior performance and robustness of GRSL in various application scenarios.

Since obtaining data labels is a time-consuming and laborious task, unsupervised feature selection has become a popular feature selection technique. However, the current unsupervised feature selection methods are facing three challenges: (1) they rely on a fixed similarity matrix derived from the original data, which will affect their performance; (2) due to the limitation of sparsity, they can only obtain sub-optimal solutions; (3) they have high computational complexity and cannot handle large-scale data. To solve this dilemma, we propose a fast unsupervised feature selection algorithm with bipartite graph and 2;0-norm constraint (BGCFS). We use the original data and the selected anchors to construct an adaptive bipartite graph in the subspace, and apply the l2,0-norm constraint to the projection matrix for feature selection. In this way, we can update the adaptive bipartite graph and the projection matrix simultaneously, and we can get the feature subset directly, without sorting the features. In addition, we propose an iterative algorithm that can solve the proposed problem globally to obtain a closed-form solution, and we provide a strict proof of convergence for it. Experiments on eight real data sets with different scales show that our method can select more valuable feature subsets more quickly

Fuzzy clustering algorithms have been widely used to reveal the possible hidden structure of data. However, with the increasing of data amount, large scale data has brought genuine challenges for fuzzy clustering. Most fuzzy clustering algorithms suffer from the long time-consumption problem since a large amount of distance calculations are involved to update the solution per iteration. To address this problem, we introduce the popular anchor graph technique into fuzzy clustering and propose a scalable fuzzy clustering algorithm referred to as Scalable Fuzzy Clustering with Anchor Graph (SFCAG). The main characteristic of SFCAG is that it addresses the scalability issue plaguing fuzzy clustering from two perspectives: anchor graph construction and membership matrix learning. Specifically, we select a small number of anchors and construct a sparse anchor graph, which is beneficial to reduce the computational complexity. We then formulate a trace ratio model, which is parameter-free, to learn the membership matrix of anchors to speed up the clustering procedure. In addition, the proposed method enjoys linear time complexity with the data size. Extensive experiments performed on both synthetic and real world datasets demonstrate the superiority (both effectiveness and scalability) of the proposed method over some representative large scale clustering methods.

Among a big body of recently developed algorithms for machine learning and data mining, a class of models using non-convex/non-smooth sparsity-inducing norms achieves promising results on many challenging tasks. An important problem faced with such models is to find an effective solution for the objective function with one or multiple intractable terms. Although a large number of optimization approaches have been developed, most of them are tailored to a specific model. Besides, these approaches generally introduce some additional parameters and no longer guarantee convergence. In this work, we first revisit some representative non-convex/non-smooth machine learning models, and then unity them into a generic formulation. Theoretically, we develop a simple yet efficient optimization framework, namely Iteratively Re-Weighted method (IRW), to solve such a class of models and provide the corresponding convergence analysis. Particularly, we validate our proposed method on two challenging machine learning tasks: multi-task regression and feature selection. Source codes are available at:https://github.com/KDD-Code/Sparse.git.

K-means is one of the most simple and popular clustering algorithms, which implemented as a standard clustering method in most of machine learning researches. The goal of K-means clustering is finding a set of cluster centers and minimizing the sum of squared distances between each sample and its nearest clustering center. In this paper, we proposed a novel K-means clustering algorithm, which reformulate the classical K-Means objective function as a trace maximization problem and then replace it with a new formulation. The proposed algorithm does not need to calculate the cluster centers in each iteration and requires fewer additional intermediate variables during the optimization process. In addition, we proposed an efficient iterative re-weighted algorithm to solve the involved optimization problem and provided the corresponding convergence analysis. The proposed algorithm keeps a consistent computational complexity as Lloyd's algorithm,$\mathcal {O}(ndk)$, but shows a faster convergence rate in experiments. Extensive experimental results on real world benchmark datasets show the effectiveness and efficiency of the proposed algorithm.

Multi-view clustering attracts considerable attention due to its effectiveness in unsupervised learning. However, previous multi-view spectral clustering methods include two separated steps: 1) Obtaining a spectral embedding; 2) Performing classical clustering methods. Although these methods have achieved promising performance, there is still some limitations. First, in computing spectral embedding, multi-view spectral clustering approaches exist high computational complexity since they usually need eigenvalue decomposition on laplacian matrix L. Its computational complexity is O(n3) where n is the number of samples; Second, in constructing similarity matrices, previous methods need to compute similarity between any two samples; Third, the two-stage approach only can obtain sub-optimal solution; Fourth, treating equally all views is unreasonable. To address these issues, we propose a Fast Multi-view Clustering via Prototype Graph (FMVPG) method. Specifically, the prototype graph is firstly constructed, and then simultaneously perform spectral embedding to obtain the real matrix and spectral rotation to get the indicator matrix. In addition, the Alternating Direction Method of Multipliers (ADMM) is used to solve the joint optimization problem. Further, we conduct extensive experiments to evaluate the proposed FMVPG approach. These experimental results show the comparable or even better clustering performance than the state-of-the-art approaches.

Spectral clustering is a vital clustering method and has been widely applied for data analysis and pattern reorganization. A routine of solving spectral clustering problem consists of two successive stages: (1) solving a relaxed continuous optimization problem to obtain a real-valued indicator solution (2) transform the real-valued indicator into a 0-1 discrete one as the final clustering result. However, we may lose the optimal solution with such a two-stage process. Besides, the spectral clustering has a high time complexity which limits the analysis of large-scale data. To alleviate these problems, this paper proposes an efficient spectral clustering framework that computes spectral embedding and improved spectral rotation simultaneously (SE-ISR). In addition, we also provide a parameter-free method (SE-ISR-PF) to automatically choose the trade-off parameter. Furthermore, with an anchor-based similarity matrix construction, it is scalable to large-scale data. An effective algorithm with a strict convergence proof is provided to solve the corresponding optimization problem. Experimental results on several benchmark datasets demonstrate that the proposed algorithm outperforms the state-of-art methods.

In recent decades, unsupervised feature selection methods have become increasingly popular. Nevertheless, most of the existing unsupervised feature selection methods suffer from two major problems that lead to suboptimal solutions. Many methods impose a hard linear projection constraint on original data, which is overly strict in nature and not suitable for dealing with data sampled from nonlinear manifolds. Second, most existing methods usel2,p-norm (02S and SF2SOG, which can simultaneously learn optimal flexible projections and obtain an orthogonal sparse projection to directly select discriminative features by applyingl2,0-norm constraint. Moreover, we propose to explore the local structure of flexible embedding through preserving the manifold structure of original data and adaptively constructing an optimal graph in subspace. Thirdly, the novel iterative optimization algorithms are presented to solve objective functions guaranteeing convergence theoretically. Various evaluation experiments on synthetic and real-world datasets demonstrate the effectiveness and superiority of our proposed methods.

Graph-based semi-supervised learning, which further utilizes graph structure behind samples for boosting semi-supervised learning, gains convincing results in several machine learning tasks. Nevertheless, existing graph-based methods have shortcomings from two aspects. On the one hand, many of them concentrate on improving label propagation over the constructed graph through time-saving methods, e.g. path searching, without giving insights on constructing a proper graph accommodated to samples. On the other hand, some models are only devoted to constructing the appropriate graph resulting in a two-stage procedure, which may incur a suboptimal scenario. In this paper, we develop a joint learning method that considers both bipartite graph construction and label propagation simultaneously. With this configuration, the constructed graph is constantly adjusted by the smoothness term in the objective as the algorithm proceeds. The time complexity of our method gets significant improvement compared with traditional graph-based methods, and the experimental results on one synthetic dataset and several real-world benchmarks demonstrate the effectiveness and scalability of our proposed method.

Given an affinity graph of data samples, graph-based clustering aims to partition these samples into disjoint groups based on the affinities, and most previous works are based on spectral clustering. However, two problems among spectral-based methods heavily affect the clustering performance. Firstly, the randomness of post-processing procedures, such as$K$-means, affects the stability of clustering. Secondly, the separated stages of spectral-based methods, including graph construction, spectral embedding learning, and clustering decision, lead to mismatched problems. In this paper, we explore a structured graph learning (SGL) framework that aims to fuse these stages to improve clustering stability. Specifically, SGL adaptively learns a structured affinity graph that contains exact$k$connected components. Each connected component corresponds to a cluster so clustering assignments can be directly obtained according to the connectivity of the learned graph. In this way, SGL avoids the randomness brought by reliance on traditional post-processing procedures. Meanwhile, the graph construction and structured graph learning procedures happen simultaneously, which alleviates the mismatched problem effectively. Moreover, we propose an efficient algorithm to solve the involved optimization problems and discuss the connections between this work and previous works. Numerical experiments on several synthetic and real datasets demonstrate the effectiveness of our methods.

Density structure and pairwise structure serve as two different but complementary perspectives for clustering. Either side of road is frequently visited and explored by multiple clustering methods. However, there are seldom approaches, which could mutually exploit both structures for clustering. To address this problem, in this paper, we develop a novel adaptive joint clustering algorithm, which combines unsupervised discrete orthogonal least squares discriminant analysis (DOLSDA) and discrete spectral clustering (DSC) with adaptive neighbors and side information into a unified model. Firstly, we extend supervised OLSDA to a discrete kernel clustering problem. To further achieve a clear pairwise structure, a new similarity with adaptive neighbors is then derived to establish sparse Laplacian matrix. In addition, side information could be incorporated to formulate clearer graph by modifying the proposed similarity. Based on the constructed graph, DSC is embedded with the discrete kernel OLSDA (DKOLSDA) clustering to exploit both cluster and pairwise data structures. Equipped with the proposed framework regarding quadratic weighted optimization, adaptive weight can be obtained automatically to leverage both unsupervised DKOLSDA and DSC. Since the unified problem is still discrete, we develop an increment scheme to achieve the optimal spectral rotation for the approximate solution to the predicted indicator.

Considering many graph clustering methods are with quadratic or cubic time complexity and need post-processing to obtain the discrete solution. Combining with the anchor graph, we study a novel graph clustering model called Rank-$r$Discrete Matrix Factorization (DMF-RR), which is linear time complexity, and motivated by nonnegative matrix factorization (NMF). Instead of constraining the factor matrices of NMF to be nonnegative as many existed methods, we constrain them to indicator matrices. Thus, DMF-RR can obtain the discrete solution by directly solving the original problem without post-processing. Furthermore, considering the greater similarity between samples of the same category, an anchor graph is constructed as an input to capture essential clustering structure by utilizing the duality information between samples and anchors. Subsequently, an efficient and simple algorithm is proposed due to the nature of indicator matrices. Extensive experiments performed on synthetic and real-world datasets demonstrate the superiority of DMF-RR.

Unsupervised feature selection can play an important role in addressing the issue of processing massive unlabelled high-dimensional data in the domain of machine learning and data mining. This paper presents a novel unsupervised feature selection method, referred to as Multi-Group Adaptive Graph Representation (MGAGR). Different from existing methods, the relationship between features is explored via the global similarity matrix, which is reconstructed by local similarities of multiple groups. Specifically, the similarity of a feature compared to other features can be represented by the linear combination of all the local similarities. The local similarity of a representative group is given a large weight to reconstruct the global similarity. Besides, an iterative algorithm is given to solve the optimization problem, in which the global similarity matrix, its corresponding reconstruction weights and the self-representation matrix are iteratively improved. Experimental results on 8 benchmark datasets demonstrates that the proposed method outperforms the state-of-the-art unsupervised feature selection methods in terms of clustering performance. The source code is available at:https://github.com/misteru/MGAGR.

Fuzzy K-Means (FKM) clustering regards each cluster as a fuzzy set and assigns each sample to multiple clusters with a certain degree of membership. However, conventional FKM methods perform clustering on original data directly where the intrinsic structure of data may be corrupted by the noise. According, the performance of these methods would be challenged. In this paper, we present a novel fuzzy K-Means clustering model to conduct clustering tasks on the flexible manifold. Technically, we perform fuzzy clustering based on the shrunk patterns which have desired manifold structure. The shrunk patterns can be viewed as an approximation to the original data; and a penalty term is employed to measure the mismatch between them. Moreover, we integrate the learning of shrunk patterns and the learning of membership degree between shrunk patterns and clusters into a unified framework. Furthermore, we extend the proposed model for projected FKM clustering to find a suitable subspace to fit the non-linear manifold structure of data, reduce the interference of the noise and redundant features and gather homogeneous samples together simultaneously. Two alternating iterative algorithms are derived to solve these two models, respectively. Extensive experimental results demonstrate the feasibility and effectiveness of our proposed clustering algorithms.

With its simplicity and effectiveness, k-means is immensely popular, but it cannot perform well on complex nonlinear datasets. Multiple kernel k-means (MKKM) demonstrates the ability to describe highly complex nonlinear separable data structures. However, its speed requirement cannot scale as well as the data size grows beyond tens of thousands. Nowadays, digital data explosion mandates more scalable clustering methods to assist the machine learning tasks in easy-to-access form. To address the issue, we propose to employ the Nystrom scheme for MKKM clustering, termed scalable multiple kernel k-means clustering. It significantly reduces the computational complexity by replacing the original kernel matrix with a low-rank approximation. Analytically and empirically, we demonstrate that our method performs as well as existing state-of-the-art methods, but at a significantly lower compute cost, allowing us to scale the method more effectively for clustering tasks.

Multiple Kernel K-means (MKKM) uses various kernels from different sources to improve clustering performance. However, most of the existing models are non-convex, which is prone to be stuck into bad local optimum, especially with noise and outliers. To address the issue, we propose a novel Self-Paced and Discrete Multiple Kernel K-Means (SPD-MKKM). It learns the MKKM model in a meaningful order by progressing both samples and kernels from easy to complex, which is beneficial to avoid bad local optimum. In addition, whereas existing methods optimize in two stages: learning the relaxation matrix and then finding the discrete one by extra discretization, our work can directly gain the discrete cluster indicator matrix without extra process. What's more, a well-designed alternative optimization is employed to reduce the overall computational complexity via using the coordinate descent technique. Finally, thorough experiments performed on real-world datasets illustrated the excellence and efficacy of our method.

Due to the exponential growth of multimedia data, multi-modal hashing as a promising technique to make cross-view retrieval scalable is attracting more and more attention. However, most of the existing multi-modal hashing methods either divide the learning process unnaturally into two separate stages or treat the discrete optimization problem simplistically as a continuous one, which leads to suboptimal results. Recently, a few discrete multi-modal hashing methods that try to address such issues have emerged, but they still ignore several important discrete constraints (such as the balance and decorrelation of hash bits). In this paper, we overcome those limitations by proposing a novel method named “Enhanced Discrete Multi-modal Hashing (EDMH)” which learns binary codes and hashing functions simultaneously from the pairwise similarity matrix of data, under the aforementioned discrete constraints. Although the model of EDMH looks a lot more complex than the other models for multi-modal hashing, we are actually able to develop a fast iterative learning algorithm for it, since the subproblems of its optimization all have closed-form solutions after introducing a couple of auxiliary variables. Our experimental results on three real-world datasets have revealed the usefulness of those previously ignored discrete constraints and demonstrated that EDMH not only performs much better than state-of-the-art competitors according to several retrieval metrics but also runs much faster than most of them.

Recently, many real-world applications exploit multi-view data, which is collected from diverse domains or obtained from various feature extractors and reflect different properties or distributions of the data. In this work, a novel unsupervised multi-view framework is proposed to cluster such data. The proposed method, called Multi-View clustering with Adaptive Sparse Memberships and Weight Allocation (MVASM), pays more attention to constructing a common membership matrix with proper sparseness over different views and learns the centroid matrix and its corresponding weight of each view. Concretely, MVASM method attempts to learn a common and flexible sparse membership matrix to indicate the clustering, which explores the underlying consensus information of multiple views, and solves the multiple centroid matrices and weights to utilize the view-specific information and further modifies the above-mentioned membership matrix. In addition, the theoretical analysis, including the determination of the power exponent parameter, convergence analysis, and complexity analysis are also presented. Compared to the state-of-the-art methods, the proposed method improves the performance of clustering on different public datasets and demonstrates its reasonability and superiority.

In this paper, a novel low-rank structural model is proposed for segmenting data drawn from a high-dimensional space. Our method is based on the fact that all groups clustered from a high-dimensional dataset are distributed in multiple low-rank subspaces. In general, it’s a very difficult task to find the low-rank structures hidden in data. Different from the classical sparse subspace clustering (SSC) and low-rank representation (LRR) which all take two steps including building the affinity matrix and spectral clustering, we introduce a new rank constraint into our model. This constraint allows our model to learn a subspace indicator which can capture different clusters directly from the data without any postprocessing. To further approximate the rank constraint, a piecewise function is utilized as the relaxing item for the proposed model. Besides, under the subspace indicator constraints, the integer programming problem is avoided, which makes our algorithm more efficient and scalable. In addition, we prove the convergence of the proposed algorithm in theory and further discuss the general case in which subspaces don’t pass through the origin. Experiment results on both synthetic and real-world datasets demonstrate that our algorithm significantly outperforms the state-of-the-art methods.

Semi-supervised learning as one of most attractive problems in machine learning research field has aroused broad attentions in recent years. In this paper, we propose a novel locality preserved dimensionality reduction framework, named Semi-supervised Adaptive Local Embedding learning (SALE), which learns a local discriminative embedding by constructing a$k_1$Nearest Neighbors ($k_1$NN) graph on labeled data, so as to explore the intrinsic structure, i.e., sub-manifolds from non-Gaussian labeled data. Then, mapping all samples into learned embedding and constructing another$k_2$NN graph on all embedded data to explore the global structure of all samples. Therefore, the unlabeled data and their corresponding labeled neighbors can be clustered into same sub-manifold, so as to improve the discriminative power of embedded data. Furthermore, we propose two semi-supervised dimensionality reduction methods with orthogonal and whitening constraints based on proposed SALE framework. An efficient alternatively iterative optimization algorithm is developed to solve the NP-hard problem in our models. Extensive experiments conducted on several synthetic and real-world data sets demonstrate the superiorities of our methods on local structure exploration and classification task.

There are many successful spectral based unsupervised dimensionality reduction methods, including Laplacian Eigenmap (LE), Locality Preserving Projection (LPP), Spectral Regression (SR), etc. We find that LPP and SR are equivalent if the symmetric similarity matrix is doubly stochastic, Positive Semi-Definite (PSD) and with rank$p$, where$p$is the reduced dimension. Since solving SR is believed faster than solving LPP based on some related literature, the discovery promotes us to seek to construct such specific similarity matrix to speed up LPP solving procedures. We then propose an unsupervised linear method called Unsupervised Large Graph Embedding (ULGE). ULGE starts with a similar idea as LPP but adopts an efficient approach to construct anchor-based similarity matrix and then performs spectral analysis on it. Moreover, since conventional anchor generation strategies suffer kinds of problems, we propose an efficient and effective anchor generation strategy, called Balanced$K$-means based Hierarchical$K$-means (BHKH). The computational complexity of ULGE can reduce to$O(ndm)$, which is a significant improvement compared to conventional methods need$O(n^2d)$at least, where$n$,$d$and$m$are the number of samples, dimensions, and anchors, respectively. Extensive experiments on several publicly available datasets demonstrate the efficiency and effectiveness of the proposed method.

Fuzzy K-Means (FKM) clustering is of great importance for analyzing unlabeled data. FKM algorithms assign each data point to multiple clusters with some degree of certainty measured by the membership function. In these methods, the fuzzy membership degree matrix is obtained based on the calculation of the distance between data points in the original space. However, this operation may lead to suboptimal results because of the influence of noises and redundant features. Besides, some FKM clustering methods ignore the importance of the weighting exponent. In this paper, we propose a novel FKM method called Fuzzy K-Means Clustering With Discriminative Embedding. Within this method, we simultaneously conduct dimensionality reduction along with fuzzy membership degree learning. To retain most information in the embedding subspace and improve the robustness of this method, principal component analysis is incorporated into our framework. An iterative optimization algorithm is proposed to solve the model. To validate the efficacy of the proposed method, we perform comprehensive analyses, including convergence behavior, parameter determination and computational complexity. Moreover, we also match a appropriate weighting exponent for each data set. Experimental results on benchmark data sets show that the proposed method is more discriminative and effective for clustering tasks.

Feature selection is one of the most important techniques to deal with the high-dimensional data for a variety of machine learning and data mining tasks, such clustering, classification, and retrieval, etc. Fuzziness is a widespread nature of data in nature human society. However, most existing feature selection methods ignore the existence of fuzziness in the data, resulting in sub-optimal feature subsets. To address the problem, we propose a novel unsupervised feature selection method, called Unsupervised Discriminative Projection for Feature Selection (UDPFS) to select discriminative features by conducting fuzziness learning and sparse learning, simultaneously. Specifically, we use projection matrix transform data as its low-dimensional representation, which are partitioned into clusters by using membership matrix with sparse constraint. In addition,$\ell _{2, 1}$-norm regularization is applied to the projection matrix. Then, a discriminative projection matrix with row sparse is obtained by perform fuzziness learning and sparse learning, simultaneously. An effective alternative optimization algorithm is proposed to solve the objective function. Evaluate experimental results on several real-world datasets show the effectiveness and superiority of the proposed unsupervised feature selection method.

Community structure division is a crucial problem in the field of network data analysis. Algorithms based on Markov chains are easy to use and provide promising solutions for community detection. In a Markov chain-based algorithm (i.e., MCL), a flow distribution matrix and a transition matrix are used to describe stochastic flows and transition probabilities, respectively, on a network. The dynamic interaction process between stochastic flows and transition probabilities in MCLs is manifested through an iterative process of updating the abovementioned two matrices. As one of the key mechanisms of MCLs, such a dynamic process for increasing the inhomogeneity directly affects the accuracy and computational cost of MCL-based methods. Inspired by a kind of positive feedback interaction of a dendritic network of tube-like amoeba cell pseudopodia (named thePhysarumforaging network), aPhysarum-inspired relationship among vertices is proposed to enhance the transition probability in the dynamic process of MCL-based community detection algorithms. Specifically, the proposed hybrid community detection algorithm can adaptively search for a better combination of parameters based on a genetic algorithm. Some experiments are carried out on both static and dynamic networks. The results show that the uniquePhysaruminspired algorithm achieved better computational efficiency and detection performance than other algorithms.

Stego-images are often contaminated by interchannel noise or active noise attack when communicating on the Web. And it is challenging to restore embedded image from corrupted stego-image. This paper studies akNN-bit approximation algorithm to remove noises in embedded image. The proposed algorithm distinguishes reliable bits from extracted bits, and estimates pixel values by keeping reliable bits unchanged and correcting unreliable bits. Specifically, the 8th (highest) unreliable bit of a pixel can be approximated with its nearest neighbor pixels. And then, if an unreliable bit locates at any one of the$5^{th}\sim 7^{th}$bits of a pixel, it is adjusted with two nearest neighbors of the pixel, where the pixel is in-between these two nearest neighbors. Finally, for other unreliable bits, each one is approximated by the maximum and minimum possible values of nearest neighbors of its pixel. We conduct experiments for illustrating the efficiency, and demonstrate that the proposed algorithm can recover the embedded images with good visual quality from corrupted stego-images.

Multi-task learning has been observed by many researchers, which supposes that different tasks can share a low-rank common yet latent subspace. It means learning multiple tasks jointly is better than learning them independently. In this paper, we propose two novel multi-task learning formulations based on two regularization terms, which can learn the optimal shared latent subspace by minimizing the exactly k minimal singular values. The proposed regularization terms are the more tight approximations of rank minimization than trace norm. But it's an NP-hard problem to solve the exact rank minimization problem. Therefore, we design a novel re-weighted based iterative strategy to solve our models, which can tactically handle the exact rank minimization problem by setting a large penalizing parameter. Experimental results on benchmark datasets demonstrate that our methods can correctly recover the low-rank structure shared across tasks, and outperform related multi-task learning methods.

Medical diagnosis through artificial intelligence has been drawing increasing attention currently. For breast lesions, the clinical ultrasound reports are the most commonly used data in the diagnosis of breast cancer. Nevertheless, the input reports always encounter the inevitable issue of data missing. Unfortunately, despite the efforts made in previous approaches that made progress on tackling data imprecision, nearly all of these approaches cannot accept inputs with data missing. A common way to alleviate the data missing issue is to fill the missing values with artificial data. However, the data filling strategy actually brings in additional noises that do not exist in the raw data. Inspired by the advantage of open world assumption, we regard the missing data in clinical ultrasound reports as non-observed terms of facts, and propose a Knowledge Graph embedding based model KGSeD with the capability of tolerating data missing, which can successfully circumvent the pollution caused by data filling. Our KGSeD is designed via an encoder-decoder framework, where the encoder incorporates structural information of the graph via embedding, and the decoder diagnose patients by inferring their links to clinical outcomes. Comparative experiments show that KGSeD achieves noticeable diagnosis performances. When data missing occurred, KGSeD yields the most stable performance over those of existing approaches, showing better tolerance to data missing.

Spectral clustering is a quite simple but effective method for solving graph clustering problem. It projects the original data points into a lower dimensional space with spectral embedding, and then relies on an algorithm to obtain the cluster labels. Since it involves eigendecomposition of the graph Laplacian matrix for embedding, spectral clustering has high time complexity and is not able to process large scale data. The performance of spectral clustering is also limited by a post-processing algorithm such as kmeans. To tackle the two issues, we propose a method called Orthogonal and Nonnegative Graph Reconstruction (ONGR) for large scale clustering. The two constraints serve as a structure constraint with which the graph reconstructed by the indicator matrix is structured. The proposed method has linear time complexity with respect to the data size that it mainly needs to implicitly construct a graph and iteratively perform economical singular value decomposition for a small size matrix. Moreover, the interpretability of the indicator matrix is offered due to the nonnegative constraint, and thus our method can provide the cluster labels with no post-processing. The experiments on benchmark datasets show the effectiveness of the proposed scalable clustering method.

Recently, with the explosive increase in Internet data, the traditional Graph-based Semi-Supervised Learning (GSSL) model is not suitable to deal with large scale data as the high computation complexity. Besides, GSSL models perform classification on a fixed input data graph. The quality of initialized graph has a great effect on the classification result. To solve this problem, in this paper, we propose a novel approach, named optimal bipartite graph-based SSL (OBGSSL). Instead of fixing the input data graph, we learn a new bipartite graph to make the result more robust. Based on the learned bipartite graph, the labels of the original data and anchors can be calculated simultaneously, which solves co-classification problem in SSL. Then, we use the label of anchor to handle out-of-sample problem, which preserves well classification performance and saves much time. The computational complexity of OBGSSL is O(ndmt+nm2), which is a significant improvement compared with traditional GSSL methods that need O(n2d+n3), where n, d, m and t are the number of samples, features anchors and iterations, respectively. Experimental results demonstrate the effectiveness and efficiency of our OBGSSL model.

Simultaneous feature selection and clustering is a major challenge in unsupervised learning. In particular, there has been significant research into saliency measures for features that result in good clustering. However, as datasets become larger and more complex, there is a need to adopt a finer-grained approach to saliency by measuring it in relation to a part of a model. Another issue is learning the feature saliency and advanced model parameters. We address the first by presenting a novel Gaussian mixture model, which explicitly models the dependency of individual mixture components on each feature giving a new component-based feature saliency measure. For the second, we use Markov Chain Monte Carlo sampling to estimate the model and hidden variables. Using a synthetic dataset, we demonstrate the superiority of our approach, in terms of clustering accuracy and model parameter estimation, over an approach using a model-based feature saliency with expectation maximisation. We performed an evaluation of our approach with six synthetic trajectory datasets obtaining an average clustering accuracy of 97 percent. To demonstrate the generality of our approach, we applied it to a network traffic flow dataset obtaining an accuracy of 93 percent for intrusion detection. Finally, we performed a comparison with state-of-the-art clustering techniques using three real-world trajectory datasets of vehicle traffic. Our approach achieved an average clustering accuracy of 96 percent compared to 77-95 percent for the other techniques. In conclusion, for the datasets considered, component based feature saliency measures gave improved clustering over those based on whole models.

Unsupervised feature selection has attracted more and more attention due to the rapid growth of the large amount of unlabelled and high-dimensional data. The performance of traditional spectral-based unsupervised methods always depends on the quality of constructed similarity matrix. However, real world data always contain a large number of noise samples and features that make the similarity matrix created by original data cannot be fully relied. We propose an unsupervised feature selection method which conducts feature selection and local structure learning simultaneously. Moreover, we add an important constraint on the similarity matrix to allow it to capture more accurate information of the data structure. To perform feature selection, orthogonal constraint and `2;p-norm are adopted on the projection matrix. An efficient and simple algorithm is derived to tackle the problem. We conduct comprehensive experiments on various benchmark data sets, including handwritten digit, face image, and biomedical data, to validate the effectiveness of the proposed approach.

Dimensionality reduction plays a significant role in high-dimensional data processing, and Linear Discriminant Analysis (LDA) is a widely used supervised dimensionality reduction approach. However, a major drawback of LDA is that it is incapable of extracting the local structure information, which is crucial for handling multimodal data. In this article, we propose a novel supervised dimensionality reduction method named Adaptive Local Linear Discriminant Analysis (ALLDA), which adaptively learns a k -nearest neighbors graph from data themselves to extract the local connectivity of data. Furthermore, the original high-dimensional data usually contains noisy and redundant features, which has a negative impact on the evaluation of neighborships and degrades the subsequent classification performance. To address this issue, our method learns the similarity matrix and updates the subspace simultaneously so that the neighborships can be evaluated in the optimal subspaces where the noises have been removed. Through the optimal graph embedding, the underlying sub-manifolds of data in intra-class can be extracted precisely. Meanwhile, an efficient iterative optimization algorithm is proposed to solve the minimization problem. Promising experimental results on synthetic and real-world datasets are provided to evaluate the effectiveness of proposed method.

Breast cancer is now considered as one of the leading causes of deaths among women all over the world. Aiming to assist clinicians in improving the accuracy of diagnostic decisions, computer-aided diagnosis (CAD) system is of increasing interest in breast cancer detection and analysis nowadays. In this paper, a novel computer-aided diagnosis scheme with human-in-the-loop is proposed to help clinicians identify the benign and malignant breast tumors in ultrasound. In this framework, feature acquisition is performed by a user-participated feature scoring scheme that is based on Breast Imaging Reporting and Data System (BI-RADS) lexicon and experience of doctors. Biclustering mining is then used as a useful tool to discover the column consistency patterns on the training data. The patterns frequently appearing in the tumors with the same label can be regarded as a potential diagnostic rule. Subsequently, the diagnostic rules are utilized to construct component classifiers of the Adaboost algorithm via a novel rules combination strategy which resolves the problem of classification in different feature spaces (PC-DFS). Finally, the AdaBoost learning is performed to discover effective combinations and integrate them into a strong classifier. The proposed approach has been validated using a large ultrasounic dataset of 1,062 breast tumor instances (including 418 benign cases and 644 malignant cases) and its performance was compared with several conventional approaches. The experimental results show that the proposed method yielded the best prediction performance, indicating a good potential in clinical applications.

Graph clustering plays an important role in data mining. Based on an input data graph, data points are partitioned into clusters. However, most existing methods keep the data graph fixed during the clustering procedure, so they are limited to exploit the implied data manifold and highly dependent on the initial graph construction. Inspired by the recent development on manifold learning, this paper proposes an Adaptive Consistency Propagation (ACP) method for graph clustering. In order to utilize the features captured from different perspectives, we further put forward the Multi-view version of the ACP model (MACP). The main contributions are threefold: (1) the manifold structure of input data is sufficiently exploited by propagating the topological connectivities between data points from near to far; (2) the optimal graph for clustering is learned by taking graph learning as a part of the optimization procedure; and (3) the negotiation among the heterogeneous features is captured by the multi-view clustering model. Extensive experiments on real-world datasets validate the effectiveness of the proposed methods on both single-and multi-view clustering, and show their superior performance over the state-of-the-arts.

Non-negative Matrix Factorization (NMF) has been one of the most popular clustering techniques in machine leaning, and involves various real-world applications. Most existing works perform matrix factorization on high-dimensional data directly. However, the intrinsic data structure is always hidden within the low-dimensional subspace. And, the redundant features within the input space may affect the final result adversely. In this paper, a new unsupervised matrix factorization method, Discrimination-aware Projected Matrix Factorization (DPMF), is proposed for data clustering. The main contributions are threefold: (1) The linear discriminant analysis is jointly incorporated into the unsupervised matrix factorization framework, so the clustering can be accomplished in the discriminant subspace. (2) The manifold regularization is introduced to perceive the geometric information, and the ℓ2,1-norm is utilized to improve the robustness. (3) An efficient optimization algorithm is designed to solve the proposed problem with proved convergence. Experimental results on one toy dataset and eight real-world benchmarks show the effectiveness of the proposed method.

Traditional graph-based Semi-Supervised Learning (SSL) methods usually contain two separate steps. First, constructing an affinity matrix. Second, inferring the unknown labels. While such a two-step method has been successful, it cannot take full advantage of the correlation between affinity matrix and label information. In order to address the above problem, we propose a novel graph-based SSL method. It can learn the affinity matrix and infer the unknown labels simultaneously. Moreover, feature selection with auto-weighting is introduced to extract the effective and robust features. Further, the proposed method learns the data similarity matrix by assigning the adaptive neighbors for each data point based on the local distance. We solve the unified problem via an alternative minimization algorithm. Extensive experimental results on synthetic data and benchmark data show that the proposed method consistently outperforms the state-of-the-art approaches.

Semi-supervised learning models for multiview data are important in image classification tasks, since heterogeneous features are easy to obtain and semi-supervised schemes are economical and effective. To model the view importance, conventional graph-based multiview learning models learn a linear combination of views while assuming a priori weights distribution. In this paper, we present a novel structural regularized semi-supervised model for multiview data, termed Adaptive MUltiview SEmi-supervised model (AMUSE). Our new model learns weights from a priori graph structure, which is more reasonable than weight regularization. Theoretical analysis reveals the significant difference between AMUSE and the prior arts. An efficient optimization algorithm is provided to solve the new model. Experimental results on six real-world data sets demonstrate the effectiveness of the structural regularized weights learning scheme.

In many real-world applications, we are often confronted with high dimensional data which are represented by various heterogeneous views. How to cluster this kind of data is still a challenging problem due to the curse of dimensionality and effectively integration of different views. To address this problem, we propose two parameter-free weighted multi-view projected clustering methods which perform structured graph learning and dimensionality reduction simultaneously. We can use the obtained structured graph directly to extract the clustering indicators, without performing other discretization procedures as previous graph-based clustering methods have to do. Moreover, two parameter-free strategies are adopted to learn an optimal weight for each view automatically, without introducing a regularization parameter as previous methods do. Extensive experiments on several public datasets demonstrate that the proposed methods outperform other state-of-the-art approaches and can be used more practically.

With the explosive growth of data, the multi-view data is widely used in many fields, such as data mining, machine learning, computer vision, and so on. Because such data always has a complex structure, i.e., many categories, many perspectives of description and high dimension, how to formulate an accurate and reliable framework for the multi-view classification is a very challenging task. In this paper, we propose a novel multi-view classification method by using multiple multi-class Support Vector Machines (SVMs) with a novel collaborative strategy. Here, each multi-class SVM embeds the scaling factor to renewedly adjust the weight allocation of all features, which is beneficial to highlight more important and discriminative features. Furthermore, we adopt the decision function values to integrate multiple multi-class learners and introduce the confidence score across multiple classes to determine the final classification result. In addition, through a series of the mathematical deduction, we bridge the proposed model with the solvable problem and solve it through an alternating iteration optimization method. We evaluate the proposed method on several image and face datasets, and the experimental results demonstrate that our proposed method performs better than other state-of-the-art learning algorithms.

In this paper, we make an extension of K-means for the clustering of multiple means. The popular K-means clustering uses only one center to model each class of data. However, the assumption on the shape of the clusters prohibits it to capture the non-convex patterns. Moreover, many categories consist of multiple subclasses which obviously cannot be represented by a single prototype. We propose a K-Multiple-Means (KMM) method to group the data points with multiple sub-cluster means into specified k clusters. Unlike the methods which use the agglomerative strategies, the proposed method formalizes the multiple-means clustering problem as an optimization problem and updates the partitions of m sub-cluster means and k clusters by an alternating optimization strategy. Notably, the partition of the original data with multiple-means representation is modeled as a bipartite graph partitioning problem with the constrained Laplacian rank. We also show the theoretical analysis of the connection between our method and the K-means clustering. Meanwhile, KMM is linear scaled with respect to n. Experimental results on several synthetic and well-known real-world data sets are conducted to show the effectiveness of the proposed algorithm.

Sparse regression based feature selection method has been extensively investigated these years. However, because it has a non-convex constraint, i.e., $\ell _{2,0}$ℓ2,0-norm constraint, this problem is very hard to solve. In this paper, unlike most of the other methods which only solve its slack version by introducing sparsity regularization into objective function forcibly, a novel framework is proposed by us to solve the original $\ell _{2,0}$ℓ2,0-norm constrained sparse regression based feature selection problem. We transform our objective function into Linear Discriminant Analysis (LDA) by using a new label coding method, thus enabling our model to calculate the ratio of inter-class scatter to intra-class scatter of features which is the most widely used feature discrimination evaluation metric. According to that ratio, features can be selected by a simple sorting method. The projection gradient descent algorithm is introduced to further improve the performance of our algorithm by using the solution obtained before as its initial solution. This ensures the stability of this iterative algorithm. We prove that the proposed method can get the global optimal solution of this non-convex problem when all features are statistically independent. For the general case where features are statistically dependent, extensive experiments on six small sample size datasets and one large-scale dataset show that our algorithm has comparable or better classification capability comparing with other eight state-of-the-art feature selection methods by the SVM classifier. We also show that our algorithm can obtain a low loss value, which means the solution of our algorithm can get very close to this NP-hard problem’s real solution. What is more, because we solve the original $\ell _{2,0}$ℓ2,0-norm constrained problem, we avoid the heavy work of tuning the regularization parameter because its meaning is explicit in our method, i.e., the number of selected features. At last, we evaluate the stability of our algorithm from two perspectives, i.e., the objective function values and the selected features, by experiments. From both perspectives, our algorithm shows satisfactory stability performance.

As one of the most widely used clustering techniques, the fuzzy K-Means (also called FKM or FCM) assigns every data point to each cluster with a certain degree of membership. However, conventional FKM approach relies on the square data fitting term which is not robust to data outliers and ignores the prior information, which leads to unsatisfactory clustering results. In this paper, we present a novel and robust fuzzy K-Means clustering algorithm, namely Embedding Fuzzy K-Means with Nonnegative Spectral Clustering via Incorporating Side Information. The proposed method combines fuzzy K-Means with nonnegative spectral clustering into a unified model, and further takes the advantage of the prior knowledge of data pairs such that the quality of similarity graph is enhanced and the clustering performance is effectively improved. Besides, the ℓ2,1-norm loss function is adopted in the objective function, which achieves better robustness to outliers. Last, experimental results on benchmark datasets verify the effectiveness and superiority of the proposed clustering method.

This paper proposes a novel algorithm, named Non-Convex Calibrated Multi-Task Learning (NC-CMTL), for learning multiple related regression tasks jointly. Instead of utilizing the nuclear norm, NC-CMTL adopts a non-convex low rank regularizer to explore the shared information among different tasks. In addition, considering that the regularization parameter for each regression task desponds on its noise level, we replace the least squares loss function by square-root loss function. Computationally, as proposed model has a nonsmooth loss function and a non-convex regularization term, we construct an efcient re-weighted method to optimize it. Theoretically, we frst present the convergence analysis of constructed method, and then prove that the derived solution is a stationary point of original problem. Particularly, the regularizer and optimization method used in this paper are also suitable for other rank minimization problems. Numerical experiments on both synthetic and real data illustrate the advantages of NC-CMTL over several state-of-the-art methods.

In this paper, we make a multiview extension of the spectral rotation technique raised in single view spectral clustering research. Since spectral rotation is closely related to the Procrustes Analysis for points matching, we point out that classical Procrustes Average approach can be used for multiview clustering. Besides, we show that direct applying Procrustes Average (PA) in multiview tasks may not be optimal theoretically and empirically, since it does not take the clustering capacity differences of different views into consideration. Other than that, we propose an Adaptively Weighted Procrustes (AWP) approach to overcome the aforementioned deficiency. Our new AWP weights views with their clustering capacities and forms a weighted Procrustes Average problem accordingly. The optimization algorithm to solve the new model is computational complexity analyzed and convergence guaranteed. Experiments on five real-world datasets demonstrate the effectiveness and efficiency of the new models.

Nonnegative matrix factorization (NMF) is one widely used feature extraction technology in the tasks of image clustering and image classification. For the former task, various unsupervised NMF methods based on the data distribution structure information have been proposed. While for the latter task, the label information of the dataset is one very important guiding. However, most previous proposed supervised NMF methods emphasis on imposing the discriminant constraints on the coefficient matrix. When dealing with new coming samples, the transpose or the pseudoinverse of the basis matrix is used to project these samples to the low dimension space. In this way, the label influence to the basis matrix is indirect. Although, there are also some methods trying to constrain the basis matrix in NMF framework, either they only restrict within-class samples or impose improper constraint on the basis matrix. To address these problems, in this article a novel NMF framework named discriminative and orthogonal subspace constraints-based nonnegative matrix factorization (DOSNMF) is proposed. In DOSNMF, the discriminative constraints are imposed on the projected subspace instead of the directly learned representation. In this manner, the discriminative information is directly connected with the projected subspace. At the same time, an orthogonal term is incorporated in DOSNMF to adjust the orthogonality of the learned basis matrix, which can ensure the orthogonality of the learned subspace and improve the sparseness of the basis matrix at the same time. This framework can be implemented in two ways. The first way is based on the manifold learning theory. In this way, two graphs, i.e., the intrinsic graph and the penalty graph, are constructed to capture the intra-class structure and the inter-class distinctness. With this design, both the manifold structure information and the discriminative information of the dataset are utilized. For convenience, we name this method as the name of the framework, i.e., DOSNMF. The second way is based on the Fisher’s criterion, we name it Fisher’s criterion-based DOSNMF (FDOSNMF). The objective functions of DOSNMF and FDOSNMF can be easily optimized using multiplicative update (MU) rules. The new methods are tested on five datasets and compared with several supervised and unsupervised variants of NMF. The experimental results reveal the effectiveness of the proposed methods.

Co-saliency detection is a newly emerging and rapidly growing research area in the computer vision community. As a novel branch of visual saliency, co-saliency detection refers to the discovery of common and salient foregrounds from two or more relevant images, and it can be widely used in many computer vision tasks. The existing co-saliency detection algorithms mainly consist of three components: extracting effective features to represent the image regions, exploring the informative cues or factors to characterize co-saliency, and designing effective computational frameworks to formulate co-saliency. Although numerous methods have been developed, the literature is still lacking a deep review and evaluation of co-saliency detection techniques. In this article, we aim at providing a comprehensive review of the fundamentals, challenges, and applications of co-saliency detection. Specifically, we provide an overview of some related computer vision works, review the history of co-saliency detection, summarize and categorize the major algorithms in this research area, discuss some open issues in this area, present the potential applications of co-saliency detection, and finally point out some unsolved challenges and promising future works. We expect this review to be beneficial to both fresh and senior researchers in this field and to give insights to researchers in other related areas regarding the utility of co-saliency detection algorithms.

Automatic age estimation is an important yet challenging problem. It has many promising applications in social media. Of the existing age estimation algorithms, the personalized approaches are among the most popular ones. However, most person-specific approaches rely heavily on the availability of training images across different ages for a single subject, which is usually difficult to satisfy in practical application of age estimation. To address this limitation, we first propose a new model called Orthogonal Gaussian Process (OGP), which is not restricted by the number of training samples per person. In addition, without sacrifice of discriminative power, OGP is much more computationally efficient than the standard Gaussian Process. Based on OGP, we then develop an effective age estimation approach, namely anisotropic OGP (A-OGP), to further reduce the estimation error. A-OGP is based on an anisotropic noise level learning scheme that contributes to better age estimation performance. To finally optimize the performance of age estimation, we propose a multifeature A-OGP fusion framework that uses multiple features combined with a random sampling method in the feature space. Extensive experiments on several public domain face aging datasets (FG-NET, MORPH Album1, and MORPH Album 2) are conducted to demonstrate the state-of-the-art estimation accuracy of our new algorithms.

The unprecedent growth of microblog services poses significant challenges on network traffic and service latency to the underlay infrastructure (i.e., geo-distributed data centers). Furthermore, the dynamic evolution in microblog status generates a huge workload on data consistence maintenance. In this article, motivated by insights of cross-media analysis-based propagation patterns, we propose a novel cache strategy for microblog service systems to reduce the inter-data center traffic and consistence maintenance cost, while achieving low service latency. Specifically, we first present a microblog classification method, which utilizes the external knowledge from correlated domains, to categorize microblogs. Then we conduct a large-scale measurement on a representative online social network system to study the category-based propagation diversity on region and time scales. These insights illustrate social common habits on creating and consuming microblogs and further motivate our architecture design. Finally, we formulate the content cache problem as a constrained optimization problem. By jointly using the Lyapunov optimization framework and simplex gradient method, we find the optimal online control strategy. Extensive trace-driven experiments further demonstrate that our algorithm reduces the system cost by 24.5% against traditional approaches with the same service latency.

Nonnegative matrix factorization (NMF) is one of the most popular data representation methods in the field of computer vision and pattern recognition. High-dimension data are usually assumed to be sampled from the submanifold embedded in the original high-dimension space. To preserve the locality geometric structure of the data,k-nearest neighbor (k-NN) graph is often constructed to encode the near-neighbor layout structure. However,k-NN graph is based on Euclidean distance, which is sensitive to noise and outliers. In this article, we propose a refined-graph regularized nonnegative matrix factorization by employing a manifold regularized least-squares regression (MRLSR) method to compute the refined graph. In particular, each sample is represented by the whole dataset regularized with ℓ2-norm and Laplacian regularizer. Then a MRLSR graph is constructed based on the representative coefficients of each sample. Moreover, we present two optimization schemes to generate refined-graphs by employing a hard-thresholding technique. We further propose two refined-graph regularized nonnegative matrix factorization methods and use them to perform image clustering. Experimental results on several image datasets reveal that they outperform 11 representative methods.

Image annotation assigns relevant tags to query images based on their semantic contents. Since Non-negative Matrix Factorization (NMF) has the strong ability to learn parts-based representations, recently, a number of algorithms based on NMF have been proposed for image annotation and have achieved good performance. However, most of the efforts have focused on the representations of images and annotations. The properties of the semantic parts have not been well studied. In this article, we revisit the sparseness-constrained NMF (sNMF) proposed by Hoyer [2004]. By endowing the sparseness constraint with a geometric interpretation and sNMF with theoretical analyses of the generalization ability, we show that NMF with such a sparseness constraint has three advantages for image annotation tasks: (i) The sparseness constraint is more ℓ 0 -norm oriented than the ℓ 1 -norm-based sparseness, which significantly enhances the ability of NMF to robustly learn semantic parts. (ii) The sparseness constraint has a large cone interpretation and thus allows the reconstruction error of NMF to be smaller, which means that the learned semantic parts are more powerful to represent images for tagging. (iii) The learned semantic parts are less correlated, which increases the discriminative ability for annotating images. Moreover, we present a new efficient large sparse cone NMF (LsCNMF) algorithm to optimize the sNMF problem by employing the Nesterov’s optimal gradient method. We conducted experiments on the PASCAL VOC07 dataset and demonstrated the effectiveness of LsCNMF for image annotation.

The K Nearest Neighbor (kNN) method has widely been used in the applications of data mining and machine learning due to its simple implementation and distinguished performance. However, setting all test data with the same k value in the previous kNN methods has been proven to make these methods impractical in real applications. This article proposes to learn a correlation matrix to reconstruct test data points by training data to assign different k values to different test data points, referred to as the Correlation Matrix kNN (CM-kNN for short) classification. Specifically, the least-squares loss function is employed to minimize the reconstruction error to reconstruct each test data point by all training data points. Then, a graph Laplacian regularizer is advocated to preserve the local structure of the data in the reconstruction process. Moreover, an ℓ 1 -norm regularizer and an ℓ 2, 1 -norm regularizer are applied to learn different k values for different test data and to result in low sparsity to remove the redundant/noisy feature from the reconstruction process, respectively. Besides for classification tasks, the kNN methods (including our proposed CM-kNN method) are further utilized to regression and missing data imputation. We conducted sets of experiments for illustrating the efficiency, and experimental results showed that the proposed method was more accurate and efficient than existing kNN methods in data-mining applications, such as classification, regression, and missing data imputation.

We study content-based learning to rank from the perspective of learning distance functions. Standardly, the two key issues of learning to rank, feature mappings and score functions, are usually modeled separately, and the learning is usually restricted to modeling a linear distance function such as the Mahalanobis distance. However, the modeling of feature mappings and score functions are mutually interacted, and the patterns underlying the data are probably complicated and nonlinear. Thus, as a general nonlinear distance family, the Bregman distance is a suitable distance function for learning to rank, due to its strong generalization ability for distance functions, and its nonlinearity for exploring the general patterns of data distributions. In this paper, we study learning to rank as a structural learning problem, and devise a Bregman distance function to build the ranking model based on structural SVM. To improve the model robustness to outliers, we develop a robust structural learning framework for the ranking model. The proposed model Robust Structural Bregman distance functions Learning to Rank (RSBLR) is a general and unified framework for learning distance functions to rank. The experiments of data ranking on real-world datasets show the superiority of this method to the state-of-the-art literature, as well as its robustness to the noisily labeled outliers.

Modeling the process of information diffusion is a challenging problem. Although numerous attempts have been made in order to solve this problem, very few studies are actually able to simulate and predict temporal dynamics of the diffusion process. In this paper, we propose a novel information diffusion model, namely GT model, which treats the nodes of a network as intelligent and rational agents and then calculates their corresponding payoffs, given different choices to make strategic decisions. By introducing time-related payoffs based on the diffusion data, the proposed GT model can be used to predict whether or not the user's behaviors will occur in a specific time interval. The user's payoff can be divided into two parts: social payoff from the user's social contacts and preference payoff from the user's idiosyncratic preference. We here exploit the global influence of the user and the social influence between any two users to accurately calculate the social payoff. In addition, we develop a new method of presenting social influence that can fully capture the temporal dynamics of social influence. Experimental results from two different datasets, Sina Weibo and Flickr demonstrate the rationality and effectiveness of the proposed prediction method with different evaluation metrics.

Spectral clustering has been playing a vital role in various research areas. Most traditional spectral clustering algorithms comprise two independent stages (e.g., first learning continuous labels and then rounding the learned labels into discrete ones), which may cause unpredictable deviation of resultant cluster labels from genuine ones, thereby leading to severe information loss and performance degradation. In this work, we study how to achieve discrete clustering as well as reliably generalize to unseen data. We propose a novel spectral clustering scheme which deeply explores cluster label properties, including discreteness, nonnegativity, and discrimination, as well as learns robust out-of-sample prediction functions. Specifically, we explicitly enforce a discrete transformation on the intermediate continuous labels, which leads to a tractable optimization problem with a discrete solution. Besides, we preserve the natural nonnegative characteristic of the clustering labels to enhance the interpretability of the results. Moreover, to further compensate the unreliability of the learned clustering labels, we integrate an adaptive robust module with ℓ2ploss to learn prediction function for grouping unseen data. We also show that the out-of-sample component can inject discriminative knowledge into the learning of cluster labels under certain conditions. Extensive experiments conducted on various data sets have demonstrated the superiority of our proposal as compared to several existing clustering approaches.

Heterogeneous face recognition, also known as cross-modality face recognition or intermodality face recognition, refers to matching two face images from alternative image modalities. Since face images from different image modalities of the same person are associated with the same face object, there should be mutual components that reflect those intrinsic face characteristics that are invariant to the image modalities. Motivated by this rationality, we propose a novel approach called Mutual Component Analysis (MCA) to infer the mutual components for robust heterogeneous face recognition. In the MCA approach, a generative model is first proposed to model the process of generating face images in different modalities, and then an Expectation Maximization (EM) algorithm is designed to iteratively learn the model parameters. The learned generative model is able to infer the mutual components (which we call the hidden factor , where hidden means the factor is unreachable and invisible, and can only be inferred from observations) that are associated with the person’s identity, thus enabling fast and effective matching for cross-modality face recognition. To enhance recognition performance, we propose an MCA-based multiclassifier framework using multiple local features. Experimental results show that our new approach significantly outperforms the state-of-the-art results on two typical application scenarios: sketch-to-photo and infrared-to-visible face recognition.

Presents the introductory editorial for this issue of the publication.

In this article, we present a novel indexing technique called EMIF (Effective Music Indexing Framework) to facilitate scalable and accurate content based music retrieval. It is designed based on a "classification-and-indexing" principle and consists of two main functionality layers: 1) a novel semantic-sensitive classification to identify input music's category and 2) multiple indexing structures - one local indexing structure corresponds to one semantic category. EMIF's layered architecture not only enables superior search accuracy but also reduces query response time significantly. To evaluate the system, a set of comprehensive experimental studies have been carried out using large test collection and EMIF demonstrates promising performance over state-of-the-art approaches.

Effective representation of image texture is important for an image-classification task. Statistical modelling in wavelet domains has been widely used to image texture representation. However, due to the intraclass complexity and interclass diversity of textures, it is hard to use a predefined probability distribution function to fit adaptively all wavelet subband coefficients of different textures. In this article, we propose a novel modelling approach, Heterogeneous and Incrementally Generated Histogram (HIGH), to indirectly model the wavelet coefficients by use of four local features in wavelet subbands. By concatenating all the HIGHs in all wavelet subbands of a texture, we can construct a nonnegative multiresolution vector (NMV) to represent a texture image. Considering the NMV’s high dimensionality and nonnegativity, we further propose a Hessian regularized discriminative nonnegative matrix factorization to compute a low-dimensional basis of the linear subspace of NMVs. Finally, we present a texture classification approach by projecting NMVs on the low-dimensional basis. Experimental results show that our proposed texture classification method outperforms seven representative approaches.

Coming with the popularity of multimedia sharing platforms such as Facebook and Flickr, recent years have witnessed an explosive growth of geographical tags on social multimedia content. This trend enables a wide variety of emerging applications, for example, mobile location search, landmark recognition, scene reconstruction, and touristic recommendation, which range from purely research prototype to commercial systems. In this article, we give a comprehensive survey on these applications, covering recent advances in recognition and mining of geographical-aware social multimedia. We review related work in the past decade regarding to location recognition, scene summarization, tourism suggestion, 3D building modeling, mobile visual search and city navigation. At the end, we further discuss potential challenges, future topics, as well as open issues related to geo-social multimedia computing, recognition, mining, and analytics.

In this paper, we propose a deterministic column-based matrix decomposition method. Conventional column-based matrix decomposition (CX) computes the columns by randomly sampling columns of the data matrix. Instead, the newly proposed method (termed as CX_D) selects columns in a deterministic manner, which well approximates singular value decomposition. The experimental results well demonstrate the power and the advantages of the proposed method upon three real-world data sets.

Spectral analysis-based dimensionality reduction algorithms are important and have been popularly applied in data mining and computer vision applications. To date many algorithms have been developed, e.g., principal component analysis, locally linear embedding, Laplacian eigenmaps, and local tangent space alignment. All of these algorithms have been designed intuitively and pragmatically, i.e., on the basis of the experience and knowledge of experts for their own purposes. Therefore, it will be more informative to provide a systematic framework for understanding the common properties and intrinsic difference in different algorithms. In this paper, we propose such a framework, named "patch alignment,” which consists of two stages: part optimization and whole alignment. The framework reveals that 1) algorithms are intrinsically different in the patch optimization stage and 2) all algorithms share an almost identical whole alignment stage. As an application of this framework, we develop a new dimensionality reduction algorithm, termed Discriminative Locality Alignment (DLA), by imposing discriminative information in the part optimization stage. DLA can 1) attack the distribution nonlinearity of measurements; 2) preserve the discriminative ability; and 3) avoid the small-sample-size problem. Thorough empirical studies demonstrate the effectiveness of DLA compared with representative dimensionality reduction algorithms.

Subspace selection is a powerful tool in data mining. An important subspace method is the Fisher-Rao linear discriminant analysis (LDA), which has been successfully applied in many fields such as biometrics, bioinformatics, and multimedia retrieval. However, LDA has a critical drawback: the projection to a subspace tends to merge those classes that are close together in the original feature space. If the separated classes are sampled from Gaussian distributions, all with identical covariance matrices, then LDA maximizes the mean value of the Kullback-Leibler (KL) divergences between the different classes. We generalize this point of view to obtain a framework for choosing a subspace by 1) generalizing the KL divergence to the Bregman divergence and 2) generalizing the arithmetic mean to a general mean. The framework is named the general averaged divergence analysis (GADA). Under this GADA framework, a geometric mean divergence analysis (GMDA) method based on the geometric mean is studied. A large number of experiments based on synthetic data show that our method significantly outperforms LDA and several representative LDA extensions.

Recently, relevance feedback (RF) in content-based image retrieval (CBIR) has been implemented as an online binary classifier to separate the positive samples from the negative samples, where both sets of samples are labeled by the user. In many applications, it is reasonable to assume that all the positive samples are alike and thus that the region of the feature space occupied by the positive samples can be described by a single hypersurface. However, for the negative samples, previous RF methods either treat each one of the negative samples as an isolated point or assume the whole negative set can be described by a single convex hypersurface. In this paper, we argue that these treatments of the negative samples are not sound. Our belief is all positive samples are included in a set and the negative samples split into a small number of subsets, each one of which has a simple distribution. Therefore, we first cluster the negative samples into several groups; for each such negative group, we build a marginal convex machine (MCM) subclassifier between it and the single positive group which results in a series of subclassifiers. These subclassifiers are then incorporated into a biased MCM (BMCM) for RF. Experiments were carried out to prove the advantages of BMCM-based RF over previous methods for RF

This paper aims to take general tensors as inputs for supervised learning. A supervised tensor learning (STL) framework is established for convex optimization based learning techniques such as support vector machines (SVM) and minimax probability machines (MPM). Within the STL framework, many conventional learning machines can be generalized to take n/sup th/-order tensors as inputs. We also study the applications of tensors to learning machine design and feature extraction by linear discriminant analysis (LDA). Our method for tensor based feature extraction is named the tenor rank-one discriminant analysis (TR1DA). These generalized algorithms have several advantages: 1) reduce the curse of dimension problem in machine learning and data mining; 2) avoid the failure to converge; and 3) achieve better separation between the different categories of samples. As an example, we generalize MPM to its STL version, which is named the tensor MPM (TMPM). TMPM learns a series of tensor projections iteratively. It is then evaluated against the original MPM. Our experiments on a binary classification problem show that TMPM significantly outperforms the original MPM.

We propose two new methods in the nonlinear kernel feature space for pixel clustering based on the traditional KMeans and Gaussian mixture model (GMM). Unlike the previous work on the kernel machines, we give out a new perspective on the new developed kernel machines. That is, kernel principle component analysis (KPCA) combined with the KMeans and the GMM are kernel KMeans (KKMeans) and kernel GMM (KGMM), respectively. In this paper, we prove the new perspective on KKMeans and give out a clear statement on the KGMM as well. Based on this new perspectives, we can implement the KKMeans and the KGMM conveniently. At the end of the paper, we utilize these new algorithms on the problem of the colour image segmentation. Based on a series of experimental results on Corel colour images, we find that the KKMeans and KGMM can outperform the traditional KMeans and GMM consistently, respectively.

General tensors can represent colour images more naturally than conventional features; however, the general tensors' stability properties are not reported and remain to be a key problem. In this paper, we use the tensor minimax probability (TMPM) to prove that the tensor representation is stable. The proof is based on the random subspace method through a large number of experiments.