Semi-supervised learning aims to generate a model with a better performance using plenty of unlabeled data. However, most existing methods treat unlabeled data equally without considering whether it is safe or not, which may lead to the degradation of prediction performance. In this paper, towards reliable semi-supervised learning, we propose a data-driven algorithm, called Reliability Propagation (RP), to learn the reliability of each unlabeled instance. The basic idea is to take local label regularity as a prior, and then perform reliability propagation on an adaptive graph. As a result, the most reliable unlabeled instances could be selected to construct a safer classifier. Beyond, the distributed RP algorithm is introduced to scale up to large volumes of data. In contrast to existing approaches, RP exploits the structural information and shed light on the soft instance selection for unlabeled data in a classifier-independent way. Experiments on both synthetic and real-world data have demonstrated that RP allows extracting most reliable unlabeled instances and supports a gained prediction performance compared to other algorithms.

Mining feature evolvable streams has gained increasing attention in recent years. However, most existing approaches are designed for stationary data streams (i.e., data streams without concept drifts) and often work with high time complexity due to the time-consuming optimization procedure. The two deficiencies thus largely limit its applications to real-world data stream scenarios. In this paper, we consider a more difficult but practical streaming setting: a data stream with both concept drifts and evolvable features. To this end, we propose a general framework for mining concept-drifting data streams with evolvable features, called FEMC, based on an efficient Feature Evolvable streaming learning and dynamic Micro-Clusters maintenance. Specifically, we derive a closed-form solution to preserve the information in vanished features by learning a weight vector on survival features. The evolving concepts, are further learnt by dynamically maintaining a set of micro-clusters with varying feature space on-the-fly. Empirical results on real-world data sets have demonstrated the benefits of the proposed framework on both clustering and classification tasks by comparing with state-of-the-art algorithms.

Multi-label classification problem has gained growing attention in recent years due to its diverse applications to real-world problems such as image annotation and query suggestions. However, traditional multi-label classification methods tend to fail due to the inconsistency between input and output space, where similar instances in the feature space may have distinct semantic labels in the output space. To eliminate the inconsistency problem, in this paper, we propose a supervised metric learning approach for multi-label classification, called MLMLI, which attempts to learn a similarity metric for multi-label data. The basic idea is to incorporate label similarity in output space as weak supervision to assign higher similarity to the pairs of instances with more similar labels. To this end, a weighted triple loss, and a step-specified coordinate descent method are employed. Different from traditional dimensionality reduction approaches, MLMLI is independent of any prior information of data, and thus enjoys a high capacity of generalization. Moreover, the metric learned by MLMLI offers a new venue for feature learning. Experiments on real-world datasets have further demonstrated the effectiveness of MLMLI and show its superiority over many state-of-the-art algorithms.

Community detection in attributed networks has gained growing attention in recent years due to the booming of network data with both topological structure and attributes of nodes. To date, numerous algorithms have been proposed to leverage both kinds of information to yield high-quality communities based on homophily assumption (i.e., nodes are likely to link with those who share similar attributes). However, these approaches tend to focus on consistent information only and fail to consider the heterogeneity between topology and attributes. In light of the problem, we propose a new algorithm called CDLM (Community Detection via Local Metric learning) for attributed networks. The key point is to combine topological structure and node attributes in local metric space and perform community detection and local metric learning iteratively. With such a strategy, the learned local distance measures will benefit the performance of community detection, and in turn, the identification of intrinsic community structure helps to eliminate the negative effects of noisy edges in learning local metrics. Notably, homogeneity and heterogeneity between topological structure and node attributes are simultaneously considered to boost the performance of community detection. Experimental results on both synthetic and real-world networks have demonstrated the effectiveness of the proposed CDLM algorithm.

The scarcity of labeled data in real streaming environments has boosted the study of online semi-supervised learning (SSL). However, existing online SSL models often rely on some specific assumptions (e.g., manifold assumption) and need to maintain some extra constraints (e.g., the Laplacian matrix) on the fly, which is usually time and resource consuming. In this paper, we propose an efficient and effective online semi-supervised learning approach via Budgeted Least Square (BLS). Specifically, we first derive both closed-form transductive and inductive solutions for kernel least squares classification in the semi-supervised setting. Then, together with online kernel learning, BLS allows a concise online update. Besides, the theoretical regret bound of BLS is analysed, and empirical experiments on both static and streaming data further demonstrate its superiority over state-of-the-art algorithms.

In this paper, we propose a prototype-based classification model for evolving data streams, called SyncStream, which allows dynamically modeling time-changing concepts, making predictions in a local fashion. Instead of learning a single model on a fixed or adaptive sliding window of historical data or ensemble learning a set of weighted base classifiers, SyncStream captures evolving concepts by dynamically maintaining a set of prototypes in a proposed P-Tree, which are obtained based on the error-driven representativeness learning and synchronization-inspired constrained clustering. To identify abrupt concept drifts in data streams, PCA and statistical analysis based heuristic approaches have been introduced. To further learn the associations among distributed data streams, the extended P-Tree structure and KNN-style strategy are introduced. We demonstrate that our new data stream classification approach has several attractive benefits: (a) SyncStream is capable of dynamically modeling the evolving concepts from even a small set of prototypes. (b) Owing to synchronization-based constrained clustering and P-Tree, SyncStream supports efficient and effective data representation and maintenance. (c) SyncStream is also tolerant of inappropriate or noisy examples via error-driven representativeness learning. (d) SyncStream allows learning relationship among distributed data streams at the instance level. The experimental results indicate its efficiency and effectiveness.

In this paper, we propose a new synchronization-inspired co-clustering algorithm by dynamic simulation, called CoSync, which aims to discover biologically relevant subgroups embedding in a given gene expression data matrix. The basic idea is to view a gene expression data matrix as a dynamical system, and the weighted two-sided interactions are imposed on each element of the matrix from both aspects of genes and conditions, resulting in the values of all element in a co-cluster synchronizing together. Experiments show that our algorithm allows uncovering high-quality co-clusterings embedded in gene expression data sets and has its superiority over many state-of-the-art algorithms.

In this paper, we introduce a novel method to discover common and distinct structural connectivity patterns between SZP and MDD via a Cluster-Driven Nonnegative Matrix Factorization (called CD-NMF). Specifically, CD-NMF is applied to decompose the joint structural connectivity map into common and distinct parts, and each part is further factorized into two sub-matrices (i.e. common/distinct basis matrix and common/distinct encoding matrix) correspondingly. By imposing the clustering constraints on common and distinct encoding matrices, the discriminative patterns as well as the common patterns between the two disorders are extracted simultaneously. Experimental results demonstrate that CD-NMF allows finding the common and distinct structural patterns effectively. More importantly, the derived distinct patterns, show powerful ability to discriminate the patients of schizophrenia and major depressive disorder.

In this paper, we propose a Reliable Semi-Supervised Learning framework, called ReSSL, for both static and streaming data. Instead of relaxing different assumptions, we do model the reliability of cluster assumption, quantify the distinct importance of clusters (or evolving micro-clusters on data streams), and integrate the cluster-level information and labeled data for prediction with a lazy learning framework. Extensive experiments demonstrate that our method has good performance compared to state-of-the-art algorithms on data sets in both static and real streaming environments.

Clustering very large datasets while preserving cluster quality remains a challenging data-mining task to date. In this paper, we propose an effective scalable clustering algorithm for large datasets that builds upon the concept of synchronization. Inherited from the powerful concept of synchronization, the proposed algorithm, CIPA (Clustering by Iterative Partitioning and Point Attractor Representations), is capable of handling very large datasets by iteratively partitioning them into thousands of subsets and clustering each subset separately. Using dynamic clustering by synchronization, each subset is then represented by a set of point attractors and outliers. Finally, CIPA identifies the cluster structure of the original dataset by clustering the newly generated dataset consisting of points attractors and outliers from all subsets. We demonstrate that our new scalable clustering approach has several attractive benefits: (a) CIPA faithfully captures the cluster structure of the original data by performing clustering on each separate data iteratively instead of using any sampling or statistical summarization technique. (b) It allows clustering very large datasets efficiently with high cluster quality. (c) CIPA is parallelizable and also suitable for distributed data. Extensive experiments demonstrate the effectiveness and efficiency of our approach.

In this paper, we introduce a new community detection algorithm, called Attractor, which automatically spots communities in a network by examining the changes of "distances" among nodes (i.e. distance dynamics). The fundamental idea is to envision the target network as an adaptive dynamical system, where each node interacts with its neighbors. The interaction will change the distances among nodes, while the distances will affect the interactions. Such interplay eventually leads to a steady distribution of distances, where the nodes sharing the same community move together and the nodes in different communities keep far away from each other. Building upon the distance dynamics, Attractor has several remarkable advantages: (a) It provides an intuitive way to analyze the community structure of a network, and more importantly, faithfully captures the natural communities (with high quality). (b) Attractor allows detecting communities on large-scale networks due to its low time complexity (O(|E|)). (c) Attractor is capable of discovering communities of arbitrary size, and thus small-size communities or anomalies, usually existing in real-world networks, can be well pinpointed. Extensive experiments show that our algorithm allows the effective and efficient community detection and has good performance compared to state-of-the-art algorithms.

Synchronization is a powerful and inherently hierarchical concept regulating a large variety of complex processes ranging from the metabolism in a cell to opinion formation in a group of individuals. Synchronization phenomena in nature have been widely investigated and models concisely describing the dynamical synchronization process have been proposed, e.g., the well-known Extensive Kuramoto Model. We explore the potential of the Extensive Kuramoto Model for data clustering. We regard each data object as a phase oscillator and simulate the dynamical behavior of the objects over time. By interaction with similar objects, the phase of an object gradually aligns with its neighborhood, resulting in a nonlinear object movement naturally driven by the local cluster structure. We demonstrate that our framework has several attractive benefits: 1) It is suitable to detect clusters of arbitrary number, shape, and data distribution, even in difficult settings with noise points and outliers. 2) Combined with the Minimum Description Length (MDL) principle, it allows partitioning and hierarchical clustering without requiring any input parameters which are difficult to estimate. 3) Synchronization faithfully captures the natural hierarchical cluster structure of the data and MDL suggests meaningful levels of abstraction. Extensive experiments demonstrate the effectiveness and efficiency of our approach.

How to address the challenges of the "curse of dimensionality" in clustering? Clustering is a powerful data mining technique for structuring and organizing vast amounts of data. However, the high-dimensional data space is usually very sparse and meaningful clusters can only be found in lower dimensional subspaces. In many applications the subspaces hosting the clusters provide valuable information for interpreting the major patterns in the data. Detection of subspace clusters is challenging since usually many of the attributes are noisy, some attributes may exhibit correlations among each other and only few of the attributes truly contribute to the cluster structure. In this paper, we propose ORSC (Arbitrarily ORiented Synchronized Clusters), a novel effective and efficient method to subspace clustering inspired by synchronization. Synchronization is a basic phenomenon prevalent in nature, capable of controlling even highly complex processes such as opinion formation in a group. Control of complex processes is achieved by simple operations based on interactions between objects. Relying on the interaction model for synchronization, our approach ORSC (1) naturally detects correlation clusters in arbitrarily oriented subspaces, including (2) arbitrarily shaped non-linear correlation clusters. Our approach is (3) robust against noise points and outliers. In contrast to previous methods, ORSC is (4) easy to parameterize, since there is no need to specify the subspace dimensionality and all interesting subspace clusters can be detected. Finally, (5) ORSC outperforms most comparison methods in terms of runtime efficiency and is highly scalable to large and high-dimensional data sets.

Synchronization is a powerful basic concept in nature regulating a large variety of complex processes ranging from the metabolism in the cell to social behavior in groups of individuals. Therefore, synchronization phenomena have been extensively studied and models robustly capturing the dynamical synchronization process have been proposed, e.g. the Extensive Kuramoto Model. Inspired by the powerful concept of synchronization, we propose Sync, a novel approach to clustering. The basic idea is to view each data object as a phase oscillator and simulate the interaction behavior of the objects over time. As time evolves, similar objects naturally synchronize together and form distinct clusters. Inherited from synchronization, Sync has several desirable properties: The clusters revealed by dynamic synchronization truly reflect the intrinsic structure of the data set, Sync does not rely on any distribution assumption and allows detecting clusters of arbitrary number, shape and size. Moreover, the concept of synchronization allows natural outlier handling, since outliers do not synchronize with cluster objects. For fully automatic clustering, we propose to combine Sync with the Minimum Description Length principle. Extensive experiments on synthetic and real world data demonstrate the effectiveness and efficiency of our approach.