International audience

International audience

International audience

We present WordGraph, a Python package for exploring the topics of documents corpora. WordGraph provides causal graphical models from text data vocabulary and proposes interactive visualizations of terms networks. Our ease-to-use package is provided with a pre-built pipeline to access the main modules through jupyter widgets. It results in the encapsulation of a whole vocabulary exploration process within a single jupyter notebook cell, with straightforward parameters settings and interactive plots. WordGraph pipeline is fully customizable by adding/removing widgets or changing default parameters. To assist users with no background in Python nor jupyter notebook, but willing to explore large corpora topics, we also propose an automatic dashboard generation from the customizable jupyter notebook pipeline in a web application style. WordGraph is available through a GitHub repository at https://github.com/MLDS-software/WordGraph.

Data clustering and representation learning play an indispensable role in data science. They are very useful to explore massive data in many fields, including information retrieval, natural language processing, bioinformatics, recommender systems, and computer vision. Despite their success, most existing clustering methods are severely challenged by the data generated by modern applications, which are typically high dimensional, noisy, heterogeneous, and sparse or even collected from multiple sources or represented by multiple views where each describes a perspective of the data. This has driven many researchers to investigate new effective clustering models to overcome these difficulties. One promising category of such models relies on representation learning. Indeed, learning a good data representation is crucial for clustering algorithms, and combining the two tasks is a common way of exploring this type of data. The idea is to embed the original data into a low dimensional latent space and then perform clustering on this new space. However, both tasks can be carried out sequentially or jointly. Many clustering algorithms, including deep learning versions, are based on these two modes of combining the two tasks.

Subspace clustering seeks to cluster high-dimensional data lying in a union of low-dimensional subspaces. It has achieved state-of-the-art results in image clustering, but text clustering of document-term matrices, has proved more impervious to advances with this approach, even though text data satisfies the assumptions of subspace clustering. We hypothesize that this is because such matrices are generally sparser and higher-dimensional than images. This, combined with the complexity of subspace clustering, which is generally cubic in the number of inputs, makes its use impractical in the context of text. Here we address these issues with a view to leveraging subspace clustering for networked (or not) text data. We first extend the concept of subspace clustering to co-clustering, which is suitable to deal with document-term matrices because of the interplay engendered between the document and word representations. We then address the sparsity problem through bilateral graph convolution, which promotes the grouping effect that has been credited for the effectiveness of some subspace clustering models. The proposed formulation results in an algorithm that is computationally/spatially efficient. Experiments using real-world datasets demonstrate the superior performance, in terms of document clustering, word clustering, and computational efficiency, of our proposed approach over the baselines and comparable methods.

Representation learning is a central problem of attributed networks (ANs) data analysis in a variety of fields. Given an attributed graph, the objectives are to obtain a representation of nodes and a partition of the set of nodes. Usually, these two objectives are pursued separately via two tasks that are performed sequentially, and any benefit that may be obtained by performing them simultaneously is lost. In this brief, we propose a power-attributed graph embedding and clustering (PAGEC for short) in which the two tasks, embedding and clustering, are considered together. To jointly encode data affinity between node links and attributes, we use a new powered proximity matrix. We formulate a new matrix decomposition model to obtain node representation and node clustering simultaneously. Theoretical analysis shows the close connections between the new proximity matrix and the random walk theory on a graph. Experimental results demonstrate that the PAGEC algorithm performs better, in terms of clustering and embedding, than state-of-the-art algorithms including deep learning methods designed for similar tasks in relation to attributed network datasets with different characteristics.

Over recent years, graph convolutional networks emerged as powerful node clustering methods and have set state of the art results for this task. In this paper, we argue that some of these methods are unnecessarily complex and propose a node clustering model that is more scalable while being more effective. The proposed model uses Laplacian smoothing to learn an initial representation of the graph before applying an efficient self-expressive subspace clustering procedure. This is performed via learning a factored coefficient matrix. These factors are then embedded into a new feature space in such a way as to generate a valid affinity matrix (symmetric and non-negative) on which an implicit spectral clustering algorithm is performed. Experiments on several real-world attributed datasets demonstrate the cost-effective nature of our method with respect to the state of the art.

Unsupervised anomaly detection seeks to identify deviant data samples in a dataset without using labels and constitutes a challenging task, particularly when the majority class is heterogeneous.This paper addresses this topic for textual data and aims to determine whether a text sample is an outlier within a potentially multi-topic corpus.To this end, it is crucial to grasp the semantic aspects of words, particularly when dealing with short texts, since it is difficult to syntactically discriminate data samples based only on a few words.Thereby we make use of word embeddings to represent each sample by a dense vector, efficiently capturing the underlying semantics.Then, we rely on the Mixture Model approach to detect which samples deviate the most from the underlying distributions of the corpus.Experiments carried out on real datasets show the effectiveness of the proposed approach in comparison to state-ofthe-art techniques both in terms of performance and time efficiency, especially when more than one topic is present in the corpus.

Over the last few years, various multi-view graph clustering methods have shown promising performances. However, we argue that these methods can have limitations. In particular, they are often unnecessarily complex, leading to scalability problems that make them prohibitive for most real-world graph applications. Furthermore, many of them can handle only specific types of multi-view graphs. Another limitation is that the process of learning graph representations is separated from the clustering process, and in some cases these methods do not even learn a graph representation, which severely restricts their flexibility and usefulness. In this paper we propose a simple yet effective linear model that addresses the dual tasks of multi-view attributed graph representation learning and clustering in a unified framework. The model starts by performing a first-order neighborhood smoothing step for the different individual views, then gives each one a weight corresponding to its importance. Finally, an iterative process of simultaneous clustering and representation learning is performed w.r.t. the importance of each view, yielding a consensus embedding and partition of the graph. Our model is generic and can deal with any type of multi-view graph. Finally, we show through extensive experimentation that this simple model consistently achieves competitive performances w.r.t. state-of-the-art multi-view attributed graph clustering models, while at the same time having training times that are shorter, in some cases by orders of magnitude.

Subspace clustering aims to cluster high dimensional data lying in a union of low-dimensional subspaces. It has shown good results on the task of image clustering but text clustering, using document-term matrices, proved more impervious to advances based on this approach. We hypothesize that this is because, compared to image data, text data is generally higher dimensional and sparser. This renders subspace clustering impractical in such a context. Here, we leverage subspace clustering for text by addressing these issues. We first extend the concept of subspace clustering to co-clustering, which has been extensively used on document-term matrices due to the resulting interplay between the document and term representations. We then address the sparsity problem through a two-way graph convolution, which promotes the grouping effect that has been credited for the effectiveness of some subspace clustering models. The proposed formulation results in an algorithm that is efficient both in terms of computational and spatial complexity. We show the competitiveness of our model w.r.t the state-of-the-art on document-term attributed graph datasets in terms of performance and efficiency.

Bipartite graphs can be used to model a wide variety of dyadic information such as user-rating, document-term, and gene-disorder pairs. Biclustering is an extension of clustering to the underlying bipartite graph induced from this kind of data. In this paper, we leverage optimal transport (OT) which has gained momentum in the machine learning community to propose a novel and scalable biclustering model that generalizes several classical biclustering approaches. We perform extensive experimentation to show the validity of our approach compared to other OT biclustering algorithms along both dimensions of the dyadic datasets.

Graphs are used in several applications to represent similarities between instances. For text data, we can represent texts by different features such as bag-of-words, static embeddings (Word2vec, GloVe, etc.), and contextual embeddings (BERT, RoBERTa, etc.), leading to multiple similarities (or graphs) based on each representation. The proposal posits that incorporating the local invariance within every graph and the consistency across different graphs leads to a consensus clustering that improves the document clustering. This problem is complex and challenged with the sparsity and the noisy data included in each graph. To this end, we rely on the modularity metric, which effectively evaluates graph clustering in such circumstances. Therefore, we present a novel approach for text clustering based on both a sparse tensor representation and graph modularity. This leads to cluster texts (nodes) while capturing information arising from the different graphs. We iteratively maximize a Tensor-based Graph Modularity criterion. Extensive experiments on benchmark text clustering datasets are performed, showing that the proposed algorithm referred to as Tensor Graph Modularity -TGM- outperforms other baseline methods in terms of clustering task. The source code is available at https://github.com/TGMclustering/TGMclustering.

Attributed graphs are used to model a wide variety of real-world networks. Recent graph convolutional network-based representation learning methods have set state-of-the-art results on the clustering of attributed graphs. However, these approaches deal with clustering as a downstream task while better performances can be attained by incorporating the clustering objective into the representation learning process. In this paper, we propose, in a unified framework, an objective function taking into account both tasks simultaneously. Based on a variant of the simple graph convolutional network, our model does clustering by minimizing the difference between the convolved node representations and their reconstructed cluster representatives. We showcase the efficiency of the derived algorithm against state-of-the-art methods both in terms of clustering performance and computational cost on thede facto benchmark graph clustering datasets. We further demonstrate the usefulness of the proposed approach for graph visualization through generating embeddings that exhibit a clustering structure.

Pre-trained Transformer-based word embeddings are now widely used in text mining where they are known to significantly improve supervised tasks such as text classification, named entity recognition and question answering. Since the Transformer models create several different embeddings for the same input, one at each layer of their architecture, various studies have already tried to identify those of these embeddings that most contribute to the success of the above-mentioned tasks. In contrast the same performance analysis has not yet been carried out in the unsupervised setting. In this paper we evaluate the effectiveness of Transformer models on the important task of text clustering. In particular, we present a clustering ensemble approach that harnesses all the network's layers. Numerical experiments carried out on real datasets with different Transformer models show the effectiveness of the proposed method compared to several baselines.

Co-clustering of document-term matrices has proved to be more effective than one-sided clustering. By their nature, text data are also generally unbalanced and directional. Recently, the von Mises-Fisher (vMF) mixture model was proposed to handle unbalanced data while harnessing the directional nature of text. In this paper we propose a novel co-clustering approach based on a matrix formulation of vMF model-based co-clustering. This formulation leads to a flexible method for text co-clustering that can easily incorporate both word-word semantic relationships and document-document similarities. By contrast with existing methods, which generally use an additive incorporation of similarities, we propose a dual multiplicative regularization that better encapsulates the underlying text data structure. Extensive evaluations on various real-world text datasets demonstrate the superior performance of our proposed approach over baseline and competitive methods, both in terms of clustering results and co-cluster topic coherence.

Biomedical scientific literature is growing at a very rapid pace, which makes increasingly difficult for human experts to spot the most relevant results hidden in the papers. Automatized information extraction tools based on text mining techniques are therefore needed to assist them in this task. In the last few years, deep neural networks-based techniques have significantly contributed to advance the state-of-the-art in this research area. Although the contribution to this progress made by supervised methods is relatively well-known, this is less so for other kinds of learning, namely unsupervised and self-supervised learning. Unsupervised learning is a kind of learning that does not require the cost of creating labels, which is very useful in the exploratory stages of a biomedical study where agile techniques are needed to rapidly explore many paths. In particular, clustering techniques applied to biomedical text mining allow to gather large sets of documents into more manageable groups. Deep learning techniques have allowed to produce new clustering-friendly representations of the data. On the other hand, self-supervised learning is a kind of supervised learning where the labels do not have to be manually created by humans, but are automatically derived from relations found in the input texts. In combination with innovative network architectures (e.g. transformer-based architectures), self-supervised techniques have allowed to design increasingly effective vector-based word representations (word embeddings). We show in this survey how word representations obtained in this way have proven to successfully interact with common supervised modules (e.g. classification networks) to whose performance they greatly contribute.

Abstract Dealing with relational learning generally relies on tools modeling relational data. An undirected graph can represent these data with vertices depicting entities and edges describing the relationships between the entities. These relationships can be well represented by multiple undirected graphs over the same set of vertices with edges arising from different graphs catching heterogeneous relations. The vertices of those networks are often structured in unknown clusters with varying properties of connectivity. These multiple graphs can be structured as a three-way tensor, where each slice of tensor depicts a graph which is represented by a count data matrix. To extract relevant clusters, we propose an appropriate model-based co-clustering capable of dealing with multiple graphs. The proposed model can be seen as a suitable tensor extension of mixture models of graphs, while the obtained co-clustering can be treated as a consensus clustering of nodes from multiple graphs. Applications on real datasets and comparisons with multi-view clustering and tensor decomposition methods show the interest of our contribution.

In this paper, we propose a unified framework for Ensemble Block Co-clustering (EBCO), which aims to fuse multiple basic co-clusterings into a consensus structured affinity matrix. Each co-clustering to be fused is obtained by applying a co-clustering method on the same document-term dataset. This fusion process reinforces the individual quality of the multiple basic data co-clusterings within a single consensus matrix. Besides, the proposed framework enables a completely unsupervised co-clustering where the number of co-clusters is automatically inferred based on the non trivial generalized modularity. We first define an explicit objective function which allows the joint learning of the basic co-clusterings aggregation and the consensus block co-clustering. Then, we show that EBCO generalizes the one side ensemble clustering to an ensemble block co-clustering context. We also establish theoretical equivalence to spectral co-clustering and weighted double spherical k-means clustering for textual data. Experimental results on various real-world document-term datasets demonstrate that EBCO is an efficient competitor to some state-of-the-art ensemble and co-clustering methods.

Nonnegative Matrix Factorization (NMF) which was originally designed for dimensionality reduction has received throughout the years a tremendous amount of attention for clustering purposes in several fields such as image processing or text mining. However, despite its mathematical elegance and simplicity, NMF has exposed a main issue which is its strong sensitivity to starting points, resulting in NMF struggling to converge toward an optimal solution. On another hand, we came to explore and discovered that even after providing a meaningful initialization, selecting the solution with the best local minimum was not always leading to the one having the best clustering quality, but somehow a better clustering could be obtained with a solution slightly off in terms of criterion. Therefore in this paper, we undertake to study the clustering characteristics and quality of a set of NMF best solutions and provide a method delivering a better partition using a consensus made of the best NMF solutions.

To deal with document clustering, we usually rely on document-term matrices. However, from additional available information like keywords, co-authors, citations we might rather exploit a reorganization of the data in the form of a tensor. In this paper, we extend the use of the Sparse Poisson Latent Block Model to deal with sparse tensor data using jointly all information arising from documents. The proposed model is parsimonious and tailored for this kind of data. To estimate the parameters, we derive a suitable tensor co-clustering algorithm. Empirical results on several real-world text datasets highlight the advantages of our proposal which improves the clustering results of documents.

Text data co-clustering is the process of partitioning the documents and words simultaneously. This approach has proven to be more useful than traditional one-sided clustering when dealing with sparsity. Among the wide range of co-clustering approaches, Non-Negative Matrix Tri-Factorization (NMTF) is recognized for its high performance, flexibility and theoretical foundations. One important aspect when dealing with text data, is to capture the semantic relationships between words since documents that are about the same topic may not necessarily use exactly the same vocabulary. However, this aspect has been overlooked by previous co-clustering models, including NMTF. To address this issue, we rely on the distributional hypothesis stating that words which co-occur frequently within the same context, e.g., a document or sentence, are likely to have similar meanings. We then propose a new NMTF model that maps frequently co-occurring words roughly to the same direction in the latent space to reflect the relationships between them. To infer the factor matrices, we derive a scalable alternating optimization algorithm, whose convergence is guaranteed. Extensive experiments, on several real-world datasets, provide strong evidence for the effectiveness of the proposed approach, in terms of co-clustering.

Text clustering, which allows to divide a dataset into groups of similar documents, plays an important role at various stages of the information retrieval process. Co-clustering is an extension of one-side clustering, and consists in simultaneously clustering the rows and columns of a data matrix. However, while co-clustering algorithms consider both dimensions of a document-term matrix, they are usually evaluated on the quality of the obtained document clusters alone. In this paper, we therefore propose an evaluation scheme that accounts for the two-dimensional nature of co-clustering algorithms, thus allowing for a more precise evaluation of their performance. Another important benefit of the proposed approach is that it does not require the use of any prior labels. This is achieved by leveraging large, public domain embedding matrices (GloVe, word2vec, FastText) to compute comparable representations of both document and term clusters. Experiments carried out on several textual datasets show that the proposed measures are both reliable and stable, and can even provide hints to improve co-clustering performance.

We present a novel generative mixture model for co-clustering text data. This model, the Sparse Poisson Latent Block Model (SPLBM), is based on the Poisson distribution, which arises naturally for contingency tables, such as document-term matrices. The advantages of SPLBM are two-fold. First, it is a rigorous statistical model which is also very parsimonious. Second, it has been designed from the ground up to deal with data sparsity problems. Extensive experiments on various real-world datasets of different size and structure provide strong evidence for the effectiveness of the proposed approach.

Spectral clustering is often carried out by combining spectral data embedding and -means clustering. However, the aims, dimensionality reduction and clustering, are usually not performed jointly. In this brief, we propose a novel approach to finding an optimal spectral embedding for identifying a partition of the set of objects; it iteratively alternates spectral embedding and clustering. In doing so, we show that our model can learn a low-dimensional representation more suited to clustering. Compared with classical spectral clustering methods, the proposed algorithm is not costly and outperforms not only these methods but also other nonnegative matrix factorization variants.

Semi-Non Negative Matrix Factorization (Semi-NMF) is one of the most popular extensions of NMF, it extends the applicable range of NMF models, to data having mixed signs, as well as strengthens their relation to clustering. However, Semi-NMF has been found to perform somewhat less than NMF, in terms of clustering, when applied to positive data such as text, which we focus on. Inspired by the recent success of neural word embedding models, e.g., word2vec, in learning high quality real valued vector representations of words, we propose to integrate a word embedding model into Semi-NMF. This allows Semi-NMF to capture more semantic relationships among words and, thereby, to infer document factors that are even better for clustering. The combination of Semi-NMF and word embedding noticeably improves the performance of NMF models, in terms of both clustering and embedding, as illustrated in our experiments.

Co-clustering has proven effective to deal with high dimensional sparse data, such as document-term matrices encountered in text mining. Apart from being high dimensional and sparse, the data sets from the aforementioned domain are also directional in nature. Most existing co-clustering approaches are, however, based on popular modelling assumptions, such as Gaussian or Multinomial, which are inadequate for directional data. Moreover, it is well known that, due to high dimensionality and sparsity, co-clustering approaches, like one-sided clustering methods, tend to generate highly skewed solutions with very unbalanced or even empty clusters, especially when the number of required clusters is large. In this paper, we rely on the recently proposed block von Mises-Fisher mixture model (dbmovMFs), which constitutes a general framework for co-clustering directional data distributed on the surface of a unit hypersphere, i.e, L2 normalized data. In order to overcome the above difficulties, we propose to modify dbmovMFs in a principled way by introducing a conscience mechanism which discourages bad local solutions having empty or very small/large clusters. This gives rise to a new scalable co-clustering algorithm which is guaranteed to increase monotonically a spherical k-means like criterion by intertwining row and column clusterings at each step. Moreover, empirical results, on several real-world datasets, provide strong support for the effectiveness of the proposed approach.

Document clustering is central in modern information retrieval applications. Among existing models, non-negative-matrix factorization (NMF) approaches have proven effective for this task. However, NMF approaches, like other models in this context, exhibit a major drawback, namely they use the bag-of-word representation and, thus, do not account for the sequential order in which words occur in documents. This is an important issue since it may result in a significant loss of semantics. In this paper, we aim to address the above issue and propose a new model which successfully integrates a word embedding model, word2vec, into an NMF framework so as to leverage the semantic relationships between words. Empirical results, on several real-world datasets, demonstrate the benefits of our model in terms of text document clustering as well as document/word embedding.

Over the last decades, several studies have demonstrated the importance of co-clustering to simultaneously produce groups of objects and features. Even to obtain object clusters only, using co-clustering is often more effective than one-way clustering, especially when considering sparse high dimensional data. In this paper, we present a novel generative mixture model for co-clustering such data. This model, the Sparse Poisson Latent Block Model (SPLBM), is based on the Poisson distribution, which arises naturally for contingency tables, such as document-term matrices. The advantages of SPLBM are two-fold. First, it is a rigorous statistical model which is also very parsimonious. Second, it has been designed from the ground up to deal with data sparsity problems. As a consequence, in addition to seeking homogeneous blocks, as other available algorithms, it also filters out homogeneous but noisy ones due to the sparsity of the data. Experiments on various datasets of different size and structure show that an algorithm based on SPLBM clearly outperforms state-of-the-art algorithms. Most notably, the SPLBM-based algorithm presented here succeeds in retrieving the natural cluster structure of difficult, unbalanced datasets which other known algorithms are unable to handle effectively.

In this work, we propose a novel way to consider the clustering and the reduction of the dimension simultaneously. Indeed, our approach takes advantage of the mutual reinforcement between data reduction and clustering tasks. The use of a low-dimensional representation can be of help in providing simpler and more interpretable solutions. We show that by doing so, our model is able to better approximate the relaxed continuous dimension reduction solution by the true discrete clustering solution. Experiment results show that our method gives better results in terms of clustering than the state-of-the-art algorithms devoted to similar tasks for data sets with different proprieties.

We propose a novel model based on the von Mises-Fisher (vMF) distribution for co-clustering high dimensional sparse matrices. While existing vMF-based models are only suitable for clustering along one dimension, our model acts simultaneously on both dimensions of a data matrix. Thereby it has the advantage of exploiting the inherent duality between rows and columns. Setting our model under the maximum likelihood (ML) approach and the classification ML (CML) approach, we derive two novel, hard and soft, co-clustering algorithms. Empirical results on numerous synthetic and real-world text datasets, demonstrate the effectiveness of our approach, for modelling high dimensional sparse data and co-clustering. Furthermore, thanks to our formulation, that performs an implicitly adaptive dimensionality reduction at each stage, our model alleviates the problem of high concentration parameters kappa’s, a well known difficulty in the classical vMF-based models.

Several studies have demonstrated the importance of co-clustering which aims to cluster simultaneously the sets of objects and features. The co-clustering is often more effective than one-side clustering, especially when considering sparse high dimensional data. In this paper, we propose a novel way to consider the co-clustering and the reduction of the dimension simultaneously. Our approach takes advantage of the mutual reinforcement between Principal Component Analysis (PCA) which provides a low-dimensional representation of data and Semi-Nonnegative Matrix Factorization (SemiNMF) that learns this low-dimensional representation and lends itself to a co-clustering interpretation. In other words, the proposed framework aims to find an optimal subspace of multi-dimensional variables for effectively identifying a partition of the set of objects. We show that by doing so, our model is able to learn low-dimensional representations that are better suited for co-clustering, outperforming not only spectral methods, but also co-clustering graph-regularized-based methods.

Spectral clustering methods use the Laplacian eigenvalues and eigenvectors to obtain a low-dimensional embedding that can be trivially clustered. For that purpose, spectral clustering is often based on a tandem approach where the two steps: affinity matrix eigendecomposition and k-means clustering, are performed separately. The potential flaw of such common practice is that the obtained relaxed continuous spectral solution can severely deviate from the true discrete clustering solution. Given the high computational cost of such spectral clustering methods, this paper provides the so-called PSDEC framework. PSDEC performs simultaneously the eigendecomposition of the affinity matrix and clustering tasks, and uses the Power method to speed up the unified process convergence. In PSDEC, the selected top eigenvectors of the Laplacian matrix can be of help in detecting a cluster structure of objects and providing simpler and more interpretable solutions. We show that by doing so, our method can learn low-dimensional representations that are better suited to clustering, outperforming not only spectral clustering algorithms but also some NMF variants.

Co-clustering is more useful than one-sided clustering when dealing with high dimensional sparse data. We propose to address the aim of document clustering with a generative model-based co-clustering approach. To this end, we rely on a particular mixture of von Mises-Fisher distributions and propose a new parsimonious model allowing to reveal a block diagonal structure as well as a good partitioning of documents and terms. Then, by setting the estimate of the model parameters under the maximum likelihood (ML) approach, we derive three novel co-clustering algorithms: a soft one and two stochastic variants. Empirical results on numerous simulated and real-world datasets, demonstrate the advantages of our approach to model and co-cluster high dimensional sparse data.

We present Coclus, a novel diagonal co-clustering algorithm which is able to effectively co-cluster binary or contingency matrices by directly maximizing an adapted version of the modularity measure traditionally used for networks. While some effective co-clustering algorithms already exist that use network-related measures (normalized cut, modularity), they do so by using spectral relaxations of the discrete optimization problems. In contrast, Coclus allows to get even better co-clusters by directly maximizing modularity using an iterative alternating optimization procedure. Extensive comparative experiments performed on various document-term datasets demonstrate that our algorithm is very effective, stable and outperforms other co-clustering algorithms.

We propose two fuzzy co-clustering algorithms based on the double Kmeans algorithm. Fuzzy approaches are known to require more computation time than hard ones but the fuzziness principle allows a description of uncertainties that often appears in real world applications. The first algorithm proposed, fuzzy double Kmeans (FDK) is a fuzzy version of double Kmeans (DK). The second algorithm, weighted fuzzy double Kmeans (W-FDK), is an extension of FDK with automated variable weighting allowing co-clustering and feature selection simultaneously. We illustrate our contribution using Monte Carlo simulations on datasets with different parameters and real datasets commonly used in the co-clustering context.

Cluster analysis is often carried out in combination with dimension reduction. The Semi-Non-negative Matrix Factorization (Semi-NMF) that learns a low-dimensional representation of a data set lends itself to a clustering interpretation. In this work we propose a novel approach to finding an optimal subspace of multi-dimensional variables for identifying a partition of the set of objects. The use of a low-dimensional representation can be of help in providing simpler and more interpretable solutions. We show that by doing so, our model is able to learn low-dimensional representations that are better suited for clustering, outperforming not only Semi-NMF, but also other NMF variants.

We propose a new theoretical framework for data visualization. This framework is based on iterative procedure looking up an appropriate approximation of the data matrix A by using two stochastic similarity matrices from the set of rows and the set of columns. This process converges to a steady state where the approximated data  is composed of g similar rows and l similar columns. Reordering A according to the first left and right singular vectors involves an optimal data reorganization revealing homogeneous block clusters. Furthermore, we show that our approach is related to a Markov chain model, to the double k-means with g ×l block clusters and to a spectral coclustering. Numerical experiments on simulated and real data sets show the interest of our approach.

The ensemble methods are very popular and can improve significantly the performance of classification and clustering algorithms. Their principle is to generate a set of different models, then aggregate them into only one. Recent works have shown that this approach can also be useful in biclustering problems.The crucial step of this approach is the consensus functions that compute the aggregation of the biclusters. We identify the main consensus functions commonly used in the clustering ensemble and show how to extend them in the biclustering context. We evaluate and analyze the performances of these consensus functions on several experiments based on both artificial and real data.

Biclustering is a major tool of data mining in many domains and many algorithms have emerged in recent years. All these algorithms aim to obtain coherent biclusters and it is crucial to have a reliable procedure for their validation. We point out the problem of size bias in biclustering evaluation and show how it can lead to wrong conclusions in a comparative study. We present the theoretical corrections for all of the most popular measures in order to remove this bias. We introduce the corrected precision-recall space that combines the advantages of corrected measures, the ease of interpretation and visualization of uncorrected measures. Numerical experiments demonstrate the interest of our approach.

We explore in this paper a novel topological organization algorithm for data clustering and visualization named TPNMF. It leads to a clustering of the data, as well as the projection of the clusters on a two-dimensional grid while preserving the topological order of the initial data. The proposed algorithm is based on a NMF (Nonnegative Matrix Factorization) formalism using a neighborhood function which take into account the topological order of the data. TPNMF was validated on variant real datasets and the experimental results show a good quality of the topological ordering and homogenous clustering.

This paper proposes a spectral algorithm for cross-topographic clustering. It leads to a simultaneous clustering on the rows and columns of data matrix, as well as the projection of the clusters on a two-dimensional grid while preserving the topological order of the initial data. The proposed algorithm is based on a spectral decomposition of this data matrix and the definition of a new matrix taking into account the co-clustering problem. The proposed approach has been validated on multiple datasets and the experimental results have shown very promising performance.

To tackle the co-clustering problem for binary and categorical data, we propose a generalized modularity measure and a spectral approximation of the modularity matrix. A spectral algorithm maximizing the modularity measure is then presented. Experimental results are performed on a variety of simulated and real-world data sets confirming the interest of the use of the modularity in co-clustering and assessing the number of clusters contexts.

One of the major tools of transcriptomics is the biclustering that simultaneously constructs a partition of both examples and genes. Several methods have been proposed for microarray data analysis that enables to identify groups of genes with similar expression profiles only under a subset of examples. We propose to improve the quality of these biclustering methods by using an ensemble approach. Our bagged biclustering method generates a collection of biclusters using the bootstrap samples of the original data and aggregate them into new biclusters. Our method improve the performance of biclustering on artificial and real datasets.

The co-clustering consists in reorganizing a data matrix into homogeneous blocks by considering simultaneously the sets of rows and columns. Setting this aim in model-based clustering, adapted block latent models were proposed for binary data and co-occurrence matrix. Regarding continuous data, the latent block model is not appropriated in many cases. As non-negative matrix factorization, it treats symmetrically the two sets, and the estimation of associated parameters requires a variational approximation. In this paper we focus on continuous data matrix without restriction to non negative matrix. We propose a parsimonious mixture model allowing to overcome the limits of the latent block model.

Co-occurrence data matrices arise frequently in various important applications such as a document clustering. By considering a multinomial mixture model, we present a new probabilistic Self-Organizing Map (SOM) for clustering and visualizing this kind of data. Contrary to SOM, our proposed learning algorithm optimizes an objective function. Its performances are evaluated by using Monte Carlo simulations and real datasets.

Visualization of the massive datasets needs new methods which are able to quickly and easily reveal their contents. The projection of the data cloud is an interesting paradigm in spite of its difficulty to be explored when data plots are too numerous. So we study a new way to show a bidimensional projection from a multidimensional data cloud: our generative model constructs a tabular view of the projected cloud. We are able to show the high densities areas by their non equidistributed discretization. This approach is an alternative to the self-organizing map when a projection does already exist. The resulting pixel views of a dataset are illustrated by projecting a data sample of real images: it becomes possible to observe how are laid out the class labels or the frequencies of a group of modalities without being lost because of a zoom enlarging change for instance. The conclusion gives perspectives to this original promising point of view to get a readable projection for a statistical data analysis of large data samples.

Summary form only given. Cluster analysis is an important tool in a variety of scientific areas such as pattern recognition, information retrieval, microarray, data mining, and so forth. Although many clustering procedures such as hierarchical clustering, k-means or self-organizing maps, aim to construct an optimal partition on the set of objects I or, sometimes, on the set of variables J, there are other methods, called block clustering methods, which consider simultaneously the two sets and organize the data into homogeneous blocks. These methods are speedy and can process large data sets. They require much less computations than if one works on I and J separately. The mixture model is undoubtedly one of the greatest contributions to clustering. Recently we have proposed a generalized EM algorithm (GEM) to maximize a variational approximation of the likelihood. The proposed algorithm is an iterative algorithm whose steps are carried out by the application of the EM algorithm on intermediate mixture models. This paper focus on the clustering context. It deals to compare block GEM and two-way EM, i.e. EM applied separately on I and J. Results on simulated data are given, confirming that block GEM gives much better performance than two-way EM.

Although many clustering procedures aim to construct an optimal partition of objects or, sometimes, of variables, there are other methods, called block clustering methods, which consider simultaneously the two sets and organize the data into homogeneous blocks. Recently, we have proposed a new mixture model called block mixture model which takes into account this situation. This model allows one to embed simultaneous clustering of objects and variables in a mixture approach. We have studied this probabilistic model under the classification likelihood approach and developed a new algorithm for simultaneous partitioning based on the Classification EM algorithm. In this paper, we consider the block clustering problem under the maximum likelihood approach and the goal of our contribution is to estimate the parameters of this model. Unfortunately, the application of the EM algorithm for the block mixture model cannot be made directly; difficulties arise due to the dependence structure in the model and approximations are required. Using a variational approximation, we propose a generalized EM algorithm to estimate the parameters of the block mixture model and, to illustrate our approach, we study the case of binary data by using a Bernoulli block mixture.