We propose an approach to learn a multiattribute utility function to model, explain or predict the value system of a Decision Maker. The main challenge of the modelling task is to describe human values and preferences in the presence of interacting attributes while keeping the utility function as simple as possible. We focus on the generalized additive decomposable utility model which allows interactions between attributes while preserving some additive decomposability of the evaluation model. We present a learning approach able to identify the factors of interacting attributes and to learn the utility functions defined on these factors. This approach relies on the determination of a sparse representation of the ANOVA decomposition of the multiattribute utility function using multiple kernel learning. It applies to both continuous and discrete attributes. Numerical tests are performed to demonstrate the practical efficiency of the learning approach.

BACKGROUND: Graphical representations are useful to model complex data in general and biological interactions in particular. Our main motivation is the comparison of metabolic networks in the wider context of developing noninvasive accurate diagnostic tools. However, comparison and classification of graphs is still extremely challenging, although a number of highly efficient methods such as graph neural networks were developed in the recent decade. Important aspects are still lacking in graph classification: interpretability and guarantees on classification quality, i.e., control of the risk level or false discovery rate control. RESULTS: In our contribution, we introduce a statistically sound approach to control the false discovery rate in a classification task for graphs in a semi-supervised setting. Our procedure identifies novelties in a dataset, where a graph is considered to be a novelty when its topology is significantly different from those in the reference class. It is noteworthy that the procedure is a conformal prediction approach, which does not make any distributional assumptions on the data and that can be seen as a wrapper around traditional machine learning models, so that it takes full advantage of existing methods. The performance of the proposed method is assessed on several standard benchmarks. It is also adapted and applied to the difficult task of classifying metabolic networks, where each graph is a representation of all metabolic reactions of a bacterium and to real task from a cancer data repository. CONCLUSIONS: Our approach efficiently controls - in highly complex data - the false discovery rate, while maximizing the true discovery rate to get the most reasonable predictive performance. This contribution is focused on confident classification of complex data, what can be further used to explore complex human pathologies and their mechanisms.

Multicriteria decision making requires defining the result of conflicting and possibly interacting criteria. Allowing criteria interactions in a decision model increases the complexity of the preference learning task due to the combinatorial nature of the possible interactions. In this paper, we propose an approach to learn a decision model in which the interaction pattern is revealed from preference data and kept as simple as possible. We consider weighted aggregation functions like multilinear utilities or Choquet integrals, admitting representations including non-linear terms measuring the joint benefit or penalty attached to some combinations of criteria. The weighting coefficients known as Möbius masses model positive or negative synergies among criteria. We propose an approach to learn the Möbius masses, based on iterative reweighted least square for sparse recovery, and dualization to improve scalability. This approach is applied to learn sparse representations of the multilinear utility model and conjunctive/disjunctive forms of the discrete Choquet integral from preferences examples, in aggregation problems possibly involving more than 20 criteria.

BACKGROUND: Current clinical routines rely more and more on "omics" data such as flow cytometry data from host and microbiota. Cohorts variability in addition to patients' heterogeneity and huge dimensions make it difficult to understand underlying structure of the data and decipher pathologies. Patients stratification and diagnostics from such complex data are extremely challenging. There is an acute need to develop novel statistical machine learning methods that are robust with respect to the data heterogeneity, efficient from the computational viewpoint, and can be understood by human experts. RESULTS: We propose a novel approach to stratify cell-based observations within a single probabilistic framework, i.e., to extract meaningful phenotypes from both patients and cells data simultaneously. We define this problem as a double clustering problem that we tackle with the proposed approach. Our method is a practical extension of the Latent Dirichlet Allocation and is used for the Double Clustering task (LDA-DC). We first validate the method on artificial datasets, then we apply our method to two real problems of patients stratification based on cytometry and microbiota data. We observe that the LDA-DC returns clusters of patients and also clusters of cells related to patients' conditions. We also construct a graphical representation of the results that can be easily understood by humans and are, therefore, of a big help for experts involved in pre-clinical research.

This paper deals with preference elicitation within Choquet Expected Utility (CEU) theory for decision making under uncertainty. We consider the Savage’s framework with a finite set of states and assume that preferences of the Decision Maker over acts are observable. The CEU model involves two parameters that must be tuned to the value system of the decision maker: a set function (capacity) modeling weights attached to events, of size exponential in the number of states, and a utility function defined on the space of outcomes. Our aim is to learn a sparse representation of the CEU model from preference data. We propose and test a preference learning approach based on a spline representation of utilities and the sparse learning of capacities to obtain CEU models achieving a good tradeoff between the aim of sparsity and the expressivity required by preference data.

Cascade learning with abstention and individualised feature selection is a class of models in high demand in personalised medical applications. The cascade consists of sequential classifiers and rejectors , where the classifiers estimate confidence of prediction, and the rejectors evaluate an expected cost-to-go of features not selected yet. The number of models is exponential in the number of features and, therefore, the challenge is to find efficient heuristics for the NP-hard problem. The state-of-the-art is based on complex deep neural networks . We introduce an efficient and robust approach based on a probabilistic graphical model representing a unified probabilistic classifier that can be applied at any stage of a multi-stage sequential model. As for the rejector, we build it on the probabilistic-based neural network that incorporates the very same probabilistic model to treat unobserved feature values. We illustrate the efficiency of the proposed method on several data sets, and compare our results to the state-of-the-art.

Integer models are of particular interest for applications where predictive models are supposed not only to be accurate but also interpretable to human experts. We introduce a novel penalty term called Facets whose primary goal is to favour integer weights. Our theoretical results illustrate the behaviour of the proposed penalty term: for small enough weights, the Facets matches the L1 penalty norm, and as the weights grow, it approaches the L2 regulariser. We provide the proximal operator associated with the proposed penalty term, so that the regularised empirical risk minimiser can be computed efficiently. We also introduce the Strongly Convex Facets, and discuss its theoretical properties. Our numerical results show that while achieving the state-of-the-art accuracy, optimisation of a loss function penalised by the proposed Facets penalty term leads to a model with a significant number of integer weights.

Graph neural networks (GNNs) have known an increasing success recently, with many GNN variants achieving state-of-the-art results on node and graph classification tasks. The proposed GNNs, however, often implement complex node and graph embedding schemes, which makes it challenging to explain their performance. In this paper, we investigate the link between a GNN’s expressiveness , that is, its ability to map different graphs to different representations, and its generalization performance in a graph classification setting. In particular, we propose a principled experimental procedure where we (i) define a practical measure for expressiveness, (ii) introduce an expressiveness-based loss function that we use to train a simple yet practical GNN that is permutation-invariant, (iii) illustrate our procedure on benchmark graph classification problems and on an original real-world application. Our results reveal that expressiveness alone does not guarantee a better performance, and that a powerful GNN should be able to produce graph representations that are well separated with respect to the class of the corresponding graphs.

An important question in microbiology is whether treatment causes changes in gut flora, and whether it also affects metabolism. The reconstruction of causal relations purely from non-temporal observational data is challenging. We address the problem of causal inference in a bivariate case, where the joint distribution of two variables is observed. We consider, in particular, data on discrete domains. The state-of-the-art causal inference methods for continuous data suffer from high computational complexity. Some modern approaches are not suitable for categorical data, and others need to estimate and fix multiple hyper-parameters. In this contribution, we introduce a novel method of causal inference which is based on the widely used assumption that if X causes Y, then P(X) and P(Y|X) are independent. We propose to explore a semi-supervised approach where P(Y|X) and P(X) are estimated from labeled and unlabeled data respectively, whereas the marginal probability is estimated potentially from much more (cheap unlabeled) data than the conditional distribution. We validate the proposed method on the standard cause-effect pairs. We illustrate by experiments on several benchmarks of biological network reconstruction that the proposed approach is very competitive in terms of computational time and accuracy compared to the state-of-the-art methods. Finally, we apply the proposed method to an original medical task where we study whether drugs confound human metagenome.

In many prediction tasks such as medical diagnostics, sequential decisions are crucial to provide optimal individual treatment. Budget in real-life applications is always limited, and it can represent any limited resource such as time, money, or side effects of medications. In this contribution, we develop a POMDP-based framework to learn cost-sensitive heterogeneous cascading systems. We provide both the theoretical support for the introduced approach and the intuition behind it. We evaluate our novel method on some standard benchmarks, and we discuss how the learned models can be interpreted by human experts.

BACKGROUND: Metabolic networks reflect the relationships between metabolites (biomolecules) and the enzymes (proteins), and are of particular interest since they describe all chemical reactions of an organism. The metabolic networks are constructed from the genome sequence of an organism, and the graphs can be used to study fluxes through the reactions, or to relate the graph structure to environmental characteristics and phenotypes. About ten years ago, Takemoto et al. (2007) stated that the structure of prokaryotic metabolic networks represented as undirected graphs, is correlated to their living environment. Although metabolic networks are naturally directed graphs, they are still usually analysed as undirected graphs. RESULTS: We implemented a pipeline to reconstruct metabolic networks from genome data and confirmed some of the results of Takemoto et al. (2007) with today data using up-to-date databases. However, Takemoto et al. (2007) used only a fraction of all available enzymes from the genome and taking into account all the enzymes we fail to reproduce the main results. Therefore, we introduce three robust measures on directed representations of graphs, which lead to similar results regardless of the method of network reconstruction. We show that the size of the largest strongly connected component, the flow hierarchy and the Laplacian spectrum are strongly correlated to the environmental conditions. CONCLUSIONS: We found a significant negative correlation between the size of the largest strongly connected component (a cycle) and the optimal growth temperature of the considered prokaryotes. This relationship holds true for the spectrum, high temperature being associated with lower eigenvalues. The hierarchy flow shows a negative correlation with optimal growth temperature. This suggests that the dynamical properties of the network are dependant on environmental factors.

Score learning aims at taking advantage of supervised learning to produce interpretable models which facilitate decision making. Scoring systems are simple classification models that let users quickly perform stratification. Ideally, a scoring system is based on simple arithmetic operations, is sparse, and can be easily explained by human experts. In this contribution, we introduce an original methodology to simultaneously learn interpretable binning mapped to a class variable, and the weights associated with these bins contributing to the score. We develop and show the theoretical guarantees for the proposed method. We demonstrate by numerical experiments on benchmark data sets that our approach is competitive compared to the state-of-the-art methods. We illustrate by a real medical problem of type 2 diabetes remission prediction that a scoring system learned automatically purely from data is comparable to one manually constructed by clinicians.

Discovering complex interactions is an important issue in numerous fields ranging from social sciences to systems biology. Over the past few decades, many network learning methods have exhibited competitive results on various types of data. A commonly reached conclusion is that some learning approaches are more advisable than others depending on the dataset type or the complexity of the underlying network. Another frequently encountered issue relates to the ever increasing number of variables that need to be simultaneously dealt with, especially when only a small number of observations is available. The ScaleNet, a novel reconstruction method which can embed different types of network discovery approaches within a spectral framework for large graphical model was introduced recently. The approach identifies sets of connected variables based on the magnitude and sign of the eigenvector elements of a normalized graph Laplacian matrix, and it learns in parallel multiple relevant sub-graphs of a large network. However, the number of eigenvectors to be used and the size of the sub-graphs are to be fixed by an expensive procedure of cross-validation. In this contribution, we propose heuristics to find both optimal number of eigenvectors and the number of nodes in the sub-networks. We illustrate by the results on standard large-scale data sets and on a real human gut graph reconstruction that the proposed approaches save computational time, i.e. are efficient, and reach the state-of-the-art performance.

Score learning aims at taking advantage of supervised learning to estimate interpretable models which facilitate decision making. Ideally, a scoring system is based on simple arithmetic operations, is sparse, and can be easily explained by human experts. In this contribution, we introduce an original methodology to simultaneously learn interpretable binning mapped to a class variable, and the weights associated with these bins contributing to the score. We show by numerical experiments on benchmark data sets that our approach is competitive compared to the state-of-the-art methods. We illustrate by a real medical problem of type 2 diabetes remission prediction that a scoring system learned automatically is comparable to one manually constructed by clinicians.

Prior knowledge integration aims at taking advantage of cheap plentiful data to improve the efficiency of supervised learning procedures. For parametric models, however, it is a challenging task. In this contribution, we introduce a novel simple methodology to incorporate prior knowledge into a supervised objective function. We propose to introduce the prior knowledge in the form of joint probability of observations and labels. We discuss the nature of features in discriminative and generative models and hence, differences in priors integration. We illustrate the efficiency of the proposed method both by synthetic data sets and by our results on a realistic large-scale sequence labeling task.

Feature selection is used to preserve significant properties of data in a compact space. In particular, feature selection is needed in applications, where information comes from multiple heterogeneous high dimensional sources. Data integration, however, is a challenge in itself. In our contribution, we introduce a feature selection framework based on powerful visualisation capabilities of self-organising maps, where the deep structure can be learned in a supervised or unsupervised manner. For a supervised version of the deep SOM, we propose to carry out inference with a linear SVM. A forward-backward procedure helps to converge to an optimal feature set. We show by experiments on real large-scale biomedical data set that the proposed methods embed data in a new compact meaningful representation, allow to visualise biomedical signatures, and also lead to a reasonable classification accuracy compared to the state-of-the-art methods.

BACKGROUND: The last decades witnessed an explosion of large-scale biological datasets whose analyses require the continuous development of innovative algorithms. Many of these high-dimensional datasets are related to large biological networks with few or no experimentally proven interactions. A striking example lies in the recent gut bacterial studies that provided researchers with a plethora of information sources. Despite a deeper knowledge of microbiome composition, inferring bacterial interactions remains a critical step that encounters significant issues, due in particular to high-dimensional settings, unknown gut bacterial taxa and unavoidable noise in sparse datasets. Such data type make any a priori choice of a learning method particularly difficult and urge the need for the development of new scalable approaches. RESULTS: We propose a consensus method based on spectral decomposition, named Spectral Consensus Strategy, to reconstruct large networks from high-dimensional datasets. This novel unsupervised approach can be applied to a broad range of biological networks and the associated spectral framework provides scalability to diverse reconstruction methods. The results obtained on benchmark datasets demonstrate the interest of our approach for high-dimensional cases. As a suitable example, we considered the human gut microbiome co-presence network. For this application, our method successfully retrieves biologically relevant relationships and gives new insights into the topology of this complex ecosystem. CONCLUSIONS: The Spectral Consensus Strategy improves prediction precision and allows scalability of various reconstruction methods to large networks. The integration of multiple reconstruction algorithms turns our approach into a robust learning method. All together, this strategy increases the confidence of predicted interactions from high-dimensional datasets without demanding computations.

Semi-supervised learning aims at taking advantage of unlabeled data to improve the efficiency of supervised learning procedures. For discriminative models however, this is a challenging task. In this contribution, we introduce an original methodology for using unlabeled data through the design of a simple semi-supervised objective function. We prove that the corresponding semi-supervised estimator is asymptotically optimal. The practical consequences of this result are discussed for the case of the logistic regression model.