This paper proposes a novel k-medoids approximation algorithm to handle large-scale datasets with reasonable computational time and memory complexity. We develop a local-search algorithm that iteratively improves the medoid selection based on the estimation of the k-medoids objective. A single batch of size m

This paper deals with uncertainty quantification and out-of-distribution detection in deep learning using Bayesian and ensemble methods. It proposes a practical solution to the lack of prediction diversity observed recently for standard approaches when used out-of-distribution (Ovadia et al., 2019; Liu et al., 2021). Considering that this issue is mainly related to a lack of weight diversity, we claim that standard methods sample in "over-restricted" regions of the weight space due to the use of "over-regularization" processes, such as weight decay and zero-mean centered Gaussian priors. We propose to solve the problem by adopting the maximum entropy principle for the weight distribution, with the underlying idea to maximize the weight diversity. Under this paradigm, the epistemic uncertainty is described by the weight distribution of maximal entropy that produces neural networks "consistent" with the training observations. Considering stochastic neural networks, a practical optimization is derived to build such a distribution, defined as a trade-off between the average empirical risk and the weight distribution entropy. We provide both theoretical and numerical results to assess the efficiency of the approach. In particular, the proposed algorithm appears in the top three best methods in all configurations of an extensive out-of-distribution detection benchmark including more than thirty competitors.

We consider the problem of collaborative learning with non-identical inter-node distributions and unbalanced training sample sizes. This problem arises in many real-world machine-learning scenarios and is particularly difficult to address due to the specificity of data architecture. Moreover, global models produced by traditional collaborative learning approaches, as federated learning methods, often fail to provide accurate models for each node due to distribution shifts. In this work, we propose to address this problem through a personalized collaborative algorithm returning a weighted average of the other nodes’ models to each node. The personalization consists in deriving a local weighting based on the optimization of an estimate of the weighted average model risk. We provide a theoretical framework of the approach by showing that the proposed optimization leads to a water-filling optimization where the optimal weighting solves a trade-off between integrating nearby nodes and minimizing the local variance. We derive theoretical insights based on the role of local bias and local variance of each model. We test our algorithm on five datasets, including four real-world ones, and show significant improvements in terms of individual node accuracy.

Physics-Informed Neural Networks (PINNs) have gained much attention in various fields of engineering thanks to their capability of incorporating physical laws into the models. However, the assessment of PINNs in industrial applications involving coupling between mechanical and thermal fields is still an active research topic. In this work, we present an application of PINNs to a non-Newtonian fluid thermo-mechanical problem which is often considered in the rubber calendering process. We demonstrate the effectiveness of PINNs when dealing with inverse and ill-posed problems, which are impractical to be solved by classical numerical discretization methods. We study the impact of the placement of the sensors and the distribution of unsupervised points on the performance of PINNs in a problem of inferring hidden physical fields from some partial data. We also investigate the capability of PINNs to identify unknown physical parameters from the measurements captured by sensors. The effect of noisy measurements is also considered throughout this work. The results of this paper demonstrate that in the problem of identification, PINNs can successfully estimate the unknown parameters using only the measurements on the sensors. In ill-posed problems where boundary conditions are not completely defined, even though the placement of the sensors and the distribution of unsupervised points have a great impact on PINNs performance, we show that the algorithm is able to infer the hidden physics from local measurements.

Budgeted learning is a research field of growing interest that aims at including real world resource constraints into the design of machine learning models, for instance to reduce real environment prediction time. One family of method to do so consists in simplifying a pre-trained machine learning model in order to fit the prediction time constraints, while keeping model’s prediction accuracy as best as possible.However we show in this work that the performance of these kinds of methods strongly depends on the pre-trained model structure. To overcome this dependence, we propose to tackle the budgeted prediction time optimization problem, by using equivalent classifiers with different structures and therefore different computation costs. The main contribution of this work is to propose an innovative evolutionary computing approach to decrease the prediction time of random forest classifiers, by using the notion of equivalence between decision trees. This method is applied for a real-time fall detection system embedding.Our genetic algorithm relies on two core operations : classifier equivalence and decision tree pruning. The first step of our method consists in building, from a pre-trained random forest, an initial population of random forests, that share the same decision function but have different structures using a randomized equivalent trees generation procedure. Then a genome reduction operation is iteratively applied on the individuals via random pruning based mutations.Our experiments show good reduction of random forests prediction time, as well as an efficient impact of using equivalent decision trees to reach better budgeted prediction time solutions. Results obtained both on a synthetic data made of gaussian-shaped clusters and on a real industrial fall detection dataset, advocate for the efficiency of our genetic random pruning approach in reducing random forests prediction time and for the use of equivalent decision trees in budgeted learning.

We present a novel instance-based approach to handle regression tasks in the context of supervised domain adaptation under an assumption of covariate shift. The approach developed in this paper is based on the assumption that the task on the target domain can be efficiently learned by adequately reweighting the source instances during training phase. We introduce a novel formulation of the optimization objective for domain adaptation which relies on a discrepancy distance characterizing the difference between domains according to a specific task and a class of hypotheses. To solve this problem, we develop an adversarial network algorithm which learns both the source weighting scheme and the task in one feed-forward gradient descent. We provide numerical evidence of the relevance of the method on public data sets for regression domain adaptation through reproducible experiments.

Transfer learning has attracted growing attention from the machine learning community. It addresses real-world issues that were not considered in the classical methods, especially with model-based methods when source data is not available. Recent work on transfer learning for Decision Trees has been proposed in the form of two algorithms named SER and STRUT. These methods tackle with success potential changes between source and target data, however several classification tasks may face imbalance data, hence degrading the performance of such transfer methods. In this paper, we study the impact of class imbalance on these algorithms and propose an adaptation for each of them, named SER* and STRUT*. These variants have been tested on real and synthetic data, these latter revealing to be of great use to control at the same time various transformations between source and target along with imbalance conditions. Results on both types of data show the benefits of our approach and suggest several perspectives for further research.