This paper proposes a novel k-medoids approximation algorithm to handle large-scale datasets with reasonable computational time and memory complexity. We develop a local-search algorithm that iteratively improves the medoid selection based on the estimation of the k-medoids objective. A single batch of size m

This paper deals with uncertainty quantification and out-of-distribution detection in deep learning using Bayesian and ensemble methods. It proposes a practical solution to the lack of prediction diversity observed recently for standard approaches when used out-of-distribution (Ovadia et al., 2019; Liu et al., 2021). Considering that this issue is mainly related to a lack of weight diversity, we claim that standard methods sample in "over-restricted" regions of the weight space due to the use of "over-regularization" processes, such as weight decay and zero-mean centered Gaussian priors. We propose to solve the problem by adopting the maximum entropy principle for the weight distribution, with the underlying idea to maximize the weight diversity. Under this paradigm, the epistemic uncertainty is described by the weight distribution of maximal entropy that produces neural networks "consistent" with the training observations. Considering stochastic neural networks, a practical optimization is derived to build such a distribution, defined as a trade-off between the average empirical risk and the weight distribution entropy. We provide both theoretical and numerical results to assess the efficiency of the approach. In particular, the proposed algorithm appears in the top three best methods in all configurations of an extensive out-of-distribution detection benchmark including more than thirty competitors.

We consider the problem of collaborative learning with non-identical inter-node distributions and unbalanced training sample sizes. This problem arises in many real-world machine-learning scenarios and is particularly difficult to address due to the specificity of data architecture. Moreover, global models produced by traditional collaborative learning approaches, as federated learning methods, often fail to provide accurate models for each node due to distribution shifts. In this work, we propose to address this problem through a personalized collaborative algorithm returning a weighted average of the other nodes’ models to each node. The personalization consists in deriving a local weighting based on the optimization of an estimate of the weighted average model risk. We provide a theoretical framework of the approach by showing that the proposed optimization leads to a water-filling optimization where the optimal weighting solves a trade-off between integrating nearby nodes and minimizing the local variance. We derive theoretical insights based on the role of local bias and local variance of each model. We test our algorithm on five datasets, including four real-world ones, and show significant improvements in terms of individual node accuracy.

We present a novel instance-based approach to handle regression tasks in the context of supervised domain adaptation under an assumption of covariate shift. The approach developed in this paper is based on the assumption that the task on the target domain can be efficiently learned by adequately reweighting the source instances during training phase. We introduce a novel formulation of the optimization objective for domain adaptation which relies on a discrepancy distance characterizing the difference between domains according to a specific task and a class of hypotheses. To solve this problem, we develop an adversarial network algorithm which learns both the source weighting scheme and the task in one feed-forward gradient descent. We provide numerical evidence of the relevance of the method on public data sets for regression domain adaptation through reproducible experiments.