Recently, Optimal Transport has been proposed as a probabilistic framework in Machine Learning for comparing and manipulating probability distributions. This is rooted in its rich history and theory, and has offered new solutions to different problems in machine learning, such as generative modeling and transfer learning. In this survey we explore contributions of Optimal Transport for Machine Learning over the period 2012 - 2023, focusing on four sub-fields of Machine Learning: supervised, unsupervised, transfer and reinforcement learning. We further highlight the recent development in computational Optimal Transport and its extensions, such as partial, unbalanced, Gromov and Neural Optimal Transport, and its interplay with Machine Learning practice.

In this article, we propose an approach for federated domain adaptation, a setting where distributional shift exists among clients and some have unlabeled data. The proposed framework, FedDaDiL, tackles the resulting challenge through dictionary learning of empirical distributions. In our setting, clients’ distributions represent particular domains, and Fed-DaDiL collectively trains a federated dictionary of empirical distributions. In particular, we build upon the Dataset Dictionary Learning framework by designing collaborative communication protocols and aggregation operations. The chosen protocols keep clients’ data private, thus enhancing overall privacy compared to its centralized counterpart. We empirically demonstrate that our approach successfully generates labeled data on the target domain with extensive experiments on (i) Caltech-Office, (ii) TEP, and (iii) CWRU benchmarks. Furthermore, we compare our method to its centralized counterpart and other benchmarks in federated domain adaptation.

In this paper, we consider the intersection of two problems in machine learning: Multi-Source Domain Adaptation (MSDA) and Dataset Distillation (DD). On the one hand, the first considers adapting multiple heterogeneous labeled source domains to an unlabeled target domain. On the other hand, the second attacks the problem of synthesizing a small summary containing all the information about the datasets. We thus consider a new problem called MSDA-DD. To solve it, we adapt previous works in the MSDA literature, such as Wasserstein Barycenter Transport and Dataset Dictionary Learning, as well as DD method Distribution Matching. We thoroughly experiment with this novel problem on four benchmarks (Caltech-Office 10, Tennessee-Eastman Process, Continuous Stirred Tank Reactor, and Case Western Reserve University), where we show that, even with as little as 1 sample per class, one achieves state-of-the-art adaptation performance.

This paper seeks to solve Multi-Source Domain Adaptation (MSDA), which aims to mitigate data distribution shifts when transferring knowledge from multiple labeled source domains to an unlabeled target domain. We propose a novel MSDA framework based on dictionary learning and optimal transport. We interpret each domain in MSDA as an empirical distribution. As such, we express each domain as a Wasserstein barycenter of dictionary atoms, which are empirical distributions. We propose a novel algorithm, DaDiL, for learning via mini-batches: (i) atom distributions; (ii) a matrix of barycentric coordinates. Based on our dictionary, we propose two novel methods for MSDA: DaDil-R, based on the reconstruction of labeled samples in the target domain, and DaDiL-E, based on the ensembling of classifiers learned on atom distributions. We evaluate our methods in 3 benchmarks: Caltech-Office, Office 31, and CRWU, where we improved previous state-of-the-art by 3.15%, 2.29%, and 7.71% in classification performance. Finally, we show that interpolations in the Wasserstein hull of learned atoms provide data that can generalize to the target domain.

We propose an algorithm to perform the low-rank Boolean Canonical Polyadic Decomposition (BCPD) of a binary tensor. The proposed approach is based on the AO-ADMM strategy introduced in [1] and uses a post-nonlinear mixture model for binary sources. We show that this new method is better suited for low-rank approximation of binary tensors compared to other similar methods. We also provide an easy-to-check uniqueness condition for the BCPD. This is the first time that such a condition is derived for Boolean decompositions.

In this paper, we address the problem of learning compact similarity-preserving embeddings for massive high-dimensional streams of data in order to perform efficient similarity search. We present a new online method for computing binary compressed representations -sketches- of high-dimensional real feature vectors. Given an expected code length c and high-dimensional input data points, our algorithm provides a c-bits binary code for preserving the distance between the points from the original high-dimensional space. Our algorithm does not require neither the storage of the whole dataset nor a chunk, thus it is fully adaptable to the streaming setting. It also provides low time complexity and convergence guarantees. We demonstrate the quality of our binary sketches through experiments on real data for the nearest neighbors search task in the online setting.

We propose in this paper a new technique to investigate the Event-Related Potentials, or Evoked-Response Potentials, in the electroencephalographic signal. The multidimensional electroencephalographic signal is first spatially filtered to enhance the Evoked-Response Potentials using the xDAWN algorithm and, second, the single trial latencies (whatever their origins: physiological or electronical) are estimated by maximizing a cross correlation without any a priori model. The performance of this approach is illustrated on two classical P300-Speller electroencephalographic databases (BCI Competition II and III). The single-trial distribution of P300 Evoked-Response Potential is deblurred using the proposed resynchronization algorithm for applications in particular to Brain Computer Interfaces.

We consider a geometrical approach for solving the Nonnegative Blind Source Separation (N-BSS) problem in the case of noiseless linear instantaneous mixture model. When the sources are nonnegative, the scatter plot of the mixed data is contained in the simplicial cone generated by the mixing matrix. The proposed method, called Simplicial Cone Shrinking Algorithm for Unmixing Nonnegative Sources (SCSA-UNS), estimates the mixing matrix and the sources by finding the Minimum Volume (MV) simplicial cone containing all the mixed data. Simulations on synthetic data shows the efficiency of the proposed method.

The separation of the informative part from an observed dataset is a significant step for dimension reduction and features extraction. In this paper, we present an approach for adaptive artifact rejection from the electroencephalogram (EEG). The main aim of our work is to increase performance of classification algorithms which have a deal with the EEG and are used in cognitive games. We provide a method to separate the EEG into informative and noised parts, select informative one and rank its dimensions. The proposed approach is based on the theoretical relation between classification accuracy, mutual information and normalized graph cut (NC) value. The presented algorithm requires a priori known class labeled EEG dataset that are utilized for a calibration phase of the brain-computer interface (BCI). Experimental results on datasets from BCI competitions show its applicability for cognitive games.

Independent Component Analysis (ICA) is a well-known technique for solving blind source separation (BSS) problem. However “classical” ICA algorithms seem not suited for non-negative sources. This paper proposes a gradient descent approach for solving the Non-Negative Independent Component Analysis problem (NNICA). NNICA original separation criterion contains the discontinuous sign function whose minimization may lead to ill convergence (local minima) especially for sparse sources. Replacing the discontinuous function by a continuous one tanh, we propose a more accurate regularized Gradient algorithm called “Exact” Regularized Gradient (ERG) for NNICA. Experiments on synthetic data with different sparsity degrees illustrate the efficiency of the proposed method and a comparison shows that the proposed ERG outperforms existing methods.

We consider the problem of pile-ups occurring in gamma spectrometry signals for HPGe detectors. The temporal signal is transformed in a sequence of busy and idle periods, each busy period being characterized by its duration and its associated energy. We present an estimator to correct the pile-up phenomenon, based on an analytical formula. Applications on simulations and real spectrometrical signals are presented, which show good adequation between what we wish to retrieve and the estimation

This article presents a problem encountered in nuclear physics, queuing theory and point processes. The studied signal consists of pulses of random length and energy, possibly sampled, whose time occurrences are points of an homogenous Poisson process. Incoming pulses can combine into pile-ups, which results in a biased estimation of the density of lengths and energies. We introduce a model based on two marked point processes and derive an analytical relation between the probability density function (pdf) of the observed pile-ups and the pdf of the pulses, that leads to an algorithm for pile-up correction, in both continuous time and discrete time signals. Simulations show cancellation of the pile-up effect and prove efficiency of the algorithms in gamma spectrometry.

We address the problem of using an array of sensors for separating the signal emitted by a narrowband source from unwanted disturbance signals (jammers and noise). The source of interest steering vector is assumed to be known exactly and the goal is to estimate the signal of interest with the maximum signal to interference plus noise ratio (SINR). The classical solution to this problem is the minimum variance beamformer (MVB). But the classical estimator of this spatial filter shows poor performance for small number of samples. We demonstrate by means of a Cramer-Rao bound calculation that every unbiased estimator of the MVB achieves low SINR, and we propose a new biased estimator which shows improved performance with small number of data.

We address the problem of using an array of sensors for detecting a narrow band source and separating its signal from unwanted disturbance signals, that is jammers and noise. The power of the desired signal is assumed to move from one level to another. This second order non-stationarity occurs, for instance, in frequency hopping systems, and more generally, at the beginning or at the end of any communication. We derive a method based on the generalized eigenstructure of two covariance matrices which requires no a priori knowledge of the array manifold, but only second order stationarity of the disturbance signals. The loss in signal to interference plus noise ratio (SINR) due to finite sample effect is calculated in closed form at the first order and validated by simulations. This last result shows that the method gives interesting performance in a wide range of situations.