While many Machine Learning methods have been developed or transposed on Riemannian manifolds to tackle data with known non-Euclidean geometry, Optimal Transport (OT) methods on such spaces have not received much attention. The main OT tool on these spaces is the Wasserstein distance, which suffers from a heavy computational burden. On Euclidean spaces, a popular alternative is the Sliced-Wasserstein distance, which leverages a closed-form solution of the Wasserstein distance in one dimension, but which is not readily available on manifolds. In this work, we derive general constructions of Sliced-Wasserstein distances on Cartan-Hadamard manifolds, Riemannian manifolds with non-positive curvature, which include among others Hyperbolic spaces or the space of Symmetric Positive Definite matrices. Then, we propose different applications such as classification of documents with a suitably learned ground cost on a manifold, and data set comparison on a product manifold. Additionally, we derive non-parametric schemes to minimize these new distances by approximating their Wasserstein gradient flows.

The generative paradigm has become increasingly important in machine learning and deep slearning models. Among popular generative models are normalizing flows, which enable exact likelihood estimation by transforming a base distribution through diffeomorphic transformations. Extending the normalizing flow framework to handle time-indexed flows provided dynamic normalizing flows, a powerful tool to model time series, stochastic processes, and neural stochastic differential equations (SDEs). In this work, we propose a novel variant of dynamic normalizing flows, a Time-Changed Normalizing Flow (TCNF), based on time deformation of a Brownian motion which constitutes a versatile and extensive family of Gaussian processes. This approach enables us to effectively model some SDEs that cannot be modeled otherwise, including standard ones such as the well-known Ornstein-Uhlenbeck process, generalizes prior methodologies, and leads to improved results and better inference and prediction capability.

In Machine Learning, Mixup is a data-dependent regularization technique that consists in creating virtual samples by linearly interpolating input signals and their associated outputs. It has been shown to significantly improve accuracy on standard datasets, in particular in the field of vision. However, authors have pointed out that Mixup can produce out-of-distribution virtual samples and even contradictions in the augmented training set, potentially resulting in adversarial effects. In this paper, we introduce Local Mixup in which distant input samples are weighted down when computing the loss. In constrained settings we demonstrate that Local Mixup can create a trade-off between bias and variance, with the extreme cases reducing to vanilla training and classical Mixup. Using standardized computer vision benchmarks, we also show that Local Mixup can improve test accuracy.

We introduce the problem of Active Few-Shot Classification (AFSC) where the objective is to classify a small, initially unlabeled, dataset given a very restrained labeling budget. This problem can be seen as a rival paradigm to classical Transductive Few-Shot Classification (TFSC), as both these approaches are applicable in similar conditions. We first propose a methodology that combines statistical inference, and an original two-tier active learning strategy that fits well into this framework. We then adapt several standard vision benchmarks from the field of TFSC. Our experiments show the potential benefits of AFSC can be substantial, with gains in average weighted accuracy of up to 10% compared to state-of-the-art TFSC methods for the same labeling budget. We believe this new paradigm could lead to new developments and standards in data-scarce learning settings.

When dealing with signals, labeling a classification dataset implies to define classes that may approximate a smoother and more complicated ground truth. For example, natural images may contain multiple objects, only one of which is labeled in many vision datasets, or classes may result from the discretization of a regression problem where targets are continuous. Using cross-entropy to train deep models on such coarse labels is likely to roughly cut through the feature space, potentially disregarding the most meaningful such features, in particular losing information on the underlying fine-grain task. In this paper we are interested in the problem of solving fine-grain classification or regression, using a model trained on coarse-grain labels only. We show that standard cross-entropy can lead to overfitting to coarse-related features. We introduce an entropy-based regularization to promote more diversity in the feature space of trained models, and empirically demonstrate the efficacy of this methodology to reach better performance on the fine-grain problems. Our results are supported by theoretical developments and empirical validation.

Over the last few years, several works have proposed deep learning architectures to learn dynamical systems from observation data with no or little knowledge of the underlying physics. A line of work relies on learning representations where the dynamics of the underlying phenomenon can be described by a linear operator, based on the Koopman operator theory. However, despite being able to provide reliable long-term predictions for some dynamical systems in ideal situations, the methods proposed so far have limitations, such as requiring to discretize intrinsically continuous dynamical systems, leading to data loss, especially when handling incomplete or sparsely sampled data. Here, we propose a new deep Koopman framework that represents dynamics in an intrinsically continuous way, leading to better performance on limited training data, as exemplified on several datasets arising from dynamical systems.

BCI Motor Imagery datasets usually are small and have different electrodes setups. When training a Deep Neural Network, one may want to capitalize on all these datasets to increase the amount of data available and hence obtain good generalization results. To this end, we introduce a spatial graph signal interpolation technique, that allows to interpolate efficiently multiple electrodes. We conduct a set of experiments with five BCI Motor Imagery datasets comparing the proposed interpolation with spherical splines interpolation. We believe that this work provides novel ideas on how to leverage graphs to interpolate electrodes and on how to homogenize multiple datasets.

When dealing with electro or magnetoencephalography records, many supervised prediction tasks are solved by working with covariance matrices to summarize the signals. Learning with these matrices requires the usage of Riemanian geometry to account for their structure. In this paper, we propose a new method to deal with distributions of covariance matrices, and demonstrate its computational efficiency on M/EEG multivariate time series. More specifically, we define a Sliced-Wasserstein distance between measures of symmetric positive definite matrices that comes with strong theoretical guarantees. Then, we take advantage of its properties and kernel methods to apply this discrepancy to brain-age prediction from MEG data, and compare it to state-of-the-art algorithms based on Riemannian geometry. Finally, we show that it is an efficient surrogate to the Wasserstein distance in domain adaptation for Brain Computer Interface applications.

Ideally, evaluation of panchromatic and multispectral image fusion requires the use of a reference image, which is only available in a reduced scale protocol. Thus, no reference metrics such as the now standard Quality No Reference (QNR) were introduced. However, the QNR contains implicit implementation parameters which have not been studied yet. Using a statistical analysis based on rank correlation, we show that those parameters have a significant effect on the QNR values. Moreover, we extend previous results indicating that the QNR has low correlation with reference metrics at reduced scale. These results raise questions about the QNR’s relevance. They call for a standardization of implicit parameters so as to compare values across works, and for the QNR to only be seen as a complementary measure to reference metrics but not as a proxy. The developed protocol is also used to find the best set of implicit parameters, but could be generalized for the assessment of other no reference metrics. Finding such well behaved no reference metric is of critical interest for the development of unsupervised machine learning methods of pansharpening.

Hyperspectral unmixing allows to represent mixed pixels as a set of pure materials weighted by their abundances. Spectral features alone are often insufficient, so it is common to rely on other features of the scene. Matrix models become insufficient when the hyperspectral image is represented as a high-order tensor with additional features in a multimodal, multi-feature framework. Tensor models such as Canonical polyadic decomposition allow for this kind of unmixing, but lack a general framework and interpretability of the results. In this paper, we propose an interpretable methodological framework for low-rank Multi-feature hyperspectral unmixing based on tensor decomposition (MultiHU-TD) which incorporates the abundance sum-to-one constraint in the Alternating optimization ADMM algorithm, and provide in-depth mathematical, physical and graphical interpretation and connections with the extended linear mixing model. As additional features, we propose to incorporate mathematical morphology and reframe a previous work on neighborhood patches within MultiHU-TD. Experiments on real hyperspectral images showcase the interpretability of the model and the analysis of the results. Python and MATLAB implementations are made available on GitHub.

Variational models are among the state-of-the-art formulations for the resolution of ill-posed inverse problems. Following recent advances in learning-based variational settings, we investigate the end-to-end learning of variational models, more precisely of the regularization term given some observation model, jointly to the associated solver, so that we can optimize the reconstruction performance. In the proposed end-to-end setting, both the variational cost and the gradient-based solver are stated as neural networks using automatic differentiation for the latter. We consider an application to inverse problems with incomplete datasets (image inpainting and multivariate time series interpolation). We experimentally illustrate that this framework can lead to a significant gain in terms of reconstruction performance, including w.r.t. the direct minimization of the variational formulation derived from the known generative model.

In machine learning, classifiers are typically susceptible to noise in the training data. In this work, we aim at reducing intra-class noise with the help of graph filtering to improve the classification performance. Considered graphs are obtained by connecting samples of the training set that belong to a same class depending on the similarity of their representation in a latent space. We show that the proposed graph filtering methodology has the effect of asymptotically reducing intra-class variance, while maintaining the mean. While our approach applies to all classification problems in general, it is particularly useful in few-shot settings, where intra-class noise can have a huge impact due to the small sample selection. Using standardized benchmarks in the field of vision, we empirically demonstrate the ability of the proposed method to slightly improve state-of-the-art results in both cases of few-shot and standard classification.

The reconstruction of sea surface geophysical variables typically relies either on Optimal Interpolation (OI) or on model-based approaches which explicitly exploit a dynamical model. While the optimal interpolation suffers from smoothing issues making it unreliable in retrieving fine scale variability, the selection and parametrization of a dynamical model, when considering model based data assimilation strategies, remains a complex issue since several trade-offs between the models complexity and its applicability in sea surface data assimilation need to be carefully addressed. In this work, and motivated by the success of artificial intelligence algorithms in various signal processing fields as well as the increasing amounts of observations and simulation datasets, we explore a data driven Koopman model and formulate the reconstruction problem as solution of the classical Kalman filter in a space of observables. The proposed architecture although linear, is shown to outperform state-of-the-art non linear data-driven filtering schemes such as the Analog Data Assimilation (AnDA).

Remote sensing data from hyperspectral cameras suffer from limited spatial resolution, in which a single pixel of a hyperspectral image may contain information from several materials in the field of view. Blind hyperspectral image unmixing is the process of identifying the pure spectra of individual materials (i.e., endmembers) and their proportions (i.e., abundances) at each pixel. In this article, we propose a novel blind hyperspectral unmixing model based on the graph total variation (gTV) regularization, which can be solved efficiently by the alternating direction method of multipliers (ADMM). To further alleviate the computational cost, we apply the Nyström method to approximate a fully connected graph by a small subset of sampled points. Furthermore, we adopt the Merriman-Bence-Osher (MBO) scheme to solve the gTV-involved subproblem in ADMM by decomposing a gray-scale image into a bitwise form. A variety of numerical experiments on synthetic and real hyperspectral images are conducted, showcasing the potential of the proposed method in terms of identification accuracy and computational efficiency.

Hyperspectral image unmixing is an inverse problem aiming at recovering the spectral signatures of pure materials of interest (called endmembers) and estimating their proportions (called abundances) in every pixel of the image. However, in spite of a tremendous applicative potential and the avent of new satellite sensors with high temporal resolution, multitemporal hyperspectral unmixing is still a relatively underexplored research avenue in the community, compared to standard image unmixing. In this paper, we propose a new framework for multitemporal unmixing and endmember extraction based on a state-space model, and present a proof of concept on simulated data to show how this representation can be used to inform multitemporal unmixing with external prior knowledge, or on the contrary to learn the dynamics of the quantities involved from data using neural network architectures adapted to the identification of dynamical systems.

Despite some promising results under ideal conditions (i.e. noise-free and complete observation), learning chaotic dynamical systems from real life data is still a very challenging task. We propose a novel framework, which combines data assimilation schemes and neural network representation, namely Auto-Encoders and Ensemble Kalman Smoother, to learn the governing equations of dynamical systems. By treating the learning as a Bayesian estimation problem, our framework can deal with noisy and partial observations. Experiments on the chaotic Lorenz-63 dynamics with different noise settings demonstrate the advantages of our method over the state-of-the-art.

The upcoming Surface Water Ocean Topography (SWOT) satellite altimetry mission is expected to yield two-dimensional high-resolution measurements of Sea Surface Height (SSH), thus allowing for a better characterization of the mesoscale and submesoscale eddy field. However, to fulfill the promises of this mission, filtering the tidal component of the SSH measurements is necessary. This challenging problem is crucial since the posterior studies done by physical oceanographers using SWOT data will depend heavily on the selected filtering schemes. In this paper, we cast this problem into a supervised learning framework and propose the use of convolutional neural networks (ConvNets) to estimate fields free of internal tide signals. Numerical experiments based on an advanced North Atlantic simulation of the ocean circulation (eNATL60) show that our ConvNet considerably reduces the imprint of the internal waves in SSH data even in regions unseen by the neural network. We also investigate the relevance of considering additional data from other sea surface variables such as sea surface temperature (SST).

The forecasting and assimilation of sea surface dynamics from satellite-derived data is a challenging issue. Data-driven approaches have arisen as promising schemes to fully exploit satellite observations. A key feature of sea surface dynamics is that they relate to partially-observed dynamics. Here, guided by physical and mathematical considerations of the underlying dynamics of a given system, we propose a novel neural networks architecture for the identification of ordinary differential equations (ODE) of partially observed systems. Numerical experiments for toy models and a sea level anomaly dynamics illustrate the relevance of the proposed scheme for forecasting and assimilation issues w.r.t. other state-of-the-art data-driven models.

In hyperspectral imaging, spectral unmixing aims at decomposing the image into a set of reference spectral signatures corresponding to the materials present in the observed scene and their relative proportions in every pixel. While a linear mixing model was used for a long time, the complex nature of the physicochemical phenomena that affect the spectra of the materials led to shift the community's attention toward algorithms accounting for the variability of the endmembers. Such intraclass variations are mainly due to local changes in the composition of the materials and to illumination changes. In the physical remote sensing community, a popular model accounting for illumination variability is the radiative transfer model proposed by Hapke. It is, however, too complex to be directly used in hyperspectral unmixing in a tractable way. Instead, the extended linear mixing model (ELMM) allows to easily unmix the hyperspectral data accounting for changing illumination conditions and to address nonlinear effects to some extent. In this letter, we show that the ELMM can be obtained from the Hapke model by successively simplifying physical assumptions, whose validity we experimentally examine, thus demonstrating its relevance to handle illumination-induced variability in the unmixing problem.

Hyperspectral image unmixing has proven to be a useful technique to interpret hyperspectral data, and is a prolific research topic in the community. Most of the approaches used to perform linear unmixing are based on convex geometry concepts, because of the strong geometrical structure of the linear mixing model. However, many algorithms based on convex geometry are still used in spite of the underlying model not considering the intra-class variability of the materials. A natural question is to wonder to what extent these concepts and tools (Intrinsic Dimensionality estimation, endmember extraction algorithms, pixel purity) are still relevant when spectral variability comes into play. In this paper, we first analyze their robustness in a case where the linear mixing model holds in each pixel, but the endmembers vary in each pixel according to a prescribed variability model. In the light of this analysis, we propose an integrated unmixing chain which tries to adress the shortcomings of the classical tools used in the linear case, based on our previously proposed extended linear mixing model. We show the interest of the proposed approach on simulated and real datasets.

This work deals with data-driven and learning-based approaches to fill space-time sampling gaps in the observation of sea surface tracers such as Sea Surface Height (SSH), Sea Surface temperature (SST), Ocean Colour,... More precisely, we jointly address field super-resolution and the prediction of subgrid-scale features, which is novel to our knowledge. From a methodological point of view, we consider deep learning architectures with a view to learning geophysically-sound differential operators (i.e. trasnport operators). Based on an Observing System Simulation Experiment representative of SWOT fast sampling phase using NATL60 simulation data, we illustrate the relevance of the proposed methodological framework which reconstructs more than 90% of the variance of the high-resolution SSH anomaly field and above 90% of the subgrid-scale variance of this anomaly. We also illustrate significant gain w.r.t. other baseline neural network architectures and further discuss the relevance of the reported contribution for other tracer fields and case studies.

In this work, we propose an alternative to the Ensemble Kalman filter through the implementation of a neural networks filtering scheme based on a parametric stochastic model. From our numerical experiment, we prove the relevance of the proposed architecture in the reconstruction of geophysical fields with respect to the state-of-the-art schemes.

Recent studies have investigated the identification of governing equations of geophysical systems from data. Here, we investigate such identification issues for ocean surface dy-namcis from ocean remote sensing data. From a methodological point of view, we address the learning of data-driven dynamical models when only provided with a noisy training dataset. We propose a novel architecture that relies on data assimilation schemes to learn the underlying dynamical model through the minimization of a reconstruction cost. We demonstrate the relevance of the proposed architecture with respect to the state-of-the-art approaches in the identification and forecasting of synthetic and real case-studies.

Climate change and anthropogenic pressure are causing an indisputable decline in biodiversity; therefore, the need of environmental knowledge is important to develop the appropriate management plans. In this context, remote sensing and, specifically, hyperspectral imagery (HSI) can contribute to the generation of vegetation maps for ecosystem monitoring. To properly obtain such information and to address the mixed pixels inconvenience, the richness of the hyperspectral data allows the application of unmixing techniques. In this sense, a problem found by the traditional linear mixing model (LMM), a fully constrained least squared unmixing (FCLSU), is the lack of ability to account for spectral variability. This paper focuses on assessing the performance of different spectral unmixing models depending on the quality and quantity of endmembers. A complex mountainous ecosystem with high spectral changes was selected. Specifically, FCLSU and 3 approaches, which consider the spectral variability, were studied: scaled constrained least squares unmixing (SCLSU), Extended LMM (ELMM) and Robust ELMM (RELMM). The analysis includes two study cases: 1) robust endmembers and 2) nonrobust endmembers. Performances were computed using the reconstructed root-mean-square error (RMSE) and classification maps taking the abundances maps as inputs. It was demonstrated that advanced unmixing techniques are needed to address the spectral variability to get accurate abundances estimations. RELMM obtained excellent RMSE values and accurate classification maps with very little knowledge of the scene and minimum effort in the selection of endmembers, avoiding the curse of dimensionality problem found in HSI.

Hyperspectral images provide much more information than conventional imaging techniques, allowing a precise identification of the materials in the observed scene, but because of the limited spatial resolution, the observations are usually mixtures of the contributions of several materials. The spectral unmixing problem aims at recovering the spectra of the pure materials of the scene (endmembers), along with their proportions (abundances) in each pixel. In order to deal with the intra-class variability of the materials and the induced spectral variability of the endmembers, several spectra per material, constituting endmember bundles, can be considered. However, the usual abundance estimation techniques do not take advantage of the particular structure of these bundles, organized into groups of spectra. In this paper, we propose to use group sparsity by introducing mixed norms in the abundance estimation optimization problem. In particular, we propose a new penalty, which simultaneously enforces group and within-group sparsity, to the cost of being nonconvex. All the proposed penalties are compatible with the abundance sum-to-one constraint, which is not the case with traditional sparse regression. We show on simulated and real datasets that well-chosen penalties can significantly improve the unmixing performance compared to classical sparse regression techniques or to the naive bundle approach.

Hyperspectral image unmixing is a source separation problem aiming at recovering the spectra of the pure materials of the observed scene (called endmembers), as well as their relative proportions in each pixel of the image (called abundances). The variability of the materials has recently received a lot of attention in the community. In particular, a consequent number of models and algorithms have been proposed to estimate pixel-wise endmembers to account for their variability. These algorithms often rely on classical endmem-ber extraction algorithms to provide reference spectra. In difficult scenarios with shadows and significant variability these algorithms may fail. In this paper, we address this issue in the Extended Linear Mixing Model framework by considering that an endmember is a direction in the feature space, rather than a single point. Under this paradigm, we show that using k- means clustering with the cosine similarity outperforms geometric endmember extraction algorithms. We also design an algorithm to refine the estimation of the endmember directions, and to account for both illumination and intrinsic variability effects. We show the potential of the proposed algorithm on a synthetic dataset using real world spectra with variability, and a challenging real dataset of a natural scene.

Hyperspectral imagery (HSI) has become an important tool in ecosystem conservation due to its capability to perform accurate spectral unmixing, for vegetation mapping and ecosystem monitoring. An issue to be solved is the spectral variability of endmembers that can be induced by sensor noise and topographic changes. This spectral variability is considered by the Extended Linear Mixing Model (ELMM), which is applied to a mountainous ecosystem with high spectral variability and radiometric changes in each swath. The results obtained are very satisfactory, achieving reasonable abundance estimations and accurate characterization of the variability within the scene. ELMM allows studying the features of each pixel, including additional information about the characterization of the mixed pixels, in HSI by taking spectral variability into account. Moreover, it is observed that ELMM is robust to the absence of pure pixels as well as to noise.

Local Spectral Unmixing (LSU) methods perform the unmixing of hyperspectral data locally in regions of the image. The endmembers and their abundances in each pixel are extracted region-wise, instead of globally to mitigate spectral variability effects, which are less severe locally. However, it requires the local estimation of the number of endmembers to use. Algorithms for intrinsic dimensionality (ID) estimation tend to overestimate the local ID, especially in small regions. The ID only provides an upper bound of the application and scale dependent number of endmembers, which leads to extract irrelevant signatures as local endmembers, associated with meaningless local abundances. We propose a method to select in each region the best subset of the locally extracted endmembers. Collaborative sparsity is used to detect spurious endmembers in each region and only keep the most influent ones. We compute an algorithmic regularization path for this problem, giving access to the sequence of successive active sets of endmembers when the regularization parameter is increased. Finally, we select the optimal set in the sense of the Bayesian Information Criterion (BIC), favoring models with a high likelihood, while penalizing those with too many endmembers. Results on real data show the interest of the proposed approach.

Spectral unmixing is a central problem in hyperspectral imagery. It is usually assuming a linear mixture model. Solving this inverse problem, however, can be seriously impacted by a wrong estimation of the number of endmembers, a bad estimation of the endmembers themselves, the spectral variability of the endmembers or the presence of nonlinearities. These problems can result in a too large number of retained endmembers. We propose to tackle this problem by introducing a new formulation for robust linear unmixing enhancing sparsity. With a single tuning parameter the optimization leads to a range of behaviors: from the standard linear model (low sparsity) to a hard classification (maximal sparsity : only one endmember is retained per pixel). We solve the proposed new functional using a computationally efficient proximal primal dual method. The experimental study, including both realistic simulated data and real data demonstrates the versatility of the proposed approach.

Hyperspectral image unmixing is a source separation problem whose goal is to identify the signatures of the materials present in the imaged scene (called endmembers), and to estimate their proportions (called abundances) in each pixel. Usually, the contributions of each material are assumed to be perfectly represented by a single spectral signature and to add up in a linear way. However, the main two limitations of this model have been identified as nonlinear mixing phenomena and spectral variability, i.e., the intraclass variability of the materials. The former limitation has been addressed by designing nonlinear mixture models, whereas the second can be dealt with by using (usually linear) space varying models. The typical example is a linear mixing model where the sources can vary from one pixel to the other. In this letter, we show that a recent variability model can also estimate the abundances of nonlinear mixtures to some extent. We make the theoretical connection between nonlinear models and this variability model, and confirm it with experiments on nonlinearly generated synthetic datasets.

We apply social ℓ-norms for the first time to the problem of hyperspectral unmixing while modeling spectral variability. These norms are built with inter-group penalties which are combined in a global intra-group penalization that can enforce selection of entire endmember bundles; this results in the selection of a few representative materials even in the presence of large endmembers bundles capturing each material's variability. We demonstrate improvements quantitatively on synthetic data and qualitatively on real data for three cases of social norms: group, elitist, and a fractional social norm, respectively. We find that the greatest improvements arise from using either the group or fractional flavor.

The intrinsic dimensionality (ID) of multivariate data is a very important concept in spectral unmixing of hyperspectral images. A good estimation of the ID is crucial for a correct retrieval of the number of endmembers (the spectral signatures of macroscopic materials) in the image, for dimensionality reduction or for subspace learning, among others. Recently, some approaches to perform spectral unmixing and superresolution locally have been proposed, which require a local estimation of the number of endmembers to use. However, the role of ID in local regions of hyperspectral images has not been properly addressed. Some important issues when dealing with small regions of hyperspectral data can seriously affect the performance of conventional hyperspectral ID estimators. We show that three factors mainly affect local ID estimation: the number of pixels in the local regions, which has to be high enough for the estimations to be relevant, the number of hyperspectral bands which complicates the estimations if the ambient space has a high dimensionality, and the noise, which can be misinterpreted as a signal when its power is important. Here, we review the hyperspectral ID estimators on the literature for local ID estimation, we show how they behave in a local setting on synthetic and real data sets, and we provide some guidelines to make proper use of these estimators in local approaches.