Trees and the associated shortest-path tree metrics provide a powerful framework for representing hierarchical and combinatorial structures in data. Given an arbitrary metric space, its deviation from a tree metric can be quantified by Gromov’s $\delta$-hyperbolicity. Nonetheless, designing algorithms that bridge an arbitrary metric to its closest tree metric is still a vivid subject of interest, as most common approaches are either heuristical and lack guarantees, or perform moderately well. In this work, we introduce a novel differentiable optimization framework, coined DeltaZero, that solves this problem. Our method leverages a smooth surrogate for Gromov’s $\delta$-hyperbolicity which enables a gradient-based optimization, with a tractable complexity. The corresponding optimization procedure is derived from a problem with better worst case guarantees than existing bounds, and is justified statistically. Experiments on synthetic and real-world datasets demonstrate that our method consistently achieves state-of-the-art distortion.

Abstract In many computer vision applications, datasets often exhibit an underlying taxonomy within the label space. To adhere to this hierarchical structure, hyperbolic spaces have emerged as an effective manifold for representation learning, thanks to their ability to encode hierarchical relationships, with little distortion, even for low-dimensional embeddings. Hyperbolic prototypical learning, where class labels are represented by prototypes, has recently demonstrated strong potential in this setting. However, existing methods generally assume a uniform distribution of prototypes, overlooking the hierarchical organization of labels that may be available for a given task. To better exploit this prior knowledge, we propose a hierarchically informed method for prototype positioning. Our approach leverages the Gromov-Wasserstein distance to align the hierarchical relationships between labels with the initial uniform spherical distribution of prototypes, leading to more structured and semantically meaningful representations. Additionally, within a deep learning framework, we propose an alternative characterization of decision boundaries using horospheres, which are level sets of the Busemann function. Geometrically, horospheres correspond to spheres tangent to the boundary of hyperbolic space at a virtual point analogous to a prototype, which makes them a compliant tool in the prototypical learning context. Accordingly, we define a new horospherical layer that can be adapted to any neural network backbone. This layer is particularly advantageous when the number of prototypes exceeds the number of classes, offering enhanced flexibility to the classifier. Through our experiments, we demonstrate that the combination of proper initialization and optimized prototype positioning significantly enhances baseline performance for image classification on hierarchical datasets. Additionally, we validate our approach in two semantic segmentation tasks, using both image and point cloud datasets, confirming its effectiveness in leveraging hierarchical label structures for improved classification performance.

While many Machine Learning methods have been developed or transposed on Riemannian manifolds to tackle data with known non-Euclidean geometry, Optimal Transport (OT) methods on such spaces have not received much attention. The main OT tool on these spaces is the Wasserstein distance, which suffers from a heavy computational burden. On Euclidean spaces, a popular alternative is the Sliced-Wasserstein distance, which leverages a closed-form solution of the Wasserstein distance in one dimension, but which is not readily available on manifolds. In this work, we derive general constructions of Sliced-Wasserstein distances on Cartan-Hadamard manifolds, Riemannian manifolds with non-positive curvature, which include among others Hyperbolic spaces or the space of Symmetric Positive Definite matrices. Then, we propose different applications such as classification of documents with a suitably learned ground cost on a manifold, and data set comparison on a product manifold. Additionally, we derive non-parametric schemes to minimize these new distances by approximating their Wasserstein gradient flows.

Learning models on one labeled dataset that generalize well on another domain is a difficult task, as several shifts might happen between the data domains. This is notably the case for lidar data, for which models can exhibit large performance discrepancies due for instance to different lidar patterns or changes in acquisition conditions. This paper addresses the corresponding Unsupervised Domain Adaptation (UDA) task for semantic segmentation. To mitigate this problem, we introduce an unsupervised auxiliary task of learning an implicit underlying surface representation simultaneously on source and target data. As both domains share the same latent representation, the model is forced to accommodate discrepancies between the two sources of data. This novel strategy differs from classical minimization of statistical divergences or lidar-specific domain adaptation techniques. Our experiments demonstrate that our method achieves a better performance than the current state of the art, both in real-to-real and synthetic-to-real scenarios.The project repository: github.com/valeoai/SALUDA

Abstract In machine learning and computer graphics, a fundamental task is the approximation of a probability density function through a well‐dispersed collection of samples. Providing a formal metric for measuring the distance between probability measures on general spaces, Optimal Transport (OT) emerges as a pivotal theoretical framework within this context. However, the associated computational burden is prohibitive in most real‐world scenarios. Leveraging the simple structure of OT in 1D, Sliced Optimal Transport (SOT) has appeared as an efficient alternative to generate samples in Euclidean spaces. This paper pushes the boundaries of SOT utilization in computational geometry problems by extending its application to sample densities residing on more diverse mathematical domains, including the spherical space 𝕊d, the hyperbolic plane ℍd, and the real projective plane ℙd. Moreover, it ensures the quality of these samples by achieving a blue noise characteristic, regardless of the dimensionality involved. The robustness of our approach is highlighted through its application to various geometry processing tasks, such as the intrinsic blue noise sampling of meshes, as well as the sampling of directions and rotations. These applications collectively underscore the efficacy of our methodology.

To leverage the large amount of unlabelled data available in remote sensing datasets, self-supervised learning (SSL) methods have recently emerged as an ubiquitous tool to pre-train robust image encoder models from unlabelled images. However, when used in a downstream setting, these models often need to be finetuned for a specific task after their pre-training. This finetuning still requires labelling information in order to train a classifier on top of the encoder while also updating the encoder weights. In this paper, we investigate the specific task of multimodal scene classification where a sample is composed of multiple views from multiple heterogeneous satellite sensors. We propose a method to improve the categorical cross-entropy finetuning process which is often used to specify the model for this downstream task. Our approach, based on the Supervised Contrastive Learning, uses the label information available to train an image encoder in a contrastive manner from multiple modalities while also training the task-specific classifier online. Such a multimodal supervised contrastive loss helps to better align representations from samples coming from multiple sensors but having the same class labels, thus improving the performance of the finetuning process. Our experiments on two public datasets including DFC2020 and Meter-ML with Sentinel-1/Sentinel-2 images show a significant gain over the baseline multimodal cross-entropy loss. All codes and datasets will be made publicly available for reproducibility upon acceptance.

Optimal transport (OT) compares probability distributions by computing a meaningful alignment between their samples. CO-optimal transport (COOT) takes this comparison further by inferring an alignment between features as well. While this approach leads to better alignments and generalizes both OT and Gromov-Wasserstein distances, we provide a theoretical result showing that it is sensitive to outliers that are omnipresent in real-world data. This prompts us to propose unbalanced COOT for which we provably show its robustness to noise in the compared datasets. To the best of our knowledge, this is the first such result for OT methods in incomparable spaces. With this result in hand, we provide empirical evidence of this robustness for the challenging tasks of heterogeneous domain adaptation with and without varying proportions of classes and simultaneous alignment of samples and features across two single-cell measurements.

When dealing with electro or magnetoencephalography records, many supervised prediction tasks are solved by working with covariance matrices to summarize the signals. Learning with these matrices requires the usage of Riemanian geometry to account for their structure. In this paper, we propose a new method to deal with distributions of covariance matrices, and demonstrate its computational efficiency on M/EEG multivariate time series. More specifically, we define a Sliced-Wasserstein distance between measures of symmetric positive definite matrices that comes with strong theoretical guarantees. Then, we take advantage of its properties and kernel methods to apply this discrepancy to brain-age prediction from MEG data, and compare it to state-of-the-art algorithms based on Riemannian geometry. Finally, we show that it is an efficient surrogate to the Wasserstein distance in domain adaptation for Brain Computer Interface applications.

With the current ubiquity of deep learning methods to solve computer vision and remote sensing specific tasks, the need for labelled data is growing constantly. However, in many cases, the annotation process can be long and tedious depending on the expertise needed to perform reliable annotations. In order to alleviate this need for annotations, several self-supervised methods have recently been proposed in the literature. The core principle behind these methods is to learn an image encoder using solely unlabelled data samples. In earth observation, there are opportunities to exploit domain-specific remote sensing image data in order to improve these methods. Specifically, by leveraging the geographical position associated with each image, it is possible to cross reference a location captured from multiple sensors, leading to multiple views of the same locations. In this paper, we briefly review the core principles behind so-called joint-embeddings methods and investigate the usage of multiple remote sensing modalities in self-supervised pre-training. We evaluate the final performance of the resulting encoders on the task of methane source classification.

Many approaches in machine learning rely on a weighted graph to encode the similarities between samples in a dataset. Entropic affinities (EAs), which are notably used in the popular Dimensionality Reduction (DR) algorithm t-SNE, are particular instances of such graphs. To ensure robustness to heterogeneous sampling densities, EAs assign a kernel bandwidth parameter to every sample in such a way that the entropy of each row in the affinity matrix is kept constant at a specific value, whose exponential is known as perplexity. EAs are inherently asymmetric and row-wise stochastic, but they are used in DR approaches after undergoing heuristic symmetrization methods that violate both the row-wise constant entropy and stochasticity properties. In this work, we uncover a novel characterization of EA as an optimal transport problem, allowing a natural symmetrization that can be computed efficiently using dual ascent. The corresponding novel affinity matrix derives advantages from symmetric doubly stochastic normalization in terms of clustering performance, while also effectively controlling the entropy of each row thus making it particularly robust to varying noise levels. Following, we present a new DR algorithm, SNEkhorn, that leverages this new affinity matrix. We show its clear superiority to state-of-the-art approaches with several indicators on both synthetic and real-world datasets.

Wasserstein distance (WD) and the associated optimal transport plan have been proven useful in many applications where probability measures are at stake. In this paper, we propose a new proxy of the squared WD, coined $\textnormal{min-SWGG}$, that is based on the transport map induced by an optimal one-dimensional projection of the two input distributions. We draw connections between $\textnormal{min-SWGG}$, and Wasserstein generalized geodesics in which the pivot measure is supported on a line. We notably provide a new closed form for the exact Wasserstein distance in the particular case of one of the distributions supported on a line allowing us to derive a fast computational scheme that is amenable to gradient descent optimization. We show that $\textnormal{min-SWGG}$, is an upper bound of WD and that it has a complexity similar to as Sliced-Wasserstein, with the additional feature of providing an associated transport plan. We also investigate some theoretical properties such as metricity, weak convergence, computational and topological properties. Empirical evidences support the benefits of $\textnormal{min-SWGG}$, in various contexts, from gradient flows, shape matching and image colorization, among others.

Individual brains vary in both anatomy and functional organization, even within a given species. Inter-individual variability is a major impediment when trying to draw generalizable conclusions from neuroimaging data collected on groups of subjects. Current co-registration procedures rely on limited data, and thus lead to very coarse inter-subject alignments. In this work, we present a novel method for inter-subject alignment based on Optimal Transport, denoted as Fused Unbalanced Gromov Wasserstein (FUGW). The method aligns two cortical surfaces based on the similarity of their functional signatures in response to a variety of stimuli, while penalizing large deformations of individual topographic organization.We demonstrate that FUGW is suited for whole-brain landmark-free alignment. The unbalanced feature allows to deal with the fact that functional areas vary in size across subjects. Results show that FUGW alignment significantly increases between-subject correlation of activity during new independent fMRI tasks and runs, and leads to more precise maps of fMRI results at the group level.

Current Graph Neural Networks (GNN) architectures generally rely on two important components: node features embedding through message passing, and aggregation with a specialized form of pooling. The structural (or topological) information is implicitly taken into account in these two steps. We propose in this work a novel point of view, which places distances to some learnable graph templates at the core of the graph representation. This distance embedding is constructed thanks to an optimal transport distance: the Fused Gromov-Wasserstein (FGW) distance, which encodes simultaneously feature and structure dissimilarities by solving a soft graph-matching problem. We postulate that the vector of FGW distances to a set of template graphs has a strong discriminative power, which is then fed to a non-linear classifier for final predictions. Distance embedding can be seen as a new layer, and can leverage on existing message passing techniques to promote sensible feature representations. Interestingly enough, in our work the optimal set of template graphs is also learnt in an end-to-end fashion by differentiating through this layer. After describing the corresponding learning procedure, we empirically validate our claim on several synthetic and real life graph classification datasets, where our method is competitive or surpasses kernel and GNN state-of-the-art approaches. We complete our experiments by an ablation study and a sensitivity analysis to parameters.

Noisy labels often occur in vision datasets, especially when they are obtained from crowdsourcing or Web scraping. We propose a new regularization method, which enables learning robust classifiers in presence of noisy data. To achieve this goal, we propose a new adversarial regularization scheme based on the Wasserstein distance. Using this distance allows taking into account specific relations between classes by leveraging the geometric properties of the labels space. Our Wasserstein Adversarial Regularization (WAR) encodes a selective regularization, which promotes smoothness of the classifier between some classes, while preserving sufficient complexity of the decision boundary between others. We first discuss how and why adversarial regularization can be used in the context of noise and then show the effectiveness of our method on five datasets corrupted with noisy labels: in both benchmarks and real datasets, WAR outperforms the state-of-the-art competitors.

Over the last years, remote sensing image (RSI) analysis has started resorting to using deep neural networks to solve most of the commonly faced problems, such as detection, land cover classification, or segmentation. As far as critical decision-making can be based upon the results of RSI analysis, it is important to clearly identify and understand potential security threats occurring in those machine learning algorithms. Notably, it has recently been found that neural networks are particularly sensitive to carefully designed attacks, generally crafted given the full knowledge of the considered deep network. In this article, we consider the more realistic but challenging case where one wants to generate such attacks in the case of a black-box neural network. In this case, only the prediction score of the network is accessible, on a specific dataset. Examples that lure away the network’s prediction, while being perceptually similar to real images, are called natural or unrestricted adversarial examples. We present an original method to generate such examples based on a variant of the Wasserstein generative adversarial network. We demonstrate its effectiveness on natural adversarial hyperspectral image generation and image modification for fooling a state-of-the-art detector. Among others, we also conduct a perceptual evaluation with human annotators to better assess the effectiveness of the proposed method. Our code is available for the community:https://github.com/PythonOT/ARWGAN.

Optimal transport distances have found many applications in machine learning for their capacity to compare non-parametric probability distributions. Yet their algorithmic complexity generally prevents their direct use on large scale datasets. Among the possible strategies to alleviate this issue, practitioners can rely on computing estimates of these distances over subsets of data, i.e. minibatches. While computationally appealing, we highlight in this paper some limits of this strategy, arguing it can lead to undesirable smoothing effects. As an alternative, we suggest that the same minibatch strategy coupled with unbalanced optimal transport can yield more robust behaviors. We discuss the associated theoretical properties, such as unbiased estimators, existence of gradients and concentration bounds. Our experimental study shows that in challenging problems associated to domain adaptation, the use of unbalanced optimal transport leads to significantly better results, competing with or surpassing recent baselines.

Dictionary learning is a key tool for representation learning, that explains the data as linear combination of few basic elements. Yet, this analysis is not amenable in the context of graph learning, as graphs usually belong to different metric spaces. We fill this gap by proposing a new online Graph Dictionary Learning approach, which uses the Gromov Wasserstein divergence for the data fitting term. In our work, graphs are encoded through their nodes’ pairwise relations and modeled as convex combination of graph atoms, i.e. dictionary elements, estimated thanks to an online stochastic algorithm, which operates on a dataset of unregistered graphs with potentially different number of nodes. Our approach naturally extends to labeled graphs, and is completed by a novel upper bound that can be used as a fast approximation of Gromov Wasserstein in the embedding space. We provide numerical evidences showing the interest of our approach for unsupervised embedding of graph datasets and for online graph subspace estimation and tracking.

Optimal transport has recently been reintroduced to the machine learning community thanks in part to novel efficient optimization procedures allowing for medium to large scale applications. We propose a Python toolbox that implements several key optimal transport ideas for the machine learning community. The toolbox contains implementations of a number of founding works of OT for machine learning such as Sinkhorn algorithm and Wasserstein barycenters, but also provides generic solvers that can be used for conducting novel fundamental research. This toolbox, named POT for Python Optimal Transport, is open source with an MIT license.

Optimal transport distances are powerful tools to compare probability distributions and have found many applications in machine learning. Yet their algorithmic complexity prevents their direct use on large scale datasets. To overcome this challenge, practitioners compute these distances on minibatches i.e., they average the outcome of several smaller optimal transport problems. We propose in this paper an analysis of this practice, which effects are not well understood so far. We notably argue that it is equivalent to an implicit regularization of the original problem, with appealing properties such as unbiased estimators, gradients and a concentration bound around the expectation, but also with defects such as loss of distance property. Along with this theoretical analysis, we also conduct empirical experiments on gradient flows, GANs or color transfer that highlight the practical interest of this strategy.

In the field of remote sensing and more specifically in Earth Observation, new data are available every day, coming from different sensors. Leveraging on those data in classification tasks comes at the price of intense labelling tasks that are not realistic in operational settings. While domain adaptation could be useful to counterbalance this problem, most of the usual methods assume that the data to adapt are comparable (they belong to the same metric space), which is not the case when multiple sensors are at stake. Heterogeneous domain adaptation methods are a particular solution to this problem. We present a novel method to deal with such cases, based on a modified cycleGAN version that incorporates classification losses and a metric space alignment term. We demonstrate its power on a land use classification tasks, with images from both Google Earth and Sentinel-2.

Optimal transport (OT) is a powerful geometric and probabilistic tool for finding correspondences and measuring similarity between two distributions. Yet, its original formulation relies on the existence of a cost function between the samples of the two distributions, which makes it impractical when they are supported on different spaces. To circumvent this limitation, we propose a novel OT problem, named COOT for CO-Optimal Transport, that simultaneously optimizes two transport maps between both samples and features, contrary to other approaches that either discard the individual features by focusing on pairwise distances between samples or need to model explicitly the relations between them. We provide a thorough theoretical analysis of our problem, establish its rich connections with other OT-based distances and demonstrate its versatility with two machine learning applications in heterogeneous domain adaptation and co-clustering/data summarization, where COOT leads to performance improvements over the state-of-the-art methods.

In this paper, we tackle the problem of reducing discrepancies between multiple domains, i.e. multi-source domain adaptation, and consider it under the target shift assumption: in all domains we aim to solve a classification problem with the same output classes, but with different labels proportions. This problem, generally ignored in the vast majority of domain adaptation papers, is nevertheless critical in real-world applications, and we theoretically show its impact on the success of the adaptation. Our proposed method is based on optimal transport, a theory that has been successfully used to tackle adaptation problems in machine learning. The introduced approach, Joint Class Proportion and Optimal Transport (JCPOT), performs multi-source adaptation and target shift correction simultaneously by learning the class probabilities of the unlabeled target sample and the coupling allowing to align two (or more) probability distributions. Experiments on both synthetic and real-world data (satellite image pixel classification) task show the superiority of the proposed method over the state-of-the-art.

This work considers the problem of computing distances between structured objects such as undirected graphs, seen as probability distributions in a specific metric space. We consider a new transportation distance ( i.e. that minimizes a total cost of transporting probability masses) that unveils the geometric nature of the structured objects space. Unlike Wasserstein or Gromov-Wasserstein metrics that focus solely and respectively on features (by considering a metric in the feature space) or structure (by seeing structure as a metric space), our new distance exploits jointly both information, and is consequently called Fused Gromov-Wasserstein (FGW). After discussing its properties and computational aspects, we show results on a graph classification task, where our method outperforms both graph kernels and deep graph convolutional networks. Exploiting further on the metric properties of FGW, interesting geometric objects such as Fr{é}chet means or barycenters of graphs are illustrated and discussed in a clustering context.

Recently used in various machine learning contexts, the Gromov-Wasserstein distance (GW) allows for comparing distributions whose supports do not necessarily lie in the same metric space. However, this Optimal Transport (OT) distance requires solving a complex non convex quadratic program which is most of the time very costly both in time and memory. Contrary to GW, the Wasserstein distance (W) enjoys several properties ({\em e.g.} duality) that permit large scale optimization. Among those, the solution of W on the real line, that only requires sorting discrete samples in 1D, allows defining the Sliced Wasserstein (SW) distance. This paper proposes a new divergence based on GW akin to SW. We first derive a closed form for GW when dealing with 1D distributions, based on a new result for the related quadratic assignment problem. We then define a novel OT discrepancy that can deal with large scale distributions via a slicing approach and we show how it relates to the GW distance while being $O(n\log(n))$ to compute. We illustrate the behavior of this so called Sliced Gromov-Wasserstein (SGW) discrepancy in experiments where we demonstrate its ability to tackle similar problems as GW while being several order of magnitudes faster to compute.

Abstract In this paper, we propose a whole animation pipeline for data‐driven character animation. Considering that the traditional animation pipeline, including skeleton reconstruction from markers, rigging, and retargeting, is subject to potential loss of information and precision, our objective is to control in real time articulated meshes from a low number of positional constraints. Our method is based on an efficient deformation technique that integrates into a volumetric control structure the high‐resolution mesh, the skeleton, and relevant marker locations. An iterative optimization method, which preserves both geometric characteristics, segment lengths and joint limits, is applied to this structure. We show the ability of our system to interactively animate and deform high‐resolution models from few positional constraints while keeping all the details of the movement.

Repeated animal censuses are crucial for wildlife parks to ensure ecological equilibriums. They are increasingly conducted using images generated by Unmanned Aerial Vehicles (UAVs), often coupled to semi-automatic object detection methods. Such methods have shown great progress also thanks to the employment of Convolutional Neural Networks (CNNs), but even the best models trained on the data acquired in one year struggle predicting animal abundances in subsequent campaigns due to the inherent shift between the datasets. In this paper we adapt a CNN-based animal detector to a follow-up UAV dataset by employing an unsupervised domain adaptation method based on Optimal Transport. We show how to infer updated labels from the source dataset by means of an ensemble of bootstraps. Our method increases the precision compared to the unmodified CNN, while not requiring additional labels from the target set.

Kernel based feature extraction method overcomes the curse of dimensionality and captures the non-linearities present in the data. However, these methods are not scalable with large number of pixels found with hyperspectral images. Thus, a small subset of pixels are randomly selected to make the solution of kernel based methods tractable. In this paper, we propose scalable nonlinear component analysis for dimensionality reduction of hyperspectral images. The proposed method relies on the randomized feature maps to capture the non-linearities between the variables in the hyperspectral data. Experiments conducted with three hyperspectral datasets show that our proposed method has provided better quality components and outperformed the state-of-the-art in terms of classification performance.

The fusion of hyperspectral and multispectral images is a crucial task nowadays for it allows the extraction of relevant information from the fused image. Fusion consists of the combination of the spectral information of the hypespectral image (h) and the spatial information of the multispectral image (m). The fused image (f) has both good spatial and spectral information. In this paper we suggest a new hyperspectral and multispectral image (h-m) fusion approach based on Optimal Transport (OT) which highlights the idea of energy transfer from the starting images m and h to the resulting image f. The simulations show that the suggested method is effective and compares competitively with other state-of-the-art methods.

This paper deals with the unsupervised domain adaptation problem, where one wants to estimate a prediction function $f$ in a given target domain without any labeled sample by exploiting the knowledge available from a source domain where labels are known. Our work makes the following assumption: there exists a non-linear transformation between the joint feature/label space distributions of the two domain $\ps$ and $\pt$. We propose a solution of this problem with optimal transport, that allows to recover an estimated target $\pt^f=(X,f(X))$ by optimizing simultaneously the optimal coupling and $f$. We show that our method corresponds to the minimization of a bound on the target error, and provide an efficient algorithmic solution, for which convergence is proved. The versatility of our approach, both in terms of class of hypothesis or loss functions is demonstrated with real world classification and regression problems, for which we reach or surpass state-of-the-art results.

Domain adaptation is one of the most challenging tasks of modern data analytics. If the adaptation is done correctly, models built on a specific data representation become more robust when confronted to data depicting the same classes, but described by another observation system. Among the many strategies proposed, finding domain-invariant representations has shown excellent properties, in particular since it allows to train a unique classifier effective in all domains. In this paper, we propose a regularized unsupervised optimal transportation model to perform the alignment of the representations in the source and target domains. We learn a transportation plan matching both PDFs, which constrains labeled samples of the same class in the source domain to remain close during transport. This way, we exploit at the same time the labeled samples in the source and the distributions observed in both domains. Experiments on toy and challenging real visual adaptation examples show the interest of the method, that consistently outperforms state of the art approaches. In addition, numerical experiments show that our approach leads to better performances on domain invariant deep learning features and can be easily adapted to the semi-supervised case where few labeled samples are available in the target domain.

Designing an effective criterion to select a subset of features is a challenging problem for hyperspectral image classification. In this paper, we develop a feature selection method to select a subset of class discriminant features for hyperspectral image classification. First, we propose a new class separability measure based on the surrogate kernel and Hilbert Schmidt independence criterion in the reproducing kernel Hilbert space. Second, we employ the proposed class separability measure as an objective function and we model the feature selection problem as a continuous optimization problem using LASSO optimization framework. The combination of the class separability measure and the LASSO model allows selecting the subset of features that increases the class separability information and also avoids a computationally intensive subset search strategy. Experiments conducted with three hyperspectral data sets and different experimental settings show that our proposed method increases the classification accuracy and outperforms the state-of-the-art methods.

One of the main objective of data fusion is the integration of several acquisition of the same physical object, in order to build a new consistent representation that embeds all the information from the different modalities. In this paper, we propose the use of optimal transport theory as a powerful mean of establishing correspondences between the modalities. After reviewing important properties and computational aspects, we showcase its application to three remote sensing fusion problems: domain adaptation, time series averaging and change detection in LIDAR data.

Generating accurate and robust classification maps from hyperspectral imagery (HSI) depends on the choice of the classifiers and input data sources. Choosing the appropriate classifier for a problem at hand is a tedious task. Multiple classifier system (MCS) combines the relative merits of various classifiers to generate robust classification maps. However, the presence of inaccurate classifiers may degrade the classification performance of MCS. In this paper, we propose an unsupervised classifier selection strategy to select an appropriate subset of accurate classifiers for the multiple classifier combination from a large pool of classifiers. The experimental results with two HSI show that the proposed classifier selection method overcomes the impact of inaccurate classifiers and significantly increases the classification accuracy.

Many spectral unmixing methods rely on the non-negative decomposition of spectral data onto a dictionary of spectral templates. In particular, state-of-the-art music transcription systems decompose the spectrogram of the input signal onto a dictionary of representative note spectra. The typical measures of fit used to quantify the adequacy of the decomposition compare the data and template entries frequency-wise. As such, small displacements of energy from a frequency bin to another as well as variations of timber can disproportionally harm the fit. We address these issues by means of optimal transportation and propose a new measure of fit that treats the frequency distributions of energy holistically as opposed to frequency-wise. Building on the harmonic nature of sound, the new measure is invariant to shifts of energy to harmonically-related frequencies, as well as to small and local displacements of energy. Equipped with this new measure of fit, the dictionary of note templates can be considerably simplified to a set of Dirac vectors located at the target fundamental frequencies (musical pitch values). This in turns gives ground to a very fast and simple decomposition algorithm that achieves state-of-the-art performance on real musical data.

We are interested in the computation of the transport map of an Optimal Transport problem. Most of the computational approaches of Optimal Transport use the Kantorovich relaxation of the problem to learn a probabilistic coupling $\mgamma$ but do not address the problem of learning the underlying transport map $\funcT$ linked to the original Monge problem. Consequently, it lowers the potential usage of such methods in contexts where out-of-samples computations are mandatory. In this paper we propose a new way to jointly learn the coupling and an approximation of the transport map. We use a jointly convex formulation which can be efficiently optimized. Additionally, jointly learning the coupling and the transport map allows to smooth the result of the Optimal Transport and generalize it to out-of-samples examples. Empirically, we show the interest and the relevance of our method in two tasks: domain adaptation and image editing.

Hyperspectral (HS) images are commonly used in the context of planetary exploration, particularly for the analysis of the composition of planets. As several instruments have been sent throughout the Solar System, a huge quantity of data is getting available for the research community. Among classical problems in the analysis of HS images, a crucial one is unsupervised nonlinear spectral unmixing, which aims at estimating the spectral signatures of elementary materials and determining their relative contribution at a subpixel level. While the unmixing problem is well studied for Earth observation, some of the traditional problems encountered with Earth images are somehow magnified in planetary exploration. Among them, large image sizes, strong nonlinearities in the mixing (often different from those found in the Earth images), and the presence of anomalies are usually impairing the unmixing algorithms. This paper presents a new method that scales favorably with the problem posed by this analysis. It performs an unsupervised unmixing jointly with anomaly-detection capacities and has a global linear complexity. Nonlinearities are handled by decomposing the HS data on an overcomplete set of spectra, combined with a specific sparse projection, which guarantees the interpretability of the analysis. A theoretical study is proposed on synthetic data sets, and results are presented over the challenging 4-Vesta asteroid data set.

Despite the popularity of mathematical morphology with remote sensing image analysis, its application to hyperspectral data remains problematic. The issue stems from the need to impose a complete lattice structure on the multi-dimensional pixel value space, that requires a vector ordering. In this article, we introduce such a supervised ordering relation, which conversely to its alternatives, has been designed to be image-specific and exploits the spectral purity of pixels. The practical interest of the resulting multivariate morphological operators is validated through classification experiments where it achieves state-of-the-art performance.

Adapting a model to changes in the data distribution is a relevant problem in machine learning and pattern recognition since such changes degrade the performances of classifiers trained on undistorted samples. This paper tackles the problem of domain adaptation in the context of hyper spectral satellite image analysis. We propose a new correlated correspondence algorithm based on network analysis. The algorithm finds a matching between two distributions, which preserves the geometrical and topological information of the corresponding graphs. We evaluate the performance of the algorithm on a shadow compensation problem in hyper spectral image analysis: the land use classification obtained with the compensated data is improved.

ABSTRACT Motion editing requires the preservation of spatial and temporal information of the motion. During editing, this information should be preserved at best. We propose a new representation of the motion based on the Laplacian expression of a 3D+t graph: the set of connected graphs given by the skeleton over time. Through this Laplacian representation of the motion, we propose an application that allows an easy and interactive editing, correction, or retargeting of a motion. The new created motion is the result of the combination of two minimizations, linear and non‐linear: the first penalizes the difference of energy between the Laplacian coordinates from an animation to the desired one. The other one preserves the length of segments. Using several examples, we demonstrate the benefits of our method and in particularly the preservation of the spatiotemporal properties of the motion in an interactive context. Copyright © 2013 John Wiley & Sons, Ltd.

A new classification technique, PerTurbo, has been investigated in the context on hyperspectral remote sensing images context. In this framework, each class is characterised by its Laplace-Beltrami operator, then approximated by the spectrum of K(S), whose terms are derived from the Gaussian kernel. The method is very simple, easy to implement and involves few parameters to tune. It also allows the definition of a simple multi-class strategy, and, as a class-wise classification method, the addition of a new class does not requires the re-training of the pre-existing class models. We conducted experiments on two datasets: results for Pavia Centre dataset are encouraging, while results obtained on Pavia University show that SVM clearly outperforms PerTurbo. Nevertheless, we believe that this difference comes from a bad parametrization of the algorithm (for which we used a rule of thumb, contrarily to the SVM procedure which was fully optimized). Hence, a systematic search for the optimal value of the parameter would improve the results. Moreover, there are several other possible improvements coming from the fields of regularization methods of dimensionality reduction techniques which let us think that this first experiment is promising. In the near future, we also plan to investigate rules leading to a better choice of s. We are also interested in studying the behavior of PerTurbo when the classes are heterogeneous with only few training samples available, or when the classes in the training set are highly unbalanced.

In this article we present a multichannel animation system for producing utterances signed in French Sign Language (LSF) by a virtual character. The main challenges of such a system are simultaneously capturing data for the entire body, including the movements of the torso, hands, and face, and developing a data-driven animation engine that takes into account the expressive characteristics of signed languages. Our approach consists of decomposing motion along different channels, representing the body parts that correspond to the linguistic components of signed languages. We show the ability of this animation system to create novel utterances in LSF, and present an evaluation by target users which highlights the importance of the respective body parts in the production of signs. We validate our framework by testing the believability and intelligibility of our virtual signer.

Abstract In a context of interactive applications, adapting motion capture data to new situations or producing variants of them are known as non trivial tasks. We propose an original method that produces motions that preserve the statistical properties of a reference motion while ensuring some constraints. This method uses principles of conditional stochastic simulation to achieve this goal. Notably, a new real time algorithm, performing sequentially and producing the desired motion is introduced. Possible applications of our method are numerous and several examples are given, along with results. Copyright © 2010 John Wiley & Sons, Ltd.

Abstract Due to the growing need for large quantities of human animation data in the entertainment industry, it has become a necessity to compress motion capture sequences in order to ease their storage and transmission. We present a novel, lossy compression method for human motion data that exploits both temporal and spatial coherence. Given one motion, we first approximate the poses manifold using Principal Geodesics Analysis (PGA) in the configuration space of the skeleton. We then search this approximate manifold for poses matching end‐effectors constraints using an iterative minimization algorithm that allows for real‐time, data‐driven inverse kinematics. The compression is achieved by only storing the approximate manifold parametrization along with the end‐effectors and root joint trajectories, also compressed, in the output data. We recover poses using the IK algorithm given the end‐effectors trajectories. Our experimental results show that considerable compression rates can be obtained using our method, with few reconstruction and perceptual errors.

Abstract In this paper a new and original technique to animate a crowd of human beings is presented. Following the success of data‐driven animation models (such as motion capture) in the context of articulated figures control, we propose to derivate a similar type of approach for crowd motions. In our framework, the motion of the crowds are represented as a time series of velocity fields estimated from a video of a real crowd. This time series is used as an input of a simple animation model that ‘advect’ people along this time‐varying flow. We demonstrate the power of our technique on both synthetic and real examples of crowd videos. We also introduce the notions of crowd motion editing and present possible extensions to our work. Copyright © 2007 John Wiley & Sons, Ltd.

This project aims at the creation of a virtual humanoid endowed with expressive gestures. More specifically, we focus our attention on expressiveness (what type of gesture: fluidity, tension, anger) and on its semantic representations. Our approach relies on a data-driven animation scheme. From motion data captured thanks to an optical system and data gloves, we try to extract significant features of communicative gestures, and to re-synthesize them afterward with style variation. The proposed model is applied to the generation of a set of French sign language (FSL) gestures. Within this framework, a database involving the whole body, hands motion and facial expressions has been built The analysis of this database makes possible information retrieval about the semantics as well as the execution style of FSL gestures. These characteristics are integrated in gesture synthesis models qualitatively evaluated by their intelligibility and the realism of the produced animations.

Abstract In this paper we address the problem of temporal alignment applied to capture communicative gestures conveying different styles. We propose a representation space that may be considered asrobustto the spatial variability induced by style. By extending a multilevel dynamic time warping algorithm, we show how this extension can fulfil the goals of time correspondence between gesture sequences while preventing jerkiness introduced by standard time warping methods. Copyright © 2006 John Wiley & Sons, Ltd.

Crowd animation and simulation have been widely studied over the last decade for many purposes: populating collaborative virtual environments, entertainment and special effects industry and finally simulating behaviors and motion of people in emergency situations for safety systems. This last topic is addressed in this paper. We propose an original enhancement of a well known physics-based animation model which allows to consider influence of gaseous phenomena such as smoke or toxic gases in the behavior of the crowd. In order to get real time performances we also propose an implementation of this framework on modern graphics hardware, which allows to simulate crowds of thousands individuals at interactive framerate.

We present in this paper a new and original application for saliency maps, intending to simulate the visual perception of a synthetic actor. Within computer graphics field, simulating virtual humans has become a challenging task. Animating such an autonomous actor within a virtual environment requires most of the time in modeling of a perception-decision-action cycle. To model a part of the perception process, we have designed a new model of saliency map, based on geometric and depth information, allowing our synthetic humanoid to perceive its environment in a biologically plausible way.

We present in this paper an original model to simulate visual perception based on the detection of salient features. Salient features correspond to the maximum of conspicuity in a static image or in a sequence of images. We intend to use this information to provide visually interesting targets that can be used in multiple contexts. The extraction process of such an information is performed through multiple steps that correspond to different features locally encoding the conspicuity of a spatial location. We mainly use for those features spatial orientation and motion information, but other types of feature could be used as well. Two kinds of application are presented in this paper: video surveillance application and simulation of the visual perception of a synthetic actor.

We present an animation technique to control a humanoid in a virtual environment. The automatic generation of humanoid motion is difficult and needs to be simplified. The solution proposed in this paper consists of controlling humanoid motion through the image it "perceives": through-the-eyes control. The considered approach is based on the visual servoing concept. It allows the automatic generation of (virtual) camera motion by simply specifying the task in the image space. This approach is suited to highly reactive contexts (video games, virtual reality). We also discuss the integration of such techniques in a more complex behavioral system.

Presents an application for image-based visual servoing: computer graphics animation. Indeed, the control of a virtual camera in a virtual environment is not a trivial problem and usually requires skilled operators. Visual servoing, a now well known technique in robotics and computer vision, consists in positioning a camera according to the informations perceived in the images. Using this method within a computer graphics context leads to a very intuitive approach of animation. Furthermore, in that case a full knowledge about the scene is available. It allows us to easily introduce constraints within the control law in order to react automatically to modifications of the environment. We apply this approach in two different contexts: highly reactive applications (virtual reality, video games) and the control of humanoid avatars.