We present Basis Restricted Elastic Shape Analysis (BaRe-ESA), a novel Riemannian framework for human body scan representation, interpolation and extrapolation. BaRe-ESA operates directly on unregistered meshes, i.e., without the need to establish prior point to point correspondences or to assume a consistent mesh structure. Our method relies on a latent space representation, which is equipped with a Riemannian (non-Euclidean) metric associated to an invariant higher-order metric on the space of surfaces. Experimental results on the FAUST and DFAUST datasets show that BaRe-ESA brings significant improvements with respect to previous solutions in terms of shape registration, interpolation and extrapolation. The efficiency and strength of our model is further demonstrated in applications such as motion transfer and random generation of body shape and pose.

This paper introduces a set of numerical methods for Riemannian shape analysis of 3D surfaces within the setting of invariant (elastic) second-order Sobolev metrics. More specifically, we address the computation of geodesics and geodesic distances between parametrized or unparametrized immersed surfaces represented as 3D meshes. Building on this, we develop tools for the statistical shape analysis of sets of surfaces, including methods for estimating Karcher means and performing tangent PCA on shape populations, and for computing parallel transport along paths of surfaces. Our proposed approach fundamentally relies on a relaxed variational formulation for the geodesic matching problem via the use of varifold fidelity terms, which enable us to enforce reparametrization independence when computing geodesics between unparametrized surfaces, while also yielding versatile algorithms that allow us to compare surfaces with varying sampling or mesh structures. Importantly, we demonstrate how our relaxed variational framework can be extended to tackle partially observed data. The different benefits of our numerical pipeline are illustrated over various examples, synthetic and real. Supplementary Information: The online version contains supplementary material available at 10.1007/s11263-022-01743-0.

This paper introduces a new framework for non-parallel emotion conversion in speech. Our framework is based on two key contributions. First, we propose a stochastic version of the popular Cycle-GAN model. Our modified loss function introduces a Kullback–Leibler (KL) divergence term that aligns the source and targetdata distributionslearned by the generators, thus overcoming the limitations of sample-wise generation. By using a variational approximation to this stochastic loss function, we show that our KL divergence term can be implemented via a paired density discriminator. We term this new architecture a variational Cycle-GAN (VCGAN). Second, we model the prosodic features of target emotion as a smooth and learnable deformation of the source prosodic features. This approach provides implicit regularization that offers key advantages in terms of better range alignment to unseen and out-of-distribution speakers. We conduct rigorous experiments and comparative studies to demonstrate that our proposed framework is fairly robust with high performance against several state-of-the-art baselines.

This paper introduces a new extension of Riemannian elastic curve matching to a general class of geometric structures, which we call (weighted) shape graphs, that allows for shape registration with partial matching constraints and topological inconsistencies. Weighted shape graphs are the union of an arbitrary number of component curves in Euclidean space with potential connectivity constraints between some of their boundary points, together with a weight function defined on each component curve. The framework of higher-order invariant Sobolev metrics is particularly well suited for constructing notions of distances and geodesics between unparametrized curves. The main difficulty in adapting this framework to the setting of shape graphs is the absence of topological consistency, which typically results in an inadequate search for an exact matching between two shape graphs. We overcome this hurdle by defining an inexact variational formulation of the matching problem between (weighted) shape graphs of any underlying topology, relying on the convenient measure representation given by varifolds to relax the exact matching constraint. We then prove the existence of minimizers to this variational problem when we choose Sobolev metrics of sufficient regularity and a total variation (TV) regularization on the weight function. We propose a numerical optimization approach which adapts the smoothed fast iterative shrinkage-thresholding algorithm (SFISTA) to deal with $TV$ norm minimization and allows us to reduce the matching problem to solving a sequence of smooth unconstrained minimization problems. We finally illustrate the capabilities of our new model through several examples showcasing its ability to tackle partially observed and topologically varying data.

We propose a novel method for emotion conversion in speech based on a chained encoder-decoder-predictor neural network architecture. The encoder constructs a latent embedding of the fundamental frequency (F0) contour and the spectrum, which we regularize using the Large Diffeomorphic Metric Mapping (LDDMM) registration framework. The decoder uses this embedding to predict the modified F0 contour in a target emotional class. Finally, the predictor uses the original spectrum and the modified F0 contour to generate a corresponding target spectrum. Our joint objective function simultaneously optimizes the parameters of three model blocks. We show that our method outperforms the existing state-of-the-art approaches on both, the saliency of emotion conversion and the quality of resynthesized speech. In addition, the LDDMM regularization allows our model to convert phrases that were not present in training, thus providing evidence for out-of-sample generalization.

Sparse dictionary learning is a popular method for representing signals as linear combinations of a few elements from a dictionary that is learned from the data. In the classical setting, signals are represented as vectors, and the dictionary learning problem is posed as a matrix factorization problem where the data matrix is approximately factorized into a dictionary matrix and a sparse matrix of coefficients. However, in many applications in computer vision and medical imaging, signals are better represented as matrices or tensors (e.g., images or videos). In such cases, instead of learning a large-scale dictionary tensor, it may be beneficial to exploit the multidimensional structure of the data to learn a more compact representation. One such approach is separable dictionary learning, where one learns separate dictionaries for different dimensions of the data (e.g., spatial and temporal dimensions of a video). However, while there has been significant recent work on separable dictionary learning, typical formulations involve solving a nonconvex optimization problem; thus, guaranteeing global optimality remains a challenge. In this work, we propose a framework that builds upon recent developments in matrix factorization to provide theoretical and numerical guarantees of global optimality for separable dictionary learning. Specifically, we prove that local minima are guaranteed to be global when some dictionary atoms and the corresponding coefficients are zero. We also propose an algorithm to find such a globally optimal solution, which alternates between following local descent steps and checking a certificate for global optimality. We illustrate our approach on diffusion magnetic resonance imaging (dMRI) data, a medical imaging modality that measures water diffusion along multiple angular directions in every voxel of a magnetic resonance imaging volume. State-of-the-art methods in dMRI either learn dictionaries only for the angular domain of the signals or in some cases learn spatial and angular dictionaries independently. In this work, we apply the proposed separable dictionary learning framework to learn spatial and angular dMRI dictionaries jointly and provide preliminary validation on denoising phantom and real dMRI brain data.

This paper introduces a general setting for the construction of data fidelity metrics between oriented or non-oriented geometric shapes like curves, curve sets or surfaces. These metrics are based on the representation of shapes as distributions of their local tangent or normal vectors and the definition of reproducing kernels on these spaces. The construction, that combines in one common setting and extends the previous frameworks of currents and varifolds, provides a very large class of kernel metrics which can be easily computed without requiring any kind of parametrization of shapes and which are smooth enough to give robustness to certain imperfections that could result e.g. from bad segmentation. We then give a sense, with synthetic examples, of the versatility and potentialities of such metrics when used in various problems like shape comparison, clustering and diffeomorphic registration.

High angular resolution diffusion imaging (HARDI) can produce better estimates of fiber orientation and richer sets of features for disease classification than diffusion tensor imaging. However, existing HARDI reconstruction algorithms require a large number of gradient directions, making the acquisition time too long to be clinically viable. State-of-the-art compressed sensing methods can reduce the number of measurements needed for accurate reconstruction by exploiting angular sparsity at each voxel, but the global sparsity level is therefore bounded below by the number of voxels. In this work, we aim to find a significantly sparser representation of HARDI by exploiting redundancies in both the spatial and angular domains jointly with a global HARDI basis. However, this leads to a massive global optimization problem over the whole brain which cannot be solved using existing sparse coding methods. We present a novel Kronecker extension to ADMM that exploits the separable spatial-angular structure of HARDI data to efficiently find a globally sparse reconstruction. We validate our method on phantom and real HARDI brain data by showing that we can achieve accurate reconstructions with a global sparsity level corresponding to less then one atom per voxel, surpassing the absolute limit of the state-of-the-art. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In this paper, we address the problem of orientation that naturally arises when representing shapes such as curves or surfaces as currents. In the field of computational anatomy, the framework of currents has indeed proved very efficient in modeling a wide variety of shapes. However, in such approaches, orientation of shapes is a fundamental issue that can lead to several drawbacks in treating certain kinds of datasets. More specifically, problems occur with structures like acute pikes because of canceling effects of currents or with data that consists in many disconnected pieces like fiber bundles for which currents require a consistent orientation of all pieces. As a promising alternative to currents, varifolds, introduced in the context of geometric measure theory by Almgren, allow the representation of any nonoriented manifold (more generally any nonoriented rectifiable set). In particular, we explain how varifolds can encode numerically nonoriented objects both from the discrete and the continuous points of view. We show various ways to build a Hilbert space structure on the set of varifolds based on the theory of reproducing kernels. We show that, unlike the currents' setting, these metrics are consistent with shape volume (Theorem 4.1), and we derive a formula for the variation of metrics with respect to the shape (Theorem 4.2). Finally, we propose a generalization to nonoriented shapes of registration algorithms in the context of large deformation diffeomorphic metric mapping (LDDMM), which we detail with a few examples in the last part of the paper.