The reliable recovery and uncertainty quantification of a fixed effect function $\mu$ in a functional mixed model, for modeling population- and object-level variability in noisily observed functional data, is a notoriously challenging task: variations along the $x$ and $y$ axes are confounded with additive measurement error, and cannot in general be disentangled. The question then as to what properties of $\mu$ may be reliably recovered becomes important. We demonstrate that it is possible to recover the size-and-shape of a square-integrable $\mu$ under a Bayesian functional mixed model. The size-and-shape of $\mu$ is a geometric property invariant to a family of space-time unitary transformations, viewed as rotations of the Hilbert space, that jointly transform the $x$ and $y$ axes. A random object-level unitary transformation then captures size-and-shape preserving deviations of $\mu$ from an individual function, while a random linear term and measurement error capture size-and-shape altering deviations. The model is regularized by appropriate priors on the unitary transformations, posterior summaries of which may then be suitably interpreted as optimal data-driven rotations of a fixed orthonormal basis for the Hilbert space. Our numerical experiments demonstrate utility of the proposed model, and superiority over the current state-of-the-art.

Topological data analysis provides a set of tools to uncover low-dimensional structure in noisy point clouds. Prominent amongst the tools is persistence homology, which summarizes birth-death times of homological features using data objects known as persistence diagrams. To better aid statistical analysis, a functional representation of the diagrams, known as persistence landscapes, enable use of functional data analysis and machine learning tools. Topological and geometric variabilities inherent in point clouds are confounded in both persistence diagrams and landscapes, and it is important to distinguish topological signal from noise to draw reliable conclusions on the structure of the point clouds when using persistence homology. We develop a framework for decomposing variability in persistence diagrams into topological signal and topological noise through alignment of persistence landscapes using an elastic Riemannian metric. Aligned landscapes (amplitude) isolate the topological signal. Reparameterizations used for landscape alignment (phase) are linked to a resolution parameter used to generate persistence diagrams, and capture topological noise in the form of geometric, global scaling and sampling variabilities. We illustrate the importance of decoupling topological signal and topological noise in persistence diagrams (landscapes) using several simulated examples. We also demonstrate that our approach provides novel insights in two real data studies.

We propose a statistical framework to analyze radiological magnetic resonance imaging (MRI) and genomic data to identify the underlying radiogenomic associations in lower grade gliomas (LGG). We devise a novel imaging phenotype by dividing the tumor region into concentric spherical layers that mimics the tumor evolution process. MRI data within each layer is represented by voxel-intensity-based probability density functions which capture the complete information about tumor heterogeneity. Under a Riemannian-geometric framework these densities are mapped to a vector of principal component scores which act as imaging phenotypes. Subsequently, we build Bayesian variable selection models for each layer with the imaging phenotypes as the response and the genomic markers as predictors. Our novel hierarchical prior formulation incorporates the interior-to-exterior structure of the layers, and the correlation between the genomic markers. We employ a computationally-efficient Expectation-Maximization-based strategy for estimation. Simulation studies demonstrate the superior performance of our approach compared to other approaches. With a focus on the cancer driver genes in LGG, we discuss some biologically relevant findings. Genes implicated with survival and oncogenesis are identified as being associated with the spherical layers, which could potentially serve as early-stage diagnostic markers for disease monitoring, prior to routine invasive approaches. We provide a R package that can be used to deploy our framework to identify radiogenomic associations.

Deep learning architectures, albeit successful in most computer vision tasks, were designed for data with an underlying Euclidean structure, which is not usually fulfilled since pre-processed data may lie on a non-linear space. In this article, we propose a geometric deep learning approach using rigid and non-rigid transformations, named KShapenet, for 2D and 3D landmark-based human motion analysis. Landmark configuration sequences are first modeled as trajectories on Kendall's shape space and then mapped to a linear tangent space. The resulting structured data are then input to a deep learning architecture, which includes a layer that optimizes over rigid and non-rigid transformations of landmark configurations, followed by a CNN-LSTM network. We apply KShapenet to 3D human landmark sequences for action and gait recognition, and 2D facial landmark sequences for expression recognition, and demonstrate the competitiveness of the proposed approach with respect to state-of-the-art.

We propose a novel framework to learn the spatiotemporal variability in longitudinal 3D shape data sets, which contain observations of objects that evolve and deform over time. This problem is challenging since surfaces come with arbitrary parameterizations and thus, they need to be spatially registered. Also, different deforming objects, hereinafter referred to as 4D surfaces, evolve at different speeds and thus they need to be temporally aligned. We solve this spatiotemporal registration problem using a Riemannian approach. We treat a 3D surface as a point in a shape space equipped with an elastic Riemannian metric that measures the amount of bending and stretching that the surfaces undergo. A 4D surface can then be seen as a trajectory in this space. With this formulation, the statistical analysis of 4D surfaces can be cast as the problem of analyzing trajectories embedded in a nonlinear Riemannian manifold. However, performing the spatiotemporal registration, and subsequently computing statistics, on such nonlinear spaces is not straightforward as they rely on complex nonlinear optimizations. Our core contribution is the mapping of the surfaces to the space of Square-Root Normal Fields (SRNF) where the [Formula: see text] metric is equivalent to the partial elastic metric in the space of surfaces. Thus, by solving the spatial registration in the SRNF space, the problem of analyzing 4D surfaces becomes the problem of analyzing trajectories embedded in the SRNF space, which has a euclidean structure. In this paper, we develop the building blocks that enable such analysis. These include: (1) the spatiotemporal registration of arbitrarily parameterized 4D surfaces even in the presence of large elastic deformations and large variations in their execution rates; (2) the computation of geodesics between 4D surfaces; (3) the computation of statistical summaries, such as means and modes of variation, of collections of 4D surfaces; and (4) the synthesis of random 4D surfaces. We demonstrate the performance of the proposed framework using 4D facial surfaces and 4D human body shapes.

Deep Learning architectures, albeit successful in most computer vision tasks, were designed for data with an underlying Euclidean structure, which is not usually fulfilled since pre-processed data may lie on a non-linear space. In this paper, we propose a geometry aware deep learning approach using rigid and non rigid transformation optimization for skeleton-based action recognition. Skeleton sequences are first modeled as trajectories on Kendall’s shape space and then mapped to the linear tangent space. The resulting structured data are then fed to a deep learning architecture, which includes a layer that optimizes over rigid and non rigid transformations of the 3D skeletons, followed by a CNN-LSTM network. The assessment on two large scale skeleton datasets, namely NTU-RGB+D and NTU-RGB+D 120, has proven that the proposed approach outperforms existing geometric deep learning methods and exceeds recently published approaches with respect to the majority of configurations.

In the shape analysis approach to computer vision problems, one treats shapes as points in an infinite-dimensional Riemannian manifold, thereby facilitating algorithms for statistical calculations such as geodesic distance between shapes and averaging of a collection of shapes. The performance of these algorithms depends heavily on the choice of the Riemannian metric. In the setting of plane curve shapes, attention has largely been focused on a two-parameter family of first order Sobolev metrics, referred to as elastic metrics. They are particularly useful due to the existence of simplifying coordinate transformations for particular parameter values, such as the well-known square-root velocity transform. In this paper, we extend the transformations appearing in the existing literature to a family of isometries, which take any elastic metric to the flat $L^2$ metric. We also extend the transforms to treat piecewise linear curves and demonstrate the existence of optimal matchings over the diffeomorphism group in this setting. We conclude the paper with multiple examples of shape geodesics for open and closed curves. We also show the benefits of our approach in a simple classification experiment.

We propose a new method for the construction and visualization of geometrically-motivated boxplot displays for elastic curve data. We use a recent shape analysis framework, based on the square-root velocity function representation of curves, to extract different sources of variability from elastic curves, which include location, scale, shape, orientation and parametrization. We then focus on constructing separate displays for these various components using the Riemannian geometry of their representation spaces. This involves computation of a median, two quartiles, and two extremes based on geometric considerations. The outlyingness of an elastic curve is also defined separately based on each of the five components. We evaluate the proposed methods using multiple simulations, and then focus our attention on real data applications. In particular, we study variability in (a) 3D spirals, (b) handwritten signatures, (c) 3D fibers from diffusion tensor magnetic resonance imaging, and (d) trajectories of the Lorenz system.

We present a novel framework to analyze hand force signals and to capture their key spatio-temporal patterns in order to characterize Rheumatoid Arthritis. We introduce a new continuous representation of hand force and derive optimal intra-class alignments using the notion of Karcher means on the quotient space under the action of the warping group. We apply this idea to temporally register hand force signal data using non-linear time warping. As a result, the original signals are separated into their phase and amplitude components. To capture the amplitude and phase variabilities in force functions we compute the dominant eigenfunctions of the covariance operator using functional principal component analysis. Finally, we use support vector machine classifiers to learn priors from current state-of-the-art features and additional features derived from our functional data analysis framework. The experimental results demonstrate that the proposed framework generates clinically relevant features leading to state-of-the-art classification performance.

Segmentation of handwritten text into individual characters is an important step in many handwriting recognition tasks. In this paper, we present two segmentation algorithms based on elastic shape analysis of parameterized, planar curves. The shape analysis methodology provides matching, comparison and averaging of handwritten curves in a unified framework, which are very useful tools for designing segmentation algorithms. The first type of segmentation can be performed by splitting a full word into individual characters using a matching function. Another type of segmentation can be obtained by matching parts of the handwritten words to a given individual character. We validate the two proposed algorithms on real handwritten signatures and words coming from the SVC 2004 and the UNIPEN ICROW 2003 datasets. We show that the proposed methods are able to successfully segment text coming from highly variable handwriting styles.

International audience

Reflection symmetry is an important feature of an object. Main goals in symmetry analysis include quantifying the amount of asymmetry in an object and finding the nearest symmetric object to a given asymmetric one. Samir et al. [19] achieved these goals using a shape distance between representations of curves termed square-root velocity functions. We extend their work by defining shape descriptors based on this representation. The descriptors are based on asymmetry measures computed for a set of reflections of a curve and are invariant to all shape preserving transformations (translation, scale, rotation and re-parameterization). We utilize these descriptors for retrieval of shapes in the Flavia leaf database and a subset of a handwritten digit dataset. We show that we outperform the commonly used angle function and other state of the art descriptors.

We study the problem of joint registration and deformation analysis of endometrial tissue using 3D magnetic resonance imaging (MRI) and 2D trans-vaginal ultrasound (TVUS) measurements. In addition to the different imaging techniques involved in the two modalities, this problem is complicated due to: 1) different patient pose during MRI and TVUS observations, 2) the 3D nature of MRI and 2D nature of TVUS measurements, 3) the unknown intersecting plane for TVUS in MRI volume, and 4) the potential deformation of endometrial tissue during TVUS measurement process. Focusing on the shape of the tissue, we use expert manual segmentation of its boundaries in the two modalities and apply, with modification, recent developments in shape analysis of parametric surfaces to this problem. First, we extend the 2D TVUS curves to generalized cylindrical surfaces through replication, and then we compare them with MRI surfaces using elastic shape analysis. This shape analysis provides a simultaneous registration (optimal reparameterization) and deformation (geodesic) between any two parametrized surfaces. Specifically, it provides optimal curves on MRI surfaces that match with the original TVUS curves. This framework results in an accurate quantification and localization of the deformable endometrial cells for radiologists, and growth characterization for gynecologists and obstetricians. We present experimental results using semi-synthetic data and real data from patients to illustrate these ideas.

Statistical shape analysis develops methods for comparisons, deformations, summarizations, and modeling of shapes in given data sets. These tasks require a fundamental tool called parallel transport of tangent vectors along arbitrary paths. This tool is essential for: (1) computation of geodesic paths using either shooting or path-straightening method, (2) transferring deformations across objects, and (3) modeling of statistical variability in shapes. Using the square-root normal field (SRNF) representation of parameterized surfaces, we present a method for transporting deformations along paths in the shape space. This is difficult despite the underlying space being a vector space because the chosen (elastic) Riemannian metric is non-standard. Using a finite-basis for representing SRNFs of shapes, we derive expressions for Christoffel symbols that enable parallel transports. We demonstrate this framework using examples from shape analysis of parameterized spherical surfaces, in the three contexts mentioned above.

The shapes of plant leaves are of great importance to plant biologists and botanists, as they can help in distinguishing plant species, measuring their health, analyzing their growth patterns, and understanding relations between various species. We propose a statistical model that uses the Squared Root Velocity Function representation and a Riemannian elastic metric to model the observed variability in the shape of plant leaves. We show that under this representation, one can compute sample means and principal modes of variations and can characterize the observed shapes using probability models, such as Gaussians, on the tangent spaces at the sample means. The approach is fully automatic and does not require precomputing correspondences between the shapes. We validate these statistical models by analyzing their classification performance on standard benchmarks and show their utility as generative models for random sampling.

Abstract We argue that full surface correspondence (registration) and optimal deformations (geodesics) are two related problems and propose a framework that solves them simultaneously. We build on the Riemannian shape analysis of anatomical and star‐shaped surfaces of Kurtek et al. and focus on articulated complex shapes that undergo elastic deformations and that may contain missing parts. Our core contribution is the re‐formulation of Kurtek et al.'s approach as a constrained optimization over all possible re‐parameterizations of the surfaces, using a sparse set of corresponding landmarks. We introduce a landmark‐constrained basis, which we use to numerically solve this optimization and therefore establish full surface registration and geodesic deformation between two surfaces. The length of the geodesic provides a measure of dissimilarity between surfaces. The advantages of this approach are: (1) simultaneous computation of full correspondence and geodesic between two surfaces, given a sparse set of matching landmarks (2) ability to handle more comprehensive deformations than nearly isometric, and (3) the geodesics and the geodesic lengths can be further used for symmetrizing 3D shapes and for computing their statistical averages. We validate the framework on challenging cases of large isometric and elastic deformations, and on surfaces with missing parts. We also provide multiple examples of averaging and symmetrizing 3D models.

This paper presents a novel Riemannian framework for shape analysis of parameterized surfaces. In particular, it provides efficient algorithms for computing geodesic paths which, in turn, are important for comparing, matching, and deforming surfaces. The novelty of this framework is that geodesics are invariant to the parameterizations of surfaces and other shape-preserving transformations of surfaces. The basic idea is to formulate a space of embedded surfaces (surfaces seen as embeddings of a unit sphere in IR3) and impose a Riemannian metric on it in such a way that the reparameterization group acts on this space by isometries. Under this framework, we solve two optimization problems. One, given any two surfaces at arbitrary rotations and parameterizations, we use a path-straightening approach to find a geodesic path between them under the chosen metric. Second, by modifying a technique presented in [25], we solve for the optimal rotation and parameterization (registration) between surfaces. Their combined solution provides an efficient mechanism for computing geodesic paths in shape spaces of parameterized surfaces. We illustrate these ideas using examples from shape analysis of anatomical structures and other general surfaces.

While signal estimation under random amplitudes, phase shifts, and additive noise is studied frequently, the problem of estimating a deterministic signal under random time-warpings has been relatively unexplored. We present a novel framework for estimating the unknown signal that utilizes the action of the warping group to form an equivalence relation between signals. First, we derive an estimator for the equivalence class of the unknown signal using the notion of Karcher mean on the quotient space of equivalence classes. This step requires the use of Fisher-Rao Riemannian metric and a square-root representation of signals to enable computations of distances and means under this metric. Then, we define a notion of the center of a class and show that the center of the estimated class is a consistent estimator of the underlying unknown signal. This estimation algorithm has many applications: (1)registration/alignment of functional data, (2) separation of phase/amplitude components of functional data, (3) joint demodulation and carrier estimation, and (4) sparse modeling of functional data. Here we demonstrate only (1) and (2): Given signals are temporally aligned using nonlinear warpings and, thus, separated into their phase and amplitude components. The proposed method for signal alignment is shown to have state of the art performance using Berkeley growth, handwritten signatures, and neuroscience spike train data.

We consider 3-D brain structures as continuous parameterized surfaces and present a metric for their comparisons that is invariant to the way they are parameterized. Past comparisons of such surfaces involve either volume deformations or nonrigid matching under fixed parameterizations of surfaces. We propose a new mathematical representation of surfaces, called q-maps, such that L² distances between such maps are invariant to re-parameterizations. This property allows for removing the parameterization variability by optimizing over the re-parameterization group, resulting in a proper parameterization-invariant distance between shapes of surfaces. We demonstrate this method in shape analysis of multiple brain structures, for 34 subjects in the Detroit Fetal Alcohol and Drug Exposure Cohort study, which results in a 91% classification rate for attention deficit hyperactivity disorder cases and controls. This method outperforms some existing techniques such as spherical harmonic point distribution model (SPHARM-PDM) or iterative closest point (ICP).

In this paper we introduce a novel Riemannian framework for shape analysis of parameterized surfaces. We derive a distance function between any two surfaces that is invariant to rigid motion, global scaling, and re-parametrization. It is the last part that presents the main difficulty. Our solution to this problem is twofold: (1) we define a special representation, called a q-map, to represent each surface, and (2) we develop a gradient-based algorithm to optimize over different re-parameterizations of a surface. The second step is akin to deforming the mesh on a fixed surface to optimize its placement. (This is different from the current methods that treat the given meshes as fixed.) Under the chosen representation, with the L2metric, the action of the re-parametrization group is by isometries. This results in, to our knowledge, the first Riemannian distance between parameterized surfaces to have all the desired invariances. We demonstrate this framework with several examples using some toy shapes, and real data with anatomical structures, and cropped facial surfaces. We also successfully demonstrate clustering and classification of these objects under the proposed metric.