Abstract. In this paper, we define and study a nested family of reproducing kernel Hilbert spaces of vector fields that is indexed by a range of scales, from which we construct a reproducing kernel Hilbert space of scale-dependent vector fields. We provide a characterization of the reproducing kernel of that space, with numerical approximations ensuring quick evaluations when this kernel does not have a closed form. We then introduce a multiscale version of the large deformation diffeomorphic metric mapping (LDDMM) problem and prove the existence of solutions. Finally, we provide numerical experiments performing landmark matching using multiscale LDDMM.

A multivariate regression model of affine and diffeomorphic transformation sequences—FineMorphs—is presented. Leveraging concepts from shape analysis, model states are optimally "reshaped" by diffeomorphisms generated by smooth vector fields during learning. Affine transformations and vector fields are optimized within an optimal control setting, and the model can naturally reduce (or increase) dimensionality and adapt to large data sets via sub-optimal vector fields. An existence proof of solution and necessary conditions for optimality for the model are derived. Experimental results on real data sets from the UCI repository are presented, with favorable results in comparison with state-of-the-art in the literature, neural ordinary differential equation models, and densely-connected neural networks in TensorFlow.

Cancer cells display massive dysregulation of key regulatory pathways due to now well-catalogued mutations and other DNA-related aberrations. Moreover, enormous heterogeneity has been commonly observed in the identity, frequency and location of these aberrations across individuals with the same cancer type or subtype, and this variation naturally propagates to the transcriptome, resulting in myriad types of dysregulated gene expression programs. Many have argued that a more integrative and quantitative analysis of heterogeneity of DNA and RNA molecular profiles may be necessary for designing more systematic explorations of alternative therapies and improving predictive accuracy. We introduce a representation of multi-omics profiles which is sufficiently rich to account for observed heterogeneity and support the construction of quantitative, integrated, metrics of variation. Starting from the network of interactions existing in Reactome, we build a library of "paired DNA-RNA aberrations" that represent prototypical and recurrent patterns of dysregulation in cancer; each two-gene "Source-Target Pair" (STP) consists of a "source" regulatory gene and a "target" gene whose expression is plausibly "controlled" by the source gene. The STP is then "aberrant" in a joint DNA-RNA profile if the source gene is DNA-aberrant (e.g., mutated, deleted, or duplicated), and the downstream target gene is "RNA-aberrant", meaning its expression level is outside the normal, baseline range. With M STPs, each sample profile has exactly one of the 2M possible configurations. We concentrate on subsets of STPs, and the corresponding reduced configurations, by selecting tissue-dependent minimal coverings, defined as the smallest family of STPs with the property that every sample in the considered population displays at least one aberrant STP within that family. These minimal coverings can be computed with integer programming. Given such a covering, a natural measure of cross-sample diversity is the extent to which the particular aberrant STPs composing a covering vary from sample to sample; this variability is captured by the entropy of the distribution over configurations. We apply this program to data from TCGA for six distinct tumor types (breast, prostate, lung, colon, liver, and kidney cancer). This enables an efficient simplification of the complex landscape observed in cancer populations, resulting in the identification of novel signatures of molecular alterations which are not detected with frequency-based criteria. Estimates of cancer heterogeneity across tumor phenotypes reveals a stable pattern: entropy increases with disease severity. This framework is then well-suited to accommodate the expanding complexity of cancer genomes and epigenomes emerging from large consortia projects.

We consider the problem of uncovering an unknown attributed graph, where both its edges and vertices are hidden from view, through a sequence of binary questions about it. In order to select questions efficiently, we define a probability distribution over graphs, with randomness not just over edges, but over vertices as well. We then sequentially select questions so as to: (1) minimize the expected entropy of the random graph, given the answers to the previous questions in the sequence; and (2) instantiate the vertices that compose the graph. We propose some basic question spaces, from which to select questions, that vary in their capacity. We apply this framework to the problem of test generation in Visual Question Answering (VQA), where semantic questions are used to evaluate vision systems over rich image representations. To do this, we use a restricted question vocabulary, resulting in image representations that take the form of scene graphs; by defining a distribution over them, a consistent set of probabilities is associated with the questions, and used in their selection.

In the field of Computational Anatomy, biological form (including our focus, neuroanatomy) is studied quantitatively through the action of the diffeomorphism group on example anatomies - a technique called diffeomorphometry. Here we design an algorithm within this framework to pass from dense objects common in neuromaging studies (binary segmentations, structural images) to a sparse representation defined on the surface boundaries of anatomical structures, and embedded into the low dimensional coordinates of a parametric model. Our main new contribution is to introduce an expanded group action to simultaneously deform surfaces through direct mapping of points, as well as images through functional composition with the inverse. This allows us to index the diffeomorphisms with respect to two-dimensional surface geometries like subcortical gray matter structures, but explicitly map onto cost functions determined by noisy 3-dimensional measurements. We consider models generated from empirical covariance of training data, as well as bandlimited (Laplace-Beltrami eigenfunction) models when no such data is available. We show applications to noisy or anomalous segmentations, and other typical problems in neuroimaging studies. We reproduce statistical results detecting changes in Alzheimer's disease, despite dimensionality reduction. Lastly we apply our algorithm to the common problem of segmenting subcortical structures from T1 MR images.

Diffeomorphic registration using optimal control on the diffeomorphism group and on shape spaces has become widely used since the development of the large deformation diffeomorphic metric mapping (LDDMM) algorithm. More recently, a series of algorithms involving sub-Riemannian constraints have been introduced in which the velocity fields that control the shapes in the LDDMM framework are constrained in accordance with a specific deformation model. Here, we extend this setting by considering, for the first time, inequality constraints in order to estimate surface deformations that only allow for atrophy, introducing for this purpose an algorithm that uses the augmented Lagrangian method. We prove the existence of solutions of the associated optimal control problem and the consistency of our approximation scheme. These developments are illustrated by numerical experiments on simulated and real data.

Lagrangian particle formulations of the large deformation diffeomorphic metric mapping algorithm only allow for the study of a single shape. In this paper, we introduce and discuss both a theoretical and practical setting for the simultaneous study of multiple shapes that are either stitched to one another or slide along a submanifold. The method is described within the optimal control formalism, and optimality conditions are given, together with the equations that are needed to implement augmented Lagrangian methods. Experimental results are provided for stitched and sliding surfaces.

In this work we propose a topology-preserving framework for tracking deforming objects in a sequence of video frames. The shape of the object is modeled as obtained through the action of a group of diffeomorphisms on the initial object boundary. We formulate a state-space model for the diffeomorphic deformation of the object and implement a particle filter on this shape space to estimate the state of the shape in each video frame. We use a practical method for sampling diffeomorphic shapes in which we generate deformations via flows of finitely generated vector fields. Based on the observations and the proposed samples we obtain an approximate estimate for the posterior distribution of the shape.

We present a new framework for learning high-dimensional multivariate probability distributions from estimated marginals. The approach is motivated by compositional models and Bayesian networks, and designed to adapt to small sample sizes. We start with a large, overlapping set of elementary statistical building blocks, or "primitives," which are low-dimensional marginal distributions learned from data. Each variable may appear in many primitives. Subsets of primitives are combined in a Lego-like fashion to construct a probabilistic graphical model; only a small fraction of the primitives will participate in any valid construction. Since primitives can be precomputed, parameter estimation and structure search are separated. Model complexity is controlled by strong biases; we adapt the primitives to the amount of training data and impose rules which restrict the merging of them into allowable compositions. The likelihood of the data decomposes into a sum of local gains, one for each primitive in the final structure. We focus on a specific subclass of networks which are binary forests. Structure optimization corresponds to an integer linear program and the maximizing composition can be computed for reasonably large numbers of variables. Performance is evaluated using both synthetic data and real datasets from natural language processing and computational biology.

We present a new diffeomorphic surface mapping algorithm under the framework of large deformation diffeomorphic metric mapping (LDDMM). Unlike existing LDDMM approaches, this new algorithm reduces the complexity of the estimation of diffeomorphic transformations by incorporating a shape prior in which a nonlinear diffeomorphic shape space is represented by a linear space of initial momenta of diffeomorphic geodesic flows from a fixed template. In addition, for the first time, the diffeomorphic mapping is formulated within a decision-theoretic scheme based on Bayesian modeling in which an empirical shape prior is characterized by a low dimensional Gaussian distribution on initial momentum. This is achieved using principal component analysis (PCA) to construct the eigenspace of the initial momentum. A likelihood function is formulated as the conditional probability of observing surfaces given any particular value of the initial momentum, which is modeled as a random field of vector-valued measures characterizing the geometry of surfaces. We define the diffeomorphic mapping as a problem that maximizes a posterior distribution of the initial momentum given observable surfaces over the eigenspace of the initial momentum. We demonstrate the stability of the initial momentum eigenspace when altering training samples using a bootstrapping method. We then validate the mapping accuracy and show robustness to outliers whose shape variation is not incorporated into the shape prior.

Protein signaling networks play a central role in transcriptional regulation and the etiology of many diseases. Statistical methods, particularly Bayesian networks, have been widely used to model cell signaling, mostly for model organisms and with focus on uncovering connectivity rather than inferring aberrations. Extensions to mammalian systems have not yielded compelling results, due likely to greatly increased complexity and limited proteomic measurements in vivo. In this study, we propose a comprehensive statistical model that is anchored to a predefined core topology, has a limited complexity due to parameter sharing and uses microarray data of mRNA transcripts as the only observable components of signaling. Specifically, we account for cell heterogeneity and a multilevel process, representing signaling as a Bayesian network at the cell level, modeling measurements as ensemble averages at the tissue level, and incorporating patient-to-patient differences at the population level. Motivated by the goal of identifying individual protein abnormalities as potential therapeutical targets, we applied our method to the RAS-RAF network using a breast cancer study with 118 patients. We demonstrated rigorous statistical inference, established reproducibility through simulations and the ability to recover receptor status from available microarray data.

In this study we present a geometric flow approach to the segmentation of three-dimensional medical images obtained from magnetic resonance imaging (MRI) or computed tomography (CT) scan methods, by minimizing a cost function. This energy term is based on the intensity of the original image and its minimum is found following a gradient descent curve in an infinite-dimensional space of diffeomorphisms (Diff) to preserve topology. The general framework is reminiscent of variational shape optimization methods, but remains closer to general developments on deformable template theory of geometric flows. In our case, the metric that provides the gradient is defined as a right invariant inner product on the tangent space (𝒱) at the identity of the group of diffeomorphisms, following the general Lie group approach suggested by Arnold [2]. To avoid local solutions of the optimization problem and to mitigate the influence of several sources of noise, a finite set of control points is defined on the boundary of the template binary images, yielding a projected gradient descent on Diff.

This paper provides a new variational paradigm to measure the smoothness of unit vector fields on spatial domains, leading to new methods for smoothing and interpolating such datasets. Our point of view is to consider unit vector fields as linear forms acting on reproducing kernel Hilbert spaces of vector fields or tensors and work with the dual norm, leading to new variational problems and algorithms. We prove, in particular, that these variational problems are well-posed, in the sense that optimal solutions exist in the space of unit vector fields. Experimental results are based on synthetic data and diffusion tensor–magnetic resonance imaging datasets.

In large-deformation diffeomorphic metric mapping (LDDMM), the diffeomorphic matching of images are modeled as evolution in time, or a flow, of an associated smooth velocity vector field v controlling the evolution. The initial momentum parameterizes the whole geodesic and encodes the shape and form of the target image. Thus, methods such as principal component analysis (PCA) of the initial momentum leads to analysis of anatomical shape and form in target images without being restricted to small-deformation assumption in the analysis of linear displacements. We apply this approach to a study of dementia of the Alzheimer type (DAT). The left hippocampus in the DAT group shows significant shape abnormality while the right hippocampus shows similar pattern of abnormality. Further, PCA of the initial momentum leads to correct classification of 12 out of 18 DAT subjects and 22 out of 26 control subjects

Geodesic spline interpolation is a simple and efficient approach for landmark matching by nonambiguous mappings (diffeomorphisms), combining classic spline interpolation and flows of diffeomorphisms. Here, we extend the method to incorporate the estimation of a affine transformation, yielding a consistent and numerically stable algorithm. A theoretical justification is provided by studying the existence of the global minimum of the energy.

This paper proposes a method to match diffusion tensor magnetic resonance images (DT-MRI) through the large deformation diffeomorphic metric mapping of vector fields, focusing on the fiber orientations, considered as unit vector fields on the image volume. We study a suitable action of diffeomorphisms on such vector fields, and provide an extension of the large deformation diffeomorphic metric-mapping framework to this type of dataset, resulting in optimizing for geodesies on the space of diffeomorphisms connecting two images. Two different distance function of vector fields are considered. Existence of the minimizers under smoothness assumptions on the compared vector fields is proved, and coarse to fine hierarchical strategies are detailed, to reduce both ambiguities and computation load. This is illustrated by numerical experiments on DT-MRI heart and brain images

One of the central problems in Automated Target Recognition is to accommodate the infinite variety of clutter in real military environments. The principle focus of our paper is on the construction of metric spaces where the metric measures the distance between objects of interest invariant to the infinite variety of clutter. Such metrics are formulated using second-order random field models. Our results indicate that this approach significantly improves detection/classification rates of targets in clutter.

In large-deformation diffeomorphic metric mapping (LDDMM), the diffeomorphic matching of images are modeled as evolution in time, or a flow, of an associated smooth velocity vector field v controlling the evolution. The initial momentum parameterizes the whole geodesic and encodes the shape and form of the target image. Thus, methods such as principal component analysis (PCA) of the initial momentum leads to analysis of anatomical shape and form in target images without being restricted to small-deformation assumption in the analysis of linear displacements. We apply this approach to a study of dementia of the Alzheimer type (DAT). The left hippocampus in the DAT group shows significant shape abnormality while the right hippocampus shows similar pattern of abnormality. Further, PCA of the initial momentum leads to correct classification of 12 out of 18 DAT subjects and 22 out of 26 control subjects.

We propose an algorithm achieving the matching of two point sets with unknown correspondence via the joint clustering of the two sets and estimation of a diffeomorphism linking the cluster centers via geodesic splines.

Visual action recognition is an important problem in computer vision. In this paper, we propose a new method to probabilistically model and recognize actions of articulated objects, such as hand or body gestures, in image sequences. Our method consists of three levels of representa- tion. At the low level, we first extract a feature vector invariant to scale and in-plane rotation by using the Fourier transform of a circular spatial histogram. Then, spectral partitioning [20] is utilized to obtain an initial clustering; this clustering is then refined using a temporal smoothness constraint. Gaussian mixture model (GMM) based clustering and density estimation in the subspace of linear discriminant analysis (LDA) are then applied to thousands of image feature vectors to obtain an intermediate level representation. Finally, at the high level we build a temporal multi- resolution histogram model for each action by aggregating the clustering weights of sampled images belonging to that action. We discuss how this high level representation can be extended to achieve temporal scaling in- variance and to include Bi-gram or Multi-gram transition information. Both image clustering and action recognition/segmentation results are given to show the validity of our three tiered representation.

Over the past several years our group has been studying biological shape in the emerging new discipline of computational anatomy (CA). CA consists of several components: (i) the construction of coordinatized anatomical manifolds, (ii) comparison of anatomical manifolds, and (iii) inference of morphometric change on anatomical manifolds. In this paper we focus on (ii) the comparison of anatomical shapes and structures in imagery via metric mapping. The purpose of this paper is to examine the generation of the geodesics associated with the metric from several points of view, the first the Euler equation describing the geodesic diffeomorphic flow, and the second the variational formulation of the geodesic in terms of the minimizing flow of vector fields which generate them.

This paper addresses the issue of motion estimation on image sequences. The standard motion equation used to complete the apparent motion of image irradiance patterns is an invariance brightness based hypothesis called the optical flow constraint. Other equations can be used, in particular the extended optical flow constraint, which is a variant of the optical flow constraint, inspired by the fluid mechanic mass conservation principle. In this paper we propose a physical interpretation of this extended optical flow equation and a new model unifying the optical flow and the extended optical flow constraints. We present results obtained for synthetic and meteorological images.

Motion estimation is one of the main subjects in computer vision. Many methods developed to compute motion make use of the optical flow hypothesis. These methods usually fail to capture motion of objects with intensity evolution. We propose a new approach to solve the motion computation problem with a different type of constancy hypothesis. Because we are mainly interested in deformable moving structures, we postulate that such a structure, within a temporal image sequence, is associated with a constant volume or a constant total intensity over time. We call this postulate the volume conservation hypothesis. Results are displayed for clouds motion and deformation on meteorological satellites images.

Abstract—We propose a general synchronous model of lattice random fields which could be used similarly to Gibbs distributions in a Bayesian framework for image analysis, leading to algorithms ideally designed for an implementation on massively parallel hardware. After a theoretical description of the model, we give an experimental illustration in the context of image restoration. Index Terms—Random fields, image processing, parallelism, Monte-Carlo sampling. ————————— — F ——————————

We define a distance between textures for texture classification from texture features based on windowed Fourier filters. The definition of the distance relies on an interpretation of our texture attributes in terms of spectral density when the texture can be considered as a Gaussian random field. The distance between textures is then defined as a symmetrized Kullback distance which is a simple function of the attributes and does not require any normalization. An experimental analysis using Gabor filters, and in particular a comparison to quadratic distances, shows the efficiency and robustness of the method.

We define distances between geometric curves by the square root of the minimal energy required to transform one curve into the other. The energy is formally defined from a left invariant Riemannian distance on an infinite dimensional group acting on the curves, which can be explicitly computed. The obtained distance boils down to a variational problem for which an optimal matching between the curves has be computed. An analysis of the distance when the curves are polygonal leads to a numerical procedure for the solution of the variational problem, which can efficiently be implemented, as illustrated by experiments.

We present and study learning rules for specific models of Boltzmann machines which extend classical sequential and reversible synchronous models.