Deep latent variable models (DLVMs) are designed to learn meaningful representations in an unsupervised manner, such that the hidden explanatory factors are interpretable by independent latent variables (aka disentanglement). The variational autoencoder (VAE) [1], [2] is a popular DLVM widely studied in disentanglement analysis due to the modeling of the posterior distribution using a factorized Gaussian distribution [3] that encourages the alignment of the latent factors with the latent axes. Several metrics have been proposed recently, assuming that the latent variables explaining the variation in data are aligned with the latent axes (cardinal directions). However, there are other DLVMs, such as the AAE and WAE-MMD (matching the aggregate posterior to the prior), where the latent variables might not be aligned with the latent axes. In this work, we propose a statistical method to evaluate disentanglement for any DLVMs in general. The proposed technique discovers the latent vectors representing the generative factors of a dataset that can be different from the cardinal latent axes. We empirically demonstrate the advantage of the method on two datasets.

The variational autoencoder (VAE) [19], [41] is a popular, deep, latent-variable model (DLVM) due to its simple yet effective formulation for modeling the data distribution. Moreover, optimizing the VAE objective function is more manageable than other DLVMs. The bottleneck dimension of the VAE is a crucial design choice, and it has strong ramifications for the model's performance, such as finding the hidden explanatory factors of a dataset using the representations learned by the VAE. However, the size of the latent dimension of the VAE is often treated as a hyperparameter estimated empirically through trial and error. To this end, we propose a statistical formulation to discover the relevant latent factors required for modeling a dataset. In this work, we use a hierarchical prior in the latent space that estimates the variance of the latent axes using the encoded data, which identifies the relevant latent dimensions. For this, we replace the fixed prior in the VAE objective function with a hierarchical prior, keeping the remainder of the formulation unchanged. We call the proposed method the automatic relevancy detection in the variational autoencoder (ARD-VAE)11https://github.com/Surojit-Utah/ARD-VAE. We demonstrate the efficacy of the ARD-VAE on multiple benchmark datasets in finding the relevant latent dimensions and their effect on different evaluation metrics, such as FID score and disentanglement analysis.

Homography estimation serves as a fundamental technique for image alignment in a wide array of applications. The advent of convolutional neural networks has introduced learning-based methodologies that have exhibited remarkable efficacy in this realm. Yet, the generalizability of these approaches across distinct domains remains underexplored. Unlike other conventional tasks, CNN-driven homography estimation models show a distinctive immunity to domain shifts, enabling seamless deployment from one dataset to another without the necessity of transfer learning. This study explores the resilience of a variety of deep homography estimation models to domain shifts, revealing that the network architecture itself is not a contributing factor to this remarkable adaptability. By closely examining the models’ focal regions and subjecting input images to a variety of modifications, we confirm that the models heavily rely on local textures such as edges and corner points for homography estimation. Moreover, our analysis underscores that the domain shift immunity itself is intricately tied to the utilization of these local textures.1

The strong radiative forcing by atmospheric methane has stimulated interest in identifying natural and anthropogenic sources of this potent greenhouse gas. Point sources are important targets for quantification, and anthropogenic targets have the potential for emissions reduction. Methane point-source plume detection and concentration retrieval have been previously demonstrated using data from the Airborne Visible InfraRed Imaging Spectrometer-Next Generation (AVIRIS-NG). Current quantitative methods have tradeoffs between computational requirements and retrieval accuracy, creating obstacles for processing real-time data or large data sets from flight campaigns. We present a new computationally efficient algorithm that applies sparsity and an albedo correction to matched the filter retrieval of trace gas concentration path length. The new algorithm was tested using the AVIRIS-NG data acquired over several point-source plumes in Ahmedabad, India. The algorithm was validated using the simulated AVIRIS-NG data, including synthetic plumes of known methane concentration. Sparsity and albedo correction together reduced the root-mean-squared error of retrieved methane concentration-path length enhancement by 60.7% compared with a previous robust matched filter method. Background noise was reduced by a factor of 2.64. The new algorithm was able to process the entire 300 flight line 2016 AVIRIS-NG India campaign in just over 8 h on a desktop computer with GPU acceleration.

Percutaneous coronary intervention (PCI) is the prevalent treatment for coronary artery disease, with hundreds of thousands of stents implanted annually. Computational studies have demonstrated the role of biomechanics in the failure of vascular stents, but clinical studies is this area are limited by a lack of understanding of the deployed stent geometry, which is required to accurately model and predict the stent-induced in vivo biomechanical environment. Herein, we present an automated method to reconstruct the 3-D deployed stent configuration through the fusion of optical coherence tomography (OCT) and micro-computed tomography ( μ CT) imaging data. In an experimental setup, OCT and μ CT data were collected in stents deployed in arterial phantoms ( n=4 ). A constrained iterative deformation process directed by diffeomorphic metric mapping was developed to deform μ CT data of a stent wireframe to the OCT-derived sparse point cloud of the deployed stent. Reconstructions of the deployed stents showed excellent agreement with the ground-truth configurations, with the distance between corresponding points on the reconstructed and ground-truth configurations of [Formula: see text]. Finally, reconstructions required <30 min of computational time. In conclusion, the developed and validated reconstruction algorithm provides a complete spatially resolved reconstruction of a deployed vascular stent from commercially available imaging modalities and has the potential, with further development, to provide more accurate computational models to evaluate the in vivo post-stent mechanical environment, as well as clinical visualization of the 3-D stent geometry immediately following PCI.

We present a novel method for image registration via a piecewise diffeomorphic deformation which accommodates sliding motion, such as that encountered at organ boundaries. Our method jointly computes the deformation as well as a coherent sliding boundary, represented by a segmentation of the domain into regions of smooth motion. Discontinuities are allowed only at the boundaries of these regions, while invertibility of the total deformation is enforced by disallowing separation or overlap between regions. Optimization alternates between discrete segmentation estimation and continuous deformation estimation. We demonstrate our method on chest 4DCT data showing sliding motion of the lungs against the thoracic cage during breathing.

Accurate motion estimation in thoracic computed tomography (CT) plays a crucial role in the diagnosis and treatment planning of lung cancer. This paper provides two key contributions to this motion estimation. First, we show we can effectively transform a CT image of effective linear attenuation coefficients to act as a density, i.e. exhibiting conservation of mass while undergoing a deformation. Second, we propose a method for diffeomorphic density registration for thoracic CT images. This algorithm uses the appropriate density action of the diffeomorphism group while offering a weighted penalty on local tissue compressibility. This algorithm appropriately models highly compressible areas of the body (such as the lungs) and incompressible areas (such as surrounding soft tissue and bones).

We address the following problem: given two smooth densities on a manifold, find an optimal diffeomorphism that transforms one density into the other. Our framework builds on connections between the Fisher--Rao information metric on the space of probability densities and right-invariant metrics on the infinite-dimensional manifold of diffeomorphisms. This optimal information transport, and modifications thereof, allow us to construct numerical algorithms for density matching. The algorithms are inherently more efficient than those based on optimal mass transport or diffeomorphic registration. Our methods have applications in medical image registration, texture mapping, image morphing, nonuniform random sampling, and mesh adaptivity. Some of these applications are illustrated in examples.

Image population analysis is the class of statistical methods that plays a central role in understanding the development, evolution, and disease of a population. However, these techniques often require excessive computational power and memory that are compounded with a large number of volumetric inputs. Restricted access to supercomputing power limits its influence in general research and practical applications. In this paper we introduce ISP, an Image-Set Processing streaming framework that harnesses the processing power of commodity heterogeneous CPU/GPU systems and attempts to solve this computational problem. In ISP, we introduce specially designed streaming algorithms and data structures that provide an optimal solution for out-of-core multiimage processing problems both in terms of memory usage and computational efficiency. ISP makes use of the asynchronous execution mechanism supported by parallel heterogeneous systems to efficiently hide the inherent latency of the processing pipeline of out-of-core approaches. Consequently, with computationally intensive problems, the ISP out-of-core solution can achieve the same performance as the in-core solution. We demonstrate the efficiency of the ISP framework on synthetic and real datasets.

Starting with a similarity function between objects, it is possible to define a distance metric (the kernel distance) on pairs of objects, and more generally on probability distributions over them. These distance metrics have a deep basis in functional analysis and geometric measure theory, and have a rich structure that includes an isometric embedding into a Hilbert space. They have recently been applied to numerous problems in machine learning and shape analysis.

This paper investigates an approach to model the space of brain images through a low-dimensional manifold. A data driven method to learn a manifold from a collections of brain images is proposed. We hypothesize that the space spanned by a set of brain images can be captured, to some approximation, by a low-dimensional manifold, i.e. a parametrization of the set of images. The approach builds on recent advances in manifold learning that allow to uncover nonlinear trends in data. We combine this manifold learning with distance measures between images that capture shape, in order to learn the underlying structure of a database of brain images. The proposed method is generative. New images can be created from the manifold parametrization and existing images can be projected onto the manifold. By measuring projection distance of a held out set of brain images we evaluate the fit of the proposed manifold model to the data and we can compute statistical properties of the data using this manifold structure. We demonstrate this technology on a database of 436 MR brain images.

The geometric median is a classic robust estimator of centrality for data in Euclidean spaces. In this paper we formulate the geometric median of data on a Riemannian manifold as the minimizer of the sum of geodesic distances to the data points. We prove existence and uniqueness of the geometric median on manifolds with non-positive sectional curvature and give sufficient conditions for uniqueness on positively curved manifolds. Generalizing the Weiszfeld procedure for finding the geometric median of Euclidean data, we present an algorithm for computing the geometric median on an arbitrary manifold. We show that this algorithm converges to the unique solution when it exists. This method produces a robust central point for data lying on a manifold, and should have use in a variety of vision applications involving manifolds. We give examples of the geometric median computation and demonstrate its robustness for three types of manifold data: the 3D rotation group, tensor manifolds, and shape spaces.

Regression analysis is a powerful tool for the study of changes in a dependent variable as a function of an independent regressor variable, and in particular it is applicable to the study of anatomical growth and shape change. When the underlying process can be modeled by parameters in a Euclidean space, classical regression techniques are applicable and have been studied extensively. However, recent work suggests that attempts to describe anatomical shapes using flat Euclidean spaces undermines our ability to represent natural biological variability. In this paper we develop a method for regression analysis of general, manifold-valued data. Specifically, we extend Nadaraya-Watson kernel regression by recasting the regression problem in terms of Frechet expectation. Although this method is quite general, our driving problem is the study anatomical shape change as a function of age from random design image data. We demonstrate our method by analyzing shape change in the brain from a random design dataset of MR images of 89 healthy adults ranging in age from 22 to 79 years. To study the small scale changes in anatomy, we use the infinite dimensional manifold of diffeomorphic transformations, with an associated metric. We regress a representative anatomical shape, as a function of age, from this population.

A primary goal of statistical shape analysis is to describe the variability of a population of geometric objects. A standard technique for computing such descriptions is principal component analysis. However, principal component analysis is limited in that it only works for data lying in a Euclidean vector space. While this is certainly sufficient for geometric models that are parameterized by a set of landmarks or a dense collection of boundary points, it does not handle more complex representations of shape. We have been developing representations of geometry based on the medial axis description or m-rep. While the medial representation provides a rich language for variability in terms of bending, twisting, and widening, the medial parameters are not elements of a Euclidean vector space. They are in fact elements of a nonlinear Riemannian symmetric space. In this paper, we develop the method of principal geodesic analysis, a generalization of principal component analysis to the manifold setting. We demonstrate its use in describing the variability of medially-defined anatomical objects. Results of applying this framework on a population of hippocampi in a schizophrenia study are presented.

Principal component analysis has proven to be useful for understanding geometric variability in populations of parameterized objects. The statistical framework is well understood when the parameters of the objects are elements of a Euclidean vector space. This is certainly the case when the objects are described via landmarks or as a dense collection of boundary points. We have been developing representations of geometry based on the medial axis description or m-rep. Although this description has proven to be effective, the medial parameters are not naturally elements of a Euclidean space. In this paper we show that medial descriptions are in fact elements of a Lie group. We develop methodology based on Lie groups for the statistical analysis of medially-defined anatomical objects.

Medial representation of a three-dimensional (3-D) object or an ensemble of 3-D objects involves capturing the object interior as a locus of medial atoms, each atom being two vectors of equal length joined at the tail at the medial point. Medial representation has a variety of beneficial properties, among the most important of which are 1) its inherent geometry, provides an object-intrinsic coordinate system and thus provides correspondence between instances of the object in and near the object(s); 2) it captures the object interior and is, thus, very suitable for deformation; and 3) it provides the basis for an intuitive object-based multiscale sequence leading to efficiency of segmentation algorithms and trainability of statistical characterizations with limited training sets. As a result of these properties, medial representation is particularly suitable for the following image analysis tasks; how each operates will be described and will be illustrated by results: segmentation of objects and object complexes via deformable models; segmentation of tubular trees, e.g., of blood vessels, by following height ridges of measures of fit of medial atoms to target images; object-based image registration via medial loci of such blood vessel trees; statistical characterization of shape differences between control and pathological classes of structures. These analysis tasks are made possible by a new form of medial representation called m-reps, which is described.

This paper presents a multiscale framework based on a medial representation for the segmentation and shape characterization of anatomical objects in medical imagery. The segmentation procedure is based on a Bayesian deformable templates methodology in which the prior information about the geometry and shape of anatomical objects is incorporated via the construction of exemplary templates. The anatomical variability is accommodated in the Bayesian framework by defining probabilistic transformations on these templates. The transformations, thus, defined are parameterized directly in terms of natural shape operations, such as growth and bending, and their locations. A preliminary validation study of the segmentation procedure is presented. We also present a novel statistical shape analysis approach based on the medial descriptions that examines shape via separate intuitive categories, such as global variability at the coarse scale and localized variability at the fine scale. We show that the method can be used to statistically describe shape variability in intuitive terms such as growing and bending.

This paper describes the generation of large deformation diffeomorphisms /spl phi/:/spl Omega/=[0,1]/sup 3//spl rlhar2//spl Omega/ for landmark matching generated as solutions to the transport equation d/spl phi/(x,t)/dt=/spl nu/(/spl phi/(x,t),t),t/spl isin/[0,1] and /spl phi/(x,0)=x, with the image map defined as /spl phi/(/spl middot/,1) and therefore controlled via the velocity field /spl nu/(/spl middot/,t),t/spl isin/[0,1]. Imagery are assumed characterized via sets of landmarks {x/sub n/, y/sub n/, n=1, 2, ..., N}. The optimal diffeomorphic match is constructed to minimize a running smoothness cost /spl par/L/spl nu//spl par//sup 2/ associated with a linear differential operator L on the velocity field generating the diffeomorphism while simultaneously minimizing the matching end point condition of the landmarks. Both inexact and exact landmark matching is studied here. Given noisy landmarks x/sub n/ matched to y/sub n/ measured with error covariances /spl Sigma//sub n/, then the matching problem is solved generating the optimal diffeomorphism /spl phi//spl circ/(x,1)=/spl int//sub 0//sup 1//spl nu//spl circ/(/spl phi//spl circ/(x,t),t)dt+x where /spl nu//spl circ/(/spl middot/)argmin/sub /spl nu/(/spl middot/)//spl int//sub 1//sup 1//spl int//sub /spl Omega///spl par/L/spl nu/(x,t)/spl par//sup 2/dxdt +/spl Sigma//sub n=1//sup N/[y/sub n/-/spl phi/(x/sub n/,1)]/sup T//spl Sigma//sub n//sup -1/[y/sub n/-/spl phi/(x/sub n/,1)]. Conditions for the existence of solutions in the space of diffeomorphisms are established, with a gradient algorithm provided for generating the optimal flow solving the minimum problem. Results on matching two-dimensional (2-D) and three-dimensional (3-D) imagery are presented in the macaque monkey.

This paper develops mathematical representations for neuro-anatomically significant substructures of the brain and their variability in a population. The focus of the paper is on the neuro-anatomical variation of the geometry and the "shape" of two-dimensional surfaces in the brain. As examples, we focus on the cortical and hippocampal surfaces in an ensemble of Macaque monkeys and human MRI brains. The "shapes" of the substructures are quantified via the construction of templates; the variations are represented by defining probabilistic deformations of the template. Methods for empirically estimating probability measures on these deformations are developed by representing the deformations as Gaussian random vector fields on the embedded sub-manifolds. The Gaussian random vector fields are constructed as quadratic mean limits using complete orthonormal bases on the sub-manifolds. The complete orthonormal bases are generated using modes of vibrations of the geometries of the brain sub-manifolds. The covariances are empirically estimated from an ensemble of brain data. Principal component analysis is presented for characterizing the "eigen-shape" of the hippocampus in an ensemble of MRI-MPRAGE whole brain images. Clustering based on eigen-shape is presented for two sub-populations of normal and schizophrenic.

This paper presents diffeomorphic transformations of three-dimensional (3-D) anatomical image data of the macaque occipital lobe and whole brain cryosection imagery and of deep brain structures in human brains as imaged via magnetic resonance imagery. These transformations are generated in a hierarchical manner, accommodating both global and local anatomical detail. The initial low-dimensional registration is accomplished by constraining the transformation to be in a low-dimensional basis. The basis is defined by the Green's function of the elasticity operator placed at predefined locations in the anatomy and the eigenfunctions of the elasticity operator. The high-dimensional large deformations are vector fields generated via the mismatch between the template and target-image volumes constrained to be the solution of a Navier-Stokes fluid model. As part of this procedure, the Jacobian of the transformation is tracked, insuring the generation of diffeomorphisms. It is shown that transformations constrained by quadratic regularization methods such as the Laplacian, biharmonic, and linear elasticity models, do not ensure that the transformation maintains topology and, therefore, must only be used for coarse global registration.