Shape graphs are complex geometrical structures commonly found in biological and anatomical systems. A shape graph is a collection of nodes, some connected by curvilinear edges with arbitrary shapes. Their high complexity stems from the large number of nodes and edges and the complex shapes of edges. With an eye toward statistical analysis, one seeks low-complexity representations that retain as much of the global structure of the original shape graphs as possible. This paper develops a framework for reducing shape graph complexity using hierarchical clustering procedures that replace groups of nodes and edges with simpler representatives. It demonstrates this framework using graphs of retinal blood vessels in two dimensions and neurons in three dimensions. The paper also presents experiments on the classification of shape graphs using progressively reduced levels of graph complexity. The accuracy of disease detection in retinal blood vessels appears to be particularly sensitive to discarding terminal edges. Accuracy in identifying neural cell types remains stable with complexity reduction.

This paper presents a generative statistical model for analyzing time series of planar shapes. Using elastic shape analysis, we separate object kinematics (rigid motions and speed variability) from morphological evolution, representing the latter through transported velocity fields (TVFs). A principal component analysis (PCA) based dimensionality reduction of the TVF representation provides a finite-dimensional Euclidean framework, enabling traditional time-series analysis. We then fit a vector auto-regressive (VAR) model to the TVF-PCA time series, capturing the statistical dynamics of shape evolution. To characterize morphological changes, we use VAR model parameters for model comparison, synthesis, and sequence classification. Leveraging these parameters, along with machine learning classifiers, we achieve high classification accuracy. Extensive experiments on cell motility data validate our approach, demonstrating its effectiveness in modeling and classifying migrating cells based on morphological evolution—marking a novel contribution to the field. • A novel method for statistical modeling and analysis of temporally-evolving shapes. • Elastic shape analysis represents a shape sequence by Euclidean time series. • Dynamic shape evolution is crucial aspect of cell migration in biology. • A detailed approach for modeling and classifying shape dynamics during cell motility.

We propose a novel framework for the statistical analysis of genus-zero 4D surfaces, i.e., 3D surfaces that deform and evolve over time. This problem is particularly challenging due to the arbitrary parameterizations of these surfaces and their varying deformation speeds, necessitating effective spatiotemporal registration. Traditionally, 4D surfaces are discretized, in space and time, before computing their spatiotemporal registrations, geodesics, and statistics. However, this approach may result in suboptimal solutions and, as we demonstrate in this paper, is not necessary. In contrast, we treat 4D surfaces as continuous functions in both space and time. We introduce Dynamic Spherical Neural Surfaces (D-SNS), an efficient smooth and continuous spatiotemporal representation for genus-0 4D surfaces. We then demonstrate how to perform core 4D shape analysis tasks such as spatiotemporal registration, geodesics computation, and mean 4D shape estimation, directly on these continuous representations without upfront discretization and meshing. By integrating neural representations with classical Riemannian geometry and statistical shape analysis techniques, we provide the building blocks for enabling full functional shape analysis. We demonstrate the efficiency of the framework on 4D human and face datasets. The source code and additional results are available at https://4d-dsns.github.io/DSNS/.

Reliability is one of the key traits of any space-based hardware. They undergo rigorous testing and evaluation prior to onboard deployment in order to guarantee fault-free operation over the mission life. Payload Controller (PLC) is package consisting of multiple boards that have different, but inter-dependent functionalities. It has a mission critical role of controlling, health monitoring and maintaining operating temperature of all payload subsystems. There is no off-the-shelf instrument that can analyze and verify PLC’s complex operations. Existing literature on functional testing of space-based hardware isolates each functionality. Automated System for Testing and Result Analysis (ASTRA) is a custom developed reconfigurable and fully integrated Automatic Test Equipment (ATE) for evaluating Payload Controller (PLC). It accurately simulates the entire PLC operational environment in line with "Test-as-you-fly" philosophy, supports macro based test scenarios and automates the verification process. Its software-defined architecture offers flexibility to support subsystem and package level testing, while its scalable and reconfigurable design accommodates various PLC configurations. This paper describes the design of ASTRA, its constituent modules and how each part contributes in accelerating and automating the testing process.

This paper proposes a dead-time control circuit to generate independent and adaptive delays for the rise and fall time duration. The circuit comprises a rise/fall time detector, rise/fall time to voltage converter and switch capacitor-based charge integrator block to generate the adaptive dead-time. The proposed adaptive dead-time controller design implemented using a 0.18μm HV CMOS process, occupies 170μm x 90μm silicon area. The results show good accuracy in the dead-time generation with an error <±3.5ns. In post-layout simulation, the design provides sinusoidal output with a very low offset voltage of 70mV.

This paper provides developments in statistical shape analysis of shape graphs, and demonstrates them using such complex objects as Retinal Blood Vessel (RBV) networks and neurons. The shape graphs are represented by sets of nodes and edges (articulated curves) connecting some nodes. The goals are to utilize nodes (locations, connectivity) and edges (edge weights and shapes) to: (1) characterize shapes, (2) quantify shape differences, and (3) model statistical variability. We develop a mathematical representation, elastic Riemannian metrics, and associated tools for shape graphs. Specifically, we derive tools for shape graph registration, geodesics, statistical summaries, shape modeling, and shape synthesis. Geodesics are convenient for visualizing optimal deformations, and PCA helps in dimension reduction and statistical modeling. One key challenge lies in comparing shape graphs with vastly different complexities (in number of nodes and edges). This paper introduces a novel multi-scale representation to handle this challenge. Using the notions of (1) "effective resistance" to cluster nodes and (2) elastic shape averaging of edge curves, it reduces graph complexity while retaining overall structures. This allows shape comparisons by bringing graphs to similar complexities. We demonstrate these ideas on 2D RBV networks from the STARE and DRIVE databases and 3D neurons from the NeuroMorpho database.

How can one analyze detailed 3D biological objects, such as neuronal and botanical trees, that exhibit complex geometrical and topological variation? In this paper, we develop a novel mathematical framework for representing, comparing, and computing geodesic deformations between the shapes of such tree-like 3D objects. A hierarchical organization of subtrees characterizes these objects - each subtree has a main branch with some side branches attached - and one needs to match these structures across objects for meaningful comparisons. We propose a novel representation that extends the Square-Root Velocity Function (SRVF), initially developed for Euclidean curves, to tree-shaped 3D objects. We then define a new metric that quantifies the bending, stretching, and branch sliding needed to deform one tree-shaped object into the other. Compared to the current metrics such as the Quotient Euclidean Distance (QED) and the Tree Edit Distance (TED), the proposed representation and metric capture the full elasticity of the branches (i.e., bending and stretching) as well as the topological variations (i.e., branch death/birth and sliding). It completely avoids the shrinkage that results from the edge collapse and node split operations of the QED and TED metrics. We demonstrate the utility of this framework in comparing, matching, and computing geodesics between biological objects such as neuronal and botanical trees. We also demonstrate its application to various shape analysis tasks such as (i) symmetry analysis and symmetrization of tree-shaped 3D objects, (ii) computing summary statistics (means and modes of variations) of populations of tree-shaped 3D objects, (iii) fitting parametric probability distributions to such populations, and (iv) finally synthesizing novel tree-shaped 3D objects through random sampling from estimated probability distributions.

This paper uses sample data to study the problem of comparing populations on finite-dimensional parallelizable Riemannian manifolds and more general trivial vector bundles. Utilizing triviality, our framework represents populations as mixtures of Gaussians on vector bundles and estimates the population parameters using a mode-based clustering algorithm. We derive a Wasserstein-type metric between Gaussian mixtures, adapted to the manifold geometry, in order to compare estimated distributions. Our contributions include an identifiability result for Gaussian mixtures on manifold domains and a convenient characterization of optimal couplings of Gaussian mixtures under the derived metric. We demonstrate these tools on some example domains, including the preshape space of planar closed curves, with applications to the shape space of triangles and populations of nanoparticles. In the nanoparticle application, we consider a sequence of populations of particle shapes arising from a manufacturing process and utilize the Wasserstein-type distance to perform change-point detection.

Sketching is a visual thinking tool available to humans for several decades. With the advent of modern sketching technologies, artists use sketches to express and iterate their ideas. To accelerate sketch-based ideation and illustration workflows, we propose a novel framework, SketchBuddy, which retrieves diverse fine-grained object suggestions to enrich a sketch and coherently inserts it into the scene. Sketchbuddy detects objects in the input sketch to estimate the scene context which is then utilized for the recommendation and insertion. We propose a novel multi-modal transformer based framework for obtaining context-aware fine-grained object recommendations. We train a CNN-based bounding box classifier to extract information from the input scene and the recommended objects to infer plausible locations for insertion. While prior works focus on sketches at object-level only, SketchBuddy is the first work in the direction of scene-level sketching assistance. Our extensive evaluations comparing SketchBuddy against competing baselines across several metrics and agreements with human preferences demonstrate its value on several aspects.

We propose a novel framework to learn the spatiotemporal variability in longitudinal 3D shape data sets, which contain observations of objects that evolve and deform over time. This problem is challenging since surfaces come with arbitrary parameterizations and thus, they need to be spatially registered. Also, different deforming objects, hereinafter referred to as 4D surfaces, evolve at different speeds and thus they need to be temporally aligned. We solve this spatiotemporal registration problem using a Riemannian approach. We treat a 3D surface as a point in a shape space equipped with an elastic Riemannian metric that measures the amount of bending and stretching that the surfaces undergo. A 4D surface can then be seen as a trajectory in this space. With this formulation, the statistical analysis of 4D surfaces can be cast as the problem of analyzing trajectories embedded in a nonlinear Riemannian manifold. However, performing the spatiotemporal registration, and subsequently computing statistics, on such nonlinear spaces is not straightforward as they rely on complex nonlinear optimizations. Our core contribution is the mapping of the surfaces to the space of Square-Root Normal Fields (SRNF) where the [Formula: see text] metric is equivalent to the partial elastic metric in the space of surfaces. Thus, by solving the spatial registration in the SRNF space, the problem of analyzing 4D surfaces becomes the problem of analyzing trajectories embedded in the SRNF space, which has a euclidean structure. In this paper, we develop the building blocks that enable such analysis. These include: (1) the spatiotemporal registration of arbitrarily parameterized 4D surfaces even in the presence of large elastic deformations and large variations in their execution rates; (2) the computation of geodesics between 4D surfaces; (3) the computation of statistical summaries, such as means and modes of variation, of collections of 4D surfaces; and (4) the synthesis of random 4D surfaces. We demonstrate the performance of the proposed framework using 4D facial surfaces and 4D human body shapes.

Advances in imaging and sequencing techniques provide increasingly informative but complex structural data for understanding the 3D genome. Segmenting 3D conformations of chromosomes into their topologically associated domains (TADs) – structurally homogeneous parts of a chromosome – is an essential tool in shape analysis. TAD segmentation allows one to divide complex chromosomal structures into smaller, simpler geometries and facilitates statistical analysis of their shapes. There are several TAD segmentation procedures in the current literature, but they often lack consistency and interpretability. We propose a novel algorithm for determining TADs, called TADBay, directly from contact matrices, which avoids the difficult step of estimating 3D conformations. While consistent with some current top-performing existing callers, it has an added ability to provide a hierarchical organization of TADs and subTADs. To validate TADBay, we utilize simulated contact matrices with ground truth TADs which can be used to benchmark TAD callers. We demonstrate the strengths of our caller through both simulated data and the IMR90 real dataset.

This paper addresses the problem of characterizing statistical distributions of cellular shape populations using shape samples from microscopy image data. This problem is challenging because of the nonlinearity and high-dimensionality of shape manifolds. The paper develops an efficient, nonparametric approach using ideas from k-modal mixtures and kernel estimators. It uses elastic shape analysis of cell boundaries to estimate statistical modes and clusters given shapes around those modes. (Notably, it uses a combination of modal distributions and ANOVA to determine k automatically.) A population is then characterized as k-modal mixture relative to this estimated clustering and a chosen kernel (e.g., a Gaussian or a flat kernel). One can compare and analyze populations using the Fisher-Rao metric between their estimated distributions. We demonstrate this approach for classifying shapes associated with migrations of entamoeba histolytica under different experimental conditions. This framework remarkably captures salient shape patterns and separates shape data for different experimental settings, even when it is difficult to discern class differences visually.

Elastic Riemannian metrics have been used successfully for statistical treatments of functional and curve shape data. However, this usage suffers from a significant restriction: the function boundaries are assumed to be fixed and matched. In practice, functional data often comes with unmatched boundaries. It happens, for example, in dynamical systems with variable evolution rates, such as COVID-19 infection rate curves associated with different geographical regions. Here, we develop a Riemannian framework that allows for partial matching, comparing, and clustering of functions with phase variability and uncertain boundaries. We extend past work by (1) Defining a new diffeomorphism group G over the positive reals that is the semidirect product of a time-warping group and a time-scaling group; (2) Introducing a metric that is invariant to the action of G; (3) Imposing a Riemannian Lie group structure on G to allow for an efficient gradient-based optimization for elastic partial matching; and (4) Presenting a modification that, while losing the metric property, allows one to control the amount of boundary disparity in the registration. We illustrate this framework by registering and clustering shapes of COVID-19 rate curves, identifying basic patterns, minimizing mismatch errors, and reducing variability within clusters compared to previous methods.

Despite enormous structural variability exhibited in 3D chromosomal conformations at a global scale, there is a significant commonality of structures visible at smaller, local levels. We hypothesize that chromosomal conformations are representable as concatenations of a handful of prototypical shapelets, termed shape letters. This is akin to expressing complicated sentences in a language using only a small set of letters. Our goal is to organize the vast variability of 3D chromosomal conformation by constructing a set of predominant shape letters, termed a shape alphabet, using statistical shape analysis of curvelets taken from training conformations. This paper utilizes conformations generated from Integrative Genome Modeling to develop a shape alphabet as follows: it first segments 3D conformations into curvelets according to their Topologically Associated Domains. It then clusters these segments, estimates mean shapes, and refines and reorders these shapes into a Chromosome Shape Alphabet. The paper demonstrates effectiveness of this construction by successfully representing independent test conformations taken from IGM and other methods such as SIMBA3D, both symbolically and structurally, using the constructed alphabet.

The problem of using covariates to predict shapes of objects in a regression setting is important in many fields. A formal statistical approach, termed Geodesic regression model, is commonly used for modeling and analyzing relationships between Euclidean predictors and shape responses. Despite its popularity, this model faces several key challenges, including (i) misalignment of shapes due to pre-processing steps, (ii) difficulties in shape alignment due to imaging heterogeneity, and (iii) lack of spatial correlation in shape structures. This paper proposes a comprehensive geodesic factor regression model that addresses all these challenges. Instead of using shapes as extracted from pre-registered data, it takes a more fundamental approach, incorporating alignment step within the proposed regression model and learns them using both pre-shape and covariate data. Additionally, it specifies spatial correlation structures using low-dimensional representations, including latent factors on the tangent space and isotropic error terms. The proposed framework results in substantial improvements in regression performance, as demonstrated through simulation studies and a real data analysis on Corpus Callosum contour data obtained from the ADNI study.

Statistical analysis of shape evolution during cell migration is important for gaining insights into biological processes. This paper develops a time-series model for temporal evolution of cellular shapes during cell motility. It uses elastic shape analysis to represent and analyze shapes of cell boundaries (as planar closed curves), thus separating cell shape changes from cell kinematics. Specifically, it utilizes Transported Square-Root Velocity Field (TSRVF), to map non-Euclidean shape sequences into a Euclidean time series. It then uses PCA to reduce Euclidean dimensions and imposes a Vector Auto-Regression (VAR) model on the resulting low-dimensional time series. Finally, it presents some results from VAR-based statistical analysis: estimation of model parameters and diagnostics, synthesis of new shape sequences, and predictions of future shapes given past shapes.

This paper presents details of the effects of shorting along the width of the air-suspended rectangular microstrip antenna (RMSA). The entire width of the patch is shorted using a number of shorting pins and as the number of shorting pins increases, the gain, bandwidth, and resonance frequency of the antenna increase. By decreasing the shorting width, the antenna can be made more compact. For the shorted compact air-suspended RMSA the effects of the finite ground plane on its performance have been analyzed. When the geometric parameters of the ground plane are changed, it affects the gain, bandwidth, resonance frequency, and back lobe radiation. Various graphs have been presented to choose the optimum performance depending on the size constraints.

Human brain functional connectivity (FC) is often measured as the similarity of functional MRI responses across brain regions when a brain is either resting or performing a task. This paper aims to statistically analyze the dynamic nature of FC by representing the collective time-series data, over a set of brain regions, as a trajectory on the space of covariance matrices, or symmetric-positive definite matrices (SPDMs). We use a recently developed metric on the space of SPDMs for quantifying differences across FC observations, and for clustering and classification of FC trajectories. To facilitate large scale and high-dimensional data analysis, we propose a novel, metric-based dimensionality reduction technique to reduce data from large SPDMs to small SPDMs. We illustrate this comprehensive framework using data from the Human Connectome Project (HCP) database for multiple subjects and tasks, with task classification rates that match or outperform state-of-the-art techniques.

Visual observations of dynamic phenomena, such as human actions, are often represented as sequences of smoothly-varying features. In cases where the feature spaces can be structured as Riemannian manifolds, the corresponding representations become trajectories on manifolds. Analysis of these trajectories is challenging due to non-linearity of underlying spaces and high-dimensionality of trajectories. In vision problems, given the nature of physical systems involved, these phenomena are better characterized on a low-dimensional manifold compared to the space of Riemannian trajectories. For instance, if one does not impose physical constraints of the human body, in data involving human action analysis, the resulting representation space will have highly redundant features. Learning an effective, low-dimensional embedding for action representations will have a huge impact in the areas of search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in the euclidean space, but its extension to Riemannian trajectories is non-trivial and remains unexplored. The difficulty lies in inherent non-linearity of the domain and temporal variability of actions that can distort any traditional metric between trajectories. To overcome these issues, we use the framework based on transported square-root velocity fields (TSRVF); this framework has several desirable properties, including a rate-invariant metric and vector space representations. We propose to learn an embedding such that each action trajectory is mapped to a single point in a low-dimensional euclidean space, and the trajectories that differ only in temporal rates map to the same point. We utilize the TSRVF representation, and accompanying statistical summaries of Riemannian trajectories, to extend existing coding methods such as PCA, KSVD and Label Consistent KSVD to Riemannian trajectories or more generally to Riemannian functions. We show that such coding efficiently captures trajectories in applications such as action recognition, stroke rehabilitation, visual speech recognition, clustering and diverse sequence sampling. Using this framework, we obtain state-of-the-art recognition results, while reducing the dimensionality/ complexity by a factor of 100-250x. Since these mappings and codes are invertible, they can also be used to interactively-visualize Riemannian trajectories and synthesize actions.

Recent developments in elastic shape analysis (ESA) are motivated by the fact that it provides a comprehensive framework for simultaneous registration, deformation, and comparison of shapes. These methods achieve computational efficiency using certain square-root representations that transform invariant elastic metrics into euclidean metrics, allowing for the application of standard algorithms and statistical tools. For analyzing shapes of embeddings of in , Jermyn et al. [1] introduced square-root normal fields (SRNFs), which transform an elastic metric, with desirable invariant properties, into the metric. These SRNFs are essentially surface normals scaled by square-roots of infinitesimal area elements. A critical need in shape analysis is a method for inverting solutions (deformations, averages, modes of variations, etc.) computed in SRNF space, back to the original surface space for visualizations and inferences. Due to the lack of theory for understanding SRNF maps and their inverses, we take a numerical approach, and derive an efficient multiresolution algorithm, based on solving an optimization problem in the surface space, that estimates surfaces corresponding to given SRNFs. This solution is found to be effective even for complex shapes that undergo significant deformations including bending and stretching, e.g., human bodies and animals. We use this inversion for computing elastic shape deformations, transferring deformations, summarizing shapes, and for finding modes of variability in a given collection, while simultaneously registering the surfaces. We demonstrate the proposed algorithms using a statistical analysis of human body shapes, classification of generic surfaces, and analysis of brain structures.

This paper develops a formal test to compare populations of shapes of objects given these population samples. The setup involves sets of shapes extracted from different images, and the goal is to equate the underlying populations and not just the individual shapes. While past works have successfully derived shape metrics and individual shape models, the problem of comparing populations using sampled shapes remains relatively unexplored. We use a novel combination of tools - elastic shape analysis of planar closed curves, spherical MDS for dimension reduction, kernel density-estimation on a unit sphere, the Fisher-Rao distance for density comparisons, and, finally, a bootstrap technique for performing a two-sample test - to reach a novel solution for population comparison. We demonstrate these ideas using some practical datasets: image-based biological cell comparisons, nano-manufacturing analysis, coffee-bean classification, and so on. This method is found to be useful in discriminating between shape sets, even those containing visually similar shapes.

We present a novel framework to analyze hand force signals and to capture their key spatio-temporal patterns in order to characterize Rheumatoid Arthritis. We introduce a new continuous representation of hand force and derive optimal intra-class alignments using the notion of Karcher means on the quotient space under the action of the warping group. We apply this idea to temporally register hand force signal data using non-linear time warping. As a result, the original signals are separated into their phase and amplitude components. To capture the amplitude and phase variabilities in force functions we compute the dominant eigenfunctions of the covariance operator using functional principal component analysis. Finally, we use support vector machine classifiers to learn priors from current state-of-the-art features and additional features derived from our functional data analysis framework. The experimental results demonstrate that the proposed framework generates clinically relevant features leading to state-of-the-art classification performance.

We study the problem of classifying actions of human subjects using depth movies generated by Kinect or other depth sensors. Representing human body as dynamical skeletons, we study the evolution of their (skeletons’) shapes as trajectories on Kendall’s shape manifold. The action data is typically corrupted by large variability in execution rates within and across subjects and, thus, causing major problems in statistical analyses. To address that issue, we adopt a recently-developed framework of Su et al. [1], [2] to this problem domain. Here, the variable execution rates correspond to re-parameterizations of trajectories, and one uses a parameterization-invariant metric for aligning, comparing, averaging, and modeling trajectories. This is based on a combination of transported square-root vector fields (TSRVFs) of trajectories and the standard Euclidean norm, that allows computational efficiency. We develop a comprehensive suite of computational tools for this application domain: smoothing and denoising skeleton trajectories using median filtering, up- and down-sampling actions in time domain, simultaneous temporal-registration of multiple actions, and extracting invertible Euclidean representations of actions. Due to invertibility these Euclidean representations allow both discriminative and generative models for statistical analysis. For instance, they can be used in a SVM-based classification of original actions, as demonstrated here using MSR Action-3D, MSR Daily Activity and 3D Action Pairs datasets. Using only the skeletal information, we achieve state-of-the-art classification results on these datasets.

This paper describes a novel framework for computing geodesic paths in shape spaces of spherical surfaces under an elastic Riemannian metric. The novelty lies in defining this Riemannian metric directly on the quotient (shape) space, rather than inheriting it from pre-shape space, and using it to formulate a path energy that measures only the normal components of velocities along the path. In other words, this paper defines and solves for geodesics directly on the shape space and avoids complications resulting from the quotient operation. This comprehensive framework is invariant to arbitrary parameterizations of surfaces along paths, a phenomenon termed as gauge invariance. Additionally, this paper makes a link between different elastic metrics used in the computer science literature on one hand, and the mathematical literature on the other hand, and provides a geometrical interpretation of the terms involved. Examples using real and simulated 3D objects are provided to help illustrate the main ideas.

Human activities observed from visual sensors often give rise to a sequence of smoothly varying features. In many cases, the space of features can be formally defined as a manifold, where the action becomes a trajectory on the manifold. Such trajectories are high dimensional in addition to being non-linear, which can severely limit computations on them. We also argue that by their nature, human actions themselves lie on a much lower dimensional manifold compared to the high dimensional feature space. Learning an accurate low dimensional embedding for actions could have a huge impact in the areas of efficient search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in ℝn, but its extension to trajectories on Riemannian manifolds is non-trivial and has remained unexplored. The challenge arises due to the inherent non-linearity, and temporal variability that can significantly distort the distance metric between trajectories. To address these issues we use the transport square-root velocity function (TSRVF) space, a recently proposed representation that provides a metric which has favorable theoretical properties such as invariance to group action. We propose to learn the low dimensional embedding with a manifold functional variant of principal component analysis (mfPCA). We show that mf-PCA effectively models the manifold trajectories in several applications such as action recognition, clustering and diverse sequence sampling while reducing the dimensionality by a factor of ~ 250×. The mfPCA features can also be reconstructed back to the original manifold to allow for easy visualization of the latent variable space.

Graph-theoretical methods have successfully provided semantic and structural interpretations of images and videos. A recent paper introduced a pattern-theoretic approach that allows construction of flexible graphs for representing interactions of actors with objects and inference is accomplished by an efficient annealing algorithm. Actions and objects are termed generators and their interactions are termed bonds; together they form high-probability configurations, or interpretations, of observed scenes. This work and other structural methods have generally been limited to analyzing short videos involving isolated actions. Here we provide an extension that uses additional temporal bonds across individual actions to enable semantic interpretations of longer videos. Longer temporal connections improve scene interpretations as they help discard (temporally) local solutions in favor of globally superior ones. Using this extension, we demonstrate improvements in understanding longer videos, compared to individual interpretations of non-overlapping time segments. We verified the success of our approach by generating interpretations for more than 700 video segments from the YouCook data set, with intricate videos that exhibit cluttered background, scenarios of occlusion, viewpoint variations and changing conditions of illumination. Interpretations for long video segments were able to yield performance increases of about 70% and, in addition, proved to be more robust to different severe scenarios of classification errors.

The side information (SI) in Distributed Video Coding (DVC) is estimated using the available decoded frames and exploited for the decoding and reconstruction of other frames. The quality of the SI has a strong impact on the performance of DVC. Here, we propose a new approach that combines both global and local SI to improve coding performance. Since the background pixels in a frame are assigned to global estimation and the foreground objects to local estimation, one needs to estimate foreground objects in the SI using the backward and forward foreground objects, the background pixels are directly taken from the global SI. Specifically, elastic curves and local motion compensation are used to generate the foreground objects masks in the SI. Experimental results show that, as far as the rate-distortion performance is concerned, the proposed approach can achieve a PSNR improvement of up to 1.39 dB for a group of picture (GOP) size of 2, and up to 4.73 dB for larger GOP sizes, with respect to the reference DISCOVER codec.

Active contour, especially in conjunction with prior-shape models, has become an important tool in image segmentation. However, most contour methods use shape priors based on similarity-shape analysis, i.e. analysis that is invariant to rotation, translation, and scale. In practice, the training shapes used for prior-shape models may be collected from viewing angles different from those for the test images and require invariance to a larger class of transformation. Using an elastic, affine-invariant shape modeling of planar curves, we propose an active contour algorithm in which the training and test shapes can be at arbitrary affine transformations, and the resulting segmentation is robust to perspective skews. We construct a shape space of affine-standardized curves and derive a statistical model for capturing class-specific shape variability. The active contour is then driven by the true gradient of a total energy composed of a data term, a smoothing term, and an affine-invariant shape-prior term. This framework is demonstrated using a number of examples involving the segmentation of occluded or noisy images of targets subject to perspective skew.

In statistical analysis of video sequences for speech recognition, and more generally activity recognition, it is natural to treat temporal evolutions of features as trajectories on Riemannian manifolds. However, different evolution patterns result in arbitrary parameterizations of these trajectories. We investigate a recent framework from statistics literature that handles this nuisance variability using a cost function/distance for temporal registration and statistical summarization & modeling of trajectories. It is based on a mathematical representation of trajectories, termed transported square-root vector field (TSRVF), and the L2 norm on the space of TSRVFs. We apply this framework to the problem of speech recognition using both audio and visual components. In each case, we extract features, form trajectories on corresponding manifolds, and compute parametrization-invariant distances using TSRVFs for speech classification. On the OuluVS database the classification performance under metric increases significantly, by nearly 100% under both modalities and for all choices of features. We obtained speaker-dependent classification rate of 70% and 96% for visual and audio components, respectively.

Segmentation of handwritten text into individual characters is an important step in many handwriting recognition tasks. In this paper, we present two segmentation algorithms based on elastic shape analysis of parameterized, planar curves. The shape analysis methodology provides matching, comparison and averaging of handwritten curves in a unified framework, which are very useful tools for designing segmentation algorithms. The first type of segmentation can be performed by splitting a full word into individual characters using a matching function. Another type of segmentation can be obtained by matching parts of the handwritten words to a given individual character. We validate the two proposed algorithms on real handwritten signatures and words coming from the SVC 2004 and the UNIPEN ICROW 2003 datasets. We show that the proposed methods are able to successfully segment text coming from highly variable handwriting styles.

We present a novel framework, based on Germander's pattern theoretic concepts, for high-level interpretation of video activities. This framework allows us to elegantly integrate ontological constraints and machine learning classifiers in one formalism to construct high-level semantic interpretations that describe video activity. The unit of analysis is a generator that could represent either an ontological label as well as a group of features from a video. These generators are linked using bonds with different constraints. An interpretation of a video is a configuration of these connected generators, which results in a graph structure that is richer than conventional graphs used in computer vision. The quality of the interpretation is quantified by an energy function that is optimized using Markov Chain Monte Carlo based simulated annealing. We demonstrate the superiority of our approach over a purely machine learning based approach (SVM) using more than 650 video shots from the You Cook dataset. This dataset is very challenging in terms of complexity of background, presence of camera motion, object occlusion, clutter, and actor variability. We find significantly improved performance in nearly all cases. Our results show that the pattern theory inference process is able to construct the correct interpretation by leveraging the ontological constraints even when the machine learning classifier is poor and the most confident labels are wrong.

Current techniques for shape analysis tend to seek invariance to similarity transformations (rotation, translation, and scale), but certain imaging situations require invariance to larger groups, such as affine or projective groups. Here we present a general Riemannian framework for shape analysis of planar objects where metrics and related quantities are invariant to affine and projective groups. Highlighting two possibilities for representing object boundaries-ordered points (or landmarks) and parameterized curves-we study different combinations of these representations (points and curves) and transformations (affine and projective). Specifically, we provide solutions to three out of four situations and develop algorithms for computing geodesics and intrinsic sample statistics, leading up to Gaussian-type statistical models, and classifying test shapes using such models learned from training data. In the case of parameterized curves, we also achieve the desired goal of invariance to re-parameterizations. The geodesics are constructed by particularizing the path-straightening algorithm to geometries of current manifolds and are used, in turn, to compute shape statistics and Gaussian-type shape models. We demonstrate these ideas using a number of examples from shape and activity recognition.

In this paper, we present an automatic approach for facial expression recognition from 3-D video sequences. In the proposed solution, the 3-D faces are represented by collections of radial curves and a Riemannian shape analysis is applied to effectively quantify the deformations induced by the facial expressions in a given subsequence of 3-D frames. This is obtained from the dense scalar field, which denotes the shooting directions of the geodesic paths constructed between pairs of corresponding radial curves of two faces. As the resulting dense scalar fields show a high dimensionality, Linear Discriminant Analysis (LDA) transformation is applied to the dense feature space. Two methods are then used for classification: 1) 3-D motion extraction with temporal Hidden Markov model (HMM) and 2) mean deformation capturing with random forest. While a dynamic HMM on the features is trained in the first approach, the second one computes mean deformations under a window and applies multiclass random forest. Both of the proposed classification schemes on the scalar fields showed comparable results and outperformed earlier studies on facial expression recognition from 3-D video sequences.

We study the problem of joint registration and deformation analysis of endometrial tissue using 3D magnetic resonance imaging (MRI) and 2D trans-vaginal ultrasound (TVUS) measurements. In addition to the different imaging techniques involved in the two modalities, this problem is complicated due to: 1) different patient pose during MRI and TVUS observations, 2) the 3D nature of MRI and 2D nature of TVUS measurements, 3) the unknown intersecting plane for TVUS in MRI volume, and 4) the potential deformation of endometrial tissue during TVUS measurement process. Focusing on the shape of the tissue, we use expert manual segmentation of its boundaries in the two modalities and apply, with modification, recent developments in shape analysis of parametric surfaces to this problem. First, we extend the 2D TVUS curves to generalized cylindrical surfaces through replication, and then we compare them with MRI surfaces using elastic shape analysis. This shape analysis provides a simultaneous registration (optimal reparameterization) and deformation (geodesic) between any two parametrized surfaces. Specifically, it provides optimal curves on MRI surfaces that match with the original TVUS curves. This framework results in an accurate quantification and localization of the deformable endometrial cells for radiologists, and growth characterization for gynecologists and obstetricians. We present experimental results using semi-synthetic data and real data from patients to illustrate these ideas.

Despite many years of research, comparing multiple protein structures simultaneously (multiple structure comparison) is still a challenging problem. Most of the previous studies have focused on similarities among subsets of residues (or atoms) from a group of proteins (local structure alignment) by minimizing some similarity scores based on root mean square deviation (RMSD) of the aligned residues. In this paper, we designed a novel mathematical and statistical framework for multiple global structure comparison (MGSC). Under this framework, a formal geodesic distance is defined for any pair of protein structures and a mean structure can be estimated for a group of protein structures. The multiple structure comparison is then conveniently performed by comparing each individual structure with the mean structure. The formal distance facilitates consistent and accurate clustering of protein structures. An efficient clustering algorithm was designed based on the developed method. Probability models can be built for groups of protein structures and used in hypothesis testing. A robust outlier detection algorithm was designed to illustrate the potential applications of the framework.

Statistical shape analysis develops methods for comparisons, deformations, summarizations, and modeling of shapes in given data sets. These tasks require a fundamental tool called parallel transport of tangent vectors along arbitrary paths. This tool is essential for: (1) computation of geodesic paths using either shooting or path-straightening method, (2) transferring deformations across objects, and (3) modeling of statistical variability in shapes. Using the square-root normal field (SRNF) representation of parameterized surfaces, we present a method for transporting deformations along paths in the shape space. This is difficult despite the underlying space being a vector space because the chosen (elastic) Riemannian metric is non-standard. Using a finite-basis for representing SRNFs of shapes, we derive expressions for Christoffel symbols that enable parallel transports. We demonstrate this framework using examples from shape analysis of parameterized spherical surfaces, in the three contexts mentioned above.

The shapes of plant leaves are of great importance to plant biologists and botanists, as they can help in distinguishing plant species, measuring their health, analyzing their growth patterns, and understanding relations between various species. We propose a statistical model that uses the Squared Root Velocity Function representation and a Riemannian elastic metric to model the observed variability in the shape of plant leaves. We show that under this representation, one can compute sample means and principal modes of variations and can characterize the observed shapes using probability models, such as Gaussians, on the tangent spaces at the sample means. The approach is fully automatic and does not require precomputing correspondences between the shapes. We validate these statistical models by analyzing their classification performance on standard benchmarks and show their utility as generative models for random sampling.

Abstract We argue that full surface correspondence (registration) and optimal deformations (geodesics) are two related problems and propose a framework that solves them simultaneously. We build on the Riemannian shape analysis of anatomical and star‐shaped surfaces of Kurtek et al. and focus on articulated complex shapes that undergo elastic deformations and that may contain missing parts. Our core contribution is the re‐formulation of Kurtek et al.'s approach as a constrained optimization over all possible re‐parameterizations of the surfaces, using a sparse set of corresponding landmarks. We introduce a landmark‐constrained basis, which we use to numerically solve this optimization and therefore establish full surface registration and geodesic deformation between two surfaces. The length of the geodesic provides a measure of dissimilarity between surfaces. The advantages of this approach are: (1) simultaneous computation of full correspondence and geodesic between two surfaces, given a sparse set of matching landmarks (2) ability to handle more comprehensive deformations than nearly isometric, and (3) the geodesics and the geodesic lengths can be further used for symmetrizing 3D shapes and for computing their statistical averages. We validate the framework on challenging cases of large isometric and elastic deformations, and on surfaces with missing parts. We also provide multiple examples of averaging and symmetrizing 3D models.

We propose a novel geometric framework for analyzing 3D faces, with the specific goals of comparing, matching, and averaging their shapes. Here we represent facial surfaces by radial curves emanating from the nose tips and use elastic shape analysis of these curves to develop a Riemannian framework for analyzing shapes of full facial surfaces. This representation, along with the elastic Riemannian metric, seems natural for measuring facial deformations and is robust to challenges such as large facial expressions (especially those with open mouths), large pose variations, missing parts, and partial occlusions due to glasses, hair, and so on. This framework is shown to be promising from both--empirical and theoretical--perspectives. In terms of the empirical evaluation, our results match or improve upon the state-of-the-art methods on three prominent databases: FRGCv2, GavabDB, and Bosphorus, each posing a different type of challenge. From a theoretical perspective, this framework allows for formal statistical inferences, such as the estimation of missing facial parts using PCA on tangent spaces and computing average shapes.

We present a Riemannian framework for analyzing signals and images in a manner that is invariant to their level of blurriness, under Gaussian blurring. Using a well known relation between Gaussian blurring and the heat equation, we establish an action of the blurring group on image space and define an orthogonal section of this action to represent and compare images at the same blur level. This comparison is based on geodesic distances on the section manifold which, in turn, are computed using a path-straightening algorithm. The actual implementations use coefficients of images under a truncated orthonormal basis and the blurring action corresponds to exponential decays of these coefficients. We demonstrate this framework using a number of experimental results, involving 1D signals and 2D images. As a specific application, we study the effect of blurring on the recognition performance when 2D facial images are used for recognizing people.

We present a Riemannian framework for analyzing shapes of planar contours in which metrics and other analyses are invariant to affine transformations and re-parameterizations of contours. Current methods that are affine invariant are restricted to point sets and do not handle full curves, while methods that analyze parameterized curves are restricted to equivalence under similarity transformation (rigid motion and scale). We construct a pre-shape manifold of standardized curves - curves whose centroid is at the origin, are of unit length, and their x and y coordinates are uncorrelated - and develop a path-straightening technique for computing geodesics on this nonlinear manifold under the elastic Riemannian metric. The removal of the rotation and the re-parameterization groups results in a quotient space, termed affine elastic shape space, and the resulting geodesic paths exhibit an improved matching of features across curves. These geodesics are used for shape comparison, retrieval, and statistical modeling of given curves. Experimental results using both simulated and real data, and an application involving pose-invariant activity recognition, demonstrate the success of this framework.

This paper presents a novel Riemannian framework for shape analysis of parameterized surfaces. In particular, it provides efficient algorithms for computing geodesic paths which, in turn, are important for comparing, matching, and deforming surfaces. The novelty of this framework is that geodesics are invariant to the parameterizations of surfaces and other shape-preserving transformations of surfaces. The basic idea is to formulate a space of embedded surfaces (surfaces seen as embeddings of a unit sphere in IR3) and impose a Riemannian metric on it in such a way that the reparameterization group acts on this space by isometries. Under this framework, we solve two optimization problems. One, given any two surfaces at arbitrary rotations and parameterizations, we use a path-straightening approach to find a geodesic path between them under the chosen metric. Second, by modifying a technique presented in [25], we solve for the optimal rotation and parameterization (registration) between surfaces. Their combined solution provides an efficient mechanism for computing geodesic paths in shape spaces of parameterized surfaces. We illustrate these ideas using examples from shape analysis of anatomical structures and other general surfaces.

We utilize ideas from two growing but disparate ideas in computer vision-shape analysis using tools from differential geometry and feature selection using machine learning-to select and highlight salient geometrical facial features that contribute most in 3-D face recognition and gender classification. First, a large set of geometries curve features are extracted using level sets (circular curves) and streamlines (radial curves) of the Euclidean distance functions of the facial surface; together they approximate facial surfaces with arbitrarily high accuracy. Then, we use the well-known Adaboost algorithm for feature selection from this large set and derive a composite classifier that achieves high performance with a minimal set of features. This greatly reduced set, consisting of some level curves on the nose and some radial curves in the forehead and cheeks regions, provides a very compact signature of a 3-D face and a fast classification algorithm for face recognition and gender selection. It is also efficient in terms of data storage and transmission costs. Experimental results, carried out using the FRGCv2 dataset, yield a rank-1 face recognition rate of 98% and a gender classification rate of 86% rate.

In recent years, RNAs have been found to have diverse functions beyond being a messenger in gene transcription. The functions of non-coding RNAs are determined by their structures. Structure comparison/alignment of RNAs provides an effective means to predict their functions. Despite many previous studies on RNA structure alignment, it is still a challenging problem to predict the function of RNA molecules based on their structure information. In this study, we developed a new RNA structure alignment method based on elastic shape analysis (ESA). ESA treats RNA structures as three dimensional curves and performs flexible alignment between two RNA molecules by bending and stretching one of the molecules to match the other. The amount of bending and stretching is quantified by a formal distance, geodesic distance. Based on ESA, a rigorous mathematical framework can be built for RNA structure comparison. Means and covariances can be computed and probability distributions can be constructed for a group of RNA structures. We further applied the method to predict functions of RNA molecules. Our method achieved good performance when tested on benchmark datasets.

We propose a novel Riemannian framework for comparing signals and images in a manner that is invariant to their levels of blur. This framework uses a log-Fourier representation of signals/images in which the set of all possible Gaussian blurs of a signal, i.e. its orbits under semigroup action of Gaussian blur functions, is a straight line. Using a set of Riemannian metrics under which the group actions are by isometries, the orbits are compared via distances between orbits. We demonstrate this framework using a number of experimental results involving 1D signals and 2D images.

While signal estimation under random amplitudes, phase shifts, and additive noise is studied frequently, the problem of estimating a deterministic signal under random time-warpings has been relatively unexplored. We present a novel framework for estimating the unknown signal that utilizes the action of the warping group to form an equivalence relation between signals. First, we derive an estimator for the equivalence class of the unknown signal using the notion of Karcher mean on the quotient space of equivalence classes. This step requires the use of Fisher-Rao Riemannian metric and a square-root representation of signals to enable computations of distances and means under this metric. Then, we define a notion of the center of a class and show that the center of the estimated class is a consistent estimator of the underlying unknown signal. This estimation algorithm has many applications: (1)registration/alignment of functional data, (2) separation of phase/amplitude components of functional data, (3) joint demodulation and carrier estimation, and (4) sparse modeling of functional data. Here we demonstrate only (1) and (2): Given signals are temporally aligned using nonlinear warpings and, thus, separated into their phase and amplitude components. The proposed method for signal alignment is shown to have state of the art performance using Berkeley growth, handwritten signatures, and neuroscience spike train data.

This paper introduces a square-root velocity (SRV) representation for analyzing shapes of curves in euclidean spaces under an elastic metric. In this SRV representation, the elastic metric simplifies to the IL(2) metric, the reparameterization group acts by isometries, and the space of unit length curves becomes the unit sphere. The shape space of closed curves is the quotient space of (a submanifold of) the unit sphere, modulo rotation, and reparameterization groups, and we find geodesics in that space using a path straightening approach. These geodesics and geodesic distances provide a framework for optimally matching, deforming, and comparing shapes. These ideas are demonstrated using: 1) shape analysis of cylindrical helices for studying protein structure, 2) shape analysis of facial curves for recognizing faces, 3) a wrapped probability distribution for capturing shapes of planar closed curves, and 4) parallel transport of deformations for predicting shapes from novel poses.

In this paper, we examine image and video-based recognition applications where the underlying models have a special structure—the linear subspace structure. We discuss how commonly used parametric models for videos and image sets can be described using the unified framework of Grassmann and Stiefel manifolds. We first show that the parameters of linear dynamic models are finite-dimensional linear subspaces of appropriate dimensions. Unordered image sets as samples from a finite-dimensional linear subspace naturally fall under this framework. We show that an inference over subspaces can be naturally cast as an inference problem on the Grassmann manifold. To perform recognition using subspace-based models, we need tools from the Riemannian geometry of the Grassmann manifold. This involves a study of the geometric properties of the space, appropriate definitions of Riemannian metrics, and definition of geodesics. Further, we derive statistical modeling of inter and intraclass variations that respect the geometry of the space. We apply techniques such as intrinsic and extrinsic statistics to enable maximum-likelihood classification. We also provide algorithms for unsupervised clustering derived from the geometry of the manifold. Finally, we demonstrate the improved performance of these methods in a wide variety of vision applications such as activity recognition, video-based face recognition, object recognition from image sets, and activity-based video clustering.

Comparison of protein structures is important for revealing the evolutionary relationship among proteins, predicting protein functions and predicting protein structures. Many methods have been developed in the past to align two or multiple protein structures. Despite the importance of this problem, rigorous mathematical or statistical frameworks have seldom been pursued for general protein structure comparison. One notable issue in this field is that with many different distances used to measure the similarity between protein structures, none of them are proper distances when protein structures of different sequences are compared. Statistical approaches based on those non-proper distances or similarity scores as random variables are thus not mathematically rigorous. In this work, we develop a mathematical framework for protein structure comparison by treating protein structures as three-dimensional curves. Using an elastic Riemannian metric on spaces of curves, geodesic distance, a proper distance on spaces of curves, can be computed for any two protein structures. In this framework, protein structures can be treated as random variables on the shape manifold, and means and covariance can be computed for populations of protein structures. Furthermore, these moments can be used to build Gaussian-type probability distributions of protein structures for use in hypothesis testing. The covariance of a population of protein structures can reveal the population-specific variations and be helpful in improving structure classification. With curves representing protein structures, the matching is performed using elastic shape analysis of curves, which can effectively model conformational changes and insertions/deletions. We show that our method performs comparably with commonly used methods in protein structure classification on a large manually annotated data set.

We consider 3-D brain structures as continuous parameterized surfaces and present a metric for their comparisons that is invariant to the way they are parameterized. Past comparisons of such surfaces involve either volume deformations or nonrigid matching under fixed parameterizations of surfaces. We propose a new mathematical representation of surfaces, called q-maps, such that L² distances between such maps are invariant to re-parameterizations. This property allows for removing the parameterization variability by optimizing over the re-parameterization group, resulting in a proper parameterization-invariant distance between shapes of surfaces. We demonstrate this method in shape analysis of multiple brain structures, for 34 subjects in the Detroit Fetal Alcohol and Drug Exposure Cohort study, which results in a 91% classification rate for attention deficit hyperactivity disorder cases and controls. This method outperforms some existing techniques such as spherical harmonic point distribution model (SPHARM-PDM) or iterative closest point (ICP).

In this paper we explore the use of shapes of elastic radial curves to model 3D facial deformations, caused by changes in facial expressions. We represent facial surfaces by indexed collections of radial curves on them, emanating from the nose tips, and compare the facial shapes by comparing the shapes of their corresponding curves. Using a past approach on elastic shape analysis of curves, we obtain an algorithm for comparing facial surfaces. We also introduce a quality control module which allows our approach to be robust to pose variation and missing data. Comparative evaluation using a common experimental setup on GAVAB dataset, considered as the most expression-rich and noise-prone 3D face dataset, shows that our approach outperforms other state-of-the-art approaches.

In this paper we introduce a novel Riemannian framework for shape analysis of parameterized surfaces. We derive a distance function between any two surfaces that is invariant to rigid motion, global scaling, and re-parametrization. It is the last part that presents the main difficulty. Our solution to this problem is twofold: (1) we define a special representation, called a q-map, to represent each surface, and (2) we develop a gradient-based algorithm to optimize over different re-parameterizations of a surface. The second step is akin to deforming the mesh on a fixed surface to optimize its placement. (This is different from the current methods that treat the given meshes as fixed.) Under the chosen representation, with the L2metric, the action of the re-parametrization group is by isometries. This results in, to our knowledge, the first Riemannian distance between parameterized surfaces to have all the desired invariances. We demonstrate this framework with several examples using some toy shapes, and real data with anatomical structures, and cropped facial surfaces. We also successfully demonstrate clustering and classification of these objects under the proposed metric.

Subspace tracking methods are widespread in signal and image processing. To reduce the influence of perturbations or outliers on the measurements, some authors have used a stochastic piecewise constant velocity model on the Grassmann manifold. This paper presents an efficient way to simulate such a model using a particular representation of the Grassmann manifold. By doing so, we can reduce the spatial and time complexity of filtering techniques based on this model. We also propose an approximation of this system which can be computed in a finite number of operations and show similar results if the subspace variation is slow.

We investigate the problem of facial expression recognition using 3D face data. Our approach is based on local shape analysis of several relevant regions of a given face scan. These regions or patches from facial surfaces are extracted and represented by sets of closed curves. A Riemannian framework is used to derive the shape analysis of the extracted patches. The applied framework permits to calculate a similarity (or dissimilarity) distances between patches, and to compute the optimal deformation between them. Once calculated, these measures are employed as inputs to a commonly used classification techniques such as AdaBoost and Support Vector Machines (SVM). A quantitative evaluation of our novel approach is conducted on a subset of the publicly available BU-3DFE database.

We present a novel statistical framework for detecting pre-determined shape classes in 2D cluttered point clouds, which are in turn extracted from images. In this model based approach, we use a 1D Poisson process for sampling points on shapes, a 2D Poisson process for points from background clutter, and an additive Gaussian model for noise. Combining these with a past stochastic model on shapes of continuous 2D contours, and optimization over unknown pose and scale, we develop a generalized likelihood ratio test for shape detection. We demonstrate the efficiency of this method and its robustness to clutter using both simulated and real data.

The seven papers in this special section focus on shape analysis and its application in image understanding.

In this paper we explore the use of shapes of noses for performing partial human biometrics. The basic idea is to represent nasal surfaces using indexed collections of iso-curves, and to analyze shapes of noses by comparing their corresponding curves. We extend past work in Riemannian analysis of shapes of closed curves in R3to obtain a similar Riemannian analysis for nasal surfaces. In particular, we obtain algorithms for computing geodesics, computing statistical means, and stochastic clustering. We demonstrate these ideas in two application contexts : authentication and identification. We evaluate performances on a large database involving 2000 scans from FRGC v2 database, and present a hierarchical organization of nose databases to allow for efficient searches.

We study the problem of identifying shape classes in point clouds. These clouds contain sampled points along contours and are corrupted by clutter and observation noise. Taking an analysis-by-synthesis approach, we simulate high-probability configurations of sampled contours using models learned from training data to evaluate the given test data. To facilitate simulations, we develop statistical models for sources of (nuisance) variability: 1) shape variations within classes, 2) variability in sampling continuous curves, 3) pose and scale variability, 4) observation noise, and 5) points introduced by clutter. The variability in sampling closed curves into finite points is represented by positive diffeomorphisms of a unit circle. We derive probability models on these functions using their square-root forms and the Fisher-Rao metric. Using a Monte Carlo approach, we simulate configurations from a joint prior on the shape-sample space and compare them to the data using a likelihood function. Average likelihoods of simulated configurations lead to estimates of posterior probabilities of different classes and, hence, Bayesian classification.

Pattern recognition in video is a challenging task because of the multitude of spatio-temporal variations that occur in different videos capturing the exact same event. While traditional pattern-theoretic approaches account for the spatial changes that occur due to lighting and pose, very little has been done to address the effect of temporal rate changes in the executions of an event. In this paper, we provide a systematic model-based approach to learn the nature of such temporal variations (time warps) while simultaneously allowing for the spatial variations in the descriptors. We illustrate our approach for the problem of action recognition and provide experimental justification for the importance of accounting for rate variations in action recognition. The model is composed of a nominal activity trajectory and a function space capturing the probability distribution of activity-specific time warping transformations. We use the square-root parameterization of time warps to derive geodesics, distance measures, and probability distributions on the space of time warping functions. We then design a Bayesian algorithm which treats the execution rate function as a nuisance variable and integrates it out using Monte Carlo sampling, to generate estimates of class posteriors. This approach allows us to learn the space of time warps for each activity while simultaneously capturing other intra- and interclass variations. Next, we discuss a special case of this approach which assumes a uniform distribution on the space of time warping functions and show how computationally efficient inference algorithms may be derived for this special case. We discuss the relative advantages and disadvantages of both approaches and show their efficacy using experiments on gait-based person identification and activity recognition.

We introduce a framework for modeling spatial patterns of shapes formed by multiple objects in an image. Our approach is graph-based where each node denotes an object and attributes of a node consist of that object's shape, position, orientation, and scale. Neighboring node are connected by edges, and they are allowed to interact in terms of their attributes/features. Similar to a Markov random field, but now applied to more sophisticated features space, the interactions are governed by energy functionals that can be internal or external. The internal energies, composed entirely of interactions between nodes, may include similarity between shapes and pose. The external energies, composed of outside influences, may include the data-likelihood term and the a-priori information about the shapes and the locations of the objects.

We propose a pattern theoretic approach for studying variability in shapes of facial surfaces. Our idea is to impose a specific, yet natural, coordinate system, called a curvilinear coordinate system, on facial surfaces. In this system, one coordinate xi1measures the distance of a point from the tip of the nose and its level curves are called the facial curves. The other coordinate xi2measures distances along these curves; level curves of this coordinate are orthogonal to the facial curves. To compare two facial surfaces we use elastic deformations that use stretching, shrinking, and bending to optimally register points across two surfaces. We will demonstrate this idea on Florida State University (FSU) 3D face database.

MOTIVATION: Genomes contain biologically significant information that extends beyond that encoded in genes. Some of this information relates to various short dispersed repeats distributed throughout the genome. The goal of this work was to combine tools for detection of statistically significant dispersed repeats in DNA sequences with tools to aid development of hypotheses regarding their possible physiological functions in an easy-to-use web-based environment. RESULTS: Ab Initio Motif Identification Environment (AIMIE) was designed to facilitate investigations of dispersed sequence motifs in prokaryotic genomes. We used AIMIE to analyze the Escherichia coli and Haemophilus influenzae genomes in order to demonstrate the utility of the new environment. AIMIE detected repeated extragenic palindrome (REP) elements, CRISPR repeats, uptake signal sequences, intergenic dyad sequences and several other over-represented sequence motifs. Distributional patterns of these motifs were analyzed using the tools included in AIMIE. AVAILABILITY: AIMIE and the related software can be accessed at our web site http://www.cmbl.uga.edu/software.html.

We propose a novel representation of continuous, closed curves in ℝ(n) that is quite efficient for analyzing their shapes. We combine the strengths of two important ideas - elastic shape metric and path-straightening methods -in shape analysis and present a fast algorithm for finding geodesics in shape spaces. The elastic metric allows for optimal matching of features while path-straightening provides geodesics between curves. Efficiency results from the fact that the elastic metric becomes the simple (2) metric in the proposed representation. We present step-by-step algorithms for computing geodesics in this framework, and demonstrate them with 2-D as well as 3-D examples.

Applications in computer vision involve statistically analyzing an important class of constrained, non-negative functions, including probability density functions (in texture analysis), dynamic time-warping functions (in activity analysis), and re-parametrization or non-rigid registration functions (in shape analysis of curves). For this one needs to impose a Riemannian structure on the spaces formed by these functions. We propose a "spherical" version of the Fisher-Rao metric that provides closed-form expressions for geodesies and distances, and allows fast computation of sample statistics. To demonstrate this approach, we present an application in planar shape classification.

Mathematical and statistical modeling of biological growth is an important problem in medical diagnostics. Here, we seek tools to analyze changes in anatomical parts using images collected over time. We introduce a structured model, called Growth by Random Iterated Diffeomorphisms (GRID), that treats a cumulative growth deformation as a composition of several elementary deformations. Each elementary deformation applies to a small region by capturing deformation local to that region and is characterized by a seed and a radial deformation pattern around that seed. These GRID variables--seed locations and radial deformation patterns---are estimated from observed images in two steps: 1) estimate a cumulative deformation over an observation interval; 2) estimate GRID variables using maximum-likelihood criterion from this estimated cumulative deformation. We demonstrate this framework using an MRI image data of a rat's brain growth. For future statistical analysis, we propose a time-varying Poisson process for the seed placements and a random drawing from a predetermined catalog of deformations for the radial deformation patterns.

Recognition of human beings using shapes of their full facial surfaces is a difficult problem. Our approach is to approximate a facial surface using a collection of (closed) facial curves, and to compare surfaces by comparing their corresponding curves. The differences between shapes of curves are quantified using lengths of geodesic paths between them on a pre-defined curve shape space. The metric for comparing facial surfaces is a composition of the metric involving individual facial curves. These ideas are demonstrated in the context of face recognition using the nearest-neighbor classifier

We study shapes of facial surfaces for the purpose of face recognition. The main idea is to 1) represent surfaces by unions of level curves, called facial curves, of the depth function and 2) compare shapes of surfaces implicitly using shapes of facial curves. The latter is performed using a differential geometric approach that computes geodesic lengths between closed curves on a shape manifold. These ideas are demonstrated using a nearest-neighbor classifier on two 3D face databases: Florida State University and Notre Dame, highlighting a good recognition performance.

We describe an approach for statistical analysis of shapes of closed curves using tools from differential geometry. This approach uses geodesic paths to define a metric on shape space, that is used to compare shapes, to compute intrinsic statistics for a set of shapes, and to define probability models on shape spaces. We demonstrate this approach using: (i) interpolation of heart-wall boundaries in echocardiographic image sequences; and (ii) a study of shapes of human silhouettes in infrared surveillance images.

Three dimensional objects viewed as surfaces or volumes embedded in /spl Ropf//sup 3/, are usually sampled along the z-dimension by planes for rendering or modeling purposes. The resulting intersections are curves or planar shapes which may in turn be modeled for parsimony of representation. Each curve or planar shape may be viewed as a point in a high dimensional manifold, thereby providing the notion of interpolation between two curves or two points on this manifold to reconstruct the subsurface that lies between the two slices. We exploit some recent results in formulating this interpolation problem as an optimization problem in /spl Ropf//sup 3/ to yield a simple interpolating spline, known as elasticae, which when evaluated at intermediate points yields curves which can in turn be instrumental in 3D reconstruction. The approach is particularly suited for interpolation between MRI slices and for modeling and reconstruction of 3D shapes.

Using a differential-geometric treatment of planar shapes, we present tools for: 1) hierarchical clustering of imaged objects according to the shapes of their boundaries, 2) learning of probability models for clusters of shapes, and 3) testing of newly observed shapes under competing probability models. Clustering at any level of hierarchy is performed using a mimimum variance type criterion criterion and a Markov process. Statistical means of clusters provide shapes to be clustered at the next higher level, thus building a hierarchy of shapes. Using finite-dimensional approximations of spaces tangent to the shape space at sample means, we (implicitly) impose probability models on the shape space, and results are illustrated via random sampling and classification (hypothesis testing). Together, hierarchical clustering and hypothesis testing provide an efficient framework for shape retrieval. Examples are presented using shapes and images from ETH, Surrey, and AMCOM databases.

Studies of hyperspectral images point to non-Gaussian statistics of pixels values, and consequently, standard Gaussian models may not perform well in hyperspectral image analysis. This paper presents novel probability models that capture non-Gaussian statistics of hyperspectral images, and uses them in automated classification of terrain sites. After the data are preprocessed using standard dimension-reduction tools, we use: 1) a nonparametric density estimate for capturing spectral variation at each site and 2) two parametric families-generalized Laplacian and Bessel K form-to capture non-Gaussian statistics of difference pixels. Assuming an Ising-type prior on site labels, favoring a smooth classification, we formulate a Markov random field-maximum a posteriori estimation problem and use a Markov chain to estimate site classifications. Results are presented from application of this framework to Washington, DC Mall and Indian Springs rural area datasets.

For analyzing shapes of planar, closed curves, we propose differential geometric representations of curves using their direction functions and curvature functions. Shapes are represented as elements of infinite-dimensional spaces and their pairwise differences are quantified using the lengths of geodesics connecting them on these spaces. We use a Fourier basis to represent tangents to the shape spaces and then use a gradient-based shooting method to solve for the tangent that connects any two shapes via a geodesic. Using the Surrey fish database, we demonstrate some applications of this approach: 1) interpolation and extrapolations of shape changes, 2) clustering of objects according to their shapes, 3) statistics on shape spaces, and 4) Bayesian extraction of shapes in low-quality images.

Although linear representations are frequently used in image analysis, their performances are seldom optimal in specific applications. This paper proposes a stochastic gradient algorithm for finding optimal linear representations of images for use in appearance-based object recognition. Using the nearest neighbor classifier, a recognition performance function is specified and linear representations that maximize this performance are sought. For solving this optimization problem on a Grassmann manifold, a stochastic gradient algorithm utilizing intrinsic flows is introduced. Several experimental results are presented to demonstrate this algorithm.

Simplicity of linear representations (of images) makes them a popular tool in imaging analysis applications such as object recognition and image classification. Although several linear representations, namely PCA (principal component analysis), ICA, and FDA (Fisher discriminant analysis), have frequently been used, these representations are generally far from optimal in terms of actual application performance. We argue that representations should be chosen with respect to the application and the databases involved. Fixing an application, say object recognition, and assuming that recognition performance is computable for any linear basis (given a classifier and a database), we propose a Monte Carlo simulated annealing method that leads to optimal linear representations by maximizing the recognition performance over all fixed-rank subspaces. We illustrate this method on two popular databases.

Low dimensional representations of images impose equivalence relations in the image space; the induced equivalence class of an image is named as its intrinsic generalization. The intrinsic generalization of a representation provides a novel way to measure its generalization and leads to more fundamental insights than the commonly used recognition performance, which is heavily influenced by the choice of training and test data. We demonstrate the limitations of linear subspace representations by sampling their intrinsic generalization, and propose a nonlinear representation that overcomes these limitations. The proposed representation projects images nonlinearly into the marginal densities of their filter responses, followed by linear projections of the marginals. We have used experiments on large datasets to show that the representations that have better intrinsic generalization also lead to a better recognition performance.

We present a geometric approach to statistical shape analysis of closed curves in images. The basic idea is to specify a space of closed curves satisfying given constraints, and exploit the differential geometry of this space to solve optimization and inference problems. We demonstrate this approach by: (i) defining and computing statistics of observed shapes, (ii) defining and learning a parametric probability model on shape space, and (iii) designing a binary hypothesis test on this space.

In this paper we study and compare the recognition performance of subspaces in two different spaces, namely the image space and spectral histogram space. In image space, each image is represented as a long vector and in the spectral histogram space, each image is represented by its histograms of the convolved images with a chosen bank of filters. Spectral histogram space is a nonlinear transformation of the image space. First principal components and independent components in the spaces are studied. Then we study different subspaces by connecting the known subspaces through geodesic curves in the projection space. Our preliminary results show the recognition performance depends more on which space to use than the different subspaces in a given space. This suggests the need to study different spaces for recognition purpose.

Seeking probability models for images, we employ a spectral approach where the images are decomposed using bandpass filters and probability models are imposed on the filter outputs (also called spectral components). We employ a (two-parameter) family of probability densities, called Bessel K forms, for modeling the marginal densities of the spectral components, and demonstrate their fit to the observed histograms for video, infrared, and range images. Motivated by object-based models for image analysis, a relationship between the Bessel parameters and the imaged objects is established. Using L/sup 2/-metric on the set of Bessel K forms, we propose a pseudometric on the image space for quantifying image similarities/differences. Some applications, including clutter classification and pruning of hypotheses for target recognition, are presented.

We present a framework for modeling background clutter in natural images. Assuming that: (i) images are made up of 2D (projected) views of 3D (real) objects, and (ii) certain simplifying conditions hold, we present an analytical density for the derivatives of the natural images. This expression is shown to match well with the observed densities (histograms).

We propose a new algorithm for image segmentation. We use the spectral histogram, which is a vector consisting of marginal distributions of responses from chosen filters as a generic feature for texture as well as intensity images. Motivated by a new segmentation energy functional, we derive an iterative and deterministic approximation algorithm for segmentation. Based on the relationships between different scales and neighboring windows, we also develop an algorithm which can automatically detect homogeneous regions in an input image, which may consist of texture regions. To reduce the boundary uncertainty due to the large spatial window used for spectral histograms, we propose a novel local feature by building precise probability models based on current segmentation results. We have applied our algorithm to intensity, texture, and natural images and obtained good results with accurate texture boundaries.

Spectral components, obtained via bandpass filtering of images, have become important tools in capturing image variability. We present a two-parameter family of probability densities, called the Bessel forms, to model the marginal densities of the spectral components. The two parameters, shape and scale parameters, are used to characterize each spectral component of the image. We derive an L/sup 2/-metric on the space of Bessel forms that leads to a metric on the space of natural images. The strength of these forms/metric is demonstrated via a study of natural clutter images.

We propose a framework for modeling clutter in natural images. Assuming that: 1) images are made up of 2D (projected) views of 3D (real) objects and 2) certain simplifying conditions hold, we derive an analytical density for natural images. This expression is shown to match well with the observed densities (histograms). In addition to deriving multidimensional densities, several extensions are also proposed.

This article investigates the asymptotic performance of Bayesian target recognition algorithms using deformable-template representations. Rigid computer-aided design (CAD) models represent the underlying targets; low-dimensional matrix Lie-groups (rotation and translation) extend them to particular instances. Remote sensors observing the targets are modeled as projective transformations, converting three-dimensional scenes into random images. Bayesian target recognition corresponds to hypothesis selection in the presence of nuisance parameters; its performance is quantified as the Bayes' error. Analytical expressions for this error probability in small noise situations are derived, yielding asymptotic error rates for exponential error probability decay.

Deformable template representations of observed imagery model the variability of target pose via the actions of the matrix Lie groups on rigid templates. In this paper, we study the construction of minimum mean squared error estimators on the special orthogonal group, SO(n), for pose estimation. Due to the nonflat geometry of SO(n), the standard Bayesian formulation of optimal estimators and their characteristics requires modifications. By utilizing Hilbert-Schmidt metric defined on GL(n), a larger group containing SO(n), a mean squared criterion is defined on SO(n). The Hilbert-Schmidt estimate (HSE) is defined to be a minimum mean squared error estimator, restricted to SO(n). The expected error associated with the HSE is shown to be a lower bound, called the Hilbert-Schmidt bound (HSB), on the error incurred by any other estimator. Analysis and algorithms are presented for evaluating the HSE and the HSB in cases of both ground-based and airborne targets.