In this paper we propose and validate a PCA-based respiratory motion model for motion compensation during image-guided cardiac interventions. In a preparatory training phase, a preoperative 3-D segmentation of the coronary arteries is automatically registered with a cardiac gated biplane cineangiogram, and used to build a respiratory motion model. This motion model is subsequently used as a prior within the intraoperative registration process for motion compensation to restrict the search space. Our hypothesis is that the use of this model-constrained registration increases the robustness and registration accuracy, especially for weak data constraints such as low signal-to-noise ratio, the lack of contrast information, or an intraoperative monoplane setting. This allows for reducing radiation exposure without compromising on registration accuracy. Synthetic data as well as phantom and clinical datasets have been used to validate the model-based registration in terms of registration accuracy, robustness and speed. We were able to significantly accelerate the intraoperative registration with a 3-D TRE of less than 2 mm for both monoplane images and intraprocedure settings with missing contrast information based on 2-D guidewire tracking, which makes it feasible for motion correction in clinical procedures.

Level set methods have been widely used in image processing and computer vision. In conventional level set formulations, the level set function typically develops irregularities during its evolution, which may cause numerical errors and eventually destroy the stability of the evolution. Therefore, a numerical remedy, called reinitialization, is typically applied to periodically replace the degraded level set function with a signed distance function. However, the practice of reinitialization not only raises serious problems as when and how it should be performed, but also affects numerical accuracy in an undesirable way. This paper proposes a new variational level set formulation in which the regularity of the level set function is intrinsically maintained during the level set evolution. The level set evolution is derived as the gradient flow that minimizes an energy functional with a distance regularization term and an external energy that drives the motion of the zero level set toward desired locations. The distance regularization term is defined with a potential function such that the derived level set evolution has a unique forward-and-backward (FAB) diffusion effect, which is able to maintain a desired shape of the level set function, particularly a signed distance profile near the zero level set. This yields a new type of level set evolution called distance regularized level set evolution (DRLSE). The distance regularization effect eliminates the need for reinitialization and thereby avoids its induced numerical errors. In contrast to complicated implementations of conventional level set formulations, a simpler and more efficient finite difference scheme can be used to implement the DRLSE formulation. DRLSE also allows the use of more general and efficient initialization of the level set function. In its numerical implementation, relatively large time steps can be used in the finite difference scheme to reduce the number of iterations, while ensuring sufficient numerical accuracy. To demonstrate the effectiveness of the DRLSE formulation, we apply it to an edge-based active contour model for image segmentation, and provide a simple narrowband implementation to greatly reduce computational cost.

We propose a framework for the nonlinear spatiotemporal registration of 4D time-series of images based on the Diffeomorphic Demons (DD) algorithm. In this framework, the 4D spatiotemporal registration is decoupled into a 4D temporal registration, defined as mapping physiological states, and a 4D spatial registration, defined as mapping trajectories of physical points. Our contribution focuses more specifically on the 4D spatial registration that should be consistent over time as opposed to 3D registration that solely aims at mapping homologous points at a given time-point. First, we estimate in each sequence the motion displacement field, which is a dense representation of the point trajectories we want to register. Then, we perform simultaneously 3D registrations of corresponding time-points with the constraints to map the same physical points over time called the trajectory constraints. Under these constraints, we show that the 4D spatial registration can be formulated as a multichannel registration of 3D images. To solve it, we propose a novel version of the Diffeomorphic Demons (DD) algorithm extended to vector-valued 3D images, the Multichannel Diffeomorphic Demons (MDD). For evaluation, this framework is applied to the registration of 4D cardiac computed tomography (CT) sequences and compared to other standard methods with real patient data and synthetic data simulated from a physiologically realistic electromechanical cardiac model. Results show that the trajectory constraints act as a temporal regularization consistent with motion whereas the multichannel registration acts as a spatial regularization. Finally, using these trajectory constraints with multichannel registration yields the best compromise between registration accuracy, temporal and spatial smoothness, and computation times. A prospective example of application is also presented with the spatiotemporal registration of 4D cardiac CT sequences of the same patient before and after radiofrequency ablation (RFA) in case of atrial fibrillation (AF). The intersequence spatial transformations over a cardiac cycle allow to analyze and quantify the regression of left ventricular hypertrophy and its impact on the cardiac function.

Spatial intensity variations caused by illumination changes have been a challenge for image segmentation and many other computer vision tasks. This paper presents a novel method for image segmentation with simultaneous estimation of illumination and reflectance images. The proposed method is based on the composition of an observed scene image with an illumination component and a reflectance component, known as intrinsic images. We define an energy functional in terms of an illumination image, the membership functions of the regions, and the corresponding reflectance constants of the regions in the scene. This energy is convex in each of its variables. By minimizing the energy, image segmentation result is obtained in the form of the membership functions of the regions. The illumination and reflectance components of the observed image are estimated simultaneously as the result of energy minimization. With illumination taken into account, the proposed method is able to segment images with non-uniform intensities caused by spatial variations in illumination. Comparisons with the state-of-the-art piecewise smooth model demonstrate the superior performance of our method.

This paper presents a new image denoising model for real color photo noise removal. Our model is implemented in the hue, saturation and intensity (HSI) space. The hue and saturation denoising are combined and implemented as a complex total variation (TV) diffusion. The intensity denoising is based on a diffusion flow to minimize a new energy functional, which is constructed with intensity component statistics. Besides the common gradient-based edge stopping functions for anisotropic diffusion, specifically for color photo denoising, we incorporate an intensity-based brightness adjusting term in the new energy, which corresponds to the noise disturbance with respect to photo intensity. In addition, we use the gradient vector flow (GVF) as the new diffusion directions for more accurate and robust denoising. Compared with previous diffusion flows only based on regular image gradients, this model provides more accurate image structure and intensity noise characterization for better denoising. Comprehensive quantitative and qualitative experiments on color photos demonstrate the improved performance of the proposed model when compared with 14 recognized approaches and 2 commercial software.

We propose a unified computational framework to build a statistical atlas of the cardiac fiber architecture from diffusion tensor magnetic resonance images (DT-MRIs). We apply this framework to a small database of nine ex vivo canine hearts. An average cardiac fiber architecture and a measure of its variability are computed using most recent advances in diffusion tensor statistics. This statistical analysis confirms the already established good stability of the fiber orientations and a higher variability of the laminar sheet orientations within a given species. The statistical comparison between the canine atlas and a standard human cardiac DT-MRI shows a better stability of the fiber orientations than their laminar sheet orientations between the two species. The proposed computational framework can be applied to larger databases of cardiac DT-MRIs from various species to better establish intraspecies and interspecies statistics on the anatomical structure of cardiac fibers. This information will be useful to guide the adjustment of average fiber models onto specific patients from in vivo anatomical imaging modalities.

The introduction of prior knowledge has greatly enhanced numerous purely low-level driven image processing algorithms. In this work, we focus on the problem of nonrigid image registration. A number of powerful registration criteria have been developed in the last decade, most prominently the criterion of maximum mutual information. Although this criterion provides for good registration results in many applications, it remains a purely low-level criterion. As a consequence, registration results will deteriorate once this low-level information is corrupted, due to noise, partial occlusions or missing image structure. In this paper, we will develop a Bayesian framework that allows to impose statistically learned prior knowledge about the joint intensity distribution into image registration methods. The prior is given by a kernel density estimate on the space of joint intensity distributions computed from a representative set of pre-registered image pairs. This nonparametric prior accurately models previously learned intensity relations between various image modalities and slice locations. Experimental results demonstrate that the resulting registration process is more robust to missing low-level information as it favors intensity correspondences statistically consistent with the learned intensity distributions.

We present a novel feature-based non-rigid image registration algorithm using a small number of automatically extracted points and their associated local salient region features. Our automatic registration is a hybrid approach co-optimizing point-based and image-based terms. Motivated by the paradigm of the TPS-RPM algorithm [6], we develop the RHDM (Robust Hybrid Deformable Matching) algorithm by alternatively optimizing correspondences and transformations for registration. The local salient region features and the geometric features, together with the softassign and deterministic annealing techniques, are used for solving correspondences. Thin-plate splines are used for generating a smooth non-rigid spatial transformation. Our algorithm is built to be extremely robust to feature extraction errors. A new dynamic outlier rejection mechanism is described for rejecting outliers and generating accurate spatial mappings. A local refinement technique is used for correcting non-exactly matched correspondences arising from image noise and irregular deformations. In contrast with the TPS-RPM algorithm, which can handle only outliers in one point set, our algorithm is able to handle a considerable number of outliers in both point sets. The experimental results demonstrate the robustness and accuracy of our algorithm.

Accurate and fast image segmentation algorithms are of paramount importance for a wide range of medical imaging applications. Level set algorithms based on narrow band implementation have been among the most widely used segmentation algorithms. However, the accuracy of standard level set algorithms is compromised by the fact that their evolution schemes deteriorate the signed distance level set functions required for accurate computation of normals and curvatures. The most common remedy is to use an ad-hoc reinitialization step to rebuild the signed distance function frequently. Meanwhile, complex upwind finite difference schemes are required for stable evolution. They together make the overall computation expensive. In this paper, we propose a novel fast narrow band distance preserving level set evolution algorithm that eliminates the need for both reinitialization and complex upwind finite difference schemes. This is achieved by incorporating into a variational level set formulation with a signed distance preserving term that regularizes the evolution. As a result, stable, accurate, fast evolution could be obtained using a simple finite difference scheme within a very narrow band, defined as the union of all 3times3 pixel blocks around the zero crossing pixels. Also, our method allows the use of larger time step to speed up the convergence while ensuring accurate result, as well as the use of more general and computational efficient initial level set functions rather than the signed distance functions required by standard level set methods. The proposed algorithm has been applied on both synthetic and real images of different modalities with promising results

In this paper, we present a new variational formulation for geometric active contours that forces the level set function to be close to a signed distance function, and therefore completely eliminates the need of the costly re-initialization procedure. Our variational formulation consists of an internal energy term that penalizes the deviation of the level set function from a signed distance function, and an external energy term that drives the motion of the zero level set toward the desired image features, such as object boundaries. The resulting evolution of the level set function is the gradient flow that minimizes the overall energy functional. The proposed variational level set formulation has three main advantages over the traditional level set formulations. First, a significantly larger time step can be used for numerically solving the evolution partial differential equation, and therefore speeds up the curve evolution. Second, the level set function can be initialized with general functions that are more efficient to construct and easier to use in practice than the widely used signed distance function. Third, the level set evolution in our formulation can be easily implemented by simple finite difference scheme and is computationally more efficient. The proposed algorithm has been applied to both simulated and real images with promising results.

In the short time since publication of Boykov and Jolly's seminal paper [2001], graph cuts have become well established as a leading method in 2D and 3D semi-automated image segmentation. Although this approach is computationally feasible for many tasks, the memory overhead and supralinear time complexity of leading algorithms results in an excessive computational burden for high-resolution data. In this paper, we introduce a multilevel banded heuristic for computation of graph cuts that is motivated by the well-known narrow band algorithm in level set computation. We perform a number of numerical experiments to show that this heuristic drastically reduces both the running time and the memory consumption of graph cuts while producing nearly the same segmentation result as the conventional graph cuts. Additionally, we are able to characterize the type of segmentation target for which our multilevel banded heuristic yields different results from the conventional graph cuts. The proposed method has been applied to both 2D and 3D images with promising results.

Geometric deformable models based on the level set method have become very popular. To overcome an inherent limitation in accuracy while maintaining computational efficiency, adaptive grid techniques using local grid refinement have been developed for use with these models. However, this strategy requires a very complex data structure, yields large numbers of contour points, and is inconsistent with our previously presented topology-preserving geometric deformable model (TGDM). In this paper, we incorporate an alternative adaptive grid technique called the moving grid method into the geometric deformable model framework. We find that it is simpler to implement than grid refinement, requiring no large, complex, hierarchical data structures. It also limits the number of contour vertices in the final contour and supports the incorporation of the topology-preserving constraint of TGDM. After presenting the algorithm, we demonstrate its performance using both simulated and real images.

Active contour and surface models, also known as deformable models, are powerful image segmentation techniques. Geometric deformable models implemented using level set methods have advantages over parametric models due to their intrinsic behavior, parameterization independence, and ease of implementation. However, a long claimed advantage of geometric deformable models-the ability to automatically handle topology changes-turns out to be a liability in applications where the object to be segmented has a known topology that must be preserved. We present a new class of geometric deformable models designed using a novel topology-preserving level set method, which achieves topology preservation by applying the simple point concept from digital topology. These new models maintain the other advantages of standard geometric deformable models including subpixel accuracy and production of nonintersecting curves or surfaces. Moreover, since the topology-preserving constraint is enforced efficiently through local computations, the resulting algorithm incurs only nominal computational overhead over standard geometric deformable models. Several experiments on simulated and real data are provided to demonstrate the performance of this new deformable model algorithm.

Reconstructing an accurate and topologically correct representation of the cortical surface of the brain is an important objective in various neuroscience applications. Most cortical surface reconstruction methods either ignore topology or correct it using manual editing or methods that lead to inaccurate reconstructions. Shattuck and Leahy recently reported a fully automatic method that yields a topologically correct representation with little distortion of the underlying segmentation. We provide an alternate approach that has several advantages over their approach, including the use of arbitrary digital connectivities, a flexible morphology-based multiscale approach, and the option of foreground-only or background-only correction. A detailed analysis of the method's performance on 15 magnetic resonance brain images is provided.

Active contour and surface models, also known as deformable models, constitute a class of powerful segmentation techniques. Geometric deformable models implemented via level-set methods have advantages over parametric ones due to their intrinsic behavior, parameterization independence, and ease of implementation. However, a long claimed advantage of geometric deformable models, the ability to automatically handle topology changes, turns out to be a liability in applications where the objects to be segmented have a known topology that must be preserved. In this paper, we present a geometric deformable model that preserves topology using the simple point concept from digital topology. This algorithm maintains the other advantages of standard geometric deformable models including sub-pixel accuracy and production of nonintersecting curves (or surfaces). Several experiments on simulated and real data are provided to demonstrate the performance of the proposed algorithm.

Reconstructing the geometry of the human cerebral cortex from MR images is an important step in both brain mapping and surgical path planning applications. Difficulties with imaging noise, partial volume averaging, image intensity inhomogeneities, convoluted cortical structures, and the requirement to preserve anatomical topology make the development of accurate automated algorithms particularly challenging. In this paper we address each of these problems and describe a systematic method for obtaining a surface representation of the geometric central layer of the human cerebral cortex. Using fuzzy segmentation, an isosurface algorithm, and a deformable surface model, the method reconstructs the entire cortex with the correct topology, including deep convoluted sulci and gyri. The method is largely automated and its results are robust to imaging noise, partial volume averaging, and image intensity inhomogeneities. The performance of this method is demonstrated, both qualitatively and quantitatively, and the results of its application to six subjects and one simulated MR brain volume are presented.

Active contours, or snakes, are used extensively in computer vision and image processing applications, particularly to locate object boundaries. A new type of external force for active contours, called gradient vector flow (GVF) was introduced recently to address problems associated with initialization and poor convergence to boundary concavities. GVF is computed as a diffusion of the gradient vectors of a gray-level or binary edge map derived from the image. In this paper, we generalize the GVF formulation to include two spatially varying weighting functions. This improves active contour convergence to long, thin boundary indentations, while maintaining other desirable properties of GVF, such as an extended capture range. The original GVF is a special case of this new generalized GVF (GGVF) model. An error analysis for active contour results on simulated test images is also presented. Aktive Umrisse, oder Schlangen, werden vielfach in Computervision- und Bildverarbeitungs-Anwendungen benutzt, um insbesondere Objektgrenzen zu lokalisieren. Ein neuer Typ äußerer Kräfte für aktive Umrisse, Gradient Vector Flow (GVF) genannt, wurde kürzlich eingeführt, um Probleme anzusprechen, die mit Initialisierung und schlechter Konvergenz zu Grenzkonkavitäten zusammenhängen. GVF wird als eine Diffusion des Gradientenvektors einer Graustufen- oder `Binary Edge'-Karte berechnet, die aus dem Bild gewonnen werden. In diesem Artikel verallgemeinern wir die GVF Formulierung, so daß zwei räumlich variierende Gewichtsfunktionen engneschlossen werden. Dies verbessert die Konvergenz aktiver Umrisse zu langen, dünnen Grenzmarkierungen, während andere wünschenswerte Eigenschaften des GVF, wie erweiterter Einfangbereich, erhalten bleiben. Das ursprüngliche GVF ist ein Spezialfall dieses neuen verallgemeinerten GVF (GGVF) Modells. Eins Fehleranalyse von Ergebnissen aktiver Umrisse mit simulierten Testbildern wird ebenfalls präsentiert. Les contours actifs, ou serpents (snakes), sont utilisés intensivement en vision par ordinateur et pour les applications de traitement d'images, particulièrement pour localiser les contours d'objects. Un nouveau type de force externe pour les contours actifs, appelè flux de vecteurs gradients (FVG) a été introduit récemment pour traiter les problémes associés à l'initialisation et la faible convergence vers des concavités dans les contours. Le FVG est calculé comme une diffusion des vecteurs gradients d'une carte des contours d'une image en niveaux de gris ou binaire. Dans cet article, nous généralisons la formulation du FVG pour y inclure deux fonctions de poids à variation spatiale. Ceci améliore la convergence des contours actifs vers les indentations de contours fines et longues, tout en maintenant les autres propriétés intéressantes des FVG comme la plage de capture étendue. Les FVG originaux sont un cas particulier des modèles de FVG généralisés. Une analyse de l'erreur des résultats de contours actifs sur des images de test synthétiques est aussi présentée.

Snakes, or active contours, are used extensively in computer vision and image processing applications, particularly to locate object boundaries. Problems associated with initialization and poor convergence to boundary concavities, however, have limited their utility. This paper presents a new external force for active contours, largely solving both problems. This external force, which we call gradient vector flow (GVF), is computed as a diffusion of the gradient vectors of a gray-level or binary edge map derived from the image. It differs fundamentally from traditional snake external forces in that it cannot be written as the negative gradient of a potential function, and the corresponding snake is formulated directly from a force balance condition rather than a variational formulation. Using several two-dimensional (2-D) examples and one three-dimensional (3-D) example, we show that GVF has a large capture range and is able to move snakes into boundary concavities.

Snakes, or active contours, are used extensively in computer vision and image processing applications, particularly to locate object boundaries. Problems associated with initialization and poor convergence to concave boundaries, however, have limited their utility. This paper develops a new external force for active contours, largely solving both problems. This external force, which we call gradient vector flow (GVF) is computed as a diffusion of the gradient vectors of a gray-level or binary edge map derived from the image. The resultant field has a large capture range and forces active contours into concave regions. Examples on simulated images and one real image are presented.

A procedure for estimating the joint probability density function (pdf) of T1, T2 and proton spin density (PD) for gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) in the brain is presented. The pdf's have numerous applications, including the study of tissue parameter variability in pathology and across populations. The procedure requires a multispectral, spin echo magnetic resonance imaging (MRI) data set of the brain. It consists of five automated steps: (i) preprocess the data to remove extracranial tissue using a sequence of image processing operators; (ii) estimate T1, T2 and PD by fitting the preprocessed data to an imaging equation; (iii) perform a fuzzy c-means clustering on the same preprocessed data to obtain a spatial map representing the membership value of the three tissue classes at each pixel location; (iv) reject estimates which are not from pure tissue or have poor fits in the parameter estimation, and classify the remaining estimates as either GM, WM or CSF; (v) compute statistics on the classified estimates to obtain a probability mass function and a Gaussian joint pdf of the tissue parameters for each tissue class. Some preliminary results are shown comparing computed pdf's of young, elderly and Alzheimer's subjects. Two brief examples applying the joint pdf's to pulse sequence optimization and generation of computational phantoms are also provided.