Patients with drug-refractory heart failure can greatly benefit from cardiac resynchronization therapy (CRT). A CRT device can resynchronize the contractions of the left ventricle (LV) leading to reduced mortality. Unfortunately, 30%-50% of patients do not respond to treatment when assessed by objective criteria such as cardiac remodeling. A significant contributing factor is the suboptimal placement of the LV lead. It has been shown that placing this lead away from scar and at the point of latest mechanical activation can improve response rates. This paper presents a comprehensive and highly automated system that uses scar and mechanical activation to plan and guide CRT procedures. Standard clinical preoperative magnetic resonance imaging is used to extract scar and mechanical activation information. The data are registered to a single 3-D coordinate system and visualized in novel 2-D and 3-D American Heart Association plots enabling the clinician to select target segments. During the procedure, the planning information is overlaid onto live fluoroscopic images to guide lead deployment. The proposed platform has been used during 14 CRT procedures and validated on synthetic, phantom, volunteer, and patient data.

Drawing a box around an intended segmentation target has become both a popular user interface and a common output for learning-driven detection algorithms. Despite the ubiquity of using a box to define a segmentation target, it is unclear in the literature whether a box is sufficient to define a unique segmentation or whether segmentation from a box is ill-posed without higher-level (semantic) knowledge of the intended target. We examine this issue by conducting a study of 14 subjects who are asked to segment a boxed target in a set of 50 real images for which they have no semantic attachment. We find that the subjects do indeed perceive and trace almost the same segmentations as each other, despite the inhomogeneity of the image intensities, irregular shapes of the segmentation targets and weakness of the target boundaries. Since the subjects produce the same segmentation, we conclude that the problem is well-posed and then provide a new segmentation algorithm from a box which achieves results close to the perceived target.

Registration and modeling of shapes are two important problems in computer vision and pattern recognition. Despite enormous progress made over the past decade, these problems are still open. In this paper, we advance the state of the art in both directions. First we consider an efficient registration method that aims to recover a one-to-one correspondence between shapes and introduce measures of uncertainties driven from the data which explain the local support of the recovered transformations. To this end, a free form deformation is used to describe the deformation model. The transformation is combined with an objective function defined in the space of implicit functions used to represent shapes. Once the registration parameters have been recovered, we introduce a novel technique for model building and statistical interpretation of the training examples based on a variable bandwidth kernel approach. The support on the kernels varies spatially and is determined according to the uncertainties of the registration process. Such a technique introduces the ability to account for potential registration errors in the model. Hand-written character recognition and knowledge-based object extraction in medical images are examples of applications that demonstrate the potentials of the proposed framework.

Reliable segmentation of the left ventricle is a long sought objective in medical imaging for automatic retrieval of anatomical and pathological measurements and detection of malfunctions. In this paper, we propose a novel model-constrained approach to address this task. The method is based on an implicit representation of the shape model used in a shape registration framework with a Thin Plate Spline transform to retrieve possible deformations. The main innovation of our approach resides in the use of uncertainties defined on the registered shape to augment the training set and improve the robustness of the statistical deformable model. We use ICA to reduce the dimensionality of the space of deformations and provide a good separation of the different deformable parts of the heart. Furthermore the estimation of uncertainties is also introduced in the segmentation process which is addressed in a variational framework where prior knowledge and visual support are considered. The method lead to very promising qualitative and quantitative experimental results in CT.

In this paper, we introduce a new technique for shape modelling in the space of implicit polynomials. Registration consists of recovering an optimal one-to-one transformation of a higher order polynomial along with uncertainties measures that are determined according to the covariance matrix of the correspondences at the zero isosurface. In the modelling phase, these measures are used to weight the importance of the training samples phase according to a variable bandwidth non-parametric density estimation process. The selection of the most appropriate kernels to represent the training set is done through the maximum likelihood criterion. Excellent results for patterns of digits, related with the registration and the modelling aspects of our approach demonstrate the potentials of our method

We demonstrate a new technique for general purpose interactive segmentation of N-dimensional images. The method creates two segments: “object” and “background”. The technical details can be found in our paper [1] in this proceedings. Below we concentrate on the actual interface. The user can enter seeds via mouse-operated brush of red (for object) or blue (for background) color. The size of the brush can be changed depending on the size of the object. The user should paint some pixels in the object of interest and some in the background. The seeds provide some clues on what the user intends to segment. As soon as initial seeds are entered, the whole image/volume can be segmented automatically. Basically, the algorithm tries to “predict” how the user would want to paint the rest of the image. Segmentation results are presented by highlighting the object and background segments with red and blue colors. Thus, the object segment appears reddish while the background appears bluish. This gives an intuitive feeling that the algorithm completes the painting started by the user. An optimal segmentation can be very efficiently recomputed when the user adds or removes any seeds. This allows the user to correct any result imperfections quickly via very intuitive interactions. If the algorithm makes a mistake, the user can add a stroke of red paint in the bluish segment (or blue paint in the reddish segment). The new segmentation would very quickly repaint the whole image to comply with additional hints from the user. Our method is not sensitive to exact positioning of seeds. Normally, the results would not change in the seeds are moved within the same object in the image or volume. Our method applies to N-D images (volumes). In case of 3D data the seeds are entered in selected representative slices. The information is automatically propagated between the slices because we compute our optimal segmentation directly in the volume. Thus, the whole volume can be segmented based on seeds in a single slice. 2. Examples

A procedure for automated 2D shape model design is presented. The system is given a set of training example shapes defined by contour point coordinates. The shapes are automatically aligned using Procrustes analysis and clustered to obtain cluster prototypes (typical objects) and statistical information about intracluster shape variation. One difference from previous methods is that the training set is first automatically clustered and shapes considered to be outliers are discarded. In this way, cluster prototypes are not distorted by outliers. A second difference is in the manner in which registered sets of points are extracted from each shape contour. We propose a flexible point matching technique that takes into account both pose/scale differences and nonlinear shape differences. The matching method is independent of the objects' initial relative position/scale and does not require any manually tuned parameters. Our shape model design method was used to learn 11 different shapes from contours that were manually traced in MR brain images. The resulting model was then employed to segment several MR brain images that were not included in the shape-training set. A quantitative analysis of our shape registration approach, within the main cluster of each structure, demonstrated results that compare very well to those achieved by manual registration; achieving an average registration error of about 1 pixel. Our approach can serve as a fully automated substitute to the tedious and time-consuming manual 2D shape registration and analysis.

We propose a method for object tracking using prototype-based deformable template models. To track an object in an image sequence, we use a criterion which combines two terms: the frame-to-frame deviations of the object shape and the fidelity of the modeled shape to the input image. The deformable template model utilizes the prior shape information which is extracted from the previous frames along with a systematic shape deformation scheme to model the object shape in a new frame. The following image information is used in the tracking process: 1) edge and gradient information: the object boundary consists of pixels with large image gradient, 2) region consistency: the same object region possesses consistent color and texture throughout the sequence, and 3) interframe motion: the boundary of a moving object is characterized by large interframe motion. The tracking proceeds by optimizing an objective function which combines both the shape deformation and the fidelity of the modeled shape to the current image (in terms of gradient, texture, and interframe motion). The inherent structure in the deformable template, together with region, motion, and image gradient cues, makes the proposed algorithm relatively insensitive to the adverse effects of weak image features and moderate amounts of occlusion.

A new fully automated shape learning method is presented. It is based on clustering a set of training shapes in the original shape space (defined by the coordinates of the contour points) and performing a Procrustes analysis on each cluster to obtain cluster prototypes and information about shape variation. The main difference from previously reported methods is that the training set is first automatically clustered and those shapes considered to be outliers are discarded. The second difference is in the manner in which registered sets of points are extracted from each shape contour. As a direct application of our shape learning method, an 11-structure shape model of brain substructures was extracted from MR image data, an eigen-shape model was automatically trained, and employed to segment several MR brain images not present in the shape-training set. A quantitative analysis of our shape registration approach, within the main cluster of each structure, shows that our results compare very well to those achieved by manual registration; achieving an average rms error of about 1 pixel. Our approach can serve as a fully automated substitute to the tedious and time-consuming manual shape registration and analysis.

An automated method for left ventricle detection in MR cardiac images is presented. Ventricle detection is the first step in a fully automated segmentation system used to compute volumetric information about the heart. Our method is based on learning the gray level appearance of the ventricle by maximizing the discrimination between positive and negative examples in a training set. The main differences from previously reported methods are feature definition and solution to the optimization problem involved in the learning process. Our method was trained on a set of 1,350 MR cardiac images from which 101,250 positive examples and 123,096 negative examples were generated. The detection results on a test set of 887 different images demonstrate an excellent performance: 98% detection rate, a false alarm rate of 0.05% of the number of windows analyzed (10 false alarms per image) and a detection time of 2 seconds per 256/spl times/256 image on a Sun Ultra 10 for an 8-scale search. The false alarms ore eventually eliminated by a position/scale consistency check along all the images that represent the same anatomical slice.

We propose a novel solution to the problem of motion compensation of coronary angiographs. As the heart is beating, it is difficult for the physician to observe closely a particular point (e.g. stenosis) on the artery tree. We propose, to rigidly compensate the sequence so that the area around the point of interest appears stable. This is a difficult problem because the arteries deform in a non-rigid manner and only their 2D X-ray projection is observed. Also, the lack of features around the selected point makes the matching subject to the aperture problem. The algorithm automatically extracts a section of the artery of interest, models it as a polyline, and tracks it. The problem is formulated as an energy minimization problem which is solved using a shortest path in a graph algorithm. The motion compensated sequence can be obtained by translating every pixel so that the point of interest remains stable. We have applied this algorithm to many examples in two sets of angiography data and have obtained excellent results.

Hybrid models are powerful tools for recovery in that they simultaneously provide a gross parametric as well as a detailed description of an object. However, it is difficult to directly employ hybrid models in the segmentation process since they are not guaranteed to locate the optimal boundaries in cross-sectional slices. Propagating 2D active contours from slice to slice, on the other hand, to delineate an object's boundaries is often effective, but may run into problems when the object's topology changes, such as at bifurcations or even in areas of high curvature. Here, we present a cooperative framework to exploit the positive aspects of both 3D hybrid model and 2D active contour approaches for segmentation and recovery. In this framework the user-defined parametric component of a 3D hybrid model provides constraints for a set of 2D segmentations performed by active contours. The same hybrid model is then fit both parametrically and locally to this segmentation. For the hybrid model fit we employ several new variations on the physically-motivated paradigm which seek to speed recovery while guaranteeing stability. A by-product of these variations is an increased generality of the method via the elimination, of some of its ad hoc parameters. We apply our cooperative framework to the recovery of branching cylindrical structures from 3D image volumes. The hybrid model we employ has a novel parametric component which is a fusion of individual cylinders. These cylinders have spines that are arbitrary space curves and cross-sections which may be any star shaped planar curve.

We propose a novel method for object tracking using prototype-based deformable template models. To track an object in an image sequence, we use a criterion which combines two terms: the deviation of the object shape from its shape in the previous frame, and the fidelity of the detected shape to the input image. Shape and gradient information are used to track the object. We have also used the consistency between corresponding object regions throughout the sequence to help in trading the object of interest. Inter-frame motion is also used to track the boundary of moving objects. We have applied the algorithm to a number of image sequences from different sources. The inherent structure in the deformable template, together with region, motion, and image gradient cues, make the algorithm relatively insensitive to the adverse effects of weak image features and moderate partial occlusion.

The paper describes an algorithm for combining color and texture information for the segmentation of color images. The algorithm uses maximum likelihood classification combined with a certainty based fusion criterion. The algorithm was validated using mosaics of real color textures. It was also tested on real outdoor color scenes and aerial images. This algorithm is part of a more complex system which is currently being designed to assist an operator in updating an old map of an area using aerial images.

This paper proposes a segmentation algorithm using deformable template models to segment a vehicle of interest both from the stationary complex background and other moving vehicles in an image sequence. We define a polygonal template to characterize a general model of a vehicle and derive a prior probability density function to constrain the template to be deformed within a set of allowed shapes. We propose a likelihood probability density function which combines motion information and edge directionality to ensure that the deformable template is contained within the moving areas in the image and its boundary coincides with strong edges with the same orientation in the image. The segmentation problem is reduced to a minimization problem and solved by the Metropolis algorithm. The system was successfully tested on 405 image sequences containing multiple moving vehicles on a highway.

This paper illustrates the advantages of using multiple cues for object matching. Given two sets of people entering and leaving a room, the goal is to identify the matching pairs assuming that the viewing aspects of the people in the two scenes are similar. Color information or 2D shape information alone is not enough to find all the matching pairs, but all the matching pairs are correctly identified when both the features are combined.>