Evolution strategies (ESs) are powerful probabilistic search and optimization algorithms gleaned from biological evolution theory. They have been successfully applied to a wide range of real world applications. The modern ESs are mainly designed for solving continuous parameter optimization problems. Their ability to adapt the parameters of the multivariate normal distribution used for mutation during the optimization run makes them well suited for this domain. In this article we describe and study mixed integer evolution strategies (MIES), which are natural extensions of ES for mixed integer optimization problems. MIES can deal with parameter vectors consisting not only of continuous variables but also with nominal discrete and integer variables. Following the design principles of the canonical evolution strategies, they use specialized mutation operators tailored for the aforementioned mixed parameter classes. For each type of variable, the choice of mutation operators is governed by a natural metric for this variable type, maximal entropy, and symmetry considerations. All distributions used for mutation can be controlled in their shape by means of scaling parameters, allowing self-adaptation to be implemented. After introducing and motivating the conceptual design of the MIES, we study the optimality of the self-adaptation of step sizes and mutation rates on a generalized (weighted) sphere model. Moreover, we prove global convergence of the MIES on a very general class of problems. The remainder of the article is devoted to performance studies on artificial landscapes (barrier functions and mixed integer NK landscapes), and a case study in the optimization of medical image analysis systems. In addition, we show that with proper constraint handling techniques, MIES can also be applied to classical mixed integer nonlinear programming problems.

Fluorescence loss in photobleaching (FLIP) is a method to study compartment connectivity in living cells. A FLIP sequence is obtained by alternatively bleaching a spot in a cell and acquiring an image of the complete cell. Connectivity is estimated by comparing fluorescence signal attenuation in different cell parts. The measurements of the fluorescence attenuation are hampered by the low signal to noise ratio of the FLIP sequences, by sudden sample shifts and by sample drift. This paper describes a method that estimates the attenuation by modeling photobleaching as exponentially decaying signals. Sudden motion artifacts are minimized by registering the frames of a FLIP sequence to target frames based on the estimated model and by removing frames that contain deformations. Linear motion (sample drift) is reduced by minimizing the entropy of the estimated attenuation coefficients. Experiments on 16 in vivo FLIP sequences of muscle cells in Drosophila show that the proposed method results in fluorescence attenuations similar to the manually identified gold standard, but with standard deviations of approximately 50 times smaller. As a result of this higher precision, cell compartment edges and details such as cell nuclei become clearly discernible. The main value of this method is that it uses a model of the bleaching process to correct motion and that the model based fluorescence intensity and attenuation estimates can be interpreted easily. The proposed method is fully automatic, and runs in approximately one minute per sequence, making it suitable for unsupervised batch processing of large data series.

This paper discusses a number of image analysis challenges emerging from longitudinal small animal molecular imaging studies. Three steps towards a quantitative 3D analysis of follow-up small animal imaging are presented: whole-body articulated registration, change visualization in follow-up data and fusion of optical and 3D structural imaging data. Several application examples are presented in the context of translational cancer research.

We study the effectiveness of stereoscopy and smooth motion as 3D cues for medical interpretation of vascular structures as obtained by 3D medical imaging techniques. We designed a user study where the user has to follow a path in a mazelike solid shaded 3D structure. The user controls rotation of the model. We measure user performance in terms of time taken and error rate. The experiment was executed with 32 (medical and non-medical) users. The results show that motion cue is more important than stereoscopy, and that stereoscopy has no added value when motion is already present, which is not consistent with previous experiments.

Visual search is a task that is performed in various application areas. Search can be aided by an automatic warning system, which highlights the sections that may contain targets and require the user's attention. The effect of imperfect automatic warnings on overall performance ultimately depends on the interplay between the user and the automatic warning system. While various user studies exist, the different studies differ in several experimental variables including the nature of the visualisation itself. Studies in the medical area remain relatively rare, even though there is a growing interest in medical screening systems. We describe an experiment where users had to perform a visual search on a vascular structure, traversing a particular vessel linearly in search of possible errors made in an automatic segmentation. We find that only the case in which the warning system generates only false positives improves user time and error performance. We discuss this finding in relation to the findings of other studies.

Automated interpretation of complex images requires elaborate knowledge and model-based image analysis, but often needs interaction with an expert as well. This research describes expert interaction with a multiagent image interpretation system using only a restricted vocabulary of high-level user interactions. The aim is to minimize inter- and intra-observer variability by keeping the total number of interactions as low and simple as possible. The multiagent image interpretation system has elaborate high-level knowledge-based control over low-level image segmentation algorithms. Agents use contextual knowledge to keep the number of interactions low but, when in doubt, present the user with the most likely interpretation of the situation. The user, in turn, can correct, supplement, and/or confirm the results of image-processing agents. This is done at a very high level of abstraction such that no knowledge of the underlying segmentation methods, parameters or agent functioning is needed. High-level interaction thereby replaces more traditional contour correction methods like inserting points and/or (re)drawing contours. This makes it easier for the user to obtain good results, while inter- and intra-observer variability are kept minimal, since the image segmentation itself remains under control of image-processing agents. The system has been applied to intravascular ultrasound (IVUS) images. Experiments show that with an average of 2-3 high-level user interactions per correction, segmentation results substantially improve while the variation is greatly reduced. The achieved level of accuracy and repeatability is equivalent to that of manual drawing by an expert.

In this paper, a statistical shape analysis method for myocardial contraction is presented that was built to detect and locate regional wall motion abnormalities (RWMA). For each slice level (base, middle, and apex), 44 short-axis magnetic resonance images were selected from healthy volunteers to train a statistical model of normal myocardial contraction using independent component analysis (ICA). A classification algorithm was constructed from the ICA components to automatically detect and localize abnormally contracting regions of the myocardium. The algorithm was validated on 45 patients suffering from ischemic heart disease. Two validations were performed; one with visual wall motion scores (VWMS) and the other with wall thickening (WT) used as references. Accuracy of the ICA-based method on each slice level was 69.93% (base), 89.63% (middle), and 72.78% (apex) when WT was used as reference, and 63.70% (base), 67.41% (middle), and 66.67% (apex) when VWMS was used as reference. From this we conclude that the proposed method is a promising diagnostic support tool to assist clinicians in reducing the subjectivity in VWMS.

This paper discusses mixed integer evolution strategies (MIES) assisted by metamodels based on radial basis function networks (RBFN). The goal is to make MIES more suitable for optimization with time consuming evaluation functions.

Manual quantitative analysis of cardiac left ventricular function using Multislice CT and MR is arduous because of the large data volume. In this paper, we present a 3-D active shape model (ASM) for semiautomatic segmentation of cardiac CT and MR volumes, without the requirement of retraining the underlying statistical shape model. A fuzzy c-means based fuzzy inference system was incorporated into the model. Thus, relative gray-level differences instead of absolute gray values were used for classification of 3-D regions of interest (ROIs), removing the necessity of training different models for different modalities/acquisition protocols. The 3-D ASM was evaluated using 25 CT and 15 MR datasets. Automatically generated contours were compared to expert contours in 100 locations. For CT, 82.4% of epicardial contours and 74.1% of endocardial contours had a maximum error of 5 mm along 95% of the contour arc length. For MR, those numbers were 93.2% (epicardium) and 91.4% (endocardium). Volume regression analysis revealed good linear correlations between manual and semiautomatic volumes, r(2) >/= 0.98. This study shows that the fuzzy inference 3-D ASM is a robust promising instrument for semiautomatic cardiac left ventricle segmentation. Without retraining its statistical shape component, it is applicable to routinely acquired CT and MR studies.

Three-dimensional (3-D) stress echocardiography is a novel technique for diagnosing cardiac dysfunction. It involves evaluating wall motion of the left ventricle, by visually analyzing ultrasound images obtained in rest and in different stages of stress. Since the acquisitions are performed minutes apart, variabilities may exist in the visualized cross-sections. To improve anatomical correspondence between rest and stress, aligning the images is essential. We developed a new intensity-based, sparse registration method to retrieve standard anatomical views from 3-D stress images that were equivalent to the manually selected views in the rest images. Using sparse image planes, the influence of common image artifacts could be reduced. We investigated different similarity measures and different levels of sparsity. The registration was tested using data of 20 patients and quantitatively evaluated based on manually defined anatomical landmarks. Alignment was best using sparse registration with two long-axis and two short-axis views; registration errors were reduced significantly, to the range of interobserver variabilities. In 91% of the cases, the registration result was qualitatively assessed as better than or equal to the manual alignment. In conclusion, sparse registration improves the alignment of rest and stress images, with a performance similar to manual alignment. This is an important step towards objective quantification in 3-D stress echocardiography.

This paper presents a novel method for registration of cardiac perfusion magnetic resonance imaging (MRI). The presented method is capable of automatically registering perfusion data, using independent component analysis (ICA) to extract physiologically relevant features together with their time-intensity behavior. A time-varying reference image mimicking intensity changes in the data of interest is computed based on the results of that ICA. This reference image is used in a two-pass registration framework. Qualitative and quantitative validation of the method is carried out using 46 clinical quality, short-axis, perfusion MR datasets comprising 100 images each. Despite varying image quality and motion patterns in the evaluation set, validation of the method showed a reduction of the average right ventricle (LV) motion from 1.26+/-0.87 to 0.64+/-0.46 pixels. Time-intensity curves are also improved after registration with an average error reduced from 2.65+/-7.89% to 0.87+/-3.88% between registered data and manual gold standard. Comparison of clinically relevant parameters computed using registered data and the manual gold standard show a good agreement. Additional tests with a simulated free-breathing protocol showed robustness against considerable deviations from a standard breathing protocol. We conclude that this fully automatic ICA-based method shows an accuracy, a robustness and a computation speed adequate for use in a clinical environment.

This paper describes a new approach to the automated segmentation of X-ray left ventricular (LV) angiograms, based on active appearance models (AAMs) and dynamic programming. A coupling of shape and texture information between the end-diastolic (ED) and end-systolic (ES) frame was achieved by constructing a multiview AAM. Over-constraining of the model was compensated for by employing dynamic programming, integrating both intensity and motion features in the cost function. Two applications are compared: a semi-automatic method with manual model initialization, and a fully automatic algorithm. The first proved to be highly robust and accurate, demonstrating high clinical relevance. Based on experiments involving 70 patient data sets, the algorithm's success rate was 100% for ED and 99% for ES, with average unsigned border positioning errors of 0.68 mm for ED and 1.45 mm for ES. Calculated volumes were accurate and unbiased. The fully automatic algorithm, with intrinsically less user interaction was less robust, but showed a high potential, mostly due to a controlled gradient descent in updating the model parameters. The success rate of the fully automatic method was 91% for ED and 83% for ES, with average unsigned border positioning errors of 0.79 mm for ED and 1.55 mm for ES.

In this paper we describe a totally new and original approach for combining global and local information in medical image processing. We implemented a virtual mobile robot and trained it using fuzzy neural networks to recognize segments of the myocardium while he navigates autonomously around the left ventricle (LV) of the heart. On its journey around the heart, the virtual exploring robot applies appropriate local edge detection to delineate fully automatically the borders of the myocardium. This may sound unconventional but it has proven effective enough to be integrated in a clinical analytical software tool.

Abstract Selection of the best set of scales is problematic when developing signal‐driven approaches for pixel‐based image segmentation. Often, different possibly conflicting criteria need to be fulfilled in order to obtain the best trade‐off between uncertainty (variance) and location accuracy. The optimal set of scales depends on several factors: the noise level present in the image material, the prior distribution of the different types of segments, the class‐conditional distributions associated with each type of segment as well as the actual size of the (connected) segments. We analyse, theoretically and through experiments, the possibility of using the overall and class‐conditional error rates as criteria for selecting the optimal sampling of the linear and morphological scale spaces. It is shown that the overall error rate is optimized by taking the prior class distribution in the image material into account. However, a uniform (ignorant) prior distribution ensures constant class‐conditional error rates. Consequently, we advocate for a uniform prior class distribution when an uncommitted, scale‐invariant segmentation approach is desired. Experiments with a neural net classifier developed for segmentation of dynamic magnetic resonance (MR) images, acquired with a paramagnetic tracer, support the theoretical results. Furthermore, the experiments show that the addition of spatial features to the classifier, extracted from the linear or morphological scale spaces, improves the segmentation result compared to a signal‐driven approach based solely on the dynamic MR signal. The segmentation results obtained from the two types of features are compared using two novel quality measures that characterize spatial properties of labelled images. Copyright © 2002 John Wiley & Sons, Ltd.

A widely used subpixel precision estimate of an object center is the weighted center of gravity (COG). We derive three maximum-likelihood estimators for the variance of the two-dimensional (2-D) COG as a function of the noise in the image. We assume that the noise is additive, Gaussian distributed and independent between neighboring pixels. Repeated experiments using 2500 generated 2-D bell-shaped markers superimposed with an increasing amount of Gaussian noise were performed, to compare the three approximations. The error of the most exact approximative variance estimate with respect to true variance was always less than 5% of the latter. This deviation decreases with increasing signal-to-noise ratio. Our second approximation to the variance estimate performed better than the third approximation, which was originally presented by Oron et al. by up to a factor approximately 10. The difference in performance between these two approximations increased with an increasing misplacement of the window in which the COG was calculated with respect to the real COG.

A novel extension of active appearance models (AAMs) for automated border detection in echocardiographic image sequences is reported. The active appearance motion model (AAMM) technique allows fully automated robust and time-continuous delineation of left ventricular (LV) endocardial contours over the full heart cycle with good results. Nonlinear intensity normalization was developed and employed to accommodate ultrasound-specific intensity distributions. The method was trained and tested on 16-frame phase-normalized transthoracic four-chamber sequences of 129 unselected infarct patients, split randomly into a training set (n = 65) and a test set (n = 64). Borders were compared to expert drawn endocardial contours. On the test set, fully automated AAMM performed well in 97% of the cases (average distance between manual and automatic landmark points was 3.3 mm, comparable to human interobserver variabilities). The ultrasound-specific intensity normalization proved to be of great value for good results in echocardiograms. The AAMM was significantly more accurate than an equivalent set of two-dimensional AAMs.

This paper presents a novel measurement technique to assess the effects of coronary brachytherapy. This new technique is based upon the conventional quantitative coronary analysis (QCA) technique, which is accepted worldwide as an accurate and reliable analysis tool for clinical trials. This paper provides the definitions and main issues important for correct brachytherapy analysis. Based on these definitions, this novel technique is implemented as an extension of conventional QCA software, as a multisegmental analysis tool. It allows to follow the influence of radiation on restenosis, and the mutual relation between intervention devices. A pilot interobserver study was performed to assess the reliability and reproducibility of the brachytherapy analysis tool, using 15 patient cases. The validation results show that the segment lengths, minimum lumen diameter, and reference diameters of the user-defined and derived (sub)segments can be assessed reproducible. However, these good results can only be obtained, when strict and extensive image acquisition and image analysis protocols are followed. From this pilot validation study presented in this paper and only based on a small number of patients, we may conclude that the software can be applied to clinical trials.

A model-based method for three-dimensional image segmentation was developed and its performance assessed in segmentation of volumetric cardiac magnetic resonance (MR) images and echocardiographic temporal image sequences. Comprehensive design of a three-dimensional (3-D) active appearance model (AAM) is reported for the first time as an involved extension of the AAM framework introduced by Cootes et al. The model's behavior is learned from manually traced segmentation examples during an automated training stage. Information about shape and image appearance of the cardiac structures is contained in a single model. This ensures a spatially and/or temporally consistent segmentation of three-dimensional cardiac images. The clinical potential of the 3-D AAM is demonstrated in short-axis cardiac MR images and four-chamber echocardiographic sequences. The method's performance was assessed by comparison with manually identified independent standards in 56 clinical MR and 64 clinical echo image sequences. The AAM method showed good agreement with the independent standard using quantitative indexes of border positioning errors, endo- and epicardial volumes, and left ventricular mass. In MR, the endocardial volumes, epicardial volumes, and left ventricular wall mass correlation coefficients between manual and AAM were R2 = 0.94, 0.97, 0.82, respectively. For echocardiographic analysis, the area correlation was R2 = 0.79. The AAM method shows high promise for successful application to MR and echocardiographic image analysis in a clinical setting.

This paper describes a multimodality framework for computer-aided myocardial viability assessment based on neuro-fuzzy techniques. The proposed approach distinguishes two main levels: the modality-independent inference level and the modality-dependent application level. This two-level distinction releases the hard constraint of multimodality image registration. An abstract description template is used to describe the different myocardial functions (contractile function, perfusion, metabolism). Parameters extracted from different image modalities are combined to derive a diagnostic image. The neuro-fuzzy techniques make our system transparent, adaptive and easily extendable. Its effectiveness and robustness are demonstrated in a positron emission tomography/magnetic resonance imaging data fusion application.

A fully automated approach to segmentation of the left and right cardiac ventricles from magnetic resonance (MR) images is reported. A novel multistage hybrid appearance model methodology is presented in which a hybrid active shape model/active appearance model (AAM) stage helps avoid local minima of the matching function. This yields an overall more favorable matching result. An automated initialization method is introduced making the approach fully automated. Our method was trained in a set of 102 MR images and tested in a separate set of 60 images. In all testing cases, the matching resulted in a visually plausible and accurate mapping of the model to the image data. Average signed border positioning errors did not exceed 0.3 mm in any of the three determined contours-left-ventricular (LV) epicardium, LV and right-ventricular (RV) endocardium. The area measurements derived from the three contours correlated well with the independent standard (r = 0.96, 0.96, 0.90), with slopes and intercepts of the regression lines close to one and zero, respectively. Testing the reproducibility of the method demonstrated an unbiased performance with small range of error as assessed via Bland-Altman statistic. In direct border positioning error comparison, the multistage method significantly outperformed the conventional AAM (p < 0.001). The developed method promises to facilitate fully automated quantitative analysis of LV and RV morphology and function in clinical setting.

The decrease in the volume of viable tumor is an indicator for the effect preoperative chemotherapy has on bone tumors. We develop an approach for segmenting dynamic perfusion MR-images into viable tumor, nonviable tumor and healthy tissue. Two cascaded feedforward neural networks are trained to perform the pixel-based segmentation. As features, we use the parameters obtained from a pharmacokinetic model of the tissue perfusion (parametric images). Additional multiscale features that incorporate contextual information are included. Experiments indicate that multiscale blurred versions of the parametric images together with a multiscale formulation of the local image entropy are the most discriminative features.

A critical aspect of the treatment of patients with heart failure is the ability to predict the success of a revascularisation procedure. This prediction is based on the assessment of myocardial viability. Positron emission tomography is considered to be the gold standard for myocardial viability studies. Most of these studies are still based on qualitative assessment of the extent and severity of the disease. More recent studies quantify the extent of viable tissue manually using interactive software. In this work an accurate automatic assessment method is presented. Our approach is based on neuro-fuzzy techniques. A self organized radial basis function network has been implemented for image segmentation and parameter extraction, and an adaptive network-based fuzzy inference system is used to combine complementary information (myocardial perfusion and metabolism) to decide on myocardial viability. This work demonstrates the efficiency and accuracy of neuro-fuzzy techniques when carefully applied to viability assessment. It is an innovative approach to viability parametric image construction.

In this paper, an unsupervised image segmentation technique is presented, which combines pyramidal image segmentation with the fuzzy c-means clustering algorithm. Each layer of the pyramid is split into a number of regions by a root labeling technique, and then fuzzy c-means is used to merge the regions of the layer with the highest image resolution. A cluster validity functional is used to find the optimal number of objects automatically. Segmentation of a number of synthetic as well as clinical images is illustrated and two fully automatic segmentation approaches are evaluated, which determine the left ventricular volume (LV) in 140 cardiovascular magnetic resonance (MR) images. First fuzzy c-means is applied without pyramids. In the second approach the regions generated by pyramidal segmentation are merged by fuzzy c-means. The correlation coefficients of manually and automatically defined LV lumen of all 140 and 20 end-diastolic images were equal to 0.86 and 0.79, respectively, when images were segmented with fuzzy c-means alone. These coefficients increased to 0.90 and 0.93 when the pyramidal segmentation was combined with fuzzy c-means. This method can be applied to any dimensional representation and at any resolution level of an image series. The evaluation study shows good performance in detecting LV lumen in MR images.

Many segmentation methods for thoracic volume data require manual input in the form of a seed point, initial contour, volume of interest etc. The aim of the work presented here is to further automate this segmentation initialization step. In this paper an anatomical modeling and matching method is proposed to coarsely segment thoracic volume data into anatomically labeled regions. An anatomical model of the thorax is constructed in two steps: 1) individual organs are modeled with blended fuzzy implicit surfaces and 2) the single organ models are grouped into a tree structure with a solid modeling technique named constructive solid geometry (CSG). The combination of CSG with fuzzy implicit surfaces allows a hierarchical scene description by means of a boundary model, which characterizes the scene volume as a boundary potential function. From this boundary potential, an energy function is defined which is minimal when the model is registered to the tissue-air transitions in thoracic magnetic resonance imaging (MRI) data. This allows automatic registration in three steps: feature detection, initial positioning and energy minimization. The model matching has been validated in phantom simulations and on 15 clinical thoracic volume scans from different subjects. In 13 of these sets the matching method accurately partitioned the image volumes into a set of volumes of interest for the heart, lungs, cardiac ventricles, and thorax outlines. The method is applicable to segmentation of various types of thoracic MR-images, provided that a large part of the thorax is contained in the image volume.

In this paper optimal filters for the detection of coronary arteries with a diameter range of 0.5-6.0 mm in digital X-ray images are derived using a computational approach. This approach is based on the two requirements for optimal detection. First, the filter should maximize the number of detected true edges and minimize the number of detected false edges. Second, if an edge has been detected, its position should be as close as possible to the true edge position in the image. Since the grey value profile in a digital X-ray image associated with an arterial vessel is asymmetric, the theory on edge detection derived by Canny has been expanded with two additional boundary constraints to make it suitable for the derivation of filters for asymmetric edges. It is demonstrated that it is possible to derive optimal filters for coronary segments. The localization error, defined by the square root of the sum of the squared systematic and random errors in the assessment of the arterial diameter, depends on the size of the coronary artery and the amount of noise in the image. In this paper, an evaluation study is described to assess the relationship between localization error and the amount of noise upon the vessel profile. For that purpose, an analytical description of the vessel profile in an angiographic image was derived. For the larger arteries the relation between noise and localization error was found to be linear and no systematic over- or underestimations were observed, even if the noise level was very high. However, it can be shown that the smallest diameter that can be measured depends on the amount of noise present in the data. Even for images that contain only a low amount of noise, arterial diameters below 0.7 mm cannot be measured accurately. If the noise in the image increases, the lowest measurable arterial diameter value also increases. Also the random error increases rapidly for vessel diameters below 1.2 mm, but with a limited amount of noise and a diameter value above 0.7 mm the random error is still acceptable [0.15 mm (21%) for 0.7-mm vessels, 0.06 mm (6%) for 1-mm vessels].

Foreshortening of vessel segments in angiographic (biplane) projection images may cause misinterpretation of the extent and degree of coronary artery disease. The views in which the object of interest are visualized with minimum foreshortening are called optimal views. The authors present a complete approach to obtain such views with computer-assisted techniques. The object of interest is first visualized in two arbitrary views. Two landmarks of the object are manually defined in the two projection images. With complete information of the projection geometry, the vector representation of the object in the three-dimensional space is computed. This vector is perpendicular to a plane in which the views are called optimal. The user has one degree of freedom to define a set of optimal biplane views. The angle between the central beams of the imaging systems can be chosen freely. The computation of the orientation of the object and of corresponding optimal biplane views have been evaluated with a simple hardware phantom. The mean and the standard deviation of the overall errors in the calculation of the optimal angulation angles were 1.8 degrees and 1.3 degrees , respectively, when the user defined a rotation angle.

As part of the development directed at the automated reporting of the location and degree of stenoses in a coronary arterial tree, the authors address the issue of automated labelling of coronary segments. With a priori information of the coronary anatomy related to the projection geometry, the branches of a coronary tree are labelled by a consistent inexact graph matching (CIGM) technique. They describe the CIGM technique, the cost functions and associated parameters, a representation of the segments of the arterial tree and their experience in labelling arteries in routinely acquired arteriograms.>

The contours of isolated objects in noisy images may be detected with a minimal cost contour detection algorithm. An algorithm that is based on the policy-iteration method for locating the closed minimal cost path is introduced. Computational results indicate that it is computationally more efficient than the dynamic programming approach. The method is applied to left ventricular contours in scintigraphic images, although it is applicable to any domain where a closed minimal cost path is to be computed in a matrix of cost coefficients.>