Deep learning applications have had a profound impact on many scientific fields, including functional genomics. Deep learning models can learn complex interactions between and within omics data; however, interpreting and explaining these models can be challenging. Interpretability is essential not only to help progress our understanding of the biological mechanisms underlying traits and diseases but also for establishing trust in these model's efficacy for healthcare applications. Recognizing this importance, recent years have seen the development of numerous diverse interpretability strategies, making it increasingly difficult to navigate the field. In this review, we present a quantitative analysis of the challenges arising when designing interpretable deep learning solutions in functional genomics. We explore design choices related to the characteristics of genomics data, the neural network architectures applied, and strategies for interpretation. By quantifying the current state of the field with a predefined set of criteria, we find the most frequent solutions, highlight exceptional examples, and identify unexplored opportunities for developing interpretable deep learning models in genomics.

Identification and detection of thin-cap fibroatheroma (TCFA) from intravascular optical coherence tomography (IVOCT) images is critical for treatment of coronary heart diseases. Recently, deep learning methods have shown promising successes in TCFA identification. However, most methods usually do not effectively utilize multi-view information or incorporate prior domain knowledge. In this paper, we propose a multi-view contour-constrained transformer network (MVCTN) for TCFA identification in IVOCT images. Inspired by the diagnosis process of cardiologists, we use contour constrained self-attention modules (CCSM) to emphasize features corresponding to salient regions (i.e., vessel walls) in an unsupervised manner and enhance the visual interpretability based on class activation mapping (CAM). Moreover, we exploit transformer modules (TM) to build global-range relations between two views (i.e., polar and Cartesian views) to effectively fuse features at multiple feature scales. Experimental results on a semi-public dataset and an in-house dataset demonstrate that the proposed MVCTN outperforms other single-view and multi-view methods. Lastly, the proposed MVCTN can also provide meaningful visualization for cardiologists via CAM.

Extracting the cerebral anterior vessel tree of patients with an intracranial large vessel occlusion (LVO) is relevant to investigate potential biomarkers that can contribute to treatment decision making. The purpose of our work is to develop a method that can achieve this from routinely acquired computed tomography angiography (CTA) and computed tomography perfusion (CTP) images. To this end, we regard the anterior vessel tree as a set of bifurcations and connected centerlines. The method consists of a proximal policy optimization (PPO) based deep reinforcement learning (DRL) approach for tracking centerlines, a convolutional neural network based bifurcation detector, and a breadth-first vessel tree construction approach taking the tracking and bifurcation detection results as input. We experimentally determine the added values of various components of the tracker. Both DRL vessel tracking and CNN bifurcation detection were assessed in a cross validation experiment using 115 subjects. The anterior vessel tree formation was evaluated on an independent test set of 25 subjects, and compared to interobserver variation on a small subset of images. The DRL tracking result achieves a median overlapping rate until the first error (1.8 mm off the reference standard) of 100, [46, 100] % on 8032 vessels over 115 subjects. The bifurcation detector reaches an average recall and precision of 76% and 87% respectively during the vessel tree formation process. The final vessel tree formation achieves a median recall of 68% and precision of 70%, which is in line with the interobserver agreement.

Augmented reality (AR) has shown potential in computer-aided surgery. It allows for the visualization of hidden anatomical structures as well as assists in navigating and locating surgical instruments at the surgical site. Various modalities (devices and/or visualizations) have been used in the literature, but few studies investigated the adequacy/superiority of one modality over the other. For instance, the use of optical see-through (OST) HMDs has not always been scientifically justified. Our goal is to compare various visualization modalities for catheter insertion in external ventricular drain and ventricular shunt procedures. We investigate two AR approaches: (1) 2D approaches consisting of a smartphone and a 2D window visualized through an OST (Microsoft HoloLens 2), and (2) 3D approaches consisting of a fully aligned patient model and a model that is adjacent to the patient and is rotationally aligned using an OST. 32 participants joined this study. For each visualization approach, participants were asked to perform five insertions after which they filled NASA-TLX and SUS forms. Moreover, the position and orientation of the needle with respect to the planning during the insertion task were collected. The results show that participants achieved a better insertion performance significantly under 3D visualizations, and the NASA-TLX and SUS forms reflected the preference of participants for these approaches compared to 2D approaches.

Intracranial vessel perforation is a peri-procedural complication during endovascular therapy (EVT). Prompt recognition is important as its occurrence is strongly associated with unfavorable treatment outcomes. However, perforations can be hard to detect because they are rare, can be subtle, and the interventionalist is working under time pressure and focused on treatment of vessel occlusions. Automatic detection holds potential to improve rapid identification of intracranial vessel perforation. In this work, we present the first study on automated perforation detection and localization on X-ray digital subtraction angiography (DSA) image series. We adapt several state-of-the-art single-frame detectors and further propose temporal modules to learn the progressive dynamics of contrast extravasation. Application-tailored loss function and post-processing techniques are designed. We train and validate various automated methods using two national multi-center datasets (i.e., MR CLEAN Registry and MR CLEAN-NoIV Trial), and one international multi-trial dataset (i.e., the HERMES collaboration). With ten-fold cross-validation, the proposed methods achieve an area under the curve (AUC) of the receiver operating characteristic of 0.93 in terms of series level perforation classification. Perforation localization precision and recall reach 0.83 and 0.70 respectively. Furthermore, we demonstrate that the proposed automatic solutions perform at similar level as an expert radiologist.

mapping. The RIM is a neural network framework that learns an iterative inference process based on the signal model, similar to conventional statistical methods for quantitative MRI (QMRI), such as the Maximum Likelihood Estimator (MLE). This framework combines the advantages of both data-driven and model-based methods, and, we hypothesize, is a promising tool for QMRI. Previously, RIMs were used to solve linear inverse reconstruction problems. Here, we show that they can also be used to optimize non-linear problems and estimate relaxometry maps with high precision and accuracy. The developed RIM framework is evaluated in terms of accuracy and precision and compared to an MLE method and an implementation of the Residual Neural Network (ResNet). The results show that the RIM improves the quality of estimates compared to the other techniques in Monte Carlo experiments with simulated data, test-retest analysis of a system phantom, and in-vivo scans. Additionally, inference with the RIM is 150 times faster than the MLE, and robustness to (slight) variations of scanning parameters is demonstrated. Hence, the RIM is a promising and flexible method for QMRI. Coupled with an open-source training data generation tool, it presents a compelling alternative to previous methods.

The Thrombolysis in Cerebral Infarction (TICI) score is an important metric for reperfusion therapy assessment in acute ischemic stroke. It is commonly used as a technical outcome measure after endovascular treatment (EVT). Existing TICI scores are defined in coarse ordinal grades based on visual inspection, leading to inter- and intra-observer variation. In this work, we present autoTICI, an automatic and quantitative TICI scoring method. First, each digital subtraction angiography (DSA) acquisition is separated into four phases (non-contrast, arterial, parenchymal and venous phase) using a multi-path convolutional neural network (CNN), which exploits spatio-temporal features. The network also incorporates sequence level label dependencies in the form of a state-transition matrix. Next, a minimum intensity map (MINIP) is computed using the motion corrected arterial and parenchymal frames. On the MINIP image, vessel, perfusion and background pixels are segmented. Finally, we quantify the autoTICI score as the ratio of reperfused pixels after EVT. On a routinely acquired multi-center dataset, the proposed autoTICI shows good correlation with the extended TICI (eTICI) reference with an average area under the curve (AUC) score of 0.81. The AUC score is 0.90 with respect to the dichotomized eTICI. In terms of clinical outcome prediction, we demonstrate that autoTICI is overall comparable to eTICI.

Instrument alignment is a common task in various surgical interventions using navigation. The goal of the task is to position and orient an instrument as it has been planned preoperatively. To this end, surgeons rely on patient-specific data visualized on screens alongside preplanned trajectories. The purpose of this manuscript is to investigate the effect of instrument visualization/non visualization on alignment tasks, and to compare it with virtual extensions approach which augments the realistic representation of the instrument with simple 3D objects. 18 volunteers performed six alignment tasks under each of the following conditions: no visualization on the instrument; realistic visualization of the instrument; realistic visualization extended with virtual elements (Virtual extensions). The first condition represents an egocentric-based alignment while the two other conditions additionally make use of exocentric depth estimation to perform the alignment. The device used was a see-through device (Microsoft HoloLens 2). The positions of the head and the instrument were acquired during the experiment. Additionally, the users were asked to fill NASA-TLX and SUS forms for each condition. The results show that instrument visualization is essential for a good alignment using see-through devices. Moreover, virtual extensions helped achieve the best performance compared to the other conditions with medians of 2 mm and 2° positional and angular error respectively. Furthermore, the virtual extensions decreased the average head velocity while similarly reducing the frustration levels. Therefore, making use of virtual extensions could facilitate alignment tasks in augmented and virtual reality (AR/VR) environments, specifically in AR navigated surgical procedures when using optical see-through devices.

Finding automatically multiple lesions in large images is a common problem in medical image analysis. Solving this problem can be challenging if, during optimization, the automated method cannot access information about the location of the lesions nor is given single examples of the lesions. We propose a new weakly supervised detection method using neural networks, that computes attention maps revealing the locations of brain lesions. These attention maps are computed using the last feature maps of a segmentation network optimized only with global image-level labels. The proposed method can generate attention maps at full input resolution without need for interpolation during preprocessing, which allows small lesions to appear in attention maps. For comparison, we modify state-of-the-art methods to compute attention maps for weakly supervised object detection, by using a global regression objective instead of the more conventional classification objective. This regression objective optimizes the number of occurrences of the target object in an image, e.g. the number of brain lesions in a scan, or the number of digits in an image. We study the behavior of the proposed method in MNIST-based detection datasets, and evaluate it for the challenging detection of enlarged perivascular spaces - a type of brain lesion - in a dataset of 2202 3D scans with point-wise annotations in the center of all lesions in four brain regions. In MNIST-based datasets, the proposed method outperforms the other methods. In the brain dataset, the weakly supervised detection methods come close to the human intrarater agreement in each region. The proposed method reaches the best area under the curve in two out of four regions, and has the lowest number of false positive detections in all regions, while its average sensitivity over all regions is similar to that of the other best methods. The proposed method can facilitate epidemiological and clinical studies of enlarged perivascular spaces and help advance research in the etiology of enlarged perivascular spaces and in their relationship with cerebrovascular diseases.

Shape analysis is increasingly becoming important to study changes in brain structures in relation to clinical neurological outcomes. This is a challenging task due to the high dimensionality of shape representations and the often limited number of available shapes. Current techniques counter the poor ratio between dimensions and sample size by using regularization in shape space, but do not take into account the spatial relations within the shapes. This can lead to models that are biologically implausible and difficult to interpret. We propose to use P-spline based regression, which combines a generalized linear model (GLM) with the coefficients described as B-splines and a penalty term that constrains the regression coefficients to be spatially smooth. Owing to the GLM, this method can naturally predict both continuous and discrete outcomes and can include non-spatial covariates without penalization. We evaluated our method on hippocampus shapes extracted from magnetic resonance (MR) images of 510 non-demented, elderly people. We related the hippocampal shape to age, memory score, and sex. The proposed method retained the good performance of current techniques, such as ridge regression, but produced smoother coefficient fields that are easier to interpret.

The goal of this paper is to increase the statistical power of crossing-fiber statistics in voxelwise analyses of diffusion-weighted magnetic resonance imaging (DW-MRI) data. In the proposed framework, a fiber orientation atlas and a model complexity atlas were used to fit the ball-and-sticks model to diffusion-weighted images of subjects in a prospective population-based cohort study. Reproducibility and sensitivity of the partial volume fractions in the ball-and-sticks model were analyzed using TBSS (tract-based spatial statistics) and compared to a reference framework. The reproducibility was investigated on two scans of 30 subjects acquired with an interval of approximately three weeks by studying the intraclass correlation coefficient (ICC). The sensitivity to true biological effects was evaluated by studying the regression with age on 500 subjects from 65 to 90 years old. Compared to the reference framework, the ICC improved significantly when using the proposed framework. Higher t-statistics indicated that regression coefficients with age could be determined more precisely with the proposed framework and more voxels correlated significantly with age. The application of a fiber orientation atlas and a model complexity atlas can significantly improve the reproducibility and sensitivity of crossing-fiber statistics in TBSS.

The collateral score is an important biomarker in decision making for endovascular treatment (EVT) of patients with ischemic stroke. The existing collateral grading systems are based on visual inspection and prone to subjective interpretation and interobserver variation. The purpose of our work is the development of an automatic collateral scoring method. In this work, we present a method that is inspired by human collateral scoring. Firstly, we define an anatomical region by atlas-based registration and extract vessel structures using a deep convolutional neural network. From this, high-level features based on the ratios of vessel length and volume of the occluded and the contralateral side are defined. Multi-class classification models are used to map the feature space to a four-grade collateral score and a quantitative score. The dataset used for training, validation and testing is from a registry of images acquired in clinical routine at multiple medical centers. The model performance is tested on 269 subjects, achieving an accuracy of 0.8. The dichotomized collateral score accuracy is 0.9. The error is comparable to the interobserver variation, the results are comparable to the performance of two radiologists with 10 to 30 years of experience.

Genome-wide association studies (GWAS) link full genome data to a handful of traits. However, in neuroimaging studies, there is an almost unlimited number of traits that can be extracted for full image-wide big data analyses. Large populations are needed to achieve the necessary power to detect statistically significant effects, emphasizing the need to pool data across multiple studies. Neuroimaging consortia, e.g., ENIGMA and CHARGE, are now analyzing MRI data from over 30,000 individuals. Distributed processing protocols extract harmonized features at each site, and pool together only the cohort statistics using meta analysis to avoid data sharing. To date, such MRI projects have focused on single measures such as hippocampal volume, yet voxelwise analyses (e.g., tensor-based morphometry; TBM) may help better localize statistical effects. This can lead to $10^{13}$1013 tests for GWAS and become underpowered. We developed an analytical framework for multi-site TBM by performing multi-channel registration to cohort-specific templates. Our results highlight the reliability of the method and the added power over alternative options while preserving single site specificity and opening the doors for well-powered image-wide genome-wide discoveries.

Multichannel image registration is an important challenge in medical image analysis. Multichannel images result from modalities such as dual-energy CT or multispectral microscopy. Besides, multichannel feature images can be derived from acquired images, for instance, by applying multiscale feature banks to the original images to register. Multichannel registration techniques have been proposed, but most of them are applicable to only two multichannel images at a time. In the present study, we propose to formulate multichannel registration as a groupwise image registration problem. In this way, we derive a method that allows the registration of two or more multichannel images in a fully symmetric manner (i.e., all images play the same role in the registration procedure), and therefore, has transitive consistency by definition. The method that we introduce is applicable to any number of multichannel images, any number of channels per image, and it allows to take into account correlation between any pair of images and not just corresponding channels. In addition, it is fully modular in terms of dissimilarity measure, transformation model, regularisation method, and optimisation strategy. For two multimodal datasets, we computed feature images from the initially acquired images, and applied the proposed registration technique to the newly created sets of multichannel images. MIND descriptors were used as feature images, and we chose total correlation as groupwise dissimilarity measure. Results show that groupwise multichannel image registration is a competitive alternative to the pairwise multichannel scheme, in terms of registration accuracy and insensitivity towards registration reference spaces.

Image registration is an important task in medical image analysis. Whereas most methods are designed for the registration of two images (pairwise registration), there is an increasing interest in simultaneously aligning more than two images using groupwise registration. Multimodal registration in a groupwise setting remains difficult, due to the lack of generally applicable similarity metrics. In this work, a novel similarity metric for such groupwise registration problems is proposed. The metric calculates the sum of the conditional entropy between each image in the group and a representative template image constructed iteratively using principal component analysis. The proposed metric is validated in extensive experiments on synthetic and intrasubject clinical image data. These experiments showed equivalent or improved registration accuracy compared to other state-of-the-art (dis)similarity metrics and improved transformation consistency compared to pairwise mutual information.

HCI plays an important role in interactive medical image segmentation. The Goals, Operators, Methods, and Selection rules (GOMS) model and the National Aeronautics and Space Administration Task Load Index (NASA-TLX) questionnaire are different methods that are often used to evaluate the HCI process. In this article, we aim at improving the HCI process of interactive segmentation using both the GOMS model and the NASA-TLX questionnaire to: 1) identify the relations between these two methods and 2) propose HCI design suggestions based on the synthesis of the evaluation results using both methods. For this, we conducted an experiment where three physicians used two interactive segmentation approaches to segment different types of organs at risk for radiotherapy planning. Using the GOMS model, we identified 16 operators and 10 methods. Further analysis discovered strong relations between the use of GOMS operators and the results of the NASA-TLX questionnaire. Finally, HCI design issues were identified, and suggestions were proposed based on the evaluation results and the identified relations.

B-splines are commonly utilized to construct the transformation model in free-form deformation (FFD) based registration. B-splines become smoother with increasing spline order. However, a higher-order B-spline requires a larger support region involving more control points, which means higher computational cost. In general, the third-order B-spline is considered as a good compromise between spline smoothness and computational cost. A lower-order function is seldom used to construct the transformation model for registration since it is less smooth. In this research, we investigated whether lower-order B-spline functions can be utilized for more efficient registration, while preserving smoothness of the deformation by using a novel random perturbation technique. With the proposed perturbation technique, the expected value of the cost function given probability density function (PDF) of the perturbation is minimized by a stochastic gradient descent optimization. Extensive experiments on 2D synthetically deformed brain images, and real 3D lung and brain scans demonstrated that the novel randomly perturbed free-form deformation (RPFFD) approach improves the registration accuracy and transformation smoothness. Meanwhile, lower-order RPFFD methods reduce the computational cost substantially.

In minimal invasive image guided catheterization procedures, physicians require information of the catheter position with respect to the patient's vasculature. However, in fluoroscopic images, visualization of the vasculature requires toxic contrast agent. Static vasculature roadmapping, which can reduce the usage of iodine contrast, is hampered by the breathing motion in abdominal catheterization. In this paper, we propose a method to track the catheter tip inside the patient's 3D vessel tree using intra-operative single-plane 2D X-ray image sequences and a peri-operative 3D rotational angiography (3DRA). The method is based on a hidden Markov model (HMM) where states of the model are the possible positions of the catheter tip inside the 3D vessel tree. The transitions from state to state model the probabilities for the catheter tip to move from one position to another. The HMM is updated following the observation scores, based on the registration between the 2D catheter centerline extracted from the 2D X-ray image, and the 2D projection of 3D vessel tree centerline extracted from the 3DRA. The method is extensively evaluated on simulated and clinical datasets acquired during liver abdominal catheterization. The evaluations show a median 3D tip tracking error of 2.3 mm with optimal settings in simulated data. The registered vessels close to the tip have a median distance error of 4.7 mm with angiographic data and optimal settings. Such accuracy is sufficient to help the physicians with an up-to-date roadmapping. The method tracks in real-time the catheter tip and enables roadmapping during catheterization procedures.

In this paper, we present a novel approach to automatic 3D facial landmarking using 2D Gabor wavelets. Our algorithm considers the face to be a surface and uses map projections to derive 2D features from raw data. Extracted features include texture, relief map, and transformations thereof. We extend an established 2D landmarking method for simultaneous evaluation of these data. The method is validated by performing landmarking experiments on two data sets using 21 landmarks and compared with an active shape model implementation. On average, landmarking error for our method was 1.9 mm, whereas the active shape model resulted in an average landmarking error of 2.3 mm. A second study investigating facial shape heritability in related individuals concludes that automatic landmarking is on par with manual landmarking for some landmarks. Our algorithm can be trained in 30 min to automatically landmark 3D facial data sets of any size, and allows for fast and robust landmarking of 3D faces.

In this work we describe a 4D registration method for on the fly stabilization of ultrasound volumes for improving image guidance for transjugular intrahepatic portosystemic shunt (TIPS) interventions. The purpose of the method is to enable a continuous visualization of the relevant anatomical planes (determined in a planning stage) in a free breathing patient during the intervention. This requires registration of the planning information to the interventional images, which is achieved in two steps. In the first step tracking is performed across the streaming input. An approximate transformation between the reference image and the incoming image is estimated by composing the intermediate transformations obtained from the tracking. In the second step a subsequent registration is performed between the reference image and the approximately transformed incoming image to account for the accumulation of error. The two step approach helps in reducing the search range and is robust under rotation. We additionally present an approach to initialize and verify the registration. Verification is required when the reference image (containing planning information) is acquired in the past and is not part of the (interventional) 4D ultrasound sequence. The verification score will help in invalidating the registration outcome, for instance, in the case of insufficient overlap or information between the registering images due to probe motion or loss of contact, respectively. We evaluate the method over thirteen 4D US sequences acquired from eight subjects. A graphics processing unit implementation runs the 4D tracking at 9 Hz with a mean registration error of 1.7 mm.

We present a new three-dimensional coupled optimal surface graph-cut algorithm to segment the wall of the carotid artery bifurcation from Magnetic Resonance (MR) images. The method combines the search for both inner and outer borders into a single graph cut and uses cost functions that integrate information from multiple sequences. Our approach requires manual localization of only three seed points indicating the start and end points of the segmentation in the internal, external, and common carotid artery. We performed a quantitative validation using images of 57 carotid arteries. Dice overlap of 0.86 ± 0.06 for the complete vessel and 0.89 ± 0.05 for the lumen compared to manual annotation were obtained. Reproducibility tests were performed in 60 scans acquired with an interval of 15 ± 9 days, showing good agreement between baseline and follow-up segmentations with intraclass correlations of 0.96 and 0.74 for the lumen and complete vessel volumes respectively.

Computer-aided diagnosis of dementia using a support vector machine (SVM) can be improved with feature selection. The relevance of individual features can be quantified from the SVM weights as a significance map (p-map). Although these p-maps previously showed clusters of relevant voxels in dementia-related brain regions, they have not yet been used for feature selection. Therefore, we introduce two novel feature selection methods based on p-maps using a direct approach (filter) and an iterative approach (wrapper). To evaluate these p-map feature selection methods, we compared them with methods based on the SVM weight vector directly, t-statistics, and expert knowledge. We used MRI data from the Alzheimer's disease neuroimaging initiative classifying Alzheimer's disease (AD) patients, mild cognitive impairment (MCI) patients who converted to AD (MCIc), MCI patients who did not convert to AD (MCInc), and cognitively normal controls (CN). Features for each voxel were derived from gray matter morphometry. Feature selection based on the SVM weights gave better results than t-statistics and expert knowledge. The p-map methods performed slightly better than those using the weight vector. The wrapper method scored better than the filter method. Recursive feature elimination based on the p-map improved most for AD-CN: the area under the receiver-operating-characteristic curve (AUC) significantly increased from 90.3% without feature selection to 92.0% when selecting 1.5%-3% of the features. This feature selection method also improved the other classifications: AD-MCI 0.1% improvement in AUC (not significant), MCI-CN 0.7%, and MCIc-MCInc 0.1% (not significant). Although the performance improvement due to feature selection was limited, the methods based on the p-map generally had the best performance, and were therefore better in estimating the relevance of individual features.

In standard B-mode ultrasound (BMUS), segmentation of the lumen of atherosclerotic carotid arteries and studying the lumen geometry over time are difficult owing to irregular lumen shapes, noise, artifacts, and echolucent plaques. Contrast enhanced ultrasound (CEUS) improves lumen visualization, but lumen segmentation remains challenging owing to varying intensities, CEUS-specific artifacts and lack of tissue visualization. To overcome these challenges, we propose a novel method using simultaneously acquired BMUS&CEUS image sequences. Initially, the method estimates nonrigid motion (NME) from the image sequences, using intensity-based image registration. The motion-compensated image sequence is then averaged to obtain a single "epitome" image with improved signal-to-noise ratio. The lumen is segmented from the epitome image through an intensity joint-histogram classification and a graph-based segmentation. NME was validated by comparing displacements with manual annotations in 11 carotids. The average root mean square error (RMSE) was 112±73 μm . Segmentation results were validated against manual delineations in the epitome images of two different datasets, respectively containing 11 (RMSE 191±43 μm) and 10 (RMSE 351±176 μm ) carotids. From the deformation fields, we derived arterial distensibility with values comparable to the literature. The average errors in all experiments were in the inter-observer variability range. To the best of our knowledge, this is the first study exploiting combined BMUS&CEUS images for atherosclerotic carotid lumen segmentation.

Automated segmentation of plaque components in carotid artery magnetic resonance imaging (MRI) is important to enable large studies on plaque vulnerability, and for incorporating plaque composition as an imaging biomarker in clinical practice. Especially supervised classification techniques, which learn from labeled examples, have shown good performance. However, a disadvantage of supervised methods is their reduced performance on data different from the training data, for example on images acquired with different scanners. Reducing the amount of manual annotations required for each new dataset will facilitate widespread implementation of supervised methods. In this paper we segment carotid plaque components of clinical interest (fibrous tissue, lipid tissue, calcification and intraplaque hemorrhage) in a multi-center MRI study. We perform voxelwise tissue classification by traditional same-center training, and compare results with two approaches that use little or no annotated same-center data. These approaches additionally use an annotated set of different-center data. We evaluate 1) a nonlinear feature normalization approach, and 2) two transfer-learning algorithms that use same and different-center data with different weights. Results showed that the best results were obtained for a combination of feature normalization and transfer learning. While for the other approaches significant differences in voxelwise or mean volume errors were found compared with the reference same-center training, the proposed approach did not yield significant differences from that reference. We conclude that both extensive feature normalization and transfer learning can be valuable for the development of supervised methods that perform well on different types of datasets.

The distributed resampling algorithm with non-proportional allocation (RNA) [1] is key to implementing particle filtering applications on parallel computer systems. We extend the original work by Bolić et al. by introducing an adaptive RNA (ARNA) algorithm, improving RNA by dynamically adjusting the particle-exchange ratio and randomizing the process ring topology. This improves the runtime performance of ARNA by about 9% over RNA with 10% particle exchange. ARNA also significantly improves the speed at which information is shared between processing elements, leading to about 20-fold faster convergence. The ARNA algorithm requires only a few modifications to the original RNA, and is hence easy to implement.

2D/3D registration of patient vasculature from preinterventional computed tomography angiography (CTA) to interventional X-ray angiography is of interest to improve guidance in percutaneous coronary interventions. In this paper we present a novel feature based 2D/3D registration framework, that is based on probabilistic point correspondences, and show its usefulness on aligning 3D coronary artery centerlines derived from CTA images with their 2D projection derived from interventional X-ray angiography. The registration framework is an extension of the Gaussian mixture model (GMM) based point-set registration to the 2D/3D setting, with a modified distance metric. We also propose a way to incorporate orientation in the registration, and show its added value for artery registration on patient datasets as well as in simulation experiments. The oriented GMM registration achieved a median accuracy of 1.06 mm, with a convergence rate of 81% for nonrigid vessel centerline registration on 12 patient datasets, using a statistical shape model. The method thereby outperformed the iterative closest point algorithm, the GMM registration without orientation, and two recently published methods on 2D/3D coronary artery registration.

Multiresolution strategies are commonly used in the nonrigid registration to avoid local minima in the optimization space. Generally, a step-by-step hierarchical approach is adopted, in which the registration starts on a level with reduced complexity (downsampled images, global transformations), then continuing to levels with increased complexity, until the finest level is reached. In this paper, we propose two alternative multiresolution strategies for both the data and transformation models, in which different resolution levels are considered simultaneously instead of subsequently. Through combining the different strategies for data and transformation, we systematically define 3 × 3 multiresolution schemes, including both existing and novel methods. Experiments on 10 pairs of computed tomography lung data sets showed that the best performing strategy resulted in a reduction of the upper quartile of the mean target registration error from 2 to 1.5 mm, compared with the conventionally hierarchical multiresolution method, while achieving smoother deformations. Experiments with intersubject registration of 18 3D T1-weighted MRI brain scans confirmed that simultaneous multiresolution strategies produce more accurate registration results (median of mean overlap increased from 0.55 to 0.57) and smoother deformation fields than the traditionally hierarchical method. Evaluation of robustness indicated that the largest differences in accuracy between methods are observed for structures with a relatively large initial misalignment.

A method for registering preoperative 3D+t coronary CTA with intraoperative monoplane 2D+t X-ray angiography images is proposed to improve image guidance during minimally invasive coronary interventions. The method uses a patient-specific dynamic coronary model, which is derived from the CTA scan by centerline extraction and motion estimation. The dynamic coronary model is registered with the 2D+t X-ray sequence, considering multiple X-ray time points concurrently, while taking breathing induced motion into account. Evaluation was performed on 26 datasets of 17 patients by comparing projected model centerlines with manually annotated centerlines in the X-ray images. The proposed 3D+t/2D+t registration method performed better than a 3D/2D registration method with respect to the accuracy and especially the robustness of the registration. Registration with a median error of 1.47 mm was achieved.

Studying joint kinematics is of interest to improve prosthesis design and to characterize postoperative motion. State of the art techniques register bones segmented from prior computed tomography or magnetic resonance scans with X-ray fluoroscopic sequences. Elimination of the prior 3D acquisition could potentially lower costs and radiation dose. Therefore, we propose to substitute the segmented bone surface with a statistical shape model based estimate. A dedicated dynamic reconstruction and tracking algorithm was developed estimating the shape based on all frames, and pose per frame. The algorithm minimizes the difference between the projected bone contour and image edges. To increase robustness, we employ a dynamic prior, image features, and prior knowledge about bone edge appearances. This enables tracking and reconstruction from a single initial pose per sequence. We evaluated our method on the distal femur using eight biplane fluoroscopic drop-landing sequences. The proposed dynamic prior and features increased the convergence rate of the reconstruction from 71% to 91%, using a convergence limit of 3 mm. The achieved root mean square point-to-surface accuracy at the converged frames was 1.48 ± 0.41 mm. The resulting tracking precision was 1-1.5 mm, with the largest errors occurring in the rotation around the femoral shaft (about 2.5° precision).

Accurate automated brain structure segmentation methods facilitate the analysis of large-scale neuroimaging studies. This work describes a novel method for brain structure segmentation in magnetic resonance images that combines information about a structure's location and appearance. The spatial model is implemented by registering multiple atlas images to the target image and creating a spatial probability map. The structure's appearance is modeled by a classifier based on Gaussian scale-space features. These components are combined with a regularization term in a Bayesian framework that is globally optimized using graph cuts. The incorporation of the appearance model enables the method to segment structures with complex intensity distributions and increases its robustness against errors in the spatial model. The method is tested in cross-validation experiments on two datasets acquired with different magnetic resonance sequences, in which the hippocampus and cerebellum were segmented by an expert. Furthermore, the method is compared to two other segmentation techniques that were applied to the same data. Results show that the atlas- and appearance-based method produces accurate results with mean Dice similarity indices of 0.95 for the cerebellum, and 0.87 for the hippocampus. This was comparable to or better than the other methods, whereas the proposed technique is more widely applicable and robust.

State of the art cardiac computed tomography (CT) enables the acquisition of imaging data of the heart over the entire cardiac cycle at concurrent high spatial and temporal resolution. However, in clinical practice, acquisition is increasingly limited to 3-D images. Estimating the shape of the cardiac structures throughout the entire cardiac cycle from a 3-D image is therefore useful in applications such as the alignment of preoperative computed tomography angiography (CTA) to intra-operative X-ray images for improved guidance in coronary interventions. We hypothesize that the motion of the heart is partially explained by its shape and therefore investigate the use of three regression methods for motion estimation from single-phase shape information. Quantitative evaluation on 150 4-D CTA images showed a small, but statistically significant, increase in the accuracy of the predicted shape sequences when using any of the regression methods, compared to shape-independent motion prediction by application of the mean motion. The best results were achieved using principal component regression resulting in point-to-point errors of 2.3±0.5 mm, compared to values of 2.7±0.6 mm for shape-independent motion estimation. Finally, we showed that this significant difference withstands small variations in important parameter settings of the landmarking procedure.

This paper presents a vessel segmentation method which learns the geometry and appearance of vessels in medical images from annotated data and uses this knowledge to segment vessels in unseen images. Vessels are segmented in a coarse-to-fine fashion. First, the vessel boundaries are estimated with multivariate linear regression using image intensities sampled in a region of interest around an initialization curve. Subsequently, the position of the vessel boundary is refined with a robust nonlinear regression technique using intensity profiles sampled across the boundary of the rough segmentation and using information about plausible cross-sectional vessel shapes. The method was evaluated by quantitatively comparing segmentation results to manual annotations of 229 coronary arteries. On average the difference between the automatically obtained segmentations and manual contours was smaller than the inter-observer variability, which is an indicator that the method outperforms manual annotation. The method was also evaluated by using it for centerline refinement on 24 publicly available datasets of the Rotterdam Coronary Artery Evaluation Framework. Centerlines are extracted with an existing method and refined with the proposed method. This combination is currently ranked second out of 10 evaluated interactive centerline extraction methods. An additional qualitative expert evaluation in which 250 automatic segmentations were compared to manual segmentations showed that the automatically obtained contours were rated on average better than manual contours.

MOTIVATION: Complete, accurate and reproducible analysis of intracellular foci from fluorescence microscopy image sequences of live cells requires full automation of all processing steps involved: cell segmentation and tracking followed by foci segmentation and pattern analysis. Integrated systems for this purpose are lacking. RESULTS: Extending our previous work in cell segmentation and tracking, we developed a new system for performing fully automated analysis of fluorescent foci in single cells. The system was validated by applying it to two common tasks: intracellular foci counting (in DNA damage repair experiments) and cell-phase identification based on foci pattern analysis (in DNA replication experiments). Experimental results show that the system performs comparably to expert human observers. Thus, it may replace tedious manual analyses for the considered tasks, and enables high-content screening. AVAILABILITY AND IMPLEMENTATION: The described system was implemented in MATLAB (The MathWorks, Inc., USA) and compiled to run within the MATLAB environment. The routines together with four sample datasets are available at http://celmia.bigr.nl/. The software is planned for public release, free of charge for non-commercial use, after publication of this article.

Studying intracellular dynamics is of fundamental importance for understanding healthy life at the molecular level and for developing drugs to target disease processes. One of the key technologies to enable this research is the automated tracking and motion analysis of these objects in microscopy image sequences. To make better use of the spatiotemporal information than common frame-by-frame tracking methods, two alternative approaches have recently been proposed, based upon either Bayesian estimation or space-time segmentation. In this paper, we propose to combine the power of both approaches, and develop a new probabilistic method to segment the traces of the moving objects in kymograph representations of the image data. It is based on variable-rate particle filtering and uses multiscale trend analysis of the extracted traces to estimate the relevant kinematic parameters. Experiments on realistic synthetically generated images as well as on real biological image data demonstrate the improved potential of the new method for the analysis of microtubule dynamics in vitro.

Cell segmentation and tracking in time-lapse fluorescence microscopy images is a task of fundamental importance in many biological studies on cell migration and proliferation. In recent years, level sets have been shown to provide a very appropriate framework for this purpose, as they are well suited to capture topological changes occurring during mitosis, and they easily extend to higher dimensional image data. This model evolution approach has also been extended to deal with many cells concurrently. Notwithstanding its high potential, the multiple-level-set method suffers from a number of shortcomings, which limit its applicability to a larger variety of cell biological imaging studies. In this paper, we propose several modifications and extensions to the coupled-active-surfaces algorithm, which considerably improve its robustness and applicability. Our algorithm was validated by comparing it to the original algorithm and two other cell segmentation algorithms. For the evaluation, four real fluorescence microscopy image datasets were used, involving different cell types and labelings that are representative of a large range of biological experiments. Improved tracking performance in terms of precision (up to 11%), recall (up to 8%), ability to correctly capture all cell division events, and computation time (up to nine times reduction) is achieved.

In the above titled paper (ibid., vol. 29, no. 3, pp. 852-867, Mar. 10), several figure citations were incorrect. The correct figure citations are provided here.

Quantitative analysis of biological image data generally involves the detection of many subresolution spots. Especially in live cell imaging, for which fluorescence microscopy is often used, the signal-to-noise ratio (SNR) can be extremely low, making automated spot detection a very challenging task. In the past, many methods have been proposed to perform this task, but a thorough quantitative evaluation and comparison of these methods is lacking in the literature. In this paper, we evaluate the performance of the most frequently used detection methods for this purpose. These include seven unsupervised and two supervised methods. We perform experiments on synthetic images of three different types, for which the ground truth was available, as well as on real image data sets acquired for two different biological studies, for which we obtained expert manual annotations to compare with. The results from both types of experiments suggest that for very low SNRs ( approximately 2), the supervised (machine learning) methods perform best overall. Of the unsupervised methods, the detectors based on the so-called h -dome transform from mathematical morphology or the multiscale variance-stabilizing transform perform comparably, and have the advantage that they do not require a cumbersome learning stage. At high SNRs ( > 5), the difference in performance of all considered detectors becomes negligible.

A novel method is presented for carotid artery vessel wall segmentation in computed tomography angiography (CTA) data. First the carotid lumen is semi-automatically segmented using a level set approach initialized with three seed points. Subsequently, calcium regions located within the vessel wall are automatically detected and classified using multiple features in a GentleBoost framework. Calcium regions segmentation is used to improve localization of the outer vessel wall because it is an easier task than direct outer vessel wall segmentation. In a third step, pixels outside the lumen area are classified as vessel wall or background, using the same GentleBoost framework with a different set of image features. Finally, a 2-D ellipse shape deformable model is fitted to a cost image derived from both the calcium and vessel wall classifications. The method has been validated on a dataset of 60 CTA images. The experimental results show that the accuracy of the method is comparable to the interobserver variability.

Visualization and quantification of small structures with computed tomography (CT) is hampered by the limited spatial resolution of the system. Histogram-based selective deblurring (HiSD) is a deconvolution method that restores small high-density structures, i.e., calcifications, of a CT image, using the high-intensity voxel information of the deconvolved image, while preserving the original hounsfield Units (HUs) in the remaining tissues. In this study, high resolution micro-CT data are used to validate the potential of HiSD to improve calcium visualization and quantification in the carotid arteries on in vivo contrast-enhanced CTA data. The evaluation is performed qualitatively and quantitatively on 15 atherosclerotic plaques obtained from ten different patients. HiSD in combination with vessel segmentation significantly improves calcification visualization and quantification on in vivo contrast-enhanced CT images. Calcification blur is reduced, while avoiding noise amplification and edge-ringing artifacts in the surrounding tissues. Calcification quantification errors are reduced by 23.5% on average.

Computed tomography (CT) has a trend towards higher resolution and higher noise. This development has increased the interest in anisotropic smoothing techniques for CT, which aim to reduce noise while preserving structures of interest. However, existing smoothing techniques are slow, which makes clinical application difficult. Furthermore, the published methods have limitations with respect to preserving small details in CT data. This paper presents a widely applicable speed optimized framework for anisotropic smoothing techniques. A second contribution of this paper is an extension to an existing smoothing technique aimed at better preserving small structures of interest in CT data. Based on second-order image structure, the method first determines an importance map, which indicates potentially relevant structures that should be preserved. Subsequently an anisotropic diffusion process is started. The diffused data is used in most parts of the images, while structures with significant second-order information are preserved. The method is qualitatively evaluated against an anisotropic diffusion method without structure preservation in an observer study to assess the improvement of 3-D visualizations of CT series and quantitatively by determining the reduction of the difference between low and high dose CT scans of in vitro carotid plaques.

Quantitative analysis of dynamic processes in living cells by means of fluorescence microscopy imaging requires tracking of hundreds of bright spots in noisy image sequences. Deterministic approaches, which use object detection prior to tracking, perform poorly in the case of noisy image data. We propose an improved, completely automatic tracker, built within a Bayesian probabilistic framework. It better exploits spatiotemporal information and prior knowledge than common approaches, yielding more robust tracking also in cases of photobleaching and object interaction. The tracking method was evaluated using simulated but realistic image sequences, for which ground truth was available. The results of these experiments show that the method is more accurate and robust than popular tracking methods. In addition, validation experiments were conducted with real fluorescence microscopy image data acquired for microtubule growth analysis. These demonstrate that the method yields results that are in good agreement with manual tracking performed by expert cell biologists. Our findings suggest that the method may replace laborious manual procedures.

Recently, foundations rooted in physics have been laid down for the goal of simulating the propagation of a guide wire inside the vasculature. At the heart of the simulation lies the fundamental task of energy minimization. The energy comes from interaction with the vessel wall and the bending of the guide wire. For the simulation to be useful in actual training, obtaining the smallest possible optimization time is key. In this paper, we, therefore, study the influence of using different optimization techniques: a semianalytical approximation algorithm, the conjugate-gradients algorithm, and an evolutionary algorithm (EA), specifically the GLIDE algorithm. Simulation performance has been measured on phantom data. The results show that a substantial reduction in time can be obtained while the error is increased only slightly if conjugate gradients or GLIDE is used.

An approach to 3-D vessel axis tracking based on surface evolution is presented. The main idea is to guide the evolution of the surface by analyzing its skeleton topology during evolution, and imposing shape constraints on the topology. For example, the intermediate topology can be processed such that it represents a single vessel segment, a bifurcation, or a more complex vascular topology. The evolving surface is then reinitialized with the newly found topology. Reinitialization is a crucial step since it creates probing behavior of the evolving front, encourages the segmentation process to extract the vascular structure of interest and reduces the risk on leaking of the curve into the background. The method was evaluated in two computed tomography angiography applications: 1) extracting the internal carotid arteries including the region in which they traverse through the skull base, which is challenging due to the proximity of bone structures and overlap in intensity values; 2) extracting the carotid bifurcations including many cases in which they are severely stenosed and contain calcifications. The vessel axis was found in 90% (18/20 internal carotids in ten patients) and 70% (14/20 carotid bifurcations in a different set of ten patients) of the cases.

In this paper, three different calibration methods for three-dimensional (3-D) freehand ultrasound (US) are evaluated. Calibration is the process of estimating the rigid transformation from US image coordinates to the coordinate system of the tracking sensor mounted onto the probe. Calibration accuracy has an important impact on quantitative studies. Geometrical precision can also be crucial in many interventions and surgery. The proposed evaluation framework relies on a single point phantom and a 3-D US phantom which mimics the US characteristics of human liver. Four quality measures are used: 3-D point localization criterion, distance and volume measurements, and shape based criterion. Results show that during the acquisition procedure, volumetric measurements and shapes of the reconstructed object depend on probe motion used, particularly fan motions for which errors are larger. It is also shown that accurate calibration is essential to obtain reliable quantitative information.

In the past few years, a number of two-dimensional (2-D) to three-dimensional (3-D) (2-D-3-D) registration algorithms have been introduced. However, these methods have been developed and evaluated for specific applications, and have not been directly compared. Understanding and evaluating their performance is therefore an open and important issue. To address this challenge we introduce a standardized evaluation methodology, which can be used for all types of 2-D-3-D registration methods and for different applications and anatomies. Our evaluation methodology uses the calibrated geometry of a 3-D rotational X-ray (3DRX) imaging system (Philips Medical Systems, Best, The Netherlands) in combination with image-based 3-D-3-D registration for attaining a highly accurate gold standard for 2-D X-ray to 3-D MR/CT/3DRX registration. Furthermore, we propose standardized starting positions and failure criteria to allow future researchers to directly compare their methods. As an illustration, the proposed methodology has been used to evaluate the performance of two 2-D-3-D registration techniques, viz. a gradient-based and an intensity-based method, for images of the spine. The data and gold standard transformations are available on the internet (http://www.isi.uu.nl/Research/Databases/).

This paper presents a new method for deformable model-based segmentation of lumen and thrombus in abdominal aortic aneurysms from computed tomography (CT) angiography (CTA) scans. First the lumen is segmented based on two positions indicated by the user, and subsequently the resulting surface is used to initialize the automated thrombus segmentation method. For the lumen, the image-derived deformation term is based on a simple grey level model (two thresholds). For the more complex problem of thrombus segmentation, a grey level modeling approach with a nonparametric pattern classification technique is used, namely k-nearest neighbors. The intensity profile sampled along the surface normal is used as classification feature. Manual segmentations are used for training the classifier: samples are collected inside, outside, and at the given boundary positions. The deformation is steered by the most likely class corresponding to the intensity profile at each vertex on the surface. A parameter optimization study is conducted, followed by experiments to assess the overall segmentation quality and the robustness of results against variation in user input. Results obtained in a study of 17 patients show that the agreement with respect to manual segmentations is comparable to previous values reported in the literature, with considerable less user interaction.

A method has been developed that, based on the guide wire position in monoplane fluoroscopic images, visualizes the approximate guide wire position in the three-dimensional (3-D) vasculature, that is obtained prior to the intervention with 3-D rotational X-ray angiography (3DRA). The method assumes the position of the guide wire in the fluoroscopic images is known. A two-dimensional feature image is determined from the 3DRA data. In this feature image, the guide wire position is determined in a two-step approach: a mincost algorithm is used to determine a suitable position for the guide wire, and subsequently a snake optimization technique is applied to move the guide wire to a better position. The resulting guide wire can then be visualized in 3-D in combination with the 3DRA dataset. The reconstruction accuracy of the method has been evaluated using a 3DRA image of a vascular phantom filled with contrast, and monoplane fluoroscopic images of the same phantom without contrast and with a guide wire inserted. The evaluation has been performed for different projection angles, and with different parameters for the method. The final result does not appear to be very sensitive to the parameters of the method. The average mean error of the estimated 3-D guide wire position is 1.5 mm, and the average tip distance is 2.3 mm. The effect of inaccurate C-arm geometry information is also investigated. Small errors in geometry information (up to 1 degrees) will slightly decrease the 3-D reconstruction accuracies, with an error of at most 1 mm. The feasibility of this approach on clinical data is demonstrated.

A method is introduced to examine the geometrical accuracy of the three-dimensional (3-D) representation of coronary arteries from multiple (two and more) calibrated two-dimensional (2-D) angiographic projections. When involving more then two projections, (multiprojection modeling) a novel procedure is presented that consists of fully automated centerline and width determination in all available projections based on the information provided by the semi-automated centerline detection in two initial calibrated projections. The accuracy of the 3-D coronary modeling approach is determined by a quantitative examination of the 3-D centerline point position and the 3-D cross sectional area of the reconstructed objects. The measurements are based on the analysis of calibrated phantom and calibrated coronary 2-D projection data. From this analysis a confidence region (alpha degrees approximately equal to [35 degrees - 145 degrees]) for the angular distance of two initial projection images is determined for which the modeling procedure is sufficiently accurate for the applied system. Within this angular border range the centerline position error is less then 0.8 mm, in terms of the Euclidean distance to a predefined ground truth. When involving more projections using our new procedure, experiments show that when the initial pair of projection images has an angular distance in the range alpha degrees approximately equal to [35 degrees - 145 degrees], the centerlines in all other projections (gamma = 0 degrees - 180 degrees) were indicated very precisely without any additional centering procedure. When involving additional projection images in the modeling procedure a more realistic shape of the structure can be provided. In case of the concave segment, however, the involvement of multiple projections does not necessarily provide a more realistic shape of the reconstructed structure.

A method is presented to interpolate between neighboring slices in a grey-scale tomographic data set. Spatial correspondence between adjacent slices is established using a nonrigid registration algorithm based on B-splines which optimizes the normalized mutual information similarity measure. Linear interpolation of the image intensities is then carried out along the directions calculated by the registration algorithm. The registration-based method is compared to both standard linear interpolation and shape-based interpolation in 20 tomographic data sets. Results show that the proposed method statistically significantly outperforms both linear and shape-based interpolation.

A method is presented that uses a vectorial multiscale feature image for wave front propagation between two or more user defined points to retrieve the central axis of tubular objects in digital images. Its implicit scale selection mechanism makes the method more robust to overlap and to the presence of adjacent structures than conventional techniques that propagate a wave front over a scalar image representing the maximum of a range of filters. The method is shown to retain its potential to cope with severe stenoses or imaging artifacts and objects with varying widths in simulated and actual two-dimensional angiographic images.

Using three-dimensional rotational X-ray angiography (3DRA), three-dimensional (3-D) information of the vasculature can be obtained prior to endovascular interventions. However, during interventions, the radiologist has to rely on fluoroscopy images to manipulate the guide wire. In order to take full advantage of the 3-D information from 3DRA data during endovascular interventions, a method is presented that yields an integrated display of the position of the guide wire and vasculature in 3-D. The method relies on an automated method that tracks the guide wire simultaneously in biplane fluoroscopy images. Based on the calibrated geometry of the C-arm, the 3-D guide-wire position is determined and visualized in the 3-D coordinate system of the vasculature. The method is evaluated in an intracranial anthropomorphic vascular phantom. The influence of the angle between projections, distortion correction of the projection images, and accuracy of geometry knowledge on the accuracy of 3-D guide-wire reconstruction from biplane images is determined. If the calibrated geometry information is used and the images are corrected for distortion, a mean distance to the reference standard of 0.42 mm and a tip distance of 0.65 mm is found, which means that accurate guide-wire reconstruction from biplane images can be performed.

A method is presented to extract and track the position of a guide wire during endovascular interventions under X-ray fluoroscopy. The method can be used to improve guide-wire visualization in low-quality fluoroscopic images and to estimate the position of the guide wire in world coordinates. A two-step procedure is utilized to track the guide wire in subsequent frames. First, a rough estimate of the displacement is obtained using a template-matching procedure. Subsequently, the position of the guide wire is determined by fitting a spline to a feature image. The feature images that have been considered enhance line-like structures on: 1) the original images; 2) subtraction images; and 3) preprocessed images in which coherent structures are enhanced. In the optimization step, the influence of the scale at which the feature is calculated and the additional value of using directional information is investigated. The method is evaluated on 267 frames from ten clinical image sequences. Using the automatic method, the guide wire could be tracked in 96% of the frames, with a similar accuracy to three observers, although the position of the tip was estimated less accurately.

Blood pool agents (BPAs) for contrast-enhanced (CE) magnetic-resonance angiography (MRA) allow prolonged imaging times for higher contrast and resolution. Imaging is performed during the steady state when the contrast agent is distributed through the complete vascular system. However, simultaneous venous and arterial enhancement in this steady state hampers interpretation. In order to improve visualization of the arteries and veins from steady-state BPA data, a semiautomated method for artery-vein separation is presented. In this method, the central arterial axis and central venous axis are used as initializations for two surfaces that simultaneously evolve in order to capture the arterial and venous parts of the vasculature using the level-set framework. Since arteries and veins can be in close proximity of each other, leakage from the evolving arterial (venous) surface into the venous (arterial) part of the vasculature is inevitable. In these situations, voxels are labeled arterial or venous based on the arrival time of the respective surface. The evolution is steered by external forces related to feature images derived from the image data and by internal forces related to the geometry of the level sets. In this paper, the robustness and accuracy of three external forces (based on image intensity, image gradient, and vessel-enhancement filtering) and combinations of them are investigated and tested on seven patient datasets. To this end, results with the level-set-based segmentation are compared to the reference-standard manually obtained segmentations. Best results are achieved by applying a combination of intensity- and gradient-based forces and a smoothness constraint based on the curvature of the surface. By applying this combination to the seven datasets, it is shown that, with minimal user interaction, artery-vein separation for improved arterial and venous visualization in BPA CE-MRA can be achieved.

Blood pool agents (BPAs) for contrast-enhanced magnetic resonance angiography (CE-MRA) allow prolonged imaging during the steady state when the agent is distributed through the complete vascular system. This increases both the spatial resolution and the contrast resolution. However, simultaneous venous and arterial enhancement hampers interpretation. For the pelvic region of the vasculature, it is shown that arterial visualization in this equilibrium phase can be improved if the central arterial axis (CAA) is known. However, manually obtaining this axis is not feasible in clinical practice. Therefore, a method is presented that utilizes images acquired during the first pass of the contrast agent to find the CAA in the steady-state data with minimum user initialization. The accuracy of the resulting CAA is compared with tracings of three observers in six patient datasets. It was found that the mean difference between the semiautomatic method and the manual delineation is 1.32 mm in the steady-state data, and that the resulting CAA was always within the arterial lumen, which is an important prerequisite for both improved visualization and segmentation.

A method for localization and segmentation of bifurcated aortic endografts in computed tomographic angiography (CTA) images is presented. The graft position is determined by detecting radiopaque markers sewn on the outside of the graft. The user indicates the first and the last marker, whereupon the remaining markers are automatically detected. This is achieved by first detecting marker-like structures through second-order scaled derivative analysis, which is combined with prior knowledge of graft shape and marker configuration. The identified marker centers approximate the graft sides and, derived from these, the central axis. The graft boundary is determined by maximizing the local gradient in the radial direction along a deformable contour passing through both sides. Three segmentation methods were tested. The first performs graft contour detection in the initial CT-slices, the second in slices that were reformatted to be orthogonal to the approximated graft axis, and the third uses the segmentation from the second method to find a more reliable approximation of the axis and subsequently performs contour detection. The methods have been applied to ten CTA images and the results were compared to manual marker indication by one observer and region growing aided segmentation by three observers. Out of a total of 266 markers, 262 were detected. Adequate approximations of the graft sides were obtained in all cases. The best segmentation results were obtained using a second iteration orthogonal to the axis determined from the first segmentation, yielding an average relative volume of overlap with the expert segmentations of 92%, while the interexpert reproducibility is 95%. The averaged difference in volume measured by the automated method and by the experts equals the difference among the experts: 3.5%.

Udgivelsesdato: June

A novel method is introduced for the generation of landmarks for three-dimensional (3-D) shapes and the construction of the corresponding 3-D statistical shape models. Automatic landmarking of a set of manual segmentations from a class of shapes is achieved by 1) construction of an atlas of the class, 2) automatic extraction of the landmarks from the atlas, and 3) subsequent propagation of these landmarks to each example shape via a volumetric nonrigid registration technique using multiresolution B-spline deformations. This approach presents some advantages over previously published methods: it can treat multiple-part structures and requires less restrictive assumptions on the structure's topology. In this paper, we address the problem of building a 3-D statistical shape model of the left and right ventricle of the heart from 3-D magnetic resonance images. The average accuracy in landmark propagation is shown to be below 2.2 mm. This application demonstrates the robustness and accuracy of the method in the presence of large shape variability and multiple objects.

Three-dimensional (3-D) imaging of the heart is a rapidly developing area of research in medical imaging. Advances in hardware and methods for fast spatio-temporal cardiac imaging are extending the frontiers of clinical diagnosis and research on cardiovascular diseases. In the last few years, many approaches have been proposed to analyze images and extract parameters of cardiac shape and function from a variety of cardiac imaging modalities. In particular, techniques based on spatio-temporal geometric models have received considerable attention. This paper surveys the literature of two decades of research on cardiac modeling. The contribution of the paper is three-fold: 1) to serve as a tutorial of the field for both clinicians and technologists, 2) to provide an extensive account of modeling techniques in a comprehensive and systematic manner, and 3) to critically review these approaches in terms of their performance and degree of clinical evaluation with respect to the final goal of cardiac functional analysis. From this review it is concluded that whereas 3-D model-based approaches have the capability to improve the diagnostic value of cardiac images, issues as robustness, 3-D interaction, computational complexity and clinical validation still require significant attention.

The decrease in the volume of viable tumor is an indicator for the effect preoperative chemotherapy has on bone tumors. We develop an approach for segmenting dynamic perfusion MR-images into viable tumor, nonviable tumor and healthy tissue. Two cascaded feedforward neural networks are trained to perform the pixel-based segmentation. As features, we use the parameters obtained from a pharmacokinetic model of the tissue perfusion (parametric images). Additional multiscale features that incorporate contextual information are included. Experiments indicate that multiscale blurred versions of the parametric images together with a multiscale formulation of the local image entropy are the most discriminative features.

Describes the use of heuristics in the determination of a minimum cost path between two points in digital images. The application of four different search methods when applied in two and three dimensional digital images is presented and evaluated. Experiments show that the number of nodes that are being addressed in the search process strongly depends on the discriminative power of the feature used. Furthermore it is shown that for a specific application, the use of a simple heuristic function leads to a considerable reduction in the number of evaluated nodes as compared with the traditional unidirectional approach.

Orbital soft-tissue motion analysis aids in the localization and diagnosis of orbital disorders. A technique has been developed to objectively quantify and visualize motion in the orbit during gaze. T1-weighted MR volume sequences are acquired during gaze and soft-tissue motion is quantified using optical flow techniques. The flow field is visualized using color-coding: orientation of the flow vector is coded by hue and magnitude by saturation of the pixel. Current clinical circumstances limit MR image acquisition to short sequences and short acquisition times. The effect of these limitations on the performance of optical flow computation has been studied for four representative optical flow algorithms: on short (nine frames) and long (21 frames) simulated sequences of rotation of a magnetic resonance (MR) imaged object, on short measured MR sequences of controlled rotation of the same object and on short MR sequences of motion in the orbit. On the short simulated and motion-controlled sequences, the Lucas and Kanade algorithm showed the best performance with respect to both accuracy and robustness. These motion estimates were accurate to within 20%. Motion in the orbit ranged between 0.05 and 0.25 mm/degree gaze. Color-coding was found to be attractive as a visualization technique, because it shows both magnitude and orientation of all flow vectors without cluttering.

A method is presented which aids the clinician in obtaining quantitative measures and a three-dimensional (3-D) representation of vessels from 3-D angiographic data with a minimum of user interaction. Based on two user defined starting points, an iterative procedure tracks the central vessel axis. During the tracking process, the minimum diameter and a surface rendering of the vessels are computed, allowing for interactive inspection of the vasculature. Applications of the method to CTA, contrast enhanced (CE)-MRA and phase contrast (PC)-MRA images of the abdomen are shown. In all applications, a long stretch of vessels with varying width is tracked, delineated, and visualized, in less than 10 s on a standard clinical workstation.

In digital subtraction angiography (DSA), patient motion is the primary cause of image quality degradation. The motion correction algorithms developed so far were not sufficiently fast so as to be suitable for integration in a clinical setting. In this paper we describe a new image registration technique for motion artifact reduction in DSA which is fully automatic, effective, and computationally very efficient. Using an image content driven control point selection mechanism and modern graphics hardware for image warping, the algorithm requires less than one second per DSA image (on average). Preliminary experiments on cerebral DSA images illustrate the applicability of the technique.

A classical approach to interpolation of sampled data is polynomial interpolation. However, from the sampling theorem it follows that the ideal approach to interpolation is to convolve the given samples with the sinc function. In this paper we study the properties of the sinc-approximating kernels that can be derived from the Lagrange central interpolation scheme. Both the finite-extent properties and the convergence property are analyzed. The Lagrange central interpolation kernels of up to ninth order are compared to cardinal splines of corresponding orders, both by spectral analysis and by rotation experiments on real-life test-images. It is concluded that cardinal spline interpolation is by far superior.

Quantification of the degree of stenosis or vessel dimensions are important for diagnosis of vascular diseases and planning vascular interventions. Although diagnosis from three-dimensional (3-D) magnetic resonance angiograms (MRA's) is mainly performed on two-dimensional (2-D) maximum intensity projections, automated quantification of vascular segments directly from the 3-D dataset is desirable to provide accurate and objective measurements of the 3-D anatomy. A model-based method for quantitative 3-D MRA is proposed. Linear vessel segments are modeled with a central vessel axis curve coupled to a vessel wall surface. A novel image feature to guide the deformation of the central vessel axis is introduced. Subsequently, concepts of deformable models are combined with knowledge of the physics of the acquisition technique to accurately segment the vessel wall and compute the vessel diameter and other geometrical properties. The method is illustrated and validated on a carotid bifurcation phantom, with ground truth and medical experts as comparisons. Also, results on 3-D time-of-flight (TOF) MRA images of the carotids are shown. The approach is a promising technique to assess several geometrical vascular parameters directly on the source 3-D images, providing an objective mechanism for stenosis grading.

Digital subtraction angiography (DSA) is a well-established modality for the visualization of blood vessels in the human body. A serious disadvantage of this technique, inherent to the subtraction operation, is its sensitivity to patient motion. The resulting artifacts frequently reduce the diagnostic value of the images. Over the past two decades, many solutions to this problem have been put forward. In this paper, we give an overview of the possible types of motion artifacts and the techniques that have been proposed to avoid them. The main purpose of this paper is to provide a detailed review and discussion of retrospective motion correction techniques that have been described in the literature, to summarize the conclusions that can be drawn from these studies, and to provide suggestions for future research.

In MR brain images, segmentation using intensity values is severely limited owing to field inhomogeneities, susceptibility artifacts and partial volume effects. Edge based segmentation methods suffer from spurious edges and gaps in boundaries. A method is presented which combines the advantages of edge based and region based segmentation. First a multiscale image representation, is constructed which favors intratissue diffusion over inter-tissue diffusion by exploiting local contrast. Subsequently a multiscale linking model (the hyperstack) is used to group voxels into a number of segments. This facilitates segmentation of grey matter, white matter and cerebrospinal fluid with minimal user interaction. Using a supervised segmentation, technique and MR simulations of a brain phantom as validation it is shown that the errors are in the order of or smaller than reported in literature.

In this paper implicit representations of deformable models for medical image enhancement and segmentation are considered. The advantage of implicit models over classical explicit models is that their topology can be naturally adapted to objects in the scene. A geodesic formulation of implicit deformable models is especially attractive since it has the energy minimizing properties of classical models. The aim of this paper is twofold. First, a modification to the customary geodesic deformable model approach is introduced by considering all the level sets in the image as energy minimizing contours. This approach is used to segment multiple objects simultaneously and for enhancing and segmenting cardiac computed tomography (CT) and magnetic resonance images. Second, the approach is used to effectively compare implicit and explicit models for specific tasks. This shows the complementary character of implicit models since in case of poor contrast boundaries or gaps in boundaries e.g. due to partial volume effects, noise, or motion artifacts, they do not perform well, since the approach is completely data-driven.

In most cases nonlinear diffusion filtering is implemented by means of explicit finite difference schemes. These algorithms are not very efficient, since they are only stable for small time steps. We address this problem by presenting unconditionally stable semi-implicit schemes which are based on an additive operator splitting (AOS). They are very efficient since they can be implemented by recursive filtering, and their separability allows a straightforward implementation in any dimension. We analyse their behaviour on a parallel computer and demonstrate that parallel AOS schemes on a modern shared-memory multiprocessor system with 8 processors allow a speed-up of two orders of magnitude in comparison to the widely-used explicit scheme on a single processor.

In optic flow based velocity estimation the image brightness constraint equation is used. However, for measurements performed at a certain scale, the brightness constraint equation does not apply. We therefore use a recently developed approach which reconciles optic flow and scale space theory. It specifically incorporates the scale (aperture) of image measurements, leading to a scheme which is essentially different from existing approaches. To obtain a unique velocity field, the data-derived information has to be augmented with physical knowledge. By keeping a strict separation between data-derived and external information, we can locally adapt or modify the user-supplied information without affecting the image-derived information. The two free scale parameters in time and space can be used for attentive vision (selecting particular velocities or objects) and to improve the reliability of velocity estimates.

Multiscale approaches are an invaluable tool for image segmentation. A vast amount of research has been devoted to the construction of different multiscale representations of an image. In this paper we use the hyperstack-a multiscale linking model for image segmentation-for an in-depth comparison of four different scale space generators with respect to segmentation results. We consider the linear (Gaussian) scale space both in the spatial and the Fourier domain, the variable conductance diffusion according to the Perona and Malik equation, and the Euclidean shortening flow. We have done experiments on MR images of the brain, for which a gold standard is available. The hyperstack proves to be rather insensitive to the underlying scale space generator.

In recent years several nonlinear diffusion schemes have been introduced. We discuss the numerical implementation of a number of current nonlinear evolution schemes, using the notion of well-posed differentiation by Gaussian kernels. The infinitesimal change of an image when increasing scale depends on the local differential invariants evaluated at the scale of the image considered, i.e. on terms of the local jet (the set of all spatial partial derivatives at that point). All these differential terms can be obtained in a well-posed fashion by a convolution of the original image with the family of the Gaussian and its derivatives. The nonlinear partial differential evaluation can thus be numerically approximated by an iterative calculation of the appropriate terms in the local jet. Examples are given for medical images.>

Differentiation is known to be ill-posed in the sense of Hadamard. The theory of regular tempered distributions and the concept of Gaussian convolution filters open the way to a well-posed differentiation process, thereby introducing the notion of scale (or: inverse resolution). There is no a priori fundamental limit to the order of differentiation of images provided they are calculated on a sufficiently high scale (relative to pixel scale and noise correlation width), and provided we have a sufficient dynamic range of intensity values. Constraints in resolution (both in the spatial and in the intensity domain) enforce a scale-dependent restriction to the accuracy with which Gaussian kernels G/sub n/(x; /spl sigma/) can be represented in a physical sense. So at a given scale /spl sigma/ (e.g. in units of the sampling scale) and a given measure of inaccuracy /spl alpha/ there is a maximal order n above which the margin /spl alpha/ is exceeded. In this paper we quantify this relation.>