Augmented Reality (AR) is considered to be a promising technology for the guidance of laparoscopic liver surgery. By overlaying pre-operative 3D information of the liver and internal blood vessels on the laparoscopic view, surgeons can better understand the location of critical structures. In an effort to enable AR, several authors have focused on the development of methods to obtain an accurate alignment between the laparoscopic video image and the pre-operative 3D data of the liver, without assessing the benefit that the resulting overlay can provide during surgery. In this paper, we present a study that aims to assess quantitatively and qualitatively the value of an AR overlay in laparoscopic surgery during a simulated surgical task on a phantom setup. We design a study where participants are asked to physically localise pre-operative tumours in a liver phantom using three image guidance conditions - a baseline condition without any image guidance, a condition where the 3D surfaces of the liver are aligned to the video and displayed on a black background, and a condition where video see-through AR is displayed on the laparoscopic video. Using data collected from a cohort of 24 participants which include 12 surgeons, we observe that compared to the baseline, AR decreases the median localisation error of surgeons on non-peripheral targets from 25.8 mm to 9.2 mm. Using subjective feedback, we also identify that AR introduces usability improvements in the surgical task and increases the perceived confidence of the users. Between the two tested displays, the majority of participants preferred to use the AR overlay instead of navigated view of the 3D surfaces on a separate screen. We conclude that AR has the potential to improve performance and decision making in laparoscopic surgery, and that improvements in overlay alignment accuracy and depth perception should be pursued in the future.

Laparoscopic Ultrasound (LUS) is recommended as a standard-of-care when performing laparoscopic liver resections as it images sub-surface structures such as tumours and major vessels. Given that LUS probes are difficult to handle and some tumours are iso-echoic, registration of LUS images to a pre-operative CT has been proposed as an image-guidance method. This registration problem is particularly challenging due to the small field of view of LUS, and usually depends on both a manual initialisation and tracking to compose a volume, hindering clinical translation. In this paper, we extend a proposed registration approach using Content-Based Image Retrieval (CBIR), removing the requirement for tracking or manual initialisation. Pre-operatively, a set of possible LUS planes is simulated from CT and a descriptor generated for each image. Then, a Bayesian framework is employed to estimate the most likely sequence of CT simulations that matches a series of LUS images. We extend our CBIR formulation to use multiple labelled objects and constrain the registration by separating liver vessels into portal vein and hepatic vein branches. The value of this new labeled approach is demonstrated in retrospective data from 5 patients. Results show that, by including a series of 5 untracked images in time, a single LUS image can be registered with accuracies ranging from 5.7 to 16.4 mm with a success rate of 78%. Initialisation of the LUS to CT registration with the proposed framework could potentially enable the clinical translation of these image fusion techniques.

Accounting for 26% of all new cancer cases worldwide, breast cancer remains the most common form of cancer in women. Although early breast cancer has a favourable long-term prognosis, roughly a third of patients suffer from a suboptimal aesthetic outcome despite breast conserving cancer treatment. Clinical-quality 3D modelling of the breast surface therefore assumes an increasingly important role in advancing treatment planning, prediction and evaluation of breast cosmesis. Yet, existing 3D torso scanners are expensive and either infrastructure-heavy or subject to motion artefacts. In this paper we employ a single consumer-grade RGBD camera with an ICP-based registration approach to jointly align all points from a sequence of depth images non-rigidly. Subtle body deformation due to postural sway and respiration is successfully mitigated leading to a higher geometric accuracy through regularised locally affine transformations. We present results from 6 clinical cases where our method compares well with the gold standard and outperforms a previous approach. We show that our method produces better reconstructions qualitatively by visual assessment and quantitatively by consistently obtaining lower landmark error scores and yielding more accurate breast volume estimates.

Estimating the 3-D pose of instruments is an important part of robotic minimally invasive surgery for automation of basic procedures as well as providing safety features, such as virtual fixtures. Image-based methods of 3-D pose estimation provide a non-invasive low cost solution compared with methods that incorporate external tracking systems. In this paper, we extend our recent work in estimating rigid 3-D pose with silhouette and optical flow-based features to incorporate the articulated degrees-of-freedom (DOFs) of robotic instruments within a gradient-based optimization framework. Validation of the technique is provided with a calibrated ex-vivo study from the da Vinci Research Kit (DVRK) robotic system, where we perform quantitative analysis on the errors each DOF of our tracker. Additionally, we perform several detailed comparisons with recently published techniques that combine visual methods with kinematic data acquired from the joint encoders. Our experiments demonstrate that our method is competitively accurate while relying solely on image data.

Vascularisation is a key feature of cancer growth, invasion and metastasis. To better understand the governing biophysical processes and their relative importance, it is instructive to develop physiologically representative mathematical models with which to compare to experimental data. Previous studies have successfully applied this approach to test the effect of various biochemical factors on tumour growth and angiogenesis. However, these models do not account for the experimentally observed dependency of angiogenic network evolution on growth-induced solid stresses. This work introduces two novel features: the effects of hapto- and mechanotaxis on vessel sprouting, and mechano-sensitive dynamic vascular remodelling. The proposed three-dimensional, multiscale, in-silico model of dynamically coupled angiogenic tumour growth is specified to in-vivo and in-vitro data, chosen, where possible, to provide a physiologically consistent description. The model is then validated against in-vivo data from murine mammary carcinomas, with particular focus placed on identifying the influence of mechanical factors. Crucially, we find that it is necessary to include hapto- and mechanotaxis to recapitulate observed time-varying spatial distributions of angiogenic vasculature.

A fully automated, unsupervised lobe segmentation algorithm is presented based on a probabilistic segmentation of the fissures and the simultaneous construction of a population model of the fissures. A two-class probabilistic segmentation segments the lung into candidate fissure voxels and the surrounding parenchyma. This was combined with anatomical information and a groupwise fissure prior to drive non-parametric surface fitting to obtain the final segmentation. The performance of our fissure segmentation was validated on 30 patients from the chronic obstructive pulmonary disease COPDGene cohort, achieving a high median F1-score of 0.90 and showed general insensitivity to filter parameters. We evaluated our lobe segmentation algorithm on the Lobe and Lung Analysis 2011 dataset, which contains 55 cases at varying levels of pathology. We achieved the highest score of 0.884 of the automated algorithms. Our method was further tested quantitatively and qualitatively on 80 patients from the COPDgene study at varying levels of functional impairment. Accurate segmentation of the lobes is shown at various degrees of fissure incompleteness for 96% of all cases. We also show the utility of including a groupwise prior in segmenting the lobes in regions of grossly incomplete fissures.

Determining corresponding regions between an MRI and an X-ray mammogram is a clinically useful task that is challenging for radiologists due to the large deformation that the breast undergoes between the two image acquisitions. In this work we propose an intensity-based image registration framework, where the biomechanical transformation model parameters and the rigid-body transformation parameters are optimised simultaneously. Patient-specific biomechanical modelling of the breast derived from diagnostic, prone MRI has been previously used for this task. However, the high computational time associated with breast compression simulation using commercial packages, did not allow the optimisation of both pose and FEM parameters in the same framework. We use a fast explicit Finite Element (FE) solver that runs on a graphics card, enabling the FEM-based transformation model to be fully integrated into the optimisation scheme. The transformation model has seven degrees of freedom, which include parameters for both the initial rigid-body pose of the breast prior to mammographic compression, and those of the biomechanical model. The framework was tested on ten clinical cases and the results were compared against an affine transformation model, previously proposed for the same task. The mean registration error was 11.6±3.8mm for the CC and 11±5.4mm for the MLO view registrations, indicating that this could be a useful clinical tool.

Preoperative diagnostic magnetic resonance (MR) breast images can provide good contrast between different tissues and 3-D information about suspicious tissues. Aligning preoperative diagnostic MR images with a patient in the theatre during breast conserving surgery could assist surgeons in achieving the complete excision of cancer with sufficient margins. Typically, preoperative diagnostic MR breast images of a patient are obtained in the prone position, while surgery is performed in the supine position. The significant shape change of breasts between these two positions due to gravity loading, external forces and related constraints makes the alignment task extremely difficult. Our previous studies have shown that either nonrigid intensity-based image registration or biomechanical modelling alone are limited in their ability to capture such a large deformation. To tackle this problem, we proposed in this paper a nonlinear biomechanical model-based image registration method with a simultaneous optimization procedure for both the material parameters of breast tissues and the direction of the gravitational force. First, finite element (FE) based biomechanical modelling is used to estimate a physically plausible deformation of the pectoral muscle and the major deformation of breast tissues due to gravity loading. Then, nonrigid intensity-based image registration is employed to recover the remaining deformation that FE analyses do not capture due to the simplifications and approximations of biomechanical models and the uncertainties of external forces and constraints. We assess the registration performance of the proposed method using the target registration error of skin fiducial markers and the Dice similarity coefficient (DSC) of fibroglandular tissues. The registration results on prone and supine MR image pairs are compared with those from two alternative nonrigid registration methods for five breasts. Overall, the proposed algorithm achieved the best registration performance on fiducial markers (target registration error, 8.44 ±5.5 mm for 45 fiducial markers) and higher overlap rates on segmentation propagation of fibroglandular tissues (DSC value > 82%).

Fractional partial differential equations can be used to model super-diffusive processes. Recent research into fractional PDEs has explored their impact on nonrigid image registration, which typically uses differential operators of integer order. In this paper, we propose a nonrigid registration algorithm that involves directly and rapidly solving a discretized fractional PDE at each time step that is subject to physically realistic image boundary conditions. The proposed algorithm is validated through registration experiments on breast MR imagery with simulated biomechanical deformations, indicating that the super-diffusive model yields lower average deformation errors than standard diffusion-based registration.

Computed Tomographic (CT) colonography is a technique used for the detection of bowel cancer or potentially precancerous polyps. The procedure is performed routinely with the patient both prone and supine to differentiate fixed colonic pathology from mobile faecal residue. Matching corresponding locations is difficult and time consuming for radiologists due to colonic deformations that occur during patient repositioning. We propose a novel method to establish correspondence between the two acquisitions automatically. The problem is first simplified by detecting haustral folds using a graph cut method applied to a curvature-based metric applied to a surface mesh generated from segmentation of the colonic lumen. A virtual camera is used to create a set of images that provide a metric for matching pairs of folds between the prone and supine acquisitions. Image patches are generated at the fold positions using depth map renderings of the endoluminal surface and optimised by performing a virtual camera registration over a restricted set of degrees of freedom. The intensity difference between image pairs, along with additional neighbourhood information to enforce geometric constraints over a 2D parameterisation of the 3D space, are used as unary and pair-wise costs respectively, and included in a Markov Random Field (MRF) model to estimate the maximum a posteriori fold labelling assignment. The method achieved fold matching accuracy of 96.0% and 96.1% in patient cases with and without local colonic collapse. Moreover, it improved upon an existing surface-based registration algorithm by providing an initialisation. The set of landmark correspondences is used to non-rigidly transform a 2D source image derived from a conformal mapping process on the 3D endoluminal surface mesh. This achieves full surface correspondence between prone and supine views and can be further refined with an intensity based registration showing a statistically significant improvement (p<0.001), and decreasing mean error from 11.9 mm to 6.0 mm measured at 1743 reference points from 17 CTC datasets.

Accurate understanding of equipment tracking error is essential for decision making in image guided surgery. For tools tracked using markers attached to a rigid body, existing error estimation methods use the assumption that the individual marker errors are independent random variables. This assumption is not valid for all tracking systems. This paper presents a method to estimate a more accurate tracking error function, consisting of a systematic and random component. The proposed method does not require detailed knowledge of the tracking system physics. Results from a pointer calibration are used to demonstrate that the proposed method provides a better match to observed results than the existing state of the art. A simulation of the pointer calibration process is then used to show that existing methods can underestimate the pointer calibration error by a factor of two. A further simulation of laparoscopic camera tracking is used to show that existing methods cannot model important variations in system performance due to the angular arrangement of the tracking markers. By arranging the markers such that the systematic errors are nearly identical for all markers, the rotational component of the tracking error can be reduced, resulting in a significant reduction in target tracking errors.

Accurate localization of the optic radiation is key to improving the surgical outcome for patients undergoing anterior temporal lobe resection for the treatment of refractory focal epilepsy. Current commercial interventional magnetic resonance imaging (MRI) scanners are capable of performing anatomical and diffusion weighted imaging and are used for guidance during various neurosurgical procedures. We present an interventional imaging workflow that can accurately localize the optic radiation during surgery. The workflow is driven by a near real-time multichannel nonrigid image registration algorithm that uses both anatomical and fractional anisotropy pre- and intra-operative images. The proposed workflow is implemented on graphical processing units and we perform a warping of the pre-operatively parcellated optic radiation to the intra-operative space in under 3 min making the proposed algorithm suitable for use under the stringent time constraints of neurosurgical procedures. The method was validated using both a numerical phantom and clinical data using pre- and post-operative images from patients who had undergone surgery for treatment of refractory focal epilepsy and shows strong correlation between the observed post-operative visual field deficit and the predicted damage to the optic radiation. We also validate the algorithm using interventional MRI datasets from a small cohort of patients. This work could be of significant utility in image guided interventions and facilitate effective surgical treatments.

There is growing clinical demand for image registration techniques that allow multimodal data fusion for accurate targeting of needle biopsy and ablative prostate cancer treatments. However, during procedures where transrectal ultrasound (TRUS) guidance is used, substantial gland deformation can occur due to TRUS probe pressure. In this paper, the ability of a statistical shape/motion model, trained using finite element simulations, to predict and compensate for this source of motion is investigated. Three-dimensional ultrasound images acquired on five patient prostates, before and after TRUS-probe-induced deformation, were registered using a nonrigid, surface-based method, and the accuracy of different deformation models compared. Registration using a statistical motion model was found to outperform alternative elastic deformation methods in terms of accuracy and robustness, and required substantially fewer target surface points to achieve a successful registration. The mean final target registration error (based on anatomical landmarks) using this method was 1.8 mm. We conclude that a statistical model of prostate deformation provides an accurate, rapid and robust means of predicting prostate deformation from sparse surface data, and is therefore well-suited to a number of interventional applications where there is a need for deformation compensation.

Mutual information (MI) registration including spatial information has been shown to perform better than the traditional MI measures for certain nonrigid registration tasks. In this work, we first provide new insight to problems of the MI-based registration and propose to use the spatially encoded mutual information (SEMI) to tackle these problems. To encode spatial information, we propose a hierarchical weighting scheme to differentiate the contribution of sample points to a set of entropy measures, which are associated to spatial variable values. By using free-form deformations (FFDs) as the transformation model, we can first define the spatial variable using the set of FFD control points, and then propose a local ascent optimization scheme for nonrigid SEMI registration. The proposed SEMI registration can improve the registration accuracy in the nonrigid cases where the traditional MI is challenged due to intensity distortion, contrast enhancement, or different imaging modalities. It also has a similar computation complexity to the registration using traditional MI measures, improving up to two orders of magnitude of computation time compared to the traditional schemes. We validate our algorithms using phantom brain MRI, simulated dynamic contrast enhanced mangetic resonance imaging (MRI) of the liver, and in vivo cardiac MRI. The results show that the SEMI registration significantly outperforms the traditional MI registration.

Magnetic resonance (MR) imaging has become a routine modality for the determination of patient cardiac morphology. The extraction of this information can be important for the development of new clinical applications as well as the planning and guidance of cardiac interventional procedures. To avoid inter- and intra-observer variability of manual delineation, it is highly desirable to develop an automatic technique for whole heart segmentation of cardiac magnetic resonance images. However, automating this process is complicated by the limited quality of acquired images and large shape variation of the heart between subjects. In this paper, we propose a fully automatic whole heart segmentation framework based on two new image registration algorithms: the locally affine registration method (LARM) and the free-form deformations with adaptive control point status (ACPS FFDs). LARM provides the correspondence of anatomical substructures such as the four chambers and great vessels of the heart, while the registration using ACPS FFDs refines the local details using a constrained optimization scheme. We validated our proposed segmentation framework on 37 cardiac MR volumes on the end-diastolic phase, displaying a wide diversity of morphology and pathology, and achieved a mean accuracy of 2.14 +/- 0.63 mm (rms surface distance) and a maximal error of 4.31 mm.

Nonparametric image registration algorithms use deformation fields to define nonrigid transformations relating two images. Typically, these algorithms operate by successively solving linear systems of partial differential equations. These PDE systems arise by linearizing the Euler-Lagrange equations associated with the minimization of a functional defined to contain an image similarity term and a regularizer. Iterative linear system solvers can be used to solve the linear PDE systems, but they can be extremely slow. Some faster techniques based on Fourier methods, multigrid methods, and additive operator splitting, exist for solving the linear PDE systems for specific combinations of regularizers and boundary conditions. In this paper, we show that Fourier methods can be employed to quickly solve the linear PDE systems for every combination of standard regularizers (diffusion, curvature, elastic, and fluid) and boundary conditions (Dirichlet, Neumann, and periodic).

Establishing spatial correspondence between features visible in X-ray mammograms obtained at different times has great potential to aid assessment and quantitation of change in the breast indicative of malignancy. The literature contains numerous nonrigid registration algorithms developed for this purpose, but existing approaches are flawed by the assumption of inappropriate 2-D transformation models and quantitative estimation of registration accuracy is limited. In this paper, we describe a novel validation method which simulates plausible mammographic compressions of the breast using a magnetic resonance imaging (MRI) derived finite element model. By projecting the resulting known 3-D displacements into 2-D and generating pseudo-mammograms from these same compressed magnetic resonance (MR) volumes, we can generate convincing images with known 2-D displacements with which to validate a registration algorithm. We illustrate this approach by computing the accuracy for two conventional nonrigid 2-D registration algorithms applied to mammographic test images generated from three patient MR datasets. We show that the accuracy of these algorithms is close to the best achievable using a 2-D one-to-one correspondence model but that new algorithms incorporating more representative transformation models are required to achieve sufficiently accurate registrations for this application.

Intraoperative freehand three-dimensional (3-D) ultrasound (3D-US) has been proposed as a noninvasive method for registering bones to a preoperative computed tomography image or computer-generated bone model during computer-aided orthopedic surgery (CAOS). In this technique, an US probe is tracked by a 3-D position sensor and acts as a percutaneous device for localizing the bone surface. However, variations in the acoustic properties of soft tissue, such as the average speed of sound, can introduce significant errors in the bone depth estimated from US images, which limits registration accuracy. We describe a new self-calibrating approach to US-based bone registration that addresses this problem, and demonstrate its application within a standard registration scheme. Using realistic US image data acquired from 6 femurs and 3 pelves of intact human cadavers, and accurate Gold Standard registration transformations calculated using bone-implanted fiducial markers, we show that self-calibrating registration is significantly more accurate than a standard method, yielding an average root mean squared target registration error of 1.6 mm. We conclude that self-calibrating registration results in significant improvements in registration accuracy for CAOS applications over conventional approaches where calibration parameters of the 3D-US system remain fixed to values determined using a preoperative phantom-based calibration.

We present a method for alignment of an interventional plan to optically tracked two-dimensional intraoperative ultrasound (US) images of the liver. Our clinical motivation is to enable the accurate transfer of information from three-dimensional preoperative imaging modalities [magnetic resonance (MR) or computed tomography (CT)] to intraoperative US to aid needle placement for thermal ablation of liver metastases. An initial rigid registration to intraoperative coordinates is obtained using a set of US images acquired at maximum exhalation. A preprocessing step is applied to both the preoperative images and the US images to produce evidence of corresponding structures. This yields two sets of images representing classification of regions as vessels. The registration then proceeds using these images. The preoperative images and plan are then warped to correspond to a single US slice acquired at an unknown point in the breathing cycle where the liver is likely to have moved and deformed relative to the preoperative image. Alignment is constrained using a patient-specific model of breathing motion and deformation. Target registration error is estimated by carrying out simulation experiments using resliced MR volumes to simulate real US and comparing the registration results to a "bronze-standard" registration performed on the full MR volume. Finally, the system is tested using real US and verified using visual inspection.

Two-dimensional (2-D)-to-three-dimensional (3-D) registration can improve visualization which may aid minimally invasive neurointerventions. Using clinical and phantom studies, two state-of-the-art approaches to rigid registration are compared quantitatively: an intensity-based algorithm using the gradient difference similarity measure; and an iterative closest point (ICP)-based algorithm. The gradient difference approach was found to be more accurate, with an average registration accuracy of 1.7 mm for clinical data, compared to the ICP-based algorithm with an average accuracy of 2.8 mm. In phantom studies, the ICP-based algorithm proved more reliable, but with more complicated clinical data, the gradient difference algorithm was more robust. Average computation time for the ICP-based algorithm was 20 s per registration, compared with 14 min and 50 s for the gradient difference algorithm.

We have developed two new algorithms for the measurement of blood flow from dynamic X-ray angiographic images. Both algorithms aim to improve on existing techniques. First, a model-based (MB) algorithm is used to constrain the concentration-distance curve matching approach. Second, a weighted optical flow algorithm (OP) is used to improve on point-based optical flow methods by averaging velocity estimates along a vessel with weighting based on the magnitude of the spatial derivative. The OP algorithm was validated using a computer simulation of pulsatile blood flow. Both the OP and the MB algorithms were validated using a physiological blood flow circuit. Dynamic biplane digital X-ray images were acquired following injection of iodine contrast medium into a variety of simulated arterial vessels. The image data were analyzed using our integrated angiographic analysis software SARA to give blood flow waveforms using the MB and OP algorithms. These waveforms were compared to flow measured using an electromagnetic flow meter (EMF). In total 4935 instantaneous measurements of flow were made and compared to the EMF recordings. It was found that the new algorithms showed low measurement bias and narrow limits of agreement and also out-performed the concentration-distance curve matching algorithm (ORG) and a modification of this algorithm (PA) in all studies.

With the introduction of cone beam (CB) scanners, cardiac volumetric computed tomography (CT) imaging has the potential to become a noninvasive imaging tool in clinical routine for the diagnosis of various heart diseases. Heart rate adaptive reconstruction schemes enable the reconstruction of high-resolution volumetric data sets of the heart. Artifacts, caused by strong heart rate variations, high heart rates and obesity, decrease the image quality and the diagnostic value of the images. The image quality suffers from streak artifacts if suboptimal scan and reconstruction parameters are chosen, demanding improved gating techniques. In this paper, an artifact analysis is carried out which addresses the artifacts due to the gating when using a three-dimensional CB cardiac reconstruction technique. An automatic and patient specific cardiac weighting technique is presented in order to improve the image quality. Based on the properties of the reconstruction algorithm, several assessment techniques are introduced which enable the quantitative determination of the cycle-to-cycle transition smoothness and phase homogeneity of the image reconstruction. Projection data of four patients were acquired using a 16-slice CBCT system in low pitch helical mode with parallel electrocardiogram recording. For each patient, image results are presented and discussed in combination with the assessment criteria.

Recent studies have shown that the surface of the brain is deformed by up to 20 mm after the skull is opened during neurosurgery, which could lead to substantial error in commercial image-guided surgery systems. We quantitatively analyze the intraoperative brain deformation of 24 subjects to investigate whether simple rules can describe or predict the deformation. Interventional magnetic resonance images acquired at the start and end of the procedure are registered nonrigidly to obtain deformation values throughout the brain. Deformation patterns are investigated quantitatively with respect to the location and magnitude of deformation, and to the distribution and principal direction of the displacements. We also measure the volume change of the lateral ventricles by manual segmentation. Our study indicates that brain shift occurs predominantly in the hemisphere ipsi-lateral to the craniotomy, and that there is more brain deformation during resection procedures than during biopsy or functional procedures. However, the brain deformation patterns are extremely complex in this group of subjects. This paper quantitatively demonstrates that brain deformation occurs not only at the surface, but also in deeper brain structure, and that the principal direction of displacement does not always correspond with the direction of gravity. Therefore, simple computational algorithms that utilize limited intraoperative information (e.g., brain surface shift) will not always accurately predict brain deformation at the lesion.

TEST 02 - Elsevier's Scopus, the largest abstract and citation database of peer-reviewed literature. Search and access research from the science, technology, medicine, social sciences and arts and humanities fields.

We propose a new method for aligning three-dimensional (3-D) magnetic resonance angiography (MRA) with 2-D X-ray digital subtraction angiograms (DSA). Our method is developed from our algorithm to register computed tomography volumes to X-ray images based on intensity matching of digitally reconstructed radiographs (DRRs). To make the DSA and DRR more similar, we transform the MRA images to images of the vasculature and set to zero the contralateral side of the MRA to that imaged with DSA. We initialize the search for a match on a user defined circular region of interest. We have tested six similarity measures using both unsegmented MRA and three segmentation variants of the MRA. Registrations were carried out on images of a physical neuro-vascular phantom and images obtained during four neuro-vascular interventions. The most accurate and robust registrations were obtained using the pattern intensity, gradient difference, and gradient correlation similarity measures, when used in conjunction with the most sophisticated MRA segmentations. Using these measures, 95% of the phantom start positions and 82% of the clinical start positions were successfully registered. The lowest root mean square reprojection errors were 1.3 mm (standard deviation 0.6) for the phantom and 1.5 mm (standard deviation 0.9) for the clinical data sets. Finally, we present a novel method for the comparison of similarity measure performance using a technique borrowed from receiver operator characteristic analysis.

This paper presents a novel method for validation of nonrigid medical image registration. This method is based on the simulation of physically plausible, biomechanical tissue deformations using finite-element methods. Applying a range of displacements to finite-element models of different patient anatomies generates model solutions which simulate gold standard deformations. From these solutions, deformed images are generated with a range of deformations typical of those likely to occur in vivo. The registration accuracy with respect to the finite-element simulations is quantified by co-registering the deformed images with the original images and comparing the recovered voxel displacements with the biomechanically simulated ones. The functionality of the validation method is demonstrated for a previously described nonrigid image registration technique based on free-form deformations using B-splines and normalized mutual information as a voxel similarity measure, with an application to contrast-enhanced magnetic resonance mammography image pairs. The exemplar nonrigid registration technique is shown to be of subvoxel accuracy on average for this particular application. The validation method presented here is an important step toward more generic simulations of biomechanically plausible tissue deformations and quantification of tissue motion recovery using nonrigid image registration. It will provide a basis for improving and comparing different nonrigid registration techniques for a diversity of medical applications, such as intrasubject tissue deformation or motion correction in the brain, liver or heart.

Previous work has suggested the existence of differences between the cerebral cortex of normal individuals, and those of patients with diseases such as epilepsy and schizophrenia. These shape abnormalities may be of developmental origin. Improved shape measures could provide useful tools for neuroscience research and patient diagnosis. We consider the theoretically desirable properties of measures of brain shape. We have implemented seven measures, three from the neuroscience literature, and four new to this field. Three of the measures are zero-order and four are second-order with respect to the surface. We validate the measures using simple geometrical shapes, and a collection of magnetic resonance scans of ten histologically normal ex vivo fetal brains with gestational ages from 19-42 weeks. We then apply the measures to MR scans from two histologically abnormal ex vivo brains. We demonstrate that our implementation of the measures is sensitive to anatomical variability rather than to the discreteness of the image data. All the measures were sensitive to changes in shape during fetal development. Several of the measures could distinguish between the normal and abnormal fetal brains. We propose a multivariate approach to studying the shape of the cerebral cortex, in which both zero-order and second-order measures are used to quantify folding.

International audience

The authors propose a novel method to register two or more optical images to a 3D surface model. The potential applications of such a registration method could be in medicine for example, in image guided interventions, surveillance and identification, industrial inspection, or telemanipulation in remote or hostile environments. Registration is performed by optimizing a similarity measure with respect to the transformation parameters. We propose a novel similarity measure based on "photo-consistency." For each surface point, the similarity measure computes how consistent the corresponding optical image information in each view is with a lighting model. The relative pose of the optical images must be known. We validate the system using data from an optical-based surface reconstruction system and surfaces derived from magnetic resonance (MR) images of the human face. We test the accuracy and robustness of the system with respect to the number of video images, video image noise, errors in surface location and area, and complexity of the matched surfaces. We demonstrate the algorithm working on 10 further optical-based reconstructions of the human head and skin surfaces derived from MR images of the heads of five volunteers. Matching four optical images to a surface model produced a 3D error of between 1.45 and 1.59 mm, at a success rate of 100 percent, where the initial misregistration was up to 16 mm or degrees from the registration position.

The problem of providing surgical navigation using image overlays on the operative scene can be split into four main tasks--calibration of the optical system; registration of preoperative images to the patient; system and patient tracking, and display using a suitable visualization scheme. To achieve a convincing result in the magnified microscope view a very high alignment accuracy is required. We have simulated an entire image overlay system to establish the most significant sources of error and improved each of the stages involved. The microscope calibration process has been automated. We have introduced bone-implanted markers for registration and incorporated a locking acrylic dental stent (LADS) for patient tracking. The LADS can also provide a less-invasive registration device with mean target error of 0.7 mm in volunteer experiments. These improvements have significantly increased the alignment accuracy of our overlays. Phantom accuracy is 0.3-0.5 mm and clinical overlay errors were 0.5-1.0 mm on the bone fiducials and 0.5-4 mm on target structures. We have improved the graphical representation of the stereo overlays. The resulting system provides three-dimensional surgical navigation for microscope-assisted guided interventions (MAGI).

We have evaluated eight different similarity measures used for rigid body registration of serial magnetic resonance (MR) brain scans. To assess their accuracy we used 33 clinical three-dimensional (3-D) serial MR images, with deformable extradural tissue excluded by manual segmentation and simulated 3-D MR images with added intensity distortion. For each measure we determined the consistency of registration transformations for both sets of segmented and unsegmented data. We have shown that of the eight measures tested, the ones based on joint entropy produced the best consistency. In particular, these measures seemed to be least sensitive to the presence of extradural tissue. For these data the difference in accuracy of these joint entropy measures, with or without brain segmentation, was within the threshold of visually detectable change in the difference images.

In this paper we present a new approach for the nonrigid registration of contrast-enhanced breast MRI. A hierarchical transformation model of the motion of the breast has been developed. The global motion of the breast is modeled by an affine transformation while the local breast motion is described by a free-form deformation (FFD) based on B-splines. Normalized mutual information is used as a voxel-based similarity measure which is insensitive to intensity changes as a result of the contrast enhancement. Registration is achieved by minimizing a cost function, which represents a combination of the cost associated with the smoothness of the transformation and the cost associated with the image similarity. The algorithm has been applied to the fully automated registration of three-dimensional (3-D) breast MRI in volunteers and patients. In particular, we have compared the results of the proposed nonrigid registration algorithm to those obtained using rigid and affine registration techniques. The results clearly indicate that the nonrigid registration algorithm is much better able to recover the motion and deformation of the breast than rigid or affine registration algorithms.

In recent years, advances in computer technology and a significant increase in the accuracy of medical imaging have made it possible to develop systems that can assist the clinician in diagnosis, planning, and treatment. This paper deals with an area that is generally referred to as computer-assisted surgery, image-directed surgery, or image-guided surgery. We report the research, development, and clinical validation performed since January 1996 in the European Applications in Surgical Interventions (EASI) project, which is funded by the European Commission in their "4th Framework Telematics Applications for Health" program. The goal of this project is the improvement of the effectiveness and quality of image-guided neurosurgery of the brain and image-guided vascular surgery of abdominal aortic aneurysms, while at the same time reducing patient risks and overall cost. We have developed advanced prototype systems for preoperative surgical planning and intraoperative surgical navigation, and we have extensively clinically validated these systems. The prototype systems and the clinical validation results are described in this paper.

All image-guided neurosurgical systems that we are aware of assume that the head and its contents behave as a rigid body. It is important to measure intraoperative brain deformation (brain shift) to provide some indication of the application accuracy of image-guided surgical systems, and also to provide data to develop and validate nonrigid registration algorithms to correct for such deformation. We are collecting data from patients undergoing neurosurgery in a high-field (1.5 T) interventional magnetic resonance (MR) scanner. High-contrast and high-resolution gradient-echo MR image volumes are collected immediately prior to surgery, during surgery, and at the end of surgery, with the patient intubated and lying on the operating table in the operative position. In this paper we report initial results from six patients: one freehand biopsy, one stereotactic functional procedure, and four resections. We investigate intraoperative brain deformation by examining threshold boundary overlays and difference images and by measuring ventricular volume. We also present preliminary results obtained using a nonrigid registration algorithm to quantify deformation. We found that some cases had much greater deformation than others, and also that, regardless of the procedure, there was very little deformation of the midline, the tentorium, the hemisphere contralateral to the procedure, and ipsilateral structures except those that are within 1 cm of the lesion or are gravitationally above the surgical site.

A comparison of six similarity measures for use in intensity-based two-dimensional-three-dimensional (2-D-3-D) image registration is presented. The accuracy of the similarity measures are compared to a "gold-standard" registration which has been accurately calculated using fiducial markers. The similarity measures are used to register a computed tomography (CT) scan of a spine phantom to a fluoroscopy image of the phantom. The registration is carried out within a region-of-interest in the fluoroscopy image which is user defined to contain a single vertebra. Many of the problems involved in this type of registration are caused by features which were not modeled by a phantom image alone. More realistic "gold-standard" data sets were simulated using the phantom image with clinical image features overlaid. Results show that the introduction of soft-tissue structures and interventional instruments into the phantom image can have a large effect on the performance of some similarity measures previously applied to 2-D-3-D image registration. Two measures were able to register accurately and robustly even when soft-tissue structures and interventional instruments were present as differences between the images. These measures were pattern intensity and gradient difference. Their registration accuracy, for all the rigid-body parameters except for the source to film translation, was within a root-mean-square (rms) error of 0.54 mm or degrees to the "gold-standard" values. No failures occurred while registering using these measures.

Registration of intraoperative fluoroscopy images with preoperative three-dimensional (3-D) CT images can be used for several purposes in image-guided surgery. On the one hand, it can be used to display the position of surgical instruments, which are being tracked by a localizer, in the preoperative CT scan. On the other hand, the registration result can be used to project preoperative planning information or important anatomical structures visible in the CT image onto the fluoroscopy image. For this registration task, a novel voxel-based method in combination with a new similarity measure (pattern intensity) has been developed. The basic concept of the method is explained at the example of two-dimensional (2-D)/3-D registration of a vertebra in an X-ray fluoroscopy image with a 3-D CT image. The registration method is described, and the results for a spine phantom are presented and discussed. Registration has been carried out repeatedly with different starting estimates to study the capture range. Information about registration accuracy has been obtained by comparing the registration results with a highly accurate "ground-truth" registration, which has been derived from fiducial markers attached to the phantom prior to imaging. In addition, registration results for different vertebrae have been compared. The results show that the rotation parameters and the shifts parallel to the projection plane can accurately be determined from a single projection. Because of the projection geometry, the accuracy of the height above the projection plane is significantly lower.

Multiple sources of 3D medical image data can be used to construct detailed patient representations. Typically registration is achieved assuming the validity of rigid body transformation. In many applications, and in particular when updating representations used for guidance during surgery and therapeutic interventions, this assumption is inappropriate. In this paper we describe a general method for 3D deformation, show how registration can incorporate a composite of rigid body and deformation components and illustrate this methodology on 3 example sets of images. The first is a repeated 3D MR scan of the abdomen of a volunteer who purposely changed position between scans; the second is an MR and CT scan of the head and neck, in which the patient was in a different position for the two scans; and the third is a set of MR and CT images of the head taken before and after epilepsy surgery. Non rigid deformation and composite warping showed significant improvement in registration accuracy in each case. 1

This paper presents a fully automated technique for the rigid body registration of clinically acquired MR and CT images of the head. We describe our multiresolution approach to the optimisation of voxel similarity measures and evaluate the performance of a number of measures when presented with clinical images with a small overlapping volume. Results are compared with those derived from manual registration by identification of corresponding point landmarks. We show that by limiting the measures to intra-cranial regions of the images, not containing deformable skin surface features, a greater accuracy may be provided for certain types of truncated image.

In this paper we present our work on using intensity feature spaces to study the relationship between voxel values in registered and unreg-istered medical images. By taking a simple image model we predict structures that we might expect to find in an intensity feature space produced from different modality images of the same scene. We show how this structure will be modified by image noise, misregistration and differing point spread functions of the two modalities. We show examples of such structure in feature spaces created from clinically ac-quired Magnetic Resonance (MR) and Positron Emission Tomography (PET) image data. We show how two simple measures of voxel sim-ilarity based on these feature space observations, a modified variance of intensity ratio and the 3rd order moment of the feature space his-togram, can be used to quantify image misregistration. The 3rd order moment measure is then used with a genetic optimisation algorithm to automatically register pre and post Gadolinium injection MR images. 1

Abstract The combination of multimodality medical images is generating increasing clinical interest, but the absence of robust automatic registration algorithms is an obstacle to these techniques entering routine clinical use. In this paper, we present a new algorithm based on surface fitting that makes use of anatomical knowledge of adjacency of identified anatomical structures to solve the 3D rigid body registration problem. The algorithm has been applied to the registration of MR CT and vascular images of the head, and has achieved accuracies similar to those obtained using interactive location of corresponding landmarks.