INTRODUCTION: This review explores the application of Artificial Intelligence (AI) in managing primary liver cancer, focusing on recent advancements. AI, particularly machine learning (ML) and deep learning (DL), shows potential in improving screening, diagnosis, treatment planning, efficacy assessment, prognosis prediction, and follow-up-crucial elements given the high mortality of liver cancer. METHODS: A systematic search was conducted in the PubMed, Scopus, Embase, and Web of Science databases, focusing on original research published until June 2024 on AI's clinical applications in liver cancer. Studies not relevant or lacking clinical evaluation were excluded. RESULTS: Out of 13,122 screened articles, 62 were selected for full review. The studies highlight significant improvements in detecting hepatocellular carcinoma and intrahepatic cholangiocarcinoma through AI. DL models show high sensitivity and specificity, particularly in early detection. In diagnosis, AI models using CT and MRI data improve precision in distinguishing benign from malignant lesions through multimodal data integration. DISCUSSION: Recent AI models outperform earlier non-neural network versions, though a gap remains between development and clinical implementation. Many models lack thorough clinical applicability assessments and external validation. CONCLUSION: AI integration in primary liver cancer management is promising but requires rigorous development and validation practices to enhance clinical outcomes fully.

Symmetry detection and morphological classification of anatomical structures play pivotal roles in medical image analysis. The application of kinematic surface fitting, a method for characterizing shapes through parametric stationary velocity fields, has shown promising results in computer vision and computer-aided design. However, existing research has predominantly focused on first order rotational velocity fields, which may not adequately capture the intricate curved and twisted nature of anatomical structures. To address this limitation, we propose an innovative approach utilizing a second order velocity field for kinematic surface fitting. This advancement accommodates higher rotational shape complexity and improves the accuracy of symmetry detection in anatomical structures. We introduce a robust fitting technique and validate its performance through testing on synthetic shapes and real anatomical structures. Our method not only enables the detection of curved rotational symmetries (core lines) but also facilitates morphological classification by deriving intrinsic shape parameters related to curvature and torsion. We illustrate the usefulness of our technique by categorizing the shape of human cochleae in terms of the intrinsic velocity field parameters. The results showcase the potential of our method as a valuable tool for medical image analysis, contributing to the assessment of complex anatomical shapes.

The variational autoencoder-based method has been widely used for modeling massive datasets. However, for 3D images, simultaneously achieving disentangled representations, low-variance Evidence Lower Bounds (ELBO), and a lightweight model remains a challenging task. In this work, we propose a Langevin dynamics-based inference framework that integrates target data information for efficient likelihood inference and disentangles appearance and morphology features via multi-scale energy-level encoding that enables unsupervised disentanglement. We adopt a quasi-symplectic integrator to handle the Hessian-related computational bottleneck that often arises in Langevin-based flow inference. We demonstrate both theoretical and empirical effectiveness of our approach compared to other methods. Experiments on public benchmarks and clinical 3D imaging datasets show that our Langevin-VAE achieves high-quality generation and learns disentangled shape and appearance representations with a model size of only 1.7M parameters. The code will be available at: https://github.com/LaplaceCenter/LangevinVAE.

Cross-modality data translation has attracted great interest in medical image computing. Deep generative models show performance improvement in addressing related challenges. Nevertheless, as a fundamental challenge in image translation, the problem of zero-shot learning cross-modality image translation with fidelity remains unanswered. To bridge this gap, we propose a novel unsupervised zero-shot learning method called Mutual Information guided Diffusion Model, which learns to translate an unseen source image to the target modality by leveraging the inherent statistical consistency of Mutual Information between different modalities. To overcome the prohibitive high dimensional Mutual Information calculation, we propose a differentiable local-wise mutual information layer for conditioning the iterative denoising process. The Local-wise-Mutual-Information-Layer captures identical cross-modality features in the statistical domain, offering diffusion guidance without relying on direct mappings between the source and target domains. This advantage allows our method to adapt to changing source domains without the need for retraining, making it highly practical when sufficient labeled source domain data is not available. We demonstrate the superior performance of MIDiffusion in zero-shot cross-modality translation tasks through empirical comparisons with other generative models, including adversarial-based and diffusion-based models. Finally, we showcase the real-world application of MIDiffusion in 3D zero-shot learning-based cross-modality image segmentation tasks.

Recently, super-resolution ultrasound imaging with ultrasound localization microscopy (ULM) has received much attention. However, ULM relies on low concentrations of microbubbles in the blood vessels, ultimately resulting in long acquisition times. Here, we present an alternative super-resolution approach, based on direct deconvolution of single-channel ultrasound radio-frequency (RF) signals with a one-dimensional dilated convolutional neural network (CNN). This work focuses on low-frequency ultrasound (1.7 MHz) for deep imaging (10 cm) of a dense cloud of monodisperse microbubbles (up to 1000 microbubbles in the measurement volume, corresponding to an average echo overlap of 94%). Data are generated with a simulator that uses a large range of acoustic pressures (5-250 kPa) and captures the full, nonlinear response of resonant, lipid-coated microbubbles. The network is trained with a novel dual-loss function, which features elements of both a classification loss and a regression loss and improves the detection-localization characteristics of the output. Whereas imposing a localization tolerance of 0 yields poor detection metrics, imposing a localization tolerance corresponding to 4% of the wavelength yields a precision and recall of both 0.90. Furthermore, the detection improves with increasing acoustic pressure and deteriorates with increasing microbubble density. The potential of the presented approach to super-resolution ultrasound imaging is demonstrated with a delay-and-sum reconstruction with deconvolved element data. The resulting image shows an order-of-magnitude gain in axial resolution compared to a delay-and-sum reconstruction with unprocessed element data.

This work addresses the problem of non-rigid registration of 3D scans, which is at the core of shape modeling techniques. Firstly, we propose a new kernel based on geodesic distances for the Gaussian Process Morphable Models (GPMMs) framework. The use of geodesic distances into the kernel makes it more adapted to the topological and geometric characteristics of the surface and leads to more realistic deformations around holes and curved areas. Since the kernel possesses hyperparameters we have optimized them for the task of face registration on the FaceWarehouse dataset. We show that the Geodesic squared exponential kernel performs significantly better than state of the art kernels for the task of face registration on all the 20 expressions of the FaceWarehouse dataset. Secondly, we propose a modification of the loss function used in the non-rigid ICP registration algorithm, that allows to weight the correspondences according to the confidence given to them. As a use case, we show that we can make the registration more robust to outliers in the 3D scans, such as non-skin parts.

We propose to learn a probabilistic motion model from a sequence of images for spatio-temporal registration. Our model encodes motion in a low-dimensional probabilistic space - the motion matrix - which enables various motion analysis tasks such as simulation and interpolation of realistic motion patterns allowing for faster data acquisition and data augmentation. More precisely, the motion matrix allows to transport the recovered motion from one subject to another simulating for example a pathological motion in a healthy subject without the need for inter-subject registration. The method is based on a conditional latent variable model that is trained using amortized variational inference. This unsupervised generative model follows a novel multivariate Gaussian process prior and is applied within a temporal convolutional network which leads to a diffeomorphic motion model. Temporal consistency and generalizability is further improved by applying a temporal dropout training scheme. Applied to cardiac cine-MRI sequences, we show improved registration accuracy and spatio-temporally smoother deformations compared to three state-of-the-art registration algorithms. Besides, we demonstrate the model's applicability for motion analysis, simulation and super-resolution by an improved motion reconstruction from sequences with missing frames compared to linear and cubic interpolation.

We propose to learn a low-dimensional probabilistic deformation model from data which can be used for the registration and the analysis of deformations. The latent variable model maps similar deformations close to each other in an encoding space. It enables to compare deformations, to generate normal or pathological deformations for any new image, or to transport deformations from one image pair to any other image. Our unsupervised method is based on the variational inference. In particular, we use a conditional variational autoencoder network and constrain transformations to be symmetric and diffeomorphic by applying a differentiable exponentiation layer with a symmetric loss function. We also present a formulation that includes spatial regularization such as the diffusion-based filters. In addition, our framework provides multi-scale velocity field estimations. We evaluated our method on 3-D intra-subject registration using 334 cardiac cine-MRIs. On this dataset, our method showed the state-of-the-art performance with a mean DICE score of 81.2% and a mean Hausdorff distance of 7.3 mm using 32 latent dimensions compared to three state-of-the-art methods while also demonstrating more regular deformation fields. The average time per registration was 0.32 s. Besides, we visualized the learned latent space and showed that the encoded deformations can be used to transport deformations and to cluster diseases with a classification accuracy of 83% after applying a linear projection.

Collecting large databases of annotated medical images is crucial for the validation and testing of feature extraction, statistical analysis, and machine learning algorithms. Recent advances in cardiac electromechanical modeling and image synthesis provided a framework to generate synthetic images based on realistic mesh simulations. Nonetheless, their potential to augment an existing database with large amounts of synthetic cases requires further investigation. We build upon these works and propose a revised scheme for synthesizing pathological cardiac sequences from real healthy sequences. Our new pipeline notably involves a much easier registration problem to reduce potential artifacts, and takes advantage of mesh correspondences to generate new data from a given case without additional registration. The output sequences are thoroughly examined in terms of quality and usability on a given application: the assessment of myocardial viability, via the generation of 465 synthetic cine MR sequences (15 healthy and 450 with pathological tissue viability [random location, extent, and grade, up to myocardial infarct]). We demonstrate that: 1) our methodology improves the state-of-the-art algorithms in terms of realism and accuracy of the simulated images and 2) our methodology is well-suited for the generation of large databases at small computational cost.

We propose a method based on deep learning to perform cardiac segmentation on short axis Magnetic resonance imaging stacks iteratively from the top slice (around the base) to the bottom slice (around the apex). At each iteration, a novel variant of the U-net is applied to propagate the segmentation of a slice to the adjacent slice below it. In other words, the prediction of a segmentation of a slice is dependent upon the already existing segmentation of an adjacent slice. The 3-D consistency is hence explicitly enforced. The method is trained on a large database of 3078 cases from the U.K. Biobank. It is then tested on the 756 different cases from the U.K. Biobank and three other state-of-the-art cohorts (ACDC with 100 cases, Sunnybrook with 30 cases, and RVSC with 16 cases). Results comparable or even better than the state of the art in terms of distance measures are achieved. They also emphasize the assets of our method, namely, enhanced spatial consistency (currently neither considered nor achieved by the state of the art), and the generalization ability to unseen cases even from other databases.

Meshes with curvilinear elements hold the appealing promise of enhanced geometric flexibility and higher-order numerical accuracy compared to their commonly-used straight-edge counterparts. However, the generation of curved meshes remains a computationally expensive endeavor with current meshing approaches: high-order parametric elements are notoriously difficult to conform to a given boundary geometry, and enforcing a smooth and non-degenerate Jacobian everywhere brings additional numerical difficulties to the meshing of complex domains. In this paper, we propose an extension of Optimal Delaunay Triangulations (ODT) to curved and graded isotropic meshes. By exploiting a continuum mechanics interpretation of ODT instead of the usual approximation theoretical foundations, we formulate a very robust geometry and topology optimization of Bézier meshes based on a new simple functional promoting isotropic and uniform Jacobians throughout the domain. We demonstrate that our resulting curved meshes can adapt to complex domains with high precision even for a small count of elements thanks to the added flexibility afforded by more control points and higher order basis functions.

We investigate uncertainty quantification under a sparse Bayesian model of medical image registration. Bayesian modelling has proven powerful to automate the tuning of registration hyperparameters, such as the trade-off between the data and regularization functionals. Sparsity-inducing priors have recently been used to render the parametrization itself adaptive and data-driven. The sparse prior on transformation parameters effectively favors the use of coarse basis functions to capture the global trends in the visible motion while finer, highly localized bases are introduced only in the presence of coherent image information and motion. In earlier work, approximate inference under the sparse Bayesian model was tackled in an efficient Variational Bayes (VB) framework. In this paper we are interested in the theoretical and empirical quality of uncertainty estimates derived under this approximate scheme vs. under the exact model. We implement an (asymptotically) exact inference scheme based on reversible jump Markov Chain Monte Carlo (MCMC) sampling to characterize the posterior distribution of the transformation and compare the predictions of the VB and MCMC based methods. The true posterior distribution under the sparse Bayesian model is found to be meaningful: orders of magnitude for the estimated uncertainty are quantitatively reasonable, the uncertainty is higher in textureless regions and lower in the direction of strong intensity gradients.

In this article, we propose a proof of concept for the automatic planning of personalized radiotherapy for brain tumors. A computational model of glioblastoma growth is combined with an exponential cell survival model to describe the effect of radiotherapy. The model is personalized to the magnetic resonance images (MRIs) of a given patient. It takes into account the uncertainty in the model parameters, together with the uncertainty in the MRI segmentations. The computed probability distribution over tumor cell densities, together with the cell survival model, is used to define the prescription dose distribution, which is the basis for subsequent Intensity Modulated Radiation Therapy (IMRT) planning. Depending on the clinical data available, we compare three different scenarios to personalize the model. First, we consider a single MRI acquisition before therapy, as it would usually be the case in clinical routine. Second, we use two MRI acquisitions at two distinct time points in order to personalize the model and plan radiotherapy. Third, we include the uncertainty in the segmentation process. We present the application of our approach on two patients diagnosed with high grade glioma. We introduce two methods to derive the radiotherapy prescription dose distribution, which are based on minimizing integral tumor cell survival using the maximum a posteriori or the expected tumor cell density. We show how our method allows the user to compute a patient specific radiotherapy planning conformal to the tumor infiltration. We further present extensions of the method in order to spare adjacent organs at risk by re-distributing the dose. The presented approach and its proof of concept may help in the future to better target the tumor and spare organs at risk.

Computer models of the heart are of increasing interest for clinical applications due to their discriminative and predictive abilities. However a single 3D simulation can be computationally expensive and long, which can make some practical applications such as the personalisation phase, or a sensitivity analysis of mechanical parameters over the simulated behaviour quite slow. In this manuscript we present a multiscale 0D/3D model which allows us to have a reliable (and extremely fast) approximation of the behaviour of the 3D model under a few simplifying assumptions. We first detail the two different models, then explain the coupling of the two models to get fast 0D approximation of 3D simulations. Finally we demonstrated how the multiscale model can speed-up an efficient optimization algorithm, which enables a fast personalisation of the 3D simulations by leveraging on the advantages of each scale. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A plethora of techniques for cardiac deformation imaging with 3D ultrasound, typically referred to as 3D speckle tracking techniques, are available from academia and industry. Although the benefits of single methods over alternative ones have been reported in separate publications, the intrinsic differences in the data and definitions used makes it hard to compare the relative performance of different solutions. To address this issue, we have recently proposed a framework to simulate realistic 3D echocardiographic recordings and used it to generate a common set of ground-truth data for 3D speckle tracking algorithms, which was made available online. The aim of this study was therefore to use the newly developed database to contrast non-commercial speckle tracking solutions from research groups with leading expertise in the field. The five techniques involved cover the most representative families of existing approaches, namely block-matching, radio-frequency tracking, optical flow and elastic image registration. The techniques were contrasted in terms of tracking and strain accuracy. The feasibility of the obtained strain measurements to diagnose pathology was also tested for ischemia and dyssynchrony.

In this paper, we describe a novel and generic approach to address fully-automatic segmentation of brain tumors by using multi-atlas patch-based voting techniques. In addition to avoiding the local search window assumption, the conventional patch-based framework is enhanced through several simple procedures: an improvement of the training dataset in terms of both label purity and intensity statistics, augmented features to implicitly guide the nearest-neighbor-search, multi-scale patches, invariance to cube isometries, stratification of the votes with respect to cases and labels. A probabilistic model automatically delineates regions of interest enclosing high-probability tumor volumes, which allows the algorithm to achieve highly competitive running time despite minimal processing power and resources. This method was evaluated on Multimodal Brain Tumor Image Segmentation challenge datasets. State-of-the-art results are achieved, with a limited learning stage thus restricting the risk of overfit. Moreover, segmentation smoothness does not involve any post-processing.

This paper describes a novel generative model for the synthesis of multi-modal medical images of pathological cases based on a single label map. Our model builds upon i) a generative model commonly used for label fusion and multi-atlas patch-based segmentation of healthy anatomical structures, ii) the Modality Propagation iterative strategy used for a spatially-coherent synthesis of subject-specific scans of desired image modalities. The expression Extended Modality Propagation is coined to refer to the extension of Modality Propagation to the synthesis of images of pathological cases. Moreover, image synthesis uncertainty is estimated. An application to Magnetic Resonance Imaging synthesis of glioma-bearing brains is i) validated on the training dataset of a Multimodal Brain Tumor Image Segmentation challenge, ii) compared to the state-of-the-art in glioma image synthesis, and iii) illustrated using the output of two different tumor growth models. Such a generative model allows the generation of a large dataset of synthetic cases, which could prove useful for the training, validation, or benchmarking of image processing algorithms.

The mathematical modeling of brain tumor growth has been the topic of numerous research studies. Most of this work focuses on the reaction-diffusion model, which suggests that the diffusion coefficient and the proliferation rate can be related to clinically relevant information. However, estimating the parameters of the reaction-diffusion model is difficult because of the lack of identifiability of the parameters, the uncertainty in the tumor segmentations, and the model approximation, which cannot perfectly capture the complex dynamics of the tumor evolution. Our approach aims at analyzing the uncertainty in the patient specific parameters of a tumor growth model, by sampling from the posterior probability of the parameters knowing the magnetic resonance images of a given patient. The estimation of the posterior probability is based on: 1) a highly parallelized implementation of the reaction-diffusion equation using the Lattice Boltzmann Method (LBM), and 2) a high acceptance rate Monte Carlo technique called Gaussian Process Hamiltonian Monte Carlo (GPHMC). We compare this personalization approach with two commonly used methods based on the spherical asymptotic analysis of the reaction-diffusion model, and on a derivative-free optimization algorithm. We demonstrate the performance of the method on synthetic data, and on seven patients with a glioblastoma, the most aggressive primary brain tumor. This Bayesian personalization produces more informative results. In particular, it provides samples from the regions of interest and highlights the presence of several modes for some patients. In contrast, previous approaches based on optimization strategies fail to reveal the presence of different modes, and correlation between parameters.

This paper presents an original method for interactions' haptic rendering when treating hyperelastic materials. Such simulations are known to be difficult due to the non-linear behavior of hyperelastic bodies; furthermore, haptic constraints enjoin contact forces to be refreshed at least at 1000 updates per second. To enforce the stability of simulations of generic objects of any range of stiffness, this method relies on implicit time integration. Soft tissues dynamics is simulated in real time (20 to 100 Hz) using the Multiplicative Jacobian Energy Decomposition (MJED) method. An asynchronous preconditioner, updated at low rates (1 to 10 Hz), is used to obtain a close approximation of the mechanical coupling of interactions. Finally, the contact problem is linearized and, using a specific-loop, it is updated at typical haptic rates (around 1000 Hz) allowing this way new simulations of prompt stiff-contacts and providing a continuous haptic feedback as well.

Quantification of cardiac deformation and strain with 3D ultrasound takes considerable research efforts. Nevertheless, a widespread use of these techniques in clinical practice is still held back due to the lack of a solid verification process to quantify and compare performance. In this context, the use of fully synthetic sequences has become an established tool for initial in silico evaluation. Nevertheless, the realism of existing simulation techniques is still too limited to represent reliable benchmarking data. Moreover, the fact that different centers typically make use of in-house developed simulation pipelines makes a fair comparison difficult. In this context, this paper introduces a novel pipeline for the generation of synthetic 3D cardiac ultrasound image sequences. State-of-the art solutions in the fields of electromechanical modeling and ultrasound simulation are combined within an original framework that exploits a real ultrasound recording to learn and simulate realistic speckle textures. The simulated images show typical artifacts that make motion tracking in ultrasound challenging. The ground-truth displacement field is available voxelwise and is fully controlled by the electromechanical model. By progressively modifying mechanical and ultrasound parameters, the sensitivity of 3D strain algorithms to pathology and image properties can be evaluated. The proposed pipeline is used to generate an initial library of 8 sequences including healthy and pathological cases, which is made freely accessible to the research community via our project web-page.

Radiofrequency ablation (RFA) is an established treatment for liver cancer when resection is not possible. Yet, its optimal delivery is challenged by the presence of large blood vessels and the time-varying thermal conductivity of biological tissue. Incomplete treatment and an increased risk of recurrence are therefore common. A tool that would enable the accurate planning of RFA is hence necessary. This manuscript describes a new method to compute the extent of ablation required based on the Lattice Boltzmann Method (LBM) and patient-specific, pre-operative images. A detailed anatomical model of the liver is obtained from volumetric images. Then a computational model of heat diffusion, cellular necrosis, and blood flow through the vessels and liver is employed to compute the extent of ablated tissue given the probe location, ablation duration and biological parameters. The model was verified against an analytical solution, showing good fidelity. We also evaluated the predictive power of the proposed framework on ten patients who underwent RFA, for whom pre- and post-operative images were available. Comparisons between the computed ablation extent and ground truth, as observed in postoperative images, were promising (DICE index: 42%, sensitivity: 67%, positive predictive value: 38%). The importance of considering liver perfusion while simulating electrical-heating ablation was also highlighted. Implemented on graphics processing units (GPU), our method simulates 1 minute of ablation in 1.14 minutes, allowing near real-time computation.

In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients-manually annotated by up to four raters-and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.

We propose a Sparse Bayesian framework for non-rigid registration. Our principled approach is flexible, in that it efficiently finds an optimal, sparse model to represent deformations among any preset, widely overcomplete range of basis functions. It addresses open challenges in state-of-the-art registration, such as the automatic joint estimate of model parameters ( e.g. noise and regularization levels). We demonstrate the feasibility and performance of our approach on cine MR, tagged MR and 3D US cardiac images, and show state-of-the-art results on benchmark datasets evaluating accuracy of motion and strain. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

We propose a technique for myocardial motion estimation based on image registration using both B-mode echocardiographic images and tissue Doppler sequences acquired interleaved. The velocity field is modeled continuously using B-splines and the spatiotemporal transform is constrained to be diffeomorphic. Images before scan conversion are used to improve the accuracy of the estimation. The similarity measure includes a model of the speckle pattern distribution of B-mode images. It also penalizes the disagreement between tissue Doppler velocities and the estimated velocity field. Registration accuracy is evaluated and compared to other alternatives using a realistic synthetic dataset, obtaining mean displacement errors of about 1 mm. Finally, the method is demonstrated on data acquired from six volunteers, both at rest and during exercise. Robustness is tested against low image quality and fast heart rates during exercise. Results show that our method provides a robust motion estimate in these situations.

This paper evaluates five 3D ultrasound tracking algorithms regarding their ability to quantify abnormal deformation in timing or amplitude. A synthetic database of B-mode image sequences modeling healthy, ischemic and dyssynchrony cases was generated for that purpose. This database is made publicly available to the community. It combines recent advances in electromechanical and ultrasound modeling. For modeling heart mechanics, the Bestel-Clement-Sorine electromechanical model was applied to a realistic geometry. For ultrasound modeling, we applied a fast simulation technique to produce realistic images on a set of scatterers moving according to the electromechanical simulation result. Tracking and strain accuracies were computed and compared for all evaluated algorithms. For tracking, all methods were estimating myocardial displacements with an error below 1 mm on the ischemic sequences. The introduction of a dilated geometry was found to have a significant impact on accuracy. Regarding strain, all methods were able to recover timing differences between segments, as well as low strain values. On all cases, radial strain was found to have a low accuracy in comparison to longitudinal and circumferential components.

We propose a new approach for the generation of synthetic but visually realistic time series of cardiac images based on an electromechanical model of the heart and real clinical 4-D image sequences. This is achieved by combining three steps. The first step is the simulation of a cardiac motion using an electromechanical model of the heart and the segmentation of the end diastolic image of a cardiac sequence. We use biophysical parameters related to the desired condition of the simulated subject. The second step extracts the cardiac motion from the real sequence using nonrigid image registration. Finally, a synthetic time series of cardiac images corresponding to the simulated motion is generated in the third step by combining the motion estimated by image registration and the simulated one. With this approach, image processing algorithms can be evaluated as we know the ground-truth motion underlying the image sequence. Moreover, databases of visually realistic images of controls and patients can be generated for which the underlying cardiac motion and some biophysical parameters are known. Such databases can open new avenues for machine learning approaches.

Cardiac fibers, as well as their local arrangement in laminar sheets, have a complex spatial variation of their orientation that has an important role in mechanical and electrical cardiac functions. In this paper, a statistical atlas of this cardiac fiber architecture is built for the first time using human datasets. This atlas provides an average description of the human cardiac fiber architecture along with its variability within the population. In this study, the population is composed of ten healthy human hearts whose cardiac fiber architecture is imaged ex vivo with DT-MRI acquisitions. The atlas construction is based on a computational framework that minimizes user interactions and combines most recent advances in image analysis: graph cuts for segmentation, symmetric log-domain diffeomorphic demons for registration, and log-Euclidean metric for diffusion tensor processing and statistical analysis. Results show that the helix angle of the average fiber orientation is highly correlated to the transmural depth and ranges from -41° on the epicardium to +66° on the endocardium. Moreover, we find that the fiber orientation dispersion across the population (±13°) is lower than for the laminar sheets (±31°) . This study, based on human hearts, extends previous studies on other mammals with concurring conclusions and provides a description of the cardiac fiber architecture more specific to human and better suited for clinical applications. Indeed, this statistical atlas can help to improve the computational models used for radio-frequency ablation, cardiac resynchronization therapy, surgical ventricular restoration, or diagnosis and followups of heart diseases due to fiber architecture anomalies.

Cardiac remodelling plays a crucial role in heart diseases. Analyzing how the heart grows and remodels over time can provide precious insights into pathological mechanisms, eventually resulting in quantitative metrics for disease evaluation and therapy planning. This study aims to quantify the regional impacts of valve regurgitation and heart growth upon the end-diastolic right ventricle (RV) in patients with tetralogy of Fallot, a severe congenital heart defect. The ultimate goal is to determine, among clinical variables, predictors for the RV shape from which a statistical model that predicts RV remodelling is built. Our approach relies on a forward model based on currents and a diffeomorphic surface registration algorithm to estimate an unbiased template. Local effects of RV regurgitation upon the RV shape were assessed with Principal Component Analysis (PCA) and cross-sectional multivariate design. A generative 3-D model of RV growth was then estimated using partial least squares (PLS) and canonical correlation analysis (CCA). Applied on a retrospective population of 49 patients, cross-effects between growth and pathology could be identified. Qualitatively, the statistical findings were found realistic by cardiologists. 10-fold cross-validation demonstrated a promising generalization and stability of the growth model. Compared to PCA regression, PLS was more compact, more precise and provided better predictions.

Reaction-diffusion based tumor growth models have been widely used in the literature for modeling the growth of brain gliomas. Lately, recent models have started integrating medical images in their formulation. Including different tissue types, geometry of the brain and the directions of white matter fiber tracts improved the spatial accuracy of reaction-diffusion models. The adaptation of the general model to the specific patient cases on the other hand has not been studied thoroughly yet. In this paper, we address this adaptation. We propose a parameter estimation method for reaction-diffusion tumor growth models using time series of medical images. This method estimates the patient specific parameters of the model using the images of the patient taken at successive time instances. The proposed method formulates the evolution of the tumor delineation visible in the images based on the reaction-diffusion dynamics; therefore, it remains consistent with the information available. We perform thorough analysis of the method using synthetic tumors and show important couplings between parameters of the reaction-diffusion model. We show that several parameters can be uniquely identified in the case of fixing one parameter, namely the proliferation rate of tumor cells. Moreover, regardless of the value the proliferation rate is fixed to, the speed of growth of the tumor can be estimated in terms of the model parameters with accuracy. We also show that using the model-based speed, we can simulate the evolution of the tumor for the specific patient case. Finally, we apply our method to two real cases and show promising preliminary results.

We propose a framework for the nonlinear spatiotemporal registration of 4D time-series of images based on the Diffeomorphic Demons (DD) algorithm. In this framework, the 4D spatiotemporal registration is decoupled into a 4D temporal registration, defined as mapping physiological states, and a 4D spatial registration, defined as mapping trajectories of physical points. Our contribution focuses more specifically on the 4D spatial registration that should be consistent over time as opposed to 3D registration that solely aims at mapping homologous points at a given time-point. First, we estimate in each sequence the motion displacement field, which is a dense representation of the point trajectories we want to register. Then, we perform simultaneously 3D registrations of corresponding time-points with the constraints to map the same physical points over time called the trajectory constraints. Under these constraints, we show that the 4D spatial registration can be formulated as a multichannel registration of 3D images. To solve it, we propose a novel version of the Diffeomorphic Demons (DD) algorithm extended to vector-valued 3D images, the Multichannel Diffeomorphic Demons (MDD). For evaluation, this framework is applied to the registration of 4D cardiac computed tomography (CT) sequences and compared to other standard methods with real patient data and synthetic data simulated from a physiologically realistic electromechanical cardiac model. Results show that the trajectory constraints act as a temporal regularization consistent with motion whereas the multichannel registration acts as a spatial regularization. Finally, using these trajectory constraints with multichannel registration yields the best compromise between registration accuracy, temporal and spatial smoothness, and computation times. A prospective example of application is also presented with the spatiotemporal registration of 4D cardiac CT sequences of the same patient before and after radiofrequency ablation (RFA) in case of atrial fibrillation (AF). The intersequence spatial transformations over a cardiac cycle allow to analyze and quantify the regression of left ventricular hypertrophy and its impact on the cardiac function.

This paper provides a formal connexion between springs and continuum mechanics in the context of one-dimensional and two-dimensional elasticity. In a first stage, the equivalence between tensile springs and the finite element discretization of stretching energy on planar curves is established. Furthermore, when considering a quadratic strain function of stretch, we introduce a new type of springs called tensile biquadratic springs. In a second stage, we extend this equivalence to non-linear membranes (St Venant-Kirchhoff materials) on triangular meshes leading to triangular biquadratic and quadratic springs. Those tensile and angular springs produce isotropic deformations parameterized by Young modulus and Poisson ratios on unstructured meshes in an efficient and simple way. For a specific choice of the Poisson ratio, 0.3, we show that regular spring-mass models may be used realistically to simulate a membrane behavior. Finally, the different spring formulations are tested in pure traction and cloth simulation experiments.

We propose a unified computational framework to build a statistical atlas of the cardiac fiber architecture from diffusion tensor magnetic resonance images (DT-MRIs). We apply this framework to a small database of nine ex vivo canine hearts. An average cardiac fiber architecture and a measure of its variability are computed using most recent advances in diffusion tensor statistics. This statistical analysis confirms the already established good stability of the fiber orientations and a higher variability of the laminar sheet orientations within a given species. The statistical comparison between the canine atlas and a standard human cardiac DT-MRI shows a better stability of the fiber orientations than their laminar sheet orientations between the two species. The proposed computational framework can be applied to larger databases of cardiac DT-MRIs from various species to better establish intraspecies and interspecies statistics on the anatomical structure of cardiac fibers. This information will be useful to guide the adjustment of average fiber models onto specific patients from in vivo anatomical imaging modalities.

Medical image analysis plays a crucial role in the diagnosis, planning, control, and follow-up of therapy. To be combined efficiently with medical robotics, medical image analysis can be supported by the development of specific computational models of the human body operating at various levels. We describe a hierarchy of these computational models, including the geometrical, physical, and physiological levels, and illustrate their potential use in a number of advanced medical applications including image-guided robot-assisted and simulated medical interventions. We conclude with scientific perspectives.

This paper presents a new three-dimensional electromechanical model of the two cardiac ventricles designed both for the simulation of their electrical and mechanical activity, and for the segmentation of time series of medical images. First, we present the volumetric biomechanical models built. Then the transmembrane potential propagation is simulated, based on FitzHugh-Nagumo reaction-diffusion equations. The myocardium contraction is modeled through a constitutive law including an electromechanical coupling. Simulation of a cardiac cycle, with boundary conditions representing blood pressure and volume constraints, leads to the correct estimation of global and local parameters of the cardiac function. This model enables the introduction of pathologies and the simulation of electrophysiology interventions. Moreover, it can be used for cardiac image analysis. A new proactive deformable model of the heart is introduced to segment the two ventricles in time series of cardiac images. Preliminary results indicate that this proactive model, which integrates a priori knowledge on the cardiac anatomy and on its dynamical behavior, can improve the accuracy and robustness of the extraction of functional parameters from cardiac images even in the presence of noisy or sparse data. Such a model also allows the simulation of cardiovascular pathologies in order to test therapy strategies and to plan interventions.

Integration of the correct tactile and kinesthetic force feedback response with an accurate computational model of a compliant environment is a formidable challenge. We examine several design issues that arise in the construction of a compliance renderer, specifically the interaction between impedances of tactile displays, impedances of robot arms, and the computational model. We also describe an implementation of a compliance rendering system combining a low-impedance robot arm for large workspace kinesthetic force feedback, a high-impedance shape display for distributed tactile feedback to the finger pad, and a real-time finite element modeler. To determine the efficacy of the integration of tactile and kinesthetic force feedback components, we conducted a study examining the user’s ability to discriminate stiffness. Subjects were able to reliably detect a 20% difference in rendered material stiffness using our compliance rendering system.

We present a new algorithm to register 3-D preoperative magnetic resonance (MR) images to intraoperative MR images of the brain which have undergone brain shift. This algorithm relies on a robust estimation of the deformation from a sparse noisy set of measured displacements. We propose a new framework to compute the displacement field in an iterative process, allowing the solution to gradually move from an approximation formulation (minimizing the sum of a regularization term and a data error term) to an interpolation formulation (least square minimization of the data error term). An outlier rejection step is introduced in this gradual registration process using a weighted least trimmed squares approach, aiming at improving the robustness of the algorithm. We use a patient-specific model discretized with the finite element method in order to ensure a realistic mechanical behavior of the brain tissue. To meet the clinical time constraint, we parallelized the slowest step of the algorithm so that we can perform a full 3-D image registration in 35 s (including the image update time) on a heterogeneous cluster of 15 personal computers. The algorithm has been tested on six cases of brain tumor resection, presenting a brain shift of up to 14 mm. The results show a good ability to recover large displacements, and a limited decrease of accuracy near the tumor resection cavity.

We propose a new model to simulate the three-dimensional (3-D) growth of glioblastomas multiforma (GBMs), the most aggressive glial tumors. The GBM speed of growth depends on the invaded tissue: faster in white than in gray matter, it is stopped by the dura or the ventricles. These different structures are introduced into the model using an atlas matching technique. The atlas includes both the segmentations of anatomical structures and diffusion information in white matter fibers. We use the finite element method (FEM) to simulate the invasion of the GBM in the brain parenchyma and its mechanical interaction with the invaded structures (mass effect). Depending on the considered tissue, the former effect is modeled with a reaction-diffusion or a Gompertz equation, while the latter is based on a linear elastic brain constitutive equation. In addition, we propose a new coupling equation taking into account the mechanical influence of the tumor cells on the invaded tissues. The tumor growth simulation is assessed by comparing the in-silico GBM growth with the real growth observed on two magnetic resonance images (MRIs) of a patient acquired with 6 mo difference. Results show the feasibility of this new conceptual approach and justifies its further evaluation.

One of the most important task in surgical simulation is the ability to cut volumetric organs. Several algorithms have already been described but none of them can actually maintain a specific and important topological property of the mesh called manifoldness. In this article we define the notion of manifoldness and we explain why it is important to preserve it. We propose a new algorithm that maintains manifoldness and that implements a very simple cut strategy: the removing of soft tissue material, which is an efficient way to simulate the action of an ultrasound cautery. Finally, we present experimental results which show the efficiency of this algorithm in a surgery simulation system.

Abstract In this article, we describe the latest developments of the minimally invasive hepatic surgery simulator prototype developed at INRIA. The goal of this simulator is to provide a realistic training test bed to perform laparoscopic procedures. Therefore, its main functionality is to simulate the action of virtual laparoscopic surgical instruments for deforming and cutting tridimensional anatomical models. Throughout this paper, we present the general features of this simulator including the implementation of several biomechanical models and the integration of two force‐feedback devices in the simulation platform. More precisely, we describe three new important developments that improve the overall realism of our simulator. First, we have developed biomechanical models, based on linear elasticity and finite element theory, that include the notion of anisotropic deformation. Indeed, we have generalized the linear elastic behaviour of anatomical models to ‘transversally isotropic’ materials, i.e. materials having a different behaviour in a given direction. We have also added to the volumetric model an external elastic membrane representing the ‘liver capsule’, a rather stiff skin surrounding the liver, which creates a kind of ‘surface anisotropy’. Second, we have developed new contact models between surgical instruments and soft tissue models. For instance, after detecting a contact with an instrument, we define specific boundary constraints on deformable models to represent various forms of interactions with a surgical tool, such as sliding, gripping, cutting or burning. In addition, we compute the reaction forces that should be felt by the user manipulating the force‐feedback devices. The last improvement is related to the problem of haptic rendering. Currently, we are able to achieve a simulation frequency of 25 Hz (visual real time) with anatomical models of complex geometry and behaviour. But to achieve a good haptic feedback requires a frequency update of applied forces typically above 300 Hz (haptic real time). Thus, we propose a force extrapolation algorithm in order to reach haptic real time. Copyright © 2002 John Wiley & Sons, Ltd.

We describe the latest developments of the minimally invasive hepatic surgery simulator prototype developed at INRIA. A key problem with such a simulator is the physical modeling of soft tissues. We propose a new deformable model based on nonlinear elasticity and the finite element method. This model is valid for large displacements, which means in particular that it is invariant with respect to rotations. This property improves the realism of the deformations and solves the problems related to the shortcomings of linear elasticity, which is only valid for small displacements. We also address the problem of anisotropic behavior, and volume variations by adding to our model incompressibility constraints. Finally, we demonstrate the relevance of this approach for the real-time simulation of laparoscopic surgical gestures on the liver.

Medical image segmentation is often a difficult task due to the low contrast, the low signal/noise ratio and the presence of outliers in images. However, it remains a critical issue for image interpretation, pattern recognition and automatic diagnosis. Deformable models are well-suited for capturing the geometry and the shape variability of anatomical structures from medical images. Indeed, they introduce an a priori knowledge in the segmentation process that increases its robustness to noise and outliers. In this paper, we address many problems related to volumetric medical image segmentation based on deformable models including model initialization, model topology, deformation behavior and image features extraction.

We describe the latest developments of the minimally invasive hepatic surgery simulator prototype developed at INRIA. The goal of this simulator is to provide a realistic training test-bed for performing laparoscopic procedures. Therefore, its main functionality is to simulate the deformation and cutting of tri-dimensional anatomical models with the help of two virtual laparoscopic surgical instruments. Throughout the paper, we present the general features of the simulator including the implementation of different bio-mechanical models based on linear elasticity and finite element theory and the integration of two force-feedback devices in the simulation platform. More precisely, we describe two important developments that improve the overall realism of the simulator. First, we can create bio-mechanical models that include the notion of anisotropic deformation. Indeed, we have generalized the linear elastic behavior of anatomical models to "transversally isotropic" materials, i.e. materials having one privileged direction of deformation. The second improvement is related to the problem of haptic rendering. Currently, we are able to achieve a simulation frequency of 25 Hz (visual real-time) with anatomical models of complex geometry and behavior. But to achieve a good haptic feedback requires a frequency update of applied forces typically above 300 Hz (haptic real-time). Thus, we propose a force extrapolation algorithm in order to reach haptic real-time.

We describe the basic components of a surgery simulator prototype developed at INRIA. After a short presentation of the geometric modeling of anatomical structures from medical images, we insist on the physical modeling components which must allow realistic interaction with surgical instruments. We present three physical models which are well suited for surgery simulation. Those models are based on linear elasticity theory and finite element modeling. The first model pre-computes the deformations and forces applied on a finite element model, therefore allowing the deformation of large structures in real-time. Unfortunately, it does not allow any topology change of the mesh therefore forbids the simulation of cutting during surgery. The second physical model is based on a dynamic law of motion and allows to simulate cutting and tearing. We called this model "tensor-mass" since it is analogous to spring-mass models for linear elasticity. This model allows volumetric deformations and cuttings, but has to be applied to a limited number of nodes to run in real-time. Finally, we propose a method for combining those two approaches into a hybrid model which may allow real time deformations and cuttings of large enough anatomical structures. This model has been implemented in a simulation system and real-time experiments are described and illustrated.

We describe a novel method for surgery simulation including a volumetric model built from medical images and an elastic modeling of the deformations. The physical model is based on elasticity theory which suitably links the shape of deformable bodies and the forces associated with the deformation. A real time computation of the deformation is possible thanks to a preprocessing of elementary deformations derived from a finite element method. This method has been implemented in a system including a force feedback device and a collision detection algorithm. The simulator works in real time with a high resolution liver model.

The robustness of shape recovery based on deformable models depends in general, on the relative difference of position and topology of the initial model with respect to the data: a close initialization with correct topology guarantees a proper recovery of the object. Furthermore, the closeness of the initial model greatly influences the time of computation needed for the recovery. In this paper, we propose a method for initializing deformable models from range data or volumetric images. The proposed method solves two distinct problems. First, we use the topological segmentation of volumetric images in order to recover the approximate topology of the object. Second, we use an efficient mesh sampling algorithm to control the number of vertices of the initial model. The method takes into account missing data and outliers.

This article reports the ideas presented by Cotin, Delingette and Ayache (1996) for developing a real-time surgery simulation system, for the training of surgeons. This system allows the interaction with volumetric deformable models of organs, and provides visual and haptic feedback in real-time. The geometry of organs is acquired from medical images. The physical properties are based on linear elasticity, and deformations are computed with finite elements. A preprocessing technique allows real-time computation of deformations and forces. The method has been extended to introduce a nonlinear behaviour closer to the biomechanical behaviour of soft tissues, while preserving real-time. We present the basic principles of the approach and results obtained with our experimental system.

Most of today's medical simulation systems are based on geometric representations of anatomical structures that take no account of their physical nature. Representing physical phenomena and, more specifically, the realistic modeling of soft tissue will not only improve current medical simulation systems but will considerably enlarge the set of applications and the credibility of medical simulation, from neurosurgery planning to laparoscopic-surgery simulation. To achieve realistic tissue deformation, it is necessary to combine deformation accuracy with computer efficiency. On the one hand, biomechanics has studied complex mathematical models and produced a large amount of experimental data for accurately representing the deformation of soft tissue. On the other hand, computer graphics has proposed many algorithms for the real-time computation of deformable bodies, often at the cost of ignoring the physics principles. The author surveys existing models of deformation in medical simulation and analyze the impediments to combining computer-graphics representations with biomechanical models. In particular, the different geometric representations of deformable tissue are compared in relation to the tasks of real-time deformation, tissue cutting, and force-feedback interaction. Last, the author inspects the potential of medical simulation under the development of this key technology.

In computer vision, two complementary approaches have been widely used to perform object reconstruction and registration. The deformable model framework locally applies internal and external forces to fit 3D data. The non-rigid registration framework iteratively computes the best global transformation that minimizes the distance between a template and the data. In this paper first we show that applying a global transformation on a surface model, is equivalent to applying an external force on a deformable model without any regularizing force. Second we propose a hybrid framework that combines the registration framework and the deformable models framework. Our hybrid deformation approach allows us to control the scale at which the model is deformed. This is clearly beneficial for performing both reconstruction and registration tasks. We show examples of this approach on active contours and deformable surfaces. Furthermore, a global transformation based on axial symmetry is introduced.

Introduces a new surface representation for recognizing curved objects. The authors approach begins by representing an object by a discrete mesh of points built from range data or from a geometric model of the object. The mesh is computed from the data by deforming a standard shaped mesh, for example, an ellipsoid, until it fits the surface of the object. The authors define local regularity constraints that the mesh must satisfy. The authors then define a canonical mapping between the mesh describing the object and a standard spherical mesh. A surface curvature index that is pose-invariant is stored at every node of the mesh. The authors use this object representation for recognition by comparing the spherical model of a reference object with the model extracted from a new observed scene. The authors show how the similarity between reference model and observed data can be evaluated and they show how the pose of the reference object in the observed scene can be easily computed using this representation. The authors present results on real range images which show that this approach to modelling and recognizing 3D objects has three main advantages: (1) it is applicable to complex curved surfaces that cannot be handled by conventional techniques; (2) it reduces the recognition problem to the computation of similarity between spherical distributions; in particular, the recognition algorithm does not require any combinatorial search; and (3) even though it is based on a spherical mapping, the approach can handle occlusions and partial views.>

Simplex meshes are simply connected meshes that are topologically dual of triangulations. In a previous work we have introduced the Simplex mesh representation for performing recognition of partially occluded smooth objects (ICCV'93, p.103-12). In this paper, we present a physically-based approach for recovering three-dimensional objects, based on the geometry of Simplex meshes. Elastic behavior is modelled by local stabilizing functionals, controlling the mean curvature through the Simplex angle extracted at each vertex. Those functionals are viewpoint-invariant, intrinsic and scale-sensitive. Unlike deformable surfaces defined on regular grids, Simplex meshes are highly adaptive structures, and we have developed a refinement process for increasing the mesh resolution at highly curved or inaccurate parts. Furthermore, operations for connecting Simplex meshes are performed to recover complex models from parts with simpler shapes.>

The authors introduce a surface representation for recognizing curved objects. The approach begins by representing an object by a discrete mesh of points built from range data or from a geometric model of the object. The mesh is computed from the data by deforming a standard shaped mesh, for example, an ellipsoid, until it fits the surface of the object. Local regularity constraints that the mesh must satisfy are defined. A canonical mapping is then defined between the mesh describing the object and a standard spherical mesh. A surface curvature index which is pose-invariant is stored at every node of the mesh. This object representation is used for recognition by comparing the spherical model of a reference object with the model extracted from a new observed scene. It is shown that the similarity between reference model and observed data can be evaluated, and it is also demonstrated that the pose of the reference object in the observed scene can be easily computed using this representation.>

A technique for constructing shape representation from images using free-form deformable surfaces is presented. The authors model an object as a closed surface that is deformed subject to attractive fields generated by input data points and features. Features affect the global shape of the surface, while data points control its local shape. This approach is used to segment objects even in cluttered or unstructured environments. The algorithm is general in that it makes few assumptions on the type of features, the nature of the data, and the type of objects. Results for a wide range of applications are presented: reconstruction of smooth isolated objects such as human faces, reconstruction of structured objects such as polyhedra, and segmentation of complex scenes with mutually occluding objects. The algorithm has been successfully tested using data from different sensors including grey-coding range finders and video cameras, using one or several images.>

The trajectory generation problem for mobile robots consists in providing a set of trajectories that are 'smooth' and meet certain boundary conditions. The authors present a method to generate curvature continuous trajectories for which the curvature profile is a polynomial function of arc length. An algorithm based on the deformation of a curve by energy minimization allows one to solve general geometric constraints which was not possible by previous methods. Furthermore, it is able to take into account the limitation of radius of curvature of the robot by controlling the extrema of curvature along the path.>