Motion estimation in echocardiography plays an important role in the characterization of cardiac function, allowing the computation of myocardial deformation indices. However, there exist limitations in clinical practice, particularly with regard to the accuracy and robustness of measurements extracted from images. We therefore propose a novel deep learning solution for motion estimation in echocardiography. Our network corresponds to a modified version of PWC-Net which achieves high performance on ultrasound sequences. In parallel, we designed a novel simulation pipeline allowing the generation of a large amount of realistic B-mode sequences. These synthetic data, together with strategies during training and inference, were used to improve the performance of our deep learning solution, which achieved an average endpoint error of 0.07 ± 0.06 mm per frame and 1.20 ± 0.67 mm between ED and ES on our simulated dataset. The performance of our method was further investigated on 30 patients from a publicly available clinical dataset acquired from a GE system. The method showed promise by achieving a mean absolute error of the global longitudinal strain of 2.5 ± 2.1% and a correlation of 0.77 compared to GLS derived from manual segmentation, much better than one of the most efficient methods in the state-of-the-art (namely the FFT-Xcorr block-matching method). We finally evaluated our method on an auxiliary dataset including 30 patients from another center and acquired with a different system. Comparable results were achieved, illustrating the ability of our method to maintain high performance regardless of the echocardiographic data processed.

3D echocardiography is an increasingly popular tool for assessing cardiac remodelling in the right ventricle (RV). It allows quantification of the cardiac chambers without any geometric assumptions, which is the main weakness of 2D echocardiography. However, regional quantification of geometry and function is limited by the lower spatial and temporal resolution and the scarcity of identifiable anatomical landmarks, especially within the ventricular cavity. We developed a technique for regionally assessing the volume of 3 relevant RV volumetric regions: apical, inlet and outflow. The proposed parcellation method is based on the geodesic distances to anatomical landmarks that are easily identifiable in the images: the apex and the tricuspid and pulmonary valves, each associated to a region. Based on these distances, we define a partition in the endocardium at end-diastole (ED). This partition is then interpolated to the blood cavity using the Laplace equation, which allows to compute regional volumes. For obtaining an end-systole (ES) partition, the endocardial partition is transported from ED to ES using a commercial image-based tracking software, and then the interpolation process is repeated. We assessed the intra- and inter-observer reproducibility using a 10-subjects dataset containing repeated quantifications of the same images, obtaining intra- and inter- observer errors (7-12% and 10-23% respectively). Finally, we propose a novel synthetic mesh generation algorithm that deforms a template mesh imposing a user-defined strain to a template mesh. We used this method to create a new dataset for involving distinct types of remodelling that were used to assess the sensitivity of the parcellation method to identify volume changes affecting different parts. We show that the parcellation method is adequate for capturing local circumferential and global circumferential and longitudinal RV remodelling, which are the most clinically relevant cases.

Clinical decision requires reasoning in the presence of imperfect data. DTs are a well-known decision support tool, owing to their interpretability, fundamental in safety-critical contexts such as medical diagnosis. However, learning DTs from uncertain data leads to poor generalization, and generating predictions for uncertain data hinders prediction accuracy. Several methods have suggested the potential of probabilistic decisions at the internal nodes in making DTs robust to uncertainty. Some approaches only employ probabilistic thresholds during evaluation. Others also consider the uncertainty in the learning phase, at the expense of increased computational complexity or reduced interpretability. The existing methods have not clarified the merit of a probabilistic approach in the distinct phases of DT learning, nor when the uncertainty is present in the training or the test data. We present a probabilistic DT approach that models measurement uncertainty as a noise distribution, independently realized: (1) when searching for the split thresholds, (2) when splitting the training instances, and (3) when generating predictions for unseen data. The soft training approaches (1, 2) achieved a regularizing effect, leading to significant reductions in DT size, while maintaining accuracy, for increased noise. Soft evaluation (3) showed no benefit in handling noise.

Decision trees (DTs) are a widely used prediction tool, owing to their interpretability. Standard learning methods follow a locally-optimal approach that trades off prediction performance for computational efficiency. Such methods can however be far from optimal, and it may pay off to spend more computational resources to increase performance. Monte Carlo tree search (MCTS) is an approach to approximate optimal choices in exponentially large search spaces. Since exploring the space of all possible DTs is computationally intractable, we propose a DT learning approach based on MCTS. To bound the branching factor of MCTS, we limit the number of decisions at each level of the search tree, and introduce mechanisms to balance exploration, DT size and the statistical significance of the predictions. To mitigate the computational cost of our method, we employ a move pruning strategy that discards some branches of the search tree, leading to improved performance. The experiments show that our approach outperformed locally optimal search in 20 out of 31 datasets, with a reduction in DT size in most of the cases.

The use of synthetic sequences is one of the most promising tools for advanced in silico evaluation of the quantification of cardiac deformation and strain through 3-D ultrasound (US) and magnetic resonance (MR) imaging. In this paper, we propose the first simulation framework which allows the generation of realistic 3-D synthetic cardiac US and MR (both cine and tagging) image sequences from the same virtual patient. A state-of-the-art electromechanical (E/M) model was exploited for simulating groundtruth cardiac motion fields ranging from healthy to various pathological cases, including both ventricular dyssynchrony and myocardial ischemia. The E/M groundtruth along with template MR/US images and physical simulators were combined in a unified framework for generating synthetic data. We efficiently merged several warping strategies to keep the full control of myocardial deformations while preserving realistic image texture. In total, we generated 18 virtual patients, each with synthetic 3-D US, cine MR, and tagged MR sequences. The simulated images were evaluated both qualitatively by showing realistic textures and quantitatively by observing myocardial intensity distributions similar to real data. In particular, the US simulation showed a smoother myocardium/background interface than the state-of-the-art. We also assessed the mechanical properties. The pathological subjects were discriminated from the healthy ones by both global indexes (ejection fraction and the global circumferential strain) and regional strain curves. The synthetic database is comprehensive in terms of both pathology and modality, and has a level of realism sufficient for validation purposes. All the 90 sequences are made publicly available to the research community via an open-access database.

Cardiac motion atlases provide a space of reference in which the motions of a cohort of subjects can be directly compared. Motion atlases can be used to learn descriptors that are linked to different pathologies and which can subsequently be used for diagnosis. To date, all such atlases have been formed and applied using data from the same modality. In this work we propose a framework to build a multimodal cardiac motion atlas from 3D magnetic resonance (MR) and 3D ultrasound (US) data. Such an atlas will benefit from the complementary motion features derived from the two modalities, and furthermore, it could be applied in clinics to detect cardiovascular disease using US data alone. The processing pipeline for the formation of the multimodal motion atlas initially involves spatial and temporal normalisation of subjects' cardiac geometry and motion. This step was accomplished following a similar pipeline to that proposed for single modality atlas formation. The main novelty of this paper lies in the use of a multi-view algorithm to simultaneously reduce the dimensionality of both the MR and US derived motion data in order to find a common space between both modalities to model their variability. Three different dimensionality reduction algorithms were investigated: principal component analysis, canonical correlation analysis and partial least squares regression (PLS). A leave-one-out cross validation on a multimodal data set of 50 volunteers was employed to quantify the accuracy of the three algorithms. Results show that PLS resulted in the lowest errors, with a reconstruction error of less than 2.3 mm for MR-derived motion data, and less than 2.5 mm for US-derived motion data. In addition, 1000 subjects from the UK Biobank database were used to build a large scale monomodal data set for a systematic validation of the proposed algorithms. Our results demonstrate the feasibility of using US data alone to analyse cardiac function based on a multimodal motion atlas.

A plethora of techniques for cardiac deformation imaging with 3D ultrasound, typically referred to as 3D speckle tracking techniques, are available from academia and industry. Although the benefits of single methods over alternative ones have been reported in separate publications, the intrinsic differences in the data and definitions used makes it hard to compare the relative performance of different solutions. To address this issue, we have recently proposed a framework to simulate realistic 3D echocardiographic recordings and used it to generate a common set of ground-truth data for 3D speckle tracking algorithms, which was made available online. The aim of this study was therefore to use the newly developed database to contrast non-commercial speckle tracking solutions from research groups with leading expertise in the field. The five techniques involved cover the most representative families of existing approaches, namely block-matching, radio-frequency tracking, optical flow and elastic image registration. The techniques were contrasted in terms of tracking and strain accuracy. The feasibility of the obtained strain measurements to diagnose pathology was also tested for ischemia and dyssynchrony.

Diagnosing and localizing myocardial infarct is crucial for early patient management and therapy planning. We propose a new method for predicting the location of myocardial infarct from local wall deformation, which has value for risk stratification from routine examinations such as (3D) echocardiography. The pipeline combines non-linear dimensionality reduction of deformation patterns and two multi-scale kernel regressions. Confidence in the diagnosis is assessed by a map of local uncertainties, which integrates plausible infarct locations generated from the space of reduced dimensionality. These concepts were tested on 500 synthetic cases generated from a realistic cardiac electromechanical model, and 108 pairs of 3D echocardiographic sequences and delayed-enhancement magnetic resonance images from real cases. Infarct prediction is made at a spatial resolution around 4 mm, more than 10 times smaller than the current diagnosis, made regionally. Our method is accurate, and significantly outperforms the clinically-used thresholding of the deformation patterns (on real data: sensitivity/specificity of 0.828/0.804, area under the curve: 0.909 versus 0.742 for the most predictive strain component). Uncertainty adds value to refine the diagnosis and eventually re-examine suspicious cases.

Quantification of cardiac deformation and strain with 3D ultrasound takes considerable research efforts. Nevertheless, a widespread use of these techniques in clinical practice is still held back due to the lack of a solid verification process to quantify and compare performance. In this context, the use of fully synthetic sequences has become an established tool for initial in silico evaluation. Nevertheless, the realism of existing simulation techniques is still too limited to represent reliable benchmarking data. Moreover, the fact that different centers typically make use of in-house developed simulation pipelines makes a fair comparison difficult. In this context, this paper introduces a novel pipeline for the generation of synthetic 3D cardiac ultrasound image sequences. State-of-the art solutions in the fields of electromechanical modeling and ultrasound simulation are combined within an original framework that exploits a real ultrasound recording to learn and simulate realistic speckle textures. The simulated images show typical artifacts that make motion tracking in ultrasound challenging. The ground-truth displacement field is available voxelwise and is fully controlled by the electromechanical model. By progressively modifying mechanical and ultrasound parameters, the sensitivity of 3D strain algorithms to pathology and image properties can be evaluated. The proposed pipeline is used to generate an initial library of 8 sequences including healthy and pathological cases, which is made freely accessible to the research community via our project web-page.

We propose a technique for myocardial motion estimation based on image registration using both B-mode echocardiographic images and tissue Doppler sequences acquired interleaved. The velocity field is modeled continuously using B-splines and the spatiotemporal transform is constrained to be diffeomorphic. Images before scan conversion are used to improve the accuracy of the estimation. The similarity measure includes a model of the speckle pattern distribution of B-mode images. It also penalizes the disagreement between tissue Doppler velocities and the estimated velocity field. Registration accuracy is evaluated and compared to other alternatives using a realistic synthetic dataset, obtaining mean displacement errors of about 1 mm. Finally, the method is demonstrated on data acquired from six volunteers, both at rest and during exercise. Robustness is tested against low image quality and fast heart rates during exercise. Results show that our method provides a robust motion estimate in these situations.

This paper evaluates five 3D ultrasound tracking algorithms regarding their ability to quantify abnormal deformation in timing or amplitude. A synthetic database of B-mode image sequences modeling healthy, ischemic and dyssynchrony cases was generated for that purpose. This database is made publicly available to the community. It combines recent advances in electromechanical and ultrasound modeling. For modeling heart mechanics, the Bestel-Clement-Sorine electromechanical model was applied to a realistic geometry. For ultrasound modeling, we applied a fast simulation technique to produce realistic images on a set of scatterers moving according to the electromechanical simulation result. Tracking and strain accuracies were computed and compared for all evaluated algorithms. For tracking, all methods were estimating myocardial displacements with an error below 1 mm on the ischemic sequences. The introduction of a dilated geometry was found to have a significant impact on accuracy. Regarding strain, all methods were able to recover timing differences between segments, as well as low strain values. On all cases, radial strain was found to have a low accuracy in comparison to longitudinal and circumferential components.

Atlases and statistical models play important roles in the personalization and simulation of cardiac physiology. For the study of the heart, however, the construction of comprehensive atlases and spatio-temporal models is faced with a number of challenges, in particular the need to handle large and highly variable image datasets, the multi-region nature of the heart, and the presence of complex as well as small cardiovascular structures. In this paper, we present a detailed atlas and spatio-temporal statistical model of the human heart based on a large population of 3D+time multi-slice computed tomography sequences, and the framework for its construction. It uses spatial normalization based on nonrigid image registration to synthesize a population mean image and establish the spatial relationships between the mean and the subjects in the population. Temporal image registration is then applied to resolve each subject-specific cardiac motion and the resulting transformations are used to warp a surface mesh representation of the atlas to fit the images of the remaining cardiac phases in each subject. Subsequently, we demonstrate the construction of a spatio-temporal statistical model of shape such that the inter-subject and dynamic sources of variation are suitably separated. The framework is applied to a 3D+time data set of 138 subjects. The data is drawn from a variety of pathologies, which benefits its generalization to new subjects and physiological studies. The obtained level of detail and the extendability of the atlas present an advantage over most cardiac models published previously.

The accurate diagnosis of osteoporosis has gained increasing importance due to the aging of our society. Areal bone mineral density (BMD) measured by dual-energy X-ray absorptiometry (DXA) is an established criterion in the diagnosis of osteoporosis. This measure, however, is limited by its two-dimensionality. This work presents a method to reconstruct both the 3D bone shape and 3D BMD distribution of the proximal femur from a single DXA image used in clinical routine. A statistical model of the combined shape and BMD distribution is presented, together with a method for its construction from a set of quantitative computed tomography (QCT) scans. A reconstruction is acquired in an intensity based 3D-2D registration process whereby an instance of the model is found that maximizes the similarity between its projection and the DXA image. Reconstruction experiments were performed on the DXA images of 30 subjects, with a model constructed from a database of QCT scans of 85 subjects. The accuracy was evaluated by comparing the reconstructions with the same subject QCT scans. The method presented here can potentially improve the diagnosis of osteoporosis and fracture risk assessment from the low radiation dose and low cost DXA devices currently used in clinical routine.

This paper presents a technique to estimate and model patient-specific pulsatility of cerebral aneurysms over one cardiac cycle, using 3D rotational X-ray angiography (3DRA) acquisitions. Aneurysm pulsation is modeled as a time varying B-spline tensor field representing the deformation applied to a reference volume image, thus producing the instantaneous morphology at each time point in the cardiac cycle. The estimated deformation is obtained by matching multiple simulated projections of the deforming volume to their corresponding original projections. A weighting scheme is introduced to account for the relevance of each original projection for the selected time point. The wide coverage of the projections, together with the weighting scheme, ensures motion consistency in all directions. The technique has been tested on digital and physical phantoms that are realistic and clinically relevant in terms of geometry, pulsation and imaging conditions. Results from digital phantom experiments demonstrate that the proposed technique is able to recover subvoxel pulsation with an error lower than 10% of the maximum pulsation in most cases. The experiments with the physical phantom allowed demonstrating the feasibility of pulsation estimation as well as identifying different pulsation regions under clinical conditions.

In this paper we compare three non-rigid registration methods for atlas-based segmentation: B-splines, morphons and a combination of morphons and demons. To assess the quality of each method, we use a data set of four patients, containing for each patient the computed tomography (CT) image and a manual segmentation of the organs at risk performed by an expert of the head and neck anatomy. Non-rigid registration algorithms have been used to match the patient and atlas images. Each deformation field, resulting from the non-rigid deformation, have been applied on the masks corresponding to segmented regions in the atlas. The atlas based segmented masks have been compared to manual segmentations performed by the expert. The results show that the combined method (morphons + demons) achieves the best performances on this dataset resulting in an average improvement of 6% with respect to morphons and 18% with respect to B-spline.

This paper introduces a new metric to gather a large collection of segmented images into a same reference system. Different positions for each subject (pose parameters) as well as high energy shape variations need to be compensated before performing statistical analysis (like principal components analysis) on the database. The atlas is obtained as the hidden variable of an expectation-maximization (EM), looking for the right signal intensity at each voxel in the collection of subjects. Each subject is aligned on the current probabilistic atlas by maximizing mutual information. A fast stochastic optimization algorithm is used for estimating pose and scale parameters and a variational approach have been designed to estimate non-rigid transformations. We illustrate the effectiveness of this method for the alignment of 31 brain segmented in 4 labels: background, white and gray matter and ventricles. Our approach has the advantage of keeping a reasonably low complexity even for large databases.

This paper introduces a new registration method estimating the displacement field of bodies which deformations are constrained by an articulated rigid body. We propose an articulated transformation model embedded in a general registration scheme. A fast stochastic gradient descent optimization strategy suitable for noisy cost functions has been chosen to maximize the mutual information metric. Once registered, we propose to propagate the deformation by a linear elastic model through the use of a tetrahedral mesh. We demonstrate this method on 3D CT of neck images where bony structures between different patient images, as vertebrae, may rigidly move while other tissues may deform.