• First multi-view infant 3D face reconstruction algorithm from triplets images (frontal, left and right) • Transformer-based multi-view 2D-3D reconstruction network • Geometric deep learning autoencoder replaces PCA-based models for richer, nonlinear facial shape representation • Cost-effective and simple approach, suitable for limited-resource settings Craniofacial anomalies are important diagnostic markers in early life. Recent studies emphasize the value of 3D imaging for extracting robust facial features that are potential indicators of disease. However, widespread availability of 3D scanning devices in hospitals remains a challenge and this type of technology may not be available in limited-resource settings. For this reason, we present a new approach to generate precise baby 3D face reconstructions from multiple uncalibrated 2D photographs acquired with a smartphone camera. The novel multi-view transformer network presented takes as input three uncalibrated photographs of the baby in frontal, left, and right pose. It then maps these images to a previously learned latent space that captures the baby’s 3D facial morphology, using a 2D vision transformer encoder. Subsequently, the estimated 3D geometry is recovered by decoding the latent vector using a 3D graph convolutional network decoder. Our network demonstrates a normalized mean error of 3.29% and a root mean square error of 2.62 mm between reconstructed and true 3D faces in the baby test dataset. These outcomes are comparable with those reported by both single-view and multi-view 2D-3D reconstruction state-of-the-art errors in adult models. To conclude, the presented Multi-view BabyMorph generates highly accurate 3D baby facial reconstructions from uncalibrated photographs, simplifying the process of 3D photogrammetry. This innovation expands access to advanced baby diagnosis tools, particularly in resource-limited settings.

3D photogrammetry is a cost-effective, non-invasive imaging modality that does not require the use of ionizing radiation or sedation. Therefore, it is specifically valuable in pediatrics and is used to support the diagnosis and longitudinal study of craniofacial developmental pathologies such as craniosynostosis — the premature fusion of one or more cranial sutures resulting in local cranial growth restrictions and cranial malformations . Analysis of 3D photogrammetry requires the identification of craniofacial landmarks to segment the head surface and compute metrics to quantify anomalies. Unfortunately, commercial 3D photogrammetry software requires intensive manual landmark placements, which is time-consuming and prone to errors. We designed and implemented SHAPE, a System for Head-shape Analysis and Pediatric Evaluation. It integrates our previously developed automated landmarking method in a visual computing pipeline to evaluate a patient’s 3D photogram while allowing for manual confirmation and correction. It also automatically computes advanced metrics to quantify craniofacial anomalies and automatically creates a report that can be uploaded to the patient’s electronic health record . We conducted a user study with a professional clinical photographer to compare SHAPE to the existing clinical workflow. We found that SHAPE allows for the evaluation of a craniofacial 3D photogram more than three times faster than the current clinical workflow ( 3 . 85 ± 0 . 99 vs. 13 . 07 ± 5 . 29 minutes, p < 0 . 001 ). Our qualitative study findings indicate that the SHAPE workflow is well aligned with the existing clinical workflow and that SHAPE has useful features and is easy to learn.

We present a 3D face reconstruction system that aims at recovering the 3D facial geometry of babies from uncalibrated photographs, BabyNet. Since the 3D facial geometry of babies differs substantially from that of adults, baby-specific facial reconstruction systems are needed. BabyNet consists of two stages: 1) a 3D graph convolutional autoencoder learns a latent space of the baby 3D facial shape; and 2) a 2D encoder that maps photographs to the 3D latent space based on representative features extracted using transfer learning. In this way, using the pre-trained 3D decoder, we can recover a 3D face from 2D images. We evaluate BabyNet and show that 1) methods based on adult datasets cannot model the 3D facial geometry of babies, which proves the need for a baby-specific method, and 2) BabyNet outperforms classical model-fitting methods even when a baby-specific 3D morphable model, such as BabyFM, is used.

Image segmentation, labeling, and landmark detection are essential tasks for pediatric craniofacial evaluation. Although deep neural networks have been recently adopted to segment cranial bones and locate cranial landmarks from computed tomography (CT) or magnetic resonance (MR) images, they may be hard to train and provide suboptimal results in some applications. First, they seldom leverage global contextual information that can improve object detection performance. Second, most methods rely on multi-stage algorithm designs that are inefficient and prone to error accumulation. Third, existing methods often target simple segmentation tasks and have shown low reliability in more challenging scenarios such as multiple cranial bone labeling in highly variable pediatric datasets. In this paper, we present a novel end-to-end neural network architecture based on DenseNet that incorporates context regularization to jointly label cranial bone plates and detect cranial base landmarks from CT images. Specifically, we designed a context-encoding module that encodes global context information as landmark displacement vector maps and uses it to guide feature learning for both bone labeling and landmark identification. We evaluated our model on a highly diverse pediatric CT image dataset of 274 normative subjects and 239 patients with craniosynostosis (age 0.63 ± 0.54 years, range 0-2 years). Our experiments demonstrate improved performance compared to state-of-the-art approaches.

Early detection of facial dysmorphology - variations of the normal facial geometry - is essential for the timely detection of genetic conditions, which has a significant impact in the reduction of the mortality and morbidity associated with them. A model encoding the normal variability in the healthy population can serve as a reference to quantify the often subtle facial abnormalities that are present in young patients with such conditions. In this paper, we present the first facial model constructed exclusively from newborn data, the Baby Face Model (BabyFM). Our model is built from 3D scans with an innovative pipeline based on least squared conformal maps (LSCM). LSCM are piece-wise linear mappings that project the training faces to a common 2D space minimising the conformal distortion. This process allows improving the correspondences between 3D faces, which is particularly important for the identification of subtle dysmorphology. We evaluate the ability of our BabyFM to recover the babys facial morphology from a set of 2D images by comparing it to state-of-the-art facial models. We also compare it to models built following an analogous pipeline to the one proposed in this paper but using nonrigid iterative closest point (NICP) to establish dense correspondences between the training faces. The results show that our model reconstructs the facial morphology of babies with significantly smaller errors than the state-of-the-art models (p = 10-4) and the “NICP models” (p <; 0.01).

Metopic craniosynostosis is a condition caused by the premature fusion of the metopic cranial suture. If untreated, it can result into brain growth restriction, increased intra-cranial pressure, visual impairment, and cognitive delay. Fronto-orbital advancement is the widely accepted surgical approach to correct cranial shape abnormalities in patients with metopic craniosynostosis, but the outcome of the surgery remains very dependent on the expertise of the surgeon because of the lack of objective and personalized cranial shape metrics to target during the intervention. We propose in this paper a locally affine diffeomorphic surface registration framework to create an optimal interventional plan personalized to each patient. Our method calculates the optimal surgical plan by minimizing cranial shape abnormalities, which are quantified using objective metrics based on a normative model of cranial shapes built from 198 healthy cases. It is guided by clinical osteotomy templates for fronto-orbital advancement, and it automatically calculates how much and in which direction each bone piece needs to be translated, rotated, and/or bent. Our locally affine framework models separately the transformation of each bone piece while ensuring the consistency of the global transformation. We used our method to calculate the optimal surgical plan for 23 patients, obtaining a significant reduction of malformations (p < 0.001) between 40.38% and 50.85% in the simulated outcome of the surgery using different osteotomy templates. In addition, malformation values were within healthy ranges (p > 0.01).

Although modern ultrasound acquisition systems allow recording of 3D echocardiographic images, tracking anatomical structures from them is still challenging. In addition, since these images are typically created from information obtained across several cardiac cycles, it is not yet possible to acquire high-quality 3D images from patients presenting varying heart rhythms. In this paper, we propose a method to estimate the motion field from multi-plane echocardiographic images of the left ventricle, which are acquired simultaneously during a single cardiac cycle. The method integrates tri-plane B-mode and tissue Doppler images acquired at different rotation angles around the long axis of the left ventricle. It uses a diffeomorphic continuous spatio-temporal transformation model with a spherical data representation for a better interpolation in the circumferential direction. This framework allows exploiting the spatial relation among the acquired planes. In addition, higher temporal resolution of the transformation in the beam direction is achieved by uncoupling the estimation of the different components of the velocity field. The method was validated using a realistic synthetic dataset including healthy and ischemic cases, obtaining errors of 0.14 ± 0.09 mm for displacements, 0.96 ± 1.03% for longitudinal strain and 3.94 ± 4.38% for radial strain estimation. In addition, the method was also demonstrated on a healthy volunteer and two patients with ischemia.

We propose a technique for myocardial motion estimation based on image registration using both B-mode echocardiographic images and tissue Doppler sequences acquired interleaved. The velocity field is modeled continuously using B-splines and the spatiotemporal transform is constrained to be diffeomorphic. Images before scan conversion are used to improve the accuracy of the estimation. The similarity measure includes a model of the speckle pattern distribution of B-mode images. It also penalizes the disagreement between tissue Doppler velocities and the estimated velocity field. Registration accuracy is evaluated and compared to other alternatives using a realistic synthetic dataset, obtaining mean displacement errors of about 1 mm. Finally, the method is demonstrated on data acquired from six volunteers, both at rest and during exercise. Robustness is tested against low image quality and fast heart rates during exercise. Results show that our method provides a robust motion estimate in these situations.

This paper evaluates five 3D ultrasound tracking algorithms regarding their ability to quantify abnormal deformation in timing or amplitude. A synthetic database of B-mode image sequences modeling healthy, ischemic and dyssynchrony cases was generated for that purpose. This database is made publicly available to the community. It combines recent advances in electromechanical and ultrasound modeling. For modeling heart mechanics, the Bestel-Clement-Sorine electromechanical model was applied to a realistic geometry. For ultrasound modeling, we applied a fast simulation technique to produce realistic images on a set of scatterers moving according to the electromechanical simulation result. Tracking and strain accuracies were computed and compared for all evaluated algorithms. For tracking, all methods were estimating myocardial displacements with an error below 1 mm on the ischemic sequences. The introduction of a dilated geometry was found to have a significant impact on accuracy. Regarding strain, all methods were able to recover timing differences between segments, as well as low strain values. On all cases, radial strain was found to have a low accuracy in comparison to longitudinal and circumferential components.