Neonatal Endotracheal Intubation (ETI) is a critical resuscitation skill that requires tremendous practice of trainees before clinical exposure. However, current manikin-based training regimen is ineffective in providing satisfactory real-time procedural guidance for accurate assessment due to the lack of see-through visualization within the manikin. The training efficiency is further reduced by the limited availability of expert instructors, which inevitably results in a long learning curve for trainees. To this end, we propose an intelligent Augmented Reality (AR) training framework that provides trainees with a complete visualization of the ETI procedure for real-time guidance and assessment. Specifically, the proposed framework is capable of capturing the motions of the laryngoscope and the manikin and offer 3D see-through visualization rendered to the head-mounted display (HMD). Furthermore, an attention-based Convolutional Neural Network (CNN) model is developed to automatically assess the ETI performance from the captured motions as well as identify regions of motions that significantly contribute to the performance evaluation. Lastly, augmented user-friendly feedback is delivered with interpretable results with the ETI scoring rubric through the color-coded motion trajectory that classifies highlighted regions that need more practice. The classification accuracy of our machine learning model is 84.6%.

Neonatal endotracheal intubation (ETI) is a complex procedure. Low intubation success rates for pediatric residents indicate the current training regimen is inadequate for achieving positive patient out-comes. Computer-based training systems in this field have been limited due to the complex nature of simulating in real-time, the anatomical structures, soft tissue deformations and frequent tool interactions with large forces which occur during actual patient intubation. This paper addresses the issues of neonatal ETI training in an attempt to bridge the gap left by traditional training methods. We propose a fully interactive physics-based virtual reality (VR) simulation framework for neonatal ETI that converts the training of this medical procedure to a completely immersive virtual environment where both visual and physical realism were achieved. Our system embeds independent dynamics models and interaction devices in separate modules while allowing them to interact with each other within the same environment, which offers a flexible solution for multi-modal medical simulation scenarios. The virtual model was extracted from CT scans of a neonatal patient, which provides realistic anatomical structures and was parameterized to allow variations in a range of features that affect the level of difficulty. Moreover, with this manikin-free VR system, we can capture and visualize an even larger set of performance parameters in relation to the internal geometric change of the virtual model for real-time guidance and post-trial assessment. Lastly, validation study results from a group of neonatologists are presented demonstrating that VR is a promising platform to train medical professionals effectively for this procedure.

In this paper, we propose our template-based non-rigid registration algorithm to address the misalignments in the frame-to-frame motion tracking with single or multiple commodity depth cameras. We analyze the deformation in the local coordinates of neighboring nodes and use this differential representation to formulate the regularization term for the deformation field in our non-rigid registration. The local coordinate regularizations vary for each pair of neighboring nodes based on the tracking status of the surface regions. We propose our tracking strategies for different surface regions to minimize misalignments and reduce error accumulation. This method can thus preserve local geometric features and prevent undesirable distortions. Moreover, we introduce a geodesic-based correspondence estimation algorithm to align surfaces with large displacements. Finally, we demonstrate the effectiveness of our proposed method with detailed experiments.

The cover image is based on the Research Article Accurate nonrigid 3D human body surface reconstruction using commodity depth sensors, by Yao Lu et al., https://doi.org/10.1002/cav.1807.

In the last decade, 3D modeling techniques enjoyed a booming development in both hardware and software. High-end hardware generates high fidelity results, but the cost is prohibitive, whereas consumer-level devices generate plausible results for entertainment purposes but are not appropriate for medical uses. We present a cost-effective and easy-to-use 3D body reconstruction system using consumer-grade depth sensors, which provides reconstructed body shapes with a high degree of accuracy and reliability appropriate for medical applications. Our surface registration framework integrates the articulated motion assumption, global loop closure constraint, and a general as-rigid-as-possible deformation model. To enhance the reconstruction quality, we propose a novel approach to accurately infer skeletal joints from anatomical data using multimodality registration. We further propose a supervised predictive model to infer the skeletal joints for arbitrary subjects independent from anatomical data reference. A rigorous validation test has been conducted on real subjects to evaluate the reconstruction accuracy and repeatability. Our system has the potential to make accurate body surface scanning systems readily available for medical professionals and the general public. The system can be used to obtain additional health data derived from 3D body shapes, such as the percentage of body fat.