Neonatal Endotracheal Intubation (ETI) is a critical resuscitation skill that requires tremendous practice of trainees before clinical exposure. However, current manikin-based training regimen is ineffective in providing satisfactory real-time procedural guidance for accurate assessment due to the lack of see-through visualization within the manikin. The training efficiency is further reduced by the limited availability of expert instructors, which inevitably results in a long learning curve for trainees. To this end, we propose an intelligent Augmented Reality (AR) training framework that provides trainees with a complete visualization of the ETI procedure for real-time guidance and assessment. Specifically, the proposed framework is capable of capturing the motions of the laryngoscope and the manikin and offer 3D see-through visualization rendered to the head-mounted display (HMD). Furthermore, an attention-based Convolutional Neural Network (CNN) model is developed to automatically assess the ETI performance from the captured motions as well as identify regions of motions that significantly contribute to the performance evaluation. Lastly, augmented user-friendly feedback is delivered with interpretable results with the ETI scoring rubric through the color-coded motion trajectory that classifies highlighted regions that need more practice. The classification accuracy of our machine learning model is 84.6%.

Neonatal endotracheal intubation (ETI) is a complex procedure. Low intubation success rates for pediatric residents indicate the current training regimen is inadequate for achieving positive patient out-comes. Computer-based training systems in this field have been limited due to the complex nature of simulating in real-time, the anatomical structures, soft tissue deformations and frequent tool interactions with large forces which occur during actual patient intubation. This paper addresses the issues of neonatal ETI training in an attempt to bridge the gap left by traditional training methods. We propose a fully interactive physics-based virtual reality (VR) simulation framework for neonatal ETI that converts the training of this medical procedure to a completely immersive virtual environment where both visual and physical realism were achieved. Our system embeds independent dynamics models and interaction devices in separate modules while allowing them to interact with each other within the same environment, which offers a flexible solution for multi-modal medical simulation scenarios. The virtual model was extracted from CT scans of a neonatal patient, which provides realistic anatomical structures and was parameterized to allow variations in a range of features that affect the level of difficulty. Moreover, with this manikin-free VR system, we can capture and visualize an even larger set of performance parameters in relation to the internal geometric change of the virtual model for real-time guidance and post-trial assessment. Lastly, validation study results from a group of neonatologists are presented demonstrating that VR is a promising platform to train medical professionals effectively for this procedure.