Existing respiratory monitoring techniques primarily focus on respiratory rate measurement, neglecting the potential of using thoracoabdominal patterns of respiration for infant lung health assessment. To bridge this gap, we exploit the unique advantage of spatial redundancy of a camera sensor to analyze the infant thoracoabdominal respiratory motion. Specifically, we propose a camera-based respiratory imaging (CRI) system that utilizes optical flow to construct a spatio-temporal respiratory imager for comparing the infant chest and abdominal respiratory motion, and employs deep learning algorithms to identify infant abdominal, thoracoabdominal synchronous, and thoracoabdominal asynchronous patterns of respiration. To alleviate the challenges posed by limited clinical training data and subject variability, we introduce a novel multiple-expert contrastive learning (MECL) strategy to CRI. It enriches training samples by reversing and pairing different-class data, and promotes the representation consistency of same-class data through multi-expert collaborative optimization. Clinical validation involving 44 infants shows that MECL achieves 70% in sensitivity and 80.21% in specificity, which validates the feasibility of CRI for respiratory pattern recognition. This work investigates a novel video-based approach for assessing the infant thoracoabdominal patterns of respiration, revealing a new value stream of video health monitoring in neonatal care.

Blood pressure (BP) is a vital physiological parameter for early warning and prompt intervention treatment in the neonatal intensive care unit (NICU). However, the application of contactless BP measurement methods in neonates remains under-explored. This proof-of-concept clinical study proposes using multi-site and multi-wavelength pulse transit time (PTT) generated from remote-Photoplethysmography (rPPG) for neonatal BP estimation. A dataset of 40 neonates was created in the NICU under three alternating phases (resting -BP measurement -resting). The spatially averaged rPPG signals from five body parts were used to calculate multiple PTT features, including multi-site PTT (MS-PTT) derived from different body parts and multi-wavelength PTT (MW-PTT) derived from different skin layers, for BP estimation. Three machine learning models, including Multivariate Linear Regression (MLR), Support Vector Regression (SVR), and Random Forest Regression (RFR), were employed for both univariate and multivariate regression. Combining MS-PTT and MW-PTT yielded the best results, achieving a mean absolute error ± standard deviation (MAE ± STD) of 7.65 ± 7.48 mmHg for SBP, 6.31 ± 5.58 mmHg for DBP, and 7.29 ± 7.29 mmHg for MBP, based on MLR with subject-dependent modeling. According to the British Hypertension Society guidelines, these results meet the requirements for Grade C. These findings provide the first clinical proof-of-concept of using camera-based MS-PTT and MW-PTT features for contactless neonatal BP estimation.

Respiratory spatial patterns describe the distribution and dynamics of lung conditions, and monitoring of their asymmetry or irregularities enables a more comprehensive assessment of respiratory functions. The feasibility of using the camera pixel array sensing combined with machine learning for analyzing respiratory spatial patterns was demonstrated, however, this approach faces challenges in patient generalization due to limited clinical data and individual respiratory variability. Data augmentation methods may address this by synthesizing new data, but they have a risk of destroying the symmetry or regular semantic information of respiratory patterns. To address this, we propose a prototype-driven hard-sample contrastive learning (PHCL) method tailored for camera-based respiratory imaging analysis. It first separates the samples into simple and hard-to-learn samples using prototypes and Gini-index distance measurement. Then it synthesizes a new feature by blending one-class simple samples and other-class hard samples to construct a transition boundary between different classes, so as to broaden the feature distribution. Then it employs contrastive learning to emphasize feature consistency between prototypes and hard same-class samples from different subjects to mitigate individual respiratory variability and refine class boundaries. Extensive experiments were conducted in the neonatal intensive care unit and the thoracic surgery department, where PHCL outperforms image augmentation and advanced feature augmentation methods by 1-10% in both accuracy and F1-score. Our work provides valuable insights into the analysis of asymmetric and irregular respiratory activities.

Heart rate variability (HRV) can be used as an effective indicator for the maturity of the neonatal autonomic nervous system (ANS) and some acute neonatal diseases (e.g., respiratory distress syndrome and late-onset sepsis). Advances in contactless sensing technology have led to the rapid development of camera-based physiological monitoring, but few studies have focused on HRV measurement in the Neonatal Intensive Care Unit. This study aims to evaluate the performance of camera-based neonatal HRV monitoring (camera-HRV) using camera–photoplethysmography (PPG), and to further investigate the feasibility of applying camera-HRV for tracking ANS maturation through quantitative analysis of 12 HRV indices on quiet infants with limited motions. The clinical experimental results show that as the ANS of newborns matures, the trends of camera-HRV indices are consistent with those obtained by electrocardiogram (ECG) and PPG. Furthermore, the performance of camera-HRV in predicting the postmenstrual age of newborns is comparable to that obtained by ECG and PPG, which can predict 63.89% of infants with an accuracy of less than ±2 weeks. In particular, camera-HRV shows excellent performance in the classification of preterm/term infants, with an optimal accuracy of 0.90 and an F1 score of 0.88. The pilot clinical study suggests that camera-HRV can be a feasible tool for tracking the development of neonatal ANS.

The infant sleep-wake behavior is an essential indicator of physiological and neurological system maturity, the circadian transition of which is important for evaluating the recovery of preterm infants from inadequate physiological function and cognitive disorders. Recently, camera-based infant sleep-wake monitoring has been investigated, but the challenges of generalization caused by variance in infants and clinical environments are not addressed for this application. In this paper, we conducted a multi-center clinical trial at four hospitals to improve the generalization of camera-based infant sleep-wake monitoring. Using the face videos of 64 term and 39 preterm infants recorded in NICUs, we proposed a novel sleep-wake classification strategy, called consistent deep representation constraint (CDRC), that forces the convolutional neural network (CNN) to make consistent predictions for the samples from different conditions but with the same label, to address the variances caused by infants and environments. The clinical validation shows that by using CDRC, all CNN backbones obtain over 85% accuracy, sensitivity, and specificity in both the cross-age and cross-environment experiments, improving the ones without CDRC by almost 15% in all metrics. This demonstrates that by improving the consistency of the deep representation of samples with the same state, we can significantly improve the generalization of infant sleep-wake classification.

Infants need adequate sleep to develop their brain and cardiovascular systems, especially for preterm infants in the Neonatal Intensive Care Unit. Camera-based infant monitoring is an emerging direction of research in video health monitoring. Intuitively, camera can easily identify the sleep-awake stage of infants by detecting the state of eyes, e.g. closed or opening. Thus in this paper, we propose to explore the unique advantage of camera-based facial analysis for sleep-awake detection, as a fundamental step toward infant sleep monitoring. A multi-center clinical trial was conducted to collect infant videos for investigating the feasibility of our proposal. A benchmark including four machine learning methods of SVM, KNN, MLP, and CNN (ResNet18) was set up to classify the sleep/awake stage of infants. To alleviate the overfitting issue caused by over-sampling of a sleeping infant, we propose to integrate ResNet18 with the contrastive learning strategy to strengthen the consistency of facial features learned from different infants. The clinical evaluation shows that all benchmarked methods obtained an accuracy above 75% while the proposed method achieved the best accuracy of 86%. This invokes further explorations of using facial/eye features of infants for sleep-awake staging, towards intelligent contactless sleep analysis of infants in combination with camera-based vital signs monitoring.

Advancements in video health monitoring opened up new opportunities for contactless infant monitoring in the Neonatal Intensive Care Unit (NICU). Several studies have demonstrated the potential of heart rate variability (HRV) as an early warning measure for acute illnesses, such as late-onset sepsis and bradycardia. Conventional HRV analysis typically relies on electrocardiography (ECG) or contact photoplethysmography (contact-PPG) that may damage infants’ fragile skin. This study aims to investigate the feasibility of camera-PPG based HRV measurement for preterm infants in NICU, i.e. an important vital sign that was less studied in this scenario. A clinical study was conducted in two hospitals involving 41 infants, focused on comparing the HRV parameters obtained from camera-PPG and contact-PPG with the reference of ECG. Results show that camera-HRV has similar performance as contact-HRV in cases without motion distortions, and it has smaller errors and better robustness in the presence of motion distortions. Strong correlations can be observed between camera-PPG, contactPPG and ECG in several HRV parameters (Mean IBI, SDNN, LF, VLF, and SD2), though the performance on short-term variability parameters need to be improved further. The findings suggest that camera-PPG could be a promising tool for motionrobust HRV estimation in NICU, offering a potential surrogate for contact-PPG in early-warning of deteriorations of preterm infants.