Existing respiratory monitoring techniques primarily focus on respiratory rate measurement, neglecting the potential of using thoracoabdominal patterns of respiration for infant lung health assessment. To bridge this gap, we exploit the unique advantage of spatial redundancy of a camera sensor to analyze the infant thoracoabdominal respiratory motion. Specifically, we propose a camera-based respiratory imaging (CRI) system that utilizes optical flow to construct a spatio-temporal respiratory imager for comparing the infant chest and abdominal respiratory motion, and employs deep learning algorithms to identify infant abdominal, thoracoabdominal synchronous, and thoracoabdominal asynchronous patterns of respiration. To alleviate the challenges posed by limited clinical training data and subject variability, we introduce a novel multiple-expert contrastive learning (MECL) strategy to CRI. It enriches training samples by reversing and pairing different-class data, and promotes the representation consistency of same-class data through multi-expert collaborative optimization. Clinical validation involving 44 infants shows that MECL achieves 70% in sensitivity and 80.21% in specificity, which validates the feasibility of CRI for respiratory pattern recognition. This work investigates a novel video-based approach for assessing the infant thoracoabdominal patterns of respiration, revealing a new value stream of video health monitoring in neonatal care.

Compared to broader physical activities, detecting nuanced respiratory movements poses a significant challenge in indoor health monitoring systems. While respiratory activity can be conceptualized as periodic chest movements akin to mechanical vibrations, uncontrollable environmental factors often introduce noise into detected radar signals. The clutter in the field of view, especially metallic objects, such as hospital steel beds, degrades the performance of radar physiological monitoring: 1) amplifying noise of multipath effects and 2) misleading the informative localization module. In this article, we propose a preprocessing scheme of Search-Voxel Ellipse Normalization for respiratory detection system, including an ellipse normalization method combined with the fitting-cost voxel selection policy, to improve the respiration detection performance using MIMO frequency modulated continuous wave radar. This article provides an in-depth assessment of the designed system, including a metal-insulated room test, involving ten participants in different postures. The results show notable performance improvements of our proposed ENDTW-MVMD method, especially in lowering the mean absolute error from the best state-of-the-art 0.93–0.75 bpm and stabilization in voxel selection. The proposed approach is thoroughly evaluated against established methods across various dimensions, such as voxel selection, independent performance, frequency estimation, and ablation studies.

Blood pressure (BP) is a vital physiological parameter for early warning and prompt intervention treatment in the neonatal intensive care unit (NICU). However, the application of contactless BP measurement methods in neonates remains under-explored. This proof-of-concept clinical study proposes using multi-site and multi-wavelength pulse transit time (PTT) generated from remote-Photoplethysmography (rPPG) for neonatal BP estimation. A dataset of 40 neonates was created in the NICU under three alternating phases (resting -BP measurement -resting). The spatially averaged rPPG signals from five body parts were used to calculate multiple PTT features, including multi-site PTT (MS-PTT) derived from different body parts and multi-wavelength PTT (MW-PTT) derived from different skin layers, for BP estimation. Three machine learning models, including Multivariate Linear Regression (MLR), Support Vector Regression (SVR), and Random Forest Regression (RFR), were employed for both univariate and multivariate regression. Combining MS-PTT and MW-PTT yielded the best results, achieving a mean absolute error ± standard deviation (MAE ± STD) of 7.65 ± 7.48 mmHg for SBP, 6.31 ± 5.58 mmHg for DBP, and 7.29 ± 7.29 mmHg for MBP, based on MLR with subject-dependent modeling. According to the British Hypertension Society guidelines, these results meet the requirements for Grade C. These findings provide the first clinical proof-of-concept of using camera-based MS-PTT and MW-PTT features for contactless neonatal BP estimation.

The monitoring of respiratory rate (RR), heart rate (HR), HR variability (HRV), and blood pressure (BP) during sleep allows for a comprehensive evaluation of sleep quality, facilitating the understanding and improvement of a person’s sleep health. Contactless physiological monitoring using cameras has gained popularity recently due to its convenient, infection-free, continuous, and versatile nature. However, the privacy concerns limit the application of camera-based solutions in sleep monitoring setups. This study proposes a novel hybrid setup that integrates camera-based seismocardiography (CamSCG) and photoplethysmography (CamPPG) for contactless measurement of RR, HR, and HRV during sleep while simultaneously estimating BP. For the proximal SCG, we employed camera-based laser speckle vibrometry to measure cardiac motions from the chest, and benchmarked it with a millimeter-wave radar (RFSCG). For the distal photoplethysmographic (PPG), a defocused camera was utilized to measure pulse signals from the facial skin while protecting privacy. In this setup, we analyzed the single-modality in measuring RR, HR, and HRV, and established two bi-modalities (CamSCG-CamPPG and RFSCG-CamPPG) to measure pulse transit time (PTT) features for BP calibration. The benchmark involving 19 subjects highlights the potential of camera-based bi-modal SCG-PPG for privacy-protected vital signs monitoring during sleep.

Cardiovascular diseases are one of the leading causes of death worldwide. Accurately capturing and analyzing the multidimensional dynamics of cardiac motion is crucial for early diagnosis and rehabilitation assessment. This study introduces a novel concept for non-contact cardiac linear vibration (SCG) and rotational components (GCGx and GCGy) decoupling and reconstruction by integrating speckle motion signals captured from two cameras with different defocus levels. The intention is to overcome the motion coupling issues inherent in single-camera imaging and improve the accuracy in characterizing the cardiac complex 3D mechanical behavior. Using a sternum-mounted inertial sensor as the reference, experiments were conducted on 42 subjects in laboratory and intensive care unit settings. The results show that the reconstructed cardiac 3D motion signals exhibit greater waveform similarity to the reference signal than the raw speckle motion signal from a single camera, with similarity indices above 87.471%. In addition, with an 8 ms tolerance error, the localization accuracy of 6 key biomarkers (aortic valve opening/closing (AO/AC), mitral valve opening/closing (MO/MC), the biomarkers corresponding to the AO event in GCGy and the MC event in GCGx) are 73.080%, 99.998%, 85.587%, 86.617%, 99.683% and 77.301%, respectively. These results also outperform those obtained from the raw speckle motion signal. These findings validate the rationale and effectiveness of using dual-camera imaging with different defocus levels to reconstruct SCG, GCGx, and GCGy, offering a promising approach for accurately capturing complex cardiac 3D motion and improving cardiac function assessment.

Cameras hold great potential in sleep monitoring and are suitable for long-term home use. However, privacy invasion is a major concern of cameras in sleep monitoring. A defocused camera offers a promising privacy-protection solution, its feasibility for physiological signal measurement has been shown but its performance on sleep staging remains unknown. We applied four levels of post-blurring (from sharp to blurred videos) to investigate the sleep staging including wakefulness (Wake), rapid eye movement (REM), light stage (Light), and deep stage (Deep). The results showed that all four levels of blurring effectively provide identity protection, but the sleep staging performance (especially the Light) was degraded by video blurring, particularly due to the distorted motion cues and high-frequency components of heart rate variability (HRV). To this end, we introduced Cardiopulmonary Coupling (CPC) to video-based sleep staging. With CPC, the accuracy (ACC) reached 69.5% (kappa = 0.59, F1-score = 0.67) with privacy protection, while under the most severe blurring, ACC was improved from 59.8% (kappa = 0.49, F1-score = 0.56) to 66.9% (kappa = 0.54, F1-score = 0.65). The main benefit of CPC is that the photoplethysmography (PPG) derived respiration (PDR) signal is more robust than inter-beat interval (IBI) under video blurring. Overall, this study explores the opportunity of using blurred videos for privacy-protected sleep staging and demonstrates the effectiveness of CPC in mitigating the degradation of motion and HRV features caused by video blurring. Cohort diversity and multimodal integration are expected to offer deeper clinical and technical insights for real-world applications.

Camera-based respiration monitoring is currently focused on the continuous measurement of respiratory rate, overlooking its potential in lung health assessment. Inspired by auscultation and palpation that use respiratory symmetry to assess the lung rehabilitation of thoracic surgery patients, we exploit the advantage of spatial redundancy of a camera sensor to replicate this clinical routine. In particular, we propose a camera-based respiratory imaging (CRI) system that leverages optical flow and deep learning algorithms to analyze the symmetric/asymmetric patterns of chest respiratory motion, and classify the subject as health, left lesion, or right lesion. To mitigate the issues of sample scarcity and subject variance, we introduce a novel multiple-prototype contrastive model (MPCM) that uses the symmetric respiration hypothesis to generate more training data, and produces multiple deep prototypes to enhance the consistency of deep representation of samples from different subjects. The clinical validation involving 45 subjects demonstrates the feasibility of CRI for lung rehabilitation assessment, where MPCM achieves above 70% in the used evaluation indices (e.g. accuracy, sensitivity, and specificity). This study demonstrates a new value stream in video health monitoring that uses camera-based respiratory imaging for the lung rehabilitation assessment of thoracic patients after surgery.

Sleep monitoring plays a vital role in various scenarios such as hospitals and living-assisted homes, contributing to the prevention of sleep accidents as well as the assessment of sleep health. Contactless camera-based sleep monitoring is promising due to its user-friendly nature and rich visual semantics. However, the privacy concern of video cameras limits their applications in sleep monitoring. In this paper, we explored the opportunity of using a defocused camera that does not allow identification of the monitored subject when measuring sleep-related parameters, as face detection and recognition are impossible on optically blurred images. We proposed a novel privacy-protected sleep parameters measurement framework, including a physiological measurement branch and a semantic analysis branch based on ResNet-18. Four important sleep parameters are measured: heart rate (HR), respiration rate (RR), sleep posture, and movement. The results of HR, RR, and movement have strong correlations with the reference (HR: R = 0.9076; RR: R = 0.9734; Movement: R = 0.9946). The overall mean absolute errors (MAE) for HR and RR are 5.2 bpm and 1.5 bpm respectively. The measurement of HR and RR achieve reliable estimation coverage of 72.1% and 93.6%, respectively. The sleep posture detection achieves an overall accuracy of 94.5%. Experimental results show that the defocused camera is promising for sleep monitoring as it fundamentally eliminates the privacy issue while still allowing the measurement of multiple parameters that are essential for sleep health informatics.

The monitoring of vital signs such as heart rate (HR) and respiratory rate (RR) during sleep is important for the assessment of sleep quality and detection of sleep disorders. Camera-based HR and RR monitoring gained popularity in sleep monitoring in recent years. However, they are all facing with serious privacy issues when using a video camera in the sleeping scenario. In this paper, we propose to use the defocused camera to measure vital signs from optically blurred images, which can fundamentally eliminate the privacy invasion as face is difficult to be identified in obtained blurry images. A spatial-redundant framework involving living-skin detection is used to extract HR and RR from the defocused camera in NIR, and a motion metric is designed to exclude outliers caused by body motions. In the benchmark, the overall Mean Absolute Error (MAE) for HR measurement is 4.4 bpm, for RR measurement is 5.9 bpm. Both have quality drops as compared to the measurement using a focused camera, but the degradation in HR is much less, i.e. HR measurement has strong correlation with the reference (R ≥ 0.90). Preliminary experiments suggest that it is feasible to use a defocused camera for cardio-respiratory monitoring while protecting the privacy. Further improvement is needed for robust RR measurement, such as by PPG-modulation based RR extraction.