Camera-based blood pressure (BP) monitoring holds significant promise for non-invasive, continuous health assessment in clinical and daily-care settings. Pulse transit time (PTT), a key physiological indicator correlated with BP, has been widely explored for camera-BP estimation. We propose a new concept that measures the PTT based on the property of skin-tissue depolarization, termed skin-tissue depolarized PTT (SD-PTT). This method extracts PTT from camera-based photoplethysmography (PPG) signals at varying tissue depths based on a single skin site with a single wavelength. By employing time-division multiplexing to sequentially illuminate the skin with 0° and 90° polarized light, SD-PTT captures PPG signals from shallow tissues (via parallel polarization) and deeper tissues (via cross polarization), quantifying the phase delay between these signals, caused by different depolarization properties of skin layers as PTT for BP estimation. Laboratory experiments with cold and heat stimulation protocols (n = 20) reveal significant variations in SD-PTT that are consistent with the regulatory mechanisms of BP (P< 0.01). The SD-PTT is further validated in an intensive care unit scenario (n = 20) by modeling the relationship between SD-PTT and BP using a random forest regressor. Although the model performance is relatively limited under strict Leave-One-Out Cross-Validation (LOOCV), under a mixed modeling approach trained with 50% of personalized data, the method achieves remarkable SD-PTT-BP calibration performance, with mean absolute errors (MAE) of 4.85mmHg for systolic BP, 2.14mmHg for diastolic BP, and 2.85mmHg for mean BP, respectively. These results, achieved through 50% personalized calibration, meet the standards of the American Association for the Advancement of Medical Instrumentation (AAMI). The SD-PTT method offers a novel solution for camera-based monitoring, with strong potential for clinical applications.

The diagnosis of peripheral artery disease (PAD) typically relies on specialized equipment such as ultrasound. The delayed PAD detection of these approaches may lead to amputation and even death. To achieve rapid and ubiquitous PAD screening, we propose a novel concept of camera-based plantar perfusion imaging (CPPI) for PAD diagnosis and severity classification. Specifically, we performed a simulation trial that used an RGB camera to record the plantar video of 20 subjects and a cuff with different pressures applied to the left leg to simulate different degrees of lower limb blockage. We generated the plantar perfusion maps using remote photoplethysmography imaging and proposed a multi-view perfusion (MVP) feature set to represent the perfusion maps for PAD classification. The experimental results show that the Pearson correlation coefficients between MVP and Doppler ultrasound (clinical reference) features were larger than 0.9. MVP feature combined with Support Vector Machine obtains 91.47% accuracy in distinguishing the normal and obstructed states, and 76.48% accuracy in differentiating four different degrees of vascular obstruction. The clinical benchmark demonstrated the potential of CPPI as a rapid, sensitive, and easy-to-use diagnostic tool for PAD, suitable for large-scale screening in home or community settings.

Existing respiratory monitoring techniques primarily focus on respiratory rate measurement, neglecting the potential of using thoracoabdominal patterns of respiration for infant lung health assessment. To bridge this gap, we exploit the unique advantage of spatial redundancy of a camera sensor to analyze the infant thoracoabdominal respiratory motion. Specifically, we propose a camera-based respiratory imaging (CRI) system that utilizes optical flow to construct a spatio-temporal respiratory imager for comparing the infant chest and abdominal respiratory motion, and employs deep learning algorithms to identify infant abdominal, thoracoabdominal synchronous, and thoracoabdominal asynchronous patterns of respiration. To alleviate the challenges posed by limited clinical training data and subject variability, we introduce a novel multiple-expert contrastive learning (MECL) strategy to CRI. It enriches training samples by reversing and pairing different-class data, and promotes the representation consistency of same-class data through multi-expert collaborative optimization. Clinical validation involving 44 infants shows that MECL achieves 70% in sensitivity and 80.21% in specificity, which validates the feasibility of CRI for respiratory pattern recognition. This work investigates a novel video-based approach for assessing the infant thoracoabdominal patterns of respiration, revealing a new value stream of video health monitoring in neonatal care.

In the context of the global damage caused by coronavirus disease 2019 (COVID-19) and the emergence of the monkeypox virus (MPXV) outbreak as a public health emergency of international concern, research into methods that can rapidly test potential therapeutics during an outbreak of a new infectious disease is urgently needed. Computational drug discovery is an effective way to solve such problems. The existence of various large open databases has mitigated the time and resource consumption of traditional drug development and improved the speed of drug discovery. However, the diversity of cell lines used in various databases remains limited, and previous drug discovery methods are ineffective for cross-cell prediction. In this study, we propose a correlation-dependent connectivity map (CDCM) to achieve cross-cell predictions of drug similarity. The CDCM mainly identifies drug-drug or disease-drug relationships from the perspective of gene networks by exploring the correlation changes between genes and identifying similarities in the effects of drugs or diseases on gene expression. We validated the CDCM on multiple datasets and found that it performed well for drug identification across cell lines. A comparison with the Connectivity Map revealed that our method was more stable and performed better across different cell lines. In the application of the CDCM to COVID-19 and MPXV data, the predictions of potential therapeutic compounds for COVID-19 were consistent with several previous studies, and most of the predicted drugs were found to be experimentally effective against MPXV. This result confirms the practical value of the CDCM. With the ability to predict across cell lines, the CDCM outperforms the Connectivity Map, and it has wider application prospects and a reduced cost of use.

In the Intensive Care Unit (ICU), monitoring a patient’s heart rate variability (HRV) can provide vital information regarding the physiological status and autonomic nervous system, serving as a potential metric for detecting adverse events in ICU. Although camera-based remote photoplethysmography (rPPG) has shown to be effective for heart rate monitoring in clinical environments, it remains insufficient research on utilizing rPPG-derived pulse rate variability (PRV) for monitoring autonomic nervous system (ANS) changes of ICU patients. In addition, the agreement between HRV and PRV is controversial, especially for ICU patients with multiple concurrent diseases. This study aims to validate the accuracy of camera-PRV measurements by analyzing eight HRV/PRV parameters from subjects with different health conditions, further discussing its feasibility as an early-warning tool. The accuracy of rPPG-derived PRV was comparable to that of contact-PPG and has high correlations with Electrocardiogram. Combined with machine learning algorithms, the system was able to accurately differentiate between healthy subjects and ICU patients (accuracy = 0.86, F1 score = 0.83, sensitivity = 0.79, specificity = 0.79). Furthermore, the system demonstrated good reliability in classifying subjects with different health status and ages. However, it could not effectively correspond to the APACHE II score when classifying ICU patients with different severities based on HRV/PRV. Because the APACHE II scoring system evaluates the changes in the patient’s ANS as well as a variety of confounding factors, of which the HRV/PRV parameters represent only a portion. In summary, the camera-PRV system was able to provide the individual physiological tracking and coarse-level modeling of the group health status. It is expected to achieve a more comprehensive health status assessment by integrating more vital signs in the future.

Respiratory spatial patterns describe the distribution and dynamics of lung conditions, and monitoring of their asymmetry or irregularities enables a more comprehensive assessment of respiratory functions. The feasibility of using the camera pixel array sensing combined with machine learning for analyzing respiratory spatial patterns was demonstrated, however, this approach faces challenges in patient generalization due to limited clinical data and individual respiratory variability. Data augmentation methods may address this by synthesizing new data, but they have a risk of destroying the symmetry or regular semantic information of respiratory patterns. To address this, we propose a prototype-driven hard-sample contrastive learning (PHCL) method tailored for camera-based respiratory imaging analysis. It first separates the samples into simple and hard-to-learn samples using prototypes and Gini-index distance measurement. Then it synthesizes a new feature by blending one-class simple samples and other-class hard samples to construct a transition boundary between different classes, so as to broaden the feature distribution. Then it employs contrastive learning to emphasize feature consistency between prototypes and hard same-class samples from different subjects to mitigate individual respiratory variability and refine class boundaries. Extensive experiments were conducted in the neonatal intensive care unit and the thoracic surgery department, where PHCL outperforms image augmentation and advanced feature augmentation methods by 1-10% in both accuracy and F1-score. Our work provides valuable insights into the analysis of asymmetric and irregular respiratory activities.

Conventional physiological monitoring relies on multiple contact-based biomedical sensors, such as electrocardiograms, pulse oximeters, and blood pressure (BP) cuffs, which often necessitate the use of cumbersome sensors (e.g., electrodes, patches, diodes) and extensive wiring. These conventional methods not only impose significant inconvenience on both caregivers and patients but also elevate the risk of patient infections. In this study, we introduce a novel non-contact, multi-parameter vital signs monitoring platform based on the red, green and near Infrared (RG-IR) spectral imaging system, which measures five critical patient parameters in the Intensive Care Unit (ICU), including heart rate (HR), heart rate variability (HRV), respiratory rate (RR), blood oxygen saturation (SpO$_{2}$) and BP. Clinical trials were conducted in the ICUs of two hospitals, involving 30 critically ill patients. The clinical outcomes indicate that our system achieves reasonable performance, with mean absolute errors of 1.89 bpm for HR, 19.04 ms for SDNN (standard deviation of normal-to-normal intervals), 0.99 bpm for RR, 2.75% for SpO$_{2}$ and 5.67-8.98 mmHg for BP, providing timely physiological information and marking a significant step toward contactless ICU monitoring. Additionally, the real-time performance of the core algorithms were validated on low-cost embedded chips, demonstrating the system's capability for edge computation and on-board patient monitoring. The proposed system has entered the registration process of the National Medical Products Administration (NMPA) as a Class II medical device (No. 2024007).

Camera-based respiration monitoring is currently focused on the continuous measurement of respiratory rate, overlooking its potential in lung health assessment. Inspired by auscultation and palpation that use respiratory symmetry to assess the lung rehabilitation of thoracic surgery patients, we exploit the advantage of spatial redundancy of a camera sensor to replicate this clinical routine. In particular, we propose a camera-based respiratory imaging (CRI) system that leverages optical flow and deep learning algorithms to analyze the symmetric/asymmetric patterns of chest respiratory motion, and classify the subject as health, left lesion, or right lesion. To mitigate the issues of sample scarcity and subject variance, we introduce a novel multiple-prototype contrastive model (MPCM) that uses the symmetric respiration hypothesis to generate more training data, and produces multiple deep prototypes to enhance the consistency of deep representation of samples from different subjects. The clinical validation involving 45 subjects demonstrates the feasibility of CRI for lung rehabilitation assessment, where MPCM achieves above 70% in the used evaluation indices (e.g. accuracy, sensitivity, and specificity). This study demonstrates a new value stream in video health monitoring that uses camera-based respiratory imaging for the lung rehabilitation assessment of thoracic patients after surgery.

Heart rate variability (HRV) can be used as an effective indicator for the maturity of the neonatal autonomic nervous system (ANS) and some acute neonatal diseases (e.g., respiratory distress syndrome and late-onset sepsis). Advances in contactless sensing technology have led to the rapid development of camera-based physiological monitoring, but few studies have focused on HRV measurement in the Neonatal Intensive Care Unit. This study aims to evaluate the performance of camera-based neonatal HRV monitoring (camera-HRV) using camera–photoplethysmography (PPG), and to further investigate the feasibility of applying camera-HRV for tracking ANS maturation through quantitative analysis of 12 HRV indices on quiet infants with limited motions. The clinical experimental results show that as the ANS of newborns matures, the trends of camera-HRV indices are consistent with those obtained by electrocardiogram (ECG) and PPG. Furthermore, the performance of camera-HRV in predicting the postmenstrual age of newborns is comparable to that obtained by ECG and PPG, which can predict 63.89% of infants with an accuracy of less than ±2 weeks. In particular, camera-HRV shows excellent performance in the classification of preterm/term infants, with an optimal accuracy of 0.90 and an F1 score of 0.88. The pilot clinical study suggests that camera-HRV can be a feasible tool for tracking the development of neonatal ANS.

The continuous vital signs monitoring allows clinicians to timely assess the physiological conditions of the patient in intensive care unit (ICU). Internet of Things (IoT) in the hospital that integrated various sensors (e.g., cameras) may enable intelligent healthcare. In this article, we proposed a remote patient monitoring system that exploits closed-circuit television (CCTV) cameras in an IoT infrastructure as optical sensors for noncontact physiological measurement, extending its scope from surveillance to warding. The proposed monitoring system implemented the latest camera photoplethysmography algorithms for cardio-respiratory measurement, particularly focused on heart rate (HR) and breathing rate (BR) as fundamental biomarkers for earlier warning of deterioration. A clinical trial involving 25 critically ill patients was carried out in ICU, where the camera focal length (i.e., key impact factor) is thoroughly investigated. The clinical results show that our system achieves a mean absolute error (MAE) of 1.3 bpm for HR and 0.7 brpm for BR in the far-focus mode; an MAE of 1.0 bpm for HR and 1.8 brpm for BR in the near-focus mode, which are in the range of clinical acceptance. The comparison between the two modes suggests that the far-focus is more suitable for monitoring patient’s vital signs in this scenario. The success rate of HR and BR in the far-focus mode is 94.5% (MAE = 2 bpm) and 96.7% (MAE = 3 brpm), respectively. The prototypes show that the increased measurement coverage and convenience of CCTV cameras in an IoT system are useful for ubiquitous patient monitoring in hospital care units.

The infant sleep-wake behavior is an essential indicator of physiological and neurological system maturity, the circadian transition of which is important for evaluating the recovery of preterm infants from inadequate physiological function and cognitive disorders. Recently, camera-based infant sleep-wake monitoring has been investigated, but the challenges of generalization caused by variance in infants and clinical environments are not addressed for this application. In this paper, we conducted a multi-center clinical trial at four hospitals to improve the generalization of camera-based infant sleep-wake monitoring. Using the face videos of 64 term and 39 preterm infants recorded in NICUs, we proposed a novel sleep-wake classification strategy, called consistent deep representation constraint (CDRC), that forces the convolutional neural network (CNN) to make consistent predictions for the samples from different conditions but with the same label, to address the variances caused by infants and environments. The clinical validation shows that by using CDRC, all CNN backbones obtain over 85% accuracy, sensitivity, and specificity in both the cross-age and cross-environment experiments, improving the ones without CDRC by almost 15% in all metrics. This demonstrates that by improving the consistency of the deep representation of samples with the same state, we can significantly improve the generalization of infant sleep-wake classification.

Heart rate variability (HRV) is a crucial metric that quantifies the variation between consecutive heartbeats, serving as a significant indicator of autonomic nervous system (ANS) activity. It has found widespread applications in clinical diagnosis, treatment, and prevention of cardiovascular diseases. In this study, we proposed an optical model for defocused speckle imaging, to simultaneously incorporate out-of-plane translation and rotation-induced motion for highly-sensitive non-contact seismocardiogram (SCG) measurement. Using electrocardiogram (ECG) signals as the gold standard, we evaluated the performance of photoplethysmogram (PPG) signals and speckle-based SCG signals in assessing HRV. The results indicated that the HRV parameters measured from SCG signals extracted from laser speckle videos showed higher consistency with the results obtained from the ECG signals compared to PPG signals. Additionally, we confirmed that even when clothing obstructed the measurement site, the efficacy of SCG signals extracted from the motion of laser speckle patterns persisted in assessing the HRV levels. This demonstrates the robustness of camera-based non-contact SCG in monitoring HRV, highlighting its potential as a reliable, non-contact alternative to traditional contact-PPG sensors.

Living-skin detection is an important step for imaging photoplethysmography and biometric anti-spoofing. In this paper, we propose a new approach that exploits spatio-temporal characteristics of structured light patterns projected on the skin surface for living-skin detection. We observed that due to the interactions between laser photons and tissues inside a multi-layer skin structure, the frequency-domain sharpness feature of laser spots on skin and non-skin surfaces exhibits clear difference. Additionally, the subtle physiological motion of living-skin causes laser interference, leading to brightness fluctuations of laser spots projected on the skin surface. Based on these two observations, we designed a new living-skin detection algorithm to distinguish skin from non-skin using spatio-temporal features of structured laser spots. Experiments in the dark chamber and Neonatal Intensive Care Unit (NICU) demonstrated that the proposed setup and method performed well, achieving a precision of 85.32%, recall of 83.87%, and F1-score of 83.03% averaged over these two scenes. Compared to the approach that only leverages the property of multilayer skin structure, the hybrid approach obtains an averaged improvement of 8.18% in precision, 3.93% in recall, and 8.64% in F1-score. These results validate the efficacy of using frequency domain sharpness and brightness fluctuations to augment the features of living-skin tissues irradiated by structured light, providing a solid basis for structured light based physiological imaging.

Blood pressure (BP) monitoring has become a daily necessity for well-being management. Camera-based BP monitoring has attracted much attention due to its comfort and convenience. It extracts photoplethysmographic (PPG) signals from the facial skin remotely and calculates morphological features for BP estimation. Although green light with strong pulsatility is commonly used, its penetration depth in skin tissues is shorter than that of the near infrared (NIR) light, indicating that the NIR light may contain more distinct PPG features (i.e., dicrotic wave) that could be more beneficial for BP estimation. In this study, we investigated the wavelength-dependency of the camera-PPG waveform for BP measurement. In particular, two morphological features (K-value and Augmentation Index) from the narrow-band green (550nm), red (660nm), and NIR (850nm) channels were measured and compared for BP calibration. Experimental data were collected from 20 healthy adult subjects using the ice water stimulation protocol. The results show that the camera-PPG waveform features have clear wavelength-dependency. The NIR channel that has deeper skin penetrability (than the green channel) but higher pulsatility (than the red channel) gives more descriptive morphological features related to BP. The conclusions drawn from this study inspire the selection of wavelength for PPG morphologv-based BP measurement.

Infants need adequate sleep to develop their brain and cardiovascular systems, especially for preterm infants in the Neonatal Intensive Care Unit. Camera-based infant monitoring is an emerging direction of research in video health monitoring. Intuitively, camera can easily identify the sleep-awake stage of infants by detecting the state of eyes, e.g. closed or opening. Thus in this paper, we propose to explore the unique advantage of camera-based facial analysis for sleep-awake detection, as a fundamental step toward infant sleep monitoring. A multi-center clinical trial was conducted to collect infant videos for investigating the feasibility of our proposal. A benchmark including four machine learning methods of SVM, KNN, MLP, and CNN (ResNet18) was set up to classify the sleep/awake stage of infants. To alleviate the overfitting issue caused by over-sampling of a sleeping infant, we propose to integrate ResNet18 with the contrastive learning strategy to strengthen the consistency of facial features learned from different infants. The clinical evaluation shows that all benchmarked methods obtained an accuracy above 75% while the proposed method achieved the best accuracy of 86%. This invokes further explorations of using facial/eye features of infants for sleep-awake staging, towards intelligent contactless sleep analysis of infants in combination with camera-based vital signs monitoring.

Advancements in video health monitoring opened up new opportunities for contactless infant monitoring in the Neonatal Intensive Care Unit (NICU). Several studies have demonstrated the potential of heart rate variability (HRV) as an early warning measure for acute illnesses, such as late-onset sepsis and bradycardia. Conventional HRV analysis typically relies on electrocardiography (ECG) or contact photoplethysmography (contact-PPG) that may damage infants’ fragile skin. This study aims to investigate the feasibility of camera-PPG based HRV measurement for preterm infants in NICU, i.e. an important vital sign that was less studied in this scenario. A clinical study was conducted in two hospitals involving 41 infants, focused on comparing the HRV parameters obtained from camera-PPG and contact-PPG with the reference of ECG. Results show that camera-HRV has similar performance as contact-HRV in cases without motion distortions, and it has smaller errors and better robustness in the presence of motion distortions. Strong correlations can be observed between camera-PPG, contactPPG and ECG in several HRV parameters (Mean IBI, SDNN, LF, VLF, and SD2), though the performance on short-term variability parameters need to be improved further. The findings suggest that camera-PPG could be a promising tool for motionrobust HRV estimation in NICU, offering a potential surrogate for contact-PPG in early-warning of deteriorations of preterm infants.

The monitoring of vital signs such as heart rate (HR) and respiratory rate (RR) during sleep is important for the assessment of sleep quality and detection of sleep disorders. Camera-based HR and RR monitoring gained popularity in sleep monitoring in recent years. However, they are all facing with serious privacy issues when using a video camera in the sleeping scenario. In this paper, we propose to use the defocused camera to measure vital signs from optically blurred images, which can fundamentally eliminate the privacy invasion as face is difficult to be identified in obtained blurry images. A spatial-redundant framework involving living-skin detection is used to extract HR and RR from the defocused camera in NIR, and a motion metric is designed to exclude outliers caused by body motions. In the benchmark, the overall Mean Absolute Error (MAE) for HR measurement is 4.4 bpm, for RR measurement is 5.9 bpm. Both have quality drops as compared to the measurement using a focused camera, but the degradation in HR is much less, i.e. HR measurement has strong correlation with the reference (R ≥ 0.90). Preliminary experiments suggest that it is feasible to use a defocused camera for cardio-respiratory monitoring while protecting the privacy. Further improvement is needed for robust RR measurement, such as by PPG-modulation based RR extraction.

The monitoring of hand hygiene activities can effectively reduce infection and contamination in the Intensive Care Unit (ICU). In this paper, we created a clinical dataset using CCTV cameras installed in ICU to explore the feasibility of recognizing the hand-washing steps of clinicians. A video processing architecture including hand landmark detection and classification is presented. Such a system can potentially be used to alarm clinicians to follow the guidelines of hand hygiene. The experimental results show that the average accuracy of our methodology can achieve 95% under the personalized model and 56% under the generalized model. The preliminary results suggest that hand hygiene is subject-dependent, which is related to individuals’ palm size and washing habits. The cross-subject modeling or subject-adaptive learning can be applied to further improve the accuracy of recognition towards a more generalized solution. The insights of the study are helpful for designing a hand hygiene scoring and alarming system as a part of hospital IoT. The hospital data and code are available at https://github.com/SunnySideUp11/Hand-Hygiene-ICU.

Lung sound analysis has been used for assessing the lung conditions of children with community-acquired pneumonia (CAP). However, the inconsistent data distribution, caused by variant CAP symptoms appearing at different ages of children, limits the generalization ability of most diagnostic models. The data scarcity will further exacerbate this problem. Therefore, we propose a contrastive embedding-based domain adaptation network (CEDANN) to eliminate individual differences and alleviate data scarcity for improving the generalization ability. It forces the embedding layer to learn the subject-independent but task-dependent features by the adversarial learning between the domain classifier and the task classifier. Multiple contrast input tuples are constructed by combining samples from different classes to increase input combinations to alleviate data scarcity. The proposed method is evaluated on a multi-center clinical dataset. The subject-independent experiments show that CEDANN improves the sensitivity from 45.93% to 59.06% and the specificity from 35.07% to 59.55% in identifying CAP-confirmed, symptomatic relief, and recovery children. It demonstrates the effectiveness of CEDANN in the diagnosis and prognosis of children CAP.

Remote photoplethysmography (rPPG) can be used to monitor driver’s cardio-respiratory functions in automotive for improving the safety of driving. To understand the challenges of rPPG in this application, we created a benchmark of latest rPPG algorithms based on the MR-NIRP Car dataset, selecting the representative methods from both the physiological model based (PBV and DIS) and deep learning based (Supervised Learning and Contrastive Learning) approaches. The experimental results show that the physiological model based methods are generally more robust in this challenging scenario with vigorous motions and dynamic lighting changes, typically DIS outperforms others, with an average MAE of 6.5 bpm on RGB videos and 15.9 bpm on NIR videos. The benchmark indicates that upgrading the single wavelength NIR setup to multi-wavelength is the essential step towards robust heart-rate monitoring in automotive.