Stress is an inherent aspect of daily life and the use of consumer wearables for stress monitoring has grown significantly. However, existing stress monitoring technologies frequently detect physiological arousal triggered by routine demands, but face challenges in distinguishing between negative stress, such as threat, and positive stress, such as challenge. This differentiation is crucial in minimizing false alarms in stress management notifications and enabling personalized interventions specifically linked to negative stress arousal. To tackle this problem, we leverage advanced biomarkers by detecting ballistocardiogram (BCG) responses through earbud motion sensors and integrating them with biomarkers derived from earbud photoplethysmography (PPG) sensors. A meticulously designed study was conducted to elicit both threat and challenge responses in a controlled laboratory environment. Participants wore custom-designed earbuds for synchronized PPG and accelerometer (BCG) data collection, along with a reference biosig-nal acquisition system. We developed a biomarker extraction pipeline utilizing state-of-the-art BCG and PPG signal processing algorithms to filter low-quality signals and accurately extract advanced physiological biomarkers that capture information related to both challenge and threat responses. Our findings reveal that algorithms applied to earbud data can effectively extract stress biomarkers for arousal detection and differentiate between threat and challenge arousal, achieving an F1-score of up to 81%.

The earbud-based ballistocardiogram (BCG) assessment holds significant promise for monitoring diverse physiological signals, including stress, cardiac activity, and blood pressure. However, unlike traditional methods that measure the force component along the head-to-foot axis for enhanced BCG signal quality, ear-worn devices are prone to orientation misalignment, leading to significant variations in BCG morphology. To address this challenge, we propose two novel algorithms: one that employs sensor-to-body-segment calibration and another that applies a calibration-free, physiologically informed axis fusion method to enhance earbud-based BCG signal assessment. We evaluate the performance of these approaches against existing methods, focusing on heart rate variability (HRV) estimation and morphological feature extraction. Through a comprehensive investigation, we aim to identify optimal strategies for obtaining high-quality BCG signals using ear-worn devices.

This study evaluates the usability of wearable head ballistocardiography (BCG) in providing accurate pulse transit time (PTT) measurements during blood pressure (BP) interventions. Head BCG is a new technique enabling measurement of proximal aortic blood ejection from sensors placed at distal sites, which envisions PTT measurement from single integrated device for cuff-less BP estimation. However, due to its low signal-to-noise ratio and sensitivity to motion artifacts, accurate beat selection is crucial to ensure the integrity of PTT calculation. In this paper, using inertial measurement unit (IMU) sensors integrated in a prototype earbud, we investigate whether the wearable head BCG signal is aligned with the ground-truth proximal reference acquired from the synchronously-recorded impedance cardiography (ICG) signal, to assess the usability of head BCG as the proximal indicator for PTT measurement at different stages of BP intervention. Wearable BCG signals showed highest reliability during resting states, with 63% of detected j-peaks aligned with ground-truth ICG signals. Beat selection via removal of IBI outliers improves the ratio of reliable peaks, at rest (68%) and during exercise (63% at intervention and 52% at plateau). Other methods, such as template matching or rejecting amplitude outliers, only improve the ratio at rest. Overall, this study reveals characteristics of distortions in the head BCG signal during intervention, as a first step toward robust solutions for PTT-based BP tracking on integrated wearable devices.

Camera-based respiratory monitoring is contactless, non-invasive, unobtrusive, and easily accessible compared to conventional wearable devices. This paper presents a novel multimodal approach to estimating breathing rate based on tracking the movement and color changes of the face through an RGB camera. A machine learning model determines the final breathing rate between two separately calculated ones from breathing motion and remote photoplethysmography (rPPG) to improve the measurement performance in a broader range of breathing frequencies. Our proposed pipeline is evaluated with 140 facial video recordings from 22 healthy subjects, including 6 controlled and 2 spontaneous breathing tasks ranging from 5 to 30 BPM. The estimation accuracy achieves 1.33 BPM mean absolute error and 86.53% pass rate within 2 BPM error criteria. To the best of our knowledge, our approach outperforms previous works that use a face region alone with a single RGB camera.

Stress can potentially have detrimental effects on both physical and mental well-being, but monitoring it can be challenging, especially in free-living conditions. One approach to address this challenge is to use earbud accelerometers to capture the ballistocardiogram (BCG) response. These sensors allow for noninvasive stress monitoring by estimating physiological indicators linked to stress, such as heart rate variability (HRV). However, ear-worn devices are susceptible to motion artifacts and can exhibit significant BCG signal morphology variations. These challenges necessitate accurate algorithms to estimate HRV for everyday use. Therefore, we developed a method to measure interbeat intervals (IBI) from BCG signals collected from an earbud. To enhance IBI estimation accuracy, we employed a Bayesian method that incorporates robust apriori IBI prediction weighting and sensor fusion techniques. We have also conducted a study involving 97 participants to assess the earbuds' ability to estimate HRV metrics and classify stressful activities. Our findings demonstrate low IBI estimation error (4.16% ± 1.90%), along with lower errors in subsequent higher-order HRV metrics compared to the state-of-the-art algorithms.

Core body temperature (CBT) is one of the critical yet under-explored phenomena in the context of stress detection. Several CBT measurement methods exist, but they are often limited in continuous CBT monitoring. Furthermore, how continuous CBT can be used to model acute stress is little explored. We address these challenges by conducting an in-lab controlled study with 97 participants who participated in baseline and stress-inducing tasks while wearing prototype earbuds capable of collecting CBT. We found that accounting for changes from individual baselines in CBT results is acute stress detection with 94.88% accuracy and 94.4% F1-score, which is 29.31% and 26.07% higher in terms of accuracy and F1-score, respectively, compared to generalized features.

Consumer hearable technologies such as earbuds are increasingly embedding physiological sensors, including photoplethysmography (PPG) and inertial measurements. They create unique opportunities to passively monitor stress and deliver digital interventions such as music. However, PPG signals recorded from ear canals are often very noisy due to head movement and fit issues. This work proposes algorithms to estimate heart rate variability (HRV) features from noisy PPG signals recorded using earbuds. We have used template matching to determine the signal quality for dynamic fine filtering around the estimated heart rate. We have also improved the inter-beat interval (IBI) outlier detection and removal algorithm using the global-local context of the input PPG signal. The mean absolute error of estimating RMSSD decreased from 70.83 milliseconds (ms) to 24.88 ms, and SDNN decreased from 46.89 ms to 16.60 ms.

The accurate estimation of peripheral capillary oxygen saturation (SpO2) is vital for monitoring respiratory health, with applications spanning medical diagnostics and fitness tracking. Remote photoplethysmography (rPPG) offers a convenient and non-contact approach for SpO2estimation. However, existing methods predominantly rely on data within the normal SpO2range, hindering their effectiveness during hypoxemia. Moreover, cross-user variations poses significant challenges for practicality. To address these limitations, we propose a simple yet effective normalization-based SpO2estimation algorithm. By aligning individual Ratio-of-Ratios (RoR) data with a standard model at the matching SpO2level, we mitigate cross-user variation, accommodate different camera configurations, and account for lighting changes. Our experiments demonstrate that the proposed method achieves an rMSE of 2.8% with leave-one-subject-out cross-validation across the full SpO2range (70%-100%), significantly outperforming existing RoR-based and CNN-based SpO2estimation approaches. Notably, our methods excel in accurately identifying hypoxemia, a critical clinical requirement. We anticipate broader applicability of our approach in rPPG-based vital sign monitoring, underlining the potential for enhancing robustness and reliability in various domains.

Camera-based pulse measurements from remote photoplethysmography (rPPG) have rapidly improved over recent years due to innovations in video processing and deep learning. However, modern data-driven solutions require large training datasets collected under diverse conditions. Collecting such training data is made more challenging by the need for time-synchronized video and physiological signals as ground truth. This paper presents a weakly supervised learning framework, Freq2Time, to train with heart rate (HR) labels. Our framework mitigates the need for simultaneous PPG or ECG as ground truth, since the HR changes relatively slowly and describes the target rPPG signal over a time interval. We show that 3D convolutional neural network (3DCNN) models trained with the Freq2Time framework give state-of-the-art HR performance with MAE of 2.86 bpm, when tested with challenging smartphone video data from 30 subjects. Additionally, our models still learn accurate rPPG time signals, allowing for other physiological metrics such as heart rate variability.

The emergence of remote vital sign measurement techniques has provided an alternative approach for monitoring vital signs without direct physical contact. However, the performance of contactless methods such as remote photoplethysmography are dependent on the accuracy of face detection algorithms. The misalignment of the face pixels from frame to frame can introduce jitters in the generated rPPG signals, and in turn, interfere with vital sign estimations. Nonetheless, the investigation into the performance of face detectors mostly focused on quantifying the accuracy of face landmarks based on an individual image. How to assess facial alignments across video frames has largely remained unknown and understudied. To address this issue, this paper introduced three novel metrics for assessing face alignment in remote vital sign detection: (1) Circular Radius, (2) Mean Offset, and (3) Percentage of Impacted Pixels. We evaluated two face detectors using proposed metrics in static and motion scenarios, where static represents no facial movement and motion scenarios involve facial movements induced by breathing. Our experiments demonstrated that employing the superior face detector recommended by our metrics resulted in a noteworthy 12.5% reduction in second-level mean absolute error and a corresponding 3.0% improvement in 5%-accuracy for remote heart rate estimation.

This paper presents algorithms to improve the estimation of heart rate (HR) and heart rate variability (HRV) from smartphone video. The remote photoplethysmogram (rPPG) signals are first extracted from the videos recorded. Next, we proposed an rPPG filter adaptively tuned by HR and respiratory rate (RR) to better enhance source signal that modulates HR. Additionally, we also addressed a unique smartphone artifact—occasionally seen in smartphone videos—by introducing a threshold-based algorithm. HR and HRV accuracies are assessed on 22 subjects who were instructed to breath at seven different RRs. The mean absolute errors of HR and standard deviation of the NN intervals (SDNN) are found to be 1.13 ± 0.68 bpm and 18.30 ± 10.33 ms respectively. Finally, we also conduct a few experiments to highlight the accuracy improvements made by the proposed algorithms.

With the popularity of non-invasive vital signs detection, remote photoplethysmography (rPPG) is drawing attention in the community. Remote PPG or rPPG signals are extracted in a contactless manner that is more prone to artifacts than PPG signals collected by wearable sensors. To develop a robust and accurate pipeline to estimate heart rate (HR) from rPPG signals, we propose a novel real-time dynamic ROI tracking algorithm that applies to slight motions and light changes. Furthermore, we develop and include a signal quality index (SQI) to improve the HR estimation accuracy. Studies have explored optimal SQIs for PPG signals, but not for remote PPG signals. In this paper, we select and test six SQIs: Perfusion, Kurtosis, Skewness, Zero-crossing, Entropy, and signal-to-noise ratio (SNR) on 124 rPPG sessions from 30 participants wearing masks. Based on the mean absolute error (MAE) of HR estimation, the optimal SQI is selected and validated by Mann–Whitney U test (MWU). Lastly, we show that the HR estimation accuracy is improved by 29% after removing outliers decided by the optimal SQI, and the best result achieves the MAE of 2.308 bpm.

Contactless measurement of vitals has been seen as a promising alternative to contact sensors for monitoring of health condition. In this paper, we focus on respiration rate (RR) as one of the fundamental biomarkers of a person’s cardio and pulmonary activities. Remote RR estimation has gained attraction due to its various potential applications; use of RGB cameras to extract remote photoplethysmography (PPG) signal from subjects’ face has been debated as one of the enabling technologies for remote RR estimation. The technology is challenged with respect to wide range of RR and non-voluntary motion in uncontrolled settings. We propose a novel methodology to enhance the remote PPG signal and remove artifacts from the respiration signal. The method achieves 3.9bpm MAE of 90% percentile (1.3bpm decrease) for estimating RR in range of 5-25bpm. We validate the performance using smartphone video recordings of 30 subjects with uniform distribution of skin tone.

We describe a novel computational framework for contactless oxygen saturation (SpO2) detection using videos recorded from human faces using smartphone cameras with ambient light. For contact pulse oximeter, a ratio of ratios (RoR) metric derived from selected regions of interest (ROI) combined with linear regression modeling is the standard approach. However, when used upon contactless remote PPG (rPPG), the assumptions of this standard approach do not hold automatically: 1) the rPPG signal is usually derived from the face area where the light reflection may not be uniform due to variation in skin tissue composition and/or lighting conditions (moles, hairs, beard, partial shadowing, etc.), 2) for most consumer-level cameras under ambient light, the rPPG signal is converted from light reflection associated with wide-band spectra, which creates complicated nonlinearity for SpO2mappings. We propose a computational framework to overcome these challenges by 1) determining and dynamically tracking the ROIs according to both spatial and color proximity, and calculating the RoR based on selected individual ROIs which have homogeneous skin reflections, and 2) using a nonlinear machine learning model to mapping the SpO2levels from RoRs derived from two different color combinations. We validated the framework with 30 healthy participants during various breathing tasks and achieved 1.24% Root Mean Square Error for across-subjects model and 1.06% for within-subject models, which surpassed the FDA-recognized ISO 81060-2-61:2017 standard.

Atrial Fibrillation (AF) is an important cardiac rhythm disorder, which if left untreated can lead to serious complications such as a stroke. AF can remain asymptomatic, and it can progressively worsen over time; it is thus a disorder that would benefit from detection and continuous monitoring with a wearable sensor. We develop an AF detection algorithm, deploy it on a smartwatch, and prospectively and comprehensively validate its performance on a real-world population that included patients diagnosed with AF. The algorithm showed a sensitivity of 87.8% and a specificity of 97.4% over every 5-minute segment of PPG evaluated. Furthermore, we introduce novel algorithm blocks and system designs to increase the time of coverage and monitor for AF even during periods of motion noise and other artifacts that would be encountered in daily-living scenarios. An average of 67.8% of the entire duration the patients wore the smartwatch produced a valid decision. Finally, we present the ability of our algorithm to function throughout the day and estimate the AF burden, a first-of-this-kind measure using a wearable sensor, showing 98% correlation with the ground truth and an average error of 6.2%.

Atrial Fibrillation (AF) is an important medical condition that can be passively detected and tracked using a smartwatch. Diagnosis and monitoring of AF can be more effective and reliable if the smartwatch senses continuously, but this can lead to significant battery consumption by the LED in the photoplethysmography (PPG) sensor. In this paper, we explore the feasibility of leveraging downsampling to achieve energy-efficient AF detection. We collect data from participants with paroxysmal AF in real ambulatory free-living environments using a commercial smartwatch and separately study the impact of uniform downsampling and compressed sensing on AF detection. Our results reveal that downsampling enables the AF detection system to consume about 77.4% less LED power than the original sampling strategy without a significant performance drop.

A long standing goal of neuroscience studies has been to understand how brain functions are related to behavior. In this paper, we investigate changes in brain functional networks under two behavioral conditions (lick (L) and no-lick (NL)) across two frequency bands. Cortical activity in Thy1-GCaMP6s transgenic calcium reporter mice is recorded during L/NL activity experiments using widefield calcium imaging. We demonstrate how cortical connectivity can be used to identify behavior-related brain states. Connectivity links that significantly contribute to the network difference of L and NL behavioral conditions are spatially localized in two frequency bands. The effectiveness of cortical networks in predicting L and NL behavior are assessed using commonly-used classifiers. Results demonstrate that frequency-dependent cortical network analysis can be utilized to decode the brain states associated with behavior.

This paper presents a novel curvature-compensation technique for bandgap reference circuits implemented in Silicon-Germanium (SiGe) BiCMOS technology. The technique utilizes the designer's access to both Si-based and SiGe-based p-n junctions. Temperature compensation is achieved in two steps: first, by weighted subtraction of two Complementary to Absolute Temperature (CTAT) currents, one proportional to the base-emitter junction of a Si BJT, and the other proportional to that of SiGe HBTs, the non-linear temperature dependent terms are compensated; and second, by adding a Proportional to Absolute Temperature (PTAT) current, the remaining linear temperature dependent terms are canceled. As a result, an almost complete temperature compensation is achieved. Based on this concept, a circuit is designed and simulated in IBM's SiGe BiCMOS 8HP technology. With a power supply of 2.5 V, simulation results show that the circuit generates an output voltage of 978.5 mV with a temperature coefficient (TC) of 1.0 ppm/°C over the temperature range of -25 °C to 125 °C.

This paper presents a low temperature coefficient BiCMOS voltage reference circuit designed in IBM's 8HP Silicon-Germanium (SiGe) technology platform. A BiCMOS compensation approach by combining the temperature properties of HBTs and CMOS transistors has been employed: the Complementary to Absolute Temperature (CTAT) current is generated by a SiGe HBT, while the Proportional to Absolute Temperature (PTAT) current is generated by MOSFETs operating in the subthreshold region. In addition, by adding a nonlinear component, a higher level of temperature compensation is achieved. Simulation results show that with a power supply of 1.2 V, the circuit generates an output voltage of 0.981 V with a temperature coefficient of 0.6 ppm/°C over the temperature range of -25°C to 125°C.