Glenn Fernandes

dblp:207/1479 · DBLP profile ↗
← Back
10ranked-venue papers
2as first author
9since 2021 · last 2025
0000-0001-9070-4594ORCID · corroborated

Domains — the database's venue-derived domains; a paper can count in several

Applied, interdisciplinary, general and emerging computing · 9 · 1 first-author · 8 since 2021Artificial intelligence and machine learning · 1 · 1 first-author · 1 since 2021Graphics, computer vision, multimedia, augmented reality and games · 1 · 1 first-author · 1 since 2021
YearPublicationVenuePosition
2025 Multi-Modal Hand-to-Mouth Gesture Recognition in Activity-Oriented RGB-Thermal Footage (Student Abstract)
abstract
Health-risk behaviors such as overeating and smoking have a profound impact on public health, making their monitoring and mitigation critical. Wearable RGB-Thermal cameras are being employed to monitor these behaviors by capturing hand-to-mouth (HTM) gestures, which are central to them. However, detection models relying on single modalities—either RGB or thermal—often struggle to accurately distinguish these confounding gestures due to inherent sensor limitations, such as sensitivity to lighting conditions or thermal occlusions. We present a family of fusion models that integrate RGB and thermal video data using early-, decision- , and a novel mid-fusion architecture, RGB-Thermal Fusion Video Network (RTFVNet), designed to enhance the recognition of HTM gestures associated with eating and smoking. Our evaluation shows that while decision fusion achieves the highest F1-score of 88% (0.44 TFLOPs), RTFVNet offers an optimal balance between performance (85%) and complexity (0.37 TFLOPs) for gesture classification of eating, smoking, and non-gesture activities.
Glenn Fernandes, Meixi Lu, Farzad Shahabi, Aggelos K. Katsaggelos, Nabil Alshurafa
AAAI1
2025 RayWatch: Hemispherical Diffusion on Wrist UV Sensor for Indoor-Outdoor Sensing
abstract
Excessive ultraviolet (UV) exposure is the principal driver of melanoma, yet at-risk individuals seldom receive timely, context-aware cues to apply protection. Existing wrist-worn UV monitors often struggle to recognize timely outdoor exposure because UV readings vary sharply with wrist orientation and sensor angle. To address this gap, we developed a wrist-watch form-factor device that embeds an AS7331 UV photodiode beneath a hemispherical polytetrafluoroethylene (PTFE) dome, which diffuses incident light and stabilizes the sensor's angular response. Ten participants wore the device during routine daily activities, generating more than 93k datapoints of annotated indoor-outdoor data. We implemented an on-device logistic-regression classifier, integrating UVA, UVB, and inertial features to distinguish indoor from outdoor contexts. Under leave-one-participant-out cross-validation, the PTFE-dome watch achieved 94% accuracy and a weighted F1 score of 0.95, outperforming both a flat-window GUVA-S12SD sensor (71% accuracy, F1 = 0.72) and a cylindrical-PTFE enclosure (84% accuracy, F1 = 0.85). By coupling a compact PTFE dome with on-device machine learning (ML), our system delivers reliable, on-wrist UV context sensing and paves the way for unobtrusive, personalized interventions to reduce cumulative UV exposure.
Harrison Dong, Glenn Fernandes, Christopher Romano, Tanmeet S. Butani, Neel Pendse, Farzad Shahabi, Tammy Stump, Nabil Alshurafa
BSN2
2025 A Multimodal AI-Enabled Framework for Characterizing Overeating Behaviors and Consumption Patterns
abstract
Overeating is a key contributor to obesity, yet identifying and characterizing its underlying causes remains challenging. While prior research has leveraged Ecological Momentary Assessment (EMA) to capture psychological and contextual factors in real-time, few studies have integrated EMA with passive sensing to uncover fine-grained, individualized consumption behaviors. In this work, we present a multimodal framework combining psychological and contextual data from a custom-built EMA app with validated camera-derived meal microstructure features from a neck-worn activity-oriented wearable camera. Across 41 participants, the camera captured 6,343 hours of footage over 312 days, yielding annotated bites, chews, meal start/end times, and dietitian-confirmed caloric intake. Using supervised contrastive learning, we generated meal-level representations, projected them using UMAP, and applied k-means clustering to identify behavioral phenotypes. We then conducted a z-score analysis to highlight features most distinctive to each cluster. Among the eight discovered groups, three consistently showed high purity for overeating meals (average purity$=0.99$), revealing nuanced, data-driven overeating phenotypes that may inform targeted intervention strategies.
Farzad Shahabi, Jessica Li, Christopher Romano, Rowan McCloskey, Glenn Fernandes, Mahdi Pedram, Jacob M. Schauer, Tammy Stump, Nabil Alshurafa
BSN5
2024 HealthSense: Unobtrusive Continuous Stress Monitoring Using a Novel Dual ECG-PPG Patch
abstract
Stress, a significant risk factor for chronic disease, manifests as changes in heart rate, respiration rate, and blood pressure. Non-invasive wearables like smartwatches can continuously track these physiological indicators to predict stress, enabling clinicians to develop and test interventions. However, most current devices are rigid and lack skin conformity, resulting in suboptimal signal quality and adherence during extended use. Furthermore, existing flexible sensors employ either electrocardiogram (ECG) or photoplethysmography (PPG), but not both, which is useful for calculating pulse arrival time (PAT) - known to correlate with stress. Addressing these challenges, we introduce HealthSense, a novel, flexible, and skin-conformable device that integrates ECG, PPG, and Inertial Measurement Unit (IMU) sensors into a single wearable. We assessed the comfort of wearing HealthSense and the feasibility of stress prediction by conducting a stress-induction study with 11 participants. Participants rated the comfort level of wearing the device on a Likert scale of 1-5, with 80% rating it as a 5 (most comfortable). Using statistical features, heart rate variability (HRV) related features, and PAT from our sensor data, we trained machine learning (ML) models to predict minute-level perceived and physiological stress with F1-scores of 85.5% and 87.7%, respectively. Additionally, using SHAP values, we identified PAT, systolic time, and pulse as the most significant contributors to the predictions. These findings enhance the understanding of physiological manifestations of stress and lays the groundwork for future stress-reduction interventions.
Glenn Fernandes, Boyang Wei, Christopher Romano, Deniz Ulusel, Henry K. Dambanemuya, Yang Gao 0025, Roozbeh Ghaffari, John A. Rogers, Nabil Alshurafa
BSN1
2024 Self-Sustaining Wearable UV Sensor for Passive and Continuous Sun Protection
abstract
Skin cancer, particularly melanoma, is a major health concern due to rising incidence rates, largely driven by ultraviolet (UV) radiation overexposure, making it essential to monitor and manage sun exposure effectively. While existing wearable UV sensors track exposure, they often rely on external power sources, limiting their battery lifetime. This study presents a self-sustaining wearable UV sensor that integrates solar energy harvesting, enabling continuous monitoring without need for frequent recharging. The device uses low-power components to measure UVA and UVB radiation with high accuracy. It is powered by a solar panel made from Ethylene Tetrafluoroethylene (ETFE), which provides continuous energy to recharge a LiPo battery. It transmits data via BLE for real-time feedback and can be used for personalized sun protection recommendations. A usability study with 10 participants demonstrated the sensor's effectiveness in raising UV awareness and encouraging sun protection habits.
Chenghong Lin, Neel Pendse, Glenn Fernandes, Nabil Alshurafa, Mahdi Pedram
BSN4
2024 When2Trigger: Evaluation Trade-Offs in Vision-Based Real-Time Eating Detection Systems
abstract
Wearable camera and thermal sensing systems are increasingly used for real-time eating detection and timely notifications to remind users to log their meals. However, confounding gestures such as irrelevant hand movements can cause false device confirmations of eating in real-time. Delaying the device confirmation of an eating episode, until the system is certain, can improve accuracy of eating detection, but prevents the capture of shorter bouts of eating. Balancing the trade-off between errors and detection delay is key to developing effective methods that provide immediate user feedback. This paper presents a real-time, hand-object-based method for automated detection of eating and drinking gestures and identifies the minimum number of gestures needed to reliably detect an eating episode. Unlike prior work, our method considers both hand motion and the object-in-hand and uses a low-power thermal sensor to reduce false positives. We evaluated our method on 36 participants, 28 of whom wore a wearable camera for up to 14 days in free-living environments. The results show that eating episodes can be accurately detected using 10 gestures or within the first 1.5 minutes of the eating episode, achieving an F1-score of 89.0%. Our findings provide evaluation guidelines for designing real-time intervention systems to address problematic eating behaviors.
Soroush Shahi, Glenn Fernandes, Christopher Romano, Nabil Alshurafa
BSN2
2022 SmartAct: Energy Efficient and Real-Time Hand-to-Mouth Gesture Detection Using Wearable RGB-T
abstract
Researchers have been leveraging wearable cameras to both visually confirm and automatically detect individuals' eating habits. However, energy-intensive tasks such as continuously collecting and storing RGB images in memory, or running algorithms in real-time to automate detection of eating, greatly impacts battery life. Since eating moments are spread sparsely throughout the day, battery life can be mitigated by recording and processing data only when there is a high likelihood of eating. We present a framework comprising a golf-ball sized wearable device using a low-powered thermal sensor array and real-time activation algorithm that activates high-energy tasks when a hand-to-mouth gesture is confirmed by the thermal sensor array. The high-energy tasks tested are turning on the RGB camera (Trigger RGB mode) and running inference on an on-device machine learning model (Trigger ML mode). Our experimental setup involved the design of a wearable camera, 6 participants collecting 18 hours of data with and without eating, the implementation of a feeding gesture detection algorithm on-device, and measures of power saving using our activation method. Our activation algorithm demonstrates an average of at-least 31.5% increase in battery life time, with minimal drop of recall (5%) and without impacting the accuracy of detecting eating (a slight 4.1% increase in F1-Score).
Soroush Shahi, Mahdi Pedram, Glenn Fernandes, Nabil Alshurafa
BSN3
2021 Users Want Diverse, Multiple, and Personalized Behavior Change Support: Need-Finding Survey
Mina Khan, Glenn Fernandes, Pattie Maes
PERSUASIVE2
2021 Improving Context-Aware Habit-Support Interventions Using Egocentric Visual Contexts
Mina Khan, Glenn Fernandes, Akash Vaish, Mayank Manuja, Pattie Maes, Agnis Stibe
PERSUASIVE2
2017 Optimizing drug-dose alerts using commercial software throughout an integrated health care system
abstract
All default electronic health record and drug reference database vendor drug-dose alerting recommendations (single dose, daily dose, dose frequency, and dose duration) were silently turned on in inpatient, outpatient, and emergency department areas for pediatric-only and nonpediatric-only populations. Drug-dose alerts were evaluated during a 3-month period. Drug-dose alerts fired on 12% of orders (104 098/834 911). System-level and drug-specific strategies to decrease drug-dose alerts were analyzed. System-level strategies included: (1) turning off all minimum drug-dosing alerts, (2) turning off all incomplete information drug-dosing alerts, (3) increasing the maximum single-dose drug-dose alert threshold to 125%, (4) increasing the daily dose maximum drug-dose alert threshold to 125%, and (5) increasing the dose frequency drug-dose alert threshold to more than 2 doses per day above initial threshold. Drug-specific strategies included changing drug-specific maximum single and maximum daily drug-dose alerting parameters for the top 22 drug categories by alert frequency. System-level approaches decreased alerting to 5% (46 988/834 911) and drug-specific approaches decreased alerts to 3% (25 455/834 911). Drug-dose alerts varied between care settings and patient populations.
Salim M. Saiyed, Peter Greco, Glenn Fernandes, David C. Kaelber
J. Am. Medical Informatics Assoc.3