Cancer survivors face unique mental health challenges, yet nearly half report unmet psychosocial needs. Smartphone interventions could help, but a major obstacle is knowing if, when, and how to intervene because inferring affective states with low-burden methods is hard. We test whether ultra-brief mobile diaries can infer contextual information approximating survivors’ affect, desire to regulate affect, and potential availability for brief digital behavioral interventions. Analyzing 24,183 entries from 407 survivors, administrative and health-related situations align with higher negative affect, whereas leisure/social situations align with higher positive affect. We introduce a Context-Aware LLM (CALLM) framework, which curates context via similarity-aligned peer cases and short personal trajectories, achieving balanced accuracy of 72.96% (positive affect), 73.29% (negative affect), 73.72% (regulation desire), and 60.09% (intervention availability), outperforming baselines. Post-hoc analyses show LLM confidence tracks accuracy, longer entries aid inference, and brief calibration improves personalization. Findings inform future just-in-time adaptive interventions for this underrepresented population.

Effective patient-provider communication is crucial in clinical care, directly impacting patient outcomes and quality of life. Traditional evaluation methods, such as human ratings, patient feedback, and provider self-assessments, are often limited by high costs and scalability issues. Although existing natural language processing (NLP) techniques show promise, they struggle with the nuances of clinical communication and require sensitive clinical data for training, reducing their effectiveness in real-world applications. Emerging large language models (LLMs) offer a new approach to assessing complex communication metrics, with the potential to advance the field through integration into passive sensing and just-in-time intervention systems. This study explores LLMs as evaluators of palliative care communication quality, leveraging their linguistic, in-context learning, and reasoning capabilities. Specifically, using simulated scripts crafted and labeled by healthcare professionals, we test proprietary models (e.g., GPT-4) and fine-tune open-source LLMs (e.g., LLaMA2) with a synthetic dataset generated by GPT-4 to evaluate clinical conversations, to identify key metrics such as “understanding” and “empathy.” Our findings demonstrated LLMs’ superior performance in evaluating clinical communication, providing actionable feedback with reasoning, and demonstrating the feasibility and practical viability of developing in-house LLMs. This research highlights LLMs’ potential to enhance patient-provider interactions and lays the groundwork for downstream steps in developing LLM-empowered clinical health systems.

Social anxiety is a common mental health condition linked to significant challenges in academic, social, and occupational functioning. A core feature is elevated momentary (state) anxiety in social situations, yet little prior work has measured or predicted fluctuations in this anxiety throughout the day. Capturing these intra-day dynamics is critical for designing realtime, personalized interventions such as Just-In-Time Adaptive Interventions (JITAIs). To address this gap, we conducted a study with socially anxious college students ($\mathrm{N}=91$; 72 after exclusions) using our custom smartwatch-based system over an average of 9.03 days (SD = 2.95). Participants received seven ecological momentary assessments (EMAs) per day to report state anxiety. We developed a base model on over 10,000 days of external heart rate data, transferred its representations to our dataset, and fine-tuned it to generate probabilistic predictions. These were combined with trait-level measures in a meta-learner. Our pipeline achieved 60.4% balanced accuracy in state anxiety detection in our dataset. To evaluate generalizability, we applied the training approach to a separate hold-out set from the TILES-18 dataset-the same dataset used for pretraining. On 10,095 once-daily EMAs, our method achieved 59.1% balanced accuracy, outperforming prior work by at least 7%.

Mobile sensing is ubiquitous and offers opportunities to gain insight into state mental health functioning. Detecting state elevations in social anxiety would be especially useful given this phenomenon is highly prevalent and impairing, but often not disclosed. In the present work, we explore the feasibility of detecting fluctuations in state social anxiety among N = 46 undergraduate students with elevated symptoms of trait social anxiety. Participants engaged in two dyadic and two group social interactions via Zoom. We evaluated participants' state anxiety levels as they anticipated, immediately after experiencing, and upon reflecting on each social interaction, spanning a time frame of 2–6 minutes. We collected biobehavioral features (i.e., PPG, EDA, skin temperature, and accelerometer) via Empatica E4 devices as they participated in the varied social contexts (e.g., dyadic vs. group; anticipating vs. experiencing the interaction; experiencing varying levels of social evaluation). We additionally measured their trait mental health functioning. Mixed-effect logistic regression and leave-one-subject-out machine learning modeling indicated biobehavioral features significantly predict state fluctuations in anxiety, though balanced accuracy tended to be modest (59%). However, our capacity to identify instances of heightened versus low state anxiety significantly increased (with balanced accuracy ranging from 69 % to 84 % across different operationalizations of state anxiety) when we integrated contextual data alongside trait mental health functioning into our predictive models. We discuss these and other findings in the context of the broader anxiety detection literature.

Correctly identifying an individual's social context from passively worn sensors holds promise for delivering just-in-time adaptive interventions (JITAIs) to treat social anxiety. In this study, we present results using passively collected data from a within-subjects experiment that assessed physiological responses across different social contexts (i.e., alone vs. with others), social phases (i.e., pre- and post-interaction vs. during an interaction), social interaction sizes (i.e., dyadic vs. group interactions), and levels of social threat (i.e., implicit vs. explicit social evaluation). Participants in the study ($N=46$) reported moderate to severe social anxiety symptoms as assessed by the Social Interaction Anxiety Scale ($\geq$34 out of 80). Univariate paired difference tests, multivariate random forest models, and cluster analyses were used to explore physiological response patterns across different social and non-social contexts. Our results suggest that social context is more reliably distinguishable than social phase, group size, or level of social threat, and that there is considerable variability in physiological response patterns even among distinguishable contexts. Implications for real-world context detection and future deployment of JITAIs are discussed.

During social interactions, understanding the in-tricacies of the context can be vital, particularly for socially anxious individuals. While previous research has found that the presence of a social interaction can be detected from ambient audio, the nuances within social contexts, which influence how anxiety provoking interactions are, remain largely unexplored. As an alternative to traditional, burdensome methods like self-report, this study presents a novel approach that harnesses ambient audio segments to detect social threat contexts. We focus on two key dimensions: number of interaction partners (dyadic vs. group) and degree of evaluative threat (explicitly evaluative vs. not explicitly evaluative). Building on data from a Zoom-based social interaction study (N=52 college students, of whom the majority N =45 are socially anxious), we employ deep learning methods to achieve strong detection performance. Under sample-wide 5-fold Cross Validation (CV), our model distinguished dyadic from group interactions with 90 % accuracy and detected evaluative threat at 83 %. Using a leave-one-group-out CV, accuracies were 82 % and 77 %, respectively. While our data are based on virtual interactions due to pandemic constraints, our method has the potential to extend to diverse real-world settings. This research underscores the potential of passive sensing and AI to differentiate intricate social contexts, and may ultimately advance the ability of context-aware digital interventions to offer personalized mental health support.

While audio data shows promise in addressing various health challenges, there is a lack of research on on-device audio processing for smartwatches. Privacy concerns make storing raw audio and performing post-hoc analysis undesirable for many users. Additionally, current on-device audio processing systems for smartwatches are limited in their feature extraction capabilities, restricting their potential for understanding user behavior and health. We developed a real-time system for on-device audio processing on smartwatches, which takes an average of 1.78 minutes (SD = 0.07 min) to extract 22 spectral and rhythmic features from a 1-minute audio sample, using a small window size of 25 milliseconds. Using these extracted audio features on a public dataset, we developed and incorporated models into a watch to classify foreground and background speech in real-time. Our Random Forest-based model classifies speech with a balanced accuracy of 80.3%.

Quality patient-provider communication is critical to improve clinical care and patient outcomes. While progress has been made with communication skills training for clinicians, significant gaps exist in how to best monitor, measure, and evaluate the implementation of communication skills in the actual clinical setting. Advancements in ubiquitous technology and natural language processing make it possible to realize more objective, real-time assessment of clinical interactions and in turn provide more timely feedback to clinicians about their communication effectiveness. In this paper, we propose CommSense, a computational sensing framework that combines smartwatch audio and transcripts with natural language processing methods to measure selected "best-practice'' communication metrics captured by wearable devices in the context of palliative care interactions, including understanding, empathy, presence, emotion, and clarity. We conducted a pilot study involving N=40 clinician participants, to test the technical feasibility and acceptability of CommSense in a simulated clinical setting. Our findings demonstrate that CommSense effectively captures most communication metrics and is well-received by both practicing clinicians and student trainees. Our study also highlights the potential for digital technology to enhance communication skills training for healthcare providers and students, ultimately resulting in more equitable delivery of healthcare and accessible, lower cost tools for training with the potential to improve patient outcomes.

Wearable devices with embedded sensors can provide personalized healthcare and wellness benefits in digital phenotyping and adaptive interventions. However, the collection, storage, and transmission of biometric data (including processed features rather than raw signals) from these devices pose significant privacy concerns. This quantitative, data-driven study examines the privacy risks associated with wearable-based digital phenotyping practices, with a focus on user reidentification (ReID), which is the process of identifying participants’ IDs from deidentified digital phenotyping datasets. We propose a machine-learning-based computational pipeline to evaluate and quantify model outcomes under various configurations, such as modality inclusion, window length, and feature type and format, to investigate the factors influencing ReID risks and their predictive trade-offs. This pipeline leverages features extracted from three wearable sensors, resulting in up to 68.43% accuracy in ReID risk for a sample size of N=45 socially anxious participants based on only descriptive features of 10-second observations. Additionally, we explore the trade-offs between privacy risks and predictive benefits by adjusting various settings (e.g., the ways to process extracted features). Our findings highlight the importance of privacy in digital phenotyping and suggest potential future directions.

While spatial-temporal environment monitoring has become an indispensable way to collect data for enabling smart cities and intelligent transportation applications, the cost to deploy, operate and maintain a sensor network with sensors and massive communication infrastructure is too high to bear. Compared to the infrastructure-based sensing approach, community sensing, or namely mobile crowdsensing, that leverage community members' mobile devices to collect data becomes a feasible way to scale up the spatial-temporal coverage of the sensing system. However, a community sensing system would need to aggregate sensors and location data from community members and thus would raise concerns on privacy and data security In this paper, we present a novel community sensing paradigm AFCS -Sensor and Location Data Aggregation-Free Community Sensing, which is designed to obtain the environment information (e.g., spatial-temporal distributions of air pollution, temperature, and bike-shares) in each subarea of the target area, without aggregating sensor and location data collected by community members. AFCS proposes to orchestrate with the Trusted Execution Environments (TEEs) of every community member's mobile device to cover the communication, computation and storage with spatial-temporal data. Further, AFCS proposes a novel Decentralized Spatial-Temporal Compressive Sensing framework based on Parallelized Stochastic Gradient Descent. Through learning the latent structure of the spatial-temporal data via decentralized optimization, AFCS approximates the value of the sensor data in each subarea (both covered and uncovered) for each sensing cycle using the sensor data locally stored in every member's TEE instance. Experiments based on real-world datasets and the Virtual Mobile Infrastructure (VMI) with TEE emulations demonstrate that AFCS exhibits low approximation error (i.e., less than 0:2°C in city-wide temperature sensing, 10 units of PM2.5 index in urban air pollution sensing, and 2 bikes in city-wide bike sharing prediction) and performs comparably to (sometimes better than) state-of-the-art algorithms based on the data aggregation and centralized computation

Bike-sharing systems have been deployed globally. One of the key issues for high-quality bike-sharing systems is to rebalance city-wide stations to maintain bike availability. Traditional strategies, such as repositioning bikes by trucks and volunteers based on historical riding records, usually operate in fixed paths and limited capacities, lacking the flexibility to cope with the highly dynamic and context dependent riding demands, and usually suffer from high costs and long delays. In this work, we propose RedPacketBike, an incentive-driven, crowd-based station rebalancing framework to effectively recruit participants from hybrid fleets (e.g., volunteer riders and hired trucks) based on the accurate forecast of bike demand leveraging deep learning techniques. First, we propose a spatiotemporal clustering method to extract bike demand hotspots from fluctuating bike usage data. Then, we build a context-aware deep neural network named BikeNet to forecast the trends of bike demand hotspots, simultaneously modeling the spatial correlations by graph convolution networks (GCN), the temporal dependencies by long short-term memory networks (RNN), and the contextual factors by autoencoders (AE). Finally, we propose a reinforcement-learning-based method to find optimal station rebalancing schemes by generating station rebalancing tasks with an integer linear programming (ILP) algorithm and allocating tasks to participants from hybrid fleets with dynamic incentive designs and reward expectations. Experiments using real-world bike-sharing system data collected from Citi Bike in New York City and Mobike in Xiamen City validate the performance of our framework, achieving a demand forecast error below 4.171 measured in MAE, and a 17.2% improvement of station availability by simulations with real-world parameter settings, outperforming the state-of-the-art baselines.

The Internet of Things (IoT) boom has revolutionized almost every corner of people’s daily lives: healthcare, environment, transportation, manufacturing, supply chain, and so on. With the recent development of sensor and communication technology, IoT artifacts, including smart wearables, cameras, smartwatches, and autonomous systems can accurately measure and perceive their surrounding environment. Continuous sensing generates massive amounts of data and presents challenges for machine learning. Deep learning models (e.g., convolution neural networks and recurrent neural networks) have been extensively employed in solving IoT tasks by learning patterns from multi-modal sensory data. Graph neural networks (GNNs), an emerging and fast-growing family of neural network models, can capture complex interactions within sensor topology and have been demonstrated to achieve state-of-the-art results in numerous IoT learning tasks. In this survey, we present a comprehensive review of recent advances in the application of GNNs to the IoT field, including a deep dive analysis of GNN design in various IoT sensing environments, an overarching list of public data and source codes from the collected publications, and future research directions. To keep track of newly published works, we collect representative papers and their open-source implementations and create a Github repository at GNN4IoT.

Mobile sensing systems have been widely used as a practical approach to collect behavioral and health-related information from individuals and to provide timely intervention to promote health and well being, such as mental health and chronic care. As the objectives of mobile sensing could be eitherpersonalized medicine for individualsorpublic health for populations, in this work, we review the design of these mobile sensing systems, and propose to categorize the design of these systems in two paradigms—1)personal sensingand 2)crowdsensingparadigms. While both sensing paradigms might incorporate common ubiquitous sensing technologies, such aswearable sensors,mobility monitoring,mobile data offloading, andcloud-based data analyticsto collect and process sensing data from individuals, we present two novel taxonomy systems based on the: 1)sensing objectives(e.g., goals of mobile health (mHealth) sensing systems and how technologies achieve the goals) and 2)the sensing systems design and implementation (D&I)(e.g., designs of mHealth sensing systems and how technologies are implemented). With respect to the two paradigms and two taxonomy systems, this work systematically reviews this field. Specifically, we first present technical reviews on the mHealth sensing systems in eight common/popular healthcare issues, ranging from depression and anxiety to COVID-19. By summarizing the mHealth sensing systems, we comprehensively survey the research works using the two taxonomy systems, where we systematically review thesensing objectivesandsensing systems D&Iwhile mapping the related research works onto the life-cycles of mHealth Sensing, i.e.: 1)sensing task creation and participation; 2)(health surveillance and data collection; and 3)data analysis and knowledge discovery. In addition to summarization, the proposed taxonomy systems also help the potential directions of mobile sensing for health from both personalized medicine and population health perspectives. Finally, we attempt to test and discuss the validity of our scientific approaches to the survey.