OBJECTIVES: We aimed to identify and map recent studies using high-frequency, time-series electronic patient-generated health data (ePGHD) to assess treatment response; characterize ePGHD types and collection methods; summarize ePGHD-based definitions of treatment response; and describe analytical approaches used. MATERIALS AND METHODS: We systematically searched 4 databases for articles published between January 2022 and June 2024, supplemented by a forward citation search until June 2025. Peer-reviewed studies were eligible if ePGHD were collected outside clinical settings, and either reported at least weekly (ie, if actively reported by participants) or summarized discretely (eg, daily) if passively collected via wearables/sensors. We screened articles for eligibility independently in duplicate and synthesized extracted data descriptively. RESULTS: Our search yielded 4030 articles, of which we included 186. Most studies collected ePGHD using mobile applications or webforms (n = 133) over 4-12 weeks (n = 67). Prior to analysis, 132 studies excluded portions or condensed ePGHD into one or more summaries. Among 172 studies estimating treatment response, 98 applied longitudinal methods (eg, mixed-effects models) that accounted for repeated measures while capturing within- and between-subject variations, whereas 74 used cross-sectional approaches. Of 18 prediction modeling studies, 16 employed machine learning techniques, with only 4 explicitly modeling repeated measures. Five studies identified clusters of response trajectories generally without incorporating temporal dependencies (eg, using K-means). DISCUSSION AND CONCLUSION: Many studies in this review did not fully leverage the high-frequency, longitudinal nature of ePGHD. Future research should adopt more appropriate and readily available analytic methods to maximize the potential of time-series ePGHD for generating insights into treatment response.

Polymyalgia rheumatica (PMR) is an inflammatory rheumatic disease primarily seen in older patients, diagnosed in primary care, and treated with oral corticosteroids. It can be challenging to distinguish PMR from conditions with similar symptoms, with an incorrect diagnosis of PMR leading to delayed diagnosis for the true condition, and unnecessary steroid treatment. A pain manikin is a measurement tool for collecting self-report pain data, where people mark the location of their pain on a diagram of the human body. Advanced analysis of PMR pain manikin data may be able to improve the accuracy of PMR diagnosis in primary care. I will develop clinically relevant summary measures for PMR data, and build a machine learning model to distinguish PMR from other painful conditions.

Yanan Ma, Chenghao Xiao, Chenhan Yuan, Sabine N Van Der Veer, Lamiece Hassan, Chenghua Lin, Goran Nenadic. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

OBJECTIVES: To assess the feasibility and implementation, usability, acceptability and efficacy of virtual reality (VR), and augmented reality (AR) smartphone applications for upskilling care home workers in hand hygiene and to explore underlying learning mechanisms. MATERIALS AND METHODS: Care homes in Northwest England were recruited. We took a mixed-methods and pre-test and post-test approach by analyzing uptake and completion rates of AR, immersive VR or non-immersive VR training, validated and bespoke questionnaires, observations, videos, and interviews. Quantitative data were analyzed descriptively. Qualitative data were analyzed using a combined inductive and deductive approach. RESULTS: Forty-eight care staff completed AR training (n = 19), immersive VR training (n = 21), or non-immersive VR training (n = 8). The immersive VR and AR training had good usability with System Usability Scale scores of 84.40 and 77.89 (of 100), respectively. They had high acceptability, with 95% of staff supporting further use. The non-immersive VR training had borderline poor usability, scoring 67.19 and only 63% would support further use. There was minimal improved knowledge, with an average of 6% increase to the knowledge questionnaire. Average hand hygiene technique scores increased from 4.77 (of 11) to 7.23 after the training. Repeated practice, task realism, feedback and reminding, and interactivity were important learning mechanisms triggered by AR/VR. Feasibility and implementation considerations included managerial support, physical space, providing support, screen size, lagging Internet, and fitting the headset. CONCLUSIONS: AR and immersive VR apps are feasible, usable, and acceptable for delivering training. Future work should explore whether they are more effective than previous training and ensure equity in training opportunities.

OBJECTIVE: Outpatient no-shows have important implications for costs and the quality of care. Predictive models of no-shows could be used to target intervention delivery to reduce no-shows. We reviewed the effectiveness of predictive model-based interventions on outpatient no-shows, intervention costs, acceptability, and equity. MATERIALS AND METHODS: Rapid systematic review of randomized controlled trials (RCTs) and non-RCTs. We searched Medline, Cochrane CENTRAL, Embase, IEEE Xplore, and Clinical Trial Registries on March 30, 2022 (updated on July 8, 2022). Two reviewers extracted outcome data and assessed the risk of bias using ROB 2, ROBINS-I, and confidence in the evidence using GRADE. We calculated risk ratios (RRs) for the relationship between the intervention and no-show rates (primary outcome), compared with usual appointment scheduling. Meta-analysis was not possible due to heterogeneity. RESULTS: We included 7 RCTs and 1 non-RCT, in dermatology (n = 2), outpatient primary care (n = 2), endoscopy, oncology, mental health, pneumology, and an magnetic resonance imaging clinic. There was high certainty evidence that predictive model-based text message reminders reduced no-shows (1 RCT, median RR 0.91, interquartile range [IQR] 0.90, 0.92). There was moderate certainty evidence that predictive model-based phone call reminders (3 RCTs, median RR 0.61, IQR 0.49, 0.68) and patient navigators reduced no-shows (1 RCT, RR 0.55, 95% confidence interval 0.46, 0.67). The effect of predictive model-based overbooking was uncertain. Limited information was reported on cost-effectiveness, acceptability, and equity. DISCUSSION AND CONCLUSIONS: Predictive modeling plus text message reminders, phone call reminders, and patient navigator calls are probably effective at reducing no-shows. Further research is needed on the comparative effectiveness of predictive model-based interventions addressed to patients at high risk of no-shows versus nontargeted interventions addressed to all patients.

OBJECTIVES: (1) Systematically review the literature on computerized audit and feedback (e-A&F) systems in healthcare. (2) Compare features of current systems against e-A&F best practices. (3) Generate hypotheses on how e-A&F systems may impact patient care and outcomes. METHODS: We searched MEDLINE (Ovid), EMBASE (Ovid), and CINAHL (Ebsco) databases to December 31, 2020. Two reviewers independently performed selection, extraction, and quality appraisal (Mixed Methods Appraisal Tool). System features were compared with 18 best practices derived from Clinical Performance Feedback Intervention Theory. We then used realist concepts to generate hypotheses on mechanisms of e-A&F impact. Results are reported in accordance with the PRISMA statement. RESULTS: Our search yielded 4301 unique articles. We included 88 studies evaluating 65 e-A&F systems, spanning a diverse range of clinical areas, including medical, surgical, general practice, etc. Systems adopted a median of 8 best practices (interquartile range 6-10), with 32 systems providing near real-time feedback data and 20 systems incorporating action planning. High-confidence hypotheses suggested that favorable e-A&F systems prompted specific actions, particularly enabled by timely and role-specific feedback (including patient lists and individual performance data) and embedded action plans, in order to improve system usage, care quality, and patient outcomes. CONCLUSIONS: e-A&F systems continue to be developed for many clinical applications. Yet, several systems still lack basic features recommended by best practice, such as timely feedback and action planning. Systems should focus on actionability, by providing real-time data for feedback that is specific to user roles, with embedded action plans. PROTOCOL REGISTRATION: PROSPERO CRD42016048695.

OBJECTIVE: The study sought to identify smartphone apps that support hand hygiene practice and to assess their content, technical and functional features, and quality. A secondary objective was to make design and research recommendations for future apps. MATERIALS AND METHODS: We searched the UK Google Play and Apple App stores for hand hygiene smartphone apps aimed at adults. Information regarding content, technical and functional features was extracted and summarized. Two raters evaluated each app, using the IMS Institute for Healthcare Informatics functionality score and the Mobile App Rating Scale (MARS). RESULTS: A total of 668 apps were identified, with 90 meeting the inclusion criteria. Most (96%) were free to download. The majority (78%) intended to educate or inform or remind users to hand wash (69%), using behavior change techniques such as personalization and prompting practice. Only 20% and 4% named a best practice guideline or had expert involvement in development, respectively. Innovative means of engagement were used in 42% (eg, virtual or augmented reality or geolocation-based reminders). Apps included an average of 2.4 out of 10 of the IMS functionality criteria (range, 0-8). The mean MARS score was 3.2 ± 0.5 out of 5, and 68% had a minimum acceptability score of 3. Two had been tested or trialed. CONCLUSIONS: Although many hand hygiene apps exist, few provide content on best practice. Many did not meet the minimum acceptability criterion for quality or were formally trialed or tested. Research should assess the feasibility and effectiveness of hand hygiene apps (especially within healthcare settings), including when and how they "work." We recommend that future apps to support hand hygiene practice are developed with infection prevention and control experts and align with best practice. Robust research is needed to determine which innovative methods of engagement create "sticky" apps.

OBJECTIVE: To investigate how the general public trades off explainability versus accuracy of artificial intelligence (AI) systems and whether this differs between healthcare and non-healthcare scenarios. MATERIALS AND METHODS: Citizens' juries are a form of deliberative democracy eliciting informed judgment from a representative sample of the general public around policy questions. We organized two 5-day citizens' juries in the UK with 18 jurors each. Jurors considered 3 AI systems with different levels of accuracy and explainability in 2 healthcare and 2 non-healthcare scenarios. Per scenario, jurors voted for their preferred system; votes were analyzed descriptively. Qualitative data on considerations behind their preferences included transcribed audio-recordings of plenary sessions, observational field notes, outputs from small group work and free-text comments accompanying jurors' votes; qualitative data were analyzed thematically by scenario, per and across AI systems. RESULTS: In healthcare scenarios, jurors favored accuracy over explainability, whereas in non-healthcare contexts they either valued explainability equally to, or more than, accuracy. Jurors' considerations in favor of accuracy regarded the impact of decisions on individuals and society, and the potential to increase efficiency of services. Reasons for emphasizing explainability included increased opportunities for individuals and society to learn and improve future prospects and enhanced ability for humans to identify and resolve system biases. CONCLUSION: Citizens may value explainability of AI systems in healthcare less than in non-healthcare domains and less than often assumed by professionals, especially when weighed against system accuracy. The public should therefore be actively consulted when developing policy on AI explainability.

OBJECTIVE: People with long-term conditions require serial clinical assessments. Digital patient-reported symptoms collected between visits can inform these, especially if integrated into electronic health records (EHRs) and clinical workflows. This systematic review identified and summarized EHR-integrated systems to remotely collect patient-reported symptoms and examined their anticipated and realized benefits in long-term conditions. MATERIALS AND METHODS: We searched Medline, Web of Science, and Embase. Inclusion criteria were symptom reporting systems in adults with long-term conditions; data integrated into the EHR; data collection outside of clinic; data used in clinical care. We synthesized data thematically. Benefits were assessed against a list of outcome indicators. We critically appraised studies using the Mixed Methods Appraisal Tool. RESULTS: We included 12 studies representing 10 systems. Seven were in oncology. Systems were technically and functionally heterogeneous, with the majority being fully integrated (data viewable in the EHR). Half of the systems enabled regular symptom tracking between visits. We identified 3 symptom report-guided clinical workflows: Consultation-only (data used during consultation, n = 5), alert-based (real-time alerts for providers, n = 4) and patient-initiated visits (n = 1). Few author-described anticipated benefits, primarily to improve communication and resultant health outcomes, were realized based on the study results, and were only supported by evidence from early-stage qualitative studies. Studies were primarily feasibility and pilot studies of acceptable quality. DISCUSSION AND CONCLUSIONS: EHR-integrated remote symptom monitoring is possible, but there are few published efforts to inform development of these systems. Currently there is limited evidence that this improves care and outcomes, warranting future robust, quantitative studies of efficacy and effectiveness.

OBJECTIVE: The study sought to explore to what extent geolocation data has been used to study serious mental illness (SMI). SMIs such as bipolar disorder and schizophrenia are characterized by fluctuating symptoms and sudden relapse. Currently, monitoring of people with an SMI is largely done through face-to-face visits. Smartphone-based geolocation sensors create opportunities for continuous monitoring and early intervention. MATERIALS AND METHODS: We searched MEDLINE, PsycINFO, and Scopus by combining terms related to geolocation and smartphones with SMI concepts. Study selection and data extraction were done in duplicate. RESULTS: Eighteen publications describing 16 studies were included in our review. Eleven studies focused on bipolar disorder. Common geolocation-derived digital biomarkers were number of locations visited (n = 8), distance traveled (n = 8), time spent at prespecified locations (n = 7), and number of changes in GSM (Global System for Mobile communications) cell (n = 4). Twelve of 14 publications evaluating clinical aspects found an association between geolocation-derived digital biomarker and SMI concepts, especially mood. Geolocation-derived digital biomarkers were more strongly associated with SMI concepts than other information (eg, accelerometer data, smartphone activity, self-reported symptoms). However, small sample sizes and short follow-up warrant cautious interpretation of these findings: of all included studies, 7 had a sample of fewer than 10 patients and 11 had a duration shorter than 12 weeks. CONCLUSIONS: The growing body of evidence for the association between SMI concepts and geolocation-derived digital biomarkers shows potential for this instrument to be used for continuous monitoring of patients in their everyday lives, but there is a need for larger studies with longer follow-up times.

Risk of psychotic relapse in schizophrenic patients is commonly measured by social functioning (SF), which focuses on patients' daily activities. Monitoring of SF usually relies on infrequent clinic visits, limiting the capacity to detect sudden changes. GPS data that is passively collected with smartphones introduce new opportunities to monitor SF. We conducted a five-day pilot study with five schizophrenic patients to assess the feasibility of this approach. Participants used a smartphone to continuously record their GPS location, and completed a paper-based SF diary to register out-of-home activities. We implemented a time-based method and a density-based method to identify the geolocations visited and then we clustered geolocations visited in places visited. Finally, we used semantic enrichment to classify places types and associated activities. We evaluated the performance of the two approaches by comparing the activities detected from the GPS data with those recorded in the SF diary. Recall was better for the density-based method, ranging from 0.686 (Standard Deviation [SD] 0.168) to 0.771 (SD 0.264) while precision was better for the time-based method (0.722 (SD 0.197) to 0.954 (SD 0.093)). To conclude, using routinely collected GPS data and relatively simple analytical methods we detected patients' out-of-home activities with moderate recall, more sophisticated analytical methods may obtain better performance.