OBJECTIVE: To characterize patient and clinician perceived barriers and facilitators to using electronic patient-generated data (PGD) in safety-net systems. MATERIALS AND METHODS: We conducted 43 semi-structured interviews (18 clinicians and 25 patients) and observed 15 patient-clinician interactions. Clinical observations were conducted in an integrated urban safety-net system. Patients who spoke English, Spanish, or Cantonese were recruited from this system. Pharmacists, nurses, and clinicians treating chronic diseases were sampled from multiple California safety-net systems. Interview guides were developed based on the Consolidated Framework for Implementation Research (CFIR) and the Behavior Change Wheel (BCW). We conducted thematic analysis through a combination of inductive and deductive coding. RESULTS: Themes most frequently identified by both clinicians and patients as impacting electronic PGD use were capability-related (knowledge about collecting and using PGD), motivation-related (preference for data sharing; attitude toward digital tools and learning how to use them; the importance of measuring the outcome for health; privacy; and patient-clinician relationship), and opportunity-related (social support). Non-English speakers expressed concerns about inconveniencing others. Clinicians also identified additional opportunity-related themes (resource availability; implementation process; external incentives). DISCUSSION: Despite the growth in electronic PGD and its potential to improve chronic disease care and outcomes, implementation in safety-net systems would benefit from consideration of capability, motivation, and opportunity-related barriers. CONCLUSION: PGD is an increasingly vital part of clinical care. If implementation is pursued without concurrently addressing factors at the patient, clinician, health system, and policy levels, barriers to adoption will persist, especially in under-resourced settings.

OBJECTIVE: To identify a brief scale to accurately assess digital skills among older adults for use in identifying need for support to use digital health tools. MATERIALS AND METHODS: Patients age ≥50 speaking English, Spanish, or Cantonese completed surveys (n = 186) assessing digital health access, use, and skills. A subsample (n = 101) completed observational task assessments gauging competency on 4 tasks essential to digital health skills: (1) launch a video visit from an email/text message hyperlink, (2) visit a specific health website, (3) sign up for a patient portal, and (4) log in to a patient portal. We used exploratory factor analysis, receiver operator characteristic, logistic regression, and dominance analysis methods to identify and evaluate a scale measuring digital skills essential to using digital health tools. RESULTS: We found that a 9-item scale demonstrated unidimensionality and reliability (Cronbach's alpha 0.93) in measuring digital skills. Mean score was 19.3 out of 36. For each task, handout/video support was inadequate in facilitating completion for one-quarter of participants. We found high accuracy of the scale in predicting digital health competency (area under the curve 0.77-0.88). DISCUSSION: The Skills Measurement and Readiness Training for Digital Health (SMART Digital Health) scale is a measure of digital skills with evidence of reliability and validity to be used as a diagnostic tool to identify patients requiring support to use digital health tools. CONCLUSION: This early work supports the identification of patients with digital literacy needs who may require interventions to effectively engage in digital health communication and management.

The 21st Century Cures Act and the rise of telemedicine led to renewed focus on patient portals. However, portal use disparities persist and are in part driven by limited digital literacy. To address digital disparities in primary care, we implemented an integrated digital health navigator program supporting portal use among patients with type II diabetes. During our pilot, we were able to enroll 121 (30.9%) patients onto the portal. Of newly enrolled or trained patients, 75 (62.0%) were Black, 13 (10.7%) were White, 23 (19.0%) were Hispanic/Latinx, 4 (3.3%) were Asian, 3 (2.5%) were of another race or ethnicity, and 3 (2.5%) had missing data. Our overall portal enrollment for clinic patients with type II diabetes increased for Hispanic/Latinx patients from 30% to 42% and Black patients from 49% to 61%. We used the Consolidated Framework for Implementation Research to understand key implementation components. Using our approach, other clinics can implement an integrated digital health navigator to support patient portal use.

We evaluated the usability of mobile COVID-19 contact tracing apps, especially for individuals with barriers to communication and limited digital literacy skills. We searched the Apple App Store, Google Play, peer-reviewed literature, and lay press to find contact tracing apps in the United States. We evaluated apps with a framework focused on user characteristics and user interface. Of the final 26 apps, 77% were on both iPhone and Android. 69% exceeded 9th grade readability, and 65% were available only in English. Only 12% had inclusive illustrations (different genders, skin tones, physical abilities). 92% alerted users of an exposure, 42% linked to a testing site, and 62% linked to a public health website within 3 clicks. Most apps alert users of COVID-19 exposure but require high English reading levels and are not fully inclusive of the U.S. population, which may limit their reach as public health tools.

The substantial expansion of secure messaging (SM) via the patient portal in the last decade suggests that it is becoming a standard of care, but few have examined SM use longitudinally. We examined SM patterns among a diverse cohort of patients with diabetes (N = 19 921) and the providers they exchanged messages with within a large, integrated health system over 10 years (2006-2015), linking patient demographics to SM use. We found a 10-fold increase in messaging volume. There were dramatic increases overall and for patient subgroups, with a majority of patients (including patients with lower income or with self-reported limited health literacy) messaging by 2015. Although more physicians than nurses and other providers messaged throughout the study, the distribution of health professions using SM changed over time. Given this rapid increase in SM, deeper understanding of optimizing the value of patient and provider engagement, while managing workflow and training challenges, is crucial.

OBJECTIVE: Providing behavioral health interventions via smartphones allows these interventions to be adapted to the changing behavior, preferences, and needs of individuals. This can be achieved through reinforcement learning (RL), a sub-area of machine learning. However, many challenges could affect the effectiveness of these algorithms in the real world. We provide guidelines for decision-making. MATERIALS AND METHODS: Using thematic analysis, we describe challenges, considerations, and solutions for algorithm design decisions in a collaboration between health services researchers, clinicians, and data scientists. We use the design process of an RL algorithm for a mobile health study "DIAMANTE" for increasing physical activity in underserved patients with diabetes and depression. Over the 1.5-year project, we kept track of the research process using collaborative cloud Google Documents, Whatsapp messenger, and video teleconferencing. We discussed, categorized, and coded critical challenges. We grouped challenges to create thematic topic process domains. RESULTS: Nine challenges emerged, which we divided into 3 major themes: 1. Choosing the model for decision-making, including appropriate contextual and reward variables; 2. Data handling/collection, such as how to deal with missing or incorrect data in real-time; 3. Weighing the algorithm performance vs effectiveness/implementation in real-world settings. CONCLUSION: The creation of effective behavioral health interventions does not depend only on final algorithm performance. Many decisions in the real world are necessary to formulate the design of problem parameters to which an algorithm is applied. Researchers must document and evaulate these considerations and decisions before and during the intervention period, to increase transparency, accountability, and reproducibility. TRIAL REGISTRATION: clinicaltrials.gov, NCT03490253.

OBJECTIVE: To determine interest in and barriers to video visits in safety-net patients with diverse age, racial/ethnic, or linguistic background. MATERIALS AND METHODS: We surveyed patients in an urban safety-net system to assess: interest in video visits; ability to successfully complete test video visits; and barriers to successful completion of test video visits. RESULTS: Among 202 participants, of which 177 (87.6%) were persons of color and 113 (55.9%) preferred non-English languages, 132 (65.3%) were interested in and 109 (54.0%) successfully completed a test video visit. Younger age, non-English preference, and prior smartphone application use were associated with interest. Over half (n = 112) reported barriers to video visits; Internet/data access was the most common barrier (n = 50, 24.8%). CONCLUSION: Safety-net patients are interested in video visits and able to successfully complete test visits. Internet or mobile data access is a common barrier in even urban safety-net settings and may impact equitable telemedicine access.

OBJECTIVE: The study sought to evaluate if peer input on outpatient cases impacted diagnostic confidence. MATERIALS AND METHODS: This randomized trial of a peer input intervention occurred among 28 clinicians with case-level randomization. Encounters with diagnostic uncertainty were entered onto a digital platform to collect input from ≥5 clinicians. The primary outcome was diagnostic confidence. We used mixed-effects logistic regression analyses to assess for intervention impact on diagnostic confidence. RESULTS: Among the 509 cases (255 control; 254 intervention), the intervention did not impact confidence (odds ratio [OR], 1.46; 95% confidence interval [CI], 0.999-2.12), but after adjusting for clinician and case traits, the intervention was associated with higher confidence (OR, 1.53; 95% CI, 1.01-2.32). The intervention impact was greater in cases with high uncertainty (OR, 3.23; 95% CI, 1.09- 9.52). CONCLUSIONS: Peer input increased diagnostic confidence primarily in high-uncertainty cases, consistent with findings that clinicians desire input primarily in cases with continued uncertainty.

Peer mentors have been proven to improve diabetes outcomes, especially among diverse patients. Delivering peer mentoring via remote strategies (phone, text, mobile applications) is critical, especially in light of the recent pandemic. We conducted a real-world evaluation of a remote diabetes intervention in a safety-net delivery system in New York. We summarized the uptake, content, and pre-post clinical effectiveness for English- and Spanish-speaking participants. Of patients who could be reached, 71% (n = 690/974) were enrolled, and 90% of those (n = 618/690) participated in coaching. Patients and mentors had a mean of 32 check-ins, and each patient set an average of 10 goals. 29% of the participants accessed the program via the smartphone application. Among participants with complete hemoglobin A1c data (n = 179), there was an absolute 1.71% reduction (P < .01). There are multiple lessons for successful implementation of remote peer coaching into settings serving diverse patients, including meaningful patient-mentor matching and addressing social determinants.

OBJECTIVE: The study sought to determine which patient characteristics are associated with the use of patient-facing digital health tools in the United States. MATERIALS AND METHODS: We conducted a literature review of studies of patient-facing digital health tools that objectively evaluated use (eg, system/platform data representing frequency of use) by patient characteristics (eg, age, race or ethnicity, income, digital literacy). We included any type of patient-facing digital health tool except patient portals. We reran results using the subset of studies identified as having robust methodology to detect differences in patient characteristics. RESULTS: We included 29 studies; 13 had robust methodology. Most studies examined smartphone apps and text messaging programs for chronic disease management and evaluated only 1-3 patient characteristics, primarily age and gender. Overall, the majority of studies found no association between patient characteristics and use. Among the subset with robust methodology, white race and poor health status appeared to be associated with higher use. DISCUSSION: Given the substantial investment in digital health tools, it is surprising how little is known about the types of patients who use them. Strategies that engage diverse populations in digital health tool use appear to be needed. CONCLUSION: Few studies evaluate objective measures of digital health tool use by patient characteristics, and those that do include a narrow range of characteristics. Evidence suggests that resources and need drive use.

OBJECTIVE: US health care institutions are implementing secure websites (patient portals) to achieve federal Meaningful Use (MU) certification. We sought to understand efforts to implement portals in "safety net" health care systems that provide services for low-income populations. MATERIALS AND METHODS: Our rapid ethnography involved visits at 4 California safety net health systems and in-depth interviews at a fifth. Visits included interviews with clinicians and executives ( n = 12), informal focus groups with front-line staff ( n = 35), observations of patient portal sign-up procedures and clinic work, review of marketing materials and portal use data, and a brief survey ( n = 45). RESULTS: Our findings demonstrate that the health systems devoted considerable effort to enlisting staff support for portal adoption and integrating portal-related work into clinic routines. Although all health systems had achieved, or were close to achieving, MU benchmarks, patients faced numerous barriers to portal use and our participants were uncertain how to achieve and sustain "meaningful use" as defined by and for their patients. DISCUSSION: Health systems' efforts to achieve MU certification united clinic staff under a shared ethos of improved quality of care. However, MU's assumptions about patients' demand for electronic access to health information and ability to make use of it directed clinics' attention to enrollment and message routing rather than to the relevance and usability of a tool that is minimally adaptable to the safety net context. CONCLUSION: We found a mismatch between MU-based metrics of patient engagement and the priorities and needs of safety net patient populations.

OBJECTIVE: Patients with limited health literacy (LHL) and limited English proficiency (LEP) experience suboptimal communication and health outcomes. Electronic health record implementation in safety net clinics may affect communication with LHL and LEP patients.We investigated the associations between safety net clinician computer use and patient-provider communication for patients with LEP and LHL. MATERIALS AND METHODS: We video-recorded encounters at 5 academically affiliated US public hospital clinics between English- and Spanish-speaking patients with chronic conditions and their primary and specialty care clinicians. We analyzed changes in communication behaviors (coded with the Roter Interaction Analysis System) with each additional point on a clinician computer use score, controlling for clinician type and visit length and stratified by English proficiency and health literacy status. RESULTS: Greater clinician computer use was associated with more biomedical statements (+12.4, P = .03) and less positive affect (-0.6, P < .01) from LEP/LHL patients. In visits with patients with adequate English proficiency/health literacy, greater clinician computer use was associated with less positive patient affect (-0.9, P < .01), fewer clinician psychosocial statements (-3.5, P < .05), greater clinician verbal dominance (+0.09, P < .01), and lower ratings on quality of care and communication. CONCLUSION: Higher clinician computer use was associated with more biomedical focus with LEP/LHL patients, and clinician verbal dominance and lower ratings with patients with adequate English proficiency and health literacy. DISCUSSION: Implementation research should explore interventions to enhance relationship-centered communication for diverse patient populations in the computer era.

OBJECTIVE: With the rapid rise in the adoption of patient portals, many patients are gaining access to their personal health information online for the first time. The objective of this study was to examine specific usability barriers to patient portal engagement among a diverse group of patients and caregivers. MATERIALS AND METHODS: We conducted interviews using performance testing and think-aloud methods with 23 patients and 2 caregivers as they first attempted to use features of a newly launched patient portal. RESULTS: In navigating the portal, participants experienced basic computer barriers (eg, difficulty using a mouse), routine computer barriers (eg, mistyping, navigation issues), reading/writing barriers, and medical content barriers. Compared to participants with adequate health literacy, participants with limited health literacy required 2 additional minutes to complete each task and were more likely to experience each type of navigational barrier. They also experienced more inaccuracies in interpreting a test result and finding a treatment plan within an after-visit summary. DISCUSSION: When using a patient portal for the first time, participants with limited health literacy completed fewer tasks unassisted, had a higher prevalence of encountering barriers, took longer to complete tasks, and had more problems accurately interpreting medical information. CONCLUSION: Our findings suggest a strong need for tailored and accessible training and support to assist all vulnerable patients and/or caregivers with portal registration and use. Measuring the health literacy of a patient population might serve as a strong proxy for identifying patients who need the most support in using health technologies.

OBJECTIVE: Online patient portals are being widely implemented; however, no studies have examined whether portals influence health behaviors or outcomes similarly across patient racial/ethnic subgroups. We evaluated longitudinal changes in statin adherence to determine whether racial/ethnic minorities initiating use of the online refill function in patient portals had similar changes over time compared with Whites. METHODS: We examined a retrospective cohort of diabetes patients who were existing patient portal users. The primary exposure was initiating online refill use (either exclusively for all statin refills or occasionally for some refills), compared with using the portal for other tasks (eg, exchanging secure messages with providers). The primary outcome was change in statin adherence, measured as the percentage of time a patient was without a supply of statins. Adjusted generalized estimating equation models controlled for race/ethnicity as a primary interaction term. RESULTS: Fifty-eight percent of patient portal users were white, and all racial/ethnic minority groups had poorer baseline statin adherence compared with Whites. In adjusted difference-in-difference models, statin adherence improved significantly over time among patients who exclusively refilled prescriptions online, even after comparing changes over time with other portal users (4% absolute decrease in percentage of time without medication). This improvement was statistically similar across all racial/ethnic groups. DISCUSSION: Patient portals may encourage or improve key health behaviors, such as medication adherence, for engaged patients, but further research will likely be required to reduce underlying racial/ethnic differences in adherence. CONCLUSION: In a well-controlled examination of diabetes patients' behavior when using a new online feature for their healthcare management, patient portals were linked to better medication adherence across all racial/ethnic groups.

BACKGROUND: The high prevalence of limited health literacy among patients threatens the success of secure electronic messaging between patients from diverse populations and their providers. OBJECTIVE: The purpose of this study is to generate hypotheses about the readability of patient and provider electronic messages. METHODS: We collected 31 patient-provider e-mail exchanges (n = 119 total messages) from a safety-net primary care clinic. We compared the messages' mean word count and Flesch-Kincaid Grade Levels (FKGLs), calculated the frequency of provider messages below an FKGL = 8, and assessed readability concordance between patients' and providers' messages. RESULTS: Patients used more words in their initial e-mails compared to providers, but the FKGLs were similar, and 68% of provider messages were written below an FKGL = 8. Of 31 exchanges, 9 (29%) contained at least one patient message with an FKGL > 3 grade levels lower than the corresponding provider message(s). CONCLUSION: Our study demonstrates that most providers are able to respond to patient electronic messages with a matching reading level.

Patient-provider relationships influence diabetes care; less is known about their impact on online patient portal use. Diabetes patients rated provider communication and trust. In this study, we linked responses to electronic medical record data on being a registered portal user and using secure messaging (SM). We specified regression models to evaluate main effects on portal use, and subgroup analyses by race/ethnicity and age. 52% of subjects were registered users; among those, 36% used SM. Those reporting greater trust were more likely to be registered users (relative risk (RR)=1.14) or SM users (RR=1.29). In subgroup analyses, increased trust was associated with being a registered user among white, Latino, and older patients, as well as SM use among white patients. Better communication ratings were also related to being a registered user among older patients. Since increased trust and communication were associated with portal use within subgroups, this suggests that patient-provider relationships encourage portal engagement.