OBJECTIVES: Post-discharge adverse events (AEs) are common and heralded by new and worsening symptoms (NWS). We evaluated the effect of electronic health record (EHR)-integrated digital tools designed to promote quality and safety in hospitalized patients on NWS and AEs after discharge. MATERIALS AND METHODS: Adult general medicine patients at a community hospital were enrolled. We implemented a dashboard which clinicians used to assess safety risks during interdisciplinary rounds. Post-implementation patients were randomized to complete a discharge checklist whose responses were incorporated into the dashboard. Outcomes were assessed using EHR review and 30-day call data adjudicated by 2 clinicians and analyzed using Poisson regression. We conducted comparisons of each exposure on post-discharge outcomes and used selected variables and NWS as independent predictors to model post-discharge AEs using multivariable logistic regression. RESULTS: A total of 260 patients (122 pre, 71 post [dashboard], 67 post [dashboard plus discharge checklist]) enrolled. The adjusted incidence rate ratios (aIRR) for NWS and AEs were unchanged in the post- compared to pre-implementation period. For patient-reported NWS, aIRR was non-significantly higher for dashboard plus discharge checklist compared to dashboard participants (1.23 [0.97,1.56], P = .08). For post-implementation patients with an AE, aIRR for duration of injury (>1 week) was significantly lower for dashboard plus discharge checklist compared to dashboard participants (0 [0,0.53], P < .01). In multivariable models, certain patient-reported NWS were associated with AEs (3.76 [1.89,7.82], P < .01). DISCUSSION: While significant reductions in post-discharge AEs were not observed, checklist participants experiencing a post-discharge AE were more likely to report NWS and had a shorter duration of injury. CONCLUSION: Interventions designed to prompt patients to report NWS may facilitate earlier detection of AEs after discharge. CLINICALTRIALS.GOV: NCT05232656.

OBJECTIVE: To evaluate the effect of electronic health record (EHR)-integrated digital health tools comprised of a checklist and video on transitions-of-care outcomes for patients preparing for discharge. MATERIALS AND METHODS: English-speaking, general medicine patients (>18 years) hospitalized at least 24 hours at an academic medical center in Boston, MA were enrolled before and after implementation. A structured checklist and video were administered on a mobile device via a patient portal or web-based survey at least 24 hours prior to anticipated discharge. Checklist responses were available for clinicians to review in real time via an EHR-integrated safety dashboard. The primary outcome was patient activation at discharge assessed by patient activation (PAM)-13. Secondary outcomes included postdischarge patient activation, hospital operational metrics, healthcare resource utilization assessed by 30-day follow-up calls and administrative data and change in patient activation from discharge to 30 days postdischarge. RESULTS: Of 673 patients approached, 484 (71.9%) enrolled. The proportion of activated patients (PAM level 3 or 4) at discharge was nonsignificantly higher for the 234 postimplementation compared with the 245 preimplementation participants (59.8% vs 56.7%, adjusted OR 1.23 [0.38, 3.96], P = .73). Postimplementation participants reported 3.75 (3.02) concerns via the checklist. Mean length of stay was significantly higher for postimplementation compared with preimplementation participants (10.13 vs 6.21, P < .01). While there was no effect on postdischarge outcomes, there was a nonsignificant decrease in change in patient activation within participants from pre- to postimplementation (adjusted difference-in-difference of -16.1% (9.6), P = .09). CONCLUSIONS: EHR-integrated digital health tools to prepare patients for discharge did not significantly increase patient activation and was associated with a longer length of stay. While issues uncovered by the checklist may have encouraged patients to inquire about their discharge preparedness, other factors associated with patient activation and length of stay may explain our observations. We offer insights for using PAM-13 in context of real-world health-IT implementations. TRIAL REGISTRATION: NIH US National Library of Medicine, NCT03116074, clinicaltrials.gov.

OBJECTIVE: To determine user and electronic health records (EHR) integration requirements for a scalable remote symptom monitoring intervention for asthma patients and their providers. METHODS: Guided by the Non-Adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework, we conducted a user-centered design process involving English- and Spanish-speaking patients and providers affiliated with an academic medical center. We conducted a secondary analysis of interview transcripts from our prior study, new design sessions with patients and primary care providers (PCPs), and a survey of PCPs. We determined EHR integration requirements as part of the asthma app design and development process. RESULTS: Analysis of 26 transcripts (21 patients, 5 providers) from the prior study, 21 new design sessions (15 patients, 6 providers), and survey responses from 55 PCPs (71% of 78) identified requirements. Patient-facing requirements included: 1- or 5-item symptom questionnaires each week, depending on asthma control; option to request a callback; ability to enter notes, triggers, and peak flows; and tips pushed via the app prior to a clinic visit. PCP-facing requirements included a clinician-facing dashboard accessible from the EHR and an EHR inbox message preceding the visit. PCP preferences diverged regarding graphical presentations of patient-reported outcomes (PROs). Nurse-facing requirements included callback requests sent as an EHR inbox message. Requirements were consistent for English- and Spanish-speaking patients. EHR integration required use of custom application programming interfaces (APIs). CONCLUSION: Using the NASSS framework to guide our user-centered design process, we identified patient and provider requirements for scaling an EHR-integrated remote symptom monitoring intervention in primary care. These requirements met the needs of patients and providers. Additional standards for PRO displays and EHR inbox APIs are needed to facilitate spread.

OBJECTIVE: The objective of this paper is to share challenges, recommendations, and lessons learned regarding the development and implementation of a Patient Safety Learning Laboratory (PSLL) project, an innovative and complex intervention comprised of a suite of Health Information Technology (HIT) tools integrated with a newly implemented Electronic Health Record (EHR) vendor system in the acute care setting at a large academic center. MATERIALS AND METHODS: The PSLL Administrative Core engaged stakeholders and study personnel throughout all phases of the project: problem analysis, design, development, implementation, and evaluation. Implementation challenges and recommendations were derived from direct observations and the collective experience of PSLL study personnel. RESULTS: The PSLL intervention was implemented on 12 inpatient units during the 18-month study period, potentially impacting 12,628 patient admissions. Challenges to implementation included stakeholder engagement, project scope/complexity, technology/governance, and team structure. Recommendations to address each of these challenges were generated, some enacted during the trial, others as lessons learned for future iterative refinements of the intervention and its implementation. CONCLUSION: Designing, implementing, and evaluating a suite of tools integrated within a vendor EHR to improve patient safety has a variety of challenges. Keys to success include continuous stakeholder engagement, involvement of systems and human factors engineers within a multidisciplinary team, an iterative approach to user-centered design, and a willingness to think outside of current workflows and processes to change health system culture around adverse event prevention.

We established a Patient Safety Learning Laboratory comprising 2 core and 3 individual project teams to introduce a suite of digital health tools integrated with our electronic health record to identify, assess, and mitigate threats to patient safety in real time. One of the core teams employed systems engineering (SE) and human factors (HF) methods to analyze problems, design and develop improvements to intervention components, support implementation, and evaluate the system of systems as an integrated whole. Of the 29 participants, 19 and 16 participated in surveys and focus groups, respectively, about their perception of SE and HF. We identified 7 themes regarding use of the 12 SE and HF methods over the 4-year project. Qualitative methods (interviews, focus, groups, observations, usability testing) were most frequently used, typically by individual project teams, and generated the most insight. Quantitative methods (failure mode and effects analysis, simulation modeling) typically were used by the SE and HF core team but generated variable insight. A decentralized project structure led to challenges using these SE and HF methods at the project and systems level. We offer recommendations and insights for using SE and HF to support digital health patient safety initiatives.

As part of an interdisciplinary acute care patient portal task force with members from 10 academic medical centers and professional organizations, we held a national workshop with 71 attendees representing over 30 health systems, professional organizations, and technology companies. Our consensus approach identified 7 key sociotechnical and evaluation research focus areas related to the consumption and capture of information from patients, care partners (eg, family, friends), and clinicians through portals in the acute and post-acute care settings. The 7 research areas were: (1) standards, (2) privacy and security, (3) user-centered design, (4) implementation, (5) data and content, (6) clinical decision support, and (7) measurement. Patient portals are not yet in routine use in the acute and post-acute setting, and research focused on the identified domains should increase the likelihood that they will deliver benefit, especially as there are differences between needs in acute and post-acute care compared to the ambulatory setting.

OBJECTIVE: To describe current practices and stakeholder perspectives of patient portals in the acute care setting. We aimed to: (1) identify key features, (2) recognize challenges, (3) understand current practices for design, configuration, and use, and (4) propose new directions for investigation and innovation. MATERIALS AND METHODS: Mixed methods including surveys, interviews, focus groups, and site visits with stakeholders at leading academic medical centers. Thematic analyses to inform development of an explanatory model and recommendations. RESULTS: Site surveys were administered to 5 institutions. Thirty interviews/focus groups were conducted at 4 site visits that included a total of 84 participants. Ten themes regarding content and functionality, engagement and culture, and access and security were identified, from which an explanatory model of current practices was developed. Key features included clinical data, messaging, glossary, patient education, patient personalization and family engagement tools, and tiered displays. Four actionable recommendations were identified by group consensus. DISCUSSION: Design, development, and implementation of acute care patient portals should consider: (1) providing a single integrated experience across care settings, (2) humanizing the patient-clinician relationship via personalization tools, (3) providing equitable access, and (4) creating a clear organizational mission and strategy to achieve outcomes of interest. CONCLUSION: Portals should provide a single integrated experience across the inpatient and ambulatory settings. Core functionality includes tools that facilitate communication, personalize the patient, and deliver education to advance safe, coordinated, and dignified patient-centered care. Our findings can be used to inform a "road map" for future work related to acute care patient portals.

Communication in acute care settings is fragmented and occurs asynchronously via a variety of electronic modalities. Providers are often not on the same page with regard to the plan of care. We designed and developed a secure, patient-centered "microblog" messaging platform that identifies care team members by synchronizing with the electronic health record, and directs providers to a single forum where they can communicate about the plan of care. The system was used for 35% of patients admitted to a medical intensive care unit over a 6-month period. Major themes in messages included care coordination (49%), clinical summarization (29%), and care team collaboration (27%). Message transparency and persistence were seen as useful features by 83% and 62% of respondents, respectively. Availability of alternative messaging tools and variable use by non-unit providers were seen as main barriers to adoption by 83% and 62% of respondents, respectively. This approach has much potential to improve communication across settings once barriers are addressed.

We implemented a web-based, patient-centered toolkit that engages patients/caregivers in the hospital plan of care by facilitating education and patient-provider communication. Of the 585 eligible patients approached on medical intensive care and oncology units, 239 were enrolled (119 patients, 120 caregivers). The most common reason for not approaching the patient was our inability to identify a health care proxy when a patient was incapacitated. Significantly more caregivers were enrolled in medical intensive care units compared with oncology units (75% vs 32%; P < .01). Of the 239 patient/caregivers, 158 (66%) and 97 (41%) inputted a daily and overall goal, respectively. Use of educational content was highest for medications and test results and infrequent for problems. The most common clinical theme identified in 291 messages sent by 158 patients/caregivers was health concerns, needs, preferences, or questions (19%, 55 of 291). The average system usability scores and satisfaction ratings of a sample of surveyed enrollees were favorable. From analysis of feedback, we identified barriers to adoption and outlined strategies to promote use.

OBJECTIVE: To assess whether integrating critical result management software--Alert Notification of Critical Results (ANCR)--with an electronic health record (EHR)-based results management application impacts closed-loop communication and follow-up of nonurgent, clinically significant radiology results by primary care providers (PCPs). MATERIALS AND METHODS: This institutional review board-approved study was conducted at a large academic medical center. Postintervention, PCPs could acknowledge nonurgent, clinically significant ANCR-generated alerts ("alerts") within ANCR or the EHR. Primary outcome was the proportion of alerts acknowledged via EHR over a 24-month postintervention. Chart abstractions for a random sample of alerts 12 months preintervention and 24 months postintervention were reviewed, and the follow-up rate of actionable alerts (eg, performing follow-up imaging, administering antibiotics) was estimated. Pre- and postintervention rates were compared using the Fisher exact test. Postintervention follow-up rate was compared for EHR-acknowledged alerts vs ANCR. RESULTS: Five thousand nine hundred and thirty-one alerts were acknowledged by 171 PCPs, with 100% acknowledgement (consistent with expected ANCR functionality). PCPs acknowledged 16% (688 of 4428) of postintervention alerts in the EHR, with the remaining in ANCR. Follow-up was documented for 85 of 90 (94%; 95% CI, 88%-98%) preintervention and 79 of 84 (94%; 95% CI, 87%-97%) postintervention alerts (P > .99). Postintervention, 11 of 14 (79%; 95% CI, 52%-92%) alerts were acknowledged via EHR and 68 of 70 (97%; 95% CI, 90%-99%) in ANCR had follow-up (P = .03). CONCLUSIONS: Integrating ANCR and EHR provides an additional workflow for acknowledging nonurgent, clinically significant results without significant change in rates of closed-loop communication or follow-up of alerts.

Failure to follow-up nonurgent, clinically significant test results (CSTRs) is an ambulatory patient safety concern. Tools within electronic health records (EHRs) may facilitate test result acknowledgment, but their utility with regard to nonurgent CSTRs is unclear. We measured use of an acknowledgment tool by 146 primary care physicians (PCPs) at 13 network-affiliated practices that use the same EHR. We then surveyed PCPs to assess use of, satisfaction with, and desired enhancements to the acknowledgment tool. The rate of acknowledgment of non-urgent CSTRs by PCPs was 78%. Of 73 survey respondents, 72 reported taking one or more actions after reviewing a CSTR; fewer (40-75%) reported that using the acknowledgment tool was helpful for a specific purpose. Forty-six (64%) were satisfied with the tool. Both satisfied and nonsatisfied PCPs reported that enhancements linking acknowledgment to routine actions would be useful. EHR vendors should consider enhancements to acknowledgment functionality to ensure follow-up of nonurgent CSTRs.

BACKGROUND AND OBJECTIVE: Physician awareness of the results of tests pending at discharge (TPADs) is poor. We developed an automated system that notifies responsible physicians of TPAD results via secure, network email. We sought to evaluate the impact of this system on self-reported awareness of TPAD results by responsible physicians, a necessary intermediary step to improve management of TPAD results. METHODS: We conducted a cluster-randomized controlled trial at a major hospital affiliated with an integrated healthcare delivery network in Boston, Massachusetts. Adult patients with TPADs who were discharged from inpatient general medicine and cardiology services were assigned to the intervention or usual care arm if their inpatient attending physician and primary care physician (PCP) were both randomized to the same study arm. Patients of physicians randomized to discordant study arms were excluded. We surveyed these physicians 72 h after all TPAD results were finalized. The primary outcome was awareness of TPAD results by attending physicians. Secondary outcomes included awareness of TPAD results by PCPs, awareness of actionable TPAD results, and provider satisfaction. RESULTS: We analyzed data on 441 patients. We sent 441 surveys to attending physicians and 353 surveys to PCPs and received 275 and 152 responses from 83 different attending physicians and 112 different PCPs, respectively (attending physician survey response rate of 63%). Intervention attending physicians and PCPs were significantly more aware of TPAD results (76% vs 38%, adjusted/clustered OR 6.30 (95% CI 3.02 to 13.16), p<0.001; 57% vs 33%, adjusted/clustered OR 3.08 (95% CI 1.43 to 6.66), p=0.004, respectively). Intervention attending physicians tended to be more aware of actionable TPAD results (59% vs 29%, adjusted/clustered OR 4.25 (0.65, 27.85), p=0.13). One hundred and eighteen (85%) and 43 (63%) intervention attending physician and PCP survey respondents, respectively, were satisfied with this intervention. CONCLUSIONS: Automated email notification represents a promising strategy for managing TPAD results, potentially mitigating an unresolved patient safety concern. CLINICAL TRIAL REGISTRATION: ClinicalTrials.gov (NCT01153451).

Physicians are often unaware of the results of tests pending at discharge (TPADs). The authors designed and implemented an automated system to notify the responsible inpatient physician of the finalized results of TPADs using secure, network email. The system coordinates a series of electronic events triggered by the discharge time stamp and sends an email to the identified discharging attending physician once finalized results are available. A carbon copy is sent to the primary care physicians in order to facilitate communication and the subsequent transfer of responsibility. Logic was incorporated to suppress selected tests and to limit notification volume. The system was activated for patients with TPADs discharged by randomly selected inpatient-attending physicians during a 6-month pilot. They received approximately 1.6 email notifications per discharged patient with TPADs. Eighty-four per cent of inpatient-attending physicians receiving automated email notifications stated that they were satisfied with the system in a brief survey (59% survey response rate). Automated email notification is a useful strategy for managing results of TPADs.

OBJECTIVE: To report the frequency, types, and causes of errors associated with outpatient computer-generated prescriptions, and to develop a framework to classify these errors to determine which strategies have greatest potential for preventing them. MATERIALS AND METHODS: This is a retrospective cohort study of 3850 computer-generated prescriptions received by a commercial outpatient pharmacy chain across three states over 4 weeks in 2008. A clinician panel reviewed the prescriptions using a previously described method to identify and classify medication errors. Primary outcomes were the incidence of medication errors; potential adverse drug events, defined as errors with potential for harm; and rate of prescribing errors by error type and by prescribing system. RESULTS: Of 3850 prescriptions, 452 (11.7%) contained 466 total errors, of which 163 (35.0%) were considered potential adverse drug events. Error rates varied by computerized prescribing system, from 5.1% to 37.5%. The most common error was omitted information (60.7% of all errors). DISCUSSION: About one in 10 computer-generated prescriptions included at least one error, of which a third had potential for harm. This is consistent with the literature on manual handwritten prescription error rates. The number, type, and severity of errors varied by computerized prescribing system, suggesting that some systems may be better at preventing errors than others. CONCLUSIONS: Implementing a computerized prescribing system without comprehensive functionality and processes in place to ensure meaningful system use does not decrease medication errors. The authors offer targeted recommendations on improving computerized prescribing systems to prevent errors.