IMPORTANCE: The US healthcare system faces significant challenges, including clinician burnout, operational inefficiencies, and concerns about patient safety. Artificial intelligence (AI), particularly generative AI, has the potential to address these challenges, but its adoption, effectiveness, and barriers to implementation are not well understood. OBJECTIVE: To evaluate the current state of AI adoption in US healthcare systems, assess successes and barriers to implementation during the early generative AI era. DESIGN, SETTING, AND PARTICIPANTS: This cross-sectional survey was conducted in Fall 2024, and included 67 health systems members of the Scottsdale Institute, a collaborative of US non-profit healthcare organizations. Forty-three health systems completed the survey (64% response rate). Respondents provided data on the deployment status and perceived success of 37 AI use cases across 10 categories. MAIN OUTCOMES AND MEASURES: The primary outcomes were the extent of AI use case development, piloting, or deployment, the degree of reported success for AI use cases, and the most significant barriers to adoption. RESULTS: Across the 43 responding health systems, AI adoption and perceptions of success varied significantly. Ambient Notes, a generative AI tool for clinical documentation, was the only use case with 100% of respondents reporting adoption activities, and 53% reported a high degree of success with using AI for Clinical Documentation. Imaging and radiology emerged as the most widely deployed clinical AI use case, with 90% of organizations reporting at least partial deployment, although successes with diagnostic use cases were limited. Similarly, many organizations have deployed AI for clinical risk stratification such as early sepsis detection, but only 38% report high success in this area. Immature AI tools were identified a significant barrier to adoption, cited by 77% of respondents, followed by financial concerns (47%) and regulatory uncertainty (40%). CONCLUSIONS AND RELEVANCE: Ambient Notes is rapidly advancing in US healthcare systems and demonstrating early success. Other AI use cases show varying degrees of adoption and success, constrained by barriers such as immature AI tools, financial concerns, and regulatory uncertainty. Addressing these challenges through robust evaluations, shared strategies, and governance models will be essential to ensure effective integration and adoption of AI into healthcare practice.

OBJECTIVES: We sought to (1) characterize the process of diagnosing pneumonia in an emergency department (ED) and (2) examine clinician reactions to a clinician-facing diagnostic discordance feedback tool. MATERIALS AND METHODS: We designed a diagnostic feedback tool, using electronic health record data from ED clinicians' patients to establish concordance or discordance between ED diagnosis, radiology reports, and hospital discharge diagnosis for pneumonia. We conducted semistructured interviews with 11 ED clinicians about pneumonia diagnosis and reactions to the feedback tool. We administered surveys measuring individual differences in mindset beliefs, comfort with feedback, and feedback tool usability. We qualitatively analyzed interview transcripts and descriptively analyzed survey data. RESULTS: Thematic results revealed: (1) the diagnostic process for pneumonia in the ED is characterized by diagnostic uncertainty and may be secondary to goals to treat and dispose the patient; (2) clinician diagnostic self-evaluation is a fragmented, inconsistent process of case review and follow-up that a feedback tool could fill; (3) the feedback tool was described favorably, with task and normative feedback harnessing clinician values of high-quality patient care and personal excellence; and (4) strong reactions to diagnostic feedback varied from implicit trust to profound skepticism about the validity of the concordance metric. Survey results suggested a relationship between clinicians' individual differences in learning and failure beliefs, feedback experience, and usability ratings. DISCUSSION AND CONCLUSION: Clinicians value feedback on pneumonia diagnoses. Our results highlight the importance of feedback about diagnostic performance and suggest directions for considering individual differences in feedback tool design and implementation.

BACKGROUND: Studies examining the effects of computerized order entry (CPOE) on medication ordering errors demonstrate that CPOE does not consistently prevent these errors as intended. We used the Agency for Healthcare Research and Quality (AHRQ) Network of Patient Safety Databases (NPSD) to investigate the frequency and degree of harm of reported events that occurred at the ordering stage, characterized by error type. MATERIALS AND METHODS: This was a retrospective observational study of safety events reported by healthcare systems in participating patient safety organizations from 6/2010 through 12/2020. All medication and other substance ordering errors reported to NPSD via common format v1.2 between 6/2010 through 12/2020 were analyzed. We aggregated and categorized the frequency of reported medication ordering errors by error type, degree of harm, and demographic characteristics. RESULTS: A total of 12 830 errors were reported during the study period. Incorrect dose accounted for 3812 errors (29.7%), followed by incorrect medication 2086 (16.3%), and incorrect duration 765 (6.0%). Of 5282 events that reached the patient and had a known level of severity, 12 resulted in death, 4 resulted in severe harm, 45 resulted in moderate harm, 341 resulted in mild harm, and 4880 resulted in no harm. CONCLUSION: Incorrect dose and incorrect drug orders were the most commonly reported and harmful types of medication ordering errors. Future studies should aim to develop and test interventions focused on CPOE to prevent medication ordering errors, prioritizing wrong-dose and wrong-drug errors.

OBJECTIVE: Little is known regarding variation among electronic health record (EHR) vendors in quality performance. This issue is compounded by selection effects in which high-quality hospitals coalesce to a subset of market leading vendors. We measured hospital performance, stratified by EHR vendor, across 4 quality metrics. MATERIALS AND METHODS: We used data on 1272 hospitals in 2018 across 4 quality measures: Leapfrog Computerized Provider Order Entry/EHR Evaluation, Centers for Medicare and Medicaid Services Hospital Compare Star Ratings, Hospital-Acquired Condition (HAC) score, and Hospital Readmission Reduction Program (HRRP) ratio. We examined score distributions and used multivariable regression to evaluate the association between vendor and score, recovering partial R2 to assess the proportion of quality variation explained by vendor. RESULTS: We found significant variation across and within EHR vendors. The largest vendor, vendor A, had the highest mean score on the Leapfrog Computerized Provider Order Entry/EHR Evaluation and HRRP ratio, vendor G had the highest Hospital Compare score, and vendor F had the highest HAC score. In adjusted models, no vendor was significantly associated with higher performance on more than 2 measures. EHR vendor explained between 1.2% (HAC) and 7.6 (HRRP) of the variation in quality performance. DISCUSSION: No EHR vendor was associated with higher quality across all measures, and the 2 largest vendors were not associated with the highest scores. Only a small fraction of quality variation was explained by EHR vendor choice. CONCLUSIONS: Top performance on quality measures can be achieved with any EHR vendor; much of quality performance is driven by the hospital and how it uses the EHR.

OBJECTIVE: The study sought to evaluate the overall performance of hospitals that used the Computerized Physician Order Entry Evaluation Tool in both 2017 and 2018, along with their performance against fatal orders and nuisance orders. MATERIALS AND METHODS: We evaluated 1599 hospitals that took the test in both 2017 and 2018 by using their overall percentage scores on the test, along with the percentage of fatal orders appropriately alerted on, and the percentage of nuisance orders incorrectly alerted on. RESULTS: Hospitals showed overall improvement; the mean score in 2017 was 58.1%, and this increased to 66.2% in 2018. Fatal order performance improved slightly from 78.8% to 83.0% (P < .001), though there was almost no change in nuisance order performance (89.0% to 89.7%; P = .43). Hospitals alerting on one or more nuisance orders had a 3-percentage-point increase in their overall score. DISCUSSION: Despite the improvement of overall scores in 2017 and 2018, there was little improvement in fatal order performance, suggesting that hospitals are not targeting the deadliest orders first. Nuisance order performance showed almost no improvement, and some hospitals may be achieving higher scores by overalerting, suggesting that the thresholds for which alerts are fired from are too low. CONCLUSIONS: Although hospitals improved overall from 2017 to 2018, there is still important room for improvement for both fatal and nuisance orders. Hospitals that incorrectly alerted on one or more nuisance orders had slightly higher overall performance, suggesting that some hospitals may be achieving higher scores at the cost of overalerting, which has the potential to cause clinician burnout and even worsen safety.

Objective: To evaluate the safety of computerized physician order entry (CPOE) and associated clinical decision support (CDS) systems in electronic health record (EHR) systems at pediatric inpatient facilities in the US using the Leapfrog Group's pediatric CPOE evaluation tool. Methods: The Leapfrog pediatric CPOE evaluation tool, a previously validated tool to assess the ability of a CPOE system to identify orders that could potentially lead to patient harm, was used to evaluate 41 pediatric hospitals over a 2-year period. Evaluation of the last available test for each institution was performed, assessing performance overall as well as by decision support category (eg, drug-drug, dosing limits). Longitudinal analysis of test performance was also carried out to assess the impact of testing and the overall trend of CPOE performance in pediatric hospitals. Results: Pediatric CPOE systems were able to identify 62% of potential medication errors in the test scenarios, but ranged widely from 23-91% in the institutions tested. The highest scoring categories included drug-allergy interactions, dosing limits (both daily and cumulative), and inappropriate routes of administration. We found that hospitals with longer periods since their CPOE implementation did not have better scores upon initial testing, but after initial testing there was a consistent improvement in testing scores of 4 percentage points per year. Conclusions: Pediatric computerized physician order entry (CPOE) systems on average are able to intercept a majority of potential medication errors, but vary widely among implementations. Prospective and repeated testing using the Leapfrog Group's evaluation tool is associated with improved ability to intercept potential medication errors.

The Office of the National Coordinator for Health Information Technology is expected to oversee creation of a Health Information Technology (HIT) Safety Center. While its functions are still being defined, the center is envisioned as a public-private entity focusing on promotion of HIT related patient safety. We propose that the HIT Safety Center leverages its unique position to work with key administrative and policy stakeholders, healthcare organizations (HCOs), and HIT vendors to achieve four goals: (1) facilitate creation of a nationwide 'post-marketing' surveillance system to monitor HIT related safety events; (2) develop methods and governance structures to support investigation of major HIT related safety events; (3) create the infrastructure and methods needed to carry out random assessments of HIT related safety in complex HCOs; and (4) advocate for HIT safety with government and private entities. The convening ability of a federally supported HIT Safety Center could be critically important to our transformation to a safe and effective HIT enabled healthcare system.

Health care providers have a basic responsibility to protect patients from accidental harm. At the institutional level, creating safe health care organizations necessitates a systematic approach. Effective use of informatics to enhance safety requires the establishment and use of standards for concept definitions and for data exchange, development of acceptable models for knowledge representation, incentives for adoption of electronic health records, support for adverse event detection and reporting, and greater investment in research at the intersection of informatics and patient safety. Leading organizations have demonstrated that health care informatics approaches can improve safety. Nevertheless, significant obstacles today limit optimal application of health informatics to safety within most provider environments. The authors offer a series of recommendations for addressing these challenges.

Computerized physician order entry (CPOE) is an application that is used to electronically write physician orders either in the hospital or in the outpatient setting. It is used in about 15% of U.S. Hospitals and a smaller percentage of ambulatory clinics. It is linked with clinical decision support, which provides much of the value of implementing it. A number of studies have assessed the impact of CPOE with respect to a variety of parameters, including costs of care, medication safety, use of guidelines or protocols, and other measures of the effectiveness or quality of care. Most of these studies have been undertaken at CPOE exemplar sites with homegrown clinical information systems. With the increasing implementation of commercial CPOE systems in various settings of care has come evidence that some implementation approaches may not achieve previously published results or may actually cause new errors or even harm. This has lead to new initiatives to evaluate CPOE systems, which have been undertaken by both vendors and other groups who evaluate vendors, focused on CPOE vendor capabilities and effective approaches to implementation that can achieve benefits seen in published studies. In addition, an electronic health record (EHR) vendor certification process is ongoing under the province of the Certification Commission for Health Information Technology (CCHIT) (which includes CPOE) that will affect the purchase and use of these applications by hospitals and clinics and their participation in public and private health insurance programs. Large employers have also joined this focus by developing flight simulation tools to evaluate the capabilities of these CPOE systems once implemented, potentially linking the results of such programs to reimbursement through pay for performance programs. The increasing role of CPOE systems in health care has invited much more scrutiny about the effectiveness of these systems in actual practice which has the potential to improve their ultimate performance.

While medications can improve patients' health, the process of prescribing them is complex and error prone, and medication errors cause many preventable injuries. Computer provider order entry (CPOE) with clinical decision support (CDS), can improve patient safety and lower medication-related costs. To realize the medication-related benefits of CDS within CPOE, one must overcome significant challenges. Healthcare organizations implementing CPOE must understand what classes of CDS their CPOE systems can support, assure that clinical knowledge underlying their CDS systems is reasonable, and appropriately represent electronic patient data. These issues often influence to what extent an institution will succeed with its CPOE implementation and achieve its desired goals. Medication-related decision support is probably best introduced into healthcare organizations in two stages, basic and advanced. Basic decision support includes drug-allergy checking, basic dosing guidance, formulary decision support, duplicate therapy checking, and drug-drug interaction checking. Advanced decision support includes dosing support for renal insufficiency and geriatric patients, guidance for medication-related laboratory testing, drug-pregnancy checking, and drug-disease contraindication checking. In this paper, the authors outline some of the challenges associated with both basic and advanced decision support and discuss how those challenges might be addressed. The authors conclude with summary recommendations for delivering effective medication-related clinical decision support addressed to healthcare organizations, application and knowledge base vendors, policy makers, and researchers.