OBJECTIVE: To determine if ChatGPT can generate useful suggestions for improving clinical decision support (CDS) logic and to assess noninferiority compared to human-generated suggestions. METHODS: We supplied summaries of CDS logic to ChatGPT, an artificial intelligence (AI) tool for question answering that uses a large language model, and asked it to generate suggestions. We asked human clinician reviewers to review the AI-generated suggestions as well as human-generated suggestions for improving the same CDS alerts, and rate the suggestions for their usefulness, acceptance, relevance, understanding, workflow, bias, inversion, and redundancy. RESULTS: Five clinicians analyzed 36 AI-generated suggestions and 29 human-generated suggestions for 7 alerts. Of the 20 suggestions that scored highest in the survey, 9 were generated by ChatGPT. The suggestions generated by AI were found to offer unique perspectives and were evaluated as highly understandable and relevant, with moderate usefulness, low acceptance, bias, inversion, redundancy. CONCLUSION: AI-generated suggestions could be an important complementary part of optimizing CDS alerts, can identify potential improvements to alert logic and support their implementation, and may even be able to assist experts in formulating their own suggestions for CDS improvement. ChatGPT shows great potential for using large language models and reinforcement learning from human feedback to improve CDS alert logic and potentially other medical areas involving complex, clinical logic, a key step in the development of an advanced learning health system.

OBJECTIVE: We describe the Clickbusters initiative implemented at Vanderbilt University Medical Center (VUMC), which was designed to improve safety and quality and reduce burnout through the optimization of clinical decision support (CDS) alerts. MATERIALS AND METHODS: We developed a 10-step Clickbusting process and implemented a program that included a curriculum, CDS alert inventory, oversight process, and gamification. We carried out two 3-month rounds of the Clickbusters program at VUMC. We completed descriptive analyses of the changes made to alerts during the process, and of alert firing rates before and after the program. RESULTS: Prior to Clickbusters, VUMC had 419 CDS alerts in production, with 488 425 firings (42 982 interruptive) each week. After 2 rounds, the Clickbusters program resulted in detailed, comprehensive reviews of 84 CDS alerts and reduced the number of weekly alert firings by more than 70 000 (15.43%). In addition to the direct improvements in CDS, the initiative also increased user engagement and involvement in CDS. CONCLUSIONS: At VUMC, the Clickbusters program was successful in optimizing CDS alerts by reducing alert firings and resulting clicks. The program also involved more users in the process of evaluating and improving CDS and helped build a culture of continuous evaluation and improvement of clinical content in the electronic health record.

The COVID-19 national emergency has led to surging care demand and the need for unprecedented telehealth expansion. Rapid telehealth expansion can be especially complex for pediatric patients. From the experience of a large academic medical center, this report describes a pathway for efficiently increasing capacity of remote pediatric enrollment for telehealth while fulfilling privacy, security, and convenience concerns. The design and implementation of the process took 2 days. Five process requirements were identified: efficient enrollment, remote ability to establish parentage, minimal additional work for application processing, compliance with guidelines for adolescent autonomy, and compliance with institutional privacy and security policies. Weekly enrollment subsequently increased 10-fold for children (age 0-12 years) and 1.2-fold for adolescents (age 13-17 years). Weekly telehealth visits increased 200-fold for children and 90-fold for adolescents. The obstacles and solutions presented in this report can provide guidance to health systems for similar challenges during the COVID-19 response and future disasters.

Emergent policy changes related to telemedicine and the Emergency Medical Treatment and Labor Act during the novel coronavirus disease 2019 (COVID-19) pandemic have created opportunities for technology-based clinical evaluation, which serves to conserve personal protective equipment (PPE) and protect emergency providers. We define electronic PPE as an approach using telemedicine tools to perform electronic medical screening exams while satisfying the Emergency Medical Treatment and Labor Act. We discuss the safety, legal, and technical factors necessary for implementing such a pathway. This approach has the potential to conserve PPE and protect providers while maintaining safe standards for medical screening exams in the emergency department for low-risk patients in whom COVID-19 is suspected.

OBJECTIVE: The general equivalence mappings (GEMs) and reimbursement mappings (RMs) facilitate translation between ICD-9-CM and ICD-10-CM. This study compared prospectively dual-encoded diagnoses assigned by professional coders with the GEMs/RMs in a clinical setting. MATERIALS AND METHODS: Professional coders manually encoded diagnoses from 100 primary care notes into both ICD-9-CM and ICD-10-CM. The investigators evaluated whether manual mappings were reproducible using the GEMs/RMs. Reproducible mappings with one ICD-9-CM and one ICD-10-CM code ("one-to-one") were classified as exact or approximate using GEMs flags. Mismatches were characterized manually. RESULTS: Manual encodings were reproducible from the forward GEMs, backward GEMs, and RMs in 85.2%, 90.4%, and 88.1% of diagnoses, respectively. For one-to-one, reproducible mappings, 61% (forward) and 63% (backward) were approximate mappings compared to 85% and 95% in the GEMs as a whole. Mismatches between manual and GEMs encodings were due to differences in coder interpretation (11%-13%), subtle hierarchical differences (52%-55%), or unknown reasons (32%-35%). DISCUSSION: This study highlights inconsistencies between manual encoding and using the GEMs/RMs. The number of approximate mappings in our population compared to all one-to-one GEMs entries supports the notion that statistics describing the GEMs as a whole might not represent the most important mappings for each organization. The mismatch characteristics highlight the subtle differences between manual encoding and using the GEMs/RMs. CONCLUSION: These results support the need for organizations to assess the GEMs and RMs in their own environment to avoid changes in reimbursement and longitudinal statistics.