BACKGROUND: Chatbots are increasingly used to deliver health education, patient engagement, and access to healthcare services. GARDE-Chat is an open-source platform designed to facilitate the development, deployment, and dissemination of chatbot-based digital health interventions across different domains and settings. MATERIALS AND METHODS: GARDE-Chat was developed through an iterative process informed by real-world use cases to guide prioritization of key features. The tool was developed as an open-source platform to promote collaboration, broad dissemination, and impact across research and clinical domains. RESULTS: GARDE-Chat's main features include (1) a visual authoring interface that allows non-programmers to design chatbots; (2) support for scripted, large language model (LLM)-based and hybrid chatbots; (3) capacity to share chatbots with researchers and institutions; (4) integration with external applications and data sources such as electronic health records and REDCap; (5) delivery via web browsers or text messaging; and (6) detailed audit log supporting analyses of chatbot user interactions. Since its first release in July 2022, GARDE-Chat has supported the development of chatbot-based interventions tested in multiple studies, including large pragmatic clinical trials addressing topics such as genetic testing, COVID-19 testing, tobacco cessation, and cancer screening. DISCUSSION: Ongoing challenges include the effort required for developing chatbot scripts, ensuring safe use of LLMs, and integrating with clinical systems. CONCLUSION: GARDE-Chat is a generalizable platform for creating, implementing, and disseminating scalable chatbot-based population health interventions. It has been validated in several studies, and it is available to researchers and healthcare systems through an open-source mechanism.

Dear Editor, We agree with Drs. Tarabichi and Thornton on the importance of smoking history documentation in the electronic health record (EHR) for determining lung cancer screening eligibility. Drs. Tarabichi and Thornton also highlight an underappreciated advantage of lung cancer risk prediction: risk-based screening thresholds are less vulnerable to sub-optimally collected pack-year information. Their findings align with other work we have done where we found that unreliable pack-year information leads to less misclassification when using risk-based selection compared to using a pack-year eligibility criterion for patient selection.1 The growing body of literature on this topic, including those by ourselves, Drs. Tarabichi and Thornton, and others, underscores the potential for leveraging EHR capabilities to optimize the identification of high-risk patients for lung cancer screening.1–5 This body of literature underscores the ubiquitous and consequential nature of EHR smoking data inaccuracies and the need for additional research to generalize findings and to identify potential solutions.

OBJECTIVE: The US Preventive Services Task Force (USPSTF) requires the estimation of lifetime pack-years to determine lung cancer screening eligibility. Leading electronic health record (EHR) vendors calculate pack-years using only the most recently recorded smoking data. The objective was to characterize EHR smoking data issues and to propose an approach to addressing these issues using longitudinal smoking data. MATERIALS AND METHODS: In this cross-sectional study, we evaluated 16 874 current or former smokers who met USPSTF age criteria for screening (50-80 years old), had no prior lung cancer diagnosis, and were seen in 2020 at an academic health system using the Epic® EHR. We described and quantified issues in the smoking data. We then estimated how many additional potentially eligible patients could be identified using longitudinal data. The approach was verified through manual review of records from 100 subjects. RESULTS: Over 80% of evaluated records had inaccuracies, including missing packs-per-day or years-smoked (42.7%), outdated data (25.1%), missing years-quit (17.4%), and a recent change in packs-per-day resulting in inaccurate lifetime pack-years estimation (16.9%). Addressing these issues by using longitudinal data enabled the identification of 49.4% more patients potentially eligible for lung cancer screening (P < .001). DISCUSSION: Missing, outdated, and inaccurate smoking data in the EHR are important barriers to effective lung cancer screening. Data collection and analysis strategies that reflect changes in smoking habits over time could improve the identification of patients eligible for screening. CONCLUSION: The use of longitudinal EHR smoking data could improve lung cancer screening.

OBJECTIVE: HL7 SMART on FHIR apps have the potential to improve healthcare delivery and EHR usability, but providers must be aware of the apps and use them for these potential benefits to be realized. The HL7 CDS Hooks standard was developed in part for this purpose. The objective of this study was to determine if contextually relevant CDS Hooks prompts can increase utilization of a SMART on FHIR medical reference app (MDCalc for EHR). MATERIALS AND METHODS: We conducted a 7-month, provider-randomized trial with 70 providers in a single emergency department. The intervention was a collection of CDS Hooks prompts suggesting the use of 6 medical calculators in a SMART on FHIR medical reference app. The primary outcome was the percentage of provider-patient interactions in which the app was used to view a recommended calculator. Secondary outcomes were app usage stratified by individual calculators. RESULTS: Intervention group providers viewed a study calculator in the app in 6.0% of interactions compared to 2.6% in the control group (odds ratio = 2.45, 95% CI, 1.2-5.2, P value .02), an increase of 130%. App use was significantly greater for 2 of 6 calculators. DISCUSSION AND CONCLUSION: Contextually relevant CDS Hooks prompts led to a significant increase in SMART on FHIR app utilization. This demonstrates the potential of using CDS Hooks to guide appropriate use of SMART on FHIR apps and was a primary motivation for the development of the standard. Future research may evaluate potential impacts on clinical care decisions and outcomes.

OBJECTIVE: We sought to ascertain perceived factors affecting women's career development efforts in the American Medical Informatics Association (AMIA) and to provide recommendations for improvements. MATERIALS AND METHODS: Data were collected using a 27-item survey administered via the AMIA newsletter and other social channels. Survey questions comprised 3 demographics, 15 Likert-scale, and 9 open-ended items. Likert-scale responses were summarized across respondent ages, career stages, and career domains, and open-ended responses were thematically analyzed. RESULTS: We received survey responses from 109 AMIA women members. Our findings demonstrate that AMIA had made strides in promoting career development, and the most effective AMIA efforts included social events (83%), panel discussions (80%), and scientific sessions (79%). However, despite these efforts, women members perceived that gender-specific challenges persisted within AMIA, and recognized the need for increased networking opportunities (96%), raising awareness of gender-specific challenges (95%), and encouraging gender proportional representation in leadership (92%). DISCUSSION: International and national biomedical informatics professional communities have put forth efforts to address gender-specific issues in career development. Yet, our study identified that some of these, including the deep-rooted gender power hierarchy and bias, are still perceived as profound in AMIA. CONCLUSION: Even though existing career development efforts for women are highly effective, important perceived gender-specific career development issues require further attention and investigation to improve existing AMIA activities.

OBJECTIVE: Our objective was to develop an evaluation framework for electronic health record (EHR)-integrated innovations to support evaluation activities at each of four information technology (IT) life cycle phases: planning, development, implementation, and operation. METHODS: The evaluation framework was developed based on a review of existing evaluation frameworks from health informatics and other domains (human factors engineering, software engineering, and social sciences); expert consensus; and real-world testing in multiple EHR-integrated innovation studies. RESULTS: The resulting Evaluation in Life Cycle of IT (ELICIT) framework covers four IT life cycle phases and three measure levels (society, user, and IT). The ELICIT framework recommends 12 evaluation steps: (1) business case assessment; (2) stakeholder requirements gathering; (3) technical requirements gathering; (4) technical acceptability assessment; (5) user acceptability assessment; (6) social acceptability assessment; (7) social implementation assessment; (8) initial user satisfaction assessment; (9) technical implementation assessment; (10) technical portability assessment; (11) long-term user satisfaction assessment; and (12) social outcomes assessment. DISCUSSION: Effective evaluation requires a shared understanding and collaboration across disciplines throughout the entire IT life cycle. In contrast with previous evaluation frameworks, the ELICIT framework focuses on all phases of the IT life cycle across the society, user, and IT levels. Institutions seeking to establish evaluation programs for EHR-integrated innovations could use our framework to create such shared understanding and justify the need to invest in evaluation. CONCLUSION: As health care undergoes a digital transformation, it will be critical for EHR-integrated innovations to be systematically evaluated. The ELICIT framework can facilitate these evaluations.

OBJECTIVE: The study sought to evaluate a novel electronic health record (EHR) add-on application for chronic disease management that uses an integrated display to decrease user cognitive load, improve efficiency, and support clinical decision making. MATERIALS AND METHODS: We designed a chronic disease management application using the technology framework known as SMART on FHIR (Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources). We used mixed methods to obtain user feedback on a prototype to support ambulatory providers managing chronic obstructive pulmonary disease. Each participant managed 2 patient scenarios using the regular EHR with and without access to our prototype in block-randomized order. The primary outcome was the percentage of expert-recommended ideal care tasks completed. Timing, keyboard and mouse use, and participant surveys were also collected. User experiences were captured using a retrospective think-aloud interview analyzed by concept coding. RESULTS: With our prototype, the 13 participants completed more recommended care (81% vs 48%; P < .001) and recommended tasks per minute (0.8 vs 0.6; P = .03) over longer sessions (7.0 minutes vs 5.4 minutes; P = .006). Keystrokes per task were lower with the prototype (6 vs 18; P < .001). Qualitative themes elicited included the desire for reliable presentation of information which matches participants' mental models of disease and for intuitive navigation in order to decrease cognitive load. DISCUSSION: Participants completed more recommended care by taking more time when using our prototype. Interviews identified a tension between using the inefficient but familiar EHR vs learning to use our novel prototype. Concept coding of user feedback generated actionable insights. CONCLUSIONS: Mixed methods can support the design and evaluation of SMART on FHIR EHR add-on applications by enhancing understanding of the user experience.

OBJECTIVE: To determine the impact of a graphical information display on diagnosing circulatory shock. MATERIALS AND METHODS: This was an experimental study comparing integrated and conventional information displays. Participants were intensivists or critical care fellows (experts) and first-year medical residents (novices). RESULTS: The integrated display was associated with higher performance (87% vs 82%; P < .001), less time (2.9 vs 3.5 min; P = .008), and more accurate etiology (67% vs 54%; P = .048) compared to the conventional display. When stratified by experience, novice physicians using the integrated display had higher performance (86% vs 69%; P < .001), less time (2.9 vs 3.7 min; P = .03), and more accurate etiology (65% vs 42%; P = .02); expert physicians using the integrated display had nonsignificantly improved performance (87% vs 82%; P = .09), time (2.9 vs 3.3; P = .28), and etiology (69% vs 67%; P = .81). DISCUSSION: The integrated display appeared to support efficient information processing, which resulted in more rapid and accurate circulatory shock diagnosis. Evidence more strongly supported a difference for novices, suggesting that graphical displays may help reduce expert-novice performance gaps.