BACKGROUND: The use of real-world data (RWD) in artificial intelligence (AI) applications for healthcare offers unique opportunities but also poses complex challenges related to interpretability, transparency, safety, efficacy, bias, equity, privacy, ethics, accountability, and stakeholder engagement. METHODS: A multi-stakeholder expert panel comprising healthcare professionals, AI developers, policymakers, and other stakeholders was assembled. Their task was to identify critical issues and formulate consensus recommendations, focusing on the responsible use of RWD in healthcare AI. The panel's work involved an in-person conference and workshop and extensive deliberations over several months. RESULTS: The panel's findings revealed several critical challenges, including the necessity for data literacy and documentation, the identification and mitigation of bias, privacy and ethics considerations, and the absence of an accountability structure for stakeholder management. To address these, the panel proposed a series of recommendations, such as the adoption of metadata standards for RWD sources, the development of transparency frameworks and instructional labels likened to "nutrition labels" for AI applications, the provision of cross-disciplinary training materials, the implementation of bias detection and mitigation strategies, and the establishment of ongoing monitoring and update processes. CONCLUSION: Guidelines and resources focused on the responsible use of RWD in healthcare AI are essential for developing safe, effective, equitable, and trustworthy applications. The proposed recommendations provide a foundation for a comprehensive framework addressing the entire lifecycle of healthcare AI, emphasizing the importance of documentation, training, transparency, accountability, and multi-stakeholder engagement.

BACKGROUND: Integrating artificial intelligence (AI) in healthcare settings has the potential to benefit clinical decision-making. Addressing challenges such as ensuring trustworthiness, mitigating bias, and maintaining safety is paramount. The lack of established methodologies for pre- and post-deployment evaluation of AI tools regarding crucial attributes such as transparency, performance monitoring, and adverse event reporting makes this situation challenging. OBJECTIVES: This paper aims to make practical suggestions for creating methods, rules, and guidelines to ensure that the development, testing, supervision, and use of AI in clinical decision support (CDS) systems are done well and safely for patients. MATERIALS AND METHODS: In May 2023, the Division of Clinical Informatics at Beth Israel Deaconess Medical Center and the American Medical Informatics Association co-sponsored a working group on AI in healthcare. In August 2023, there were 4 webinars on AI topics and a 2-day workshop in September 2023 for consensus-building. The event included over 200 industry stakeholders, including clinicians, software developers, academics, ethicists, attorneys, government policy experts, scientists, and patients. The goal was to identify challenges associated with the trusted use of AI-enabled CDS in medical practice. Key issues were identified, and solutions were proposed through qualitative analysis and a 4-month iterative consensus process. RESULTS: Our work culminated in several key recommendations: (1) building safe and trustworthy systems; (2) developing validation, verification, and certification processes for AI-CDS systems; (3) providing a means of safety monitoring and reporting at the national level; and (4) ensuring that appropriate documentation and end-user training are provided. DISCUSSION: AI-enabled Clinical Decision Support (AI-CDS) systems promise to revolutionize healthcare decision-making, necessitating a comprehensive framework for their development, implementation, and regulation that emphasizes trustworthiness, transparency, and safety. This framework encompasses various aspects including model training, explainability, validation, certification, monitoring, and continuous evaluation, while also addressing challenges such as data privacy, fairness, and the need for regulatory oversight to ensure responsible integration of AI into clinical workflow. CONCLUSIONS: Achieving responsible AI-CDS systems requires a collective effort from many healthcare stakeholders. This involves implementing robust safety, monitoring, and transparency measures while fostering innovation. Future steps include testing and piloting proposed trust mechanisms, such as safety reporting protocols, and establishing best practice guidelines.

In a recent paper by Alison McCoy et al “Clinician collaboration to improve clinical decision support: the Clickbusters initiative,” the author noted how the acceptance rates of alerts were quite poor with override rates of ∼90%. The acceptance rates cited were for drug-drug, drug-allergy alerts, and other drug safety alerts. An area of confusion exists generally around equating medication decision support (MDS) override rates with those of system alerts. The term system alerts refers to a form of clinical decision support (CDS) that cover a wide range of topics, is available at multiple times during clinical workflow, can be triggered by diagnoses orders, medications, labs, etc., and content usually has to be configured and built. MDS is another form of CDS but is only available during medication ordering and e-prescribing.1,2 MDS provides drug-drug, drug-allergy, drug-lab, etc., at the point of order, uses a drug database as its knowledge base, content often is pre-built, has very high override rates, and is delivered quite differently than system alerts. The knowledge base and rules engine for system alerts and MDS are usually located quite separately in the clinical information system. If one looks at papers on system alerts, at least 2 prior studies and 1 recent study suggest significantly higher acceptance rates.3–5 Our study described in “A Framework for Usable and Effective Clinical Decision Support: Experience from the iCPR Randomized Clinical Trial” found that study alerts were adopted by 57.4% of users and accepted by 42.7% of users, respectively.3 In the “Impact of an Electronic Clinical Decision Support Tool for Emergency Department Patients With Pneumonia,” the authors cite an acceptance rate of 62.6%.4 A subsequent publication study looking at alerts for problem list deficiencies had an acceptance rate of 22.1%.5

Recent advances in the science and technology of artificial intelligence (AI) and growing numbers of deployed AI systems in healthcare and other services have called attention to the need for ethical principles and governance. We define and provide a rationale for principles that should guide the commission, creation, implementation, maintenance, and retirement of AI systems as a foundation for governance throughout the lifecycle. Some principles are derived from the familiar requirements of practice and research in medicine and healthcare: beneficence, nonmaleficence, autonomy, and justice come first. A set of principles follow from the creation and engineering of AI systems: explainability of the technology in plain terms; interpretability, that is, plausible reasoning for decisions; fairness and absence of bias; dependability, including "safe failure"; provision of an audit trail for decisions; and active management of the knowledge base to remain up to date and sensitive to any changes in the environment. In organizational terms, the principles require benevolence-aiming to do good through the use of AI; transparency, ensuring that all assumptions and potential conflicts of interest are declared; and accountability, including active oversight of AI systems and management of any risks that may arise. Particular attention is drawn to the case of vulnerable populations, where extreme care must be exercised. Finally, the principles emphasize the need for user education at all levels of engagement with AI and for continuing research into AI and its biomedical and healthcare applications.

The development and implementation of clinical decision support (CDS) that trains itself and adapts its algorithms based on new data-here referred to as Adaptive CDS-present unique challenges and considerations. Although Adaptive CDS represents an expected progression from earlier work, the activities needed to appropriately manage and support the establishment and evolution of Adaptive CDS require new, coordinated initiatives and oversight that do not currently exist. In this AMIA position paper, the authors describe current and emerging challenges to the safe use of Adaptive CDS and lay out recommendations for the effective management and monitoring of Adaptive CDS.

OBJECTIVE: We explored patients' and clinicians' perspectives on electronic health record (EHR)-generated outpatient after-visit summaries (AVSs) to inform efforts to maximize the document's utility. MATERIALS AND METHODS: This qualitative study involved focus groups and semistructured interviews with patients ( n = 39) and clinicians ( n = 56) in adult primary care practices serving socioeconomically diverse communities in New York City; Long Island, New York; and Chicago, Illinois. Focus group and interview transcripts were coded and analyzed following standard qualitative methods. RESULTS: Core themes included the use and purpose of the AVS, content modification and prioritization, formatting improvements, customization, privacy and accuracy concerns, and clinician workflow concerns. While most patients valued the document as a visit summary, others considered it a general summary of their health and health care issues, useful for sharing with family or clinicians even if they had access to their health records via web portals. Patients expressed a preference for the order of content items, and many wanted the reasons for medications and referrals stated. Additionally, some patients were confused by multiple medication lists indicating started, stopped, and modified medications, and a single "current" medication list was preferred by both patients and doctors. Concerns were raised about the risk of violating patient privacy and challenges to clinician workflow. DISCUSSION: The AVS is valued by patients and clinicians. Both groups have identified numerous ways it can be improved, but also several obstacles to improvement and effective use. CONCLUSION: EHR vendors should work with stakeholder groups to improve the AVS to ensure that this important communication device achieves its patient-centered potential.

AMIA has been at the forefront of advancing scientific research and education in informatics as well as public policy around issues related to clinical informatics. AMIA’s multidisciplinary, interprofessional membership and strategy reflect this – we have seen a rise in the number of submissions to the AMIA Annual Symposium related to the applied areas of informatics, and more emphasis in our clinical research informatics community on applied aspects of clinical research. An important trend to follow is the professionalization of the informatics field. The rise of accreditation in training programs, the growth of the clinical informatics subspecialty board diplomates (1000+), and the future development of the advanced health informatics certification has, and will, lead to an increase in the number of informatics professionals in leadership roles within their healthcare organizations. This professionalization is in part being driven by the measurable growth in investments in the applied use of information technology in healthcare with emphasis on the deployment and utilization of electronic health records. We have seen a rapid increase in the adoption of electronic health records and other health information technology, and with it, the rise in leadership positions that recognize the importance of informatics to use technology in a strategic way.

BACKGROUND: Health information exchange (HIE) is a potentially powerful technology that can improve the quality of care delivered in emergency departments, but little is known about emergency physicians' current perceptions of HIE. OBJECTIVES: This study sought to assess emergency physicians' perceived needs and knowledge of HIE. METHODS: A questionnaire was developed based on heuristics from the literature and implemented in a Web-based tool. The survey was sent as a hyperlink via e-mail to 371 attending emergency physicians at 12 hospitals in New York City. RESULTS: The response rate was 58% (n = 216). Although 63% said more than one quarter of their patients would benefit from external health information, the barriers to obtain it without HIE are too high--85% said it was difficult or very difficult to obtain external data, taking an average of 66 minutes, 72% said that their attempts fail half of the time, and 56% currently attempt to obtain external data less than 10% of the time. When asked to create a rank-order list, electrocardiograms (ECGs) were ranked the highest, followed by discharge summaries. Respondents also chose images over written reports for ECGs and X-rays, but preferred written reports for advanced imaging and cardiac studies. CONCLUSION: There is a strong perceived need for HIE, most respondents were not aware of HIE prior to this study, and there are certain types of data and presentations of data that are preferred by emergency physicians in the New York City region.

Dr. Patterson's letter was greatly appreciated and reinforces the major arguments in our paper: that not all order entry systems are created equal and that user satisfaction is important to the acceptance of order entry systems and must be assessed after system implementation.1 Mount Sinai's decision to purchase and implement a computerized physician order entry (CPOE) system was made in the early 1990s. An appropriate system selection committee was formed and reviewed many available proprietary CPOE systems. On the basis of the committee's recommendations, Mount Sinai then chose the CPOE system, a system that other major institutions at that time had either selected or implemented. The committee did have prior knowledge about the available literature assessing CPOE systems, and, as a result, steps were undertaken to carefully implement the system. Thus implementation of our system was performed with few of the problems described by Massaro.1,2 As previously mentioned, user feedback is paramount to the development of a usable system. In our study, we gathered qualitative and quantitative data, which is now the basis for a redesign effort of the Mount Sinai CPOE system. We intend to measure how these new changes will influence physician acceptance of the system. It is our belief that user satisfaction will improve through the incorporation of this feedback into the system's user interface. All CPOE systems are not equal. Although inefficient proprietary systems do exist, this is no reflection on medical informatics. Instead, the fact that poorly designed systems exist should emphasize the importance of the specialty of medical informatics in collecting and utilizing user feedback to help design and redesign CPOE systems that are highly usable and therefore accepted by clinicians.—Harvey J. Murff, MD, and Joseph Kannry, MD

OBJECTIVES: In the wake of the Institute of Medicine report, To Err Is Human: Building a Safer Health System (LT Kohn, JM Corrigan, MS Donaldson, eds; Washington, DC: National Academy Press, 1999), numerous advisory panels are advocating widespread implementation of physician order entry as a means to reduce errors and improve patient safety. Successful implementation of an order entry system requires that attention be given to the user interface. The authors assessed physician satisfaction with the user interface of two different order entry systems-a commercially available product, and the Department of Veterans Affairs Computerized Patient Record System (CPRS). DESIGN AND MEASUREMENT: A standardized instrument for measuring user satisfaction with physician order entry systems was mailed to internal medicine and medicine-pediatrics house staff physicians. The subjects answered questions on each system using a 0 to 9 scale. RESULTS: The survey response rates were 63 and 64 percent for the two order entry systems. Overall, house staff were dissatisfied with the commercial system, giving it an overall mean score of 3.67 (95 percent confidence interval [95%CI], 3.37-3.97). In contrast, the CPRS had a mean score of 7.21 (95% CI, 7.00-7.43), indicating that house staff were satisfied with the system. Overall satisfaction was most strongly correlated with the ability to perform tasks in a "straightforward" manner. CONCLUSIONS: User satisfaction differed significantly between the two order entry systems, suggesting that all order entry systems are not equally usable. Given the national usage of the two order entry systems studied, further studies are needed to assess physician satisfaction with use of these same systems at other institutions.

OBJECTIVE: To examine the issues involved in mapping an existing structured controlled vocabulary, the Medical Entities Dictionary (MED) developed at Columbia University, to an institutional vocabulary, the laboratory and pharmacy vocabularies of the Yale New Haven Medical Center. DESIGN: 200 Yale pharmacy terms and 200 Yale laboratory terms were randomly selected from database files containing all of the Yale laboratory and pharmacy terms. These 400 terms were then mapped to the MED in three phases: mapping terms, mapping relationships between terms, and mapping attributes that modify terms. RESULTS: 73% of the Yale pharmacy terms mapped to MED terms. 49% of the Yale laboratory terms mapped to MED terms. After certain obsolete and otherwise inappropriate laboratory terms were eliminated, the latter rate improved to 59%. 23% of the unmatched Yale laboratory terms failed to match because of differences in granularity with MED terms. The Yale and MED pharmacy terms share 12 of 30 distinct attributes. The Yale and MED laboratory terms share 14 of 23 distinct attributes. CONCLUSION: The mapping of an institutional vocabulary to a structured controlled vocabulary requires that the mapping be performed at the level of terms, relationships, and attributes. The mapping process revealed the importance of standardization of local vocabulary subsets, standardization of attribute representation, and term granularity.