BACKGROUND: The use of real-world data (RWD) in artificial intelligence (AI) applications for healthcare offers unique opportunities but also poses complex challenges related to interpretability, transparency, safety, efficacy, bias, equity, privacy, ethics, accountability, and stakeholder engagement. METHODS: A multi-stakeholder expert panel comprising healthcare professionals, AI developers, policymakers, and other stakeholders was assembled. Their task was to identify critical issues and formulate consensus recommendations, focusing on the responsible use of RWD in healthcare AI. The panel's work involved an in-person conference and workshop and extensive deliberations over several months. RESULTS: The panel's findings revealed several critical challenges, including the necessity for data literacy and documentation, the identification and mitigation of bias, privacy and ethics considerations, and the absence of an accountability structure for stakeholder management. To address these, the panel proposed a series of recommendations, such as the adoption of metadata standards for RWD sources, the development of transparency frameworks and instructional labels likened to "nutrition labels" for AI applications, the provision of cross-disciplinary training materials, the implementation of bias detection and mitigation strategies, and the establishment of ongoing monitoring and update processes. CONCLUSION: Guidelines and resources focused on the responsible use of RWD in healthcare AI are essential for developing safe, effective, equitable, and trustworthy applications. The proposed recommendations provide a foundation for a comprehensive framework addressing the entire lifecycle of healthcare AI, emphasizing the importance of documentation, training, transparency, accountability, and multi-stakeholder engagement.

BACKGROUND: Integrating artificial intelligence (AI) in healthcare settings has the potential to benefit clinical decision-making. Addressing challenges such as ensuring trustworthiness, mitigating bias, and maintaining safety is paramount. The lack of established methodologies for pre- and post-deployment evaluation of AI tools regarding crucial attributes such as transparency, performance monitoring, and adverse event reporting makes this situation challenging. OBJECTIVES: This paper aims to make practical suggestions for creating methods, rules, and guidelines to ensure that the development, testing, supervision, and use of AI in clinical decision support (CDS) systems are done well and safely for patients. MATERIALS AND METHODS: In May 2023, the Division of Clinical Informatics at Beth Israel Deaconess Medical Center and the American Medical Informatics Association co-sponsored a working group on AI in healthcare. In August 2023, there were 4 webinars on AI topics and a 2-day workshop in September 2023 for consensus-building. The event included over 200 industry stakeholders, including clinicians, software developers, academics, ethicists, attorneys, government policy experts, scientists, and patients. The goal was to identify challenges associated with the trusted use of AI-enabled CDS in medical practice. Key issues were identified, and solutions were proposed through qualitative analysis and a 4-month iterative consensus process. RESULTS: Our work culminated in several key recommendations: (1) building safe and trustworthy systems; (2) developing validation, verification, and certification processes for AI-CDS systems; (3) providing a means of safety monitoring and reporting at the national level; and (4) ensuring that appropriate documentation and end-user training are provided. DISCUSSION: AI-enabled Clinical Decision Support (AI-CDS) systems promise to revolutionize healthcare decision-making, necessitating a comprehensive framework for their development, implementation, and regulation that emphasizes trustworthiness, transparency, and safety. This framework encompasses various aspects including model training, explainability, validation, certification, monitoring, and continuous evaluation, while also addressing challenges such as data privacy, fairness, and the need for regulatory oversight to ensure responsible integration of AI into clinical workflow. CONCLUSIONS: Achieving responsible AI-CDS systems requires a collective effort from many healthcare stakeholders. This involves implementing robust safety, monitoring, and transparency measures while fostering innovation. Future steps include testing and piloting proposed trust mechanisms, such as safety reporting protocols, and establishing best practice guidelines.

OBJECTIVE: The increasing demands for curated, high-quality research data are driving the emergence of a novel registry type. The need to assemble, curate, and export this data grows, and the conventional simplicity of registry models is driving the need for advanced, multimodal data registries-the dawn of the next-generation registry. MATERIALS AND METHODS: The article provides an outline of the technology roles and responsibilities needed for successful implementations of next-generation registries. RESULTS: We propose a framework for the planning, construction, maintenance, and sustainability of this new registry type. DISCUSSION: A rubric of organizational, computational, and human resource needs is discussed in detail, backed by over 40 years of combined in-the-field experiences by the authors. CONCLUSIONS: A novel field, registry science, within the clinical research informatics domain, has arisen to offer its insights into conceiving, structuring, and sustaining this new breed of tools.

In the wake of Coronavirus disease 2019 (COVID-19), several nations have sought to implement digital vaccine passports (DVPs) to enable the resumption of international travel. Comprising a minimum dataset for each unique individual, DVPs have the makings of a global electronic health record, broaching key issues involved in building a global digital health ecosystem. Debate simulations offer a safe, interactive space to foster participatory policy discussions for advancing digital health diplomacy. This study used an online simulation of a Model World Health Assembly to critically analyze the sociotechnical issues associated with the global implementation of DVPs, and to generate useful insights and questions about the role of diplomacy in global digital health. The debate arguments addressed and provided insights into the technological, scientific, ethical, legal, policy, and societal aspects of DVPs. Reflecting on the simulation, we discuss its opportunities and challenges for the digitalization, decolonization, decentralization, and democratization of participatory policymaking.

The Journal of the American Medical Informatics Association (JAMIA) focus issue on Global Health Informatics (GHI) examines how health informatics can address global health issues. The selected papers discuss how informatics can improve health outcomes, especially in low- and middle-income countries (LMICs). The articles in this issue come from a broad range of researchers, practitioners, and policymakers, provide an overview of GHI initiatives and encourage researchers and practitioners to share ideas and best practices. GHI aims to harness digital health and informatics knowledge and tools to improve the care of individuals and populations in LMIC through achieving these 5 aims: effectiveness, efficiencies, patient satisfaction, provider well-being, and equity. A central consideration is the global north-south divide and the sociotechnical determinants of and impacts on the digital health maturity of LMIC. This issue seeks to document and understand how health informatics approaches have been undertaken in multiple LMIC scenarios and environments and how they can demonstrate the scientific impact of informatics-based best practices applied to COVID-19 and beyond. Increasing the understanding of current GHI work is important for the entire field of biomedical and health informatics; arguably, global health touches upon all 5 of the recognized domains of biomedical informatics, from translational to public health. This focus Issue aims to advance the science and application of GHI in sustainable, scalable, and capacity-building health information technology (HIT) systems that improve individual patient and community health outcomes. In addition, this focus Issue sought to bring to highlight specific innovative health informatics approaches and solutions developed and evaluated to address the COVID-19 pandemic preparedness and response across LMIC. Multiple opportunities exist to leverage and integrate lessons learned from these approaches into our US-based institutions. We share the contents and constraints of this focus issue and identify opportunities for health informatics and the American Medical Informatics Association (AMIA). COVID-19 and other pandemics of noncommunicable diseases have tested the resilience and fitness-for-purpose of HIT systems in all countries, subnationally and internationally, exposing their structural and functional inadequacies and inequities across all the “divides.” Massive investments into HIT systems have helped the response. However, the mainly ad hoc development of new and repurposed digital tools, without explicit reference to or compliance with interoperability standards, is worrisome in terms of an integrated (and interoperable) global response to “long COVID” and the next pandemic. On the other hand, the response to incorporating COVID-19 terminology in WHO ICD-11 was timely, auguring well for the health and informatics communities.1 The pandemic has also highlighted the need to develop guidance and regulatory frameworks for creating, managing, and procuring HIT systems. The GHI community needs to be vigilant that this COVID-driven investment is built by emphasizing interoperability standards and seamless implementation into the healthcare workflow. AMIA and the American College of Medical Informatics (ACMI) have roles in collecting and creating the evidence to support the GHI community to achieve the quintuple aims and improve individual and organizational digital capability maturity concurrently. The call for current research and perspectives was intentionally broad to encourage submissions from multiple authors representing diverse regions. Solicited topics included data governance and sharing, interoperability, data-driven policymaking, clinical-based solutions, telemedicine, consumer-based solutions, and specific COVID-19 informatics responses, including vaccination distribution and vaccine passports. This issue includes 11 articles summarized in Table 1 that reflect these broad topics of interest.2–12 Papers included in this focus issue, grouped by primary theme mHealth COVID-19 mHealth Data governance COVID-19 mHealth COVID-19 Data analysis COVID-19 Interoperability and standards COVID-19 Data analysis COVID-19 Misinformation and global health COVID-19 mHealth COVID-19 mHealth Data governance COVID-19 mHealth COVID-19 Data analysis COVID-19 Interoperability and standards COVID-19 Data analysis COVID-19 Misinformation and global health COVID-19 Papers included in this focus issue, grouped by primary theme mHealth COVID-19 mHealth Data governance COVID-19 mHealth COVID-19 Data analysis COVID-19 Interoperability and standards COVID-19 Data analysis COVID-19 Misinformation and global health COVID-19 mHealth COVID-19 mHealth Data governance COVID-19 mHealth COVID-19 Data analysis COVID-19 Interoperability and standards COVID-19 Data analysis COVID-19 Misinformation and global health COVID-19 Specific GHI insights came from multiple countries, including Brazil, India, Kenya, Pakistan, Uganda, Vietnam, and other regions. Country-specific papers addressed multiple health informatics areas, including database structure within the Observational Medical Outcomes Partnership (OMOP) model,9 specific mobile health (mHealth) applications,7 the implementation and impact of telehealth and digital health wallets (DHW),10 and, from Uganda, a study of COVID-19 on clinical care through the application of interrupted time series and regression models using data from a country-wide HIT system.8 There is a focus on the eHealth component (mHealth and telehealth) during COVID-19. This reflects a long history of emphasis on “shiny” tools without emphasizing vital standards-based informatics (especially usability testing) and data science foundation and the paucity of enterprise-wide platforms to support the spread and scale-up of the tools and data. The limited findings of the OMOP paper support this view from the data networks perspective.9 Governance is growing in importance, and funded organizations like the Health Data Collaborative are promising. This issue includes an exemplar using the application of health data governance principles on simulated global electronic health records informed by data within a digital health passport.5 Global inequity, such as the global north-south divide, must be addressed explicitly in informatics education and professional practice. The north-south divide is real, and only 1 paper expressly discussed it. This used an online simulation of educational activities around digital vaccine passports to address border controls in pandemics, “vaccine nationalism,” and inequity. Digital health diplomacy was identified as an important part of education and training to achieve sociotechnical harmonization and cooperation across the digital spectrum of technical, syntactic, semantic, services, and regulatory layers of interoperability.5 “Scoping Review of Health Information Technology Usability Methods Leveraged in Africa” explored the state of health information technology (HIT) usability (effectiveness, efficiency, and satisfaction) evaluation in Africa.2 Developing and enhancing usable HIT is critical to promoting equitable health service delivery and high-quality care in Africa. Of the 73 articles included in this review, 20 (27%) are from the last 2 years and 31 (42%) from the previous 3 to 5 years. The findings suggest that early-stage evaluations (Stages 1 and 2) and interactions (Types 0 and 1) should receive special attention to ensure HIT usability before implementing HIT in the field. The authors recommend that researchers must incorporate theoretical frameworks in future studies to provide a more robust rationale for their work and enhance the rigor of usability evaluation research. “Clinical Information System (CIS): Implementation in Developing Countries: Requirements, Success Factors, and Recommendations” by Tun and Madanian systematically reviewed the literature on the success factors in Clinical Information System (CIS) implementation for developing countries.3 The aim was to create a set of recommendations, presented as a framework, to enhance the success rate in implementing CIS in these countries. A systematic literature review was conducted, and 83 articles were analyzed using thematic and cross-case analysis to identify the requirements and success factors. Six major requirement categories were identified: project management, financial resources, government involvement and support, human resources, organizational, and technical requirements. The study provided a set of recommendations presented in a framework based on the project management lifecycle approach. The proposed framework could support CIS implementation in developing countries. Future studies should focus on identifying barriers to CIS implementation and conducting country-specific empirical studies based on the findings. It is recommended that projects in developing countries include the results of this research included in Appendix E (page 65). “A Scoping Review of Digital Health Interventions for Combating COVID-19 Misinformation and Disinformation” examined digital health interventions (DHIs) that mitigate COVID-19 misinformation and disinformation seeding and spread.12 With the digitization of information dissemination and community outreach activities through online social media, there is a unique opportunity to capture, monitor, examine, and evaluate the components operationalized in trending misinformation (including semantics, syntax, framing, and behavioral constructs) that are permeating public health risk communications and diffusing in vulnerable communities globally. The study screened 1666 articles from PubMed, PsychINFO, and Web of Science and included 17 experimental and interventional studies that developed and tested public consumer-facing DHIs. The results showed that most studies used social media in DHIs, but there was a lack of platform-agnostic generalizability. Only half of the studies specified a theory or model to guide DHIs, and only 1 DHI was evaluated for user perceptions and acceptance. Although critical appraisal and meta-synthesis of articles are not required in scoping reviews, this paper performed an initial summarization. The conclusion suggests the need for community engagement and theory-guided engineering of equitable DHIs that consider the problem of misinformation and disinformation through a multilevel lens. Future scoping reviews can present a more comprehensive view of the research area by exploring domain areas, search terms, and digital health interventions outside the domain of COVID-19, identifying domain gaps, and improving quality in the literature. “Utilization of Telehealth to Manage the COVID-19 Pandemic in Low and Middle-Income countries: A Scoping Review” explored the utilization of telehealth technology to manage the COVID-19 pandemic in LMIC.4 This scoping review provided insight into the use of telehealth technology for different specialties and highlighted the challenges. The scoping review conducted in 2022 analyzed 18 articles published since 2020 on the use of telehealth for COVID-19 in LMIC (South Asia, sub-Saharan Africa, the Middle East and North Africa, and East Asia and Oceania). They included telehealth interventions such as teleconsultation, telecoaching, teledermatology, televisit, mhealth applications, telerehabilitation, telepharmacy, and telepsychiatry. WhatsApp was the most common way for service delivery, and patients and healthcare providers were satisfied with the services in most cases. Although telehealth use was limited in LMIC during the pandemic, it was effective and had positive outcomes. Difficulties with a network connection and accessing high-speed Internet were among the main challenges in using telehealth technology. According to the results, the outcomes of the telehealth interventions in terms of the providers’ and patients’ satisfaction and clinical improvements were positive in most studies indicating that telehealth technology in LMIC is feasible. Patient privacy and information security were among the reported challenges. The governments and health authorities in LMIC need to develop practical guidelines supported by legal bodies to guarantee telehealth technology’s ethical and legal aspects. The economic aspect of telehealth is also important for patients and healthcare providers. Therefore, transparent legislation for the costs and payments for virtual health care services is required. Moreover, training the end-users and providing adequate technical support can encourage patients, their carers, and healthcare providers to use the technology more than before. The study recommends further investigation into similar interventions’ characteristics and clinical effectiveness in different countries, including LMIC. “The Global Health Informatics Landscape and JAMIA” assessed the state of GHI in selected informatics journals including JAMIA.11 The perspective applied criteria for articles about LMIC, international health, and underserved populations and compared JAMIA to 3 other health informatics journals. The analysis found that JAMIA is a key target journal for GHI dissemination with a high impact factor and that it and other journals play a crucial role in strengthening GHI worldwide. The authors make recommendations for future directions for GHI innovation, evaluation, and policy and the role that journals like JAMIA can play in strengthening GHI worldwide. “Analysis of User Interactions with a Digital Health Wallet for Enabling Care Continuity in the Context of an Ongoing Pandemic” investigated user interactions with a DHW system for addressing care continuity challenges in chronic disease management during the COVID-19 pandemic in Kenya.10 The feasibility study analyzed user interaction log data generated by clinicians, nurses, and patients during the deployment of a DHW. The Hamming distance from Information Theory was used to quantify deviations from predetermined workflow sequences supported by the DHW platform. User interaction log analysis is a valuable alternative method for generating and quantifying user experiences in ongoing pandemics. This can be used to identify potential bottlenecks in patient and provider-facing documentation and data-sharing applications such as the DHW. The results showed that nurses interacted with all the relevant user interface elements for triage, clinicians interacted with only 43% of the relevant aspects for consultation, and patients interacted with 67% of the relevant elements. Most deviations from the predetermined workflow sequences pertained to users returning to previous steps in their usage workflow. The conclusion suggests that user interaction log analysis is a valuable alternative method for generating and quantifying user experiences in ongoing pandemics. However, researchers should consider potential disruptions and use multiple approaches to investigate user experiences of health technology during pandemics. Two papers included in this focus issue concentrated on informatics approaches to problems that persist in many settings but in the context of the COVID-19 pandemic. The article “Integrating Real-World Data From The OMOP Data and Health to COVID-19 in the Global a study that used a health informatics framework to and evidence of COVID-19 in and The study COVID-19 records from and analyzed using a network The study analyzed 2 COVID-19 in and The results showed that COVID-19 outcomes were more in the and with health with international findings. The study that a health informatics framework for and COVID-19 data from the Global can provide a potential framework for global knowledge and clinical in response to pandemics and other healthcare The article “A of Two from LMIC the attention on mHealth in LMIC to the potential they can With the of many are exploring as a of impact for individuals in LMIC. The COVID-19 pandemic has also many governments in LMIC to use to support and manage their pandemic and growing digital and COVID-19 mHealth to and public health practices. the in how were and including telemedicine, and and information were practices for included to emphasizing in and using a and for but of services, but data on their impact is future are and are creating an on best practices and global health and the of different and LMIC are in the that mHealth must be in the context of technology The article the of COVID-19 on Health A of using and is of the articles to evaluate data from an Health Information System Data from the system through was used to the impact of on new and interrupted time series and regression evaluation showed within a Multiple regression models were used to quantify the potential impact of the COVID-19 pandemic. and models to the need for HIT in global health This study also the need for model of and the need for tools to robust impact the this article was of the that evaluated the impact of COVID-19 on care delivery in and that a system is in LMIC and can be used for data and Digital Health WHO how environments have used simulation models to evaluate the impact of solutions to implementation and Digital vaccine passports have been proposed as a to a global electronic health the on 1 proposed digital health vaccine and how this could among diverse and challenges for and were can health informatics domains that need to be addressed in LMIC including and public and within the This work how an can and an international in from simulation to “A for a for mHealth Data Governance In Low Middle-Income evaluated how the Health Data Governance developed and in 2022 by digital health are to be for LMIC health informatics the proposed health data governance principles to mHealth data governance helped 3 data and to ensure equitable from health with patients, and promoting health by enhancing health systems and This application of the Health Data Governance principles to mobile health the digital that can including attention to data the development of to data sharing, and the role of interoperability in mHealth technology. of the common findings across the studies The growing use of telehealth and mobile in and the potential they can bring to support and manage COVID-19 pandemic in The of usability in applications The potential of health informatics frameworks for generating and evidence of COVID-19 in The of data privacy and governance implementing mHealth and the need for a comprehensive data-sharing for mHealth data sharing, and governance in The use of simulation models and regression models can be used to understand the impact of solutions in The need for community engagement and the of public and within the The articles in the focus issue also the evidence in highlight DHI and and DHI address health system challenges and enhance the quality of health and DHI are within digital health applications and information including such as to achieve health Misinformation and disinformation presented a public health and were in the context of the COVID-19 information the effectiveness of health interventions to health DHIs have been developed around the to address misinformation and include 3 that to identify or misinformation There has been an in online platforms in LMIC and social media and can information for health information in social media it for users to from platforms have also the spread of a to have also used to address studies DHI behavioral and such as risk and to COVID-19 misinformation or of the COVID-19 pandemic, and the study limited evaluation of the role of DHIs in addressing misinformation or especially in LMIC. for included a lack of community engagement and research in the development and evaluation of the DHIs. DHIs, and an are important to enhance user and ensure research is to evaluate user and satisfaction with the DHIs. Telehealth was an effective for providing healthcare services during the COVID-19 pandemic. Telehealth and clinical healthcare services, and training information and Telehealth services were and platforms such as and and were telehealth services The and used platforms such as WhatsApp and were and used platforms in showed a in the use and of telehealth services the of the COVID-19 showed positive and in using this The of services include improving to healthcare services, and in most critical components for the implementation of telehealth in LMIC include legal and regulatory identifying telehealth users and telehealth technology financial telehealth and training and healthcare and Telehealth has in technical and challenges The barriers to implementing telehealth in LMIC are the high development technical and Internet The of the technology and and of are among other factors should be especially in Internet and and more and development investment are critical for telehealth and security were other challenges of using telehealth technology in There is a need to develop practical guidelines and evaluate the economic aspect of telehealth technology. training the end-users and providing adequate technical support can encourage individuals to use the technology more than before. This focus issue has to the multiple informatics challenges in LMIC including limited electronic health systems used the of that have been developed to than clinical lack of work that can support interoperability and data sharing, and health informatics limited by resources, to training and among many The COVID-19 pandemic the of HIT support for and health in LMIC. of the that were recognized in the during the pandemic of electronic to share clinical data within the healthcare system as well as public health, limited vaccine within the community and public health were in LMIC as a of informatics The lack of research and development and experimental studies in the focus issue reflects the of digital health maturity in the the of digital tools, a for health information and such as a are The scoping reviews reflect a to guide the and training in LMIC. This suggests that the biomedical and health informatics researchers in LMIC are their best within the and financial The principles are but the implementation and evaluation of a need for more to the of digital health The COVID-19 investments in HIT in are an opportunity to an interoperability standards-based informatics (especially usability testing) and an enterprise-wide to support the and and of the tools and data that have been and to be A global to this enterprise-wide with a common data model and an explicit and governance structure to achieve the quintuple aims a long way to achieving the of COVID-19 has investments in informatics (including public health in LMIC, but the investment support for a robust foundation for standards-based digital and public A framework for GHI must consider the and evaluation of the capability maturity of the digital health in health organizations and individual digital there is a need to develop an equitable based on understanding the impact of This to work is critical to and improve the robust evaluation research such as AMIA and with similar organizations in other countries through the Medical Informatics Association and of Health Informatics can play a role in conducting the and the for to achieve can include work with our governments and institutions. authors to the and of the authors the of the The data this article are in the article and the special issue is a of the Health and is for the in this not the or of the

Actualizing the vision of Global Digital Health is a central issue on the Global Health Diplomacy agenda. The COVID-reinforced need for accelerated digital health progress will require political structures and processes to build a foundation for Global Digital Health. Simultaneously, Global Health Diplomacy uses digital technologies in its enactment. Both phenomena have driven interest in the term "Digital Health Diplomacy." A review of the literature revealed 2 emerging but distinct definitions that have been published very recently, each with its associated discourse and practice. This multiplicity of ideas demonstrates the myriad ways in which global digital and political systems are becoming increasingly entangled. Untangling these, this paper proposes and discusses 3 dimensions of Digital Health Diplomacy: "Diplomacy for digital health," "Digital health for diplomacy," and "Digital health in diplomacy." It calls upon digital health professionals, diplomats, political and social scientists, epidemiologists, and clinicians to discuss, critique, and advance this emerging domain.

There has been a tremendous increase in the costs of caring for older adults owing to the fact that societies are aging around around the world. This has led to a decrease in the number of caregivers who are able to assist. Investigative studies indicate that older adults require social as well as physical support for their well-being which prompted researchers to use social and cognitive robots and advanced human machine interaction devices. However, most of these studies have shortcomings when it comes to providing means of a natural interaction with the machine. With speech being the most natural way for human communication and the huge developments in the Internet of Things and smart homes, equipping a robotic system with powerful natural speech interaction capabilities to maintain a conversation with an elderly while being linked to other smart home devices shows a promising direction. This paper describes a scalable and expandable system with main goal of designing a natural speech-enabled system for older adults that is capable of linking to multiple active agents with minimal integration efforts. The system makes use of the power of commercially available digital assistant systems, integrated with an intelligent conversational agent, robotics, and smart wearables. The main advantage of the system is that it could provide a portion of the population, namely older adults and the disabled, the flexibility of interacting naturally with powerful social robots in smart home environments, hence providing them with much needed independence.