The Journal of the American Medical Informatics Association (JAMIA) focus issue on Global Health Informatics (GHI) examines how health informatics can address global health issues. The selected papers discuss how informatics can improve health outcomes, especially in low- and middle-income countries (LMICs). The articles in this issue come from a broad range of researchers, practitioners, and policymakers, provide an overview of GHI initiatives and encourage researchers and practitioners to share ideas and best practices. GHI aims to harness digital health and informatics knowledge and tools to improve the care of individuals and populations in LMIC through achieving these 5 aims: effectiveness, efficiencies, patient satisfaction, provider well-being, and equity. A central consideration is the global north-south divide and the sociotechnical determinants of and impacts on the digital health maturity of LMIC. This issue seeks to document and understand how health informatics approaches have been undertaken in multiple LMIC scenarios and environments and how they can demonstrate the scientific impact of informatics-based best practices applied to COVID-19 and beyond. Increasing the understanding of current GHI work is important for the entire field of biomedical and health informatics; arguably, global health touches upon all 5 of the recognized domains of biomedical informatics, from translational to public health. This focus Issue aims to advance the science and application of GHI in sustainable, scalable, and capacity-building health information technology (HIT) systems that improve individual patient and community health outcomes. In addition, this focus Issue sought to bring to highlight specific innovative health informatics approaches and solutions developed and evaluated to address the COVID-19 pandemic preparedness and response across LMIC. Multiple opportunities exist to leverage and integrate lessons learned from these approaches into our US-based institutions. We share the contents and constraints of this focus issue and identify opportunities for health informatics and the American Medical Informatics Association (AMIA). COVID-19 and other pandemics of noncommunicable diseases have tested the resilience and fitness-for-purpose of HIT systems in all countries, subnationally and internationally, exposing their structural and functional inadequacies and inequities across all the “divides.” Massive investments into HIT systems have helped the response. However, the mainly ad hoc development of new and repurposed digital tools, without explicit reference to or compliance with interoperability standards, is worrisome in terms of an integrated (and interoperable) global response to “long COVID” and the next pandemic. On the other hand, the response to incorporating COVID-19 terminology in WHO ICD-11 was timely, auguring well for the health and informatics communities.1 The pandemic has also highlighted the need to develop guidance and regulatory frameworks for creating, managing, and procuring HIT systems. The GHI community needs to be vigilant that this COVID-driven investment is built by emphasizing interoperability standards and seamless implementation into the healthcare workflow. AMIA and the American College of Medical Informatics (ACMI) have roles in collecting and creating the evidence to support the GHI community to achieve the quintuple aims and improve individual and organizational digital capability maturity concurrently. The call for current research and perspectives was intentionally broad to encourage submissions from multiple authors representing diverse regions. Solicited topics included data governance and sharing, interoperability, data-driven policymaking, clinical-based solutions, telemedicine, consumer-based solutions, and specific COVID-19 informatics responses, including vaccination distribution and vaccine passports. This issue includes 11 articles summarized in Table 1 that reflect these broad topics of interest.2–12 Papers included in this focus issue, grouped by primary theme mHealth COVID-19 mHealth Data governance COVID-19 mHealth COVID-19 Data analysis COVID-19 Interoperability and standards COVID-19 Data analysis COVID-19 Misinformation and global health COVID-19 mHealth COVID-19 mHealth Data governance COVID-19 mHealth COVID-19 Data analysis COVID-19 Interoperability and standards COVID-19 Data analysis COVID-19 Misinformation and global health COVID-19 Papers included in this focus issue, grouped by primary theme mHealth COVID-19 mHealth Data governance COVID-19 mHealth COVID-19 Data analysis COVID-19 Interoperability and standards COVID-19 Data analysis COVID-19 Misinformation and global health COVID-19 mHealth COVID-19 mHealth Data governance COVID-19 mHealth COVID-19 Data analysis COVID-19 Interoperability and standards COVID-19 Data analysis COVID-19 Misinformation and global health COVID-19 Specific GHI insights came from multiple countries, including Brazil, India, Kenya, Pakistan, Uganda, Vietnam, and other regions. Country-specific papers addressed multiple health informatics areas, including database structure within the Observational Medical Outcomes Partnership (OMOP) model,9 specific mobile health (mHealth) applications,7 the implementation and impact of telehealth and digital health wallets (DHW),10 and, from Uganda, a study of COVID-19 on clinical care through the application of interrupted time series and regression models using data from a country-wide HIT system.8 There is a focus on the eHealth component (mHealth and telehealth) during COVID-19. This reflects a long history of emphasis on “shiny” tools without emphasizing vital standards-based informatics (especially usability testing) and data science foundation and the paucity of enterprise-wide platforms to support the spread and scale-up of the tools and data. The limited findings of the OMOP paper support this view from the data networks perspective.9 Governance is growing in importance, and funded organizations like the Health Data Collaborative are promising. This issue includes an exemplar using the application of health data governance principles on simulated global electronic health records informed by data within a digital health passport.5 Global inequity, such as the global north-south divide, must be addressed explicitly in informatics education and professional practice. The north-south divide is real, and only 1 paper expressly discussed it. This used an online simulation of educational activities around digital vaccine passports to address border controls in pandemics, “vaccine nationalism,” and inequity. Digital health diplomacy was identified as an important part of education and training to achieve sociotechnical harmonization and cooperation across the digital spectrum of technical, syntactic, semantic, services, and regulatory layers of interoperability.5 “Scoping Review of Health Information Technology Usability Methods Leveraged in Africa” explored the state of health information technology (HIT) usability (effectiveness, efficiency, and satisfaction) evaluation in Africa.2 Developing and enhancing usable HIT is critical to promoting equitable health service delivery and high-quality care in Africa. Of the 73 articles included in this review, 20 (27%) are from the last 2 years and 31 (42%) from the previous 3 to 5 years. The findings suggest that early-stage evaluations (Stages 1 and 2) and interactions (Types 0 and 1) should receive special attention to ensure HIT usability before implementing HIT in the field. The authors recommend that researchers must incorporate theoretical frameworks in future studies to provide a more robust rationale for their work and enhance the rigor of usability evaluation research. “Clinical Information System (CIS): Implementation in Developing Countries: Requirements, Success Factors, and Recommendations” by Tun and Madanian systematically reviewed the literature on the success factors in Clinical Information System (CIS) implementation for developing countries.3 The aim was to create a set of recommendations, presented as a framework, to enhance the success rate in implementing CIS in these countries. A systematic literature review was conducted, and 83 articles were analyzed using thematic and cross-case analysis to identify the requirements and success factors. Six major requirement categories were identified: project management, financial resources, government involvement and support, human resources, organizational, and technical requirements. The study provided a set of recommendations presented in a framework based on the project management lifecycle approach. The proposed framework could support CIS implementation in developing countries. Future studies should focus on identifying barriers to CIS implementation and conducting country-specific empirical studies based on the findings. It is recommended that projects in developing countries include the results of this research included in Appendix E (page 65). “A Scoping Review of Digital Health Interventions for Combating COVID-19 Misinformation and Disinformation” examined digital health interventions (DHIs) that mitigate COVID-19 misinformation and disinformation seeding and spread.12 With the digitization of information dissemination and community outreach activities through online social media, there is a unique opportunity to capture, monitor, examine, and evaluate the components operationalized in trending misinformation (including semantics, syntax, framing, and behavioral constructs) that are permeating public health risk communications and diffusing in vulnerable communities globally. The study screened 1666 articles from PubMed, PsychINFO, and Web of Science and included 17 experimental and interventional studies that developed and tested public consumer-facing DHIs. The results showed that most studies used social media in DHIs, but there was a lack of platform-agnostic generalizability. Only half of the studies specified a theory or model to guide DHIs, and only 1 DHI was evaluated for user perceptions and acceptance. Although critical appraisal and meta-synthesis of articles are not required in scoping reviews, this paper performed an initial summarization. The conclusion suggests the need for community engagement and theory-guided engineering of equitable DHIs that consider the problem of misinformation and disinformation through a multilevel lens. Future scoping reviews can present a more comprehensive view of the research area by exploring domain areas, search terms, and digital health interventions outside the domain of COVID-19, identifying domain gaps, and improving quality in the literature. “Utilization of Telehealth to Manage the COVID-19 Pandemic in Low and Middle-Income countries: A Scoping Review” explored the utilization of telehealth technology to manage the COVID-19 pandemic in LMIC.4 This scoping review provided insight into the use of telehealth technology for different specialties and highlighted the challenges. The scoping review conducted in 2022 analyzed 18 articles published since 2020 on the use of telehealth for COVID-19 in LMIC (South Asia, sub-Saharan Africa, the Middle East and North Africa, and East Asia and Oceania). They included telehealth interventions such as teleconsultation, telecoaching, teledermatology, televisit, mhealth applications, telerehabilitation, telepharmacy, and telepsychiatry. WhatsApp was the most common way for service delivery, and patients and healthcare providers were satisfied with the services in most cases. Although telehealth use was limited in LMIC during the pandemic, it was effective and had positive outcomes. Difficulties with a network connection and accessing high-speed Internet were among the main challenges in using telehealth technology. According to the results, the outcomes of the telehealth interventions in terms of the providers’ and patients’ satisfaction and clinical improvements were positive in most studies indicating that telehealth technology in LMIC is feasible. Patient privacy and information security were among the reported challenges. The governments and health authorities in LMIC need to develop practical guidelines supported by legal bodies to guarantee telehealth technology’s ethical and legal aspects. The economic aspect of telehealth is also important for patients and healthcare providers. Therefore, transparent legislation for the costs and payments for virtual health care services is required. Moreover, training the end-users and providing adequate technical support can encourage patients, their carers, and healthcare providers to use the technology more than before. The study recommends further investigation into similar interventions’ characteristics and clinical effectiveness in different countries, including LMIC. “The Global Health Informatics Landscape and JAMIA” assessed the state of GHI in selected informatics journals including JAMIA.11 The perspective applied criteria for articles about LMIC, international health, and underserved populations and compared JAMIA to 3 other health informatics journals. The analysis found that JAMIA is a key target journal for GHI dissemination with a high impact factor and that it and other journals play a crucial role in strengthening GHI worldwide. The authors make recommendations for future directions for GHI innovation, evaluation, and policy and the role that journals like JAMIA can play in strengthening GHI worldwide. “Analysis of User Interactions with a Digital Health Wallet for Enabling Care Continuity in the Context of an Ongoing Pandemic” investigated user interactions with a DHW system for addressing care continuity challenges in chronic disease management during the COVID-19 pandemic in Kenya.10 The feasibility study analyzed user interaction log data generated by clinicians, nurses, and patients during the deployment of a DHW. The Hamming distance from Information Theory was used to quantify deviations from predetermined workflow sequences supported by the DHW platform. User interaction log analysis is a valuable alternative method for generating and quantifying user experiences in ongoing pandemics. This can be used to identify potential bottlenecks in patient and provider-facing documentation and data-sharing applications such as the DHW. The results showed that nurses interacted with all the relevant user interface elements for triage, clinicians interacted with only 43% of the relevant aspects for consultation, and patients interacted with 67% of the relevant elements. Most deviations from the predetermined workflow sequences pertained to users returning to previous steps in their usage workflow. The conclusion suggests that user interaction log analysis is a valuable alternative method for generating and quantifying user experiences in ongoing pandemics. However, researchers should consider potential disruptions and use multiple approaches to investigate user experiences of health technology during pandemics. Two papers included in this focus issue concentrated on informatics approaches to problems that persist in many settings but in the context of the COVID-19 pandemic. The article “Integrating Real-World Data From The OMOP Data and Health to COVID-19 in the Global a study that used a health informatics framework to and evidence of COVID-19 in and The study COVID-19 records from and analyzed using a network The study analyzed 2 COVID-19 in and The results showed that COVID-19 outcomes were more in the and with health with international findings. The study that a health informatics framework for and COVID-19 data from the Global can provide a potential framework for global knowledge and clinical in response to pandemics and other healthcare The article “A of Two from LMIC the attention on mHealth in LMIC to the potential they can With the of many are exploring as a of impact for individuals in LMIC. The COVID-19 pandemic has also many governments in LMIC to use to support and manage their pandemic and growing digital and COVID-19 mHealth to and public health practices. the in how were and including telemedicine, and and information were practices for included to emphasizing in and using a and for but of services, but data on their impact is future are and are creating an on best practices and global health and the of different and LMIC are in the that mHealth must be in the context of technology The article the of COVID-19 on Health A of using and is of the articles to evaluate data from an Health Information System Data from the system through was used to the impact of on new and interrupted time series and regression evaluation showed within a Multiple regression models were used to quantify the potential impact of the COVID-19 pandemic. and models to the need for HIT in global health This study also the need for model of and the need for tools to robust impact the this article was of the that evaluated the impact of COVID-19 on care delivery in and that a system is in LMIC and can be used for data and Digital Health WHO how environments have used simulation models to evaluate the impact of solutions to implementation and Digital vaccine passports have been proposed as a to a global electronic health the on 1 proposed digital health vaccine and how this could among diverse and challenges for and were can health informatics domains that need to be addressed in LMIC including and public and within the This work how an can and an international in from simulation to “A for a for mHealth Data Governance In Low Middle-Income evaluated how the Health Data Governance developed and in 2022 by digital health are to be for LMIC health informatics the proposed health data governance principles to mHealth data governance helped 3 data and to ensure equitable from health with patients, and promoting health by enhancing health systems and This application of the Health Data Governance principles to mobile health the digital that can including attention to data the development of to data sharing, and the role of interoperability in mHealth technology. of the common findings across the studies The growing use of telehealth and mobile in and the potential they can bring to support and manage COVID-19 pandemic in The of usability in applications The potential of health informatics frameworks for generating and evidence of COVID-19 in The of data privacy and governance implementing mHealth and the need for a comprehensive data-sharing for mHealth data sharing, and governance in The use of simulation models and regression models can be used to understand the impact of solutions in The need for community engagement and the of public and within the The articles in the focus issue also the evidence in highlight DHI and and DHI address health system challenges and enhance the quality of health and DHI are within digital health applications and information including such as to achieve health Misinformation and disinformation presented a public health and were in the context of the COVID-19 information the effectiveness of health interventions to health DHIs have been developed around the to address misinformation and include 3 that to identify or misinformation There has been an in online platforms in LMIC and social media and can information for health information in social media it for users to from platforms have also the spread of a to have also used to address studies DHI behavioral and such as risk and to COVID-19 misinformation or of the COVID-19 pandemic, and the study limited evaluation of the role of DHIs in addressing misinformation or especially in LMIC. for included a lack of community engagement and research in the development and evaluation of the DHIs. DHIs, and an are important to enhance user and ensure research is to evaluate user and satisfaction with the DHIs. Telehealth was an effective for providing healthcare services during the COVID-19 pandemic. Telehealth and clinical healthcare services, and training information and Telehealth services were and platforms such as and and were telehealth services The and used platforms such as WhatsApp and were and used platforms in showed a in the use and of telehealth services the of the COVID-19 showed positive and in using this The of services include improving to healthcare services, and in most critical components for the implementation of telehealth in LMIC include legal and regulatory identifying telehealth users and telehealth technology financial telehealth and training and healthcare and Telehealth has in technical and challenges The barriers to implementing telehealth in LMIC are the high development technical and Internet The of the technology and and of are among other factors should be especially in Internet and and more and development investment are critical for telehealth and security were other challenges of using telehealth technology in There is a need to develop practical guidelines and evaluate the economic aspect of telehealth technology. training the end-users and providing adequate technical support can encourage individuals to use the technology more than before. This focus issue has to the multiple informatics challenges in LMIC including limited electronic health systems used the of that have been developed to than clinical lack of work that can support interoperability and data sharing, and health informatics limited by resources, to training and among many The COVID-19 pandemic the of HIT support for and health in LMIC. of the that were recognized in the during the pandemic of electronic to share clinical data within the healthcare system as well as public health, limited vaccine within the community and public health were in LMIC as a of informatics The lack of research and development and experimental studies in the focus issue reflects the of digital health maturity in the the of digital tools, a for health information and such as a are The scoping reviews reflect a to guide the and training in LMIC. This suggests that the biomedical and health informatics researchers in LMIC are their best within the and financial The principles are but the implementation and evaluation of a need for more to the of digital health The COVID-19 investments in HIT in are an opportunity to an interoperability standards-based informatics (especially usability testing) and an enterprise-wide to support the and and of the tools and data that have been and to be A global to this enterprise-wide with a common data model and an explicit and governance structure to achieve the quintuple aims a long way to achieving the of COVID-19 has investments in informatics (including public health in LMIC, but the investment support for a robust foundation for standards-based digital and public A framework for GHI must consider the and evaluation of the capability maturity of the digital health in health organizations and individual digital there is a need to develop an equitable based on understanding the impact of This to work is critical to and improve the robust evaluation research such as AMIA and with similar organizations in other countries through the Medical Informatics Association and of Health Informatics can play a role in conducting the and the for to achieve can include work with our governments and institutions. authors to the and of the authors the of the The data this article are in the article and the special issue is a of the Health and is for the in this not the or of the

The rapid increase of interest in, and use of, artificial intelligence (AI) in computer applications has raised a parallel concern about its ability (or lack thereof) to provide understandable, or explainable, output to users. This concern is especially legitimate in biomedical contexts, where patient safety is of paramount importance. This position paper brings together seven researchers working in the field with different roles and perspectives, to explore in depth the concept of explainable AI, or XAI, offering a functional definition and conceptual framework or model that can be used when considering XAI. This is followed by a series of desiderata for attaining explainability in AI, each of which touches upon a key domain in biomedicine.

Acute kidney injury (AKI) is a frequent complication in hospitalized patients, and is associated with worse short and long-term outcomes. An early prediction of AKI to detect the patients at risk could be a first step in the discovery and assessment of new therapies, and in improvements of patient outcomes. The advances in clinical informatics and the increasing availability of electronic medical records have allowed the development of predictive models of AKI diagnosis. In this research work we provide a consistent reproducible ETL pipeline for Intensive Care Unit (ICU) data, in particular regarding the MIMIC III database, to support the early prediction of AKI. Then, we build different predictive models aimed at early identifying subjects who could experience AKI syndrome in their next 7 days after the ICU admission. The entire procedure is based on a recently proposed rolling observational window approach. We consider two predictive models, Gradient Boosting Decision Trees and Support Vector Machines, via different platforms.

Temporal phenotyping enables clinicians to better understand observable characteristics of a disease as it progresses. Modelling disease progression that captures interactions between phenotypes is inherently challenging. Temporal models that capture change in disease over time can identify the key features that characterize disease subtypes that underpin these trajectories. These models will enable clinicians to identify early warning signs of progression in specific sub-types and therefore to make informed decisions tailored to individual patients. In this paper, we explore two approaches to building temporal phenotypes based on the topology of data: topological data analysis and pseudo time-series. Using type 2 diabetes data, we show that the topological data analysis approach is able to identify disease trajectories and that pseudo time-series can infer a state space model characterized by transitions between hidden states that represent distinct temporal phenotypes. Both approaches highlight lipid profiles as key factors in distinguishing the phenotypes.

OBJECTIVES: We propose a one-shot, privacy-preserving distributed algorithm to perform logistic regression (ODAL) across multiple clinical sites. MATERIALS AND METHODS: ODAL effectively utilizes the information from the local site (where the patient-level data are accessible) and incorporates the first-order (ODAL1) and second-order (ODAL2) gradients of the likelihood function from other sites to construct an estimator without requiring iterative communication across sites or transferring patient-level data. We evaluated ODAL via extensive simulation studies and an application to a dataset from the University of Pennsylvania Health System. The estimation accuracy was evaluated by comparing it with the estimator based on the combined individual participant data or pooled data (ie, gold standard). RESULTS: Our simulation studies revealed that the relative estimation bias of ODAL1 compared with the pooled estimates was <3%, and the ratio of standard errors was <1.25 for all scenarios. ODAL2 achieved higher accuracy (with relative bias <0.1% and ratio of standard errors <1.05). In real data analysis, we investigated the associations of 100 medications with fetal loss during pregnancy. We found that ODAL1 provided estimates with relative bias <10% for 85% of medications, and ODAL2 has relative bias <10% for 99% of medications. For communication cost, ODAL1 requires transferring p numbers from each site to the local site and ODAL2 requires transferring (p×p+p) numbers from each site to the local site, where p is the number of parameters in the regression model. CONCLUSIONS: This study demonstrates that ODAL is privacy-preserving and communication-efficient with small bias and high statistical efficiency.

OBJECTIVE: We developed and evaluated a privacy-preserving One-shot Distributed Algorithm to fit a multicenter Cox proportional hazards model (ODAC) without sharing patient-level information across sites. MATERIALS AND METHODS: Using patient-level data from a single site combined with only aggregated information from other sites, we constructed a surrogate likelihood function, approximating the Cox partial likelihood function obtained using patient-level data from all sites. By maximizing the surrogate likelihood function, each site obtained a local estimate of the model parameter, and the ODAC estimator was constructed as a weighted average of all the local estimates. We evaluated the performance of ODAC with (1) a simulation study and (2) a real-world use case study using 4 datasets from the Observational Health Data Sciences and Informatics network. RESULTS: On the one hand, our simulation study showed that ODAC provided estimates nearly the same as the estimator obtained by analyzing, in a single dataset, the combined patient-level data from all sites (ie, the pooled estimator). The relative bias was <0.1% across all scenarios. The accuracy of ODAC remained high across different sample sizes and event rates. On the other hand, the meta-analysis estimator, which was obtained by the inverse variance weighted average of the site-specific estimates, had substantial bias when the event rate is <5%, with the relative bias reaching 20% when the event rate is 1%. In the Observational Health Data Sciences and Informatics network application, the ODAC estimates have a relative bias <5% for 15 out of 16 log hazard ratios, whereas the meta-analysis estimates had substantially higher bias than ODAC. CONCLUSIONS: ODAC is a privacy-preserving and noniterative method for implementing time-to-event analyses across multiple sites. It provides estimates on par with the pooled estimator and substantially outperforms the meta-analysis estimator when the event is uncommon, making it extremely suitable for studying rare events and diseases in a distributed manner.

The detection, modeling and characterization of complex patterns of association in bioinformatics has focused on feature interactions and, more recently, instance-subgroup specific associations (e.g. genetic heterogeneity). Previously, attribute tracking was proposed as an instance-linked memory approach, leveraging the incremental learning of learning classifier systems (LCSs) to track which features were most useful in making class predictions within individual instances. These 'attribute tracking' signatures could later be used to characterize patterns of association in the data. While effective, true underlying patterns remain difficult to characterize in noisy problems, and the original approach places equal weight on tracked feature scores obtained early as well as late in learning. In this work we investigate alternative strategies for attribute tracking scoring, including the adoption of a time recency update scheme taken from reinforcement learning, to gain insight into how to optimize this approach to improve modeling performance and downstream pattern interpretability. We report mixed results over a variety of performance metrics that point to promising future directions for building effective building blocks and improving model interpretability.

Background: Globally, 36% of deaths among children can be attributed to environmental factors. However, no comprehensive list of environmental exposures exists. We seek to address this gap by developing a literature-mining algorithm to catalog prenatal environmental exposures. Methods: We designed a framework called. PEPPER: Prenatal Exposure PubMed ParsER to a) catalog prenatal exposures studied in the literature and b) identify study type. Using PubMed Central, PEPPER classifies article type (methodology, systematic review) and catalogs prenatal exposures. We coupled PEPPER with the FDA's food additive database to form a master set of exposures. Results: We found that of 31 764 prenatal exposure studies only 53.0% were methodology studies. PEPPER consists of 219 prenatal exposures, including a common set of 43 exposures. PEPPER captured prenatal exposures from 56.4% of methodology studies (9492/16 832 studies). Two raters independently reviewed 50 randomly selected articles and annotated presence of exposures and study methodology type. Error rates for PEPPER's exposure assignment ranged from 0.56% to 1.30% depending on the rater. Evaluation of the study type assignment showed agreement ranging from 96% to 100% (kappa = 0.909, p < .001). Using a gold-standard set of relevant prenatal exposure studies, PEPPER achieved a recall of 94.4%. Conclusions: Using curated exposures and food additives; PEPPER provides the first comprehensive list of 219 prenatal exposures studied in methodology papers. On average, 1.45 exposures were investigated per study. PEPPER successfully distinguished article type for all prenatal studies allowing literature gaps to be easily identified.

OBJECTIVE: Cultural and health service obstacles affect the quality of pregnancy care that women from vulnerable populations receive. Using a participatory design approach, the Stress in Pregnancy: Improving Results with Interactive Technology group developed specifications for a suite of eHealth applications to improve the quality of perinatal mental health care. MATERIALS AND METHODS: We established a longitudinal participatory design group consisting of low-income women with a history of antenatal depression, their prenatal providers, mental health specialists, an app developer, and researchers. The group met 20 times over 24 months. Applications were designed using rapid prototyping. Meetings were documented using field notes. RESULTS AND DISCUSSION: The group achieved high levels of continuity and engagement. Three apps were developed by the group: an app to support high-risk women after discharge from hospital, a screening tool for depression, and a patient decision aid for supporting treatment choice. CONCLUSION: Longitudinal participatory design groups are a promising, highly feasible approach to developing technology for underserved populations.

OBJECTIVE: To review the published, peer-reviewed literature on clinical research data warehouse governance in distributed research networks (DRNs). MATERIALS AND METHODS: Medline, PubMed, EMBASE, CINAHL, and INSPEC were searched for relevant documents published through July 31, 2013 using a systematic approach. Only documents relating to DRNs in the USA were included. Documents were analyzed using a classification framework consisting of 10 facets to identify themes. RESULTS: 6641 documents were retrieved. After screening for duplicates and relevance, 38 were included in the final review. A peer-reviewed literature on data warehouse governance is emerging, but is still sparse. Peer-reviewed publications on UK research network governance were more prevalent, although not reviewed for this analysis. All 10 classification facets were used, with some documents falling into two or more classifications. No document addressed costs associated with governance. DISCUSSION: Even though DRNs are emerging as vehicles for research and public health surveillance, understanding of DRN data governance policies and procedures is limited. This is expected to change as more DRN projects disseminate their governance approaches as publicly available toolkits and peer-reviewed publications. CONCLUSIONS: While peer-reviewed, US-based DRN data warehouse governance publications have increased, DRN developers and administrators are encouraged to publish information about these programs.

SUMMARY: We have developed medpie, a software package for preparing medical message board corpora and extracting patient mentions and statistics for drugs, herbs and adverse effects experienced from them. The package is divided into web-crawling, HTML-cleaning, de-identification and information extraction modules. It also includes a sample controlled vocabulary of drugs, herbs and adverse effect terms. AVAILABILITY: http://www.cis.upenn.edu/~ungar/medpie.zip. DEPENDENCIES: Python 2.6 or 2.7.

The AMIA Public Health Informatics 2011 Conference brought together members of the public health and health informatics communities to revisit the national agenda developed at the AMIA Spring Congress in 2001, assess the progress that has been made in the past decade, and develop recommendations to further guide the field. Participants met in five discussion tracks: technical framework; research and evaluation; ethics; education, professional training, and workforce development; and sustainability. Participants identified 62 recommendations, which clustered into three key themes related to the need to (1) enhance communication and information sharing within the public health informatics community, (2) improve the consistency of public health informatics through common public health terminologies, rigorous evaluation methodologies, and competency-based training, and (3) promote effective coordination and leadership that will champion and drive the field forward. The agenda and recommendations from the meeting will be disseminated and discussed throughout the public health and informatics communities. Both communities stand to gain much by working together to use these recommendations to further advance the application of information technology to improve health.

There are millions of public posts to medical message boards by users seeking support and information on a wide range of medical conditions. It has been shown that these posts can be used to gain a greater understanding of patients' experiences and concerns. As investigators continue to explore large corpora of medical discussion board data for research purposes, protecting the privacy of the members of these online communities becomes an important challenge that needs to be met. Extant entity recognition methods used for more structured text are not sufficient because message posts present additional challenges: the posts contain many typographical errors, larger variety of possible names, terms and abbreviations specific to Internet posts or a particular message board, and mentions of the authors' personal lives. The main contribution of this paper is a system to de-identify the authors of message board posts automatically, taking into account the aforementioned challenges. We demonstrate our system on two different message board corpora, one on breast cancer and another on arthritis. We show that our approach significantly outperforms other publicly available named entity recognition and de-identification systems, which have been tuned for more structured text like operative reports, pathology reports, discharge summaries, or newswire.

There are millions of public posts to medical message boards by users seeking support and information on a wide range of medical conditions. It has been shown that these posts can be used to gain a greater understanding of patients' experiences and concerns. As investigators continue to explore large corpora of medical discussion board data for research purposes, protecting the privacy of the members of these online communities becomes an important challenge that needs to be met. Extant entity recognition methods used for more structured text are not sufficient because message posts present additional challenges: the posts contain many typographical errors, larger variety of possible names, terms and abbreviations specific to Internet posts or a particular message board, and mentions of the authors' personal lives. The main contribution of this paper is a system to de-identify the authors of discussion board posts automatically, taking into account the aforementioned challenges. We demonstrate our system on two different message board corpora, one on breast cancer and another on arthritis. We show that our approach significantly outperforms other publicly available de-identification systems, which have been tuned for more structured text like operative reports, pathology reports and discharge summaries. Our software will be available for download as open source code in the near future.

The Core Content for Clinical Informatics defines the boundaries of the discipline and informs the Program Requirements for Fellowship Education in Clinical Informatics. The Core Content includes four major categories: fundamentals, clinical decision making and care process improvement, health information systems, and leadership and management of change. The AMIA Board of Directors approved the Core Content for Clinical Informatics in November 2008.

The Program Requirements for Fellowship Education identify the knowledge and skills that physicians must master through the course of a training program to be certified in the subspecialty of clinical informatics. They also specify accreditation requirements for clinical informatics training programs. The AMIA Board of Directors approved this document in November 2008.

In its 1999 report, the Institute of Medicine identified medical error as an important factor in mortality of hospitalized patients, contributing to as many as 98,000 deaths annually. 1 These errors are costly as well, incurring an annual cost of as much as 30 billion dollars in lost income and excessive health care expenditures. As prevalent as these errors are, 35% of internal medicine clerkship directors had, in 2000, “little or no familiarity with the Institute of Medicine's report.” 2 This unacceptable level of error in health care, and lack of awareness of its importance send a clear call to educators to address the issue of patient safety. This call extends to all dimensions of medical education, including undergraduate medical education. 3 In fact, one could argue that patient safety belongs first in a medical school curriculum, and that it should be an ongoing educational endeavor, one that continues throughout a physician's career. However, one may justifiably wonder about the optimal method for implementing a patient safety curriculum in the medical school. Many approaches to undergraduate medical education have included an informatics-based platform. In particular, the design and implementation of computer-based instructional materials have revolutionized the way preclinical and clinical subjects have been taught in medical schools over the past decade. Very few of these materials, however, have focused on patient safety, and given their success in preclinical and clinical domains, an informatics-centric approach to teaching patient safety should be considered. The evidence of need for a medical school curriculum related to patient safety is compelling. It has been observed that physician trainees use the mechanisms of denial, discounting, and distancing to define and defend medical error. 4 In a review of medical error, Lester and Tritter noted that recurrent themes in errors, such as uncertainty, fallibility, and exclusivity of judgment, are probably rooted in medical education. 5 Others 6 , 7 have called for the implementation of programs and materials necessary for teaching medication or general patient safety in medical schools. Most compelling of all, Rosebraugh et al. 3 surveyed medical school clerkship program directors to ascertain the prevalence and depth of educational programs related to medical errors. Only 16% of respondents provided formal lectures about medication errors, yet most (65%) said they would incorporate short educational modules about errors and adverse events, if they were available. Boreham et al. 8 concluded that medical errors result from the “lack of a knowledge base which integrate[s] scientific knowledge with clinical know-how.” Clearly, there is a real need for patient safety curriculum material that is useful at the undergraduate medical education level for reducing medical errors at all levels and that will provide the knowledge base medical students require for safe medical practice. The patient safety curriculum also needs to be carried forward into the practical setting of clerkships in organizations with cultures that support improvement and safety. James Reason described two types of error: active and latent. 9Active errors are produced by human hands and are possibly addressable through appropriate education and practice. Latent errors result from poorly designed or implemented systems or procedures. These errors are likely addressable through the education of system developers and management and tangentially through the education of users. Either way, informatics has an important role to play in the development of educational materials that can attenuate the risk of medical error committed by medical students. This paper provides a summary of how the articles included in this special JAMIA supplement may be used to design or refine medical school curricula in patient safety. Medical students could benefit from a problem-based learning curriculum in patient safety that lays the groundwork for understanding the types of errors, especially as they occur in the context of technology-laden patient care and research settings. The problem-based learning format would encourage discussion, collaboration, quality improvement, and the value of learning from error in a variety of informatics domains related to clinical practice. To this end, four such domains are discussed: clinical systems, human factors and communication, knowledge representation, and protection of confidentiality. The presence of clinical information systems is increasing in every patient care setting. These systems support specialized, often complex, tasks, such as order entry, laboratory and radiology reporting, clinical data entry and retrieval, and decision support. Medical students will be exposed to these systems earlier in their career; they will be expected to train on these systems quickly and to use them accurately as soon as they arrive on the inpatient floor or in the outpatient clinic for their first clinical rotation. Often, an information system educates the user, wittingly or unwittingly. Systems that use clinical guidelines for improving care have burgeoned over the past decade, but it is not yet clear that guideline-based systems actually improve care, because the extent of physician adherence to guidelines in the first place is unknown. Goldstein et al. report on a guideline-based decision support system (ATHENA) to improve care for hypertension. This paper provides an excellent introduction to the design, implementation, use, and limitations of clinical decision support systems, all of which would be helpful for educating medical students about the appropriate role and use of these systems. In addition to Goldstein et al, Advani et al. address the problem of how these systems are used in practice by means of a specialized quality indicator language that can be used by guideline authors so that adherence to their intentions can be scored for quality assessment. Medical students could benefit from using this tool, both in formulating guidelines as practice exercises and in assessing the quality of care in a clinical setting. Other clinical information systems focus more on monitoring. Boëlle et al. report on a surveillance system for capturing adverse anesthesia events. This system provides a good example of how specialized surveillance systems can easily be implemented, even in a complicated setting such as the operating room. Also in the anesthesia domain, Sawa and Ohno-Machado report on a decision support system that uses set theory to reduce errors. This system generates dynamic checklists of intraoperative problems, which would be useful in teaching medical students about the kinds of problems that can arise in the operating room and in developing strategies for dealing with them. Another approach to surveillance, using the ICD-9 as a representation of chief complaint and therefore a sensitive and predictive sensor of epidemics, is reported by Tsui et al. Although the focus of this paper was on bioterrorism surveillance, the methods could be applied to the surveillance of medical errors. Given the ubiquity of the ICD-9 coding system, this paper provides medical students with a good introduction to how the system can be used for surveillance purposes. In another approach to surveillance, Einbinder and Scully used a retrospective approach to identifying adverse drug events (ADEs) from a clinical data warehouse. They found that a substantial number of ADEs occurred annually in their site, and these results should be useful to medical students in understanding the magnitude of the problem. In their work correlating ADEs with medication errors, Gandhi et al. found no difference in the rates of these events between manual and computerized prescribing systems. This suggests that practitioners should be equally vigilant in prescribing on either type of system. Expanding the domain to the more general adverse event (AE), Murff et al. found an even larger prevalence: they identified AEs in 31% of patients on retrospective review of discharge summaries using an electronic abstracting tool. The increasing interest in patient safety has led to identifying the factors associated with human-computer interaction and communication that may increase the likelihood of medical errors. McKnight et al. found that providers have difficulty obtaining the type of information they need, even though such information is available. Access to this information is a key variable in the successful meeting of diagnostic and treatment needs, and the authors propose that this phenomenon suggests a computerized-solution. Furthermore, awareness of this problem in itself is an important step in mitigating the risks associated with poor communication. Zhang et al. postulate that the problem of medical errors will not be solved within medicine. Rather, this is a problem for cognitive science and engineering, as it is primarily a human factors issue. They argue that there is a system hierarchy of medical errors grounded in cognitive processes. This paper provides an excellent means for stimulating discussion of a potentially controversial subject that the problem of medical errors may eventually be solved by non-medical disciplines. Nowhere in clinical practice is the realm of human factors and communication research more germane than in high-throughput clinical areas such as operating rooms and emergency departments. Weinger and Slagle performed a task analysis and workload assessment to evaluate clinical decision making among anesthesiologists. This research focused on the identification of nonroutine events associated with seven surgical procedures, and the prospective collection of these events as a means of identifying changes in clinical processes and practices to improve patient safety. Moss et al. documented the patterns of communication of an operating room charge nurse. Strategies for improving communication, such as an electronic operating room schedule, available throughout the hospital's clinical areas, as well as the implementation of an asynchronous messaging system, were offered as two ways of reducing the communication overload the charge nurse often experiences. The development of knowledge representation suitable for capturing adverse events relies on agreement by a number of regulatory and professional bodies as to the form and content of controlled vocabularies, ontologies, and taxonomies. Nebeker et al. provide a review of the problem of conflicting taxonomies and even definitions as they pertain to adverse drug events. Rather than offer a solution to the problem, this paper is an excellent resource for understanding the problems associated with defining, describing, and reporting ADEs. Stetson et al. describe their initial efforts at developing an ontology, using the Unified Medical Language System, to model medical errors. The authors use a conceptual graph notation to define the schemas for communication space, information needs, and errors, which they considered to be foundational information for identifying the concepts related to medical errors. The confidentiality of patient information is coming under ever-increasing scrutiny, especially with the advent of HIPAA. Clinicians need to become more aware of the need for preserving confidentiality, as additional types of information become available and are represented in print and electronic media. Breaches of confidentiality can be seen rightly as medical errors in their own right, because they potentially stand to compromise patient care and confidence. Medical students need to learn early on in their careers that preserving patient confidentiality is a sacred trust that is not limited solely to clinical data but rather extends to data collected in research settings as well. Two papers in this issue address the data needs of researchers, in balance with the need to preserve confidentiality. Dreiseitl et al. and Ohno-Machado et al. discuss the use of anonymized data in research, where sensitive data is purposely ambiguated to avoid linkage or identification. Such data can be disambiguated, however, through the use of the anonymization algorithm originally used on the data. Ohno-Machado et al. go so far as to show that inferences, such as those obtained from predictive models, can be constructed from ambiguated data, which further demonstrates that reliance on the ability of anonymization algorithms to preserve confidentiality may be misplaced and should be reconsidered. These discoveries have tremendous implications for preserving the anonymity of people represented in research or other data, and these implications should be discussed with medical students, whether or not they plan on a research career. The education of clinicians to the importance of avoiding medical errors and mitigating risks to patients should be paramount in any medical school curriculum. Yet very few training programs identify formal education in medical errors or patient safety. More positively, however, most programs would use patient safety training materials if they were available. The papers in this supplement of JAMIA have a special informatics focus on at least four different dimensions, which address the needs of users, developers, and ultimately, patients as the primary stakeholders in patient safety. To provide the best care possible, medical students need to understand the implications of information technology in medicine: that as attractive as it is in improving patient care, practitioners have a responsibility to understand its supportive, and at times potentially compromising, role in patient safety. In addition, medical students need to understand the extent of medical knowledge held by their patients and sources of information available to patients about their own health. As informatics solutions increasingly help solve the problem of reducing the prevalence and effect of medical errors, it is critical that the papers in this supplement be incorporated into a medical school curriculum about patient safety. Doing so will help students understand the source of the problem, its effects, and possible solutions.

The learning classifier system (LCS) integrates a rule-based system with reinforcement learning and genetic algorithm-based rule discovery. This investigation reports on the design, implementation, and evaluation of EpiCS, a LCS adapted for knowledge discovery in epidemiologic surveillance. Using data from a large, national child automobile passenger protection program, EpiCS was compared with C4. 5 and logistic regression to evaluate its ability to induce rules from data that could be used to classify cases and to derive estimates of outcome risk, respectively. The rules induced by EpiCS were less parsimonious than those induced by C4.5, but were potentially more useful to investigators in hypothesis generation. Classification performance of C4.5 was superior to that of EpiCS (P<0.05). However, risk estimates derived by EpiCS were significantly more accurate than those derived by logistic regression (P<0.05).