OBJECTIVE: To examine how social media analytics have been applied in research on firearm injury exposure, with a focus on informatics approaches, analytical methodologies, and public health surveillance applications. MATERIALS AND METHODS: Following the PRISMA-ScR framework for scoping reviews, we systematically searched 5 databases (Web of Science, Scopus, PubMed, IEEE Xplore, ACM Digital Library) for studies published 2014-2025 that used social media analytics to investigate firearm injury exposure. The most recent search was conducted on February 18, 2025. Two independent reviewers screened studies using standardized criteria and extracted study characteristics via Covidence. Inter-rater reliability was found to be (Cohen's κ = 0.63). Of 742 initial records, 16 studies met the inclusion criteria. RESULTS: All included studies (16/16; 100%) used X (formerly Twitter) as the data source. Analytical approaches were natural language processing (n = 12), topic modeling (n = 8), and sentiment analysis (n = 6). Most studies were US-based (n = 12) and examined direct exposure (eg, witnessing shootings) and indirect exposure (eg, media coverage). Key informatics applications included sentiment detection, temporal discourse pattern analysis, and computational methods for evaluating community-level impacts. DISCUSSION: Findings revealed significant methodological homogeneity, with overreliance on X and limited use of longitudinal designs. Implementations of analytics methods varied considerably. Lack of platform diversity and standardization limits generalizability and integration with public health surveillance systems, impeding translation into policy and intervention. CONCLUSION: Social media analytics represent a promising tool for advancing public health informatics related to firearm injuries. Future research should employ platform diversity, longitudinal approaches, and computational metrics to enhance integration with health information systems.

OBJECTIVES: Advancing health through informatics requires attending to justice. Recent policy changes in the United States have introduced significant barriers to promoting justice within informatics due to targeted funding cuts and hostility to science, especially science that prioritizes justice. MATERIALS AND METHODS: We present five key principles for advancing a justice-oriented informatics agenda, synthesized from our workshop held at the American Medical Informatics Association 2022 Annual Symposium. RESULTS: These principles are: (1) Recognize knowledge and methodologies across communities; (2) Acknowledge historical and cultural contexts of interactions; (3) Facilitate transparency and accountability through clear measures and metrics; (4) Foster trust and sustainability; and (5) Equitably allocate compensation and resources. DISCUSSION AND CONCLUSION: We discuss barriers to implementing these principles that have arisen since the 2022 workshop and provide recommendations for moving towards justice-oriented informatics. We offer examples of how these principles may be used to frame challenges and adapt to new barriers within BMI.

A Learning Health System (LHS) is one “in which science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the delivery process and new knowledge captured as an integral by-product of the delivery experience.”1 For almost 2 decades, there has been a clear recognition of the criticality of informatics infrastructure, technologies, and processes as the underpinning of an LHS.2 The Journal of the American Medical Informatics Association (JAMIA) has been a premier venue for publishing on the topic. For example, the April 2014 issue included multiple papers that described scalable infrastructure for an LHS.3–5 A 2015 paper by Friedman et al. reported on a National Science Foundation workshop that resulted in a research agenda for the high-functioning LHS and called for an interdisciplinary science of learning systems.6 Despite some progress toward an LHS, in a Perspective in this issue, Gunderson, Embi, Friedman, and Melton argue that the informatics community has been relatively slow to formalize LHS as a priority area.7 To generate informatics priorities for an LHS, they compiled results from a short survey of LHS leaders and American Medical Informatics Association (AMIA) members, discussion from an LHS reception at the AMIA annual meeting, and a follow-up survey to inform priorities at the intersection of LHS and informatics. Through a thematic analysis, the authors identified 7 opportunities at the intersection of LHS and informatics: Understanding and Context, Shared Resources, Collaboration, LHS Education, Data and Data Exchange, Demonstration and Evaluation, and Patient Centeredness (ie, inclusion of patient voices). They also identified immediate LHS informatics priorities: (1) establish informatics LHS forum(s); (2) disseminate case reports of LHS informatics successes and failures; (3) create LHS informatics education resources; and (4) advance understanding of LHS principles in informatics.

My nursing education more than 40 years ago and subsequent clinical experience included consideration of individuals in the context of family and broader environment and thinking about dyads, families, organizations, and communities as “units” of assessment, analysis, and intervention. In this editorial, I highlight papers not because of a common informatics theme, but because they address biomedical and health informatics issues and tools beyond the individual (ie, dyads, caregivers, families, clinical teams, and geographic location). To inform point-of-care needs assessment for the substance-exposed dyad, Bourgeois et al.1 examined the current state of birthing person and infant/child (dyad) data-sharing capabilities from the perspective health information exchange (HIE) standards and HIE network capabilities for data exchange. They cross-mapped a set of dyadic data elements focused on pediatric development and longitudinal supportive care for substance-exposed dyads: 70 birthing person and 110 infant/child elements to identify definitional alignment to standardized data fields within national healthcare data exchange standards, the United States Core Data for Interoperability (USCDI) version 4 (v4) and Fast Healthcare Interoperability Resources (FHIR) release 4 (R4), and applicable structured vocabulary standards or terminology. More than 88% of dyadic data elements cross-mapped to at least 1 USCDI standardized data field and to FHIR R4. Among the 13 HIE networks surveyed or interviewed, 9 responded; of those, 75% reported supporting USCDI versions 1 or 2 and the capability to use FHIR suggesting sufficient HIE support for substance-exposed dyads.

The volume of literature related to the application of large language models (LLMs) in the biomedical and health domain has dramatically increased in the last 2 years. This is reflected in the 2024 focus issue1 in the Journal of the American Medical Informatics Association (JAMIA) on the topic as well as papers in every issue. In this issue, I highlight 5 papers. Two papers provide frameworks for development, implementation, and evaluation of LLMs in clinical settings2,3 and 2 focus on use of LLMs to facilitate systematic review processes.4,5 The fifth paper is a scoping review that summarizes the literature on applying natural language processing including LLMs to genomic sequencing data.6 Liu, McCoy, and Wright2 conducted a systematic review and meta-analysis to synthesize recent research on retrieval-augmented generation (RAG) and LLMs in biomedicine and provide clinical development and implementation guidelines to improve effectiveness. They specifically examined studies that compared baseline LLM performance with RAG performance. In the 20 studies in the review, resources used for RAG varied from single sources to large data sets and reflected different RAG strategies by stage: pre-retrieval, retrieval, and post-retrieval. Studies reflected human (n = 9), automated (n = 8), and human and automated (n = 3) evaluation methods. In a random-effect meta-analysis model that used odds ratio as the effect size, the pooled effect size was 1.35 indicating better performance with RAG. Based on their study findings, the authors developed the GUIDE-RAG (Guidelines for Unified Implementation and Development of Enhanced LLM Applications with RAG in Clinical Settings) Framework which specifies best practices by RAG stage. They also specify 3 future research directions: (1) system-level enhancement: the combination of RAG and agent, (2) knowledge-level enhancement: deep integration of knowledge into LLM, and (3) integration-level enhancement: integrating RAG systems within electronic health records. Hong et al.3 propose a unified evaluation framework that bridges qualitative and quantitative methods to assess LLM performance in healthcare settings. The framework maps evaluation aspects (linguistic quality, efficiency, content integrity, trustworthiness, and usefulness) to qualitative human assessments and quantitative metrics. They demonstrate the applicability of the framework by evaluating the Epic In-Basket feature, which uses LLM to generate patient message replies. Clinician decisions to use LLM-generated patient message drafts correlated strongly with quantitative metrics. This finding suggests that quantitative metrics have the potential to reduce human effort in the evaluation of LLM output. The authors also note that the framework can serve as the foundation relevant for derivation of benchmarks that can be applied to further LLM monitoring and evaluation in healthcare settings. In the context of living systematic reviews, Khan et al.4 focused on the laborious data extraction process typically undertaken by 2 human reviewers. The dataset for the analysis comprised 22 publications from a published living systematic review including 23 variables related to trial, population, and outcomes data. The dataset was split into prompt development (n = 5) and test datasets (n = 17) and data were extracted by GPT-4-turbo and Claude-3-Opus. Concordant LLM responses were 96% in the prompt development dataset and 87% in the test dataset; accuracy of the concordant responses against the human gold standard for the datasets was 0.99 and 0.94, respectively. The accuracy of discordant responses was 0.41 for GPT-4-turbo and 0.50 for Claude-3-Opus. These findings suggest that concordant responses by the LLMs are likely to be accurate and can facilitate the process of living systematic reviews. Li et al.5 developed and validated an LLM-assisted system for conducting systematic literature reviews in the domain of health technology assessment. The system comprises 5 modules: (1) literature search query setup; (2) study protocol setup using population, intervention/comparison, outcome, and study type criteria; (3) LLM-assisted abstract screening; (4) LLM-assisted data extraction; and (5) data summarization. The system collects information on disagreements between the LLM and human reviewers regarding inclusion/exclusion decisions and rationales. The system was evaluated using 4 datasets of PubMed abstracts on 3 tasks: (1) recommending inclusion/exclusion decisions during abstract screening, (2) providing valid rationales for abstract exclusion, and (3) extracting relevant information from abstracts. Across tasks and datasets, the system attained accuracy of at least 84% demonstrating potential to streamline systematic reviews. The scoping review by Chen et al.6 focused on applying natural language processing techniques, particularly LLMs and transformer architectures, to decipher genomic sequencing data. The analysis of the 26 studies in the review suggests that tokenization and transformer models enhance the processing and understanding of the complex structure of genomic data. The authors suggest the application of natural language processing including LLMs has the potential to drive advancements in personalized medicine by offering more efficient and scalable solutions for genomic analysis. The papers in this issue demonstrate the rapid progress in the application of LLMs for a variety of applications and tasks in health and biomedicine. Moreover, the papers by Liu et al.2 and Hong et al.3 offer frameworks that can promote comparisons across applications and tasks and contribute to generalizable findings and scalable solutions. None declared. None declared. Not applicable.

In the first issue of 2024, the 4 editors of the Journal of the American Medical Association (JAMIA) over its first 30 years, delineated an additional goal for JAMIA which is to: exemplify best practices in publishing to advance health equity and justice through promoting (1) diversity, equity, and inclusion in editorial team and processes; (2) use of inclusive and non-stigmatizing language; and (3) innovative, rigorous, and transparent quantitative and qualitative research methods that address fairness and mitigate epistemic injustice.1 In support of this goal, in this editorial, I highlight 2 review papers focused on equity related to digital health tools,2,3 a perspective from the American Medical Informatics Association’s (AMIA) Global Health Informatics Working Group focused on equitable and representative academic partnerships in global health informatics research,4 a paper that examines the role of the patient trust in artificial intelligence (AI),5 and a perspective from the AMIA Climate, Health, and Informatics Working Group that includes examination of the association between climate change and social vulnerability.6 Kim and Backonja2 conducted a scoping review to identify and describe published frameworks and concepts relevant to digital health equity—the opportunity for all to engage with digital health tools to support good health outcomes—interventions. Using the socio-ecological model to inform the analysis, they identified 243 concepts from 42 frameworks using grouped into 43 categories that included characteristics of individuals, communities, and organizations; societal context; perceived intervention value and impacts on individuals, community members, and the organization; partnerships; and access to digital health services, in-person services, digital services, and data and information. Based on their analysis, the authors suggest a consolidated definition of digital health equity as “a multi-level socio-ecological concept that results from fair and just opportunities for everyone to attain their highest level of health through access to technology-enabled health resources and services.” Arthur et al3 examined the equity implications of extended reality (XR) technologies for health and procedural anxiety through a systematic review. They extracted equity-relevant data from reviews of patient-directed XR technologies for health and procedural anxiety and deductively categorized data along multiple axes: (1) review-level or within-review extraction; (2) equity-relevant characteristics; (3) whether statements relate to availability, accessibility, or acceptability of relevant XR interventions; and (4) whether statements relate to availability, accessibility, or acceptability of the health or procedural contexts linked to XR interventions. Subsequently, they developed a novel implementation-focused framework. The “double jeopardy, common impact” framework outlines unique pathways through which XR technologies could help address health disparities but also have the potential to accelerate or even generate inequity across different systems, communities, and individuals. Given the latter concern, the authors emphasize the need for taking a cautious, inclusive approach to XR implementation in future programs. Campbell et al4 report on principles and implementation strategies for equitable and representative academic partnerships in global health informatics research. The foundation for the principles was a workshop organized by the AMIA Global Health Informatics Working Group, at the 2023 AMIA Annual Symposium that included the presentation of 5 case studies reflecting incorporating principles of health equity into projects in low-and-middle-income countries and with Indigenous communities in the United States. This was followed by focus group discussions. The 5 core principles are: (1) inclusion and participation in ethical, sustainable collaborations; (2) engaging community-based participatory research approaches; (3) stakeholder engagement; (4) scalability and sustainability; and (5) representation in knowledge creation. The authors concluded that “equitable, sustainable, and scalable global health informatics projects require intentional integration of community and stakeholder perspectives in project development, implementation, and knowledge creation processes.” Nong and Ji5 surveyed a cross-sectional national sample of US adults (n = 2039) and applied population weights to produced national estimates of patient expectations of healthcare AI. Almost 20% of patients expected AI to improve their relationship with their doctor and to increase affordability of health care, while 30% expected it to improve their access to care. They also found that trust in providers and the healthcare system is positively associated with expectations of AI controlling for demographic factors, general attitudes toward technology, and other healthcare-related variables. The authors also suggest that prioritization of patient benefits is important to preserving or promoting trust. However, the relationship between expectations of AI and trust in providers and the healthcare system raises concerns about health-disparate populations who may have lower trust in providers and healthcare systems. Building on the findings of a mini-summit held during the AMIA 2023 Annual Symposium, Schleyer et al6 offer a call to action to the informatics community on its role in climate change. Hosted by the AMIA Climate, Health, and Informatics Working Group (at the time, an AMIA Discussion Forum), the International Medical Informatics Association (IMIA), the International Academy of Health Sciences Informatics (IAHSI), and the Regenstrief Institute, the mini-summit summit discussion was motived by 2 questions to participants: What evidence-based professional practices can individuals, groups, and organizations in healthcare apply or implement now to help mitigate or adapt to climate change? and What research should the informatics community conduct to help the healthcare profession mitigate or adapt to climate change? The resulting mini-summit action of “assess community vulnerability” is of particular relevance to the focus of this editorial. The associated implementation strategies included: (1) identify populations at increased risk from climate impacts, and (2) develop resource allocation plans based on social vulnerability indices. The mini-summit reflects an important step in helping the informatics community to set application and research priorities at the intersection of climate, health, and informatics. As envisioned by our double issue focused on health equity in 20197 and reflected in our recent focus issue on returning value from the All of Us Research Program, the topics of diversity, equity, and inclusion remain relevant to publishing in JAMIA and to advancing the science and application of biomedical and health informatics.8 None declared. None declared. Not applicable.

Papers on large language models (LLMs) continue to be a focus of many submissions to the Journal of the American Medical Informatics Association (JAMIA). The studies that we publish continue to evolve in their sophistication, rigor, and relevance to clinical tasks and synthesis of scientific literature. In this editorial, I highlight 2 studies focused on a variety of clinical tasks,1,2 a psychometric study in which the authors developed an instrument to measure the quality of LLM-generated summaries,3 and 2 studies related to synthesis of scientific literature.4,5 Dorfner et al1 compared the performance of biomedically fine-tuned LLMs against their general-purpose counterparts across clinical case challenges from 2 medical journals and on multiple clinical tasks (information extraction, question answering, document summarization, and clinical coding). The authors used relevant benchmarks and evaluation metrics for each task General-purpose LLMs and larger biomedical LLMs outperformed small biomedical LLMs on case challenges from Journal of the American Medical Association and New England Journal of Medicine. Similarly, general-purpose LLMs often achieved higher scores in text generation, question answering, and coding. Biomedical LLMs also showed a higher tendency to hallucinate than general LLMs. The authors’ findings suggest that fine-tuning LLMs on biomedical data may not yield the anticipated benefits and that alternative approaches, such as retrieval augmentation, should be further explored for effective and reliable clinical integration of LLMs.

I typically focus my monthly editorial on a particular theme. However, as I enter my last 18 months of my 8-year term as Editor-in-Chief of the Journal of the American Medical Informatics Association (JAMIA), I have been reflecting on what has intrigued and motivated me over my 40-year career in biomedical and health informatics. Three things stand out for me—the breadth of application areas for our common and rapidly evolving methods, the interaction between humans and technology, and the core focus on human health. The 5 highlighted articles reflect these interests. I selected this article because it captures the spirit of medical informatics pioneer, Warner Slack, and his advocacy for the capture and use of patient-generated data in clinical care.1 He et al2 conducted a systematic review and meta-analysis that examined the effectiveness of electronic patient-reported outcome measures (ePROMs) to triage and schedule appointments for adult patients with chronic medical conditions. They limited their search to randomized controlled trials (RCTs) that compared use of ePROMs to facilitate flexible scheduling of appointments (intervention) with conventional scheduling practices (control) with the primary outcome of difference in healthcare utilization (number of outpatient clinic visits, telephone consultations, and unplanned visits). Among the 17 RCTs representing 6469 patients in the review, 6 RCTs had sufficient data for the meta-analysis which indicated that the ePROM-informed group had significantly fewer outpatient clinic visits. In terms of secondary outcomes (n = 10 studies), the ePROM-informed group had improved disease control in 2 studies and improved cancer survival in 2 studies. These results support the promise of incorporating ePROMS into outpatient clinic visit scheduling as a strategy for increasing efficiency without compromising disease outcomes. In the current context of ubiquitous algorithms, this article reiterates the importance of understanding the perspective of those who will be receiving the algorithmic output—a key interest of mine. Williams and co-authors describe the process of incorporating end-user perspectives into the development and implementation planning of a prediction algorithm for new perinatal depression onset.3 This included conducting a focus group (n = 12 providers) and 4 virtual community engagement studios (n = 21 patients). They applied rapid qualitative analysis to code provider and patient perspectives on the patient-facing prediction algorithm’s completeness, interpretability, and acceptability. Providers and patients agreed on the interpretability of the prediction algorithm’s variables and to ensure its completeness, they discussed additional variables believed to be predictive of depression. However, in terms of acceptability of the patient-facing screener, patients expressed a desire to discuss screening results with their provider, while providers were concerned about time for such discussions. Both groups identified the need to connect patients to relevant resources post-screening as well as the limited availability of such resources. Such engagement is critical to successful design and implementation of predictive algorithms in general, but even more important for those that are patient-facing. Dianna Forsythe, another medical informatics pioneer and my colleague at University of California, San Francisco, brought people and organizations into the mix at the American Medical Informatics Association (AMIA) in the 1990s and continues to influence the thinking in the field.4 This article led by long-time JAMIA Associate Editor, Eric Poon, uses a case study approach to survey health systems about organizational priorities, successes, and barriers to implementation during the early generative artificial intelligence (AI) era.5 Forty-three of 67 health systems members of the Scottsdale Institute, a collaborative of US non-profit healthcare organizations, provided data on the deployment status and perceived success of 37 AI use cases across 10 categories. They also identified organizational priorities and barriers to implementation. The most frequently reported top 2 priorities were: reducing caregiver burden and improving satisfaction (72%), patient safety/quality (56%), and workflow efficiency/productivity (53%). The most frequently deployed AI use cases were imaging and radiology in at least limited areas (90%), early detection of sepsis (67%), ambient notes (60%), risk of clinical deterioration (56%), predicting risk unplanned readmission (52%), and in-basket automation (51%). Organizations reported a high degree of success in only a few AI use-case categories: clinical documentation (53%), clinical risk stratification (38%), revenue cycle (23%), and clinical diagnosis (19%). Reported barriers to adoption included: immature AI tools (77%), financial concerns (47%), and regulatory uncertainty (40%). Such studies provide the foundation for national and organizational strategies to address barriers to success of AI implementation. Clinical research informatics became more visible in the AMIA community in the mid-2000s and continues to evolve rapidly.6 In my opinion, one of the most important clinical research informatics tools is the widely deployed Research Electronic Data Capture (REDCap).7 Cheng et al.8 report on the development and evaluation of a framework for creating external modules enabling project-specific REDCap custom functionality (External Module [EM] Framework). The EM Framework, which was developed collaboratively with REDCap Consortium members, provides guidance and standard processes for developers to ensure basic functionality, compatibility, and security across REDCap instances and includes an optional dissemination mechanism, the REDCap Repository of External Modules (Repo), for sharing with other members in the REDCap Consortium. During 2017-2024, 356 external modules were published to the Repo from 59 institutions and used on 29 485 projects at 2107 institutions in 67 countries. Moreover, features from 22 external modules have been integrated into the core REDCap code serving >7700 REDCap Consortium members in 160 countries providing evidence of dissemination into the global research community. As a long-time standards advocate and former nursing terminology expert, the article by Fan et al9 caught my attention because of its use of the Clinical Care Classification (CCC), developed by nursing informatic pioneer, Virginia Saba, as part of a semi-automated pipeline to align electronic health record (EHR) flowsheets and measures, a key part of nursing documentation, between 2 organizations. They transformed flowsheet templates and measures into template-measures (T-M) pairs and aligned them through exact, lexical, or semantic matching. Among 31 255 unique T-M pairs in acute care units and 27 012 T-M pairs in intensive care units, the alignment rate for flowsheet and concept matching was 63% and 53% when matches were restricted to top-ranked CCC concept and 96.5% and 96% when expanded to top 3 CCC concepts. The study findings support the feasibility of leveraging a semi-automated pipeline to streamline the EHR flowsheet alignment and accelerate the manual concept mapping. I hope you have enjoyed my JAMIA selections and I encourage you to select yours from this issue! None declared. Not applicable.

Manuscripts related to predictive models and large language models predominate submissions to the Journal of the American Medical Informatics Association (JAMIA) and are frequently highlighted in my editorials. However, in this editorial, I focus on the human side of our field of biomedical and health informatics. A Perspective describes the processes used to draft, refine, and distribute American Medical Informatics Association (AMIA) Inclusive Language and Context Style Guidelines (the “Guidelines”)—a set of principles for describing those who participate in practice, education, or research initiatives.1 Three highlighted articles focus on aspects of clinical care: intrashospital transport, use of patient-generated data (PGD), and medication administration technologies (MAT).2–4 The fifth article reports a comparative study of trajectories of policy context and digital development in 4 Canadian academic health centers over 30 years.5 In 2023, the AMIA Inclusive Language and Context Style Guidelines were approved by the Board of Directors and made publicly available at AMIA Inclusive Language and Context Style Guidelines | AMIA—American Medical Informatics Association. In a Perspective endorsed by the Board of Directors, Bear Don’t Walk IV and colleagues report on the approach to drafting, refining, and distributing the Guidelines.1 In 2021, AMIA’s Diversity, Equity, and Inclusion (DEI) Task Force (subsequently DEI Committee) initiated actions to establish best practices for language use in biomedical and health informatics research as a strategy to strengthen science and practice. Their iterative approach included consulting existing language guides, AMIA member reviews, external expert reviews, webinars, and workshops. The Guidelines include 4 principles for consideration in scientific communications: plurality, precision, transparency, and destigmatization. Each principle is defined along with relevant examples of use. Notably, the Guidelines are a living document that will continue to be updated with input and feedback. I encourage authors submitting to JAMIA to review the Guidelines and to assess their relevance and application to the submitted work.

In this editorial, I highlight 4 papers that address aspects of clinical decision-making and artificial intelligence (AI). Two papers focus on the explainability of the output of AI-based methods and its relationship to clinical decision-making.1,2 A systematic review examines how electronic health record (EHR) data are summarized, visualized, and aligned with clinical reasoning and decision-making tasks.3 The fourth paper compares the diagnostic performance of clinicians and an AI-based model in wound image assessment and examines the influence of dimensions of clinician expertise on performance.4 Lastly, I highlight a Perspective that explores the potential influence of federal government Executive Orders on research in biomedical and health informatics including that focused on clinical AI algorithms.5 Nycklemoe et al1 developed an explainer algorithm, the pediatric Calculated Assessment of Risk and Triage (pCART) Explainer, that uses clinician EHR documentation to generate text-based explanations for deterioration risk prediction alerts from the pCART risk prediction model. The pCART model produces a risk score and a graphical user interface enables visualization of pCART trends, along with a highlighted list of abnormal vital signs or laboratory results. For children at risk for deterioration, pCART Explainer augments these physiological variables by highlighting alert-relevant contextual information from clinician documentation (eg, rapid breathing, abdominal distension). The pCART Explainer pipeline includes: (1) clinical Text And Knowledge Extraction System (cTAKES) natural language processing engine to convert text to concept unique identifiers (CUIs); (2) Self-Alignment Pretraining for Biomedical Entity Representations (SapBERT) to represent the CUIs as embeddings; and (3) the Label-Aware Attention model to convert CUI embeddings to document embeddings. The study supported the face validity of text-based explanations, but formal evaluation is required to assess the influence on clinician decision-making behaviors. Bauer and Michalowski2 critically reviewed healthcare provider evaluation of explanations produced by explainable AI-based methods to support point-of-care, patient-specific, clinical decision-making within medical settings. Twenty-five studies of 2673 retrieved met inclusion criteria for the review. All studies included physicians as participants with others (i.e., nurses, pharmacists) in a few studies and the majority focused on imaging. Only 2 studies included a formal evaluation framework, and a broad variety of instruments were used for evaluation. Their synthesis of instruments across studies suggests a potential common measure of clinical explainability with 3 indicators: interpretability, fidelity, and clinical value. The authors suggest that “future research should aim to clarify and expand key concepts in healthcare provider evaluation, propose a comprehensive evaluation model positioned in current theoretical knowledge, and develop a valid instrument to support comparisons.” Fan et al3 conducted a systematic review that examines how EHR data are summarized, visualized, and aligned with clinical reasoning and decision-making tasks. Data extracted from 112 studies that met inclusion criteria included EHR data types, information summarization methods, visualization strategies, clinician characteristics, and evaluations. They found 2 primary visualization strategies: timeline-based presentations emphasizing temporal trends and longitudinal tracking, and snapshot-based approaches focusing on status overviews and rapid assessments. They organized extracted data into data-information-visualization (data-info-vis) flows with 3 representative flows emerging: (1) use of structured data to generate patterns for temporal visualizations, supporting tasks such as diagnosis and patient management; (2) abstraction of data into miniature charts, aiding situation-aware understanding and knowledge synthesis; and (3) application of high-level visual metaphors for complex and overarching tasks, such as achieving better care. The findings of the systematic review provide insights for developing visualization-based clinical decision-support tools. A substantial body of recent literature has compared AI-based models and clinician performance on a variety of tasks. Using a diagnostic task involving 30 chronic wound images with and without maceration, Kücking et al4 compared the performance of clinicians (n = 481) and an AI-based model (Convolutional Neural Network) and examined the influence of dimensions of clinician expertise on performance. There were no significant differences in performance. The authors examined predictors of performance using multiple measures of expertise (formal qualification, work experience, self-confidence, and wound focus) finding that only formal qualification and self-confidence significantly predicted clinician accuracy. The authors highlight the importance of defining clinician expertise in comparative studies. In a Perspective, Hochheiser and Visweswaran5 discuss the implications of executive orders (e.g., diversity, equity, and inclusion, sex, gender) issued by the U.S. federal government for biomedical informatics research. These include, but are not limited to, hindering research on critical topics such as bias and fairness in clinical AI and machine learning algorithms, narrowing the scope of questions addressed by informatics research, limiting the understanding of the effectiveness of informatics resources or who they benefit and obstructing efforts to enhance the diversity of perspectives within the field. The authors call upon the American Medical Informatics Association and the broader informatics community to support inclusive perspectives to advance research, health care, and health. Supporting clinical decision-making has consistently been a key focus of research in biomedical and health informatics. Although AI methods have advanced throughout decades, the papers highlighted in this editorial remind us of the continued importance of clinical decision-making as a key target to advance healthcare quality and equity. None declared. None declared. Not applicable.

In this month’s editorial, I highlight 5 papers characterized by their focus on types of data that are less frequently addressed in publications in the Journal of the American Medical Informatics Association. This includes social and behavioral determinants of health (SBDH)1 and negative patient descriptors2 in clinical notes, clinical research data,3 and genomics data.4,5 Gu et al1 report on the development and evaluation of SBDH-Reader to extract 6 categories of SBDH from unstructured clinical notes: employment, housing, marital status, and use of alcohol, tobacco, and drugs. The authors developed SBDH-Reader through GPT-4o prompt engineering using the MIMIC-III database (2001-2012, 7225 notes from 6382 patients) and externally validated it using data from The University of Texas Southwestern Medical Center (2022-2023, 971 notes from 437 patients). F1 scores in the validation data set ranged from 0.96 (employment, housing) to 0.99 (tobacco use). These findings demonstrate the power of using a general-purpose large language model without task-specific fine-tuning and support the scalability of the approach.

In this editorial, I highlight 3 papers focused on ethics frameworks for artificial intelligence (AI)1–3 which assess bias from the perspective of AI-based applications in health care as well as 2 papers that address the role of AI-based technologies in patient communication4,5 and within the broader context of their studies, consider the potential for AI-based applications to mitigate clinician biases. Chan et al1 conducted a scoping review to examine the real-world impact of AI ethics frameworks across a decade in health care. Eligible studies reported primary research on the qualitative or quantitative impacts of AI ethics frameworks implemented in health care. The search and screening strategies resulted in 16 of 1807 records that met inclusion criteria, comprising 5 preliminary initiatives testing guidelines in practice, 5 case studies, 5 implementation studies, and 1 comparative case study. Narrative synthesis revealed that the frameworks were implemented for 3 key purposes: (1) to develop new AI governance structures and guidelines, (2) as ethical review assessment systems for adopting clinical AI technologies, and (3) as ethical “audit” tools for identifying ethical risks including bias. Real-world impact was predominantly reported through qualitative improvements in process measures (eg, improved trust in AI). Notably, no studies in the review demonstrated a direct link between ethics frameworks and health-related outcome measures such as patient safety. The authors conclude that implementation and impact evaluation of ethics frameworks for AI in health care remain limited. They suggest that stronger evidence is needed to link ethics frameworks with measurable health outcomes. In a Case Study, Koski et al2 report on the processes and recommendations of a multi-stakeholder expert panel (eg, healthcare professionals, AI developers, policymakers) that was tasked with identifying critical issues and formulating consensus recommendations related to the responsible use of real-world data (RWD) in healthcare AI. The panel’s deliberations revealed several critical challenges including: (1) need for data literacy and documentation, (2) identification and mitigation of bias, (3) privacy and ethical considerations, and (4) absence of an accountability structure for stakeholder management. To address these challenges, the panel proposed 15 recommendations in 5 keys areas: (1) adoption of metadata standards for RWD sources; (2) development of transparency frameworks and instructional labels likened to 'nutrition labels’ for AI applications; (3) provision of crossdisciplinary training materials; (4) implementation of bias detection and mitigation strategies; and (5) establishment of ongoing monitoring and update processes. The authors emphasize the importance of guidelines and resources focused on the responsible use of RWD in healthcare AI for developing safe, effective, equitable, and trustworthy applications, as well as the criticality of multi-stakeholder engagement. In a Perspective, Tsoi et al3 share their experiences in developing a responsible AI framework to assess AI solutions at a healthcare organization. The framework includes an intake survey with descriptive information and 21 items aligned with institutional goals to promote fairness (4 items), transparency (7 items), accountability (5 items), and trustworthiness (5 items). At least 5 clinical, analytical, and operational experts rate each of the 21 items on a 5-point Likert scale. The ratings and subsequent consensus recommendation are reviewed by the AI and Automation Advisory group and organizational leadership for final decision making. Of 12 AI solutions evaluated—most in radiology, 10 were conditionally approved. Challenges identified with the process included limited availability of information from vendors to assess fairness and transparency and reliance on volunteer engagement for evaluation of responses. To address scalability and resource constraints, the authors suggest that future iterations of the framework should consider tiered evaluation based on risk likelihood of the AI-based solution. Dellavalle et al4 conducted a mixed-methods study with patient-users of a healthcare system multi-task chatbot integrated with an electronic health record. They surveyed chatbot users, with 20% of 3089 users responding. They also conducted semi-structured interviews with 46 patients who used the chatbot and 2 chatbot developers. The authors analyzed survey data with descriptive statistics and Chi-square tests, interview data with a modified grounded theory approach, and integrated the data with mixed methods approaches. Of relevance to mitigating potential clinician bias, some patients preferred to discuss sensitive topics (such as mental health or gender-affirming care) with chatbots due to less worry about clinician judgment and perceived ability to avoid unpleasant interactions. Patients also preferred chatbots for administrative tasks due to convenience but preferred clinicians for diagnostic tasks. Langevin et al5 conducted remote interviews with 16 primary care practitioners in remote interviews to brainstorm future technologies for improving clinician awareness of implicit bias in patient-provider communication. Participants also completed an online survey in which they rated the priority of educational strategies that could complement the technology. The interview comprised two components. The provocative design component included a short film-based scenario depicting positive and negative provider communication in primary care visits, after which the participants were asked about what they noticed in the scenes and how future technology that monitors clinical communication might help to improve patient-provider interactions. In the second component, participants were asked to reflect on intervention strategies for future communication feedback technology to address implicit bias. The authors performed inductive-deductive thematic analysis of the interview data with implicit bias recognition and management domains as a priori codes. Participants proposed how future technology could improve clinician awareness of implicit bias, but some providers expressed concerns regarding feedback fatigue and the potential impact of technology on reducing time spent with patients. It was also noted that such provider-specific technology-based interventions must occur within an overall organizational context, including education and monitoring. The three papers focused on AI ethics frameworks reflect increased awareness and activity but highlight the need for continued evaluation, and Chan et al. specifically call for action related to linking such frameworks to measurable health outcomes. The latter two studies remind us that humans as well as algorithms exhibit biases. Indeed, several recent papers6,7 in the Journal of the American Medical Informatics Association identified the presence of stigmatizing and privileging language reflective of clinician bias in clinical notes. Consideration of sources of bias and mitigating strategies is critical to advancing the role of AI-based applications in health care. None to declare. None to declare. Not applicable.

Writing this editorial between the 2025 National Academy of Medicine annual meeting, which focused on “Frontiers of AI & Health: Care, Discovery, and Education” and featured multiple American Medical Informatics Association (AMIA) members as speakers, and our annual AMIA Symposium, I am struck by how some aspects of our field have become increasingly relevant to the broader biomedical research and health enterprise but also how the rapidity of AI development in industry will increasingly influence the ecosystem in which we work. In this December 2025 editorial, I highlight articles that reflect key recurring themes (eg, user-centered approaches,1 interoperability frameworks2) from our field of biomedical and health informatics and evolving methods that address privacy preservation,3 distributed learning,4 and causal pathways.5 Saleh and Johnson report on the development and validation of a framework for assessing the appropriateness of automating clinical orders in electronic health records (EHRs).1 Their multi-phase study comprised: (1) conduct and analysis of focus groups to identify key themes with subsequent integration with existing literature to identify desiderata; (2) validation survey in which clinicians were asked to review 10 use cases and rate their perceptions of appropriateness, cognitive support, and patient safety; and (3) analysis of 1 year of order-based alerts and orders to identify candidate orders for automation and their impact. The first phase resulted in 8 desiderata for automated order appropriateness: logical consistency, data provenance, order transparency, context permanence, monitoring plans, trigger consistency, care team empowerment, and system accountability. In the validation phase, clinicians rated use cases meeting the desiderata significantly higher on appropriateness, cognitive support, and patient safety than use cases that did not meet the desiderata. The authors conclude that the desiderata are useful to systematically identify appropriate orders for automation and to govern them although further research is needed to validate operational scalability. Engelke et al2 introduce FHIR-Former, an open-source framework integrating Fast Healthcare Interoperability Resources (FHIR) with large language models (LLMs) to automate and standardize clinical prediction and classification tasks. Fast Healthcare Interoperability Resources-Former operates through a 3-stage pipeline: (1) task selection and preprocessing of Health Level Seven (HL7) FHIR-based structured and unstructured patient data; (2) fine-tuning large language models using established machine learning frameworks to enhance model performance; and (3) generation of clinical risk scores that are automatically integrated back into FHIR RiskAssessment resources. The authors demonstrated FHIR-Former’s capability using four prediction tasks: 30-day readmission risk, in-hospital mortality, imaging studies performed within the first 24 hours of hospital admission and ICD-10-GM codes after the first day of admission until the end of hospitalization. The respective F1 scores ranged from .48 for ICD-10-GM codes to .71 for 30-day readmission risk. The findings support the promise of the approach. Importantly, FHIR-Former eliminates institution-specific preprocessing by adapting to diverse FHIR implementations. Further assessment is needed in other settings and for other prediction tasks. Data synthesis using generative AI methods can enable the sharing of high-quality data while preserving the privacy of patients. Pilgrim, Kababji, Liu, and El-Emam address the question of whether it is useful to synthesize an entire dataset when only a task-relevant subset is needed by evaluating the influence of the number of variables in the training dataset on fidelity, utility, and privacy of the synthetic data.3 They used 12 cross-sectional medical datasets to define downstream tasks with corresponding core variables and derived 6354 variants by adding adjunct variables to the core. They generated synthetic data using 7 different generative models and evaluated for fidelity, downstream utility, and privacy. They applied mixed-effects models to assess the effect of adjunct variables on the respective evaluation metric, accounting for the medical dataset as a random component. Fidelity and utility remained stable across the great majority of synthetic data models and privacy was not influenced by the number of adjunct variables, thus suggesting that fidelity, utility, and privacy are preserved when generating a more comprehensive medical dataset rather than only the task-relevant subset. Given the evidence for racial disparities in kidney transplant outcomes, Wang et al4 developed a novel decentralized multisite approach to quantitatively assess the effect of site of care on racial disparities between non-Hispanic Black (NHB) and non-Hispanic White (NHW) patients in posttransplantation survival times. The “Communication-Efficient Distributed Analysis for Racial Disparity in Time-to-event Data” (CEDAR-t2e) algorithm comprises 2 models: (1) estimation of the site-specific proportional hazards model for time-to-event outcomes in a distributed manner; and (2) calculation of how long the kidney failure time of NHB patients would be extended if they had been admitted to transplant centers in the same distribution as NHW patients were admitted. When the authors applied the algorithm to United States Renal Data System data covering 39 043 patients across 73 transplant centers, they found no evidence suggesting the presence of site-of-care-associated racial disparities in posttransplantation survival times. The quantitative approach for evaluating site-of-care associated racial disparities is potentially generalizable to other time-to-event outcomes. Based on an argument that externalizing behaviors (eg, aggression, hyperactivity, defiance) in children are influenced by complex interplays between genetic predispositions and environmental factors, particularly parental behaviors. Wei and Peng5 developed “Hillclimb-Causal Inference,” a causal discovery approach that integrates the Hill Climb search algorithm with a customized Linear Gaussian Bayesian Information Criterion. They applied the method to data from the Adolescent Brain Cognitive Development Study including parental behavior assessments, children’s genotypes (ie, polygenic risk scores), and externalizing behavior measures. After they identified the causal pathways, the authors employed structural equation modeling to quantify the relationships within the model. Their findings across direct and indirect paths provide evidence that parental substance use (ie, alcohol, drugs, and tobacco) exerts larger effects than genetic risk on children’s externalizing behaviors; thus, suggesting potential targets for prevention and intervention. Moreover, the Hillclimb-Causal Inference framework provides a general approach to map causal pathways in developmental psychiatry and related domains. Highlighted articles and others in this issue continue to address key conceptual and methodological issues as we strive to influence health and health equity for all. None declared. None declared. Not applicable.

Since its first volume in 1994, the Journal of the American Medical Informatics Association has published >1000 papers tagged as artificial intelligence (AI) in PubMed. Most of these papers do not include AI in their title but include methods and technologies long considered components of the broader term. The 9 AI papers in JAMIA’s first volume included the topics of natural language processing,1–3 machine learning,4 expert systems,4,5 computerized decision support,6 diagnostic reasoning,7 language schemas,3 concept modeling,8 and neural networks.9 In recent years, as computational power and volume, and types of data have increased, AI has become a ubiquitous term in our daily lives. In this editorial, we highlight 5 papers that include AI in their title and explicitly address a hot topic related to the application of AI in health. Two Brief Communications evaluate implementation of ambient scribe technology in an academic medical center.10,11 A Research and Applications paper summarizes use and impact health system-wide access to generative AI.12 Two Perspectives address important topics for scalability of AI in practice: using human factors methods to mitigate bias resulting from user interfaces, and regulation of AI.13,14

BACKGROUND: Mild cognitive impairment and early-stage dementia significantly impact healthcare utilization and costs, yet more than half of affected patients remain underdiagnosed. This study leverages audio-recorded patient-nurse verbal communication in home healthcare settings to develop an artificial intelligence-based screening tool for early detection of cognitive decline. OBJECTIVE: To develop a speech processing algorithm using routine patient-nurse verbal communication and evaluate its performance when combined with electronic health record (EHR) data in detecting early signs of cognitive decline. METHOD: We analyzed 125 audio-recorded patient-nurse verbal communication for 47 patients from a major home healthcare agency in New York City. Out of 47 patients, 19 experienced symptoms associated with the onset of cognitive decline. A natural language processing algorithm was developed to extract domain-specific linguistic and interaction features from these recordings. The algorithm's performance was compared against EHR-based screening methods. Both standalone and combined data approaches were assessed using F1-score and area under the curve (AUC) metrics. RESULTS: The initial model using only patient-nurse verbal communication achieved an F1-score of 85 and an AUC of 86.47. The model based on EHR data achieved an F1-score of 75.56 and an AUC of 79. Combining patient-nurse verbal communication with EHR data yielded the highest performance, with an F1-score of 88.89 and an AUC of 90.23. Key linguistic indicators of cognitive decline included reduced linguistic diversity, grammatical challenges, repetition, and altered speech patterns. Incorporating audio data significantly enhanced the risk prediction models for hospitalization and emergency department visits. DISCUSSION: Routine verbal communication between patients and nurses contains critical linguistic and interactional indicators for identifying cognitive impairment. Integrating audio-recorded patient-nurse communication with EHR data provides a more comprehensive and accurate method for early detection of cognitive decline, potentially improving patient outcomes through timely interventions. This combined approach could revolutionize cognitive impairment screening in home healthcare settings.

In this editorial, I focus on studies that address organizational and individual expertise and experience in engaging with digital health innovations. This is consistent with the epistemic injustice component of the additional goal for JAMIA that was described in the first editorial of 2024: To exemplify best practices in publishing to advance health equity and justice through promoting (1) diversity, equity, and inclusion in editorial team and processes; (2) use of inclusive and nonstigmatizing language; and (3) innovative, rigorous, and transparent quantitative and qualitative research methods that address fairness and mitigate epistemic injustice.1 Epistemic injustice is the failure to treat individuals as knowers.2 The complex digital health ecosystem demands consideration of the knowledge of all stakeholders to advance the goals of quality and equity. The papers in this issue consider the knowledge of digital health companies, community-based organizations (CBOs), staff members in Federally Qualified Health Centers (FHQCs), and healthcare consumers and patients.

The number of predictive models based on electronic health record (EHR) data is expanding resulting in identification of challenges not only in model development and validation (bias, cross-site and cross-setting differences) but also in implementation in practice including monitoring of performance changes that may occur due to dataset shifts in clinical environments. In this editorial, I highlight a systematic review, three research papers, and a perspective that contribute to advancing the science for addressing these challenges. Chen et al. conducted a systematic review of artificial intelligence (AI) models developed using EHR data to identify key biases, strategies for detecting and mitigating bias throughout model development, and metrics for bias assessment.1 Twenty of 450 retrieved articles met inclusion criteria with most models developed for predictive tasks. No models in the review had been deployed in real-world settings at the time of the review. Twenty-five percent of the studies focused on detection of biases through fairness metrics such as statistical parity, equal opportunity, and predictive equity. The remainder proposed strategies for mitigating biases and predominantly involved data collection and preprocessing techniques such as resampling and reweighting. This review highlights the importance of bias detection and mitigation strategies so that predictive models advance health equity rather than exacerbate existing disparities.

The Journal of the American Medical Informatics Association (JAMIA) has published papers on large language models (LLM) over the last 5 years. For example, a 2019 paper by Si et al. explored utilizing LLM for clinical concept extraction, including comparing Bidirectional Encoder Representations from Transformers (BERT) to traditional word embedding methods (word2vec, GloVe, fastText). Their experiments demonstrated that contextual embeddings encode valuable semantic information not accounted for in traditional word representations.1 The broad release of ChatGPT 3.5 resulted in a flurry of submissions to JAMIA and motivated a forthcoming focus issue on ChatGPT and LLM in Biomedicine and Health with a particular emphasis on methodological innovation as well as associated ethical, legal, and social implications. In this issue, I highlight 5 papers that explore different areas of application of LLMs including generative AI. Xie et al. evaluated an epilepsy-specific LLM (ClinicalBERT that had been fine-tuned on 700 manually annotated epileptologist notes) for intrinsic bias and used LLM-extracted outcomes to determine if demographic groups varied in freedom from seizures at each office visit.2 In a sample of 84 675 clinic visits from 25 612 unique patients, they found little evidence of bias in prediction accuracy and confidence of outcome classifications across demographic groups (race, ethnicity, sex, income, and health insurance). Females, those with public insurance, and those living in lower-income zip code areas had significantly worse outcomes, that is, freedom from seizures. This study contributes to the body of evidence regarding application of LLMs to examine health disparities.

In this editorial, I highlight five papers that focus on methodological and logistical aspects of using electronic health record (EHR) data for research. Using primary care EHR data from >6 million people with Multiple Long-Term Conditions in England, Beaney et al compared the performance of unsupervised representations of sequences of disease codes generated by bag-of-words versus sequence-based natural language processing (NLP) algorithms at predicting clinically relevant outcomes.1 They generated an unsupervised vector representation of patient time-ordered sequences of diseases using 2 input strategies (disease categories versus diagnostic codes) and different NLP algorithms (Latent Dirichlet Allocation, doc2vec, 2 transformer models designed for EHRs), and a transformer architecture, EHR-BERT, that incorporated sociodemographic information. They compared the performance of each of these representations (without fine-tuning) as inputs into a logistic classifier to predict 1-year mortality, healthcare use, and new disease diagnosis. Patient representations generated using disease categories performed similarly to those using diagnostic codes as inputs. Additionally, patient representations generated by sequence-based algorithms performed consistently better than bag-of-words methods in predicting clinical endpoints, with the highest performance for EHR-BERT across all tasks. Their findings suggest that models can equally manage smaller or larger vocabularies for prediction of the studied outcomes and that transformer models may be useful for generating multi-purpose representations, even without fine-tuning.

Much of the current buzz in our field is around large language models and generative artificial intelligence (AI), and these topics are reflected in all issues of the Journal of the American Medical Informatics Association (JAMIA) this year including the forthcoming September 2024 focus issue on ChatGPT and Large Language Models. However, in this editorial, I highlight 5 publications that focus on the foundational elements of standards and frameworks that have been regularly featured throughout JAMIA’s 30-year history. Motivated by a long-standing collaborative arrangement between SNOMED International and the World Health Organization (WHO) to support interoperability between the International Classification of Diseases (ICD) and SNOMED CT,1 Fung and his co-authors conducted a pilot project that bidirectionally mapped endocrine concepts between SNOMED CT and the ICD-11 Foundation.2 In phase 1 mapping from ICD-11 to SNOMED CT, they found that 59% of 637 ICD-11 Foundation entities had an exact match in SNOMED CT. In phase 2 mapping from SNOMED CT to ICD-11, only 32% of 1893 SNOMED CT concepts had an exact match in the ICD-11 Foundation with post-coordination of concepts resulting in an additional 15% of exact matches. The authors encountered challenges such as non-synonymous synonyms, mismatches in granularity, composite conditions, and residual categories. Based on the mapping pilot, the authors make 3 recommendations related to mapping between the coding systems: (1) clarify goals and use cases; (2) provide adequate resources; and (3) set up a road map. Most importantly, they recommend that if “the ultimate goal is to maximize interoperability between SNOMED CT and ICD-11, the direct use of SNOMED CT as an ontology to build the ICD-11 Foundation is a better solution than a map.”

The 2023 Annual Report from the Johns Hopkins Center for Gun Violence Solutions reports an analysis of the 2021 firearm fatality data released from the Centers for Disease Control and Prevention in January of 2023.1 The report highlighted alarming facts: In the area of school shootings, the Washington Post reports that 352 000 children have experienced gun violence at school since Columbine in 1999 (https://www.washingtonpost.com/education/interactive/school-shootings-database/). Moreover, the median age of a school shooter is 16 years, and children were responsible for more than half the US school shootings. The Post also found that when the source of the gun could be determined in school shootings by children, 86% of the weapons were found in the homes of friends, relatives, or parents. A 2022 study reports that 4.6 million children live in homes with at least 1 gun that is loaded and unlocked. Thus, increasing the risk of gun violence among children and youth.2

Eta S. Berner, EdD, FACMI, FIAHSI, died on December 11, 2023, only a month after interacting with many colleagues at the American Medical Informatics Association (AMIA) Annual Symposium in New Orleans. As co-workers, collaborators, and friends we each have personal memories that we hold near and dear. With this editorial, we celebrate her professional contributions. Dr Berner was an author of 20 papers published in Journal of the American Medical Informatics Association (JAMIA),1–20 the most recent in the February 2024 issue. Her 1994 New England Journal of Medicine paper on evaluation of diagnostic decision support systems set the standard for rigorous evaluation of such systems.21 Several of her early JAMIA papers reflected her expertise in decision support,1,4,5 while others represented Dr Berner’s commitment to competency-based informatics education through her own work11,12 as well as her substantial contributions to efforts within AMIA.14 A third area of important contribution published in JAMIA was in the area of professional ethics for the field.8,13,16,17 In addition to her authorship contributions, Dr Berner served multiple terms on the JAMIA Editorial Board providing thoughtful peer-reviews on hundreds of manuscripts.

In 2015, the White House and the National Institutes of Health announced the inception of the All of Us Research Program “to bring us closer to curing diseases like cancer and diabetes, and to give each of us access to the personalized information we need to keep ourselves and our families healthier.” Now approaching the first decade, the vision to enhance innovation in biomedical research remains strong with the goal of moving the United States into an era where medical treatment and other health interventions can be tailored to individuals. As of October 2024, there are over 842 000 participants who have consented to participate in the All of Us Research Program. The resulting data set, which is accessed through the All of Us Public Data Brower (aggregated data only) or Researcher Workbench reflects three novel aspects: (1) enriched enrollment for racial, ethnic, sexual, gender, and geographic minority populations to correct for past sampling bias in precision medicine studies, (2) inclusion of social determinants of health (SDoH), electronic health record (EHR), and genomic data, and (3) designed for use by scientists with a broad variety of backgrounds and different research (eg, research-intensive universities, community-based organizations) and educational settings (eg, Historically Black Colleges and Universities, high schools).

OBJECTIVE: Developing targeted, culturally competent educational materials is critical for participant understanding of engagement in a large genomic study that uses computational pipelines to produce genome-informed risk assessments. MATERIALS AND METHODS: Guided by the Smerecnik framework that theorizes understanding of multifactorial genetic disease through 3 knowledge types, we developed English and Spanish infographics for individuals enrolled in the Electronic Medical Records and Genomics Network. Infographics were developed to explain concepts in lay language and visualizations. We conducted iterative sessions using a modified "think-aloud" process with 10 participants (6 English, 4 Spanish-speaking) to explore comprehension of and attitudes towards the infographics. RESULTS: We found that all but one participant had "awareness knowledge" of genetic disease risk factors upon viewing the infographics. Many participants had difficulty with "how-to" knowledge of applying genetic risk factors to specific monogenic and polygenic risks. Participant attitudes towards the iteratively-refined infographics indicated that design saturation was reached. DISCUSSION: There were several elements that contributed to the participants' comprehension (or misunderstanding) of the infographics. Visualization and iconography techniques best resonated with those who could draw on prior experiences or knowledge and were absent in those without. Limited graphicacy interfered with the understanding of absolute and relative risks when presented in graph format. Notably, narrative and storytelling theory that informed the creation of a vignette infographic was most accessible to all participants. CONCLUSION: Engagement with the intended audience who can identify strengths and points for improvement of the intervention is necessary to the development of effective infographics.

OBJECTIVES: Integration of social determinants of health into health outcomes research will allow researchers to study health inequities. The All of Us Research Program has the potential to be a rich source of social determinants of health data. However, user-friendly recommendations for scoring and interpreting the All of Us Social Determinants of Health Survey are needed to return value to communities through advancing researcher competencies in use of the All of Us Research Hub Researcher Workbench. We created a user guide aimed at providing researchers with an overview of the Social Determinants of Health Survey, recommendations for scoring and interpreting participant responses, and readily executable R and Python functions. TARGET AUDIENCE: This user guide targets registered users of the All of Us Research Hub Researcher Workbench, a cloud-based platform that supports analysis of All of Us data, who are currently conducting or planning to conduct analyses using the Social Determinants of Health Survey. SCOPE: We introduce 14 constructs evaluated as part of the Social Determinants of Health Survey and summarize construct operationalization. We offer 30 literature-informed recommendations for scoring participant responses and interpreting scores, with multiple options available for 8 of the constructs. Then, we walk through example R and Python functions for relabeling responses and scoring constructs that can be directly implemented in Jupyter Notebook or RStudio within the Researcher Workbench. Full source code is available in supplemental files and GitHub. Finally, we discuss psychometric considerations related to the Social Determinants of Health Survey for researchers.

OBJECTIVE: To pilot test an infographic-based health communication intervention that our team rigorously designed and explore whether its implementation leads to better health outcomes among Latino persons with HIV (PWH). MATERIALS AND METHODS: Latino PWH (N = 30) living in New York City received the intervention during health education sessions at 3 study visits that occurred approximately 3 months apart. At each visit, participants completed baseline or follow-up assessments and laboratory data were extracted from patient charts. We assessed 6 outcomes (HIV-related knowledge, self-efficacy to manage HIV, adherence to antiretroviral therapy, CD4 count, viral load, and current and overall health status) selected according to a conceptual model that describes pathways through which communication influences health outcomes. We assessed changes in outcomes over time using quantile and generalized linear regression models controlling for the coronavirus disease 2019 (COVID-19) research pause and new patient status (new/established) at the time of enrollment. RESULTS: Most participants were male (60%) and Spanish-speaking (60%); 40% of participants identified as Mixed Race/Mestizo, 13.3% as Black, 13.3% as White, and 33.3% as "other" race. Outcome measures generally improved after the second intervention exposure. Following the third intervention exposure (after the COVID-19 research pause), only the improvements in HIV-related knowledge and current health status were statistically significant. DISCUSSION AND CONCLUSION: Our infographic-based health communication intervention may lead to better health outcomes among Latino PWH, but larger trials are needed to establish efficacy. From this work, we contribute suggestions for effective infographic use for patient-provider communication to enhance patient education in clinical settings.

OBJECTIVES: Advances in informatics research come from academic, nonprofit, and for-profit industry organizations, and from academic-industry partnerships. While scientific studies of commercial products may offer critical lessons for the field, manuscripts authored by industry scientists are sometimes categorically rejected. We review historical context, community perceptions, and guidelines on informatics authorship. PROCESS: We convened an expert panel at the American Medical Informatics Association 2022 Annual Symposium to explore the role of industry in informatics research and authorship with community input. The panel summarized session themes and prepared recommendations. CONCLUSIONS: Authorship for informatics research, regardless of affiliation, should be determined by International Committee of Medical Journal Editors uniform requirements for authorship. All authors meeting criteria should be included, and categorical rejection based on author affiliation is unethical. Informatics research should be evaluated based on its scientific rigor; all sources of bias and conflicts of interest should be addressed through disclosure and, when possible, methodological mitigation.

OBJECTIVE: We aimed to evaluate the feasibility of using ChatGPT as programming support for nursing PhD students conducting analyses using the All of Us Researcher Workbench. MATERIALS AND METHODS: 9 students in a PhD-level nursing course were prospectively randomized into 2 groups who used ChatGPT for programming support on alternating assignments in the workbench. Students reported completion time, confidence, and qualitative reflections on barriers, resources used, and the learning process. RESULTS: The median completion time was shorter for novices and certain assignments using ChatGPT. In qualitative reflections, students reported ChatGPT helped generate and troubleshoot code and facilitated learning but was occasionally inaccurate. DISCUSSION: ChatGPT provided cognitive scaffolding that enabled students to move toward complex programming tasks using the All of Us Researcher Workbench but should be used in combination with other resources. CONCLUSION: Our findings support the feasibility of using ChatGPT to help PhD nursing students use the All of Us Researcher Workbench to pursue novel research directions.

Phenotyping is an increasingly important area for the application of informatics processes and methods. From a biological perspective, phenotype generally refers to observable traits which are influenced by genotype and environment factors. In the context of electronic health records (EHRs), a “computable phenotype,” or simply “phenotype,” is a clinical condition or characteristic that can be ascertained by means of a computerized query to an EHR system or clinical data repository using a defined set of data elements and logical expressions.1 Somewhat broader than that definition, Richesson et al2, on behalf of the NIH Health Care Systems Collaboratory, defined EHR-based phenotyping as activities and applications that use EHR data exclusively to describe clinical characteristics, events, and service patterns for specific patient populations. Spinazze et al3 further specified digital phenotyping as “the process of inferring individual behavior from digital data generated through human interaction with electronic devices, including both physical hardware and software.” In this Editorial, I highlight 5 papers related to phenotyping. A methodical review summarizes multiple challenges related to EHR phenotyping4 while four articles describe the application of informatics innovations to improve electronic phenotyping.5–8

In this editorial, I highlight 5 papers that address innovative informatics interventions—3 research studies and 2 reviews. The papers reflect a variety of information technologies and processes including mobile health (mHealth),1 behavioral nudges in the electronic health record (EHR),2 adaptive intervention framework,3 predictive models,4 and artificial intelligence (eg, machine learning, data mining, natural language processing).5 The interventions were designed to address important clinical and public health problems such as adherence to antiretroviral therapy for persons living with HIV (PLWH),1 opioid use disorder,3 and pain assessment and management,5 as well as aspects of healthcare quality including no-show rates for appointments4 and erroneous decisions, waste, and misuse of resources due to EHR choice architecture for clinician orders.3 Schnall et al1 completed a randomized controlled trial (RCT) to assess the efficacy of an mHealth self-management interventions for PLWH on antiretroviral adherence and viral suppression. The novel WiseApp intervention comprised testimonials of lived experiences, push-notification reminders, medication trackers, health surveys, chat rooms, and a daily “To-Do” list. In a sample of PLWH with suboptimal adherence, daily antiretroviral adherence was significantly higher in the WiseApp group as compared to attention control group from day 1 to day 59 but not for days 60–120. There were no significant differences in viral suppression. The authors suggest that there is a need for booster or a combination intervention approaches after the initial effects of the mHealth intervention wanes.

Throughout the decades of JAMIA’s existence, the relationship between automation and clinical workflow has remained a hot topic.1–4 A 2021 JAMIA issue focused on the relationship between health information technology and clinician burnout.5 In this editorial, I highlight four research and application papers and a brief communication that address aspects of clinician workflow. One paper asks the provocative question of “Are we there yet?”6 and the five highlighted papers suggest that despite positive effects of health information technology on quality in some instances and the advances in the science of workflow research,7,8 much remains to be done to match health information technology and clinician workflow. Three of the five studies emphasize the role of qualitative research in understanding this relationship6,9,10 and another addresses the discrepancy between subjective perceptions and objective measures of clinician efficiency.11 Moy et al9 conducted semi-structured interviews with a national sample (n = 24) of US prescribing providers and registered nurses who practice in the adult emergency department setting and use the Epic electronic health record (EHR) to understand perceptions of the role of EHRs and workflow fragmentation on clinician documentation burden in emergency departments. They analyzed interview transcripts using inductive thematic analysis and finalized six themes through a consensus process. EHR factors perceived to contribute to documentation burden included (1) lack of advanced EHR capabilities; (2) EHR documentation not optimized for clinicians; (3) EHR work volume hinders communication between clinicians internal and external to the EHR; (4) poor user interface design impacts clinician documentation habits; (5) high volume of manual EHR work; and (6) blockages in EHR impede documentation efficiency. Their findings highlight the importance of designing EHRs that align with clinical workflows to alleviate clinician documentation burden.

In my inaugural editorial as the Journal of the American Medical Informatics Association’s (JAMIA) Editor-in-Chief, I urged biomedical and health informaticians to focus on “doing what matters most” rather than on incremental methodological improvements.1 Little did I know in January of 2019, that soon we would be engaged in the COVID-19 pandemic resulting in >150 JAMIA papers on the topic over the last 3 years. Certainly, the COVID-19 pandemic strained our public health system and revealed gaps in infrastructure. Moreover, some health inequities were exacerbated. In this editorial, I highlight the work of the American College of Medical Informatics on public health challenges and opportunities.2 I also summarize innovative research that addresses significant public health problems: homelessness,3 suicide,4 controlled medication addiction and overdose,5 and inequities related to transgender and gender diverse (TGD) persons.6 The 2022 American College of Medical Informatics (ACMI) symposium focused on the national public health information systems (PHIS) infrastructure to support public health goals. The paper in this issue summarizes the strengths, weaknesses, threats, and opportunities (SWOT) identified by attendees regarding the ability of the current PHIS infrastructure to meet public health goals.2 A related paper details the recommendations generated from the Symposium proceedings.7 The qualitative SWOT analysis revealed 57 factors that were organized into 22 themes using the Informatics Stack—a heuristic previously used by Lehmann to teach medical informatics.8 The analysis revealed three overarching opportunities: (1) addressing the needs for sustainable funding, (2) leveraging existing infrastructure and processes for information exchange and system development that meets public health goals, and (3) preparing the public health workforce to benefit from available resources. The authors conclude that “the PHIS is unarguably overdue for a strategically designed, technology-enabled, information infrastructure for delivering day-to-day essential public health services and to respond effectively to public health emergencies.”

Public discourse about artificial intelligence (AI) and generative AI, in particular, is ubiquitous. AI has been a focus of research in biomedical and health informatics since its inception and in publications the Journal of American Medical Informatics Association since its inaugural issue that included a threaded bibliography on medical diagnostic decision support systems by Dr. Randy Miller (a future JAMIA Editor-in-Chief).1 In that same issue and apropos of the title of this editorial, Dr. Ted Shortliffe’s provocative editorial was entitled “Dehumanization of patient care—are computers the problem or the solution?”2 This month I highlight 5 papers focused on AI that provide key lessons about the importance of keeping the human in the loop. Lyell et al3 examined real-world safety problems involving machine learning (ML)-enabled medical devices by analyzing safety events reported to the US Food and Drug Administration’s Manufacturer and Use Facility Device Experience program. Using an existing framework for safety problems with health information technology, they identified whether a reported problem was due to the ML device or its use, and key contributors to the problem. They also classified the consequences of events. The majority of the 266 safety events were associated with ML devices that primarily used image-based data as compared to signal-based data. Ninety-three percent of problems involved the ML device with 82% related to data acquisition and <10% to algorithm errors. Sixteen percent of the events resulted in harm. Use problems (7%) were 4 times more likely than device problems to cause harm. This study highlights the need to approach ML device safety from a whole-system perspective including user interactions with devices rather than focusing only on the algorithm.

I write this editorial from beautiful Sydney, Australia, where Journal of the American Medical Informatics Association (JAMIA) Associate Editor, Farah Magrabi, is serving as Co-Chair of a very exciting MedInfo 2023. I am privileged to represent JAMIA in a panel on trends in health informatics publishing at the conference. I am also delighted to report that the Fellows of the International Academy of Health Sciences Informatics met on the first day of MedInfo and endorsed a statement on climate change that builds upon our efforts in JAMIA to heighten the awareness of climate change and the role of health informatics and data science in measuring, monitoring, and mitigating climate change. This included a focus issue on climate change led by former JAMIA Associate Editor, Enrico Coiera, and Farah Magrabi.1–3 The statement is available at: https://imia-medinfo.org/wp/iahsi/. In this issue, I highlight a set of papers related to the implementation of decision support including clinical decision support (CDS) that uses AI-based approaches. Three papers introduce frameworks for informing clinical implementation of models.4–6 Another focuses on the differences between expected and deployed model performance in a Pediatric Intensive Care Unit.7 The fifth highlighted paper is a scoping review of population-segmentation models.8

JAMIA’s founding Editor, William W. Stead, entitled his inaugural editorial “JAMIA—why?”1 Two of the multiple reasons for establishing JAMIA informed my selection of papers to highlight in this issue. One reason was to provide a forum for identification of career paths, definition of curricula, and consideration of credentialing in medical informatics. Dr. Stead also envisioned JAMIA as a vehicle for reaching beyond the medical informatics community to educate the public about the potential of the field for improving the healthcare environment and the challenges that must be overcome to achieve that potential. Taken together, these reasons for JAMIA and the substantial progress in these areas by AMIA contribute to defining our field’s relationship with society—a concept well-known to us from clinical disciplines as a social contract. JAMIA has published a substantial number of papers, primarily AMIA position papers, on the competencies and core educational content for our field throughout its history ranging from defining core content and competencies for educational programs2–6 to certification as a medical subspecialty7 or for advanced health informatics practice.8–10 Two of the highlighted papers in this editorial are AMIA position papers that build upon this prior work by defining the foundational domains and competencies for baccalaureate health informatics education11 and mapping the delineation of practice to the AMIA foundational domains for applied health informatics at the Master’s level12 while a third characterizes the experiences of clinical informatics fellow graduates during fellowship.13

Since its first issue, the Journal of the American Medical Informatics Association has published hundreds of papers about standards to support interoperability among various types of electronic health data and information systems. In this editorial, I highlight 2 perspectives from 2 long-time thought leaders in biomedical and health informatics.1,2 This is followed by 2 Research and Application papers that focus on terminology standards, the Unified Medical Language System (UMLS) and SNOMED-CT.3,4 The last highlighted paper addresses information blocking practices related to specific classes in the United States Core Data for Interoperability Version 1 (USCDI v1).5 Donald W. Simborg, MD, founding member of Health Level 7 (HL7), reflects on its origins and impact on interoperability in hospitals.1 He particularly notes that although HL7 is the most widely used data-interchange protocol worldwide in health care, the promise of open architecture envisioned by its founders did not prevail in healthcare as it did in other industries due to some unique characteristics of the healthcare environment. Enrico Coiera proposes a framework to support the scientific research of standards so that they can be better measured, evaluated, and designed.2 He argues that fitting work using conformance services is needed to repair these gaps between a standard and what is required for real-world use and identifies 3 strategies: (1) universal conformance (all agents access the same standard); (2) mediated conformance (an interoperability layer supports heterogeneous agents); and (3) localized conformance (autonomous adaptive agents manage their own needs). Coiera’s approach conceptually decouples interoperability and standardization, and he concludes that although “standards facilitate interoperability, interoperability is achievable without standardization.”

I point readers to 2 Editorials in this first issue of 2024—one focused on the Journal of the American Medical Informatics Association (JAMIA) by its 4 Editors-in-Chief1 and the second on treating the climate and nature crisis as an indivisible global health emergency.2 In the following paragraphs, I highlight 5 papers (3 studies and 2 reviews) in this issue that address aspects of user interfaces (UIs) for a variety of different types of electronic systems and users. Staes et al3 applied an interdisciplinary user-centered design approach for an interface to support communication of machine learning-based prognosis for patients with advanced solid tumors by incorporating oncologists’ needs and feedback throughout 5 rounds of iterative design. Oncologists emphasized the importance of interpretability over explainability and indicated that appropriate use of such a tool should be in the context of a clinician-patient conversation. The resulting UI was divided into 7 sections that included visualization to support oncologists to “tell a story” as they discussed prognosis during a clinical encounter.

OBJECTIVE: The aim of this study was to explore the state of health information technology (HIT) usability evaluation in Africa. MATERIALS AND METHODS: We searched three electronic databases: PubMed, Embase, and Association for Computing Machinery. We categorized the stage of evaluations, the type of interactions assessed, and methods applied using Stead's System Development Life Cycle (SDLC) and Bennett and Shackel's usability models. RESULTS: Analysis of 73 of 1002 articles that met inclusion criteria reveals that HIT usability evaluations in Africa have increased in recent years and mainly focused on later SDLC stage (stages 4 and 5) evaluations in sub-Saharan Africa. Forty percent of the articles examined system-user-task-environment (type 4) interactions. Most articles used mixed methods to measure usability. Interviews and surveys were often used at each development stage, while other methods, such as quality-adjusted life year analysis, were only found at stage 5. Sixty percent of articles did not include a theoretical model or framework. DISCUSSION: The use of multistage evaluation and mixed methods approaches to obtain a comprehensive understanding HIT usability is critical to ensure that HIT meets user needs. CONCLUSIONS: Developing and enhancing usable HIT is critical to promoting equitable health service delivery and high-quality care in Africa. Early-stage evaluations (stages 1 and 2) and interactions (types 0 and 1) should receive special attention to ensure HIT usability prior to implementing HIT in the field.

Journal Article Standardized nursing terminologies come of age: advancing quality of care, population health, and health equity across the care continuum Get access Karen A Monsen, PhD, RN, FAMIA, FNAP, FAAN, Karen A Monsen, PhD, RN, FAMIA, FNAP, FAAN School of Nursing, University of Minnesota, Minneapolis, MN, United States Corresponding author: Karen A. Monsen, PhD, RN, FAMIA, FNAP, FAAN, School of Nursing, University of Minnesota, 5-140 Weaver-Densford Hall, 308 Harvard Street SE, Minneapolis, MN 55455 ([email protected]) https://orcid.org/0000-0003-0196-9799 Search for other works by this author on: Oxford Academic PubMed Google Scholar Laura Heermann Langford, PhD, RN, FAMIA, FHL7, Laura Heermann Langford, PhD, RN, FAMIA, FHL7 Logica, Salt Lake City, UT, United States Search for other works by this author on: Oxford Academic PubMed Google Scholar Suzanne Bakken, PhD, MS, BSN, FAAN, FACMI, FIAHSI, Suzanne Bakken, PhD, MS, BSN, FAAN, FACMI, FIAHSI Columbia University, New York, NY, United States https://orcid.org/0000-0001-6202-6001 Search for other works by this author on: Oxford Academic PubMed Google Scholar Karen Dunn Lopez, PhD, MPH, RN, FAAN Karen Dunn Lopez, PhD, MPH, RN, FAAN College of Nursing, University of Iowa, Iowa City, IA, United States Search for other works by this author on: Oxford Academic PubMed Google Scholar Journal of the American Medical Informatics Association, Volume 30, Issue 11, November 2023, Pages 1757–1759, https://doi.org/10.1093/jamia/ocad173 Published: 19 October 2023 Article history Editorial decision: 14 August 2023 Received: 14 August 2023 Published: 19 October 2023

In this editorial, the first 4 Editors-in-Chief of the Journal of the American Medical Informatics Association (JAMIA) reflect on its history and future.The origins and characterization of each Editor's era represent the "lived experience" of each Editor rather than a comparison of common metrics over time.We also qualitatively assess JAMIA's progress in meeting its original vision and goals, and posit considerations for its future.Table 1 summarizes key JAMIA-related events.

In 2020 and 2021, Journal of the American Medical Informatics Association (JAMIA) published many papers related to the COVID-19 pandemic. Despite the COVID-19 pandemic’s prominence as a public health issue, other consequential public problems continue to plague our society and, in some instances, have been exacerbated by the pandemic. In this editorial, I highlight papers related to suicide, opioid use disorder, and child abuse. Given that accurate identification of self-harm presentations to Emergency Departments (EDs) can lead to more timely mental health support, aid in understanding the burden of suicidal intent in a population, and support evaluation of public health initiatives related to suicide prevention, Rozova et al1 developed an automated system for the detection of self-harm presentations from brief ED nursing triage notes. They applied natural language processing to 477 627 free-text triage notes from ED presentations in a single site; 1.4% were labeled as related...

Since its inaugural issue, JAMIA has been a premier venue for publishing about electronic health records (EHRs). In this editorial, I highlight 4 of a dozen papers in this issue that address some aspect of EHRs.1–4 The fifth highlight acknowledges the contributions of Karen Greenwood to AMIA.5 While EHRs serve the key purposes of clinical documentation and billing, one of the promises of EHRs is the ability to share data across sites. Bernstam et al1 quantified the interoperability of real-world EHR implementations for selected structured data (6 medications and 6 laboratory tests) by de-identifying and aggregating data across 68 oncology sites that implemented 1 of 5 EHR vendor products. They calculated inter- and intra-EHR vendor interoperability scores finding that the mean intra-EHR vendor interoperability score was 0.68 as compared to a mean of 0.22 for intersystem interoperability, when weighted by number of systems of each type. These...

In this editorial, I focus on 5 papers that highlight the centrality of the patient. Two papers address the issue of “patient work.” The first reports qualitative research to identify preferences for informatics tools to support patient work related to affordability of medications and tests1 while the second reports on an informatics tool that enabled patients to identify and document diagnostic concerns through a review of their online visit notes.2 The third paper describes the contribution of wearable consumer device data (ie, patient-generated data) and electronic health record (EHR) data in identifying patients with atrial fibrillation who may be eligible for anticoagulation therapy,3 and the fourth provides an overview of the technical landscape for capturing, standardizing, and integrating patient-generated data for decision support for clinicians and patients.4 The fifth paper reminds us that the “patient” is not always an individual and that the patient may play different roles including that of a research participant but must remain at the center of informatics tool development in order to advance health equity.

In this editorial, I highlight 5 papers that address expanded data sources and services to understand, contextualize, promote, and predict individual health with careful consideration of bias. Coiera et al1 promote the idea of family informatics to create a set of digital services to support the family network. Two studies in this issue examine the role of social determinants of health (SDOH) in predictive model performance as a strategy for identifying and mitigating bias.2,3 Lastly, two papers are about data sharing for a variety of purposes. One explores willingness to share as a potential bias in analytic data sets and algorithms4 while the other evaluates a privacy-protecting framework for one type of data.5 As a group, these papers provide additional foundation to advance health equity. In a perspective, Coiera et al1 highlight the importance of understanding individuals in the context of their family and argue that this may require new classes of digital services (ie, family informatics) to address important chronic health challenges such as obesity, mental health, and substance abuse, and to support acute health challenges, and promote self-management capacity. They conceptualize the family network as a multiagent system with distributed cognition. They propose that digital tools can address family needs in four key areas: (1) sensing and monitoring; (2) communicating and sharing; (3) deciding and acting; and (4) treating and preventing illness. Juhn et al2 applied machine learning models for predicting asthma exacerbation in children with asthma. They measured one SDOH, socioeconomic status (SES), using the HOUsing-based SocioEconomic Status measure (HOUSES) index, to assess its influence on predictive model performance. They also compared incompleteness of EHR information relevant to asthma care by SES. Those with lower SES had a higher proportion of missing information relevant to asthma care (eg, asthma severity). The HOUSES index enables assessment of SES bias in predictive model performance. Amrollahi et al3 compared the performance of sepsis readmission prediction models with and without inclusion of SDOH. They used data from the All of Us Research Program participants across 35 hospitals (n = 8935 septic index encounters) to develop a multicenter validated sepsis-related unplanned 30-day readmission models with and without SDOH to predict 30-day unplanned readmissions. Incorporation of SDOH factors (eg, economic stability) into the model of clinical and demographic features improved area under the receiver operating characteristic curve significantly (from 0.75 to 0.80; P < .001). Research participant willingness to share types of data sources can influence the representativeness of samples in analytic datasets. Joseph et al4 examined the willingness of participants in the National Institutes of Health All of Us Research Program to share EHR information. In a sample of 25 852 participants (White—66.5%, Black—18.7%, Hispanic—7.7%, female—32.5%), 2.3% declined to share EHR data. Younger age (1.26 [1.19–1.33]), female sex (1.74 [1.42–2.14]), and education >high school (2.44 [1.86–3.21]), but not race or ethnicity, were significantly associated with decline to share EHR data. Concerns about privacy may limit willingness to share data. Bonomi et al5 propose a privacy-protecting method for sharing one type of data, individual-level electrocardiography (ECG) time-series data. Their approach leverages dimensional reduction technique and random sampling to achieve privacy protection against an informed adversarial model while enabling useful aggregate-level analysis while maintaining the usability for data analytics. Their evaluation of the approach on two real-world ECG data sets demonstrated significant reduction in privacy risks while retaining data usability for tasks such as predictive modeling and clustering. None declared.

In this editorial, I highlight 5 papers with consideration of ethical perspectives. In the first, the authors of an AMIA position paper provide a set of guiding principles related to selection of venues for AMIA conferences and events using an ethical perspective that delineates a set of rights for AMIA members and associated institutional obligations for AMIA.1 Subsequently, I apply a public health ethics framework comprising 7 principles2 to the remaining 4 papers that address critical topics in biomedical and health informatics including bias in models, particularly artificial intelligence (AI) models,3,4 and protection of individual privacy.5,6 The public health ethics principles include well-established bioethical obligations related to non-maleficence (do no harm), beneficence (produce benefit), respect for autonomy (eg, informed consent, privacy), and justice (provide equal opportunity to benefit) as well as 3 additional principles. Population health maximization is the obligation to maximize...

Throughout its existence, the Journal of the American Medical Informatics Association (JAMIA) has served as a premier dissemination venue for research related to semantic interoperability. Hundreds of papers on papers on topics such as ontologies, concept-oriented terminologies, clinical data models, information models, document architectures, and other health information exchange standards have advanced knowledge about semantic interoperability. Half of the papers in JAMIA’s first issue, including my own1 addressed this topic. In this editorial, I highlight 5 papers related to semantic interoperability in clinical care and biomedical and health research. Two research and applications papers and a scoping review focus on one or more aspects related to the Health Level 7 (HL7) Fast Healthcare Interoperability Resources (FHIR) draft standard.2–4 A fourth compares real-world electronic prescriptions (e-prescriptions) to RxNorm, a widely implemented standard for medication ingredients, strengths, and doses.5 The fifth paper proposes a data model standard for research administrative data.6 Toward the goal of facilitating interoperability with existing systems, Brandt et al2 developed and validated an FHIR-based phenotyping tool, the Phenotype Execution Modeling Architecture (PhEMA) Workbench, to author, validate, and share electronic health record (EHR)-based phenotype definitions. They executed a thrombotic event phenotype definition at 3 sites: Mayo Clinic, Northwestern Medicine, and Weill Cornell Medicine, and used manual review to determine precision and recall. The thrombotic event phenotype definition comprised 11 Clinical Query Language statements and 24 value sets containing a total of 834 codes. Precision ranged from 95% to 100% and recall from 84% to 100%. The study findings suggest that the use of a formal representation and a phenotype definition that integrates with existing standards-compliant systems ease automation and have the potential to decrease human error.

In my first year as Editor-in-Chief of the Journal of the American Medical Informatics Association (JAMIA), I published an editorial focused on advancing biomedical and health informatics knowledge through reviews of existing research.1 In the 2019 editorial, I delineated the criteria and best practices for reviews in JAMIA including: (1) address a topic central to biomedical and health informatics and relevant to the JAMIA readership; (2) employ a formal search strategy of multiple databases; (3) depict the flow of information with a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram; (4) document who (minimum of 2) was involved in the flow process, the level of agreement between them, how discrepancies were resolved, and how the process was managed; (5) if a systematic review, apply a method of quality assessment; (6) incorporate a synthesis approach (eg, narrative, tabular, graphical, meta-analysis); (7) analyze what is known, what remains unknown, uncertainty around findings, recommendations for future research, and where relevant recommendations for practice; and (8) register review protocol. In this editorial, I highlight 5 reviews; 4 focus on clinical decision support (CDS)2–5 and the last on shared tasks for natural language processing (NLP) challenges that use electronic health record (EHR) data.6

Meeting the information and communication needs of health disparate populations in a culturally congruent and health literate manner is foundational to advancing health equity, health status, and health-related quality of life. The Journal of the American Medical Informatics Association has published seminal papers on this topic over the last 15 years.1–10 Papers have addressed important factors related to meeting information and communication needs including health literacy, numeracy, and graph literacy1,2,4; described innovative solutions for meeting information and communication needs3–10; and conducted comparative evaluations of visual strategies.6–9 In this editorial, I briefly summarize 5 papers that contribute to the literature on these topics. In 2 papers, the authors report user studies of data visualizations of longitudinal data, that is, systolic blood pressure values11 and patient-reported outcomes (PROs) that include measures of health and graph literacy.12 A review paper focuses on the natural language processing literature for automatic simplification of existing biomedical text for consumers and identifies strategies for addressing challenges to progress in the field.13 As the foundation for public health dashboards that effectively communicate with a variety of audiences including the lay public, Ansari and Martin14 describe formative work for the creation of a heuristic evaluation checklist for public health dashboards. In the fifth paper, Valdez et al15 highlight the disability community as a health disparate population and offer a set of guidelines for effective engagement to create digital health technologies that more fully meet the information and communication needs of all disabled individuals. As formative research to inform the design of a shared decision-making tool, Shaffer et al11 assessed the influence of patient health literacy, numeracy, and graph literacy on perceptions of hypertension control using different forms of data visualization for displaying blood pressure over time. They asked participants (n = 1079 individuals with hypertension who were predominantly white with greater than a high school education) to review and make judgments about hypertension control and need for medication change in 12 vignettes that systematically varied in the mean systolic blood pressure (130 and 145 mmHg), blood pressure standard deviation (15 and 25 mmHg), and form of data visualization (data table, graph with raw values, graph with smoothed values). Participants subsequently completed measures of literacy, graph literacy, and subjective and objective numeracy. The analysis demonstrated a significant main effect of data visualization type on perceptions of hypertension control, need for medication change, perceived comprehension, and alarm. Furthermore, graph literacy, subjective numeracy, and health literacy were all significant predictors of all primary outcomes, while objective numeracy was not. Those with the lowest levels of graph literacy were unable to distinguish between cases of controlled and uncontrolled hypertension with any form of data visualization. These findings are useful for informing the design of information visualizations about blood pressure for shared decision-making but additional studies with a representative sample of those who experience hypertension are needed. Snyder et al12 explicitly addressed the topic of graph literacy through conducting user testing to assess comprehension, utility, and preference of longitudinal PRO visualizations designed for prostate cancer survivors with limited literacy. Building upon their prior work co-designing longitudinal PRO visualizations,3 they engaged 18 prostate cancer survivors (50% African American) to assess 4 prototypes (Meter, Words, Comic, and Emoji) from the perspectives of comprehension (gist and verbatim), utility, and preference. Most participants had less than a college degree (95%), inadequate health literacy (78%), and low graph literacy (89%). Among the 4 prototypes, Meter had the best gist comprehension while Emoji had the highest verbatim comprehension. Words and Comic scored lower than Emoji and Meter for comprehension. For utility, Emoji was rated the highest, Meter and Words were rated mid-range, and Comic the lowest rated. Preference was quite variable with the ranking of first choice being Emoji, Words, Comic, and Meter with 72% ranking Meter as their second choice. This study highlights the importance of rating multiple dimensions in assessing visualization prototypes. Larger studies are needed to assess the statistical significance of the differences among the prototype ratings. Ondov et al13 conducted a review of the literature on application of natural language processing for automatic simplification of existing biomedical text for consumers. As background for their characterization of studies, they discuss the strengths and challenges of procedural, statistical, and neural based approaches to text simplification. Their search of Google Scholar, Semantic Scholar, PubMed, Association for Computational Linguistics (ACL) Anthology, and Database Programming Languages (DBPL) retrieved 46 relevant papers for the review spanning 7 natural languages. Thirty-two papers in the review describe tools or methods, 13 present datasets or resources, and 9 describe influence of text simplification on human comprehension. In terms of methods, the chief focus of 22 papers was procedural (ie, rule-based) while 10 reflected a neural focus. They conclude that the scarcity of parallel data sources, which are required for statistical and neural based approaches, has led to continued development of procedural methods despite the promise of neural approaches, and that high-quality parallel data is a prerequisite for developing fully automated biomedical text simplification. Ansari and Martin14 systematically evaluated the usability of 13 publicly available sexually transmitted infection dashboards on state health department websites in the United States. To identify usability problems, 6 reviewers with usability knowledge who varied in content expertise reviewed the dashboard using a rubric based on 11 principles derived from the information visualization literature: spatial organization, information coding, consistency, removal of extraneous ink, recognition rather than recall, minimal action, dataset reduction, flexibility to user experience, understandability of contents, scientific integrity, and readability. Data analysis included quantitative description of usability scores and qualitative synthesis of textual comments. The reviewers identified the most major usability problems related to the principles of understandability of contents, flexibility, and scientific integrity. The identification of usability problems informed development of a checklist to improve performance related to the 11 heuristic principles. Testing with the varied target audiences including the lay public for such dashboards is needed to further examine their usability. In a Perspective, Valdez et al15 urge the informatics community to expand our impact by focusing on the disability community as a health disparity population. They highlight the need to approach disability “from a more holistic framework, simultaneously accounting for multiple forms of disability and the ways disability intersects with other forms of identity.” Toward this goal, they offer a set of guidelines for effective engagement as a foundation for creating digital health technologies which more fully meet the needs of all disabled individuals. Their recommendations for creating accessible digital technologies fall into 5 broad categories: (1) understanding the broader disability context (eg, respect historical context and reasons for mistrust); (2) partnering with the disability community (eg, develop strong relationships with community organizations that comprise and serve the disability community); (3) intentionally engaging a diverse sample (eg, purposefully sample across individuals with a wide range of diagnoses, functional abilities, and disability identities); (4) creating responsive and accommodating study procedures (eg, prepare alternative approaches to collecting data such as alternative phrasing of questions to be asked, alternative technology platforms to be used during usability evaluation); and (5) building foundation for future work (eg, develop inclusive informatics programs which recruit and train disabled individuals and those with intersecting, marginalized identities). To address many of the challenges raised in the papers highlighted in this issue, JAMIA has issued a call for papers for a Focus Issue on Visualization of Health Data for Lay Audiences with Guest Associate Editors Adriana Arcia, Natalie Benda, Amanda Makulec, and Danny Wu (Chair, AMIA Visual Analytics Working Group). Details are provided at: https://academic.oup.com/jamia/pages/call-for-papers. I urge you to consider how your theoretical, methodological, and empirical work fits this topic and look forward to receiving your submissions. None declared.

In this editorial, we highlight 5 manuscripts that address aspects of clinician interaction with the electronic health record (EHR). A Perspective describes the efforts of the Office of the National Coordinator for Health Information Technology (ONC) in establishing priorities for workflow automation in healthcare settings.1 A second Perspective summarizes a panel sponsored by the American College of Medical Informatics (ACMI) at the 2021 AMIA Symposium that examined a provocative question: Are EHRs dumbing down clinicians?2 Three manuscripts focus on what can be learned from EHR audit log data.3–5 First, Zayas-Cabán et al1 describe how the ONC led a multidisciplinary effort of stakeholders from industrial engineering, computer science, and finance to investigate automation in health care. They define automation as “the creation and application of technology to monitor and control the delivery of products and services.”6 The process of key informant interviews, focus groups, and a review of pertinent literature identified 6 priorities and their related strategies for advancing workflow automation. These priorities focused on leveraging high quality, interoperable data, and engaging clinicians in the EHR design, implementation, and evaluation processes. Then, relevant, and effective workflows could be identified that add value, not a burden, in clinician processes. The supporting strategies revolved around education, convening multiple interested parties, prioritizing appropriate workflows, and using policies and the market to incentivize automated solutions' development, testing, and evaluation. Examples of priority workflows for automation include reimbursement and prior authorizations, medication reconciliation, care management for newly diagnosed patients, and public health and adverse event reporting. The goal is to upgrade EHR technology to increase efficiency, improve health outcomes, and deliver more value for patients, caregivers, clinicians, and staff involved in healthcare. These priorities and strategies provide some solutions to the inadequacies of EHRs as outlined by Melton et al.2

OBJECTIVE: The purpose of this study is to describe online health information seeking among a sample of transgender and gender diverse (TGD) people compared with cisgender sexual minority people to explore associations with human papillomavirus (HPV) vaccination, and whether general health literacy and eHealth literacy moderate this relationship. MATERIALS AND METHODS: We performed a cross-sectional online survey of TGD and cisgender sexual minority participants from The PRIDE Study, a longitudinal, U.S.-based, national health study of sexual and gender minority people. We employed multivariable logistic regression to model the association of online health information seeking and HPV vaccination. RESULTS: The online survey yielded 3258 responses. Compared with cisgender sexual minority participants, TGD had increased odds of reporting HPV vaccination (aOR, 1.5; 95% CI, 1.1-2.2) but decreased odds when they had looked for information about vaccines online (aOR, 0.7; 95% CI, 0.5-0.9). TGD participants had over twice the odds of reporting HPV vaccination if they visited a social networking site like Facebook (aOR, 2.4; 95% CI, 1.1-5.6). No moderating effects from general or eHealth literacy were observed. DISCUSSION: Decreased reporting of HPV vaccination among TGD people after searching for vaccine information online suggests vaccine hesitancy, which may potentially be related to the quality of online content. Increased reporting of vaccination after using social media may be related to peer validation. CONCLUSIONS: Future studies should investigate potential deterrents to HPV vaccination in online health information to enhance its effectiveness and further explore which aspects of social media might increase vaccine uptake among TGD people.

More than a decade ago, Embi and Payne1 proposed a definition of clinical research informatics as “the subdomain of biomedical informatics concerned with the development, application, and evaluation of theories, methods, and systems to optimize the design and conduct of clinical research and the analysis, interpretation, and dissemination of the information generated.” Three years later, a Journal of the American Medical Informatics Association (JAMIA) supplement on clinical research informatics was published. This included an article by Kahn and Weng2 that presented a conceptual model that was used to highlight 18 articles in the issue. In 2018, JAMIA Editor-in-Chief Lucila Ohno-Machado highlighted multiple clinical research informatics articles and predicted the growth of clinical research informatics as a subspecialization in biomedical informatics.3 Five articles in this first issue of 2021,4–8 and many more in the 2019 and 2020 volumes of JAMIA, support the accuracy of Ohno-Machado’s prediction. In a scoping review, Rogers et al4 analyzed 89 articles that examined the contemporary use of real-world data (ie, routinely collected healthcare data) for clinical trial conduct in the United States. Real-world data sources included electronic health records (EHRs) (n = 59), administrative claims (n = 29), and registries (n = 26). Fifty-seven percent of the articles focused on clinical trial process tasks (eg, planning, screening/recruitment, follow-up assessment), while 38% were generalizability assessments. All articles reported data-related challenges including missingness. Notably, <10% of trials using real-world data for trial process tasks evaluated medications or procedures. This review identified both challenges and opportunities for clinical research informatics.

As this March issue of the Journal of the American Medical Informatics Association (JAMIA) is published, we have experienced a year of sheltering-in-place, wearing masks, frequent handwashing, and COVID-19 testing. For some of us, this year also included COVID-19 infection and loss of family, friends, and colleagues—and more recently, COVID-19 vaccination. I accepted JAMIA’s first COVID-related paper on March 19, 2020 less than 24 hours after its submission,1 and the accepted version was available online within days. To date, JAMIA has received almost 400 COVID-related submissions and published 69 contributions in issues since June 2020. The March issue includes 7 papers and correspondence related to COVID-19 including a call from World Health Organization authors to strengthen data in response to COVID-19 and beyond,2 a report on virtual care expansion in the Veterans Health Administration,3 and correspondence about telemedicine, privacy, and information security in the age of COVID-19.4 In this editorial, I highlight 5 papers that reflect the breadth of biomedical and health informatics approaches to addressing the COVID-19 pandemic. Haendel and colleagues provide an overview of the rationale, design, infrastructure, and deployment of the National COVID Cohort Collaborative (N3C).5 N3C (covid.cd2h.org), an open science community focused on analyzing individual-level data from many centers, was developed by the Clinical and Translational Award Program, the National Center for Translational Sciences, and the scientific community to enable rapid collaboration among clinicians, researchers, and data scientists to identify treatments and specialized care and subsequently reduce the immediate and long-term consequences of COVID-19. To overcome technical, regulatory, policy, and governance barriers to sharing and harmonizing individual-level clinical data, N3C developed (a) legal agreements and governance for organizations and researchers; (b) data extraction scripts to identify and ingest positive, negative, and possible COVID-19 cases; (c) a data quality assurance and harmonization pipeline to create a single harmonized dataset; (d) a secure data enclave with data, machine learning, and statistical analytics tools; (e) dissemination mechanisms; and (f) a synthetic data pilot to democratize data access. There are 3 datasets for analysis: synthetic, deidentified, and limited. The Attribution and Publication Policy encompasses all N3C contributions as reflected in the author contribution statement for this paper. Analyses posted within the N3C enclave leverage the contributor attribution model to track the transitive credit of all upstream contributors. The N3C infrastructure is designed to be scalable and extensible to other topics. Sun and colleagues designed COVID-19 Trial Finder, an open-source semantic search engine, to facilitate patient-centered search of COVID-19 trials.6 It is powered by a machine-readable dataset for all COVID-19 trials in the United States. COVID-19 Trial Finder also includes a web-based visualization of the geographic distribution of COVID-19 trials. The initial search is by location and radius distance from trial sites; this is refined through a set of dynamically generated medical questions to assess patient eligibility for nearby COVID-19 trials. They assessed the precision of COVID-19 Trial Finder for COVID-19 using 20 case reports from LitCOVID. Overall precision across the 20 cases was 79.76%, although it varied widely across cases. The major factor contributing to imprecision was the inability to generate relevant questions in some instances which prevented filtering out irrelevant trials. The system (https://covidtrialx.dbmi.columbia.edu) and its source code (https://github.com/WengLab-InformaticsResearch/COVID19-TrialFinder) are accessible online. Hassandoust, Akhaghpour, and Johnson examine individual privacy concerns and intention to adopt contact tracing mobile applications through a situational privacy calculus model.7 Using structural equation modeling and a national sample (N = 853) of survey respondents, they found that risk beliefs, perceived individual and societal benefits to public health, privacy concerns, privacy protection initiatives (legal and technical protection), and technology features (anonymity and use of less sensitive data) influenced intention to install a contact tracing mobile application. The relationship between trust in public health authorities and intention was indirect. Sex, education, and past invasion of privacy had no significant influence on intention. Study findings provide the foundation for actions to address the identified factors and, subsequently, increase adoption of contact tracing mobile applications. Oiao and colleagues describe and evaluate the Focal Loss bAsed Neural Network EnsembLe (FLANNEL) approach for COVID-19 detection in chest x-ray images.8 They constructed a dataset from 2 publicly available sources comprising 5508 chest x-ray images for 2874 patients with 4 classes of diagnoses: normal, bacterial pneumonia, non-COVID-19 viral pneumonia, and COVID-19 pneumonia. The addition of focal loss to the neural network approach was designed to address class imbalance. FLANNEL consistently outperformed baseline models in COVID-19 identification achieving a precision of 0.78, recall of 0.86, and F1 of 0.82. These promising findings highlight the potential of the FLANNEL approach for differentiating COVID-19 from other types of pneumonias. To estimate the hospitalization risk for people with comorbidities infected by SARS-CoV-2, Gao and Dong applied a Bayesian approach designed to overcome challenges with traditional biostatistical requirements for risk estimates.9 These include data about the number of infected people who were not hospitalized and their comorbidities. The former is particularly problematic for COVID-19, where many individuals are asymptomatic, and not all those infected have been tested. Using 2 large-scale datasets (2491 patients from COVID-NET and 5700 patients from New York hospitals), they estimated the posterior distribution of the risk ratio using the observed frequency of comorbidities in hospitalized COVID-19 patients and the prevalence of comorbidities in the general population. They found that cardiovascular diseases carried the highest hospitalization risk for COVID-19 patients, followed by diabetes, chronic respiratory disease, hypertension, and obesity. This innovative approach has the potential to assist with resource planning to manage the COVID-19 pandemic. There is no doubt that the COVID-19 pandemic has highlighted the need for and visibility of biomedical and health informatics. For JAMIA, this is reflected in a dramatic increase in Altmetric Attention Score which represents the mentions that papers are receiving in news outlets and social media. As always, I hope that you will consider JAMIA as a publication venue for informatics and data science research relevant to the challenging public health problems facing our world. None declared.

The articles highlighted in this issue focus on data generated, viewed, or interacted with by patients or consumers. Such data are increasingly part of healthcare and research processes. Several papers also address the issue of racial bias or disparities.1–3 A variety of data sources, informatics processes, and tools are illustrated through the papers including machine learning,1,4 mapping to an information model,2 open notes,3 EHR-enabled dashboard,5 and conversational agent.4 Lwowski and Rios explored the fairness of different machine learning methods for the specific task of detecting influenza-related content by comparing the performance of each machine learning model (support vector machine, convolutional neural network, bidirectional long short-term memory) on tweets written in Standard American English (SAE) vs African American English (AAE).1 The datasets for training and testing matched real-world scenarios in which there is a large imbalance between SAE and AAE examples with the latter ranging from 2–5% in both datasets. Fairness was defined as equality of opportunity and predictive equality. The former assumes that the false negative rate is equal between 2 groups and the latter is a measure of the difference between the false positive rates of 2 groups. The authors illustrate the potential influence on racial inequalities in influenza. If social media is used to hot spot influenza, a high false negative rate could lead to inadequate resources (ie, lost opportunity) to fight the virus. In contrast, in a study of misinformation, a high false positive rate means that accurate information in AAE text is identified as misinformation and could exacerbate vaccination disparities. The analysis showed that all machine learning methods were unfair on both datasets with differences between SAE and AAE ranging from 0.01 to 0.23 for false negative rates and 0.01 to 0.35 for false positive rates. They recommend assessing fairness along with traditional evaluation metrics noting that the study’s purpose will inform whether false positive or false negative rate is more important. Within the context of the All of Us Research Program, which collects family health history from multiple sources, Cronin et al compared data completed by participants via survey with electronic health record (EHR) data mapped to the observational medical outcomes partnership (OMOP) data model.2 In their analysis of 58 872 participants with family health history data, 63% had survey data only, 26% had EHR data only, and 10.5% had both. Among those with a medically actionable genetic disorder, the predominant data source for family health history was survey (89%) with only 2% from both sources. For all disorders except breast cancer, surveys contributed at least 90% of the data. There was little overlap for those with both data sources. Challenges exist with both sources. Response to family health history questions in the All of Us survey is low in general and even lower for populations under-represented in biomedical research. One EHR-related challenge is that mapping of EHR data to the OMOP can lead to loss of information. Study findings suggest that multiple data sources may provide more accurate family health history information than a single source alone. The open notes movement has made clinician visit notes accessible to patients and families. Lam, Bourgeois, Dong, and Bell analyzed survey data from 2 academic medical centers to describe patient and family attitudes, experiences, and barriers related to speaking up about perceived serious note errors.3 In a sample of 6913 adult patients and 3672 pediatric families, 93% agreed that reporting mistakes improves patient safety. Among those who read a note, 17% perceived at least 1 mistake and 44% of those considered the mistake to be serious, yet only 56% contacted their provider. Individuals who self-identified as Black or African American, Asian, “other,” or “multiple” race(s) or who reported poorer health were less likely to contact their provider. Common barriers to reporting a mistake were not knowing how to report a mistake (61%) and avoiding perception as a “troublemaker” (34%). To discover solutions for overcoming these barriers, qualitative comments were categorized as (a) clear instructions about how to report and whom to report to (65%), (b) cultural changes to encourage reporting (24%), and (c) ideas for patient–clinician collaboration on note accuracy (17%). Dalal and coauthors evaluated the effect of EHR-integrated digital health tools comprised of a checklist and video on transitions-of-care outcomes for patients preparing for discharge using a pre- (n=245) and postimplementation (n=234) cohort design.5 The patient digital tools were administered on a mobile device via a patient portal or web-based survey at least 24 hours prior to anticipated discharge. Clinicians could review checklist responses via an EHR-integrated safety dashboard. Patient cohorts varied significantly only in proportion of Hispanics and number of non-English speakers which were higher in the preimplementation cohort. There were no significant differences in patient activation score between pre- and postimplementation, but length of stay was significantly higher. The authors suggest that the checklist may have encouraged patients to inquire about their discharge preparedness and that other factors associated with patient activation and length of stay may explain their findings. McKillop et al report on the use the Watson Assistant platform to develop conversational agents to deliver COVID-19 related information.4 Watson Assistant uses natural language processing and machine learning in intent understanding, entity extraction, query expansion, and finding answers through estimating document relevancy. Between March 30 and August 10, 2020, 37 institutions in 9 countries deployed conversational agents using Watson Assistant. This included 24 governmental agencies, 7 employers, 5 provider organizations, and 1 health plan resulting in > 6.8 million messages delivered through the platform with an average of 1.9–3.5 conversational turns per session. This study supports the ability of a wide variety of organizations to develop conversational agents during a free trial use period of Watson Assistant. In 1976, informatics pioneer, Warner Slack, said that the patient is the “largest and least utilized resource in healthcare.”6 Thirty-five years later, this remains true. However, I believe this statement can be expanded to included patient- and consumer-generated data. As experts in biomedical and health informatics, we must not only think about how to integrate informatics tools into the daily lives of patients and consumers, but also how we learn from the data they generate to advance discoveries, improve healthcare systems, and increase health equity. None declared.

In a 2015 JAMIA article, Friedman et al, reporting on the proceedings of a National Science Foundation workshop, called for a “science of learning systems” for healthcare.1 Other articles in JAMIA have explicitly addressed the learning health system,2,3 and many more have addressed specific components of such a system. In this editorial, I highlight 5 articles related to the science of learning systems. Three articles recognize clinician expertise as a key knowledge asset in a learning system and describe ways of mining and supporting clinician expertise within the context of an electronic health record (EHR).4–6 Two additional articles address critical questions related to institutional knowledge assets, the practical implementation of predictive models in clinician workflow,7 and transportability of phenotype algorithms across settings.8 Arguing that there are signals of clinicians’ expert and knowledge-driven behaviors within EHRs that can be exploited to support clinical prediction, Rossetti et al describe the iterative development of the Healthcare Process Modeling Framework to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals).4 The framework was developed and evaluated using modeling and simulation testing based on the Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) predictive model which detects and leverages signals of clinician expertise for prediction of patient trajectories. The framework includes a 3-step modeling technique: (1) identification of patterns of clinical behaviors from user interaction through mining EHR data; (2) interpretation of patterns as proxies of an individual’s decisions, knowledge, and expertise; and (3) use of patterns in predictive models for associations with outcomes. Jung and colleagues characterized clinician EHR activities as tasks and leveraged unsupervised learning approaches to learn tasks from sequences of events in EHR audit logs.5 Using EHR audit logs, they developed metrics to characterize the prevalence of unique events and event repetition and categorized the tasks into 4 complexity profiles. They compared the profiles on performance time, event type (eg, view problem list, select patient from lookup), and clinician prevalence, and number of unique clinicians who were observed performing these tasks. To evaluate their methods, they subsequently applied the methods to audit logs generated by 33 neonatal intensive care unit nurses across 57 234 sessions and 81 tasks, finding significant differences in performance time for task complexity profiles. However, there were no significant differences in clinician prevalence or in the frequency of viewing and modifying event types among task complexity profiles. Such methods that learn from audit log data may be useful in assisting organizations to refine EHR workflows to support clinical tasks. Morris and colleagues, reflecting deep expertise in informatics, patient safety, and quality management, argue that most current clinical decision-support (CDS) tools or aids lack detail and neither reduce burden nor enable replicable actions.6 To address this issue, they propose a feasible subset of therapeutic decision-support tools based on credible clinical outcome evidence: computer protocols leading to replicable clinician actions (eActions). Beyond most current CDS, eActions emphasize consistent decisions and actions (ie, behaviors across clinicians) when faced with the same patient input data. Because eActions can reduce unwarranted variation, increase quality of clinical care and research, and reduce EHR noise, they are a critical component of a learning healthcare system. JAMIA receives many submissions reporting the development and validation of predictive models. However, little is known about implementation of such models as important knowledge assets into clinical workflow. Jung et al argue that the benefit of using a predictive model for identifying patients for interventions is highly dependent on the capacity to execute the workflow triggered by the model.7 To address this issue, the authors provide a framework for quantifying the impact of healthcare delivery factors and work capacity constraints on achieved benefit of the predictive model. This is illustrated through analyzing the impact of triggering an Advanced Care Planning workflow based on predictions of 12-month mortality. Such frameworks are necessary to guide decision makers on if and how to integrate predictive models into clinical workflow within the context of a learning health system. Another critical question for a learning health system is whether a knowledge asset developed in 1 population or institution is transportable to another? Geva et al tested the transportability of multimodal automated phenotyping (MAP), a scalable, high-throughput phenotyping method, developed using EHR data from an adult population to a pediatric population.8 They applied MAP to a pediatric population enrolled in a biobank and evaluated performance against physician-reviewed medical records. In addition, they compared performance of MAP at the pediatric institution and the original adult institution for 6 validated phenotypes, finding that MAP performed equally well in both contexts. The participants in the National Science Foundation workshop referenced in the introduction identified 4 system requirements for a learning health system: (1) an economically stable and governable learning health system; (2) a learning health system trusted and valued by all stakeholders; (3) an adaptable, self-improving, stable, certifiable, and responsive learning health system; and (4) a learning health system capable of engendering a virtuous cycle of health improvement.1 I contend that the articles highlighted in this issue represent advances in the science of learning systems and contribute to enabling requirements 2–4. Innovations in informatics and data science remain critical to reach the vision of a learning health system. None declared.

As I write this editorial in May of 2021, there are broad indications of reopening and decreasing COVID-19 pandemic restrictions in the US, while there are major pandemic hotspots globally. Like many others, I am hopeful that the lessons from the pandemic can be applied to major public health issues in the future. How transferrable are the theories, models, algorithms, and informatics-based solutions that we’ve developed? Through the years, we’ve certainly argued that this is a fundamental characteristic of informatics as a scientific field. I highlight 5 COVID-related studies in this issue, and I ask you to reflect, as I have done, on the key lessons for the future. In partnership with state and local public health agencies as well as health systems, Dixon et al describe the development and implementation of population-level dashboards that are deployed on top of a statewide health information exchange.1 Two dashboards collate information on individuals tested for and infected with COVID-19. The primary dashboard enables authorized users working in public health agencies to monitor populations in detail. In contrast, a public version provides higher-level situational awareness to inform ongoing pandemic response efforts in communities. Over the span of 2 months, the dashboards were accessed by 74 317 distinct users, indicating substantial use. In terms of usefulness, the private dashboard enabled detection of a local community outbreak associated with a meat-packing plant. The authors call for continued investment in a statewide health information exchange as a critical component of public health infrastructure. Klann and co-authors describe the development and validation of a computable phenotype for COVID-19 severity by the Consortium for Clinical Characterization of COVID-19 by EHR (4CE).2 4CE is an international collaboration that is addressing COVID-19 through federated analyses of electronic health record (EHR) data. They developed an EHR-based severity phenotype, consisting of 6 code classes using patient hospitalization data, and validated the phenotype in twelve 4CE international sites against the outcomes of intensive care unit admission and/or death. The full 4CE severity phenotype developed by experts had a pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of intensive care unit admission and/or death; however, the sensitivity of individual code categories for acuity had high variability. The authors also conducted a pilot in 1 site in which they compared selected predictors of severity between a machine learning approach and the 4CE phenotype with mean areas under the curve reported as 0.956 (95% confidence interval, 0.952–0.959) and 0.903 (95% confidence interval, 0.886–0.921), respectively. The authors suggest that the severity phenotype comprising 6 code classes was resilient to coding variability across institutions, but they raised the concern that machine learning approaches may overfit hospital-specific orders. These findings contribute to the literature about generic vs institution-specific approaches. Malden, Heeney, Bates, and Sheikh conducted a qualitative study to develop an in-depth understanding of how hospitals with a long history of health information technology (HIT) use responded to the COVID-19 pandemic from an HIT perspective.3 Informed by a topic guide, they interviewed 44 healthcare professionals with a background in informatics from 6 hospitals internationally via videoconference. They applied thematic analysis to develop a coding framework and identify emerging themes. This resulted in 3 themes and 6 subthemes. Key findings included (a) HIT was employed to manage time and resources during a surge in patient numbers through fast-tracked governance procedures and the creation of real-time bed capacity tracking within EHRs; (b) improving the integration of different hospital systems was important across sites; (c) use of hard-stop alerts and order sets was perceived as effective in helping to respond to potential medication shortages and select available drug treatments; (d) use of information from multiple data sources to develop alerts facilitated patient treatment; and (e) risk of nosocomial infections was reduced through upscaling/optimization of telehealth and remote working capabilities. Because of these changes, informaticians felt more valued by hospital management than prior to the COVID-19 pandemic. All study findings have relevance for the future application of HIT along with informatics expertise to address other significant public health concerns. Given that the requirements for facial mask wearing are decreasing, particularly for those who have completed their COVID vaccination course, what can we learn from 2 papers in this issue that focus on the topic of facial masks? He et al analyzed a total of 771 268 US-based tweets from January to October 2020.4 They first developed machine learning classifiers to identify and categorize relevant tweets and subsequently performed a qualitative content analysis of a subset of the tweets to understand the rationale of those who opposed mask wearing. Among 267 152 tweets that contained personal opinions about wearing facial masks to prevent the spread of COVID-19, the proportion of antimask tweets stayed constant at about the 10% level throughout the study period. Although negative effects, lack of effectiveness, and being unnecessary or inappropriate for certain people or under certain circumstances, were cited as reasons not to wear masks, such tweets were significantly less likely than promask tweets to cite external sources of evidence to support the arguments. The combination of machine learning classifiers and qualitative analysis to inform health communication offers a method that can be applied to other topics and data sources. Mercaldo and Santone designed a transfer learning approach that exploited the MobileNetV2 model to identify face mask violations using a data set of 4095 images of people with and without masks.5 They obtained an accuracy of 0.98 in face mask detection. While the study was motivated by the use case of the COVID-19 pandemic, transfer learning approaches are not widely reported in the biomedical and health informatics literature. This study may inspire other applications of transfer learning. JAMIA has published almost 100 COVID-related papers and will continue to do so, but as always our priority is on innovative and generalizable findings. None declared.

Almost 30 years ago, I published an article in JAMIA on the importance of an informatics infrastructure for quality assessment and improvement1; and about 20 years ago, I coedited a JAMIA supplement with Leslie Lenert, now an associate editor, on the role of informatics in patient safety and quality.2 Through the years, I’ve also had the privilege of participating in key Institute of Medicine (now National Academy of Medicine) reports on the intersection of informatics with safety and quality and the challenges that we still face in achieving the promise of informatics in this regard.3–5 Certainly the context has changed through the decades, with more recent attention to social determinants of health and health equity,6,7 but healthcare safety and quality remain a significant focus of clinical informaticians and what we publish in JAMIA. In this issue, I highlight 5 articles that address safety or quality from a variety of perspectives, including healthcare and public health . In an innovative multisite study, Willis et al investigated the user-centered design requirements for a theory-informed, peer mentoring-based, informatics intervention to activate patients undergoing outpatient hemodialysis in preventing a major safety concern, intradialytic hypotension (IDH).8 They conducted observations, patient interviews, and patient focus groups that included participatory design activities. Application of inductive and deductive qualitative data analytic techniques resulted in themes and design principles linked to constructs from social, cognitive, and self-determination theories. Patients identified characteristics of a desirable informatics intervention for IDH prevention including one that: (a) collapses distance between patients, peers, and family; (b) harnesses patients’ strength of character and resolve in all parts of their life; and (c) respects and supports patients’ needs, preferences, and choices. To enable the desired characteristics, the authors found that the design of the informatics intervention must support: depth of interpersonal connections; positivity; individual choice and initiative; and comprehension of connections and possible actions. The resulting informatics intervention will be evaluated in a pragmatic cluster-randomized controlled trial in 28 hemodialysis facilities in 4 US regions. A second qualitative study in this issue focuses on primary care teams’ perceptions of an event notification intervention, which was implemented to improve care coordination for geriatric patients in 2 Veterans Health Administration (VHA) medical centers.9 Building upon health information exchange, the alert notified primary care teams of non-VHA hospital admissions and emergency department (ED) visits. The researchers collected data through semistructured interviews of primary care team physicians, nurses, and medical assistants. The study design and analysis were guided by the Consolidated Framework for Implementation Research. Team members found the alerts were considered necessary, helpful for filling information gaps, and effective in supporting timely follow-up care. Concerns and suggestions for improvement included distinguishing alerts from other VHA notices, additional data on patients’ diagnosis and discharge instructions, and notifying additional team members to ensure alerts were acted upon. The authors identified the need to explore the optimal amount and types of information and delivery method across sites and test the integration of alerts into broader care coordination efforts. Two studies in this issue address prediction of key quality indicators including length of stay, mortality, and hospital admission after an ED encounter. In a retrospective cohort study (n = 6521 admissions) using the Multiparameter Intelligent Monitoring of Intensive Care III (MIMIC-III) database, Huang and coauthors compared the ability of text extracted using a “bag of words” approach from physician and nursing notes that were written in the first 48 hours of admission, to predict intensive care unit (ICU) length of stay and mortality using 3 methods (gradient boosting, logistic regression, and random forest).10 For the primary outcome of composite score of ICU length of stay ≥7 days or in-hospital mortality, the gradient boosting model had better performance than the logistic regression and random forest models. Nursing notes achieved higher area under the curve (AUC) than physician notes across all models, 0.826 and 0.796, respectively, for the gradient boosting model. The highest was for the combined notes (0.839); this along with the overlap rate of 0.38 between highly predictive words in the physician and nursing notes for the gradient boosting models suggests that physician and nursing notes are uniquely useful in predicting ICU outcomes. In a single-center prospective observational study in a tertiary pediatric hospital, Barak-Corren et al compared the accuracy of computer versus physician predictions of hospital admission from the ED encounter and explored the potential synergies of hybrid physician-computer models.11 From the perspective of safety and quality, early identification of likely admissions has the potential to reduce ED boarding times while waiting for hospital bed assignment, improve patient flow, and reduce errors, subsequently leading to improved care, enhanced patient satisfaction, and decreased ED overcrowding. Nine ED attending physicians predicted the likelihood of admission for 192 ED pediatric patients prior to the disposition decision. In addition, a random forest computer model predicted hospital admission for the cohort based on data available within the first hour of the ED encounter. Twenty-eight percent of patients were admitted. The positive predictive value for the prediction of admission was 66% for the clinicians, 73% for the computer model, and 86% for a hybrid model combining the 2. Researchers also found that physicians relied more heavily on patient clinical appearance while the computer model considered data such as rate of prior admissions or distance traveled to the hospital. The findings related to the combined model suggest support for Friedman’s fundamental theorem of medical informatics which stipulates that “a person working in partnership with an informatics resource is better than the same person unassisted.”12 The authors propose several scenarios for achieving integration of such a prediction model into the ED clinician workflow. Last, in a perspective, Lenert, Ding, and Jacobs highlight the challenges associated with promoting public safety through vaccination in the context of the current COVID-19 pandemic.13 They advocate for public health-healthcare system partnerships and “an information ecosystem with n-direction communication among public health and population health providers (ie, healthcare systems) with the goal of an independent but coordinated response to the challenges of the pandemic or other public health emergency at a population level.”13 They explicate 3 types of informatics innovations necessary to support such an ecosystem: (a) a national patient identifier for public health emergency purposes; (b) population-level data exchange for immunization data through combining approaches from existing immunization information systems and Flat Fast Healthcare Interoperability Resources standard protocols for bulk transfer of data; and (c) computable electronic quality measures. The authors contend that a comprehensive information ecosystem that links public health and population health providers is needed to address the challenges of COVID-19 pandemic and future challenges. AMIA’s practice analyses for the clinical informatics subspecialty certification for physicians and health informatics certification document the emphasis of informatics practice on patient safety and quality.14,15 Recognizing the complexity of the health issues that face the US and the world, JAMIA remains committed to advancing the science and application of informatics to achieve equitable, efficient, safe, and high-quality care throughout the broad health system. None declared.

In this Editorial, I highlight 2 critical issues for advancing the impact of the science and application of biomedical and health informatics methods, processes, and tools: replication studies and diversity, equity, and inclusion. In a previous article in Journal of the American Medical Informatics Association, Coiera et al1 asked if health informatics had a replication crisis and concluded that “taking replication seriously is essential if biomedical and health informatics is to be an evidence-based discipline.” In this issue, Coiera and Tong2 assess the frequency, fidelity, and impact of replication studies in the clinical decision support system (CDSS) literature. They identified CDSS replications across 28 health and biomedical informatics journals, assessed fidelity to the original study using 5 categories (identical, substitutable, in-class, augmented, out-of-class) and an innovative framework comprising 7 domains (investigators, method, population, intervention, setting, comparator, and outcome [IMPISCO]). Only 12 of 4063 publications retrieved from their search strategy were identified as actual replications; 6 related to one computer-based order entry study and replicated but which did not reproduce the findings of the original study. The authors conclude that “attention to replication should improve the efficiency and effectiveness of CDSS research” and call for characterization of core CDSS principles that require replication, identification of past replication data, and conduct of missing replication studies. The methods developed for this study, including the IMPISCO framework, fidelity scores, and fidelity heat map, as well as a proposed reporting structure for clearly identifying replication studies, provide a foundation not only for examining replication in CDSSs, but also for other informatics innovations. Four articles in the issue address an aspect of diversity, equity, and inclusion.3–6 Apathy et al3 examined the “advanced use” digital divide between critical access hospitals (CAHs) and non-CAHs by measuring electronic health record adoption and advanced use over time using American Hospital Association Information Technology survey data ((2008-2018) for patient engagement and clinical data analytics domains. The authors used a linear probability regression for each domain with year-CAH interactions to measure temporal changes in the relationship between CAH status and advanced use. In 2018, there were no differences in electronic health record adoption by CAH status; however, CAHs were less likely to demonstrate advanced use. The temporal analyses suggest that the advanced use divide has persisted for patient engagement and widened for clinical data analytics. The authors recommend that “policymakers should consider partnering with vendors to develop implementation guides and standards for functions like dashboards and high-risk patient identification algorithms to better support CAH adoption .” In a Perspective, Valdez et al4 contend that the widespread use of telehealth resulting from the COVID-19 (coronavirus disease 2019) pandemic has the potential to further exacerbate inequities faced by people with disabilities and that concerns about such inequities are expressed less frequently than those related to older adults, race, ethnicity, and socioeconomic status. In terms of intersectionality, people with disabilities are most often overrepresented in these categories. Because of the heterogeneity of the community with disabilities, the potential benefits (eg, improved convenience, lower transportation costs) and adverse consequences (eg, lack of access to those with communication-related disabilities) of the movement to telehealth will be differentially experienced. They propose 8 design considerations to maximize usability and usefulness of telehealth technologies for people with disabilities and provide implementation and policy considerations. The authors conclude that while the option to engage with telehealth may result in reduced barriers to care for some people with disabilities, inadequate attention to the design, implementation, and policy dimensions may be detrimental to others and must be addressed to mitigate health inequities faced by the disability community. In a second Perspective, authors from health and technology sectors discuss TechQuity using the working definition of “the strategic development and deployment of technology to advance health equity” and call for a commitment to action steps for achieving it.5 They argue that TechQuity requires that scientific as well as business strategy, product, and economic contributions create opportunities for all. The proposed recommendations for action steps include (1) invest in people and communities; (2) be trustworthy , collect data that are relevant to diverse communities, and keep these data secure; (3) use artificial intelligence and analytics to promote health equity; (4) integrate purchasers of technology in driving change; and (5) develop innovative partnerships that engage diverse communities. To compare men and women on the scholarly dissemination and receipt of awards at the Annual American Medical Informatics Association Symposium, Hartzler et al6 analyzed 2017-2020 American Medical Informatics Association submissions for differences in panels, papers, podium abstracts, posters, workshops, and awards for men compared with women. Labeling men and women Symposium authors and reviewers using Genderize.io, the authors compared submission and acceptance rates, performed regression analyses to evaluate the impact of the assumed gender, and performed sentiment analysis of reviewer comments. Although more men (60%) than women (40%) led submissions, acceptance rates were similar. Women-led submissions increased over the 4 years, but women were underrepresented in podium abstracts, panels, and workshops. In terms of reviewers, men provided longer reviews and increased the odds of rejection, while women’s reviews had more positive comments. Across awards in 3 categories (Research, Leadership, and Signature) in years 2017-2020, 49% of recipients were women with percentage of women by award category being 47%, 40%, and 58%, respectively. The authors conclude that there are opportunities to improve gender parity in some dissemination and award areas. Journal of the American Medical Informatics Association is committed advancing the impact of the science and application of biomedical and health informatics methods, processes, and tools. The articles highlighted in this Editorial not have only explicated critical issues that limit impact, but also have identified multiple steps for action. Linking the 2 highlighted issues, I believe that multiple domains (eg, investigators, population, and setting) of Coiera and Tong’s IMPISCO framework would support examination of the intersection of replication and diversity, equity, and inclusion as we continue to build the evidence base of our discipline across populations and settings. None.

The Editorial in this month’s issue calls for emergency action to limit global temperature increases and restore biodiversity and was published online on September 6, 2021, by more than 100 health and science journals.1 Based on decades of science that have documented the effects of climate change on health, the editorial argues that “only fundamental and equitable changes to societies will reverse our current trajectory” and urges immediate actions despite the continuing concerns raised by the COVID-19 (coronavirus disease 2019) pandemic. With equity at the center of the global response, the authors argue that wealthier nations must do more and do it faster. Many of the strategies delineated in the Editorial are consistent with Journal of the American Medical Informatics Association’s commitment to health equity. Our editorial leadership team is deliberating on our unique response and expertise as a biomedical and health informatics journal. As a result, readers will see more content at the intersection of climate, health, and informatics in future issues.

In January 2019, during my first month as JAMIA Editor-in-Chief, I was the lead author of a paper published in the American Journal of Public Health that focused on consumer information technologies (CIT) comprising mHealth, telehealth, and social media and made recommendations for enhancing their contribution to advancing health equity.1 Although the specific focus of the paper was behavioral interventions, I selected the 5 papers highlighted in this Editorial because they reflected several recommendations from the paper that are more generally relevant. Intervention design should: (A) integrate methods that facilitate the alignment of intervention focus, CIT platform, and user characteristics such as cultural beliefs, preferences, and functional, digital, and health literacy as well as the ecological context of use and (B) incorporate mechanisms of action for behavior change and persuasive design principles to sustain user engagement with CIT-enabled engagement. Advance multilevel interventions by linking mHealth and social media-based interventions with the healthcare system through electronic health record (EHR)-based approaches including clinical decision support, tethered patient portals, and clinical dashboards. Choudhury et al2 report the findings of quasi-controlled intervention to augment maternal health awareness among tribal pregnant mothers in India as a strategy for addressing maternal health disparities. The study provides an excellent illustration of addressing recommendation 1A.1 They compared oral education delivered by Hindi-speaking community health workers (standard of care village) to standard of care plus mHealth (intervention village) in women (n = 740 in each group) from 2 independent villages with similar sociodemographics on awareness of selected maternal health concerns. These included danger signs of pregnancy (eg, vaginal bleeding, severe blurring of vision) as well as prevention strategies (eg, tetanus injection, iron tablet consumption). The community health workers also ensured ambulance availability if needed and financial incentives for women delivering at the hospital for both groups. The mHealth application comprised 4 modules (registration, antenatal care, intranatal care, and postnatal care) and was used by the women in the presence of the community health worker. Aspects of the application that supported low literacy users included photographs and voice prompts. There were significant differences between standard of care and intervention groups on all awareness measures although the control group also improved on most measures. The study did not report maternal health outcomes but demonstrates promising findings related to increasing awareness of maternal health topics among women with low literacy by augmenting a community health worker intervention with mHealth. In a systematic review of 74 studies, Liu et al3 focus on user interface and persuasive design features in mHealth apps for older adults. While carefully considering user characteristics as in recommendation 1A, the review also highlights persuasive design features as emphasized in recommendation 1B.1 The authors extracted and synthesized recommendations related to user interface considerations for older adults from the 74 studies into 3 categories; 4 recommendations targeted cognitive and memory deterioration (eg, simple and consistent layout, easy navigation), 3 targeted perceptual capability (eg, font, color, audio), and 2 targeted motor coordination (eg, use of simple gestures such as tapping, minimize text input). The authors also classified persuasive features identified in studies into 5 categories: reminders (n = 20), social features (n = 17), game elements (n = 7), personalized interventions (n = 13), and health education (n = 27). A total of 37 studies in the review addressed both user interface and persuasive design features, both critical to use of mHealth apps by older adults. The authors note the lack of application of theory in the selection of design features in all, but 5 studies reviewed. They also call for the evaluation of specific design features to augment overall evaluation to advance the knowledge base of what works for specific populations as well as more rigorous evaluation designs. There is increased attention and federal requirements to the integration of a variety of apps including mHealth apps into electronic health record systems. Two papers in this issue address the topic of integration and address the second recommendation of Bakken et al.1 Barker and Johnson4 summarized the ecosystem of apps and software integrated with certified health information technology through characterizing the app market by EHR app gallery and type of app; tracking changes in the EHR app galleries from the end of 2019 through 2020; and examining how apps connect to EHR data systems, and support for the Health Level Seven (HL7) Fast Healthcare Interoperability Resources (FHIR) standard. They developed a program that gathered data from the public app galleries hosted by Allscripts, athenahealth, Cerner Corporation, Epic Systems Corporation, and Substitutable Medical Apps & Reusable Technology (SMART). The functionalities of 734 apps discovered through this process were classified according to 5 categories: Administrative (42%), Clinical Use (38%), Care Management (31%), Patient Engagement (20%), and Research (5%); a single app could have functionality in multiple categories. Clinical and care management apps supported the FHIR standard at a higher rate than administrative apps. While not specific to mHealth apps, the types of apps in Care Management and Patient Engagement categories reflect tasks typically supported in mHealth apps such as disease management, care planning, medication management, and patient education. However, they note that their approach may not represent all apps integrated with the EHRs. In particular, patient-facing apps may be more widely marketed in smartphone app stores, such as the Apple or Google app stores. Rudin et al5 sought to determine EHR integration requirements for a scalable remote symptom monitoring intervention for asthma patients and their providers. Guided by the Non-Adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework, they conducted a user-center design process with English- and Spanish-speaking patients as well as providers. Methods included: secondary analysis of interviews (n = 26) conducted for a previous feasibility study, design sessions (n = 21), and a primary care provider (PCP) survey (n = 55). In addition to functional requirements for the technology (patient app and provider dashboard), workflow needs of nurses and PCPs, and high-level user interface design needs for the purpose of collecting patient-reported outcome (PRO) data and monitoring asthma symptoms between visits, they identified 3 EHR integration requirements. First, to support PCPs access to the asthma PRO dashboard via a navigation bar from patient charts before or during a visit, the third-party application needed to be registered as an EHR extension. Second, a data services application programming interface was required to send EHR inbox notifications to nurses and PCPs for patient callback requests and previsit reminders. Third, to send previsit tips to patients and reminders to PCPs, the application required access to visit schedules and the identity of the PCP. While the authors implemented custom solutions to meet these EHR integration requirements due to existing expertise, they plan to explore FHIR as a future solution. Reflecting consideration of the ecological context of use (recommendation 1A),1 Alford-Teaster et al6 developed a measure of geographic access to telehealth to enable assessment of the contribution of telehealth access to alleviating the disparities in healthcare access in rural areas and for disadvantaged populations. The 2-step virtual catchment area (2SVCA) method is an enhancement to the standard 2-step floating catchment area (2SFCA) method. 2SVCA considers durability and speed of broadband access. They demonstrate the use of the measure through a case study of Vermont showing an increase in access as measured by 2SVCA as compared to 2SFCA. While appropriately acknowledging several limitations of their method, the authors argue that such a measure will enable policy analysis to assess the impact of the dramatic shift in increase of telehealth services and address questions such as: Will telehealth close the gaps or enlarge the divides? What policy or strategy can effectively mitigate any negative outcomes of telehealth while preserving and enhancing positive gains? Although the highlighted studies reflect a very small sample of the research being conducted and much remains to be done, such studies suggest that there is progress in contextualization, inclusion of persuasive design features, and integration of CIT with EHRs. None declared.

Dr. Randolph (Randy) A. Miller, founding Associate Editor and second Editor-in-Chief of the Journal of the American Medical Informatics Association (JAMIA), has been selected to receive the 2021 Morris F. Collen Award from the American College of Medical Informatics for his sustained and innovative contributions to the field including his pioneering research in clinical decision support systems. We have created an online collection of selected papers from among Dr. Miller’s publications in JAMIA that is available at: https://academic.oup.com/jamia/pages/morris-collen-award. The collection includes his paper in JAMIA’s inaugural issue, Medical Diagnostic Decision Support Systems—Past, Present, And Future: A Threaded Bibliography and Brief Commentary, which has been cited 569 times to date.1 This issue of JAMIA includes a historical review by Miller and Shortliffe along with three papers focused on clinical decision support.2–5 Given that Dr. Miller has played a significant role in advancing rigor in evaluation...

A 2019 Editorial by Associate Editor, Lenert,1 on the science of informatics and predictive analytics concluded that “It is not enough just to build tools that predict and describe them; authors who want to publish in Journal of the American Medical Informatics Association need to write about the science that ensures that they are predicting something that matters.” Our issues typically contain multiple predictive model papers. In this Editorial, I highlight 4 papers that reflect methodological innovation and predicting something that matters2–5 as well as a systematic review of development and validation of models predicting postacute care destination after adult inpatient hospitalization.6 Although the COVID-19 pandemic has dominated other public health issues in national attention, the opioid crisis has not abated. Through an academic-state health agency partnership, Ripperger et al2 developed and validated algorithms for predicting 30-day fatal and nonfatal opioid-related overdose. They created a 6-year observational cohort (n = 3 041 668 patients with 71 479 191 controlled substance prescriptions) using statewide data sources including prescription drug monitoring program data, Hospital Discharge Data System data, and Tennessee vital records as well as socioeconomic indicators. Data were divided into 75% training (10 subsets), 5% development, and 20% testing partitions to train, ensemble, and calibrate 10 random forest (RF) “weak learner” models. Validation was performed using area under the receiver operating curve, area under the precision-recall curve, risk concentration, and Spiegelhalter z-test statistic. Discrimination and calibration improved after ensembling. Risk concentration captured 47–52% of cases in the top quantiles of predicted probabilities. Such methods may complement traditional epidemiologic methods of risk identification, but prospective validation is needed. Clinicians often find it difficult to identify individuals at high risk for suicide. Bayramli et al3 argue that current algorithmic approaches for suicide risk detection fail to optimize temporal information to improve predictions. To address this concern, they developed and validated a temporally enhanced variant of the RF model—Omni-Temporal Balanced Random Forests (OT-BRF)—that incorporates temporal information in every tree within the forest using a corpus of longitudinal electronic health record data including clinician notes (1998–2018) from the Mass General Brigham Health System. They compared OT-BRF performance to a baseline Naive Bayes Classifier and 2 standard versions of balanced RFs. RF models were more accurate than Naive Bayesian classifiers at predicting suicide risk in advance and the proposed OT-BRF model performed best yielding a sensitivity of 0.339 at 95% specificity. The authors concluded that temporal variables such as visit frequency play an important role in detecting suicide risk and should be included in predictive models to improve performance. Yang et al4 also examined the role of temporal information, that is, whether data that are temporarily unavailable at prediction time can be used to improve the performance of a risk model. Their use case for addressing this issue was hospital-acquired infections (HAIs) which are associated with significant morbidity, mortality, and prolonged hospital length of stay. Perioperative risk prediction models typically use pre- and interoperative data to predict risk at the end of surgery when postoperative data is not yet available. They incorporated the temporarily unavailable data into 12 logistic/linear regression and deep learning models using a variety of intermediate representations of the data to predict 8 HAI outcomes. Although performance varied across models and outcomes, in all instances the integration of postoperative (ie, temporarily unavailable data at prediction time) improved performance as compared to the baseline model. Disentangling physiological patterns of menstruation from self-tracking behaviors (eg, skipping tracking) is necessary for the development of predictive models for menstrual cycle start dates. Li et al5 used data from a menstrual tracker with >2 million cycles to generate a probabilistic predictive model that (1) accounts explicitly for self-tracking adherence; (2) updates predictions as a given cycle and supports interpretable insight into how these predictions change over time; and (3) enables modeling of an individual’s cycle length history while incorporating population-level information. They compared the generative probabilistic predictive model with 5 baselines (mean, median, convolutional neural network, recurrent neural network, and long short-term memory network), finding that the model yields better predictions of next cycle start date and consistently outperforms the baseline models as the cycle evolves. The model also provides predictions of skipped tracking probabilities. The authors’ machine learning approach to modeling self-tracked cycle lengths separated true cycle changes from self-tracking behavior thus enabling more informed predictions and insights into the underlying observed data structure. Kennedy et al6 conducted a systematic review of development and validation of models predicting postacute care destination after adult inpatient hospitalization. Methods followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and the Cochrane Prognosis Methods Group criteria. Data were extracted based on the Critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS) checklist, and studies were evaluated based on predictor variables, validation, performance in validation, risk of bias, and applicability using the Prediction Model Risk of Bias Assessment Tool (PROBAST). Of the 35 models in 28 articles, 18 models were internally validated, 10 were externally validated, and 7 underwent both types of validation. Most models were developed using regression, demonstrated risk of bias, and had not been tested or implemented beyond original studies. The authors suggest that future studies should ensure the rigorous variable selection and follow Transparent Reporting of a multivariate prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines. Lu et al7 conducted an environmental scan summarizing existing reporting guidelines for clinical prediction models using that resulted in 220 reporting “atoms.” They organized the atoms into 8 stages of the creation and evaluation of a machine learning model to guide care: Use Case, Model Formulation, Model Development, Fairness in Model Development, Practical Feasibility, Utility Assessment, Deployment Design, Deployed Model (including Execution and Workflow), and Prospective Evaluation. This study and individual guidelines, such as those mentioned in the systematic review above, will inform the work of a JAMIA working group focused on providing clear guidance to JAMIA authors on content and methodological expectations. On a more personal note, I am writing this Editorial 4 days after the death of Dr. Virginia K. Saba, nursing informatics icon, who published a paper in the inaugural volume of JAMIA on the importance of standardized nursing terminologies.8 At a recent dinner with friends the night before I received the Sigma Theta Tau International Virginia K. Saba Nursing Informatics Leadership Award, I told Virginia that her terminology work on the Clinical Care Classification was being used in ways that she never imagined including the clinical prediction work of nurse informaticians Sarah Collins Rossetti, Kenrick Cato, and Patricia Dykes that we published in JAMIA.9 Virginia’s legacy lives on through such research and the multitude of individuals, including me, that she inspired and mentored. None declared.

Diana Forsythe, PhD, was a scholar of biomedical informatics, medical anthropology, artificial intelligence (AI), and feminism. With her upbringing by 2 renowned computer scientists, Drs. Alexandra Illmer Forsythe and George Forsythe, she was aware of the hard problems in computer science during her early years. Although she pursued a graduate degree in cultural anthropology and social demography, she went on to introduce methods, frameworks, and insights from the social sciences and the study of science and technology to the nascent fields of AI and biomedical informatics in the 1980s and1990s. The scope of her work was foundational in establishing people and organizational studies as a subdiscipline within biomedical informatics and as a working group with the American Medical Informatics Association (AMIA). Indeed, the current JAMIA editor-in-chief—a colleague of Diana’s at University of California, San Francisco—asked her to attend and reflect upon a 1997 AMIA workshop that brought together nursing vocabulary developers and other key stakeholders to address the topic of implementing nursing vocabularies in computer-based systems. Her reflections, published in JAMIA,1 pointed out the importance of culture and embedded practice in concept naming and questioned the need for a single nursing vocabulary. During her relatively brief career, Diana challenged researchers to understand that technology is never neutral and that attitudes and perspectives of researchers and technology developers profoundly influence fundamental aspects of technology design. Her work consistently challenged the field to pay attention to how people intended to use the technology, the social context within which the technology was implemented, and the potential broader and unintended impacts of technology. Through the rigorous application of qualitative methods through the lens of anthropology, her work identified how these factors influenced the intended users of technology in ways that could be detrimental. Throughout her publications and in the landmark posthumously published collection of essays, Studying Those Who Study Us: An Anthropologist in the World of Artificial Intelligence,2 Diana questioned inherent assumptions about the design and implementation of technology in medicine. In particular, her work reflected and advocated for the rigorous application of social science theory and methods in biomedical informatics research and practice. She also introduced feminist perspectives into analyses of social and technical contexts, emphasizing how the lived experiences of women intersected with technology development, use, and implementation. Toward this end, recent efforts such as the Women in AMIA Initiative and the AMIA First Look Program have sought to raise the visibility of the contributions of women and individuals from groups typically underrepresented in AMIA. As a fierce advocate for feminism and ethnography (ie, a subdiscipline of anthropology), Diana’s interactions with biomedical informatics, at the time a field largely dominated by men with computer science and information technology (IT) perspectives, were not always smooth and seamless. Her work, both as a researcher and an anthropologist, was deeply rooted in a sense of social justice and shed light on the needs and rights of disempowered communities. Additionally, Diana and her colleagues engaged in robust debates regarding rigor, reliability, and validity in qualitative research, including the importance of truly understanding and engaging with social sciences. Since her untimely death in 1997, Diana’s continued influence on the field lives on. As a foundational leader in establishing the important research space that sits at the intersection of social sciences and biomedical informatics, her work sheds light on the relevance of social sciences methods and theories, notably the importance of subjective experience and context, in informatics and computer science fields, which have a dominant perspective of objective reality. Those of us who work in people and organizational spaces in the biomedical informatics field have been profoundly influenced by Diana’s work and have also sought to build on the foundations that she established. The sustained impact of her body of research on the field can be witnessed through discussions at panels at the AMIA Symposium3 and through awards that bear her name including the annual AMIA Diana Forsythe Award honoring an outstanding publication at the intersection of the social sciences and biomedical informatics, the annual American Anthropological Association (AAA) Diana Forsythe Prize, and the Forsythe Dissertation Award for Social Studies of Science, Technology, and Health at the University of California, San Francisco or Stanford University. The definition of biomedical informatics has rapidly evolved over the last 25 years and is recognized today as a truly interdisciplinary field.4,5 This broader definition of the field recognizes the social sciences, human factors engineering, cognitive sciences, and multiple other disciplines as core contributors to continued progress in the field. However, too often, a gap still exists in the published biomedical informatics literature in capturing and representing these perspectives critical to the current healthcare context as new advancements and technologies (eg, mobile health, social media, health information exchange) emerge and as we strive to increase diversity of perspectives in the field. Over the last thirty years, researchers have continued to expand the rigorous application of qualitative methods in informatics and to include representation of patient voices and other stakeholder perspectives in technology design efforts. Thus, the time is ideal for a special issue focused on the continuing legacy of Diana Forsythe in biomedical informatics, identifying the current status of qualitative methods and ethnography in biomedical informatics, and charting a path toward the future. The purpose of our Special Issue was two-fold: to highlight the continued presence of people and organizational focused work in biomedical and health informatics; and to explore future directions for this critically important subdiscipline moving forward. In honoring and continuing Diana’s legacy, this special issue focuses on innovative and interdisciplinary scholarship at the intersection of biomedical informatics and the social sciences. In particular, it highlights the advances in knowledge about: (a) the methods and theories used to understand problems at the intersection of social sciences and informatics; (b) the impact of women and feminism in shaping the field of informatics; and (c) the role of human meaning in developing and implementing health IT and computational tools. Fourteen articles were accepted to this Special Issue from a total of 27 submissions. With the exception of a scoping review,6 all articles reported on original research studies at the intersection of informatics and the social sciences (Table 1). The majority of research studies were conducted in the United States,7–16 with the exception of 3 studies conducted in the United Kingdom,17 Netherlands,18 and Saudi Arabia.19 More than half of the studies were multisite investigations,7,13–18 while 5 studies were single-site8–12 and 1 study used an online context, LinkedIn.19 Most studies received some form of intramural or extramural funding,7,8,10,11,13–15,17 acknowledging the relevance and value of applying methods from social sciences to not only understand but also address research questions and clinical problems within biomedical informatics. In addition to the studies targeted at examining practices of patients and clinicians,8–12,14–17,19 there has been a recent shift toward broadening the scope of the stakeholder population under inquiry—for example, studies included healthcare researchers,13 women managers in the biomedical informatics field,19 and scribes.7 All but 1 study14 used multiple methods for collecting data. Six studies utilized qualitative methods to understand current workflows and gather user needs and design guidelines for health IT,7,8,12,14,16,17 while the remaining studies adopted mixed-methods approaches supported by observations, interviews, and to design and of health Most studies used a by research questions a of were used to the of the at the collection or the interdisciplinary of biomedical informatics, the reported in these studies were from other fields such as human computer computer supported work and social sciences. of studies that qualitative of JAMIA and were in publication over studies were conducted while studies were conducted Studies used interviews, observations, and have typically been the of but there has been an of on healthcare there has been a of qualitative in scope has been with the of qualitative methods used in other fields, such as and science and technology included for and of a broader of qualitative methods by and as as an of in to that to and 27 25 27 during a perspective of that some to included at the of the healthcare and with a or to work sought with the Most were to to the of to patient information of and of to work is in role for have human than in the of be new the role of needs to to be there could be an role for or as as of a new of healthcare and 3 patients and patient of patient The 3 of patient were identified in the patient the patient and and were were with in the shift on the factors can be identified that influence patient and and and future research by the include and in for and methods of critically the culture and of and and to such as and and Women managers in the field of biomedical informatics experiences and experiences meaning of biomedical informatics, meaning of health IT future and and meaning of Saudi in and in understanding the field of informatics not that was an feminist theory have The to be a women not to by attention to the understanding that women not how it to systems. Social have an impact on how women experience and on regarding of health IT and the biomedical informatics field. Studies at and the lived experiences of women in science and technology to the understanding of how these experiences knowledge of these fields in the context of healthcare advancements and work to patient to of cognitive to patient and through practices and patients to could patient to information and a to and individuals into patient and to information and on the of and and interviews, information and and for and and 5 key that health and also identified 3 practice of with health and in and to is and approaches to on context including health, social and the use, and be that the patient and the is not a for or to that the needs of the of health, the of and the and social in which to and to about of and of experience to and include technology to or new of and such information to on and systems. and in perspectives on and collecting on and methods, typically in This the need for and with mixed-methods from to for for future and but not was to working on and health and were critical within to technology included in in information and while included for and to the design of IT that can the needs of population health researchers in such as use of social media, information and tools. be by practice and of that can and human for practices in and for use during and interviews, over collection of of key to identifying and engaging user groups as users of the technology, or the as a as an and and human and in to on the social of with the that in technology the technology to a but of the technology in it to embedded in the the toward also a shift from with technology users to with patients as users and with healthcare as 2 design and of and and of the technology be to and patient and as as understand were multiple of design in informatics example, design researchers to insights and design that not have from or groups and the design a space individuals who could a included or and to individuals and included to medical and with could be to a meaning individuals have over experiences in the can technology through design. 27 and of over the and how this to and and was for patient and nursing time and of was not always and in for patients were both a key of and patient as a lens for understanding work, that can be by informatics to the of and to patient for the field to and design for include the of clinical experiences for technology developers to to the lives of technology and users of clinical to new technologies with and on the needs of patients and in patient during 3 1 medical 1 1 IT IT 1 3 1 1 research and 3 1 and through healthcare and to a health as and the is on the and needs of health to the application in the current health of diversity to a of of the group of patients with in new and were In to the design of patient in design and implementation and need to to with stakeholders to not only understand or to design but also to ways to such into design. a design culture in which the of in is as and a the fields of and on the and health informatics research and design on the is interactions with and with and and was was the as as that but only were regarding were in in the is foundational to of is key to cognitive and to cognitive The the of users were to in during these cognitive reported that and to were key to of The from this have researchers to a on application for for user in the clinical As of and have been in the in to impact on and to clinical and interviews, observations, qualitative Six of the with with 3 by than knowledge and The used health IT to increase to and patient with by health IT and social of and social with health IT for and and of health IT to were to This study how on a social to and with social from and and health IT that to can to health The an for the design and implementation of efforts to and clinical efforts. that qualitative of JAMIA and were in publication over studies were conducted while studies were conducted Studies used interviews, observations, and have typically been the of but there has been an of on healthcare there has been a of qualitative in scope has been with the of qualitative methods used in other fields, such as and science and technology included for and of a broader of qualitative methods by and as as an of in to that to and 27 25 27 during a perspective of that some to included at the of the healthcare and with a or to work sought with the Most were to to the of to patient information of and of to work is in role for have human than in the of be new the role of needs to to be there could be an role for or as as of a new of healthcare and 3 patients and patient of patient The 3 of patient were identified in the patient the patient and and were were with in the shift on the factors can be identified that influence patient and and and future research by the include and in for and methods of critically the culture and of and and to such as and and Women managers in the field of biomedical informatics experiences and experiences meaning of biomedical informatics, meaning of health IT future and and meaning of Saudi in and in understanding the field of informatics not that was an feminist theory have The to be a women not to by attention to the understanding that women not how it to systems. Social have an impact on how women experience and on regarding of health IT and the biomedical informatics field. Studies at and the lived experiences of women in science and technology to the understanding of how these experiences knowledge of these fields in the context of healthcare advancements and work to patient to of cognitive to patient and through practices and patients to could patient to information and a to and individuals into patient and to information and on the of and and interviews, information and and for and and 5 key that health and also identified 3 practice of with health and in and to is and approaches to on context including health, social and the use, and be that the patient and the is not a for or to that the needs of the of health, the of and the and social in which to and to about of and of experience to and include technology to or new of and such information to on and systems. and in perspectives on and collecting on and methods, typically in This the need for and with mixed-methods from to for for future and but not was to working on and health and were critical within to technology included in in information and while included for and to the design of IT that can the needs of population health researchers in such as use of social media, information and tools. be by practice and of that can and human for practices in and for use during and interviews, over collection of of key to identifying and engaging user groups as users of the technology, or the as a as an and and human and in to on the social of with the that in technology the technology to a but of the technology in it to embedded in the the toward also a shift from with technology users to with patients as users and with healthcare as 2 design and of and and of the technology be to and patient and as as understand were multiple of design in informatics example, design researchers to insights and design that not have from or groups and the design a space individuals who could a included or and to individuals and included to medical and with could be to a meaning individuals have over experiences in the can technology through design. 27 and of over the and how this to and and was for patient and nursing time and of was not always and in for patients were both a key of and patient as a lens for understanding work, that can be by informatics to the of and to patient for the field to and design for include the of clinical experiences for technology developers to to the lives of technology and users of clinical to new technologies with and on the needs of patients and in patient during 3 1 medical 1 1 IT IT 1 3 1 1 research and 3 1 and through healthcare and to a health as and the is on the and needs of health to the application in the current health of diversity to a of of the group of patients with in new and were In to the design of patient in design and implementation and need to to with stakeholders to not only understand or to design but also to ways to such into design. a design culture in which the of in is as and a the fields of and on the and health informatics research and design on the is interactions with and with and and was was the as as that but only were regarding were in in the is foundational to of is key to cognitive and to cognitive The the of users were to in during these cognitive reported that and to were key to of The from this have researchers to a on application for for user in the clinical As of and have been in the in to impact on and to clinical and interviews, observations, qualitative Six of the with with 3 by than knowledge and The used health IT to increase to and patient with by health IT and social of and social with health IT for and and of health IT to were to This study how on a social to and with social from and and health IT that to can to health The an for the design and implementation of efforts to and clinical efforts. medical informatics computer supported health information human computer health information for patient of studies that qualitative of JAMIA and were in publication over studies were conducted while studies were conducted Studies used interviews, observations, and have typically been the of but there has been an of on healthcare there has been a of qualitative in scope has been with the of qualitative methods used in other fields, such as and science and technology included for and of a broader of qualitative methods by and as as an of in to that to and 27 25 27 during a perspective of that some to included at the of the healthcare and with a or to work sought with the Most were to to the of to patient information of and of to work is in role for have human than in the of be new the role of needs to to be there could be an role for or as as of a new of healthcare and 3 patients and patient of patient The 3 of patient were identified in the patient the patient and and were were with in the shift on the factors can be identified that influence patient and and and future research by the include and in for and methods of critically the culture and of and and to such as and and Women managers in the field of biomedical informatics experiences and experiences meaning of biomedical informatics, meaning of health IT future and and meaning of Saudi in and in understanding the field of informatics not that was an feminist theory have The to be a women not to by attention to the understanding that women not how it to systems. Social have an impact on how women experience and on regarding of health IT and the biomedical informatics field. Studies at and the lived experiences of women in science and technology to the understanding of how these experiences knowledge of these fields in the context of healthcare advancements and work to patient to of cognitive to patient and through practices and patients to could patient to information and a to and individuals into patient and to information and on the of and and interviews, information and and for and and 5 key that health and also identified 3 practice of with health and in and to is and approaches to on context including health, social and the use, and be that the patient and the is not a for or to that the needs of the of health, the of and the and social in which to and to about of and of experience to and include technology to or new of and such information to on and systems. and in perspectives on and collecting on and methods, typically in This the need for and with mixed-methods from to for for future and but not was to working on and health and were critical within to technology included in in information and while included for and to the design of IT that can the needs of population health researchers in such as use of social media, information and tools. be by practice and of that can and human for practices in and for use during and interviews, over collection of of key to identifying and engaging user groups as users of the technology, or the as a as an and and human and in to on the social of with the that in technology the technology to a but of the technology in it to embedded in the the toward also a shift from with technology users to with patients as users and with healthcare as 2 design and of and and of the technology be to and patient and as as understand were multiple of design in informatics example, design researchers to insights and design that not have from or groups and the design a space individuals who could a included or and to individuals and included to medical and with could be to a meaning individuals have over experiences in the can technology through design. 27 and of over the and how this to and and was for patient and nursing time and of was not always and in for patients were both a key of and patient as a lens for understanding work, that can be by informatics to the of and to patient for the field to and design for include the of clinical experiences for technology developers to to the lives of technology and users of clinical to new technologies with and on the needs of patients and in patient during 3 1 medical 1 1 IT IT 1 3 1 1 research and 3 1 and through healthcare and to a health as and the is on the and needs of health to the application in the current health of diversity to a of of the group of patients with in new and were In to the design of patient in design and implementation and need to to with stakeholders to not only understand or to design but also to ways to such into design. a design culture in which the of in is as and a the fields of and on the and health informatics research and design on the is interactions with and with and and was was the as as that but only were regarding were in in the is foundational to of is key to cognitive and to cognitive The the of users were to in during these cognitive reported that and to were key to of The from this have researchers to a on application for for user in the clinical As of and have been in the in to impact on and to clinical and interviews, observations, qualitative Six of the with with 3 by than knowledge and The used health IT to increase to and patient with by health IT and social of and social with health IT for and and of health IT to were to This study how on a social to and with social from and and health IT that to can to health The an for the design and implementation of efforts to and clinical efforts. that qualitative of JAMIA and were in publication over studies were conducted while studies were conducted Studies used interviews, observations, and have typically been the of but there has been an of on healthcare there has been a of qualitative in scope has been with the of qualitative methods used in other fields, such as and science and technology included for and

Biomedical informatics pioneer Warner V. Slack, MD (1933–2018) is widely credited with saying “Patients are the most underused resource in healthcare” in the 1970s. In a previous editorial, I highlighted the continued relevance of consumer- and patient-oriented perspectives in biomedical and health informatics.1 Additionally, a recent American Medical Informatics Association analysis related to physician clinical informatics subspecialty practice2 as well as existing definitions for other health professionals (eg, nursing informatics3) emphasize the importance of this focus. Five articles in this issue reflect innovative biomedical and health informatics approaches for discovery, application, and analysis related to consumers and patients. Haldar et al4 examined patient experiences of undesirable events related to safety and quality of care. Based on a survey of 242 patients and caregivers during a hospital stay, the authors developed a 4-stage conceptual model that reflects patient experiences, from when they first encounter undesirable events, when they could intervene, when visible (eg, pain) and invisible (eg, lack of trust) harms emerge, what types of harms they experience, and what they do in response to harms. They also identified opportunities for informatics solutions related to each stage including “speaking up” risk assessments, apps for navigating invisible harm and reporting options. Their article illustrates how patient-oriented conceptual model development informed discovery of novel targets for informatics interventions.

Innovation is key for scientific advancement. One of the most common reasons that well-written manuscripts are rejected from Journal of the American Medical Informatics Association (JAMIA) is lack of innovation from the perspective of biomedical and health informatics. Oxford defines innovation (in something) as “a new idea, way of doing something, etc. that has been introduced or discovered.”1 Innovation in biomedical and health informatics can take multiple forms (eg, conceptual, topical, methodological, or application domains) and is relevant across manuscript types. In this editorial, I highlight 5 articles that illustrate different aspects of innovation. A research study by Kuo et al2 reflects innovation in its development of a framework that combines level-wise model learning, blockchain-based model dissemination, and a hierarchical consensus algorithm to construct generalizable predictive models using cross-institutional approaches. The framework is designed to take advantage of the privacy-preserving characteristics of blockchain ledger technology while considering the topology of large-scale research enterprises, which the authors characterize as a network of networks. As compared to centralized server privacy-preserving approaches, the peer-to-peer blockchain approach has advantages related to provenance as well as immutability and transparency of the models. They created an implementation of the framework called HierarchicalChain (Hierarchical privacy-preserving modeling on blockChain) and evaluated it using 3 healthcare and genomic datasets comparing HierarchicalChain’s predictive correctness, learning iteration, and execution time with a state-of-the-art method designed for flattened network topology. The authors found that HierarchicalChain improves the predictive correctness for small training datasets and provides comparable correctness results with the competing method with higher learning iteration and similar per-iteration execution time.

In my first editorial as Journal of the American Medical Informatics Association Editor-in-Chief, entitled “Doing What Matters Most,” I called for a consequentialist approach to biomedical informatics that is tied to improving our outcome of interest—human health.1 I further noted that this requires focusing our informatics research and its translation in practice on important health issues and the challenges facing our healthcare system. In this editorial, I highlight 4 clinical informatics articles that reflect a consequentialist perspective; 3 address an aspect of healthcare efficiency2–4 and the fourth describes an approach for mitigating the prescription opioid epidemic.5 In my inaugural editorial, I also noted that a consequentialist approach did not mean a lack of attention to methodological rigor. Thus, the fifth article highlighted from this issue concentrates on a methodological concern: predictive model calibration.6 Electronic health record (EHR)–associated clinician burnout remains a hot topic in clinical informatics and beyond7 and will be the focus of a 2021 Special Issue. See for the call for papers at https://academic.oup.com/jamia/pages/call-for-papers-clinician-burnout. Of high relevance to this topic, 2 articles in this issue address the need for standardized EHR metrics related to clinician efficiency.2,3

This issue of Journal of the American Medical Informatics Association issue includes 6 articles1–6 and a Correspondence7 that address the coronavirus disease 2019 (COVID-19) pandemic. We published the articles through Advanced Access immediately after acceptance to disseminate innovative informatics strategies and thought-provoking perspectives to inform clinical practice as well as policy decision making. This Open Access content is also available at jamia.org and on our publisher’s COVID-19 hub (https://academic.oup.com/journals/pages/coronavirus). The role of biomedical and health informatics has been critical in the system response to the COVID-19 pandemic. Thus, it is fitting that this issue starts off with an American Medical Informatics Association Position Paper that describes a health informatics practice analysis8 that complements the previously published American Medical Informatics Association clinical informatics subspecialty practice analysis.9 As compared with the latter, which focused on physicians, the focus of Gadd et al8 is on health informatics professionals comprising practitioners with clinical (eg, dentistry, nursing, pharmacy), public health, health informatics, or computer science training. The authors applied 2 methods to meet the practice analysis objective of developing a comprehensive and current description of what health informatics professionals do and what they need to know. First, 6 independent subject matter expert panels contributed to the development of a draft health informatics delineation of practice. Second, an online survey was distributed to health informatics professionals to validate the draft delineation of practice by rating the draft items related to domain, tasks, knowledge, and skills; qualitative feedback was also provided on the completeness of the delineation of practice. Informed by a sample of >1000 survey participants, this resulted in 5 domains, 74 tasks, and 144 knowledge and skill statements. Study findings will inform health informatics certification, accreditation, and education activities. The 4 COVID-19 articles highlighted in this editorial reflect the 5 domains identified in the health informatics practice analysis: foundational knowledge; enhancing health decision making, processes, and outcomes; health information systems; data governance, management, and analytics; and leadership, professionalism, strategy, and transformation,8 as well as similar domains in the physician clinical informatics subspecialty practice analysis.9 The 3 clinical articles illustrate the important relationships among technical knowledge and skills domains and those focused on decision making, processes, and outcomes, and leadership. Moreover, 2 articles highlight the important linkage between rapidly evolving federal policy and informatics practice during the pandemic.3,4 Reeves et al,1 from University of California, San Diego, describe the rapid implementation of technological support for optimizing clinical management of the COVID-19 pandemic from the perspective of an academic medical center. Critical to these efforts was the establishment of an Incident Command Center on February 5, 2020, for 24-hour monitoring and adaptation to rapidly evolving conditions and recommendations on a local, state, federal, and global scale. A second significant component informing the response was an assessment of the current state with regard to this context, which revealed institutional needs requiring technology support. This included the design and implementation of electronic health record (EHR)–based rapid screening processes, as well as expansion of system-level EHR documentation templates (eg, urgent care/emergency department screening or testing), clinical decision support (eg, isolation, who should be tested), reporting tools (eg, operational dashboard and tracking system for persons under investigation), and patient-facing technology (eg, video visits for outpatient encounters) related to COVID-19. The inclusion of information services representation in the Incident Command Center enabled real-time identification of failures and successes and a focus on evolving needs, which was foundational to building cohesive systems as an institutional response to the COVID-19 pandemic. Judson et al,2 from University of California, San Francisco, rapidly deployed a patient-facing self-triage and self-scheduling tool on their patient portal using a toolkit provided by their EHR vendor. They made the tool available to primary care patients with active portal accounts (about two-thirds of their 90 000 patients). Through the UCSF Coronavirus Symptom Checker module, basic demographic information is populated from the EHR, and asymptomatic patients are asked about exposure history and then provided relevant information. In contrast, symptomatic patients are triaged into 1 of 4 categories (emergent, urgent, nonurgent, or self-care) and subsequently connected to care via telephone hotline or self-scheduling. All responses and interactions are stored in the EHR. During the first 16 days of use, the tool was accessed 1129 times by 950 unique patients. The triage dispositions of the 72% of symptomatic patients were emergent (24%), urgent (24%), nonurgent (12%), and self-care (40%). The primary benefit of the tool beyond its efficiency for patients is prevention of unnecessary in-person encounters, which diminishes patient exposure, decreases personal protective equipment (PPE) use, and enables clinicians to focus on more acutely ill patients. In a Perspective, Turer et al,3 from Vanderbilt University Medical Center, describe an approach they call electronic PPE (ePPE) within the context of emergent policy changes related to telemedicine and the Emergency Medical Treatment and Labor Act during the COVID-19 pandemic. As distinct from telemedicine, they define ePPE as the use of telemedicine tools by on-site medical providers to perform electronic medical screening exams while limiting physical proximity. The authors discuss the safety, legal, and technical factors necessary for implementing such a pathway. In terms of safety, they recommend performing medical screening exams using ePPE only on “low-risk patients (ie, 4 [less urgent] to 5 [nonurgent]) with reassuring vital signs, few comorbidities, and chief complaints suggesting lower respiratory infection (fever, cough, shortness of breath).” Legally, ePPE is supported by a March 30, 2020, Centers for Medicare and Medicaid Services update to Emergency Medical Treatment and Labor Act enforcement that allows for on-site and off-site medical screening exams by qualified medical personnel using telemedicine equipment. From a technical perspective, they recommend using consumer devices such as FaceTime, Skype, and Zoom instead of dedicated telemedicine platforms because of their familiarity to providers. The approach of ePPE has the potential to facilitate more frequent patient-provider interactions in other settings while reducing exposure and conserving PPE. In a Perspective focused on balancing health privacy, health information exchange (HIE), and research in the context of the COVID-19 pandemic, Lenert and McSwain4 argue that “our current regulations on the flows of information for clinical care and research are antiquated and often conflict at the state and federal levels” and call for proposed changes to privacy regulations. They recommend consideration of 3 possible actions to enable the rapid communication of required health data necessary for a pandemic response by waiving the current legal barriers to HIE while ensuring the privacy of individual health information: The enactment of the Health Insurance Portability and Accountability Act’s complete federal preemption of all other data sharing and consent laws. The Office for Civil Rights should create a safe harbor business associate agreement that covers entities and that other supporting organizations can rapidly adopt for HIE about COVID-19. The Office for Civil Rights should issue guidance that clarifies that there is no requirement for minimal information in exchange of data for care of patients, and that transmission of minimal information does not apply to public health entities during this crisis. The authors conclude that use of emergency federal powers to create a unified framework for data exchange is an essential step toward effective response to the clinical, public health, and research challenges of the COVID-19 epidemic. In my first editorial as Editor-in-Chief, I called for a consequentialist informatics approach in which we focus our informatics research and its translation in practice on important health issues.10 The articles in this issue that focus on COVID-19 exemplify this approach and highlight the centrality of informatics in combating this devastating pandemic by doing what matters most. None declared.

A systematic review in this issue of Journal of the American Medical Informatics Association (JAMIA) focused on teledentistry reminds us that telemedicine and telehealth approaches have been around for decades.1 However, there is no doubt that the coronavirus disease 2019 (COVID-19) pandemic has dramatically changed not only the frequency of patient-clinician visits conducted via technology across a distance, but also the emergence of widespread electronic personal protective equipment (ePPE) when the patient and clinician are in the same setting, as highlighted by Turer et al2 and Wosik et al3 in last month’s JAMIA. Moreover, the variety of technologies for achieving such interactions has dramatically changed as illustrated by 4 articles in this issue.4–7 However, in another highlighted article in this issue, Ramsetty and Adams8 delineate the digital divide issues that make access to telehealth technologies difficult for vulnerable populations, including the homeless,...

Our field of biomedical and health informatics frequently considers the question of the impact of our research and its application on discovery, care delivery, health, and health equity. In this editorial I highlight 5 papers that illustrate scalable tools and standards-based infrastructure. Such innovations are components of the essential foundation for realizing the impact of informatics. Two papers address the critical issue of privacy preservation in secondary use of electronic health record (EHR) data,1,2 while a third focuses on information retrieval for COVID-19-related questions.3 Two additional papers focus on data definitions, codified vocabularies, and other components that enable semantic interoperability and their particular role in the COVID-19 pandemic.4,5 Carrell et al describe the hiding in plain sight (HIPS) approach which replaces personally identifying information (PII) tagged by a deidentification system with resynthesized content with the intent of making it harder to detect unredacted PPI.1 They used 2000 representative clinical documents from each of 2 healthcare settings to generate 2 deidentified 100-document corpora (200 documents total) where PII tagged by a typical automated machine-learned tagger was replaced by HIPS resynthesized content with a 10% leaky PII rate. Two readers from the originating institution and 2 external readers conducted aggressive reidentification attacks to isolate leaked PII. Mean recall and precision, respectively, varied for patient ages (9%, 37%), dates (32%, 26%), doctor names (25%, 37%), organization names (45%, 55%), and patient names (23%, 57%). Both recall and precision were higher for internal than external readers. While the HIPS results were superior to published findings of traditional redaction, they were inferior to a human adversary augmented by machine learning, suggesting the need for further refinement. Lee and colleagues address another aspect of privacy preservation based on the contention that unique trajectories of patients over time make it easier to reidentify patients.2 They focus their attention on set-valued sequences that describe chronological medical conditions of patients with the goal of learning and synthesizing realistic sequences of EHR data. They developed the dual adversarial autoencoder (DAAE) that learns set-valued sequences of medical entities by combining a recurrent autoencoder with 2 generative adversarial networks. They evaluated the performance of DAAE for diagnostic codes in the context of predictive modeling and plausibility as well as privacy preservation using MIMIC-III and the University of Texas Physicians clinical database. Their findings support the adequacy of DAAE performance for predictive modeling and clinical plausibility. In addition, the differentially private optimization aspect of their approach enabled generation of synthetic sequences without increasing the privacy leakage of patient data. Roberts et al describe the ongoing Text REtrieval Conference (TREC)-COVID information retrieval (IR) shared task whose goal is to galvanize the informatics community and provide the necessary data to help answer 6 important questions using the COVID-19 Open Research Dataset.3 These are: (1) What are the appropriate IR modalities (ad hoc search, filtering, question-answering, etc) for this kind of event? (2) What are effective methods for customizing the search engine to the specific needs of the situation? (3) Can existing data be leveraged (eg, via machine learning) to improve the search engine? (4) Can event-specific training data be created fast enough to have an impact? (5) How does one quantitatively evaluate the search engine’s performance? (6) How likely is it that different search engines have divergent enough performance to merit a quantitative comparison during a crisis? Participants are given about week from topic release (30 initial topics with 5 additional topics in each release) to result submission and can submit up to 1000 documents for each topic (eg, coronavirus hydroxy-chloroquine, coronavirus social distancing impact). Topics and data submissions are available at https://ir.nist.gov/covidSubmit/data.html. Performance is ranked per round. The processes and outcomes of the TREC-COVID shared task serve multiple purposes that influence discovery, care, and public health: (1) immediate support for researchers and clinicians fighting the pandemic; (2) development of a new IR evaluation process as the document collection, state of knowledge, and user interests evolve; and (3) a collection and approach to developing and implementing systems capable of satisfying information needs during pandemics. Standardized vocabularies enable secondary use of EHR data for research and health information exchange. Dong and coauthors describe the development and evaluation of a rule-based tool called COVID-19 TestNorm that automatically normalizes local COVID-19 testing names to standard Logical Object Identifiers, Names, and Codes (LOINC).5 Using 568 test names (454 for development and 114 for testing) collected from 8 healthcare systems, COVID-19 TestNorm achieved an accuracy of 97.4% on the test set. COVID-19 TestNorm is available as an open-source package for developers and as an online web application for end users (https://clamp.uth.edu/covid/loinc.php). The Centers for Disease Control and Prevention (CDC) COVID-19 Information Management Repository was created to address the need for public health and healthcare stakeholders to easily obtain access to comprehensive and up-to-date information management resources including those aimed at improving interoperability.5 Garcia et al provide an overview of 6 categories of COVID-19 resources in the Repository: (1) General (eg, LOINC Minimum Data Set for Public Health Emergency Operations Centers); (2) Emergency Medical Services (eg, National Emergency Medical Services Information System version 3 data dictionary); (3) Clinical Encounter (eg, SNOMED codes for COVID-19 patient encounters); (4) COVID-19 Public Health Reporting and Surveillance (eg, CDC COVID-19 Patient Impact and Hospital Capacity Module Form); (5) Laboratory Data Exchange and Laboratory Surveillance (eg, LOINC special use COVID-19 laboratory codes); (6) Geospatial Data Sets and Reference Sources (eg, CDC World COVID-19 Map). The repository is publicly available, distributed through CDC’s Public Health Information Network Vocabulary Access and Distribution System (https://phinvads.cdc.gov/vads/SearchVocab.action). The COVID-19 pandemic has certainly accelerated the application of biomedical and health informatics research to support discovery, care delivery, and health. At JAMIA, we remain particularly interested in innovative, scalable, and generalizable approaches that address important challenges to human health and health equity. None declared.

In June 2020, the American Medical Informatics Association (AMIA) Board of Directors unanimously approved the creation of the AMIA Diversity, Equity and Inclusion (DEI) Task Force to advise AMIA on specific, actionable steps to further address matters of racial diversity, equity and inclusion.1 As Editor-in-Chief of the Journal of the American Medical Informatics Association, I’m honored to serve on the Task Force along with other AMIA members, all committed to addressing DEI from their perspectives as informaticians and, for most, also from their lived experience as Black, Indigenous, or other persons of color. In parallel, the Journal of the American Medical Informatics Association editorial team has been considering how we can best advance DEI in our policies and practices and also address recently proposed recommendations for publishing on racial health disparities in a manner that appropriately considers the role of structural racism in disparities.2 I look forward to sharing the outcomes of our deliberations in the future. In this editorial, I highlight 5 articles in this issue that address at least 1 aspect of DEI in biomedical and health informatics or health care.

In celebration of the 200th birthday of Florence Nightingale, widely acknowledged as the founder of modern nursing, the World Health Organization declared 2020 as the International Year of the Nurse and Midwife. Nursing informatics is a key component of the broader field of biomedical and health informatics as well as the Nursing Informatics Working Group, a large and vibrant working group within the American Medical Informatics Association community. The American Nurses Association (ANA) defined nursing informatics as a specialty and published the first standards for its practice more than 25 years ago.1,2 As described in Journal of the American Medical Informatics Association (JAMIA), the definition has evolved through the years.3 The most recent ANA definition is the following: Nursing informatics is a specialty that integrates nursing science, computer science, and information science to manage and communicate data, information, and knowledge in nursing practice. Nursing informatics facilitates the integration of data, information, and knowledge to support patients, nurses, and other providers in their decision making in all roles and settings. This support is accomplished through the use of information structures, information processes, and information technology.4 In this editorial, we highlight 5 articles in this issue by or about nurses. We also announce the launch of a JAMIA Virtual Collection focused on nursing in celebration of the International Year of the Nurse and Midwife. Two articles focus on the “support patients” aspect of the ANA definition. Ryan Shaw, from the Duke University School of Nursing, and colleagues examined the use of multiple mobile health (mHealth) technologies to generate and transmit data from diverse patients with type 2 diabetes mellitus in between clinic visits.5 In a longitudinal feasibility trial, 60 patients were asked to self-monitor for 6 months using a variety of mHealth technologies (wireless glucometer, cellular scale, wrist-worn accelerometer, and medication adherence text message surveys) provided at baseline. In a diverse sample (60% Black/African American), they found that engagement with devices was highest for physical activity (ie, Fitbit), followed by glucose and weight. Fitbit engagement (87%) did not vary by age, hemoglobin A1C level, or race. However, engagement with the other mHealth technologies differed according to 1 or more of these variables. The authors concluded that it was feasible for participants from different socioeconomic, educational, and racial backgrounds to use and track relevant diabetes-related data from multiple mHealth technologies for 6 months, although engagement varied by patient characteristics. Patient-reported outcomes (PROs) are increasingly a component of self-monitoring of health status. Information visualizations may help patients interpret and contextualize their PROs. Turchoie et al6 assessed hospitalized patients’ objective comprehension (primary outcome) of text-only, nongraph, and graph visualizations that display longitudinal PROs. Forty patients viewed 4 conditions in a counterbalanced design that controlled for potential order effects: (1) text only, (2) text plus visual analogy, (3) text plus number line, and (4) text plus line graph. They assessed objective comprehension using the International Organization for Standardization protocol. Secondary outcomes included response times, preferences, risk perceptions, and behavioral intentions. Comprehension scores were 83% for text plus visual analogy, 70% for text plus number line, 62% for text only, and 60% for text plus line graph; the differences between text plus visual analogy and the latter 2 conditions were statistically significant. Interestingly, in participants who comprehended at least 1 condition, 14% preferred a condition that they did not comprehend. These findings highlight the need for assessing comprehension rather than preference in assessments of information visualizations. Two articles address aspects of quality and safety in nursing. Moore et al7 conducted a systematic review of the literature on the impact of health information technology (HIT) on nurses’ time. Following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and conducting a search of articles, published between January 2004 and December 2019, 33 articles met inclusion criteria: had comparison group in the design, measured the time taken to carry out documentation or medication administration, documented the quantitative estimates of time differences between the comparisons, had nurses as subjects, and was conducted in a care home, hospital, or community clinic. Twenty-one studies reported the impact of 12 different HIT implementations on nurses’ documentation time and showed an increase in time. In contrast, the time spent carrying out medication administration after bar code medication administration implementation was reduced by 33%. In some instances, nurses’ time after HIT implementation was redistributed to “value-adding” activities, such as direct patient care and interprofessional communication. Best practice guidelines are a key component of quality and safety across health professions and healthcare organizations. Multiple organizations have initiatives related to collection of such data as part of their quality and safety programs. In 2012, the Registered Nurses’ Association of Ontario launched the Nursing Quality Indicators for Reporting and Evaluation (NQuIRE) database to collect data on quality indicators derived from Registered Nurses’ Association of Ontario best practice guidelines. In a case report in this issue, Naik et al8 present a method to standardize data quality assessments of the NQuIRE database by developing a data quality framework and assessing key dimensions of the framework (integrity, relevance, interpretability, coherence, timeliness, institutional environment) using a data quality index. The data quality index is a single key performance metric for assessing the quality of the database, which is based on an aggregation of normalized measures across the 6 framework dimensions. This approach may be useful to others wishing to develop a data quality framework for a health data repository. Nurses most often practice as part of an interprofessional team. Members of interprofessional teams share distinct information, have specific information needs, and communicate what they know through various mechanisms and pathways. Recognizing unmet information needs is an important component of patient safety in care delivery. Cohen et al9 identified the unmet information needs of clinical teams delivering care to complex patients and then generated design principles to address those needs. The observational study involved care teams in 9 community health centers and included observations and interviews focused on their use of the electronic health record (EHR) when caring for patients with complex medical needs as well as needs related to social determinants of health (SDH). Nurse practitioners, registered nurses, and registered nurse care coordinators were among the types of nurses observed and interviewed. Analyses of >300 hours of observations and 51 interviews identified 4 major categories of information needs, related to consistency of SDH documentation, SDH information prioritization and changes to this prioritization, initiation and follow-up of community resource referrals, and timely communication of SDH information. There were 10 unmet information needs within these categories that were judged to be addressable through the EHR. They proposed 5 EHR design recommendations: enhance the flexibility of EHR documentation workflows, expand the ability to exchange information within teams and between systems, balance innovation and standardization of HIT systems, organize and simplify information displays, and prioritize and reduce information. To complement the articles in this issue, JAMIA is launching a Virtual Collection focused on nursing in celebration of the International Year of the Nurse and Midwife. The Collection is located at https://academic.oup.com/jamia/pages/nursing-collection. It contains a subset of the >200 articles published by or about nurses in JAMIA from 1994 to the present. We are proud to be part of the nursing informatics community. In our roles as JAMIA Editor-in-Chief and Chair of the Nursing Informatics Working Group of the American Medical Informatics Association, we hope that all biomedical and health informaticians will join us in celebrating the International Year of the Nurse and Midwife by reading aa article in this issue and by visiting the Virtual Collection. SB drafted the highlight section of the editorial. SB and GA contributed to the remaining sections and approved the final draft of the editorial. None declared.

In chronic conditions, patients and providers need support in understanding and managing illness over time. Focusing on endometriosis, an enigmatic chronic condition, we conducted interviews with specialists and focus groups with patients to elicit their work in care specifically pertaining to dealing with an enigmatic disease, both independently and in partnership, and how technology could support these efforts. We found that the work to care for the illness, including reflecting on the illness experience and planning for care, is significantly compounded by the complex nature of the disease: enigmatic condition means uncertainty and frustration in care and management; the multi-factorial and systemic features of endometriosis without any guidance to interpret them overwhelm patients and providers; the different temporal resolutions of this chronic condition confuse both patients and provides; and patients and providers negotiate medical knowledge and expertise in an attempt to align their perspectives. We note how this added complexity demands that patients and providers work together to find common ground and align perspectives, and propose three design opportunities (considerations to construct a holistic picture of the patient, design features to reflect and make sense of the illness, and opportunities and mechanisms to correct misalignments and plan for care) and implications to support patients and providers in their care work. Specifically, the enigmatic nature of endometriosis necessitates complementary approaches from human-centered computing and artificial intelligence, and thus opens a number of future research avenues.

In my pursuit of the JAMIA Editor-in-Chief position, I was inspired by Sandro Galea, a physician and epidemiologist, who argued in 2013 for a consequentialist epidemiology in which greater attention is paid to consequences as compared to deontological norms.1 Derived from moral philosophy, the former focuses on the outcomes of actions while the latter emphasizes adherence to set of norms or rules for the actions, eg, in the instance of research, its methodological standards. In this, my first editorial, I share a few thoughts on what it would mean for JAMIA to reflect a consequentialist perspective. Our field of biomedical and health informatics (including its subspecialties) has distinguished itself from the foundational sciences that inform it (eg, computer science, information science, decision science) by its motivation to improve human health, as illustrated by the definitions of biomedical informatics, nursing informatics, and clinical research informatics (Box 1).2–4 There is no doubt that relevance to human health is fundamental to what we do as informaticians, but a consequentialist perspective requires doing what matters most5 to improving our outcome of interest—human health. This suggests the importance of focusing our informatics research and its translation in practice on important health issues (eg, cancer, Alzheimer’s disease, opioid use disorder, antimicrobial resistance, family caregiver role strain), upstream social determinants of health, and the challenges facing our healthcare system (eg, complexity, cost, suboptimal patient engagement, insufficient coordination, mismatch between clinician needs and the available tools to support them). Moreover, a consequentialist perspective requires constant attention to the “so what?” question.6 Example Definitions Biomedical informatics is the interdisciplinary field that studies and pursues the effective uses of biomedical data, information, and knowledge for scientific inquiry, problem solving, and decision making, motivated by efforts to improve human health. Clinical research informatics involves the use of informatics in the discovery and management of new knowledge relating to health and disease. It includes management of information related to clinical trials and also involves informatics related to secondary research use of clinical data. Clinical research informatics and translational bioinformatics are the primary domains related to informatics activities to support translational research. Nursing informatics is the specialty that integrates nursing science with multiple information management and analytical sciences to identify, define, manage, and communicate data, information, knowledge, and wisdom in nursing practice. Nursing informatics supports nurses, consumers, patients, the interprofessional healthcare team, and other stakeholders in their decision making in all roles and settings to achieve desired outcomes. This support is accomplished through the use of information structures, information processes, and information technology. Aligned with a consequentialist approach, the first special focus issue during my tenure as Editor-in-Chief will be on health equity; details are provided in the Call for Papers (https://academic.oup.com/jamia/pages/cfp_health_equity). Several papers in the current issue are consistent with a consequentialist perspective, given the importance of the issues addressed to human health. For example, the mental health of military veterans is a major concern. Denneson et al7 demonstrate the influence of a web-based educational program for veterans who read their mental health notes online on their activation and perceived efficacy in healthcare interactions. Grasso et al8 lay the foundation for decreasing health disparities in lesbian, gay, bisexual, transgender, and queer people by presenting recommendations for planning and implementing the collection of accurate sexual orientation and gender identity information in electronic health records (EHRs). Zhan et al9 report a case study that combined survey and social media to address the significant public health issue of e-cigarette use and propose an innovative framework for social media data triangulation. This does not mean that JAMIA will decrease its emphasis on theoretical soundness and methodological quality. In fact, the Editorial team has worked to expand our instructions to authors to provide additional guidance for data science, qualitative, and mixed methods submissions and plans to expand guidance in other areas. Several articles in this issue reflect advances in methodological approaches. Fareed et al10 extend the evidence base on studying portal use by applying a hierarchical clustering algorithm to audit log files of >200 inpatients; the resulting clusters suggest implications for tailored engagement approaches. Turer et al11 report the development and validation of an algorithm that uses combinations of extractable EHR indicators (diagnosis codes, orders for laboratories, medications, and referrals) to detect recommended clinician behaviors: with attention to overweight/obesity/body mass index alone or with attention to hypertension/other comorbidities, or neither. Murray et al12 adapt methods that allow for automated “noisy labeling” of positive and negative controls to create a machine learning silver standard as compared to a time-consuming clinician gold standard for identification of systemic lupus erythematosus. However, a consequentialist perspective implies the need for greater attention to generalizability and external validity rather than simply focusing on internal validity. For example, how representative are the study sample and setting as compared with the target population or setting, and the subsamples analyzed? With regard to increasing representativeness of study samples, Pfaff et al13 applied multiple informatics methods to create an electronic recruitment workflow that includes a vendor-based EHR and its associated portal with direct electronic messaging to potential recipients as well as REDCap. Although this workflow yielded lower enrollment rates than clinic recruitment, enrollment rates were higher than traditional letters, and the workflow yielded benefits in patient population access. As you prepare your submissions to JAMIA, I ask you to think about the relevance to human health in terms of doing what matters most and to clearly delineate your answer to the “so what?” question.

In a recent Annals of Internal Medicine opinion piece, Downing et al1 argued that while physicians may identify the electronic health record (EHR) as a source of dissatisfaction and burnout, the real culprit is the documentation burden caused by regulations related to billing and reimbursement rather than the EHR itself and that improvements in informatics platform alone are insufficient to solve the issue. They support this argument by comparing physician documentation burden and EHR satisfaction in the United States with other countries such as Australia, the Netherlands, Singapore, and Denmark, many of whom have implemented an EHR that is also widely implemented in ambulatory care in the United States. These comparisons suggest that documentation burden is lower and EHR satisfaction is higher. Of course, physicians are only 1 user of the EHR. While there is a large body of literature on nurse burnout, studies related to nursing and the EHR have focused primarily on lack of features and functions to meet nursing needs.2 Burnout has been studied in advanced practice registered nurses who use the EHR similarly to physicians in the ambulatory care setting. A survey conducted by Harris et al3 in Rhode Island found that among advanced practice registered nurses using EHRs, insufficient time for documentation and the EHR adding to daily frustration were significant predictors of burnout. There is little literature on EHR-related burnout for other healthcare professionals such as dentists, pharmacists, medical social workers, and physical therapists. Like many clinicians drawn to biomedical and health informatics, I was attracted by my belief in the power of biomedical and health informatics to improve healthcare processes and health outcomes. Through my years in the field, I’ve grown to appreciate the policy context and I’m proud to be part of the American Medical Informatics Association, an organization that is known for its acumen regarding the intersection of health policy and informatics. Thus, while I strongly support Downing et al’s call for regulatory reform, I believe that innovation from the field is also necessary to help mitigate clinician burnout. Consequently, I highlight papers in this issue that focus specifically on clinicians and EHRs. Adding to the body of literature on EHRs and physician burnout, Gardner et al4 examined the relationship between physician EHR-related stress and burnout through a 2017 survey of all Rhode Island practicing physicians. Twenty-six percent of the 1792 respondents reported burnout and 70% reported health information technology–related stress. Physicians reporting poor or marginal time for documentation, reporting moderately high or excessive time spent on EHRs at home, and agreeing that EHRs add to their daily frustration had significantly higher odds of burnout as compared with those who did not. Colicchio and Cimino5 qualitatively synthesized 23 qualitative and mixed-methods studies about the use of EHR systems to support creation and use of clinical documentation. Two findings of the review are particularly relevant to the topic of clinician burnout. Five studies on note purposes found that nonclinical purposes have become more common. Note-entry studies (n = 6) revealed that EHR interfaces affect what clinicians document. They called for more research to investigate approaches to capture and represent clinicians’ reasoning and improve note entry and retrieval or reading. Two papers in this issue report on the design of innovations to improve clinician performance in common tasks that have potential to influence workload and burnout. Belden et al6 applied human factors and interaction design principles to iteratively create a medication timeline visualization intended to improve ease of use, speed, and accuracy in the ambulatory care of chronic disease. A pilot evaluation of the timeline visualization as compared to tabular presentation of the same information showed improved physician performance in 5 common medication-related tasks: (1) identify current prescription on a medication history, (2) identify past prescription on a medication history, (3) identify length of time medication has been prescribed, (4) identify new prescription in given time interval, and (5) identify dosage change in given time interval. Using a mixed methods design and scenario-based evaluation, Hosseini et al7 evaluated an information system designed to reconcile information across multiple electronic documents containing patient health records from a health information exchange network. Consolidation with as compared with without the information system resulted in higher accuracy, lower perceived workload, and shorter information reconciliation time in a small sample of physicians. The authors concluded that automating retrieval and reconciliation of information across multiple electronic documents is a promising approach for reducing healthcare providers’ task complexity and workload. Addressing an area of clinician workload that is on the rise, Herr et al8 argued that a first step in determining whether pharmacogenomics (PGx) clinical decision support will be effective in improving prescribing decisions is to examine process measures such as physician uptake and response. Their multisite pilot study focused on 2 prerequisites for examining these process measures—characterization of alert design within the eMERGE Network, and establishing a method for sharing PGx alert response data for aggregate analysis. Although the 6 pilot sites successfully shared response data, the authors concluded that the variation in PGx alert design is a barrier to multisite PGx clinical decision support studies. Many EHR innovations target the individual clinicians. While this is a necessary component, EHR innovations cannot help to mitigate clinician burnout without careful consideration of the socioecological context in which these innovations occur, including organizational culture, healthcare marketplace, technology ecosystem, and national policy.

Regardless of the type of biomedical and health informatics research conducted (eg computational, randomized controlled trials, qualitative, mixed methods), transparency, reproducibility, and replicability are crucial to scientific rigor, open science, and advancing the knowledge base of our field and its application across practice domains. These principles are also essential to high-quality publications in Journal of the American Medical Informatics Association (JAMIA). Transparency is reflected by explicit, clear, and open communication about the methods and procedures used to obtain the research results and is foundational to reproducibility (ability to repeatedly obtain the same results from data) and replicability (ability of other investigators to observe the same result under identical conditions).1,2 In the following paragraphs, I summarize key strategies from a number of authors1–3 as well as my own thoughts in 4 categories (data, code, connect, publish) and, when applicable, describe their relationship to publishing in JAMIA. While the principles apply across types of research, the relevance of some strategies varies. Record how each result was produced.3 The analysis workflow, including pre- and postprocessing steps, should be presented in your JAMIA manuscript as a figure or in an online supplement. Record all intermediate results.3 Deposit data in a repository. JAMIA authors can deposit their data in Dryad free of charge (https://datadryad.org/journal/1067-5027). Dryad provides a basic level of curation, assignment of a digital object identifier, and long-term data storage. JAMIA articles that include deposited data will receive additional promotion by both the American Medical Informatics Association and Oxford University Press. Publish a dataset paper if the dataset is unique and available for reuse by others. For JAMIA, a dataset paper is considered a Research and Applications paper. Avoid manual data manipulation steps.3 Record and, if possible, archive the exact versions of all external programs used.3 This includes not only data analysis programs, but also algorithms used to extract, filter, and reduce multidimensionality of the data at the various stages of the analysis workflow. These programs should be reported in your JAMIA manuscript. Apply and record version-control strategies to all custom scripts (eg, Git).3 Record underlying random seeds and which analysis steps involve randomness for analyses that include randomness.3 This should be reported in your JAMIA manuscript. Store raw data behind and the code used to create plot or figures.3 Generate layered output for inspection in levels of detail (eg, use HTML file hypertext links to go from summary to detail).3 Layered output can be included in the online supplement for your JAMIA manuscript. Connect textual statements to underlying results.3 Such annotations are a typical part of qualitative research and supported by multiple software packages (eg, ATLAS.ti, Dedoose, NVivo). Quantitative packages include the Sweave function in R that creates dynamic reports for integration into LaTeX or LyX documents. Literate programming approaches combine analysis code, plots, and text narrative that can be shared: for example, Jupyter Notebooks (for R, Python, and Julia; http://jupyter.org), R Markdown (http://rmarkdown.rstudio.com), and matlabweb (https://www.ctan.org/pkg/matlabweb).2 The site for your literate programming content should be referenced in your JAMIA manuscript. Register your research protocol.1,4 While registration of research protocols for clinical trials is routine at ClinicalTrials.gov/, other registries are emerging. For example, PROSPERO (https://www.crd.york.ac.uk/prospero/) is an international register for prospective systematic reviews. Your registered protocol should be noted in your JAMIA manuscript. Make a draft version of a manuscript available on a preprint server such as arXiv (https://arxiv.org/) or bioRxiv (https://www.biorxiv.org/). This does not preclude publication of the manuscript in JAMIA. Attend to general guidelines and checklists for publishing research and JAMIA author guidelines when preparing a JAMIA manuscript. General guidelines vary by research design and purpose, and include CONSORT (Consolidated Standards of Reporting Trials), (Strengthening the Reporting of Observational Studies in Epidemiology), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and SRQR (Standards for Reporting Qualitative Research). Many guidelines are available on the EQUATOR (Enhancing the Quality and Transparency of Health Research) Network website (https://www.equator-network.org/reporting-guidelines/). Provide public access to scripts, runs, and results.1,3 Multiple generic and health research-specific platforms support public access, and include GitHub (https://github.com/); CIELO (Collaborative Informatics Environment for Learning on Health Outcomes),5 which is now part of the Clinical and Translational Science Award Clinical Data to Health program (https://ctsa.ncats.nih.gov/cd2h/cd2h-labs/); and literate programming packages (eg, Jupyter Notebook, R Markdown). The site relevant to the contents of your JAMIA manuscript should be referenced. Clearly disclose research outcomes as primary, secondary, or exploratory in experimental research.1 Such specification is foundational to distinguish hypothesis testing from hypothesis-generating analyses. For JAMIA, an online supplement provides the opportunity to balance manuscript length with full disclosure. Gonul et al7 describe the development and validation of a template-based digital intervention design framework to support just-in-time adaptable interventions. Their automated approach provides the foundation for transparency, reproducibility, and replicability by (1) enabling experts to explicitly specify decision points, intervention options, tailoring variables and decision rules (through a rule definition language), and proximal or distal outcomes; and (2) dynamically tailoring intervention delivery strategies with respect to timing, frequency, and type (content) of interventions based on a personalization algorithm. The authors provide evidence for the extensibility of their design as well as preliminary validation of the personalization algorithm. Mercaldo et al8 report the study design used for the eMERGE (Electronic Medical Records and Genomic) Network’s survey of perspectives on broad consent and data sharing in biomedical research. To ensure that understudied populations were adequately represented (eg, minorities, those from rural areas, those with low educational attainment), they combined electronic health record data and U.S. Census data to construct sampling strata by imputing missing electronic health record data using the most frequent (mode) value from the patient’s census block group. Reproducibility and replicability are enabled by provision of details about the algorithmic approach as well as an example from 1 study site. Moreover, the authors are transparent about the limitations of their approach. To complement existing approaches for screening, brief intervention, and referral to treatment programs at trauma centers that have proven efficacy for reducing alcohol consumption and decreasing injury recurrence, Afshar et al9 applied the clinical Text Analysis and Knowledge Extraction System (cTAKES) natural language processor and machine learning to clinical emergency department notes for trauma admissions using the as the reference standard. Regarding transparency, the authors specified hypotheses about the expected performance of cTAKES on the corpus and developed the rule-based keyword algorithm that in advance of the cTAKES results. To support reproducibility and replicability, the reference standard (Alcohol Use Disorders Identification Test) and various programs or frameworks used are described, the supplement lists the Unified Medical Language System semantic types used for alcoholic beverages, and the source code is publicly available in Apache cTAKES SVN (subversion) repository. Lu et al,10 from the Lister Hill National Center for Biomedical Communications at the National Library of Medicine, developed a novel spell-checking tool, CSpell, that handles nonword errors, real-word errors, word boundary infractions, punctuation errors, and combinations of these errors and infractions for consumer-generated questions. Their approach uses dual embedding within Word2vec for context-dependent corrections in combination with dictionary-based corrections in a 2-stage ranking system. They also developed various splitters and handlers to correct word boundary infractions. The dual-embedding model shows a significant improvement in F1 score compared with the general practice of using cosine similarity with word vectors in Word2vec for context ranking and the 2-stage ranking system shows an almost 5% improvement in F1 score compared with the best 1-stage ranking system. In support of reproducibility and replicability, the software and the CSpell test set are available at https://umlslex.nlm.nih.gov/cSpell. The scientific community in general, our informatics community, and JAMIA are on a journey toward increased transparency, reproducibility, and replicability. I ask the JAMIA readers and authors to engage with the editorial team to make our journey one that meets the needs of our biomedical and health informatics community while advancing an open science framework.

Rigorous reviews that adhere to methodological standards can advance biomedical and health informatics knowledge by synthesizing research and assessing its quality, identifying knowledge gaps, and making recommendations for research, practice, or policy. Thus, reviews are an important manuscript type for Journal of the American Medical Informatics Association (JAMIA) and complement other types of research papers. Reviews can be characterized based on 4 methodological aspects: search strategy (formal or informal), appraisal of quality (present or absent), synthesis (narrative or quantitative), and analysis (eg, quantity, quality, themes, knowledge gaps, limitations, recommendations).1 In its 25 years, JAMIA has published more than 150 reviews; the frequency of reviews has increased in recent years with increasing awareness of the role of high-quality reviews as a foundation for future research on a topic. This has included scoping reviews that provide a preliminary assessment of potential size and scope of available research literature, but do not include a formal appraisal of study quality.2 Most recent JAMIA reviews are identified as systematic reviews that include a formal search strategy, appraisal of study quality, and a narrative3 or quantitative (ie, meta-analysis)4 synthesis of findings. JAMIA has also published critical reviews that synthesize the literature conceptually and offer key recommendations for the field.5

As I was selecting articles to highlight in this month’s editorial, I was struck by phrases such as data scientist, blockchain-distributed ledger technology, and consumer health informatics that weren’t around when I published an article in the first issue of Journal of the American Medical Informatics Association (JAMIA) 25 years ago or in the earlier decades when the field of medical informatics was founded. The fact that I was selecting the articles and writing the editorial on Presidents’ Day weekend led me to think about founding mothers and fathers and if and how the fundamentals of our field of biomedical and health informatics have changed through the subsequent years. In this editorial, I briefly summarize and comment on 5 articles in this issue from that perspective. Data science has been a topic of interest in the biomedical and health informatics community for almost a decade and was the focus of a 2013 JAMIA editorial by Lucila Ohno-Machado1 and a special issue in 2018; the latter characterized biomedical informatics and data science as evolving fields with significant overlap.2 In the current issue, Meyer content analyzed 3 months of job descriptions for data scientists across healthcare sectors (eg, academia, healthcare systems, pharma, insurance) by applying inductive and deductive qualitative approaches.3 In the sample of 198 job descriptions comprising more than 3200 skills, two-thirds listed statistics, R, machine learning, and storytelling as requirements. Half of the job descriptions included 6 additional skills: Python, communicating findings, developing products, data-driven problem solving, data manipulation, and developing algorithms. Mining unstructured data or text only appeared in 28.2% of the descriptions. Multidisciplinary team collaboration and data visualization were listed in 42.4% and 36.9% of the descriptions, respectively. The required skill sets reflect a variety of settings as well as the fact that 41.4% of the positions required only a bachelor’s degree, whereas 38.4% required a master’s degree and 9.1% a doctoral degree (e.g., PhD, MD). I was surprised that mining unstructured data or text was not a component of more job descriptions given the ubiquity of text in health data and the importance of natural language processing, in particular, in biomedical and health informatics. For example, a keyword search of natural language processing and JAMIA in PubMed yields 284 publications including articles led by pioneers Naomi Sager4 and Carol Friedman5 in JAMIA’s second issue. JAMIA’s first of 181 articles with machine learning as a keyword also appeared in 1994.6 My perspective is in agreement with the 2018 editorial—overlap exists. While the programming languages have changed, the foundational skills of a data scientist from the sample of job descriptions look like a familiar subset from my > a quarter of century old training in medical informatics.

For this issue, I am highlighting 5 articles chosen to reflect different aspects of our field of biomedical and health informatics as well as different types of Journal of the American Medical Informatics Association (JAMIA) articles. The topical editorial, which focuses on ClinicalTrials.gov, is provided by long-time Associate Editor Betsy Humphreys.1 Informaticians are keenly aware of differences in information technology (IT) maturity in various healthcare implementations and the importance of considering the stage of IT maturity when examining the relationship between IT and health processes and outcomes at setting, provider, and patient levels within and across organizations. Alexander et al2 describe the development of a nursing home IT Maturity Staging Model designed to capture stages of IT maturity. The focus of Phase I was to develop a preliminary model. Phase II involved three rounds of questionnaires administered to a Delphi panel of expert nursing home administrators to evaluate the validity of Phase I model. The model evolved from 5 to 7stages. The stages of the final model range from Stage 0 (non existent IT solutions or electronic medical record) to Stage 6 (use of data by nursing home resident and/or resident representative to generate clinical data and drive self-management).

An American Medical Informatics Association position paper reporting a clinical informatics practice analysis1 leads off this issue of Journal of the American Medical Informatics Association and is contextualized by an editorial from Doug Fridsma2 in which he notes that “Recognition of the unique set of knowledge and skills associated with clinical/health informatics practice will create opportunities and raise expectations for informatics professionals.” Regarding expectations for informatics professionals, the clinical practice analysis includes multiple specific tasks related to consumers and patients with associated areas of knowledge such as social determinants of health, use of patient-generated data, and consumer-facing health informatics applications (eg, patient portals, mobile health apps and devices, disease management, patient education, behavior modification). To emphasize this aspect of our field, I highlight other articles in this issue that explicitly reflect a consumer- or patient-oriented perspective. Bajracharya et al3 examined patients’ experiences of completing a 39-item family history via a patient portal; the results were summarized and integrated into their electronic health record (EHR). About one-third of 4223 patients who completed the family history also completed an online survey. Inductive analysis of free-text responses generated 5 main themes. On the positive side, patients described feeling empowered through sharing their family history and anticipated future predictive value from having the information integrated into the EHR. However, although the history tool was considered easy to use, it was also characterized as tedious, with some respondents raising concerns about validity (eg, not reflecting the complexity of family history) and privacy (eg, possible influence on insurance and employment). These findings suggest not only the continued promise of such an approach, but also the need for solutions that better address patient concerns.

Electronic health record (EHR)-based medication alerts have been the focus of decades of biomedical and health informatics research as well as health services research. The editorial by Associate Editor Julia Adler-Milstein characterizes the similarities and differences between health informatics and health services research and points out the requirement of informatics innovation for publication in JAMIA.1 Clinical decision support (CDS) for medication management including dose, route, contraindications, and drug–drug and drug–disease interactions is a core component of commercial EHRs. This begs the question: is there still a need for informatics innovation in EHR-based medication alerts? In this issue of JAMIA, 5 papers reflect biomedical informatics innovation related to EHR-based medication alerts. In a systematic review of 39 studies, Hussein, Reynolds, and Zheng examined medication safety alert fatigue, finding that interruptive CDS was least accepted by alert recipients.2 Among alternative models such as risk stratification tiers, providing shortcuts for common corrections, and tailoring to role (eg, pharmacist vs prescriber), only the last increased alert acceptance. The authors concluded that improved CDS interaction design and role tailoring may reduce medication safety alert fatigue. As a strategy to reduce alert burden, Bubp and colleagues developed a criteria-based scoring tool for assessing drug–disease knowledge base content, created a subset of alerts based upon scores, and implemented the alert subset across multiple Kaiser Permanente regions.3 The scoring identified 1211 out of 4111 contraindicated drug–disease pairs for inclusion in the subset. After clinician review, the subset was reduced to 1189 drug–disease alerts. Deployment of the score-based drug–disease alert subset resulted in a decrease in monthly alerts from 32 045 to 1168 with monthly acceptance rates ranging from 20.2%–29.8%. Wright et al examined the availability and use of structured override reasons for drug–drug interaction alerts in EHRs across 10 clinical sites in the US.4 The analysis resulted in 177 unique override reasons comprising 12 categories. The number of unique override reasons ranged from 3–100 per site. Three categories accounted for 78% of all overrides: “will monitor or take precautions,” “not clinically significant,” and “benefit outweighs risk.” Some override reasons attested to a future action (such as decreasing a dose or ordering monitoring tests), which would require an additional step after the alert is overridden. Among the implications of the study findings was the need to make alerts actionable. The purpose of the study by Monsen and colleagues was to determine if medication cost transparency alerts provided at the time of prescribing would lead ambulatory prescribers to reduce their use of high-cost medications.5 They deployed provider-level alerts to 1896 prescribers in ambulatory practices (58 primary care and 152 specialty care clinics) of a single health system. Prescribers in the randomly assigned intervention arm received a computerized alert whenever they ordered a medication from a high-cost medication class for which a lower-cost, equally effective, and safe alternative was available. Prescribing volume for the high-cost medications overall decreased by 32% (p < 0.0001) as compared to the 24-week baseline period supporting the efficacy of the strategy in a single health system. Tsopra et al designed AntibioHelp, a CDS system for antibiotic treatment which displays recommended and nonrecommended antibiotics and their properties, weighted by degree of importance based on clinical practice guidelines.6 One aspect of their study used information visualization to help general practitioners extrapolate guidelines to patients for whom there was no explicit guideline recommendation. In a small sample study, they found that visualization of weighted antibiotic properties did indeed help the general practitioners to extrapolate recommendations to their patients and increased their confidence in prescriptions for these patients as well. Despite decades of research and integration of medication management functions into commercial EHRs, there remains a need for biomedical informatics research to decrease alert fatigue and provide CDS to prescribers and others that matches their needs and supports the safety and value of the medication management process.

In my editorial1 that accompanied Gardner et al’s article2 on the impact of health information technology (HIT) on physician stress and burnout, I noted that “EHR innovations cannot help to mitigate clinician burnout without careful consideration of the socioecological context in which these innovations occur, including organizational culture, healthcare marketplace, technology ecosystem, and national policy.” I focused my editorial on the topic of clinician burnout to encourage continued discourse on this incredibly important topic. In a letter to the editor, Zsenits et al3 contend that the impact of HIT on physician burnout is unknown for now and provide a methodological critique of Gardner et al’s article. The authors’ response follows.4 With the publication of this correspondence in Journal of the American Medical Informatics Association, I aim to highlight of the importance of applying a variety of research methods to the important topic of HIT-related clinician burnout so that we can fully understand the complexity of the phenomenon to drive and create solutions that allow the HIT and clinician to each do what they do best. I look forward to future articles in Journal of the American Medical Informatics Association that continue to build the evidence base on this topic.

Founded in 1984, the American College of Medical Informatics (ACMI) is a college of elected Fellows from the United States and abroad who have made significant and sustained contributions to the field of biomedical informatics. On November 4, 2018, the 2018 Cohort of Fellows was introduced to the College and attendees at the American Medical Informatics Association (AMIA) Annual Symposium as well as to the public through Tweets. This article includes the introduction for each Fellow in the 2018 Cohort, which was read by Christopher G. Chute (ACMI President), Suzanne Bakken (ACMI Past President), or William M. Tierney (ACMI President-Elect); a somewhat tongue-in-cheek Tweet created by Gretchen Purcell Jackson or James J. Cimino; and a link to their Journal of theAmerican Medical Informatics Association (JAMIA) and JAMIA Open publications. This is followed by the traditional closing remarks of welcome into ACMI. Gregory L. Alexander,PhD, RN, FAAN, FACMI Potter-Brinton Endowed Professor, Sinclair School of Nursing and Department of Health Management and Informatics, University of Missouri

Objective: To determine the effects of an inpatient portal intervention on patient activation, patient satisfaction, patient engagement with health information, and 30-day hospital readmissions. Methods and Materials: From March 2014 to May 2017, we enrolled 426 English- or Spanish-speaking patients from 2 cardiac medical-surgical units at an urban academic medical center. Patients were randomized to 1 of 3 groups: 1) usual care, 2) tablet with general Internet access (tablet-only), and 3) tablet with an inpatient portal. The primary study outcome was patient activation (Patient Activation Measure-13). Secondary outcomes included all-cause readmission within 30 days, patient satisfaction, and patient engagement with health information. Results: There was no evidence of a difference in patient activation among patients assigned to the inpatient portal intervention compared to usual care or the tablet-only group. Patients in the inpatient portal group had lower 30-day hospital readmissions (5.5% vs. 12.9% tablet-only and 13.5% usual care; P = 0.044). There was evidence of a difference in patient engagement with health information between the inpatient portal and tablet-only group, including looking up health information online (89.6% vs. 51.8%; P < 0.001). Healthcare providers reported that patients found the portal useful and that the portal did not negatively impact healthcare delivery. Conclusions: Access to an inpatient portal did not significantly improve patient activation, but it was associated with looking up health information online and with a lower 30-day hospital readmission rate. These results illustrate benefit of providing hospitalized patients with real-time access to their electronic health record data while in the hospital. Trial Registration: ClinicalTrials.gov Identifier: NCT01970852.

OBJECTIVE: Natural language processing (NLP) of symptoms from electronic health records (EHRs) could contribute to the advancement of symptom science. We aim to synthesize the literature on the use of NLP to process or analyze symptom information documented in EHR free-text narratives. MATERIALS AND METHODS: Our search of 1964 records from PubMed and EMBASE was narrowed to 27 eligible articles. Data related to the purpose, free-text corpus, patients, symptoms, NLP methodology, evaluation metrics, and quality indicators were extracted for each study. RESULTS: Symptom-related information was presented as a primary outcome in 14 studies. EHR narratives represented various inpatient and outpatient clinical specialties, with general, cardiology, and mental health occurring most frequently. Studies encompassed a wide variety of symptoms, including shortness of breath, pain, nausea, dizziness, disturbed sleep, constipation, and depressed mood. NLP approaches included previously developed NLP tools, classification methods, and manually curated rule-based processing. Only one-third (n = 9) of studies reported patient demographic characteristics. DISCUSSION: NLP is used to extract information from EHR free-text narratives written by a variety of healthcare providers on an expansive range of symptoms across diverse clinical specialties. The current focus of this field is on the development of methods to extract symptom information and the use of symptom information for disease classification tasks rather than the examination of symptoms themselves. CONCLUSION: Future NLP studies should concentrate on the investigation of symptoms and symptom documentation in EHR free-text narratives. Efforts should be undertaken to examine patient characteristics and make symptom-related NLP algorithms or pipelines and vocabularies openly available.

Objective: To systematically synthesize the literature on information visualizations of symptoms included as National Institute of Nursing Research common data elements and designed for use by patients and/or healthcare providers. Methods: We searched CINAHL, Engineering Village, PsycINFO, PubMed, ACM Digital Library, and IEEE Explore Digital Library to identify peer-reviewed studies published between 2007 and 2017. We evaluated the studies using the Mixed Methods Appraisal Tool (MMAT) and a visualization quality score, and organized evaluation findings according to the Health Information Technology Usability Evaluation Model. Results: Eighteen studies met inclusion criteria. Ten of these addressed all MMAT items; 13 addressed all visualization quality items. Symptom visualizations focused on pain, fatigue, and sleep and were represented as graphs (n = 14), icons (n = 4), and virtual body maps (n = 2). Studies evaluated perceived ease of use (n = 13), perceived usefulness (n = 12), efficiency (n = 9), effectiveness (n = 5), preference (n = 6), and intent to use (n = 3). Few studies reported race/ethnicity or education level. Conclusion: The small number of studies for each type of information visualization limit generalizable conclusions about optimal visualization approaches. User-centered participatory approaches for information visualization design and more sophisticated evaluation designs are needed to assess which visualization elements work best for which populations in which contexts.

Health informatics studies the use of information technology to improve human health. As informaticists, we seek to reduce the gaps between current healthcare practices and our societal goals for better health and healthcare quality, safety, or cost. It is time to recognize health equity as one of these societal goals-a point underscored by this Journal of the American Medical Informatics Association Special Focus Issue, "Health Informatics and Health Equity: Improving our Reach and Impact." This Special Issue highlights health informatics research that focuses on marginalized and underserved groups, health disparities, and health equity. In particular, this Special Issue intentionally showcases high-quality research and professional experiences that encompass a broad range of subdisciplines, methods, marginalized populations, and approaches to disparities. Building on this variety of submissions and other recent developments, we highlight contents of the Special Issue and offer an assessment of the state of research at the intersection of health informatics and health equity.

Objective: Unintentional medication discrepancies contribute to preventable adverse drug events in patients. Patient engagement in medication safety beyond verbal participation in medication reconciliation is limited. We conducted a pilot study to determine whether patients' use of an electronic home medication review tool could improve medication safety during hospitalization. Materials and Methods: Patients were randomized to use a tool before or after hospital admission medication reconciliation to review and modify their home medication list. We assessed the quantity, potential severity, and potential harm of patients' and clinicians' medication changes. We also surveyed clinicians to assess the tool's usefulness. Results: Of 76 patients approached, 65 (86%) participated. Forty-eight (74%) made changes to their home medication list [before: 29 (81%), after: 19 (66%), p = .170]. Before group participants identified 57 changes that clinicians subsequently missed on admission medication reconciliation. Thirty-nine (74%) had a significant or greater potential severity, and 19 (36%) had a greater than 50-50 chance of harm. After group patients identified 68 additional changes to their reconciled medication lists. Fifty-one (75%) had a significant or greater potential severity, and 33 (49%) had a greater than 50-50 chance of harm. Clinicians reported believing that the tool would save time, and patients would supply useful information. Discussion: The results demonstrate a high willingness of patients to engage in medication reconciliation, and show that patients were able to identify important medication discrepancies and often changes that clinicians missed. Conclusion: Engaging patients in admission medication reconciliation using an electronic home medication review tool may improve medication safety during hospitalization.

Objectives: The Internet and social media are revolutionizing how social support is exchanged and perceived, making online health communities (OHCs) one of the most exciting research areas in health informatics. This paper aims to provide a framework for organizing research of OHCs and help identify questions to explore for future informatics research. Based on the framework, we conceptualize OHCs from a social support standpoint and identify variables of interest in characterizing community members. For the sake of this tutorial, we focus our review on online cancer communities. Target audience: The primary target audience is informaticists interested in understanding ways to characterize OHCs, their members, and the impact of participation, and in creating tools to facilitate outcome research of OHCs. OHC designers and moderators are also among the target audience for this tutorial. Scope: The tutorial provides an informatics point of view of online cancer communities, with social support as their leading element. We conceptualize OHCs according to 3 major variables: type of support, source of support, and setting in which the support is exchanged. We summarize current research and synthesize the findings for 2 primary research questions on online cancer communities: (1) the impact of using online social support on an individual's health, and (2) the characteristics of the community, its members, and their interactions. We discuss ways in which future research in informatics in social support and OHCs can ultimately benefit patients.

OBJECTIVE: To collaborate with community members to develop tailored infographics that support comprehension of health information, engage the viewer, and may have the potential to motivate health-promoting behaviors. METHODS: The authors conducted participatory design sessions with community members, who were purposively sampled and grouped by preferred language (English, Spanish), age group (18-30, 31-60, >60 years), and level of health literacy (adequate, marginal, inadequate). Research staff elicited perceived meaning of each infographic, preferences between infographics, suggestions for improvement, and whether or not the infographics would motivate health-promoting behavior. Analysis and infographic refinement were iterative and concurrent with data collection. RESULTS: Successful designs were information-rich, supported comparison, provided context, and/or employed familiar color and symbolic analogies. Infographics that employed repeated icons to represent multiple instances of a more general class of things (e.g., apple icons to represent fruit servings) were interpreted in a rigidly literal fashion and thus were unsuitable for this community. Preliminary findings suggest that infographics may motivate health-promoting behaviors. DISCUSSION: Infographics should be information-rich, contextualize the information for the viewer, and yield an accurate meaning even if interpreted literally. CONCLUSION: Carefully designed infographics can be useful tools to support comprehension and thus help patients engage with their own health data. Infographics may contribute to patients' ability to participate in the Learning Health System through participation in the development of a robust data utility, use of clinical communication tools for health self-management, and involvement in building knowledge through patient-reported outcomes.

OBJECTIVE: We compare 5 health informatics research projects that applied community-based participatory research (CBPR) approaches with the goal of extending existing CBPR principles to address issues specific to health informatics research. MATERIALS AND METHODS: We conducted a cross-case analysis of 5 diverse case studies with 1 common element: integration of CBPR approaches into health informatics research. After reviewing publications and other case-related materials, all coauthors engaged in collaborative discussions focused on CBPR. Researchers mapped each case to an existing CBPR framework, examined each case individually for success factors and barriers, and identified common patterns across cases. RESULTS: Benefits of applying CBPR approaches to health informatics research across the cases included the following: developing more relevant research with wider impact, greater engagement with diverse populations, improved internal validity, more rapid translation of research into action, and the development of people. Challenges of applying CBPR to health informatics research included requirements to develop strong, sustainable academic-community partnerships and mismatches related to cultural and temporal factors. Several technology-related challenges, including needs to define ownership of technology outputs and to build technical capacity with community partners, also emerged from our analysis. Finally, we created several principles that extended an existing CBPR framework to specifically address health informatics research requirements. CONCLUSIONS: Our cross-case analysis yielded valuable insights regarding CBPR implementation in health informatics research and identified valuable lessons useful for future CBPR-based research. The benefits of applying CBPR approaches can be significant, particularly in engaging populations that are typically underserved by health care and in designing patient-facing technology.

OBJECTIVE: Prior studies of computing applications that support patients' medication knowledge and self-management offer valuable insights into effective application design, but do not address inpatient settings. This study is the first to explore the design and usefulness of patient-facing tools supporting inpatient medication management and tracking. MATERIALS AND METHODS: We designed myNYP Inpatient, a custom personal health record application, through an iterative, user-centered approach. Medication-tracking tools in myNYP Inpatient include interactive views of home and hospital medication data and features for commenting on these data. In a two-phase pilot study, patients used the tools during cardiothoracic postoperative care at Columbia University Medical Center. In Phase One, we provided 20 patients with the application for 24-48 h and conducted a closing interview after this period. In Phase Two, we conducted semi-structured interviews with 12 patients and 5 clinical pharmacists who evaluated refinements to the tools based on the feedback received during Phase One. RESULTS: Patients reported that the medication-tracking tools were useful. During Phase One, 14 of the 20 participants used the tools actively, to review medication lists and log comments and questions about their medications. Patients' interview responses and audit logs revealed that they made frequent use of the hospital medications feature and found electronic reporting of questions and comments useful. We also uncovered important considerations for subsequent design of such tools. In Phase Two, the patients and pharmacists participating in the study confirmed the usability and usefulness of the refined tools. CONCLUSIONS: Inpatient medication-tracking tools, when designed to meet patients' needs, can play an important role in fostering patient participation in their own care and patient-provider communication during a hospital stay.

BACKGROUND: Self-monitoring is an integral component of many chronic diseases; however few theoretical frameworks address how individuals understand self-monitoring data and use it to guide self-management. PURPOSE: To articulate a theoretical framework of sensemaking in diabetes self-management that integrates existing scholarship with empirical data. METHODS: The proposed framework is grounded in theories of sensemaking adopted from organizational behavior, education, and human-computer interaction. To empirically validate the framework the researchers reviewed and analyzed reports on qualitative studies of diabetes self-management practices published in peer-reviewed journals from 2000 to 2015. RESULTS: The proposed framework distinguishes between sensemaking and habitual modes of self-management and identifies three essential sensemaking activities: perception of new information related to health and wellness, development of inferences that inform selection of actions, and carrying out daily activities in response to new information. The analysis of qualitative findings from 50 published reports provided ample empirical evidence for the proposed framework; however, it also identified a number of barriers to engaging in sensemaking in diabetes self-management. CONCLUSIONS: The proposed framework suggests new directions for research in diabetes self-management and for design of new informatics interventions for data-driven self-management.

OBJECTIVE: To systematically review existing literature regarding patient engagement technologies used in the inpatient setting. METHODS: PubMed, Association for Computing Machinery (ACM) Digital Library, Institute of Electrical and Electronics Engineers (IEEE) Xplore, and Cochrane databases were searched for studies that discussed patient engagement ('self-efficacy', 'patient empowerment', 'patient activation', or 'patient engagement'), (2) involved health information technology ('technology', 'games', 'electronic health record', 'electronic medical record', or 'personal health record'), and (3) took place in the inpatient setting ('inpatient' or 'hospital'). Only English language studies were reviewed. RESULTS: 17 articles were identified describing the topic of inpatient patient engagement. A few articles identified design requirements for inpatient engagement technology. The remainder described interventions, which we grouped into five categories: entertainment, generic health information delivery, patient-specific information delivery, advanced communication tools, and personalized decision support. CONCLUSIONS: Examination of the current literature shows there are considerable gaps in knowledge regarding patient engagement in the hospital setting and inconsistent use of terminology regarding patient engagement overall. Research on inpatient engagement technologies has been limited, especially concerning the impact on health outcomes and cost-effectiveness.

OBJECTIVES: To develop a method for investigating co-authorship patterns and author team characteristics associated with the publications in high-impact journals through the integration of public MEDLINE data and institutional scientific profile data. METHODS: For all current researchers at Columbia University Medical Center, we extracted their publications from MEDLINE authored between years 2007 and 2011 and associated journal impact factors, along with author academic ranks and departmental affiliations obtained from Columbia University Scientific Profiles (CUSP). Chi-square tests were performed on co-authorship patterns, with Bonferroni correction for multiple comparisons, to identify team composition characteristics associated with publication impact factors. We also developed co-authorship networks for the 25 most prolific departments between years 2002 and 2011 and counted the internal and external authors, inter-connectivity, and centrality of each department. RESULTS: Papers with at least one author from a basic science department are significantly more likely to appear in high-impact journals than papers authored by those from clinical departments alone. Inclusion of at least one professor on the author list is strongly associated with publication in high-impact journals, as is inclusion of at least one research scientist. Departmental and disciplinary differences in the ratios of within- to outside-department collaboration and overall network cohesion are also observed. CONCLUSIONS: Enrichment of co-authorship patterns with author scientific profiles helps uncover associations between author team characteristics and appearance in high-impact journals. These results may offer implications for mentoring junior biomedical researchers to publish on high-impact journals, as well as for evaluating academic progress across disciplines in modern academic medical centers.

OBJECTIVE: Publications are a key data source for investigator profiles and research networking systems. We developed ReCiter, an algorithm that automatically extracts bibliographies from PubMed using institutional information about the target investigators. METHODS: ReCiter executes a broad query against PubMed, groups the results into clusters that appear to constitute distinct author identities and selects the cluster that best matches the target investigator. Using information about investigators from one of our institutions, we compared ReCiter results to queries based on author name and institution and to citations extracted manually from the Scopus database. Five judges created a gold standard using citations of a random sample of 200 investigators. RESULTS: About half of the 10,471 potential investigators had no matching citations in PubMed, and about 45% had fewer than 70 citations. Interrater agreement (Fleiss' kappa) for the gold standard was 0.81. Scopus achieved the best recall (sensitivity) of 0.81, while name-based queries had 0.78 and ReCiter had 0.69. ReCiter attained the best precision (positive predictive value) of 0.93 while Scopus had 0.85 and name-based queries had 0.31. DISCUSSION: ReCiter accesses the most current citation data, uses limited computational resources and minimizes manual entry by investigators. Generation of bibliographies using named-based queries will not yield high accuracy. Proprietary databases can perform well but requite manual effort. Automated generation with higher recall is possible but requires additional knowledge about investigators.

This paper describes our considerations and methods for implementing an open-source centralized research data repository (CRDR) and reports its impact on retrospective outcomes research capacity in the urology department at Columbia University. We performed retrospective pretest and post-test analyses of user acceptance, workflow efficiency, and publication quantity and quality (measured by journal impact factor) before and after the implementation. The CRDR transformed the research workflow and enabled a new research model. During the pre- and post-test periods, the department's average annual retrospective study publication rate was 11.5 and 25.6, respectively; the average publication impact score was 1.7 and 3.1, respectively. The new model was adopted by 62.5% (5/8) of the clinical scientists within the department. Additionally, four basic science researchers outside the department took advantage of the implemented model. The average proximate time required to complete a retrospective study decreased from 12 months before the implementation to <6 months after the implementation. Implementing a CRDR appears to be effective in enhancing the outcomes research capacity for one academic department.

We describe a clinical research visit scheduling system that can potentially coordinate clinical research visits with patient care visits and increase efficiency at clinical sites where clinical and research activities occur simultaneously. Participatory Design methods were applied to support requirements engineering and to create this software called Integrated Model for Patient Care and Clinical Trials (IMPACT). Using a multi-user constraint satisfaction and resource optimization algorithm, IMPACT automatically synthesizes temporal availability of various research resources and recommends the optimal dates and times for pending research visits. We conducted scenario-based evaluations with 10 clinical research coordinators (CRCs) from diverse clinical research settings to assess the usefulness, feasibility, and user acceptance of IMPACT. We obtained qualitative feedback using semi-structured interviews with the CRCs. Most CRCs acknowledged the usefulness of IMPACT features. Support for collaboration within research teams and interoperability with electronic health records and clinical trial management systems were highly requested features. Overall, IMPACT received satisfactory user acceptance and proves to be potentially useful for a variety of clinical research settings. Our future work includes comparing the effectiveness of IMPACT with that of existing scheduling solutions on the market and conducting field tests to formally assess user adoption.

Usability factors are a major obstacle to health information technology (IT) adoption. The purpose of this paper is to review and categorize health IT usability study methods and to provide practical guidance on health IT usability evaluation. 2025 references were initially retrieved from the Medline database from 2003 to 2009 that evaluated health IT used by clinicians. Titles and abstracts were first reviewed for inclusion. Full-text articles were then examined to identify final eligibility studies. 629 studies were categorized into the five stages of an integrated usability specification and evaluation framework that was based on a usability model and the system development life cycle (SDLC)-associated stages of evaluation. Theoretical and methodological aspects of 319 studies were extracted in greater detail and studies that focused on system validation (SDLC stage 2) were not assessed further. The number of studies by stage was: stage 1, task-based or user-task interaction, n=42; stage 2, system-task interaction, n=310; stage 3, user-task-system interaction, n=69; stage 4, user-task-system-environment interaction, n=54; and stage 5, user-task-system-environment interaction in routine use, n=199. The studies applied a variety of quantitative and qualitative approaches. Methodological issues included lack of theoretical framework/model, lack of details regarding qualitative study approaches, single evaluation focus, environmental factors not evaluated in the early stages, and guideline adherence as the primary outcome for decision support system evaluations. Based on the findings, a three-level stratified view of health IT usability evaluation is proposed and methodological guidance is offered based upon the type of interaction that is of primary interest in the evaluation.

OBJECTIVE: Meaningful use of electronic health records (EHRs) is dependent on accurate clinical documentation. Documenting common goals in the intensive care unit (ICU), such as sedation and ventilator management plans, may increase collaboration and decrease patient length of stay. This study analyzed the degree to which goals stated were present in the EHR. DESIGN: Descriptive correlational study of common goals verbally stated during daily ICU interdisciplinary rounds compared with the presence of those goals, and actions related to those goals, documented in the EHR over the subsequent 24 h for 28 patients over 15 days. The study setting was a neurovascular ICU with a fully implemented electronic nursing and physician documentation system. MEASUREMENTS: Descriptive statistics and χ(2) analyses were used to assess differences in EHR documentation of stated goals and goal-related actions. Inter-coder reliability was performed on 16 (13%) of the 127 stated goals. RESULTS: One-quarter of the stated goals were not documented in the EHR. If a goal was not documented, actions related to that goal were 60% less likely to be documented. The attending physician note contained 81% of the stated ventilator weaning goals, but only 49% of the sedation weaning goals; additionally, sedation goals were not part of the structured nursing documentation. Inter-coder reliability (κ) was greater than 0.82. LIMITATIONS: Observations in a single ICU setting at a large academic medical center using a commercial EHR. CONCLUSION: The current documentation tools available in EHRs may not be sufficient to capture common goals of ICU patient care.

OBJECTIVE: The goals of this study were to explore the information needs of case managers who provide services to persons living with HIV (PLWH) and to assess the applicability of the Information Needs Event Taxonomy in a new population. DESIGN: The study design was observational with data collection via an online survey. MEASUREMENTS: Responses to open-ended survey questions about the information needs of case managers (n=94) related to PLWH of three levels of care complexity were categorized using the Information Needs Event Taxonomy. RESULTS: The most frequently identified needs were related to patient education resources (33%), patient data (23%), and referral resources (22%) accounting for 79% of all (N=282) information needs. LIMITATIONS: Study limitations include selection bias, recall bias, and a relatively narrow focus of the study on case-manager information needs in the context of caring for PLWH. CONCLUSION: The study findings contribute to the evidence base regarding information needs in the context of patient interactions by: (1) supporting the applicability of the Information Needs Event Taxonomy and extending it through addition of a new generic question; (2) providing a foundation for the addition of context-specific links to external information resources within information systems; (3) applying a new approach for elicitation of information needs; and (4) expanding the literature regarding addressing information needs in community-based settings for HIV services.

OBJECTIVE: Although electronic notes have advantages compared to handwritten notes, they take longer to write and promote information redundancy in electronic health records (EHRs). We sought to quantify redundancy in clinical documentation by studying collections of physician notes in an EHR. DESIGN AND METHODS: We implemented a retrospective design to gather all electronic admission, progress, resident signout and discharge summary notes written during 100 randomly selected patient admissions within a 6 month period. We modified and applied a Levenshtein edit-distance algorithm to align and compare the documents written for each of the 100 admissions. We then identified and measured the amount of text duplicated from previous notes. Finally, we manually reviewed the content that was conserved between note types in a subsample of notes. MEASUREMENTS: We measured the amount of new information in a document, which was calculated as the number of words that did not match with previous documents divided by the length, in words, of the document. Results are reported as the percentage of information in a document that had been duplicated from previously written documents. RESULTS: Signout and progress notes proved to be particularly redundant, with an average of 78% and 54% information duplicated from previous documents respectively. There was also significant information duplication between document types (eg, from an admission note to a progress note). CONCLUSION: The study established the feasibility of exploring redundancy in the narrative record with a known sequence alignment algorithm used frequently in the field of bioinformatics. The findings provide a foundation for studying the usefulness and risks of redundancy in the EHR.

OBJECTIVE: Clinical informatics intervention research suffers from a lack of attention to external validity in study design, implementation, evaluation, and reporting. This hampers the ability of others to assess the fit of a clinical informatics intervention with demonstrated efficacy in one setting for implementation in their setting. The objective of this model formulation paper is to demonstrate the applicability of the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework with proposed extensions to clinical informatics intervention research and describe the framework's role in facilitating the translation of evidence into practice and generation of evidence from practice. Both aspects are essential to reap the clinical and public health benefits of clinical informatics research. DESIGN: We expanded RE-AIM through the addition of assessment questions relevant to clinical informatics intervention research including those related to predisposing, enabling, and reinforcing factors and validated it with two case studies. RESULTS: The first case study supported the applicability of RE-AIM to inform real world implementation of a clinical informatics intervention with demonstrated efficacy in randomized controlled trials (RCTs)--the Choice (Creating better Health Outcomes by Improving Communication about Patients' Experiences) intervention. The second, an RCT of a personal digital assistant-based decision support system for guideline-based care, illustrated how RE-AIM can be used to inform the design of an efficacy RCT that captures essential contextual details typically lacking in RCT design and reporting. CONCLUSION: The case studies validate, through example, the applicability of RE-AIM to inform the design, implementation, evaluation, and reporting of clinical informatics intervention studies.

To effectively meet clinician information needs at the point of care, we must understand how their needs are dependent on both context and clinician type. The Infobutton Manager (IM), accessed through a clinical information system, anticipates the clinician's questions and provides links to pertinent electronic resources. We conducted an observational usefulness case study of medical residents (MDs), nurse practitioners (NPs), registered nurses (RNs), and a physician assistant (PA), using the IM in a laboratory setting. Generic question types and success rates for each clinician's information needs were characterized. Question type frequency differed by clinician type. All clinician types asked for institution-specific protocols. The MDs asked about unfamiliar domains, RNs asked about physician order rationales, and NPs asked questions similar to both MDs and RNs. Observational data suggest that IM success rates may be improved by tailoring anticipated questions to clinician type. Clinicians reported that a more visible Infobutton may increase use.

The authors summarize their experience in iteratively testing the adequacy of three versions of the Health Level Seven (HL7) Logical Observation Identifiers Names and Codes (LOINC) Clinical Document Ontology (CDO) to represent document names at Columbia University Medical Center. The percentage of documents fully represented increased from 23.4% (Version 1) to 98.5% (Version 3). The proportion of unique representations increased from 7.9% (Analysis 1) to 39.4% (Analysis 4); the proportion reflects the level of specificity in the document names as well as the completeness and level of granularity of the CDO. The authors shared the findings of each analysis with the Clinical LOINC committee and participated in the decision-making regarding changes to the CDO on the basis of those analyses and those conducted by the Department of Veterans Affairs. The authors encourage other institutions to actively engage in testing healthcare standards and participating in standards development activities to increase the likelihood that the evolving standards will meet institutional needs.

OBJECTIVE: To develop an electronic health record that facilitates rapid capture of detailed narrative observations from clinicians, with partial structuring of narrative information for integration and reuse. DESIGN: We propose a design in which unstructured text and coded data are fused into a single model called structured narrative. Each major clinical event (e.g., encounter or procedure) is represented as a document that is marked up to identify gross structure (sections, fields, paragraphs, lists) as well as fine structure within sentences (concepts, modifiers, relationships). Marked up items are associated with standardized codes that enable linkage to other events, as well as efficient reuse of information, which can speed up data entry by clinicians. Natural language processing is used to identify fine structure, which can reduce the need for form-based entry. VALIDATION: The model is validated through an example of use by a clinician, with discussion of relevant aspects of the user interface, data structures and processing rules. DISCUSSION: The proposed model represents all patient information as documents with standardized gross structure (templates). Clinicians enter their data as free text, which is coded by natural language processing in real time making it immediately usable for other computation, such as alerts or critiques. In addition, the narrative data annotates and augments structured data with temporal relations, severity and degree modifiers, causal connections, clinical explanations and rationale. CONCLUSION: Structured narrative has potential to facilitate capture of data directly from clinicians by allowing freedom of expression, giving immediate feedback, supporting reuse of clinical information and structuring data for subsequent processing, such as quality assurance and clinical research.

OBJECTIVE: The Tailored Interventions for management of DEpressive Symptoms (TIDES) program was designed based on social cognitive theory to provide tailored, computer-based education on key elements and self-care strategies for depressive symptoms in persons living with HIV/AIDS (PLWHAs). DESIGN AND MEASUREMENT: Based on an extension of the Technology Acceptance Model (TAM), a cross-sectional design was used to assess the acceptance of the HIV TIDES prototype and explore the relationships among system acceptance factors proposed in the conceptual model. RESULTS: Thirty-two PLWHAs were recruited from HIV/AIDS clinics. The majority were African American (68.8%), male (65.6%), with high school or lower education (68.7%), and in their 40s (62.5%). PARTICIPANTS spent an average of 10.4 minutes (SD = 5.6) using HIV TIDES. The PLWHAs rated the system as easy to use (Mean = 9.61, SD = 0.76) and useful (Mean = 9.50, SD = 1.16). The high ratings of behavior intention to use (Mean = 9.47, SD = 1.24) suggest that HIV TIDES has the potential to be accepted and used by PLWHAs. Four factors were positively correlated with behavioral intention to use: perceived usefulness (r = 0.61), perceived ease of use (r = 0.61), internal control (r = 0.59), and external control (r = 0.46). Computer anxiety (r = -0.80), tailoring path (r = 0-.35) and depressive symptoms (r = -0.49) were negatively correlated with behavioral intention to use. CONCLUSION: The results of this study provide evidence of the acceptability of HIV TIDES by PLWHAs. Individuals are expected to be empowered through participating in the interactive process to generate their self-care plan. HIV TIDES enables information sharing about depression prevention and health promotion and has the potential to reframe the traditional patient-provider relationship.

OBJECTIVES: This study sought to design and validate a reliable instrument to assess the quality of physician documentation. DESIGN: Adjectives describing clinician attitudes about high-quality clinical documentation were gathered through literature review, assessed by clinical experts, and transformed into a semantic differential scale. Using the scale, physicians and nurse practitioners scored the importance of the adjectives for describing quality in three note types: admission, progress, and discharge notes. Psychometric methods including exploratory factor analysis were applied to provide preliminary evidence for the construct validity and internal consistency reliability. RESULTS: A 22-item Physician Documentation Quality Instrument (PDQI) was developed. Exploratory factor analysis (n = 67 clinician respondents) on three note types resulted in solutions ranging from four (discharge) to six (admission and progress) factors, and explained 65.8% (discharge) to 73% (admission and progress) of the variance. Each factor solution was unique. However, four sets of items consistently factored together across all note types: (1) up-to-date and current; (2) brief, concise, succinct; (3) organized and structured; and (4) correct, comprehensible, consistent. Internal consistency reliabilities were: admission note (factor scales = 0.52-88, overall = 0.86), progress note (factor scales = 0.59-0.84, overall = 0.87), and discharge summary (factor scales = 0.76-0.85, overall = 0.88). CONCLUSION: The exploratory factor analyses and reliability analyses provide preliminary evidence for the construct validity and internal consistency reliability of the PDQI. Two novel dimensions of the construct for document quality were developed related to form (Well-formed, Compact). Additional work is needed to assess intrarater and interrater reliability of applying of the proposed instrument and to examine the reproducibility of the factors in other samples.

OBJECTIVE: The use of health information technology (HIT) for the support of communication processes and data and information access in acute care settings is a relatively new phenomenon. A means of evaluating the impact of HIT in hospital settings is needed. The purpose of this research was to design and psychometrically evaluate the Impact of Health Information Technology scale (I-HIT). I-HIT was designed to measure the perception of nurses regarding the ways in which HIT influences interdisciplinary communication and workflow patterns and nurses' satisfaction with HIT applications and tools. DESIGN: Content for a 43-item tool was derived from the literature, and supported theoretically by the Coiera model and by nurse informaticists. Internal consistency reliability analysis using Cronbach's alpha was conducted on the 43-item scale to initiate the item reduction process. Items with an item total correlation of less than 0.35 were removed, leaving a total of 29 items. MEASUREMENTS: Item analysis, exploratory principal component analysis and internal consistency reliability using Cronbach's alpha were used to confirm the 29-item scale. RESULTS: Principal components analysis with Varimax rotation produced a four-factor solution that explained 58.5% of total variance (general advantages, information tools to support information needs, information tools to support communication needs, and workflow implications). Internal consistency of the total scale was 0.95 and ranged from 0.80-0.89 for four subscales. CONCLUSION: I-HIT demonstrated psychometric adequacy and is recommended to measure the impact of HIT on nursing practice in acute care settings.

OBJECTIVES: To describe the development and validation of low literacy English and Spanish versions of the 26-item Telemedicine Satisfaction and Usefulness Questionnaire (TSUQ), report telemedicine satisfaction and usefulness ratings of urban and rural participants in the Informatics for Diabetes Education and Telemedicine (IDEATel) project, and explore relationships between utilization and perceptions of satisfaction and usefulness. METHODS: Data sources included TSUQ, utilization data from IDEATel log files, and sociodemographic data from the annual IDEATel interview. Psychometric analyses were conducted to examine the reliability and validity of TSUQ. Data were analyzed using descriptive, correlational techniques. RESULTS: The principal components factor analysis extracted two factors (Video Visits, alpha=.96, and Use and Impact, alpha=.92) that explained 63.6% of the variance in TSUQ satisfaction scores. All satisfaction and usefulness items had mean scores of greater than 4 on a 5-point scale. Those from urban areas reported significantly higher ratings on both factors than rural participants as did those who did not know how to use a computer at baseline. Mean frequency of utilization of IDEATel components was highest for blood sugar testing followed by web site for reviewing results, blood pressure testing, video visits, and ADA educational Web pages. Associations between utilization and perceptions of satisfaction and usefulness varied among IDEATel components. CONCLUSION: Psychometric analyses support the construct validity and internal consistency reliability of TSUQ, which is available in both English and Spanish at a readability level of 8th grade. Both rural and urban participants reported high levels of satisfaction and found all IDEATel components useful. Further work is needed to examine the relationships between utilization and perceptions of satisfaction and usefulness and to explore the effects of location (urban versus rural) and ethnicity on satisfaction with telemedicine services.

OBJECTIVE: The authors aimed to (1) formally represent OASIS-B1 concepts using the Logical Observation Identifiers, Names, and Codes (LOINC) semantic structure; (2) demonstrate integration of OASIS-B1 concepts into a concept-oriented terminology, the Medical Entities Dictionary (MED); (3) examine potential hierarchical structures within LOINC among OASIS-B1 and other nursing terms; and (4) illustrate a Web-based implementation for OASIS-B1 data entry using Dialogix, a software tool with a set of functions that supports complex data entry. DESIGN AND MEASUREMENTS: Two hundred nine OASIS-B1 items were dissected into the six elements of the LOINC semantic structure and then integrated into the MED hierarchy. Each OASIS-B1 term was matched to LOINC-coded nursing terms, Home Health Care Classification, the Omaha System, and the Sign and Symptom Check-List for Persons with HIV, and the extent of the match was judged based on a scale of 0 (no match) to 4 (exact match). OASIS-B1 terms were implemented as a Web-based survey using Dialogix. RESULTS: Of 209 terms, 204 were successfully dissected into the elements of the LOINC semantics structure and integrated into the MED with minor revisions of MED semantics. One hundred fifty-one OASIS-B1 terms were mapped to one or more of the LOINC-coded nursing terms. CONCLUSION: The LOINC semantic structure offers a standard way to add home health care data to a comprehensive patient record to facilitate data sharing for monitoring outcomes across sites and to further terminology management, decision support, and accurate information retrieval for evidence-based practice. The cross-mapping results support the possibility of a hierarchical structure of the OASIS-B1 concepts within nursing terminologies in the LOINC database.

The AMIA 2003 Spring Congress entitled "Bridging the Digital Divide: Informatics and Vulnerable Populations" convened 178 experts including medical informaticians, health care professionals, government leaders, policy makers, researchers, health care industry leaders, consumer advocates, and others specializing in health care provision to underserved populations. The primary objective of this working congress was to develop a framework for a national agenda in information and communication technology to enhance the health and health care of underserved populations. Discussions during four tracks addressed issues and trends in information and communication technologies for underserved populations, strategies learned from successful programs, evaluation methodologies for measuring the impact of informatics, and dissemination of information for replication of successful programs. Each track addressed current status, ideal state, barriers, strategies, and recommendations. Recommendations of the breakout sessions were summarized under the overarching themes of Policy, Funding, Research, and Education and Training. The general recommendations emphasized four key themes: revision in payment and reimbursement policies, integration of health care standards, partnerships as the key to success, and broad dissemination of findings including specific feedback to target populations and other key stakeholders.

OBJECTIVE: The purposes of the study were (1) to evaluate the usefulness of the International Standards Organization (ISO) Reference Terminology Model for Nursing Diagnoses as a terminology model for defining nursing diagnostic concepts in the Medical Entities Dictionary (MED) and (2) to create the additional hierarchical structures required for integration of nursing diagnostic concepts into the MED. DESIGN AND MEASUREMENTS: The authors dissected nursing diagnostic terms from two source terminologies (Home Health Care Classification and the Omaha System) into the semantic categories of the ISO model. Consistent with the ISO model, they selected Focus and Judgment as required semantic categories for creating intensional definitions of nursing diagnostic concepts in the MED. Because the MED does not include Focus and Judgment hierarchies, the authors developed them to define the nursing diagnostic concepts. RESULTS: The ISO model was sufficient for dissecting the source terminologies into atomic terms. The authors identified 162 unique focus concepts from the 266 nursing diagnosis terms for inclusion in the Focus hierarchy. For the Judgment hierarchy, the authors precoordinated Judgment and Potentiality instead of using Potentiality as a qualifier of Judgment as in the ISO model. Impairment and Alteration were the most frequently occurring judgments. CONCLUSIONS: Nursing care represents a large proportion of health care activities; thus, it is vital that terms used by nurses are integrated into concept-oriented terminologies that provide broad coverage for the domain of health care. This study supports the utility of the ISO Reference Terminology Model for Nursing Diagnoses as a facilitator for the integration process.

The AMIA Annual Symposium began life as the Symposium on Computer Applications in Medical Care (SCAMC) in 1977. Inaugurated as a multidisciplinary meeting with multiple sponsors, the Symposium quickly became a force in the development of the field of medical informatics. The authors summarize the 25-year history of the meeting and its proceedings, drawing on information in the printed programs and proceedings and on the personal recollections of some Symposium organizers and attendees. They also present the results of a study of the extent to which Symposium papers have been cited in the journal literature.

In the past, both health professionals and the public have assumed that the commitment of health professionals to obtaining the best possible outcomes for their patients would guarantee safe and effective health care. Through the work of the Institute of Medicine (IOM) and other leading organizations, however, it has become obvious both to researchers and to society as a whole, that commitment to the betterment of health does not guarantee safe health care practices. As highlighted in “To Err is Human,” 1 a shockingly large number of errors occur, despite the best intentions of providers, and many errors have fatal results. Although there is some debate about the exact number of people who die each year as a result of errors, 2 by and large, deaths do not occur because of individual negligence—rather they result from flawed systems in which people fail to deliver the quality of a care to which they aspire. How can health professionals perform as they both wish and must? A second IOM report released in 20001, “Crossing the Quality Chasm,” 3 identified the critical role that information technology will play in engineering health care systems that produce care that is “safe, effective, patient-centered, timely, efficient, and equitable” (p. 164). As is well recognized by most readers of JAMIA , the needed information technologies are not yet readily available or easily implementable. Improving safety through information technology is an inherently cross-disciplinary effort that requires close collaboration among computer scientists, engineers, and social scientists for a successful endeavor. Where could such a diverse group of people come together to present work and advance this nascent field? This special issue presents papers, posters, and panel discussions from the 2001 AMIA Fall Symposium addressing the issues of patient safety. Three additional papers provide guidance for integrating patient safety concepts into continuing medical education, undergraduate medical education, and nursing informatics curricula. This meeting was one of the first major scientific meetings devoted to medical computing after the new funding initiatives to develop specific research support for the safety area. Papers were part of the first-ever meeting track devoted to the issue of patient safety. These results represent the new wine—first fruits of researchers coming to the safety arena as a result of national initiatives—as well as the continuing contributions of stalwarts in the field. As is typical of Fall Symposium meetings, the papers in this supplement present conceptual models, technological developments that are evolving and still being evaluated, and a snapshot of ongoing research related to computer safety systems. The breadth of work highlighted in this issue reveals that the pursuit of safer practices requires a multifaceted approach. Work presented in the track included work on clinical systems, human factors, knowledge representation, and protecting confidentiality. As of result of the 9-11 terrorist attacks, we have expanded the focus of safety work presented in this issue to include work related to detection of bioterrorist incidents and other issues relevant to homeland security. It is our hope that this issue provides clinicians and educators with a glimpse into the future of the role of technology in improvement of safety and quality of care. As efforts continue to expand funding for use of informatics to enhance safety, we hope that this issue also provides important feedback to policy makers about the types of programs ongoing in the research community and the needs of that community for research support.

In order to understand the differing perceptions of information needs and communication patterns of healthcare professionals as they relate to medical errors, we conducted a survey and 5 focus group sessions of inpatient physicians and nurses. Although nurses and physicians stated differing information needs, both groups expressed significant problems with obtaining patient, domain and institution-specific information in a timely manner. Identification of appropriate providers and establishing contact with those people was perceived as the most pressing communication need. All focus group participants felt that communication difficulties were common and could give examples in which such difficulties led to adverse events. Our studies suggest that information needs and communication difficulties are common and can lead to medical errors or near misses. Many of these problems may be amenable to information technology solutions.

Medical errors are common, costly and often preventable. Work in understanding the proximal causes of medical errors demonstrates that systems failures predispose to adverse clinical events. Most of these systems failures are due to lack of appropriate information at the appropriate time during the course of clinical care. Problems with clinical communication are common proximal causes of medical errors. We have begun a project designed to measure the impact of wireless computing on medical errors. We report here on our efforts to develop an ontology representing the intersection of medical errors, information needs and the communication space. We will use this ontology to support the collection, storage and interpretation of project data. The ontology's formal representation of the concepts in this novel domain will help guide the rational deployment of our informatics interventions. A real-life scenario is evaluated using the ontology in order to demonstrate its utility.

In response to the need to support diverse and complex information requirements, nursing has developed a number of different terminology systems. The two main kinds of systems that have emerged are enumerative systems and combinatorial systems, although some systems have characteristics of both approaches. Differences in the structure and content of terminology systems, while useful at a local level, prevent effective wider communication, information sharing, integration of record systems, and comparison of nursing elements of healthcare information at a more global level. Formal nursing terminology systems present an alternative approach. This paper describes a number of recent initiatives and explains how these emerging approaches may help to augment existing nursing terminology systems and overcome their limitations through mediation. The development of formal nursing terminology systems is not an end in itself and there remains a great deal of work to be done before success can be claimed. This paper presents an overview of the key issues outstanding and provides recommendations for a way forward.

The contention of the author is that an informatics infrastructure is essential for evidenced-based practice. Five building blocks of an informatics infrastructure for evidence-based practice are proposed: 1) standardized terminologies and structures, 2) digital sources of evidence, 3) standards that facilitate health care data exchange among heterogeneous systems, 4) informatics processes that support the acquisition and application of evidence to a specific clinical situation, and 5) informatics competencies. Selected examples illustrate how each of these building blocks supports the application of evidence to practice and the building of evidence from practice. Although a number of major challenges remain, medical informatics can provide solutions that have the potential to decrease unintended variation in practice and health care errors.

Following the Seventh International Congress on Nursing Informatics, the triennial meeting of the International Medical Informatics Association Nursing Informatics (IMIA-NI) Special Interest Group, multidisciplinary experts convened in Rotorua, New Zealand, to examine the topic of evidence-based practice and outcomes from an informatics perspective. Using the framework described in this issue,1 participants critically analyzed the state of the science of the building blocks for an informatics infrastructure for evidence-based practice, and discussed how those components support the building of evidence, the accessing of evidence, and the application of evidence to practice in acute care, primary care, and community/home-care settings. In addition, through work in small groups, the participants identified issues and knowledge gaps that are barriers to the application of evidence in practice. Summaries of the presentations and group discussions will be published in the post-conference proceedings. The articles in this issue of JAMIA reflect only a narrow portion of the breadth of topics discussed over the course of the working conference; thus, the topics are briefly summarized prior to presenting a call to action for nursing informatics. Presentations related to the state of the science served as triggers for the small-group discussions. For the building block of standardized terminologies and data structures, panel members summarized selected terminology-related activities in Asia, South America, the United Kingdom, and the United States and described initiatives related to nursing concept representation in the International Council of Nursing (ICN) and the European Committee on Standardization. Of particular relevance to the international conference was a report on development of an International Standards Organization (ISO) standard for a reference terminology model for nursing under the auspices of IMIA-NI and ICN.2 Two articles related to these presentations appear in this issue. Coenen et al.3 present a summary of those activities and initiatives in an international context, and Hardiker and Rector4 report the benefits gained from implementing a formal, concept-oriented representation of the International Classification of the Nursing Practice and illustrate the manner in which formal terminologies and the more traditional enumerated classifications interrelate.4 Discussions related to access and quality of digital sources of evidence were stimulated by three presentations. Wiechula et al.5 described the activities of the Joanna Briggs Institute for Evidence Based in Nursing and Midwifery and highlighted two essential roles of evidence-based research groups in facilitating access to the evidence—providing information and providing those who use it (e.g., clinicians, researchers, and consumers) with support to manage the information. He emphasized the fact that, regardless of quality and relevancy of the evidence, access alone is insufficient to change practice. Mendonça and Cimino6 illustrated the integration of heterogeneous sources of evidence to answer clinical questions through the presentation of a case study of New York Presbyterian Hospital. Application of evidence in practice was also the focus of the presentation by Entwistle,7 which demonstrated a context-specific approach for integrating practice guideline information with information in the electronic health record. The third set of presentations related to applying the evidence in practice using informatics processes. Sermeus and Hoy8 described WiseCare (Workflow Information Systems for European Nursing Care), a large collaborative research project that aimed to improve cancer nursing practice through the integration and use of information technology. The development of the choice (Creating Better Health Outcomes by Improving Communication about Patients' Expectations) decision support system was used by Ruland9 to illustrate how collection of a type of patient-related evidence (i.e., patient preference for functional performance) could be incorporated into the nursing assessment and care planning process. In a third presentation, Epping10 reviewed the development of critical pathways in the Netherlands. Skiba and McCormick provided visions for the future. Skiba11 focused on the role of emerging technologies in retrieving and applying evidence from the perspective of computing power and speed, information infrastructure (i.e., interconnected networks of computers, devices, and software), and human connections. In contrast, McCormick12 challenged the group to think about the impact of the changing nature of evidence, given the mapping of the human genome and the need for formal representation of that evidence in computer-based systems. Moreover, she identified new potential roles for nurses and described the centrality of informatics competencies to those roles. A number of barriers to building evidence from practice and accessing and applying evidence to practice were identified by conference participants. The following challenges represent a call to action for nursing informatics as opposed to the profession of nursing or medical informatics in general: Develop concept representation standards beyond those focused on reference terminologies, e.g., an international standard for a nursing minimum data set that supports evidence-based practice and outcomes research and a standard for the structure of an electronic nursing admission assessment. Develop informatics-supported critiquing tools that address the full range of nursing research methodologies, including qualitative research methods, and incorporate criteria that address relevance to clinical nursing practice. Integrate information retrieval and presentation approaches based on standardized nursing terminologies and the context-specific needs of nurses across the care continuum. Use informatics processes and information technologies to provide a mechanism by which nurses can view the effectiveness of application of evidence in practice over time. Define informatics competencies for emerging roles related to the changing nature of the evidence available for application to practice.—

Current nursing terminology efforts have converged toward meeting the demand for a reference terminology for nursing concepts by building on the foundation of existing interface and administrative terminologies and by collaborating with terminology efforts across the spectrum of health care. In this article, the authors illustrate how collaboration is promoting convergence toward a reference terminology for nursing by briefly summarizing a wide range of exemplary activities. These include: 1) the International Classification of Nursing Practice (ICNP) activities of the International Council of Nurses (ICN), 2) work in Brazil and Korea that has contributed to, and been stimulated by, ICNP developments, 3) efforts in the United States to improve understanding of the different types of terminologies needed in nursing and to promote harmonization and linking among them, and 4) current nursing participation in major multi-disciplinary standards initiatives. Although early nursing terminology work occurred primarily in isolation and resulted in some duplicative efforts, the activities summarized in this article demonstrate a tremendous level of collaboration and convergence not only in the discipline of nursing but in multi-disciplinary standards initiatives. These efforts are an important prerequisite for ensuring that nursing concepts are represented in computer-based systems in a manner that facilitates multi-purpose use at local, national, regional, and international levels.

The representation of data, information, and knowledge in computer-based systems is essential to achieving the goals of improving the processes and outcomes of health care and building health care knowledge. The papers by Cimino1 and Chute2 in this issue focus on a specific and very significant area of knowledge representation in health care, i.e., terminological knowledge. From the perspective of a decade's experience in development and use, Cimino utilizes the case of the Medical Entities Dictionary (MED) to illustrate the impact on a clinical enterprise of the application of knowledge-based approaches to terminology development and management. Proof of concepts are provided in five areas: merging data and application knowledge; smarter retrievals from the record; “just-in-time” education; expert systems; and data mining. He then generalizes from the MED experience in a discussion of the manner in which the knowledge in terminologies supports the transformation of coded patient data into new knowledge.

The "vocabulary problem" has long plagued the developers, implementers, and users of computer-based systems. The authors review selected activities of the Health Level 7 (HL7) Vocabulary Technical Committee that are related to vocabulary domain specification for HL7 coded data elements. These activities include: 1) the development of two sets of principles to provide guidance to terminology stakeholders, including organizations seeking to deploy HL7-compliant systems, terminology developers, and terminology integrators; 2) the completion of a survey of terminology developers; 3) the development of a process for HL7 registration of terminologies; and 4) the maintenance of vocabulary domain specification tables. As background, vocabulary domain specification is defined and the relationship between the HL7 Reference Information Model and vocabulary domain specification is described. The activities of the Vocabulary Technical Committee complement the efforts of terminology developers and other stakeholders. These activities are aimed at realizing semantic interoperability in the context of the HL7 Message Development Framework, so that information exchange and use among disparate systems can occur for the delivery and management of direct clinical care as well as for purposes such as clinical research, outcome research, and population health management.

OBJECTIVE: The purpose of this study was to test the adequacy of the Clinical LOINC (Logical Observation Identifiers, Names, and Codes) semantic structure as a terminology model for standardized assessment measures. METHODS: After extension of the definitions, 1, 096 items from 35 standardized assessment instruments were dissected into the elements of the Clinical LOINC semantic structure. An additional coder dissected at least one randomly selected item from each instrument. When multiple scale types occurred in a single instrument, a second coder dissected one randomly selected item representative of each scale type. RESULTS: The results support the adequacy of the Clinical LOINC semantic structure as a terminology model for standardized assessments. Using the revised definitions, the coders were able to dissect into the elements of Clinical LOINC all the standardized assessment items in the sample instruments. Percentage agreement for each element was as follows: component, 100 percent; property, 87.8 percent; timing, 82.9 percent; system/sample, 100 percent; scale, 92.6 percent; and method, 97.6 percent. DISCUSSION: This evaluation was an initial step toward the representation of standardized assessment items in a manner that facilitates data sharing and re-use. Further clarification of the definitions, especially those related to time and property, is required to improve inter-rater reliability and to harmonize the representations with similar items already in LOINC.

OBJECTIVE: A type definition, as a component of the categorical structures of a concept-oriented terminology, must support nonambiguous concept representations and, consequently, comparisons of data that are represented using different terminologies. The purpose of the study was to evaluate the adequacy and utility of a proposed type definition for nursing activity concepts. DESIGN: Nursing activity terms (n = 1039) from patient charts and intervention terms from two nursing terminologies (Home Health Care Classification and Omaha System) were decomposed into the attributes of the proposed type definition-Delivery Mode, Activity Focus, and Recipient. MEASUREMENTS: Attributes of the type definition were coded as present or absent for each term by multiple raters. In addition, Delivery Mode was rated as Explicit or Implicit and Recipient was rated as Explicit, Implicit, or Ambiguous. The data were summarized using descriptive statistics. Inter-rater reliabilities were calculated for each attribute of the type definition. RESULTS: All attributes of the type definition were present in 73.9 percent of the chart terms, 91.3 percent of Home Health Care Classification intervention terms, and 63.5 percent of Omaha System intervention terms. While Delivery Mode and Activity Focus were almost universally present, Recipient was problematic. It was rated as ambiguous in 4.8 percent of the chart terms, 8.7 percent of Home Health Care Classification intervention terms, and 36.5 percent of Omaha System intervention terms. CONCLUSIONS: The study findings supported the adequacy and utility of the type definition. Further research is needed to refine the type definition and its use for representing nursing activity concepts within a concept-oriented terminological system.

Practice guidelines are an integral part of evidence-based health care delivery. When the authors decided to install the clinical documentation component of an electronic health record in a nurse practitioner faculty practice, however, they found that they lacked the resources to integrate it immediately with other systems and components that would support the processing of clinical rules. They were thus challenged to devise an initial approach for decision support related to clinical practice guidelines that did not include interfacing with an inference engine and set of decision rules. The authors developed a prototypic application within the WAVE electronic health record that demonstrates the feasibility of representing a guideline as structured encoded text organized into an online patient-encounter template. Although this approach may be more broadly applicable, it is described within the context of the management of diabetes mellitus by nurse practitioners. The advantages of the approach relate primarily to the integration of the guideline recommendations with the encounter form, the online interaction of the clinician with the system, and the ease of creation and modification of the guideline-based encounter form. However, there are several limitations of the current approach as a result of the inability to do inference and the lack of integration with patient-specific data to trigger specific rules.

Building on the work of previous authors, the Computer-based Patient Record Institute (CPRI) Work Group on Codes and Structures has described features of a classification scheme for implementation within a computer-based patient record. The authors of the current study reviewed the evaluation literature related to six major nursing vocabularies (the North American Nursing Diagnosis Association Taxonomy 1, the Nursing Interventions Classification, the Nursing Outcomes Classification, the Home Health Care Classification, the Omaha System, and the International Classification for Nursing Practice) to determine the extent to which the vocabularies include the CPRI features. None of the vocabularies met all criteria. The Omaha System, Home Health Care Classification, and International Classification for Nursing Practice each included five features. Criteria not fully met by any systems were clear and non-redundant representation of concepts, administrative cross-references, syntax and grammar, synonyms, uncertainty, context-free identifiers, and language independence.

Our premise is that from the perspective of maximum flexibility of data usage by computer-based record (CPR) systems, existing nursing classification systems are necessary, but not sufficient, for representing important aspects of "what nurses do." In particular, we have focused our attention on those classification systems that represent nurses' clinical activities through the abstraction of activities into categories of nursing interventions. In this theoretical paper, we argue that taxonomic, combinatorial vocabularies capable of coding atomic-level nursing activities are required to effectively capture in a reproducible and reversible manner the clinical decisions and actions of nurses, and that, without such vocabularies and associated grammars, potentially important clinical process data is lost during the encoding process. Existing nursing intervention classification systems do not fulfill these criteria. As background to our argument, we first present an overview of the content, methods, and evaluation criteria used in previous studies whose focus has been to evaluate the effectiveness of existing coding and classification systems. Next, using the Ingenerf typology of taxonomic vocabularies, we categorize the formal type and structure of three existing nursing intervention classification system--Nursing Interventions Classification, Omaha System, and Home Health Care Classification. Third, we use records from home care patients to show examples of lossy data transformation, the loss of potentially significant atomic data, resulting from encoding using each of the three systems. Last, we provide an example of the application of a formal representation methodology (conceptual graphs) which we believe could be used as a model to build the required combinatorial, taxonomic vocabulary for representing nursing interventions.

In recent decades there have been major advances in the creation and implementation of information technologies and in the development of measures of health care quality. The premise of this article is that informatics provides essential infrastructure for quality assessment and improvement in nursing. In this context, the term quality assessment and improvement comprises both short-term processes such as continuous quality improvement (CQI) and long-term outcomes management. This premise is supported by 1) presentation of a historical perspective on quality assessment and improvement; 2) delineation of the types of data required for quality assessment and improvement; and 3) description of the current and potential uses of information technology in the acquisition, storage, transformation, and presentation of quality data, information, and knowledge.

OBJECTIVE: To analyze the terms used by nurses in a variety of data sources and to test the feasibility of using SNOMED III to represent nursing terms. DESIGN: Prospective research design with manual matching of terms to the SNOMED III vocabulary. MEASUREMENTS: The terms used by nurses to describe patient problems during 485 episodes of care for 201 patients hospitalized for Pneumocystis carinii pneumonia were identified. Problems from four data sources (nurse interview, intershift report, nursing care plan, and nurse progress note/flowsheet) were classified based on the substantive area of the problem and on the terminology used to describe the problem. A test subset of the 25 most frequently used terms from the two written data sources (nursing care plan and nurse progress note/flowsheet) were manually matched to SNOMED III terms to test the feasibility of using that existing vocabulary to represent nursing terms. RESULTS: Nurses most frequently described patient problems as signs/symptoms in the verbal nurse interview and intershift report. In the written data sources, problems were recorded as North American Nursing Diagnosis Association (NANDA) terms and signs/symptoms with similar frequencies. Of the nursing terms in the test subset, 69% were represented using one or more SNOMED III terms.